diff --git a/doc/src/Bayesian/.chapter11.copyright b/doc/src/Bayesian/.chapter11.copyright new file mode 100644 index 000000000..98e7e541c --- /dev/null +++ b/doc/src/Bayesian/.chapter11.copyright @@ -0,0 +1 @@ +{'holder': ['Morten Hjorth-Jensen'], 'year': '1999-2020', 'license': 'Released under CC Attribution-NonCommercial 4.0 license', 'cite doconce': False} \ No newline at end of file diff --git a/doc/src/LectureNotes/VaryDimensionPredicted12.csv b/doc/src/LectureNotes/VaryDimensionPredicted12.csv new file mode 100644 index 000000000..40e4fae96 --- /dev/null +++ b/doc/src/LectureNotes/VaryDimensionPredicted12.csv @@ -0,0 +1,20 @@ +-3.077640548999999864e-02 +-8.336233265999999642e-02 +-1.446729566999999939e-01 +-2.116753731999999888e-01 +-2.830637391999999974e-01 +-3.581341341000000011e-01 +-4.364624349999999819e-01 +-5.177783846000000301e-01 +-6.019067271000000385e-01 +-6.887363571000000295e-01 +-7.782028951999999666e-01 +-8.702784033999999558e-01 +-9.671434760093688965e-01 +-1.065430045127868652e+00 +-1.166525721549987793e+00 +-1.269154906272888184e+00 +-1.373405933380126953e+00 +-1.478558778762817383e+00 +-1.584262609481811523e+00 +-1.689914822578430176e+00 diff --git a/doc/src/LectureNotes/_build/.doctrees/chapter1.doctree b/doc/src/LectureNotes/_build/.doctrees/chapter1.doctree index 3657701e8..c49566105 100644 Binary files a/doc/src/LectureNotes/_build/.doctrees/chapter1.doctree and b/doc/src/LectureNotes/_build/.doctrees/chapter1.doctree differ diff --git a/doc/src/LectureNotes/_build/.doctrees/chapter10.doctree b/doc/src/LectureNotes/_build/.doctrees/chapter10.doctree new file mode 100644 index 000000000..be9382f0d Binary files /dev/null and b/doc/src/LectureNotes/_build/.doctrees/chapter10.doctree differ diff --git a/doc/src/LectureNotes/_build/.doctrees/chapter11.doctree b/doc/src/LectureNotes/_build/.doctrees/chapter11.doctree new file mode 100644 index 000000000..49850d19b Binary files /dev/null and b/doc/src/LectureNotes/_build/.doctrees/chapter11.doctree differ diff --git a/doc/src/LectureNotes/_build/.doctrees/chapter2.doctree b/doc/src/LectureNotes/_build/.doctrees/chapter2.doctree index 94baf2ba4..235df8a6e 100644 Binary files a/doc/src/LectureNotes/_build/.doctrees/chapter2.doctree and b/doc/src/LectureNotes/_build/.doctrees/chapter2.doctree differ diff --git a/doc/src/LectureNotes/_build/.doctrees/chapter3.doctree b/doc/src/LectureNotes/_build/.doctrees/chapter3.doctree index eff307cdb..967e6fc11 100644 Binary files a/doc/src/LectureNotes/_build/.doctrees/chapter3.doctree and b/doc/src/LectureNotes/_build/.doctrees/chapter3.doctree differ diff --git a/doc/src/LectureNotes/_build/.doctrees/chapter4.doctree b/doc/src/LectureNotes/_build/.doctrees/chapter4.doctree index ad50241a3..86ee05621 100644 Binary files a/doc/src/LectureNotes/_build/.doctrees/chapter4.doctree and b/doc/src/LectureNotes/_build/.doctrees/chapter4.doctree differ diff --git a/doc/src/LectureNotes/_build/.doctrees/chapter6.doctree b/doc/src/LectureNotes/_build/.doctrees/chapter6.doctree index dd463ca53..6f2ff1d4f 100644 Binary files a/doc/src/LectureNotes/_build/.doctrees/chapter6.doctree and b/doc/src/LectureNotes/_build/.doctrees/chapter6.doctree differ diff --git a/doc/src/LectureNotes/_build/.doctrees/chapter8.doctree b/doc/src/LectureNotes/_build/.doctrees/chapter8.doctree index 9daeb94bf..d11e1ed3e 100644 Binary files a/doc/src/LectureNotes/_build/.doctrees/chapter8.doctree and b/doc/src/LectureNotes/_build/.doctrees/chapter8.doctree differ diff --git a/doc/src/LectureNotes/_build/.doctrees/chapter9.doctree b/doc/src/LectureNotes/_build/.doctrees/chapter9.doctree index d851430c8..69683e397 100644 Binary files a/doc/src/LectureNotes/_build/.doctrees/chapter9.doctree and b/doc/src/LectureNotes/_build/.doctrees/chapter9.doctree differ diff --git a/doc/src/LectureNotes/_build/.doctrees/environment.pickle b/doc/src/LectureNotes/_build/.doctrees/environment.pickle index a0d0a0bc1..633a94b37 100644 Binary files a/doc/src/LectureNotes/_build/.doctrees/environment.pickle and b/doc/src/LectureNotes/_build/.doctrees/environment.pickle differ diff --git a/doc/src/LectureNotes/_build/.doctrees/testbook/_build/jupyter_execute/chapter2.doctree b/doc/src/LectureNotes/_build/.doctrees/testbook/_build/jupyter_execute/chapter2.doctree index 9cb6b7b93..e79a62e73 100644 Binary files a/doc/src/LectureNotes/_build/.doctrees/testbook/_build/jupyter_execute/chapter2.doctree and b/doc/src/LectureNotes/_build/.doctrees/testbook/_build/jupyter_execute/chapter2.doctree differ diff --git a/doc/src/LectureNotes/_build/.doctrees/testbook/chapter2.doctree b/doc/src/LectureNotes/_build/.doctrees/testbook/chapter2.doctree index 596968e92..6d07bc7e6 100644 Binary files a/doc/src/LectureNotes/_build/.doctrees/testbook/chapter2.doctree and b/doc/src/LectureNotes/_build/.doctrees/testbook/chapter2.doctree differ diff --git a/doc/src/LectureNotes/_build/html/_images/chapter10_11_326.png b/doc/src/LectureNotes/_build/html/_images/chapter10_11_326.png new file mode 100644 index 000000000..805fe41b2 Binary files /dev/null and b/doc/src/LectureNotes/_build/html/_images/chapter10_11_326.png differ diff --git a/doc/src/LectureNotes/_build/html/_images/chapter10_11_328.png b/doc/src/LectureNotes/_build/html/_images/chapter10_11_328.png new file mode 100644 index 000000000..15cd8bd4a Binary files /dev/null and b/doc/src/LectureNotes/_build/html/_images/chapter10_11_328.png differ diff --git a/doc/src/LectureNotes/_build/html/_images/chapter10_11_687.png b/doc/src/LectureNotes/_build/html/_images/chapter10_11_687.png new file mode 100644 index 000000000..69b7bde49 Binary files /dev/null and b/doc/src/LectureNotes/_build/html/_images/chapter10_11_687.png differ diff --git a/doc/src/LectureNotes/_build/html/_images/chapter10_11_688.png b/doc/src/LectureNotes/_build/html/_images/chapter10_11_688.png new file mode 100644 index 000000000..a6433f182 Binary files /dev/null and b/doc/src/LectureNotes/_build/html/_images/chapter10_11_688.png differ diff --git a/doc/src/LectureNotes/_build/html/_images/chapter10_1_0.png b/doc/src/LectureNotes/_build/html/_images/chapter10_1_0.png new file mode 100644 index 000000000..1d4c16704 Binary files /dev/null and b/doc/src/LectureNotes/_build/html/_images/chapter10_1_0.png differ diff --git a/doc/src/LectureNotes/_build/html/_images/chapter10_1_197.png b/doc/src/LectureNotes/_build/html/_images/chapter10_1_197.png new file mode 100644 index 000000000..35d7d55c4 Binary files /dev/null and b/doc/src/LectureNotes/_build/html/_images/chapter10_1_197.png differ diff --git a/doc/src/LectureNotes/_build/html/_images/chapter10_59_2.png b/doc/src/LectureNotes/_build/html/_images/chapter10_59_2.png new file mode 100644 index 000000000..bb70c1d74 Binary files /dev/null and b/doc/src/LectureNotes/_build/html/_images/chapter10_59_2.png differ diff --git a/doc/src/LectureNotes/_build/html/_images/chapter10_7_411.png b/doc/src/LectureNotes/_build/html/_images/chapter10_7_411.png new file mode 100644 index 000000000..33b4a158d Binary files /dev/null and b/doc/src/LectureNotes/_build/html/_images/chapter10_7_411.png differ diff --git a/doc/src/LectureNotes/_build/html/_images/chapter10_7_413.png b/doc/src/LectureNotes/_build/html/_images/chapter10_7_413.png new file mode 100644 index 000000000..93e7d1fc9 Binary files /dev/null and b/doc/src/LectureNotes/_build/html/_images/chapter10_7_413.png differ diff --git a/doc/src/LectureNotes/_build/html/_images/chapter10_9_421.png b/doc/src/LectureNotes/_build/html/_images/chapter10_9_421.png new file mode 100644 index 000000000..ec63b7726 Binary files /dev/null and b/doc/src/LectureNotes/_build/html/_images/chapter10_9_421.png differ diff --git a/doc/src/LectureNotes/_build/html/_images/chapter10_9_423.png b/doc/src/LectureNotes/_build/html/_images/chapter10_9_423.png new file mode 100644 index 000000000..b7afeb9ff Binary files /dev/null and b/doc/src/LectureNotes/_build/html/_images/chapter10_9_423.png differ diff --git a/doc/src/LectureNotes/_build/html/_images/chapter2_178_0.png b/doc/src/LectureNotes/_build/html/_images/chapter2_178_0.png index 80d8fd112..e1fe92955 100644 Binary files a/doc/src/LectureNotes/_build/html/_images/chapter2_178_0.png and b/doc/src/LectureNotes/_build/html/_images/chapter2_178_0.png differ diff --git a/doc/src/LectureNotes/_build/html/_images/chapter2_184_1.png b/doc/src/LectureNotes/_build/html/_images/chapter2_184_1.png index c7b66ed0b..c5fadabc8 100644 Binary files a/doc/src/LectureNotes/_build/html/_images/chapter2_184_1.png and b/doc/src/LectureNotes/_build/html/_images/chapter2_184_1.png differ diff --git a/doc/src/LectureNotes/_build/html/_images/chapter4_278_2.png b/doc/src/LectureNotes/_build/html/_images/chapter4_278_2.png index a56a44f78..2abb9e89d 100644 Binary files a/doc/src/LectureNotes/_build/html/_images/chapter4_278_2.png and b/doc/src/LectureNotes/_build/html/_images/chapter4_278_2.png differ diff --git a/doc/src/LectureNotes/_build/html/_images/chapter6_134_1.png b/doc/src/LectureNotes/_build/html/_images/chapter6_134_1.png new file mode 100644 index 000000000..eccdd272b Binary files /dev/null and b/doc/src/LectureNotes/_build/html/_images/chapter6_134_1.png differ diff --git a/doc/src/LectureNotes/_build/html/_images/chapter6_134_2.png b/doc/src/LectureNotes/_build/html/_images/chapter6_134_2.png new file mode 100644 index 000000000..e3bbd6804 Binary files /dev/null and b/doc/src/LectureNotes/_build/html/_images/chapter6_134_2.png differ diff --git a/doc/src/LectureNotes/_build/html/_images/chapter6_138_0.png b/doc/src/LectureNotes/_build/html/_images/chapter6_138_0.png new file mode 100644 index 000000000..cae5147e9 Binary files /dev/null and b/doc/src/LectureNotes/_build/html/_images/chapter6_138_0.png differ diff --git a/doc/src/LectureNotes/_build/html/_images/chapter6_138_1.png b/doc/src/LectureNotes/_build/html/_images/chapter6_138_1.png new file mode 100644 index 000000000..f889a46a4 Binary files /dev/null and b/doc/src/LectureNotes/_build/html/_images/chapter6_138_1.png differ diff --git a/doc/src/LectureNotes/_build/html/_images/chapter8_77_1.png b/doc/src/LectureNotes/_build/html/_images/chapter8_77_1.png index 50f345c7f..1bc84b602 100644 Binary files a/doc/src/LectureNotes/_build/html/_images/chapter8_77_1.png and b/doc/src/LectureNotes/_build/html/_images/chapter8_77_1.png differ diff --git a/doc/src/LectureNotes/_build/html/_sources/chapter10.ipynb b/doc/src/LectureNotes/_build/html/_sources/chapter10.ipynb new file mode 100644 index 000000000..be47c97e1 --- /dev/null +++ b/doc/src/LectureNotes/_build/html/_sources/chapter10.ipynb @@ -0,0 +1,3815 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Recurrent Neural Networks\n", + "\n", + "[Overview video](https://www.youtube.com/watch?v=SEnXr6v2ifU&ab_channel=AlexanderAmini).\n", + "See also lecture on Thursday October 22 and examples from [week 42](https://compphysics.github.io/MachineLearning/doc/pub/week42/html/week42.html).\n", + "\n", + "[IN5400 at UiO Lecture](https://www.uio.no/studier/emner/matnat/ifi/IN5400/v20/material/week10/in5400_2020_week10_recurrent_neural_network.pdf)\n", + "\n", + "[CS231 at Stanford Lecture](https://www.youtube.com/watch?v=6niqTuYFZLQ&list=PLzUTmXVwsnXod6WNdg57Yc3zFx_f-RYsq&index=10&ab_channel=StanfordUniversitySchoolofEngineering)\n", + "\n", + "## Recurrent neural networks: Overarching view\n", + "\n", + "Till now our focus has been, including convolutional neural networks\n", + "as well, on feedforward neural networks. The output or the activations\n", + "flow only in one direction, from the input layer to the output layer.\n", + "\n", + "A recurrent neural network (RNN) looks very much like a feedforward\n", + "neural network, except that it also has connections pointing\n", + "backward. \n", + "\n", + "RNNs are used to analyze time series data such as stock prices, and\n", + "tell you when to buy or sell. In autonomous driving systems, they can\n", + "anticipate car trajectories and help avoid accidents. More generally,\n", + "they can work on sequences of arbitrary lengths, rather than on\n", + "fixed-sized inputs like all the nets we have discussed so far. For\n", + "example, they can take sentences, documents, or audio samples as\n", + "input, making them extremely useful for natural language processing\n", + "systems such as automatic translation and speech-to-text.\n", + "\n", + "\n", + "\n", + "\n", + "## Set up of an RNN\n", + "\n", + "\n", + "Text to come.\n", + "\n", + "\n", + "## A simple example" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "%matplotlib inline\n", + "\n", + "# Start importing packages\n", + "import pandas as pd\n", + "import numpy as np\n", + "import matplotlib.pyplot as plt\n", + "import tensorflow as tf\n", + "from tensorflow.keras import datasets, layers, models\n", + "from tensorflow.keras.layers import Input\n", + "from tensorflow.keras.models import Model, Sequential \n", + "from tensorflow.keras.layers import Dense, SimpleRNN, LSTM, GRU\n", + "from tensorflow.keras import optimizers \n", + "from tensorflow.keras import regularizers \n", + "from tensorflow.keras.utils import to_categorical \n", + "\n", + "\n", + "\n", + "# convert into dataset matrix\n", + "def convertToMatrix(data, step):\n", + " X, Y =[], []\n", + " for i in range(len(data)-step):\n", + " d=i+step \n", + " X.append(data[i:d,])\n", + " Y.append(data[d,])\n", + " return np.array(X), np.array(Y)\n", + "\n", + "step = 4\n", + "N = 1000 \n", + "Tp = 800 \n", + "\n", + "t=np.arange(0,N)\n", + "x=np.sin(0.02*t)+2*np.random.rand(N)\n", + "df = pd.DataFrame(x)\n", + "df.head()\n", + "\n", + "plt.plot(df)\n", + "plt.show()\n", + "\n", + "values=df.values\n", + "train,test = values[0:Tp,:], values[Tp:N,:]\n", + "\n", + "# add step elements into train and test\n", + "test = np.append(test,np.repeat(test[-1,],step))\n", + "train = np.append(train,np.repeat(train[-1,],step))\n", + " \n", + "trainX,trainY =convertToMatrix(train,step)\n", + "testX,testY =convertToMatrix(test,step)\n", + "trainX = np.reshape(trainX, (trainX.shape[0], 1, trainX.shape[1]))\n", + "testX = np.reshape(testX, (testX.shape[0], 1, testX.shape[1]))\n", + "\n", + "model = Sequential()\n", + "model.add(SimpleRNN(units=32, input_shape=(1,step), activation=\"relu\"))\n", + "model.add(Dense(8, activation=\"relu\")) \n", + "model.add(Dense(1))\n", + "model.compile(loss='mean_squared_error', optimizer='rmsprop')\n", + "model.summary()\n", + "\n", + "model.fit(trainX,trainY, epochs=100, batch_size=16, verbose=2)\n", + "trainPredict = model.predict(trainX)\n", + "testPredict= model.predict(testX)\n", + "predicted=np.concatenate((trainPredict,testPredict),axis=0)\n", + "\n", + "trainScore = model.evaluate(trainX, trainY, verbose=0)\n", + "print(trainScore)\n", + "\n", + "index = df.index.values\n", + "plt.plot(index,df)\n", + "plt.plot(index,predicted)\n", + "plt.axvline(df.index[Tp], c=\"r\")\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## An extrapolation example\n", + "\n", + "The following code provides an example of how recurrent neural\n", + "networks can be used to extrapolate to unknown values of physics data\n", + "sets. Specifically, the data sets used in this program come from\n", + "a quantum mechanical many-body calculation of energies as functions of the number of particles." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "\n", + "# For matrices and calculations\n", + "import numpy as np\n", + "# For machine learning (backend for keras)\n", + "import tensorflow as tf\n", + "# User-friendly machine learning library\n", + "# Front end for TensorFlow\n", + "import tensorflow.keras\n", + "# Different methods from Keras needed to create an RNN\n", + "# This is not necessary but it shortened function calls \n", + "# that need to be used in the code.\n", + "from tensorflow.keras import datasets, layers, models\n", + "from tensorflow.keras.layers import Input\n", + "from tensorflow.keras import regularizers\n", + "from tensorflow.keras.models import Model, Sequential\n", + "from tensorflow.keras.layers import Dense, SimpleRNN, LSTM, GRU\n", + "# For timing the code\n", + "from timeit import default_timer as timer\n", + "# For plotting\n", + "import matplotlib.pyplot as plt\n", + "\n", + "\n", + "# The data set\n", + "datatype='VaryDimension'\n", + "X_tot = np.arange(2, 42, 2)\n", + "y_tot = np.array([-0.03077640549, -0.08336233266, -0.1446729567, -0.2116753732, -0.2830637392, -0.3581341341, -0.436462435, -0.5177783846,\n", + "\t-0.6019067271, -0.6887363571, -0.7782028952, -0.8702784034, -0.9649652536, -1.062292565, -1.16231451, \n", + "\t-1.265109911, -1.370782966, -1.479465113, -1.591317992, -1.70653767])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Formatting the Data\n", + "\n", + "The way the recurrent neural networks are trained in this program\n", + "differs from how machine learning algorithms are usually trained.\n", + "Typically a machine learning algorithm is trained by learning the\n", + "relationship between the x data and the y data. In this program, the\n", + "recurrent neural network will be trained to recognize the relationship\n", + "in a sequence of y values. This is type of data formatting is\n", + "typically used time series forcasting, but it can also be used in any\n", + "extrapolation (time series forecasting is just a specific type of\n", + "extrapolation along the time axis). This method of data formatting\n", + "does not use the x data and assumes that the y data are evenly spaced.\n", + "\n", + "For a standard machine learning algorithm, the training data has the\n", + "form of (x,y) so the machine learning algorithm learns to assiciate a\n", + "y value with a given x value. This is useful when the test data has x\n", + "values within the same range as the training data. However, for this\n", + "application, the x values of the test data are outside of the x values\n", + "of the training data and the traditional method of training a machine\n", + "learning algorithm does not work as well. For this reason, the\n", + "recurrent neural network is trained on sequences of y values of the\n", + "form ((y1, y2), y3), so that the network is concerned with learning\n", + "the pattern of the y data and not the relation between the x and y\n", + "data. As long as the pattern of y data outside of the training region\n", + "stays relatively stable compared to what was inside the training\n", + "region, this method of training can produce accurate extrapolations to\n", + "y values far removed from the training data set.\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "# FORMAT_DATA\n", + "def format_data(data, length_of_sequence = 2): \n", + " \"\"\"\n", + " Inputs:\n", + " data(a numpy array): the data that will be the inputs to the recurrent neural\n", + " network\n", + " length_of_sequence (an int): the number of elements in one iteration of the\n", + " sequence patter. For a function approximator use length_of_sequence = 2.\n", + " Returns:\n", + " rnn_input (a 3D numpy array): the input data for the recurrent neural network. Its\n", + " dimensions are length of data - length of sequence, length of sequence, \n", + " dimnsion of data\n", + " rnn_output (a numpy array): the training data for the neural network\n", + " Formats data to be used in a recurrent neural network.\n", + " \"\"\"\n", + "\n", + " X, Y = [], []\n", + " for i in range(len(data)-length_of_sequence):\n", + " # Get the next length_of_sequence elements\n", + " a = data[i:i+length_of_sequence]\n", + " # Get the element that immediately follows that\n", + " b = data[i+length_of_sequence]\n", + " # Reshape so that each data point is contained in its own array\n", + " a = np.reshape (a, (len(a), 1))\n", + " X.append(a)\n", + " Y.append(b)\n", + " rnn_input = np.array(X)\n", + " rnn_output = np.array(Y)\n", + "\n", + " return rnn_input, rnn_output\n", + "\n", + "\n", + "# ## Defining the Recurrent Neural Network Using Keras\n", + "# \n", + "# The following method defines a simple recurrent neural network in keras consisting of one input layer, one hidden layer, and one output layer.\n", + "\n", + "def rnn(length_of_sequences, batch_size = None, stateful = False):\n", + " \"\"\"\n", + " Inputs:\n", + " length_of_sequences (an int): the number of y values in \"x data\". This is determined\n", + " when the data is formatted\n", + " batch_size (an int): Default value is None. See Keras documentation of SimpleRNN.\n", + " stateful (a boolean): Default value is False. See Keras documentation of SimpleRNN.\n", + " Returns:\n", + " model (a Keras model): The recurrent neural network that is built and compiled by this\n", + " method\n", + " Builds and compiles a recurrent neural network with one hidden layer and returns the model.\n", + " \"\"\"\n", + " # Number of neurons in the input and output layers\n", + " in_out_neurons = 1\n", + " # Number of neurons in the hidden layer\n", + " hidden_neurons = 200\n", + " # Define the input layer\n", + " inp = Input(batch_shape=(batch_size, \n", + " length_of_sequences, \n", + " in_out_neurons)) \n", + " # Define the hidden layer as a simple RNN layer with a set number of neurons and add it to \n", + " # the network immediately after the input layer\n", + " rnn = SimpleRNN(hidden_neurons, \n", + " return_sequences=False,\n", + " stateful = stateful,\n", + " name=\"RNN\")(inp)\n", + " # Define the output layer as a dense neural network layer (standard neural network layer)\n", + " #and add it to the network immediately after the hidden layer.\n", + " dens = Dense(in_out_neurons,name=\"dense\")(rnn)\n", + " # Create the machine learning model starting with the input layer and ending with the \n", + " # output layer\n", + " model = Model(inputs=[inp],outputs=[dens])\n", + " # Compile the machine learning model using the mean squared error function as the loss \n", + " # function and an Adams optimizer.\n", + " model.compile(loss=\"mean_squared_error\", optimizer=\"adam\") \n", + " return model" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Predicting New Points With A Trained Recurrent Neural Network" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "def test_rnn (x1, y_test, plot_min, plot_max):\n", + " \"\"\"\n", + " Inputs:\n", + " x1 (a list or numpy array): The complete x component of the data set\n", + " y_test (a list or numpy array): The complete y component of the data set\n", + " plot_min (an int or float): the smallest x value used in the training data\n", + " plot_max (an int or float): the largest x valye used in the training data\n", + " Returns:\n", + " None.\n", + " Uses a trained recurrent neural network model to predict future points in the \n", + " series. Computes the MSE of the predicted data set from the true data set, saves\n", + " the predicted data set to a csv file, and plots the predicted and true data sets w\n", + " while also displaying the data range used for training.\n", + " \"\"\"\n", + " # Add the training data as the first dim points in the predicted data array as these\n", + " # are known values.\n", + " y_pred = y_test[:dim].tolist()\n", + " # Generate the first input to the trained recurrent neural network using the last two \n", + " # points of the training data. Based on how the network was trained this means that it\n", + " # will predict the first point in the data set after the training data. All of the \n", + " # brackets are necessary for Tensorflow.\n", + " next_input = np.array([[[y_test[dim-2]], [y_test[dim-1]]]])\n", + " # Save the very last point in the training data set. This will be used later.\n", + " last = [y_test[dim-1]]\n", + "\n", + " # Iterate until the complete data set is created.\n", + " for i in range (dim, len(y_test)):\n", + " # Predict the next point in the data set using the previous two points.\n", + " next = model.predict(next_input)\n", + " # Append just the number of the predicted data set\n", + " y_pred.append(next[0][0])\n", + " # Create the input that will be used to predict the next data point in the data set.\n", + " next_input = np.array([[last, next[0]]], dtype=np.float64)\n", + " last = next\n", + "\n", + " # Print the mean squared error between the known data set and the predicted data set.\n", + " print('MSE: ', np.square(np.subtract(y_test, y_pred)).mean())\n", + " # Save the predicted data set as a csv file for later use\n", + " name = datatype + 'Predicted'+str(dim)+'.csv'\n", + " np.savetxt(name, y_pred, delimiter=',')\n", + " # Plot the known data set and the predicted data set. The red box represents the region that was used\n", + " # for the training data.\n", + " fig, ax = plt.subplots()\n", + " ax.plot(x1, y_test, label=\"true\", linewidth=3)\n", + " ax.plot(x1, y_pred, 'g-.',label=\"predicted\", linewidth=4)\n", + " ax.legend()\n", + " # Created a red region to represent the points used in the training data.\n", + " ax.axvspan(plot_min, plot_max, alpha=0.25, color='red')\n", + " plt.show()\n", + "\n", + "# Check to make sure the data set is complete\n", + "assert len(X_tot) == len(y_tot)\n", + "\n", + "# This is the number of points that will be used in as the training data\n", + "dim=12\n", + "\n", + "# Separate the training data from the whole data set\n", + "X_train = X_tot[:dim]\n", + "y_train = y_tot[:dim]\n", + "\n", + "\n", + "# Generate the training data for the RNN, using a sequence of 2\n", + "rnn_input, rnn_training = format_data(y_train, 2)\n", + "\n", + "\n", + "# Create a recurrent neural network in Keras and produce a summary of the \n", + "# machine learning model\n", + "model = rnn(length_of_sequences = rnn_input.shape[1])\n", + "model.summary()\n", + "\n", + "# Start the timer. Want to time training+testing\n", + "start = timer()\n", + "# Fit the model using the training data genenerated above using 150 training iterations and a 5%\n", + "# validation split. Setting verbose to True prints information about each training iteration.\n", + "hist = model.fit(rnn_input, rnn_training, batch_size=None, epochs=150, \n", + " verbose=True,validation_split=0.05)\n", + "\n", + "for label in [\"loss\",\"val_loss\"]:\n", + " plt.plot(hist.history[label],label=label)\n", + "\n", + "plt.ylabel(\"loss\")\n", + "plt.xlabel(\"epoch\")\n", + "plt.title(\"The final validation loss: {}\".format(hist.history[\"val_loss\"][-1]))\n", + "plt.legend()\n", + "plt.show()\n", + "\n", + "# Use the trained neural network to predict more points of the data set\n", + "test_rnn(X_tot, y_tot, X_tot[0], X_tot[dim-1])\n", + "# Stop the timer and calculate the total time needed.\n", + "end = timer()\n", + "print('Time: ', end-start)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Other Things to Try\n", + "\n", + "\n", + "Changing the size of the recurrent neural network and its parameters\n", + "can drastically change the results you get from the model. The below\n", + "code takes the simple recurrent neural network from above and adds a\n", + "second hidden layer, changes the number of neurons in the hidden\n", + "layer, and explicitly declares the activation function of the hidden\n", + "layers to be a sigmoid function. The loss function and optimizer can\n", + "also be changed but are kept the same as the above network. These\n", + "parameters can be tuned to provide the optimal result from the\n", + "network. For some ideas on how to improve the performance of a\n", + "[recurrent neural network](https://danijar.com/tips-for-training-recurrent-neural-networks)." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "def rnn_2layers(length_of_sequences, batch_size = None, stateful = False):\n", + " \"\"\"\n", + " Inputs:\n", + " length_of_sequences (an int): the number of y values in \"x data\". This is determined\n", + " when the data is formatted\n", + " batch_size (an int): Default value is None. See Keras documentation of SimpleRNN.\n", + " stateful (a boolean): Default value is False. See Keras documentation of SimpleRNN.\n", + " Returns:\n", + " model (a Keras model): The recurrent neural network that is built and compiled by this\n", + " method\n", + " Builds and compiles a recurrent neural network with two hidden layers and returns the model.\n", + " \"\"\"\n", + " # Number of neurons in the input and output layers\n", + " in_out_neurons = 1\n", + " # Number of neurons in the hidden layer, increased from the first network\n", + " hidden_neurons = 500\n", + " # Define the input layer\n", + " inp = Input(batch_shape=(batch_size, \n", + " length_of_sequences, \n", + " in_out_neurons)) \n", + " # Create two hidden layers instead of one hidden layer. Explicitly set the activation\n", + " # function to be the sigmoid function (the default value is hyperbolic tangent)\n", + " rnn1 = SimpleRNN(hidden_neurons, \n", + " return_sequences=True, # This needs to be True if another hidden layer is to follow\n", + " stateful = stateful, activation = 'sigmoid',\n", + " name=\"RNN1\")(inp)\n", + " rnn2 = SimpleRNN(hidden_neurons, \n", + " return_sequences=False, activation = 'sigmoid',\n", + " stateful = stateful,\n", + " name=\"RNN2\")(rnn1)\n", + " # Define the output layer as a dense neural network layer (standard neural network layer)\n", + " #and add it to the network immediately after the hidden layer.\n", + " dens = Dense(in_out_neurons,name=\"dense\")(rnn2)\n", + " # Create the machine learning model starting with the input layer and ending with the \n", + " # output layer\n", + " model = Model(inputs=[inp],outputs=[dens])\n", + " # Compile the machine learning model using the mean squared error function as the loss \n", + " # function and an Adams optimizer.\n", + " model.compile(loss=\"mean_squared_error\", optimizer=\"adam\") \n", + " return model\n", + "\n", + "# Check to make sure the data set is complete\n", + "assert len(X_tot) == len(y_tot)\n", + "\n", + "# This is the number of points that will be used in as the training data\n", + "dim=12\n", + "\n", + "# Separate the training data from the whole data set\n", + "X_train = X_tot[:dim]\n", + "y_train = y_tot[:dim]\n", + "\n", + "\n", + "# Generate the training data for the RNN, using a sequence of 2\n", + "rnn_input, rnn_training = format_data(y_train, 2)\n", + "\n", + "\n", + "# Create a recurrent neural network in Keras and produce a summary of the \n", + "# machine learning model\n", + "model = rnn_2layers(length_of_sequences = 2)\n", + "model.summary()\n", + "\n", + "# Start the timer. Want to time training+testing\n", + "start = timer()\n", + "# Fit the model using the training data genenerated above using 150 training iterations and a 5%\n", + "# validation split. Setting verbose to True prints information about each training iteration.\n", + "hist = model.fit(rnn_input, rnn_training, batch_size=None, epochs=150, \n", + " verbose=True,validation_split=0.05)\n", + "\n", + "\n", + "# This section plots the training loss and the validation loss as a function of training iteration.\n", + "# This is not required for analyzing the couple cluster data but can help determine if the network is\n", + "# being overtrained.\n", + "for label in [\"loss\",\"val_loss\"]:\n", + " plt.plot(hist.history[label],label=label)\n", + "\n", + "plt.ylabel(\"loss\")\n", + "plt.xlabel(\"epoch\")\n", + "plt.title(\"The final validation loss: {}\".format(hist.history[\"val_loss\"][-1]))\n", + "plt.legend()\n", + "plt.show()\n", + "\n", + "# Use the trained neural network to predict more points of the data set\n", + "test_rnn(X_tot, y_tot, X_tot[0], X_tot[dim-1])\n", + "# Stop the timer and calculate the total time needed.\n", + "end = timer()\n", + "print('Time: ', end-start)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Other Types of Recurrent Neural Networks\n", + "\n", + "Besides a simple recurrent neural network layer, there are two other\n", + "commonly used types of recurrent neural network layers: Long Short\n", + "Term Memory (LSTM) and Gated Recurrent Unit (GRU). For a short\n", + "introduction to these layers see \n", + "and .\n", + "\n", + "The first network created below is similar to the previous network,\n", + "but it replaces the SimpleRNN layers with LSTM layers. The second\n", + "network below has two hidden layers made up of GRUs, which are\n", + "preceeded by two dense (feeddorward) neural network layers. These\n", + "dense layers \"preprocess\" the data before it reaches the recurrent\n", + "layers. This architecture has been shown to improve the performance\n", + "of recurrent neural networks (see the link above and also\n", + "." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "def lstm_2layers(length_of_sequences, batch_size = None, stateful = False):\n", + " \"\"\"\n", + " Inputs:\n", + " length_of_sequences (an int): the number of y values in \"x data\". This is determined\n", + " when the data is formatted\n", + " batch_size (an int): Default value is None. See Keras documentation of SimpleRNN.\n", + " stateful (a boolean): Default value is False. See Keras documentation of SimpleRNN.\n", + " Returns:\n", + " model (a Keras model): The recurrent neural network that is built and compiled by this\n", + " method\n", + " Builds and compiles a recurrent neural network with two LSTM hidden layers and returns the model.\n", + " \"\"\"\n", + " # Number of neurons on the input/output layer and the number of neurons in the hidden layer\n", + " in_out_neurons = 1\n", + " hidden_neurons = 250\n", + " # Input Layer\n", + " inp = Input(batch_shape=(batch_size, \n", + " length_of_sequences, \n", + " in_out_neurons)) \n", + " # Hidden layers (in this case they are LSTM layers instead if SimpleRNN layers)\n", + " rnn= LSTM(hidden_neurons, \n", + " return_sequences=True,\n", + " stateful = stateful,\n", + " name=\"RNN\", use_bias=True, activation='tanh')(inp)\n", + " rnn1 = LSTM(hidden_neurons, \n", + " return_sequences=False,\n", + " stateful = stateful,\n", + " name=\"RNN1\", use_bias=True, activation='tanh')(rnn)\n", + " # Output layer\n", + " dens = Dense(in_out_neurons,name=\"dense\")(rnn1)\n", + " # Define the midel\n", + " model = Model(inputs=[inp],outputs=[dens])\n", + " # Compile the model\n", + " model.compile(loss='mean_squared_error', optimizer='adam') \n", + " # Return the model\n", + " return model\n", + "\n", + "def dnn2_gru2(length_of_sequences, batch_size = None, stateful = False):\n", + " \"\"\"\n", + " Inputs:\n", + " length_of_sequences (an int): the number of y values in \"x data\". This is determined\n", + " when the data is formatted\n", + " batch_size (an int): Default value is None. See Keras documentation of SimpleRNN.\n", + " stateful (a boolean): Default value is False. See Keras documentation of SimpleRNN.\n", + " Returns:\n", + " model (a Keras model): The recurrent neural network that is built and compiled by this\n", + " method\n", + " Builds and compiles a recurrent neural network with four hidden layers (two dense followed by\n", + " two GRU layers) and returns the model.\n", + " \"\"\" \n", + " # Number of neurons on the input/output layers and hidden layers\n", + " in_out_neurons = 1\n", + " hidden_neurons = 250\n", + " # Input layer\n", + " inp = Input(batch_shape=(batch_size, \n", + " length_of_sequences, \n", + " in_out_neurons)) \n", + " # Hidden Dense (feedforward) layers\n", + " dnn = Dense(hidden_neurons/2, activation='relu', name='dnn')(inp)\n", + " dnn1 = Dense(hidden_neurons/2, activation='relu', name='dnn1')(dnn)\n", + " # Hidden GRU layers\n", + " rnn1 = GRU(hidden_neurons, \n", + " return_sequences=True,\n", + " stateful = stateful,\n", + " name=\"RNN1\", use_bias=True)(dnn1)\n", + " rnn = GRU(hidden_neurons, \n", + " return_sequences=False,\n", + " stateful = stateful,\n", + " name=\"RNN\", use_bias=True)(rnn1)\n", + " # Output layer\n", + " dens = Dense(in_out_neurons,name=\"dense\")(rnn)\n", + " # Define the model\n", + " model = Model(inputs=[inp],outputs=[dens])\n", + " # Compile the mdoel\n", + " model.compile(loss='mean_squared_error', optimizer='adam') \n", + " # Return the model\n", + " return model\n", + "\n", + "# Check to make sure the data set is complete\n", + "assert len(X_tot) == len(y_tot)\n", + "\n", + "# This is the number of points that will be used in as the training data\n", + "dim=12\n", + "\n", + "# Separate the training data from the whole data set\n", + "X_train = X_tot[:dim]\n", + "y_train = y_tot[:dim]\n", + "\n", + "\n", + "# Generate the training data for the RNN, using a sequence of 2\n", + "rnn_input, rnn_training = format_data(y_train, 2)\n", + "\n", + "\n", + "# Create a recurrent neural network in Keras and produce a summary of the \n", + "# machine learning model\n", + "# Change the method name to reflect which network you want to use\n", + "model = dnn2_gru2(length_of_sequences = 2)\n", + "model.summary()\n", + "\n", + "# Start the timer. Want to time training+testing\n", + "start = timer()\n", + "# Fit the model using the training data genenerated above using 150 training iterations and a 5%\n", + "# validation split. Setting verbose to True prints information about each training iteration.\n", + "hist = model.fit(rnn_input, rnn_training, batch_size=None, epochs=150, \n", + " verbose=True,validation_split=0.05)\n", + "\n", + "\n", + "# This section plots the training loss and the validation loss as a function of training iteration.\n", + "# This is not required for analyzing the couple cluster data but can help determine if the network is\n", + "# being overtrained.\n", + "for label in [\"loss\",\"val_loss\"]:\n", + " plt.plot(hist.history[label],label=label)\n", + "\n", + "plt.ylabel(\"loss\")\n", + "plt.xlabel(\"epoch\")\n", + "plt.title(\"The final validation loss: {}\".format(hist.history[\"val_loss\"][-1]))\n", + "plt.legend()\n", + "plt.show()\n", + "\n", + "# Use the trained neural network to predict more points of the data set\n", + "test_rnn(X_tot, y_tot, X_tot[0], X_tot[dim-1])\n", + "# Stop the timer and calculate the total time needed.\n", + "end = timer()\n", + "print('Time: ', end-start)\n", + "\n", + "\n", + "# ### Training Recurrent Neural Networks in the Standard Way (i.e. learning the relationship between the X and Y data)\n", + "# \n", + "# Finally, comparing the performace of a recurrent neural network using the standard data formatting to the performance of the network with time sequence data formatting shows the benefit of this type of data formatting with extrapolation.\n", + "\n", + "# Check to make sure the data set is complete\n", + "assert len(X_tot) == len(y_tot)\n", + "\n", + "# This is the number of points that will be used in as the training data\n", + "dim=12\n", + "\n", + "# Separate the training data from the whole data set\n", + "X_train = X_tot[:dim]\n", + "y_train = y_tot[:dim]\n", + "\n", + "# Reshape the data for Keras specifications\n", + "X_train = X_train.reshape((dim, 1))\n", + "y_train = y_train.reshape((dim, 1))\n", + "\n", + "\n", + "# Create a recurrent neural network in Keras and produce a summary of the \n", + "# machine learning model\n", + "# Set the sequence length to 1 for regular data formatting \n", + "model = rnn(length_of_sequences = 1)\n", + "model.summary()\n", + "\n", + "# Start the timer. Want to time training+testing\n", + "start = timer()\n", + "# Fit the model using the training data genenerated above using 150 training iterations and a 5%\n", + "# validation split. Setting verbose to True prints information about each training iteration.\n", + "hist = model.fit(X_train, y_train, batch_size=None, epochs=150, \n", + " verbose=True,validation_split=0.05)\n", + "\n", + "\n", + "# This section plots the training loss and the validation loss as a function of training iteration.\n", + "# This is not required for analyzing the couple cluster data but can help determine if the network is\n", + "# being overtrained.\n", + "for label in [\"loss\",\"val_loss\"]:\n", + " plt.plot(hist.history[label],label=label)\n", + "\n", + "plt.ylabel(\"loss\")\n", + "plt.xlabel(\"epoch\")\n", + "plt.title(\"The final validation loss: {}\".format(hist.history[\"val_loss\"][-1]))\n", + "plt.legend()\n", + "plt.show()\n", + "\n", + "# Use the trained neural network to predict the remaining data points\n", + "X_pred = X_tot[dim:]\n", + "X_pred = X_pred.reshape((len(X_pred), 1))\n", + "y_model = model.predict(X_pred)\n", + "y_pred = np.concatenate((y_tot[:dim], y_model.flatten()))\n", + "\n", + "# Plot the known data set and the predicted data set. The red box represents the region that was used\n", + "# for the training data.\n", + "fig, ax = plt.subplots()\n", + "ax.plot(X_tot, y_tot, label=\"true\", linewidth=3)\n", + "ax.plot(X_tot, y_pred, 'g-.',label=\"predicted\", linewidth=4)\n", + "ax.legend()\n", + "# Created a red region to represent the points used in the training data.\n", + "ax.axvspan(X_tot[0], X_tot[dim], alpha=0.25, color='red')\n", + "plt.show()\n", + "\n", + "# Stop the timer and calculate the total time needed.\n", + "end = timer()\n", + "print('Time: ', end-start)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Solving ODEs with Deep Learning\n", + "\n", + "The Universal Approximation Theorem states that a neural network can\n", + "approximate any function at a single hidden layer along with one input\n", + "and output layer to any given precision. \n", + "\n", + "\n", + "\n", + "## Ordinary Differential Equations\n", + "\n", + "An ordinary differential equation (ODE) is an equation involving functions having one variable.\n", + "\n", + "In general, an ordinary differential equation looks like" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "
\n", + "\n", + "$$\n", + "\\begin{equation} \\label{ode} \\tag{1}\n", + "f\\left(x, \\, g(x), \\, g'(x), \\, g''(x), \\, \\dots \\, , \\, g^{(n)}(x)\\right) = 0\n", + "\\end{equation}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "where $g(x)$ is the function to find, and $g^{(n)}(x)$ is the $n$-th derivative of $g(x)$.\n", + "\n", + "The $f\\left(x, g(x), g'(x), g''(x), \\, \\dots \\, , g^{(n)}(x)\\right)$ is just a way to write that there is an expression involving $x$ and $g(x), \\ g'(x), \\ g''(x), \\, \\dots \\, , \\text{ and } g^{(n)}(x)$ on the left side of the equality sign in ([1](#ode)).\n", + "The highest order of derivative, that is the value of $n$, determines to the order of the equation.\n", + "The equation is referred to as a $n$-th order ODE.\n", + "Along with ([1](#ode)), some additional conditions of the function $g(x)$ are typically given\n", + "for the solution to be unique.\n", + "\n", + "\n", + "## The trial solution\n", + "\n", + "Let the trial solution $g_t(x)$ be" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "
\n", + "\n", + "$$\n", + "\\begin{equation}\n", + "\tg_t(x) = h_1(x) + h_2(x,N(x,P))\n", + "\\label{_auto1} \\tag{2}\n", + "\\end{equation}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "where $h_1(x)$ is a function that makes $g_t(x)$ satisfy a given set\n", + "of conditions, $N(x,P)$ a neural network with weights and biases\n", + "described by $P$ and $h_2(x, N(x,P))$ some expression involving the\n", + "neural network. The role of the function $h_2(x, N(x,P))$, is to\n", + "ensure that the output from $N(x,P)$ is zero when $g_t(x)$ is\n", + "evaluated at the values of $x$ where the given conditions must be\n", + "satisfied. The function $h_1(x)$ should alone make $g_t(x)$ satisfy\n", + "the conditions.\n", + "\n", + "But what about the network $N(x,P)$?\n", + "\n", + "\n", + "As described previously, an optimization method could be used to minimize the parameters of a neural network, that being its weights and biases, through backward propagation.\n", + "\n", + "\n", + "\n", + "## Minimization process\n", + "\n", + "For the minimization to be defined, we need to have a cost function at hand to minimize.\n", + "\n", + "It is given that $f\\left(x, \\, g(x), \\, g'(x), \\, g''(x), \\, \\dots \\, , \\, g^{(n)}(x)\\right)$ should be equal to zero in ([1](#ode)).\n", + "We can choose to consider the mean squared error as the cost function for an input $x$.\n", + "Since we are looking at one input, the cost function is just $f$ squared.\n", + "The cost function $c\\left(x, P \\right)$ can therefore be expressed as" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "C\\left(x, P\\right) = \\big(f\\left(x, \\, g(x), \\, g'(x), \\, g''(x), \\, \\dots \\, , \\, g^{(n)}(x)\\right)\\big)^2\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If $N$ inputs are given as a vector $\\boldsymbol{x}$ with elements $x_i$ for $i = 1,\\dots,N$,\n", + "the cost function becomes" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "
\n", + "\n", + "$$\n", + "\\begin{equation} \\label{cost} \\tag{3}\n", + "\tC\\left(\\boldsymbol{x}, P\\right) = \\frac{1}{N} \\sum_{i=1}^N \\big(f\\left(x_i, \\, g(x_i), \\, g'(x_i), \\, g''(x_i), \\, \\dots \\, , \\, g^{(n)}(x_i)\\right)\\big)^2\n", + "\\end{equation}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The neural net should then find the parameters $P$ that minimizes the cost function in\n", + "([3](#cost)) for a set of $N$ training samples $x_i$.\n", + "\n", + "\n", + "## Minimizing the cost function using gradient descent and automatic differentiation\n", + "\n", + "To perform the minimization using gradient descent, the gradient of $C\\left(\\boldsymbol{x}, P\\right)$ is needed.\n", + "It might happen so that finding an analytical expression of the gradient of $C(\\boldsymbol{x}, P)$ from ([3](#cost)) gets too messy, depending on which cost function one desires to use.\n", + "\n", + "Luckily, there exists libraries that makes the job for us through automatic differentiation.\n", + "Automatic differentiation is a method of finding the derivatives numerically with very high precision.\n", + "\n", + "\n", + "\n", + "## Example: Exponential decay\n", + "\n", + "An exponential decay of a quantity $g(x)$ is described by the equation" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "
\n", + "\n", + "$$\n", + "\\begin{equation} \\label{solve_expdec} \\tag{4}\n", + " g'(x) = -\\gamma g(x)\n", + "\\end{equation}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "with $g(0) = g_0$ for some chosen initial value $g_0$.\n", + "\n", + "The analytical solution of ([4](#solve_expdec)) is" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "
\n", + "\n", + "$$\n", + "\\begin{equation}\n", + " g(x) = g_0 \\exp\\left(-\\gamma x\\right)\n", + "\\label{_auto2} \\tag{5}\n", + "\\end{equation}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Having an analytical solution at hand, it is possible to use it to compare how well a neural network finds a solution of ([4](#solve_expdec)).\n", + "\n", + "\n", + "\n", + "## The function to solve for\n", + "\n", + "The program will use a neural network to solve" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "
\n", + "\n", + "$$\n", + "\\begin{equation} \\label{solveode} \\tag{6}\n", + "g'(x) = -\\gamma g(x)\n", + "\\end{equation}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "where $g(0) = g_0$ with $\\gamma$ and $g_0$ being some chosen values.\n", + "\n", + "In this example, $\\gamma = 2$ and $g_0 = 10$.\n", + "\n", + "\n", + "## The trial solution\n", + "To begin with, a trial solution $g_t(t)$ must be chosen. A general trial solution for ordinary differential equations could be" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "g_t(x, P) = h_1(x) + h_2(x, N(x, P))\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "with $h_1(x)$ ensuring that $g_t(x)$ satisfies some conditions and $h_2(x,N(x, P))$ an expression involving $x$ and the output from the neural network $N(x,P)$ with $P $ being the collection of the weights and biases for each layer. For now, it is assumed that the network consists of one input layer, one hidden layer, and one output layer.\n", + "\n", + "\n", + "## Setup of Network\n", + "\n", + "In this network, there are no weights and bias at the input layer, so $P = \\{ P_{\\text{hidden}}, P_{\\text{output}} \\}$.\n", + "If there are $N_{\\text{hidden} }$ neurons in the hidden layer, then $P_{\\text{hidden}}$ is a $N_{\\text{hidden} } \\times (1 + N_{\\text{input}})$ matrix, given that there are $N_{\\text{input}}$ neurons in the input layer.\n", + "\n", + "The first column in $P_{\\text{hidden} }$ represents the bias for each neuron in the hidden layer and the second column represents the weights for each neuron in the hidden layer from the input layer.\n", + "If there are $N_{\\text{output} }$ neurons in the output layer, then $P_{\\text{output}} $ is a $N_{\\text{output} } \\times (1 + N_{\\text{hidden} })$ matrix.\n", + "\n", + "Its first column represents the bias of each neuron and the remaining columns represents the weights to each neuron.\n", + "\n", + "It is given that $g(0) = g_0$. The trial solution must fulfill this condition to be a proper solution of ([6](#solveode)). A possible way to ensure that $g_t(0, P) = g_0$, is to let $F(N(x,P)) = x \\cdot N(x,P)$ and $A(x) = g_0$. This gives the following trial solution:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "
\n", + "\n", + "$$\n", + "\\begin{equation} \\label{trial} \\tag{7}\n", + "g_t(x, P) = g_0 + x \\cdot N(x, P)\n", + "\\end{equation}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Reformulating the problem\n", + "\n", + "We wish that our neural network manages to minimize a given cost function.\n", + "\n", + "A reformulation of out equation, ([6](#solveode)), must therefore be done,\n", + "such that it describes the problem a neural network can solve for.\n", + "\n", + "The neural network must find the set of weights and biases $P$ such that the trial solution in ([7](#trial)) satisfies ([6](#solveode)).\n", + "\n", + "The trial solution" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "g_t(x, P) = g_0 + x \\cdot N(x, P)\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "has been chosen such that it already solves the condition $g(0) = g_0$. What remains, is to find $P$ such that" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "
\n", + "\n", + "$$\n", + "\\begin{equation} \\label{nnmin} \\tag{8}\n", + "g_t'(x, P) = - \\gamma g_t(x, P)\n", + "\\end{equation}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "is fulfilled as *best as possible*.\n", + "\n", + "\n", + "## More technicalities\n", + "\n", + "The left hand side and right hand side of ([8](#nnmin)) must be computed separately, and then the neural network must choose weights and biases, contained in $P$, such that the sides are equal as best as possible.\n", + "This means that the absolute or squared difference between the sides must be as close to zero, ideally equal to zero.\n", + "In this case, the difference squared shows to be an appropriate measurement of how erroneous the trial solution is with respect to $P$ of the neural network.\n", + "\n", + "This gives the following cost function our neural network must solve for:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "\\min_{P}\\Big\\{ \\big(g_t'(x, P) - ( -\\gamma g_t(x, P) \\big)^2 \\Big\\}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "(the notation $\\min_{P}\\{ f(x, P) \\}$ means that we desire to find $P$ that yields the minimum of $f(x, P)$)\n", + "\n", + "or, in terms of weights and biases for the hidden and output layer in our network:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "\\min_{P_{\\text{hidden} }, \\ P_{\\text{output} }}\\Big\\{ \\big(g_t'(x, \\{ P_{\\text{hidden} }, P_{\\text{output} }\\}) - ( -\\gamma g_t(x, \\{ P_{\\text{hidden} }, P_{\\text{output} }\\}) \\big)^2 \\Big\\}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "for an input value $x$.\n", + "\n", + "\n", + "## More details\n", + "\n", + "If the neural network evaluates $g_t(x, P)$ at more values for $x$, say $N$ values $x_i$ for $i = 1, \\dots, N$, then the *total* error to minimize becomes" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "
\n", + "\n", + "$$\n", + "\\begin{equation} \\label{min} \\tag{9}\n", + "\\min_{P}\\Big\\{\\frac{1}{N} \\sum_{i=1}^N \\big(g_t'(x_i, P) - ( -\\gamma g_t(x_i, P) \\big)^2 \\Big\\}\n", + "\\end{equation}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Letting $\\boldsymbol{x}$ be a vector with elements $x_i$ and $C(\\boldsymbol{x}, P) = \\frac{1}{N} \\sum_i \\big(g_t'(x_i, P) - ( -\\gamma g_t(x_i, P) \\big)^2$ denote the cost function, the minimization problem that our network must solve, becomes" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "\\min_{P} C(\\boldsymbol{x}, P)\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In terms of $P_{\\text{hidden} }$ and $P_{\\text{output} }$, this could also be expressed as\n", + "\n", + "$$\n", + "\\min_{P_{\\text{hidden} }, \\ P_{\\text{output} }} C(\\boldsymbol{x}, \\{P_{\\text{hidden} }, P_{\\text{output} }\\})\n", + "$$\n", + "\n", + "\n", + "## A possible implementation of a neural network\n", + "\n", + "For simplicity, it is assumed that the input is an array $\\boldsymbol{x} = (x_1, \\dots, x_N)$ with $N$ elements. It is at these points the neural network should find $P$ such that it fulfills ([9](#min)).\n", + "\n", + "First, the neural network must feed forward the inputs.\n", + "This means that $\\boldsymbol{x}s$ must be passed through an input layer, a hidden layer and a output layer. The input layer in this case, does not need to process the data any further.\n", + "The input layer will consist of $N_{\\text{input} }$ neurons, passing its element to each neuron in the hidden layer. The number of neurons in the hidden layer will be $N_{\\text{hidden} }$.\n", + "\n", + "\n", + "## Technicalities\n", + "\n", + "For the $i$-th in the hidden layer with weight $w_i^{\\text{hidden} }$ and bias $b_i^{\\text{hidden} }$, the weighting from the $j$-th neuron at the input layer is:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "\\begin{aligned}\n", + "z_{i,j}^{\\text{hidden}} &= b_i^{\\text{hidden}} + w_i^{\\text{hidden}}x_j \\\\\n", + "&=\n", + "\\begin{pmatrix}\n", + "b_i^{\\text{hidden}} & w_i^{\\text{hidden}}\n", + "\\end{pmatrix}\n", + "\\begin{pmatrix}\n", + "1 \\\\\n", + "x_j\n", + "\\end{pmatrix}\n", + "\\end{aligned}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Final technicalities I\n", + "\n", + "The result after weighting the inputs at the $i$-th hidden neuron can be written as a vector:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "\\begin{aligned}\n", + "\\boldsymbol{z}_{i}^{\\text{hidden}} &= \\Big( b_i^{\\text{hidden}} + w_i^{\\text{hidden}}x_1 , \\ b_i^{\\text{hidden}} + w_i^{\\text{hidden}} x_2, \\ \\dots \\, , \\ b_i^{\\text{hidden}} + w_i^{\\text{hidden}} x_N\\Big) \\\\\n", + "&=\n", + "\\begin{pmatrix}\n", + " b_i^{\\text{hidden}} & w_i^{\\text{hidden}}\n", + "\\end{pmatrix}\n", + "\\begin{pmatrix}\n", + "1 & 1 & \\dots & 1 \\\\\n", + "x_1 & x_2 & \\dots & x_N\n", + "\\end{pmatrix} \\\\\n", + "&= \\boldsymbol{p}_{i, \\text{hidden}}^T X\n", + "\\end{aligned}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Final technicalities II\n", + "\n", + "The vector $\\boldsymbol{p}_{i, \\text{hidden}}^T$ constitutes each row in $P_{\\text{hidden} }$, which contains the weights for the neural network to minimize according to ([9](#min)).\n", + "\n", + "After having found $\\boldsymbol{z}_{i}^{\\text{hidden}} $ for every $i$-th neuron within the hidden layer, the vector will be sent to an activation function $a_i(\\boldsymbol{z})$.\n", + "\n", + "In this example, the sigmoid function has been chosen to be the activation function for each hidden neuron:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "f(z) = \\frac{1}{1 + \\exp{(-z)}}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "It is possible to use other activations functions for the hidden layer also.\n", + "\n", + "The output $\\boldsymbol{x}_i^{\\text{hidden}}$ from each $i$-th hidden neuron is:\n", + "\n", + "$$\n", + "\\boldsymbol{x}_i^{\\text{hidden} } = f\\big( \\boldsymbol{z}_{i}^{\\text{hidden}} \\big)\n", + "$$\n", + "\n", + "The outputs $\\boldsymbol{x}_i^{\\text{hidden} } $ are then sent to the output layer.\n", + "\n", + "The output layer consists of one neuron in this case, and combines the\n", + "output from each of the neurons in the hidden layers. The output layer\n", + "combines the results from the hidden layer using some weights $w_i^{\\text{output}}$\n", + "and biases $b_i^{\\text{output}}$. In this case,\n", + "it is assumes that the number of neurons in the output layer is one.\n", + "\n", + "\n", + "## Final technicalities III\n", + "\n", + "\n", + "The procedure of weighting the output neuron $j$ in the hidden layer to the $i$-th neuron in the output layer is similar as for the hidden layer described previously." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "\\begin{aligned}\n", + "z_{1,j}^{\\text{output}} & =\n", + "\\begin{pmatrix}\n", + "b_1^{\\text{output}} & \\boldsymbol{w}_1^{\\text{output}}\n", + "\\end{pmatrix}\n", + "\\begin{pmatrix}\n", + "1 \\\\\n", + "\\boldsymbol{x}_j^{\\text{hidden}}\n", + "\\end{pmatrix}\n", + "\\end{aligned}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Final technicalities IV\n", + "\n", + "Expressing $z_{1,j}^{\\text{output}}$ as a vector gives the following way of weighting the inputs from the hidden layer:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "\\boldsymbol{z}_{1}^{\\text{output}} =\n", + "\\begin{pmatrix}\n", + "b_1^{\\text{output}} & \\boldsymbol{w}_1^{\\text{output}}\n", + "\\end{pmatrix}\n", + "\\begin{pmatrix}\n", + "1 & 1 & \\dots & 1 \\\\\n", + "\\boldsymbol{x}_1^{\\text{hidden}} & \\boldsymbol{x}_2^{\\text{hidden}} & \\dots & \\boldsymbol{x}_N^{\\text{hidden}}\n", + "\\end{pmatrix}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In this case we seek a continuous range of values since we are approximating a function. This means that after computing $\\boldsymbol{z}_{1}^{\\text{output}}$ the neural network has finished its feed forward step, and $\\boldsymbol{z}_{1}^{\\text{output}}$ is the final output of the network.\n", + "\n", + "\n", + "## Back propagation\n", + "\n", + "The next step is to decide how the parameters should be changed such that they minimize the cost function.\n", + "\n", + "The chosen cost function for this problem is" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "C(\\boldsymbol{x}, P) = \\frac{1}{N} \\sum_i \\big(g_t'(x_i, P) - ( -\\gamma g_t(x_i, P) \\big)^2\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In order to minimize the cost function, an optimization method must be chosen.\n", + "\n", + "Here, gradient descent with a constant step size has been chosen.\n", + "\n", + "\n", + "## Gradient descent\n", + "\n", + "The idea of the gradient descent algorithm is to update parameters in\n", + "a direction where the cost function decreases goes to a minimum.\n", + "\n", + "In general, the update of some parameters $\\boldsymbol{\\omega}$ given a cost\n", + "function defined by some weights $\\boldsymbol{\\omega}$, $C(\\boldsymbol{x},\n", + "\\boldsymbol{\\omega})$, goes as follows:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "\\boldsymbol{\\omega}_{\\text{new} } = \\boldsymbol{\\omega} - \\lambda \\nabla_{\\boldsymbol{\\omega}} C(\\boldsymbol{x}, \\boldsymbol{\\omega})\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "for a number of iterations or until $ \\big|\\big| \\boldsymbol{\\omega}_{\\text{new} } - \\boldsymbol{\\omega} \\big|\\big|$ becomes smaller than some given tolerance.\n", + "\n", + "The value of $\\lambda$ decides how large steps the algorithm must take\n", + "in the direction of $ \\nabla_{\\boldsymbol{\\omega}} C(\\boldsymbol{x}, \\boldsymbol{\\omega})$.\n", + "The notation $\\nabla_{\\boldsymbol{\\omega}}$ express the gradient with respect\n", + "to the elements in $\\boldsymbol{\\omega}$.\n", + "\n", + "In our case, we have to minimize the cost function $C(\\boldsymbol{x}, P)$ with\n", + "respect to the two sets of weights and biases, that is for the hidden\n", + "layer $P_{\\text{hidden} }$ and for the output layer $P_{\\text{output}\n", + "}$ .\n", + "\n", + "This means that $P_{\\text{hidden} }$ and $P_{\\text{output} }$ is updated by" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "\\begin{aligned}\n", + "P_{\\text{hidden},\\text{new}} &= P_{\\text{hidden}} - \\lambda \\nabla_{P_{\\text{hidden}}} C(\\boldsymbol{x}, P) \\\\\n", + "P_{\\text{output},\\text{new}} &= P_{\\text{output}} - \\lambda \\nabla_{P_{\\text{output}}} C(\\boldsymbol{x}, P)\n", + "\\end{aligned}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## The code for solving the ODE" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "import autograd.numpy as np\n", + "from autograd import grad, elementwise_grad\n", + "import autograd.numpy.random as npr\n", + "from matplotlib import pyplot as plt\n", + "\n", + "def sigmoid(z):\n", + " return 1/(1 + np.exp(-z))\n", + "\n", + "# Assuming one input, hidden, and output layer\n", + "def neural_network(params, x):\n", + "\n", + " # Find the weights (including and biases) for the hidden and output layer.\n", + " # Assume that params is a list of parameters for each layer.\n", + " # The biases are the first element for each array in params,\n", + " # and the weights are the remaning elements in each array in params.\n", + "\n", + " w_hidden = params[0]\n", + " w_output = params[1]\n", + "\n", + " # Assumes input x being an one-dimensional array\n", + " num_values = np.size(x)\n", + " x = x.reshape(-1, num_values)\n", + "\n", + " # Assume that the input layer does nothing to the input x\n", + " x_input = x\n", + "\n", + " ## Hidden layer:\n", + "\n", + " # Add a row of ones to include bias\n", + " x_input = np.concatenate((np.ones((1,num_values)), x_input ), axis = 0)\n", + "\n", + " z_hidden = np.matmul(w_hidden, x_input)\n", + " x_hidden = sigmoid(z_hidden)\n", + "\n", + " ## Output layer:\n", + "\n", + " # Include bias:\n", + " x_hidden = np.concatenate((np.ones((1,num_values)), x_hidden ), axis = 0)\n", + "\n", + " z_output = np.matmul(w_output, x_hidden)\n", + " x_output = z_output\n", + "\n", + " return x_output\n", + "\n", + "# The trial solution using the deep neural network:\n", + "def g_trial(x,params, g0 = 10):\n", + " return g0 + x*neural_network(params,x)\n", + "\n", + "# The right side of the ODE:\n", + "def g(x, g_trial, gamma = 2):\n", + " return -gamma*g_trial\n", + "\n", + "# The cost function:\n", + "def cost_function(P, x):\n", + "\n", + " # Evaluate the trial function with the current parameters P\n", + " g_t = g_trial(x,P)\n", + "\n", + " # Find the derivative w.r.t x of the neural network\n", + " d_net_out = elementwise_grad(neural_network,1)(P,x)\n", + "\n", + " # Find the derivative w.r.t x of the trial function\n", + " d_g_t = elementwise_grad(g_trial,0)(x,P)\n", + "\n", + " # The right side of the ODE\n", + " func = g(x, g_t)\n", + "\n", + " err_sqr = (d_g_t - func)**2\n", + " cost_sum = np.sum(err_sqr)\n", + "\n", + " return cost_sum / np.size(err_sqr)\n", + "\n", + "# Solve the exponential decay ODE using neural network with one input, hidden, and output layer\n", + "def solve_ode_neural_network(x, num_neurons_hidden, num_iter, lmb):\n", + " ## Set up initial weights and biases\n", + "\n", + " # For the hidden layer\n", + " p0 = npr.randn(num_neurons_hidden, 2 )\n", + "\n", + " # For the output layer\n", + " p1 = npr.randn(1, num_neurons_hidden + 1 ) # +1 since bias is included\n", + "\n", + " P = [p0, p1]\n", + "\n", + " print('Initial cost: %g'%cost_function(P, x))\n", + "\n", + " ## Start finding the optimal weights using gradient descent\n", + "\n", + " # Find the Python function that represents the gradient of the cost function\n", + " # w.r.t the 0-th input argument -- that is the weights and biases in the hidden and output layer\n", + " cost_function_grad = grad(cost_function,0)\n", + "\n", + " # Let the update be done num_iter times\n", + " for i in range(num_iter):\n", + " # Evaluate the gradient at the current weights and biases in P.\n", + " # The cost_grad consist now of two arrays;\n", + " # one for the gradient w.r.t P_hidden and\n", + " # one for the gradient w.r.t P_output\n", + " cost_grad = cost_function_grad(P, x)\n", + "\n", + " P[0] = P[0] - lmb * cost_grad[0]\n", + " P[1] = P[1] - lmb * cost_grad[1]\n", + "\n", + " print('Final cost: %g'%cost_function(P, x))\n", + "\n", + " return P\n", + "\n", + "def g_analytic(x, gamma = 2, g0 = 10):\n", + " return g0*np.exp(-gamma*x)\n", + "\n", + "# Solve the given problem\n", + "if __name__ == '__main__':\n", + " # Set seed such that the weight are initialized\n", + " # with same weights and biases for every run.\n", + " npr.seed(15)\n", + "\n", + " ## Decide the vales of arguments to the function to solve\n", + " N = 10\n", + " x = np.linspace(0, 1, N)\n", + "\n", + " ## Set up the initial parameters\n", + " num_hidden_neurons = 10\n", + " num_iter = 10000\n", + " lmb = 0.001\n", + "\n", + " # Use the network\n", + " P = solve_ode_neural_network(x, num_hidden_neurons, num_iter, lmb)\n", + "\n", + " # Print the deviation from the trial solution and true solution\n", + " res = g_trial(x,P)\n", + " res_analytical = g_analytic(x)\n", + "\n", + " print('Max absolute difference: %g'%np.max(np.abs(res - res_analytical)))\n", + "\n", + " # Plot the results\n", + " plt.figure(figsize=(10,10))\n", + "\n", + " plt.title('Performance of neural network solving an ODE compared to the analytical solution')\n", + " plt.plot(x, res_analytical)\n", + " plt.plot(x, res[0,:])\n", + " plt.legend(['analytical','nn'])\n", + " plt.xlabel('x')\n", + " plt.ylabel('g(x)')\n", + " plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## The network with one input layer, specified number of hidden layers, and one output layer\n", + "\n", + "It is also possible to extend the construction of our network into a more general one, allowing the network to contain more than one hidden layers.\n", + "\n", + "The number of neurons within each hidden layer are given as a list of integers in the program below." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "import autograd.numpy as np\n", + "from autograd import grad, elementwise_grad\n", + "import autograd.numpy.random as npr\n", + "from matplotlib import pyplot as plt\n", + "\n", + "def sigmoid(z):\n", + " return 1/(1 + np.exp(-z))\n", + "\n", + "# The neural network with one input layer and one output layer,\n", + "# but with number of hidden layers specified by the user.\n", + "def deep_neural_network(deep_params, x):\n", + " # N_hidden is the number of hidden layers\n", + "\n", + " N_hidden = np.size(deep_params) - 1 # -1 since params consists of\n", + " # parameters to all the hidden\n", + " # layers AND the output layer.\n", + "\n", + " # Assumes input x being an one-dimensional array\n", + " num_values = np.size(x)\n", + " x = x.reshape(-1, num_values)\n", + "\n", + " # Assume that the input layer does nothing to the input x\n", + " x_input = x\n", + "\n", + " # Due to multiple hidden layers, define a variable referencing to the\n", + " # output of the previous layer:\n", + " x_prev = x_input\n", + "\n", + " ## Hidden layers:\n", + "\n", + " for l in range(N_hidden):\n", + " # From the list of parameters P; find the correct weigths and bias for this layer\n", + " w_hidden = deep_params[l]\n", + "\n", + " # Add a row of ones to include bias\n", + " x_prev = np.concatenate((np.ones((1,num_values)), x_prev ), axis = 0)\n", + "\n", + " z_hidden = np.matmul(w_hidden, x_prev)\n", + " x_hidden = sigmoid(z_hidden)\n", + "\n", + " # Update x_prev such that next layer can use the output from this layer\n", + " x_prev = x_hidden\n", + "\n", + " ## Output layer:\n", + "\n", + " # Get the weights and bias for this layer\n", + " w_output = deep_params[-1]\n", + "\n", + " # Include bias:\n", + " x_prev = np.concatenate((np.ones((1,num_values)), x_prev), axis = 0)\n", + "\n", + " z_output = np.matmul(w_output, x_prev)\n", + " x_output = z_output\n", + "\n", + " return x_output\n", + "\n", + "# The trial solution using the deep neural network:\n", + "def g_trial_deep(x,params, g0 = 10):\n", + " return g0 + x*deep_neural_network(params, x)\n", + "\n", + "# The right side of the ODE:\n", + "def g(x, g_trial, gamma = 2):\n", + " return -gamma*g_trial\n", + "\n", + "# The same cost function as before, but calls deep_neural_network instead.\n", + "def cost_function_deep(P, x):\n", + "\n", + " # Evaluate the trial function with the current parameters P\n", + " g_t = g_trial_deep(x,P)\n", + "\n", + " # Find the derivative w.r.t x of the neural network\n", + " d_net_out = elementwise_grad(deep_neural_network,1)(P,x)\n", + "\n", + " # Find the derivative w.r.t x of the trial function\n", + " d_g_t = elementwise_grad(g_trial_deep,0)(x,P)\n", + "\n", + " # The right side of the ODE\n", + " func = g(x, g_t)\n", + "\n", + " err_sqr = (d_g_t - func)**2\n", + " cost_sum = np.sum(err_sqr)\n", + "\n", + " return cost_sum / np.size(err_sqr)\n", + "\n", + "# Solve the exponential decay ODE using neural network with one input and one output layer,\n", + "# but with specified number of hidden layers from the user.\n", + "def solve_ode_deep_neural_network(x, num_neurons, num_iter, lmb):\n", + " # num_hidden_neurons is now a list of number of neurons within each hidden layer\n", + "\n", + " # The number of elements in the list num_hidden_neurons thus represents\n", + " # the number of hidden layers.\n", + "\n", + " # Find the number of hidden layers:\n", + " N_hidden = np.size(num_neurons)\n", + "\n", + " ## Set up initial weights and biases\n", + "\n", + " # Initialize the list of parameters:\n", + " P = [None]*(N_hidden + 1) # + 1 to include the output layer\n", + "\n", + " P[0] = npr.randn(num_neurons[0], 2 )\n", + " for l in range(1,N_hidden):\n", + " P[l] = npr.randn(num_neurons[l], num_neurons[l-1] + 1) # +1 to include bias\n", + "\n", + " # For the output layer\n", + " P[-1] = npr.randn(1, num_neurons[-1] + 1 ) # +1 since bias is included\n", + "\n", + " print('Initial cost: %g'%cost_function_deep(P, x))\n", + "\n", + " ## Start finding the optimal weights using gradient descent\n", + "\n", + " # Find the Python function that represents the gradient of the cost function\n", + " # w.r.t the 0-th input argument -- that is the weights and biases in the hidden and output layer\n", + " cost_function_deep_grad = grad(cost_function_deep,0)\n", + "\n", + " # Let the update be done num_iter times\n", + " for i in range(num_iter):\n", + " # Evaluate the gradient at the current weights and biases in P.\n", + " # The cost_grad consist now of N_hidden + 1 arrays; the gradient w.r.t the weights and biases\n", + " # in the hidden layers and output layers evaluated at x.\n", + " cost_deep_grad = cost_function_deep_grad(P, x)\n", + "\n", + " for l in range(N_hidden+1):\n", + " P[l] = P[l] - lmb * cost_deep_grad[l]\n", + "\n", + " print('Final cost: %g'%cost_function_deep(P, x))\n", + "\n", + " return P\n", + "\n", + "def g_analytic(x, gamma = 2, g0 = 10):\n", + " return g0*np.exp(-gamma*x)\n", + "\n", + "# Solve the given problem\n", + "if __name__ == '__main__':\n", + " npr.seed(15)\n", + "\n", + " ## Decide the vales of arguments to the function to solve\n", + " N = 10\n", + " x = np.linspace(0, 1, N)\n", + "\n", + " ## Set up the initial parameters\n", + " num_hidden_neurons = np.array([10,10])\n", + " num_iter = 10000\n", + " lmb = 0.001\n", + "\n", + " P = solve_ode_deep_neural_network(x, num_hidden_neurons, num_iter, lmb)\n", + "\n", + " res = g_trial_deep(x,P)\n", + " res_analytical = g_analytic(x)\n", + "\n", + " plt.figure(figsize=(10,10))\n", + "\n", + " plt.title('Performance of a deep neural network solving an ODE compared to the analytical solution')\n", + " plt.plot(x, res_analytical)\n", + " plt.plot(x, res[0,:])\n", + " plt.legend(['analytical','dnn'])\n", + " plt.ylabel('g(x)')\n", + " plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Example: Population growth\n", + "\n", + "A logistic model of population growth assumes that a population converges toward an equilibrium.\n", + "The population growth can be modeled by" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "
\n", + "\n", + "$$\n", + "\\begin{equation} \\label{log} \\tag{10}\n", + "\tg'(t) = \\alpha g(t)(A - g(t))\n", + "\\end{equation}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "where $g(t)$ is the population density at time $t$, $\\alpha > 0$ the growth rate and $A > 0$ is the maximum population number in the environment.\n", + "Also, at $t = 0$ the population has the size $g(0) = g_0$, where $g_0$ is some chosen constant.\n", + "\n", + "In this example, similar network as for the exponential decay using Autograd has been used to solve the equation. However, as the implementation might suffer from e.g numerical instability\n", + "and high execution time (this might be more apparent in the examples solving PDEs),\n", + "using a library like TensorFlow is recommended.\n", + "Here, we stay with a more simple approach and implement for comparison, the simple forward Euler method.\n", + "\n", + "\n", + "## Setting up the problem\n", + "\n", + "Here, we will model a population $g(t)$ in an environment having carrying capacity $A$.\n", + "The population follows the model" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "
\n", + "\n", + "$$\n", + "\\begin{equation} \\label{solveode_population} \\tag{11}\n", + "g'(t) = \\alpha g(t)(A - g(t))\n", + "\\end{equation}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "where $g(0) = g_0$.\n", + "\n", + "In this example, we let $\\alpha = 2$, $A = 1$, and $g_0 = 1.2$.\n", + "\n", + "\n", + "## The trial solution\n", + "\n", + "We will get a slightly different trial solution, as the boundary conditions are different\n", + "compared to the case for exponential decay.\n", + "\n", + "A possible trial solution satisfying the condition $g(0) = g_0$ could be\n", + "\n", + "$$\n", + "h_1(t) = g_0 + t \\cdot N(t,P)\n", + "$$\n", + "\n", + "with $N(t,P)$ being the output from the neural network with weights and biases for each layer collected in the set $P$.\n", + "\n", + "The analytical solution is\n", + "\n", + "$$\n", + "g(t) = \\frac{Ag_0}{g_0 + (A - g_0)\\exp(-\\alpha A t)}\n", + "$$\n", + "\n", + "\n", + "## The program using Autograd\n", + "\n", + "The network will be the similar as for the exponential decay example, but with some small modifications for our problem." + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "import autograd.numpy as np\n", + "from autograd import grad, elementwise_grad\n", + "import autograd.numpy.random as npr\n", + "from matplotlib import pyplot as plt\n", + "\n", + "def sigmoid(z):\n", + " return 1/(1 + np.exp(-z))\n", + "\n", + "# Function to get the parameters.\n", + "# Done such that one can easily change the paramaters after one's liking.\n", + "def get_parameters():\n", + " alpha = 2\n", + " A = 1\n", + " g0 = 1.2\n", + " return alpha, A, g0\n", + "\n", + "def deep_neural_network(P, x):\n", + " # N_hidden is the number of hidden layers\n", + " N_hidden = np.size(P) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n", + "\n", + " # Assumes input x being an one-dimensional array\n", + " num_values = np.size(x)\n", + " x = x.reshape(-1, num_values)\n", + "\n", + " # Assume that the input layer does nothing to the input x\n", + " x_input = x\n", + "\n", + " # Due to multiple hidden layers, define a variable referencing to the\n", + " # output of the previous layer:\n", + " x_prev = x_input\n", + "\n", + " ## Hidden layers:\n", + "\n", + " for l in range(N_hidden):\n", + " # From the list of parameters P; find the correct weigths and bias for this layer\n", + " w_hidden = P[l]\n", + "\n", + " # Add a row of ones to include bias\n", + " x_prev = np.concatenate((np.ones((1,num_values)), x_prev ), axis = 0)\n", + "\n", + " z_hidden = np.matmul(w_hidden, x_prev)\n", + " x_hidden = sigmoid(z_hidden)\n", + "\n", + " # Update x_prev such that next layer can use the output from this layer\n", + " x_prev = x_hidden\n", + "\n", + " ## Output layer:\n", + "\n", + " # Get the weights and bias for this layer\n", + " w_output = P[-1]\n", + "\n", + " # Include bias:\n", + " x_prev = np.concatenate((np.ones((1,num_values)), x_prev), axis = 0)\n", + "\n", + " z_output = np.matmul(w_output, x_prev)\n", + " x_output = z_output\n", + "\n", + " return x_output\n", + "\n", + "\n", + "def cost_function_deep(P, x):\n", + "\n", + " # Evaluate the trial function with the current parameters P\n", + " g_t = g_trial_deep(x,P)\n", + "\n", + " # Find the derivative w.r.t x of the trial function\n", + " d_g_t = elementwise_grad(g_trial_deep,0)(x,P)\n", + "\n", + " # The right side of the ODE\n", + " func = f(x, g_t)\n", + "\n", + " err_sqr = (d_g_t - func)**2\n", + " cost_sum = np.sum(err_sqr)\n", + "\n", + " return cost_sum / np.size(err_sqr)\n", + "\n", + "# The right side of the ODE:\n", + "def f(x, g_trial):\n", + " alpha,A, g0 = get_parameters()\n", + " return alpha*g_trial*(A - g_trial)\n", + "\n", + "# The trial solution using the deep neural network:\n", + "def g_trial_deep(x, params):\n", + " alpha,A, g0 = get_parameters()\n", + " return g0 + x*deep_neural_network(params,x)\n", + "\n", + "# The analytical solution:\n", + "def g_analytic(t):\n", + " alpha,A, g0 = get_parameters()\n", + " return A*g0/(g0 + (A - g0)*np.exp(-alpha*A*t))\n", + "\n", + "def solve_ode_deep_neural_network(x, num_neurons, num_iter, lmb):\n", + " # num_hidden_neurons is now a list of number of neurons within each hidden layer\n", + "\n", + " # Find the number of hidden layers:\n", + " N_hidden = np.size(num_neurons)\n", + "\n", + " ## Set up initial weigths and biases\n", + "\n", + " # Initialize the list of parameters:\n", + " P = [None]*(N_hidden + 1) # + 1 to include the output layer\n", + "\n", + " P[0] = npr.randn(num_neurons[0], 2 )\n", + " for l in range(1,N_hidden):\n", + " P[l] = npr.randn(num_neurons[l], num_neurons[l-1] + 1) # +1 to include bias\n", + "\n", + " # For the output layer\n", + " P[-1] = npr.randn(1, num_neurons[-1] + 1 ) # +1 since bias is included\n", + "\n", + " print('Initial cost: %g'%cost_function_deep(P, x))\n", + "\n", + " ## Start finding the optimal weigths using gradient descent\n", + "\n", + " # Find the Python function that represents the gradient of the cost function\n", + " # w.r.t the 0-th input argument -- that is the weights and biases in the hidden and output layer\n", + " cost_function_deep_grad = grad(cost_function_deep,0)\n", + "\n", + " # Let the update be done num_iter times\n", + " for i in range(num_iter):\n", + " # Evaluate the gradient at the current weights and biases in P.\n", + " # The cost_grad consist now of N_hidden + 1 arrays; the gradient w.r.t the weights and biases\n", + " # in the hidden layers and output layers evaluated at x.\n", + " cost_deep_grad = cost_function_deep_grad(P, x)\n", + "\n", + " for l in range(N_hidden+1):\n", + " P[l] = P[l] - lmb * cost_deep_grad[l]\n", + "\n", + " print('Final cost: %g'%cost_function_deep(P, x))\n", + "\n", + " return P\n", + "\n", + "if __name__ == '__main__':\n", + " npr.seed(4155)\n", + "\n", + " ## Decide the vales of arguments to the function to solve\n", + " Nt = 10\n", + " T = 1\n", + " t = np.linspace(0,T, Nt)\n", + "\n", + " ## Set up the initial parameters\n", + " num_hidden_neurons = [100, 50, 25]\n", + " num_iter = 1000\n", + " lmb = 1e-3\n", + "\n", + " P = solve_ode_deep_neural_network(t, num_hidden_neurons, num_iter, lmb)\n", + "\n", + " g_dnn_ag = g_trial_deep(t,P)\n", + " g_analytical = g_analytic(t)\n", + "\n", + " # Find the maximum absolute difference between the solutons:\n", + " diff_ag = np.max(np.abs(g_dnn_ag - g_analytical))\n", + " print(\"The max absolute difference between the solutions is: %g\"%diff_ag)\n", + "\n", + " plt.figure(figsize=(10,10))\n", + "\n", + " plt.title('Performance of neural network solving an ODE compared to the analytical solution')\n", + " plt.plot(t, g_analytical)\n", + " plt.plot(t, g_dnn_ag[0,:])\n", + " plt.legend(['analytical','nn'])\n", + " plt.xlabel('t')\n", + " plt.ylabel('g(t)')\n", + "\n", + " plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Using forward Euler to solve the ODE\n", + "\n", + "A straightforward way of solving an ODE numerically, is to use Euler's method.\n", + "\n", + "Euler's method uses Taylor series to approximate the value at a function $f$ at a step $\\Delta x$ from $x$:\n", + "\n", + "$$\n", + "f(x + \\Delta x) \\approx f(x) + \\Delta x f'(x)\n", + "$$\n", + "\n", + "In our case, using Euler's method to approximate the value of $g$ at a step $\\Delta t$ from $t$ yields" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "\\begin{aligned}\n", + " g(t + \\Delta t) &\\approx g(t) + \\Delta t g'(t) \\\\\n", + " &= g(t) + \\Delta t \\big(\\alpha g(t)(A - g(t))\\big)\n", + "\\end{aligned}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "along with the condition that $g(0) = g_0$.\n", + "\n", + "Let $t_i = i \\cdot \\Delta t$ where $\\Delta t = \\frac{T}{N_t-1}$ where $T$ is the final time our solver must solve for and $N_t$ the number of values for $t \\in [0, T]$ for $i = 0, \\dots, N_t-1$.\n", + "\n", + "For $i \\geq 1$, we have that" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "\\begin{aligned}\n", + "t_i &= i\\Delta t \\\\\n", + "&= (i - 1)\\Delta t + \\Delta t \\\\\n", + "&= t_{i-1} + \\Delta t\n", + "\\end{aligned}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now, if $g_i = g(t_i)$ then" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "
\n", + "\n", + "$$\n", + "\\begin{equation}\n", + " \\begin{aligned}\n", + " g_i &= g(t_i) \\\\\n", + " &= g(t_{i-1} + \\Delta t) \\\\\n", + " &\\approx g(t_{i-1}) + \\Delta t \\big(\\alpha g(t_{i-1})(A - g(t_{i-1}))\\big) \\\\\n", + " &= g_{i-1} + \\Delta t \\big(\\alpha g_{i-1}(A - g_{i-1})\\big)\n", + " \\end{aligned}\n", + "\\end{equation} \\label{odenum} \\tag{12}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "for $i \\geq 1$ and $g_0 = g(t_0) = g(0) = g_0$.\n", + "\n", + "Equation ([12](#odenum)) could be implemented in the following way,\n", + "extending the program that uses the network using Autograd:" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "# Assume that all function definitions from the example program using Autograd\n", + "# are located here.\n", + "\n", + "if __name__ == '__main__':\n", + " npr.seed(4155)\n", + "\n", + " ## Decide the vales of arguments to the function to solve\n", + " Nt = 10\n", + " T = 1\n", + " t = np.linspace(0,T, Nt)\n", + "\n", + " ## Set up the initial parameters\n", + " num_hidden_neurons = [100,50,25]\n", + " num_iter = 1000\n", + " lmb = 1e-3\n", + "\n", + " P = solve_ode_deep_neural_network(t, num_hidden_neurons, num_iter, lmb)\n", + "\n", + " g_dnn_ag = g_trial_deep(t,P)\n", + " g_analytical = g_analytic(t)\n", + "\n", + " # Find the maximum absolute difference between the solutons:\n", + " diff_ag = np.max(np.abs(g_dnn_ag - g_analytical))\n", + " print(\"The max absolute difference between the solutions is: %g\"%diff_ag)\n", + "\n", + " plt.figure(figsize=(10,10))\n", + "\n", + " plt.title('Performance of neural network solving an ODE compared to the analytical solution')\n", + " plt.plot(t, g_analytical)\n", + " plt.plot(t, g_dnn_ag[0,:])\n", + " plt.legend(['analytical','nn'])\n", + " plt.xlabel('t')\n", + " plt.ylabel('g(t)')\n", + "\n", + " ## Find an approximation to the funtion using forward Euler\n", + "\n", + " alpha, A, g0 = get_parameters()\n", + " dt = T/(Nt - 1)\n", + "\n", + " # Perform forward Euler to solve the ODE\n", + " g_euler = np.zeros(Nt)\n", + " g_euler[0] = g0\n", + "\n", + " for i in range(1,Nt):\n", + " g_euler[i] = g_euler[i-1] + dt*(alpha*g_euler[i-1]*(A - g_euler[i-1]))\n", + "\n", + " # Print the errors done by each method\n", + " diff1 = np.max(np.abs(g_euler - g_analytical))\n", + " diff2 = np.max(np.abs(g_dnn_ag[0,:] - g_analytical))\n", + "\n", + " print('Max absolute difference between Euler method and analytical: %g'%diff1)\n", + " print('Max absolute difference between deep neural network and analytical: %g'%diff2)\n", + "\n", + " # Plot results\n", + " plt.figure(figsize=(10,10))\n", + "\n", + " plt.plot(t,g_euler)\n", + " plt.plot(t,g_analytical)\n", + " plt.plot(t,g_dnn_ag[0,:])\n", + "\n", + " plt.legend(['euler','analytical','dnn'])\n", + " plt.xlabel('Time t')\n", + " plt.ylabel('g(t)')\n", + "\n", + " plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Example: Solving the one dimensional Poisson equation\n", + "\n", + "The Poisson equation for $g(x)$ in one dimension is" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "
\n", + "\n", + "$$\n", + "\\begin{equation} \\label{poisson} \\tag{13}\n", + " -g''(x) = f(x)\n", + "\\end{equation}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "where $f(x)$ is a given function for $x \\in (0,1)$.\n", + "\n", + "The conditions that $g(x)$ is chosen to fulfill, are" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "\\begin{align*}\n", + " g(0) &= 0 \\\\\n", + " g(1) &= 0\n", + "\\end{align*}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This equation can be solved numerically using programs where e.g Autograd and TensorFlow are used.\n", + "The results from the networks can then be compared to the analytical solution.\n", + "In addition, it could be interesting to see how a typical method for numerically solving second order ODEs compares to the neural networks.\n", + "\n", + "\n", + "## The specific equation to solve for\n", + "\n", + "Here, the function $g(x)$ to solve for follows the equation" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "-g''(x) = f(x),\\qquad x \\in (0,1)\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "where $f(x)$ is a given function, along with the chosen conditions" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "
\n", + "\n", + "$$\n", + "\\begin{aligned}\n", + "g(0) = g(1) = 0\n", + "\\end{aligned}\\label{cond} \\tag{14}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In this example, we consider the case when $f(x) = (3x + x^2)\\exp(x)$.\n", + "\n", + "For this case, a possible trial solution satisfying the conditions could be" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "g_t(x) = x \\cdot (1-x) \\cdot N(P,x)\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The analytical solution for this problem is" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "g(x) = x(1 - x)\\exp(x)\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Solving the equation using Autograd" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "import autograd.numpy as np\n", + "from autograd import grad, elementwise_grad\n", + "import autograd.numpy.random as npr\n", + "from matplotlib import pyplot as plt\n", + "\n", + "def sigmoid(z):\n", + " return 1/(1 + np.exp(-z))\n", + "\n", + "def deep_neural_network(deep_params, x):\n", + " # N_hidden is the number of hidden layers\n", + " N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n", + "\n", + " # Assumes input x being an one-dimensional array\n", + " num_values = np.size(x)\n", + " x = x.reshape(-1, num_values)\n", + "\n", + " # Assume that the input layer does nothing to the input x\n", + " x_input = x\n", + "\n", + " # Due to multiple hidden layers, define a variable referencing to the\n", + " # output of the previous layer:\n", + " x_prev = x_input\n", + "\n", + " ## Hidden layers:\n", + "\n", + " for l in range(N_hidden):\n", + " # From the list of parameters P; find the correct weigths and bias for this layer\n", + " w_hidden = deep_params[l]\n", + "\n", + " # Add a row of ones to include bias\n", + " x_prev = np.concatenate((np.ones((1,num_values)), x_prev ), axis = 0)\n", + "\n", + " z_hidden = np.matmul(w_hidden, x_prev)\n", + " x_hidden = sigmoid(z_hidden)\n", + "\n", + " # Update x_prev such that next layer can use the output from this layer\n", + " x_prev = x_hidden\n", + "\n", + " ## Output layer:\n", + "\n", + " # Get the weights and bias for this layer\n", + " w_output = deep_params[-1]\n", + "\n", + " # Include bias:\n", + " x_prev = np.concatenate((np.ones((1,num_values)), x_prev), axis = 0)\n", + "\n", + " z_output = np.matmul(w_output, x_prev)\n", + " x_output = z_output\n", + "\n", + " return x_output\n", + "\n", + "def solve_ode_deep_neural_network(x, num_neurons, num_iter, lmb):\n", + " # num_hidden_neurons is now a list of number of neurons within each hidden layer\n", + "\n", + " # Find the number of hidden layers:\n", + " N_hidden = np.size(num_neurons)\n", + "\n", + " ## Set up initial weigths and biases\n", + "\n", + " # Initialize the list of parameters:\n", + " P = [None]*(N_hidden + 1) # + 1 to include the output layer\n", + "\n", + " P[0] = npr.randn(num_neurons[0], 2 )\n", + " for l in range(1,N_hidden):\n", + " P[l] = npr.randn(num_neurons[l], num_neurons[l-1] + 1) # +1 to include bias\n", + "\n", + " # For the output layer\n", + " P[-1] = npr.randn(1, num_neurons[-1] + 1 ) # +1 since bias is included\n", + "\n", + " print('Initial cost: %g'%cost_function_deep(P, x))\n", + "\n", + " ## Start finding the optimal weigths using gradient descent\n", + "\n", + " # Find the Python function that represents the gradient of the cost function\n", + " # w.r.t the 0-th input argument -- that is the weights and biases in the hidden and output layer\n", + " cost_function_deep_grad = grad(cost_function_deep,0)\n", + "\n", + " # Let the update be done num_iter times\n", + " for i in range(num_iter):\n", + " # Evaluate the gradient at the current weights and biases in P.\n", + " # The cost_grad consist now of N_hidden + 1 arrays; the gradient w.r.t the weights and biases\n", + " # in the hidden layers and output layers evaluated at x.\n", + " cost_deep_grad = cost_function_deep_grad(P, x)\n", + "\n", + " for l in range(N_hidden+1):\n", + " P[l] = P[l] - lmb * cost_deep_grad[l]\n", + "\n", + " print('Final cost: %g'%cost_function_deep(P, x))\n", + "\n", + " return P\n", + "\n", + "## Set up the cost function specified for this Poisson equation:\n", + "\n", + "# The right side of the ODE\n", + "def f(x):\n", + " return (3*x + x**2)*np.exp(x)\n", + "\n", + "def cost_function_deep(P, x):\n", + "\n", + " # Evaluate the trial function with the current parameters P\n", + " g_t = g_trial_deep(x,P)\n", + "\n", + " # Find the derivative w.r.t x of the trial function\n", + " d2_g_t = elementwise_grad(elementwise_grad(g_trial_deep,0))(x,P)\n", + "\n", + " right_side = f(x)\n", + "\n", + " err_sqr = (-d2_g_t - right_side)**2\n", + " cost_sum = np.sum(err_sqr)\n", + "\n", + " return cost_sum/np.size(err_sqr)\n", + "\n", + "# The trial solution:\n", + "def g_trial_deep(x,P):\n", + " return x*(1-x)*deep_neural_network(P,x)\n", + "\n", + "# The analytic solution;\n", + "def g_analytic(x):\n", + " return x*(1-x)*np.exp(x)\n", + "\n", + "if __name__ == '__main__':\n", + " npr.seed(4155)\n", + "\n", + " ## Decide the vales of arguments to the function to solve\n", + " Nx = 10\n", + " x = np.linspace(0,1, Nx)\n", + "\n", + " ## Set up the initial parameters\n", + " num_hidden_neurons = [200,100]\n", + " num_iter = 1000\n", + " lmb = 1e-3\n", + "\n", + " P = solve_ode_deep_neural_network(x, num_hidden_neurons, num_iter, lmb)\n", + "\n", + " g_dnn_ag = g_trial_deep(x,P)\n", + " g_analytical = g_analytic(x)\n", + "\n", + " # Find the maximum absolute difference between the solutons:\n", + " max_diff = np.max(np.abs(g_dnn_ag - g_analytical))\n", + " print(\"The max absolute difference between the solutions is: %g\"%max_diff)\n", + "\n", + " plt.figure(figsize=(10,10))\n", + "\n", + " plt.title('Performance of neural network solving an ODE compared to the analytical solution')\n", + " plt.plot(x, g_analytical)\n", + " plt.plot(x, g_dnn_ag[0,:])\n", + " plt.legend(['analytical','nn'])\n", + " plt.xlabel('x')\n", + " plt.ylabel('g(x)')\n", + " plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Comparing with a numerical scheme\n", + "\n", + "The Poisson equation is possible to solve using Taylor series to approximate the second derivative.\n", + "\n", + "Using Taylor series, the second derivative can be expressed as\n", + "\n", + "$$\n", + "g''(x) = \\frac{g(x + \\Delta x) - 2g(x) + g(x-\\Delta x)}{\\Delta x^2} + E_{\\Delta x}(x)\n", + "$$\n", + "\n", + "where $\\Delta x$ is a small step size and $E_{\\Delta x}(x)$ being the error term.\n", + "\n", + "Looking away from the error terms gives an approximation to the second derivative:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "
\n", + "\n", + "$$\n", + "\\begin{equation} \\label{approx} \\tag{15}\n", + "g''(x) \\approx \\frac{g(x + \\Delta x) - 2g(x) + g(x-\\Delta x)}{\\Delta x^2}\n", + "\\end{equation}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If $x_i = i \\Delta x = x_{i-1} + \\Delta x$ and $g_i = g(x_i)$ for $i = 1,\\dots N_x - 2$ with $N_x$ being the number of values for $x$, ([15](#approx)) becomes" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "\\begin{aligned}\n", + "g''(x_i) &\\approx \\frac{g(x_i + \\Delta x) - 2g(x_i) + g(x_i -\\Delta x)}{\\Delta x^2} \\\\\n", + "&= \\frac{g_{i+1} - 2g_i + g_{i-1}}{\\Delta x^2}\n", + "\\end{aligned}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Since we know from our problem that" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "\\begin{aligned}\n", + "-g''(x) &= f(x) \\\\\n", + "&= (3x + x^2)\\exp(x)\n", + "\\end{aligned}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "along with the conditions $g(0) = g(1) = 0$,\n", + "the following scheme can be used to find an approximate solution for $g(x)$ numerically:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "
\n", + "\n", + "$$\n", + "\\begin{equation}\n", + " \\begin{aligned}\n", + " -\\Big( \\frac{g_{i+1} - 2g_i + g_{i-1}}{\\Delta x^2} \\Big) &= f(x_i) \\\\\n", + " -g_{i+1} + 2g_i - g_{i-1} &= \\Delta x^2 f(x_i)\n", + " \\end{aligned}\n", + "\\end{equation} \\label{odesys} \\tag{16}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "for $i = 1, \\dots, N_x - 2$ where $g_0 = g_{N_x - 1} = 0$ and $f(x_i) = (3x_i + x_i^2)\\exp(x_i)$, which is given for our specific problem.\n", + "\n", + "The equation can be rewritten into a matrix equation:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "\\begin{aligned}\n", + "\\begin{pmatrix}\n", + "2 & -1 & 0 & \\dots & 0 \\\\\n", + "-1 & 2 & -1 & \\dots & 0 \\\\\n", + "\\vdots & & \\ddots & & \\vdots \\\\\n", + "0 & \\dots & -1 & 2 & -1 \\\\\n", + "0 & \\dots & 0 & -1 & 2\\\\\n", + "\\end{pmatrix}\n", + "\\begin{pmatrix}\n", + "g_1 \\\\\n", + "g_2 \\\\\n", + "\\vdots \\\\\n", + "g_{N_x - 3} \\\\\n", + "g_{N_x - 2}\n", + "\\end{pmatrix}\n", + "&=\n", + "\\Delta x^2\n", + "\\begin{pmatrix}\n", + "f(x_1) \\\\\n", + "f(x_2) \\\\\n", + "\\vdots \\\\\n", + "f(x_{N_x - 3}) \\\\\n", + "f(x_{N_x - 2})\n", + "\\end{pmatrix} \\\\\n", + "\\boldsymbol{A}\\boldsymbol{g} &= \\boldsymbol{f},\n", + "\\end{aligned}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "which makes it possible to solve for the vector $\\boldsymbol{g}$.\n", + "\n", + "\n", + "## Setting up the code\n", + "\n", + "We can then compare the result from this numerical scheme with the output from our network using Autograd:" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "import autograd.numpy as np\n", + "from autograd import grad, elementwise_grad\n", + "import autograd.numpy.random as npr\n", + "from matplotlib import pyplot as plt\n", + "\n", + "def sigmoid(z):\n", + " return 1/(1 + np.exp(-z))\n", + "\n", + "def deep_neural_network(deep_params, x):\n", + " # N_hidden is the number of hidden layers\n", + " N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n", + "\n", + " # Assumes input x being an one-dimensional array\n", + " num_values = np.size(x)\n", + " x = x.reshape(-1, num_values)\n", + "\n", + " # Assume that the input layer does nothing to the input x\n", + " x_input = x\n", + "\n", + " # Due to multiple hidden layers, define a variable referencing to the\n", + " # output of the previous layer:\n", + " x_prev = x_input\n", + "\n", + " ## Hidden layers:\n", + "\n", + " for l in range(N_hidden):\n", + " # From the list of parameters P; find the correct weigths and bias for this layer\n", + " w_hidden = deep_params[l]\n", + "\n", + " # Add a row of ones to include bias\n", + " x_prev = np.concatenate((np.ones((1,num_values)), x_prev ), axis = 0)\n", + "\n", + " z_hidden = np.matmul(w_hidden, x_prev)\n", + " x_hidden = sigmoid(z_hidden)\n", + "\n", + " # Update x_prev such that next layer can use the output from this layer\n", + " x_prev = x_hidden\n", + "\n", + " ## Output layer:\n", + "\n", + " # Get the weights and bias for this layer\n", + " w_output = deep_params[-1]\n", + "\n", + " # Include bias:\n", + " x_prev = np.concatenate((np.ones((1,num_values)), x_prev), axis = 0)\n", + "\n", + " z_output = np.matmul(w_output, x_prev)\n", + " x_output = z_output\n", + "\n", + " return x_output\n", + "\n", + "def solve_ode_deep_neural_network(x, num_neurons, num_iter, lmb):\n", + " # num_hidden_neurons is now a list of number of neurons within each hidden layer\n", + "\n", + " # Find the number of hidden layers:\n", + " N_hidden = np.size(num_neurons)\n", + "\n", + " ## Set up initial weigths and biases\n", + "\n", + " # Initialize the list of parameters:\n", + " P = [None]*(N_hidden + 1) # + 1 to include the output layer\n", + "\n", + " P[0] = npr.randn(num_neurons[0], 2 )\n", + " for l in range(1,N_hidden):\n", + " P[l] = npr.randn(num_neurons[l], num_neurons[l-1] + 1) # +1 to include bias\n", + "\n", + " # For the output layer\n", + " P[-1] = npr.randn(1, num_neurons[-1] + 1 ) # +1 since bias is included\n", + "\n", + " print('Initial cost: %g'%cost_function_deep(P, x))\n", + "\n", + " ## Start finding the optimal weigths using gradient descent\n", + "\n", + " # Find the Python function that represents the gradient of the cost function\n", + " # w.r.t the 0-th input argument -- that is the weights and biases in the hidden and output layer\n", + " cost_function_deep_grad = grad(cost_function_deep,0)\n", + "\n", + " # Let the update be done num_iter times\n", + " for i in range(num_iter):\n", + " # Evaluate the gradient at the current weights and biases in P.\n", + " # The cost_grad consist now of N_hidden + 1 arrays; the gradient w.r.t the weights and biases\n", + " # in the hidden layers and output layers evaluated at x.\n", + " cost_deep_grad = cost_function_deep_grad(P, x)\n", + "\n", + " for l in range(N_hidden+1):\n", + " P[l] = P[l] - lmb * cost_deep_grad[l]\n", + "\n", + " print('Final cost: %g'%cost_function_deep(P, x))\n", + "\n", + " return P\n", + "\n", + "## Set up the cost function specified for this Poisson equation:\n", + "\n", + "# The right side of the ODE\n", + "def f(x):\n", + " return (3*x + x**2)*np.exp(x)\n", + "\n", + "def cost_function_deep(P, x):\n", + "\n", + " # Evaluate the trial function with the current parameters P\n", + " g_t = g_trial_deep(x,P)\n", + "\n", + " # Find the derivative w.r.t x of the trial function\n", + " d2_g_t = elementwise_grad(elementwise_grad(g_trial_deep,0))(x,P)\n", + "\n", + " right_side = f(x)\n", + "\n", + " err_sqr = (-d2_g_t - right_side)**2\n", + " cost_sum = np.sum(err_sqr)\n", + "\n", + " return cost_sum/np.size(err_sqr)\n", + "\n", + "# The trial solution:\n", + "def g_trial_deep(x,P):\n", + " return x*(1-x)*deep_neural_network(P,x)\n", + "\n", + "# The analytic solution;\n", + "def g_analytic(x):\n", + " return x*(1-x)*np.exp(x)\n", + "\n", + "if __name__ == '__main__':\n", + " npr.seed(4155)\n", + "\n", + " ## Decide the vales of arguments to the function to solve\n", + " Nx = 10\n", + " x = np.linspace(0,1, Nx)\n", + "\n", + " ## Set up the initial parameters\n", + " num_hidden_neurons = [200,100]\n", + " num_iter = 1000\n", + " lmb = 1e-3\n", + "\n", + " P = solve_ode_deep_neural_network(x, num_hidden_neurons, num_iter, lmb)\n", + "\n", + " g_dnn_ag = g_trial_deep(x,P)\n", + " g_analytical = g_analytic(x)\n", + "\n", + " # Find the maximum absolute difference between the solutons:\n", + "\n", + " plt.figure(figsize=(10,10))\n", + "\n", + " plt.title('Performance of neural network solving an ODE compared to the analytical solution')\n", + " plt.plot(x, g_analytical)\n", + " plt.plot(x, g_dnn_ag[0,:])\n", + " plt.legend(['analytical','nn'])\n", + " plt.xlabel('x')\n", + " plt.ylabel('g(x)')\n", + "\n", + " ## Perform the computation using the numerical scheme\n", + "\n", + " dx = 1/(Nx - 1)\n", + "\n", + " # Set up the matrix A\n", + " A = np.zeros((Nx-2,Nx-2))\n", + "\n", + " A[0,0] = 2\n", + " A[0,1] = -1\n", + "\n", + " for i in range(1,Nx-3):\n", + " A[i,i-1] = -1\n", + " A[i,i] = 2\n", + " A[i,i+1] = -1\n", + "\n", + " A[Nx - 3, Nx - 4] = -1\n", + " A[Nx - 3, Nx - 3] = 2\n", + "\n", + " # Set up the vector f\n", + " f_vec = dx**2 * f(x[1:-1])\n", + "\n", + " # Solve the equation\n", + " g_res = np.linalg.solve(A,f_vec)\n", + "\n", + " g_vec = np.zeros(Nx)\n", + " g_vec[1:-1] = g_res\n", + "\n", + " # Print the differences between each method\n", + " max_diff1 = np.max(np.abs(g_dnn_ag - g_analytical))\n", + " max_diff2 = np.max(np.abs(g_vec - g_analytical))\n", + " print(\"The max absolute difference between the analytical solution and DNN Autograd: %g\"%max_diff1)\n", + " print(\"The max absolute difference between the analytical solution and numerical scheme: %g\"%max_diff2)\n", + "\n", + " # Plot the results\n", + " plt.figure(figsize=(10,10))\n", + "\n", + " plt.plot(x,g_vec)\n", + " plt.plot(x,g_analytical)\n", + " plt.plot(x,g_dnn_ag[0,:])\n", + "\n", + " plt.legend(['numerical scheme','analytical','dnn'])\n", + " plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Partial Differential Equations\n", + "\n", + "A partial differential equation (PDE) has a solution here the function\n", + "is defined by multiple variables. The equation may involve all kinds\n", + "of combinations of which variables the function is differentiated with\n", + "respect to.\n", + "\n", + "In general, a partial differential equation for a function $g(x_1,\\dots,x_N)$ with $N$ variables may be expressed as" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "
\n", + "\n", + "$$\n", + "\\begin{equation} \\label{PDE} \\tag{17}\n", + " f\\left(x_1, \\, \\dots \\, , x_N, \\frac{\\partial g(x_1,\\dots,x_N) }{\\partial x_1}, \\dots , \\frac{\\partial g(x_1,\\dots,x_N) }{\\partial x_N}, \\frac{\\partial g(x_1,\\dots,x_N) }{\\partial x_1\\partial x_2}, \\, \\dots \\, , \\frac{\\partial^n g(x_1,\\dots,x_N) }{\\partial x_N^n} \\right) = 0\n", + "\\end{equation}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "where $f$ is an expression involving all kinds of possible mixed derivatives of $g(x_1,\\dots,x_N)$ up to an order $n$. In order for the solution to be unique, some additional conditions must also be given.\n", + "\n", + "\n", + "## Type of problem\n", + "\n", + "The problem our network must solve for, is similar to the ODE case.\n", + "We must have a trial solution $g_t$ at hand.\n", + "\n", + "For instance, the trial solution could be expressed as" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "\\begin{align*}\n", + " g_t(x_1,\\dots,x_N) = h_1(x_1,\\dots,x_N) + h_2(x_1,\\dots,x_N,N(x_1,\\dots,x_N,P))\n", + "\\end{align*}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "where $h_1(x_1,\\dots,x_N)$ is a function that ensures $g_t(x_1,\\dots,x_N)$ satisfies some given conditions.\n", + "The neural network $N(x_1,\\dots,x_N,P)$ has weights and biases described by $P$ and $h_2(x_1,\\dots,x_N,N(x_1,\\dots,x_N,P))$ is an expression using the output from the neural network in some way.\n", + "\n", + "The role of the function $h_2(x_1,\\dots,x_N,N(x_1,\\dots,x_N,P))$, is to ensure that the output of $N(x_1,\\dots,x_N,P)$ is zero when $g_t(x_1,\\dots,x_N)$ is evaluated at the values of $x_1,\\dots,x_N$ where the given conditions must be satisfied. The function $h_1(x_1,\\dots,x_N)$ should alone make $g_t(x_1,\\dots,x_N)$ satisfy the conditions.\n", + "\n", + "\n", + "\n", + "## Network requirements\n", + "\n", + "The network tries then the minimize the cost function following the\n", + "same ideas as described for the ODE case, but now with more than one\n", + "variables to consider. The concept still remains the same; find a set\n", + "of parameters $P$ such that the expression $f$ in ([17](#PDE)) is as\n", + "close to zero as possible.\n", + "\n", + "As for the ODE case, the cost function is the mean squared error that\n", + "the network must try to minimize. The cost function for the network to\n", + "minimize is" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "C\\left(x_1, \\dots, x_N, P\\right) = \\left( f\\left(x_1, \\, \\dots \\, , x_N, \\frac{\\partial g(x_1,\\dots,x_N) }{\\partial x_1}, \\dots , \\frac{\\partial g(x_1,\\dots,x_N) }{\\partial x_N}, \\frac{\\partial g(x_1,\\dots,x_N) }{\\partial x_1\\partial x_2}, \\, \\dots \\, , \\frac{\\partial^n g(x_1,\\dots,x_N) }{\\partial x_N^n} \\right) \\right)^2\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## More details\n", + "\n", + "If we let $\\boldsymbol{x} = \\big( x_1, \\dots, x_N \\big)$ be an array containing the values for $x_1, \\dots, x_N$ respectively, the cost function can be reformulated into the following:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "C\\left(\\boldsymbol{x}, P\\right) = f\\left( \\left( \\boldsymbol{x}, \\frac{\\partial g(\\boldsymbol{x}) }{\\partial x_1}, \\dots , \\frac{\\partial g(\\boldsymbol{x}) }{\\partial x_N}, \\frac{\\partial g(\\boldsymbol{x}) }{\\partial x_1\\partial x_2}, \\, \\dots \\, , \\frac{\\partial^n g(\\boldsymbol{x}) }{\\partial x_N^n} \\right) \\right)^2\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If we also have $M$ different sets of values for $x_1, \\dots, x_N$, that is $\\boldsymbol{x}_i = \\big(x_1^{(i)}, \\dots, x_N^{(i)}\\big)$ for $i = 1,\\dots,M$ being the rows in matrix $X$, the cost function can be generalized into" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "C\\left(X, P \\right) = \\sum_{i=1}^M f\\left( \\left( \\boldsymbol{x}_i, \\frac{\\partial g(\\boldsymbol{x}_i) }{\\partial x_1}, \\dots , \\frac{\\partial g(\\boldsymbol{x}_i) }{\\partial x_N}, \\frac{\\partial g(\\boldsymbol{x}_i) }{\\partial x_1\\partial x_2}, \\, \\dots \\, , \\frac{\\partial^n g(\\boldsymbol{x}_i) }{\\partial x_N^n} \\right) \\right)^2.\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Example: The diffusion equation\n", + "\n", + "In one spatial dimension, the equation reads" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "\\frac{\\partial g(x,t)}{\\partial t} = \\frac{\\partial^2 g(x,t)}{\\partial x^2}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "where a possible choice of conditions are" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "\\begin{align*}\n", + "g(0,t) &= 0 ,\\qquad t \\geq 0 \\\\\n", + "g(1,t) &= 0, \\qquad t \\geq 0 \\\\\n", + "g(x,0) &= u(x),\\qquad x\\in [0,1]\n", + "\\end{align*}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "with $u(x)$ being some given function.\n", + "\n", + "\n", + "## Defining the problem\n", + "\n", + "For this case, we want to find $g(x,t)$ such that" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "
\n", + "\n", + "$$\n", + "\\begin{equation}\n", + " \\frac{\\partial g(x,t)}{\\partial t} = \\frac{\\partial^2 g(x,t)}{\\partial x^2}\n", + "\\end{equation} \\label{diffonedim} \\tag{18}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "and" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "\\begin{align*}\n", + "g(0,t) &= 0 ,\\qquad t \\geq 0 \\\\\n", + "g(1,t) &= 0, \\qquad t \\geq 0 \\\\\n", + "g(x,0) &= u(x),\\qquad x\\in [0,1]\n", + "\\end{align*}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "with $u(x) = \\sin(\\pi x)$.\n", + "\n", + "First, let us set up the deep neural network.\n", + "The deep neural network will follow the same structure as discussed in the examples solving the ODEs.\n", + "First, we will look into how Autograd could be used in a network tailored to solve for bivariate functions.\n", + "\n", + "\n", + "\n", + "\n", + "## Setting up the network using Autograd\n", + "\n", + "The only change to do here, is to extend our network such that\n", + "functions of multiple parameters are correctly handled. In this case\n", + "we have two variables in our function to solve for, that is time $t$\n", + "and position $x$. The variables will be represented by a\n", + "one-dimensional array in the program. The program will evaluate the\n", + "network at each possible pair $(x,t)$, given an array for the desired\n", + "$x$-values and $t$-values to approximate the solution at." + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "def sigmoid(z):\n", + " return 1/(1 + np.exp(-z))\n", + "\n", + "def deep_neural_network(deep_params, x):\n", + " # x is now a point and a 1D numpy array; make it a column vector\n", + " num_coordinates = np.size(x,0)\n", + " x = x.reshape(num_coordinates,-1)\n", + "\n", + " num_points = np.size(x,1)\n", + "\n", + " # N_hidden is the number of hidden layers\n", + " N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n", + "\n", + " # Assume that the input layer does nothing to the input x\n", + " x_input = x\n", + " x_prev = x_input\n", + "\n", + " ## Hidden layers:\n", + "\n", + " for l in range(N_hidden):\n", + " # From the list of parameters P; find the correct weigths and bias for this layer\n", + " w_hidden = deep_params[l]\n", + "\n", + " # Add a row of ones to include bias\n", + " x_prev = np.concatenate((np.ones((1,num_points)), x_prev ), axis = 0)\n", + "\n", + " z_hidden = np.matmul(w_hidden, x_prev)\n", + " x_hidden = sigmoid(z_hidden)\n", + "\n", + " # Update x_prev such that next layer can use the output from this layer\n", + " x_prev = x_hidden\n", + "\n", + " ## Output layer:\n", + "\n", + " # Get the weights and bias for this layer\n", + " w_output = deep_params[-1]\n", + "\n", + " # Include bias:\n", + " x_prev = np.concatenate((np.ones((1,num_points)), x_prev), axis = 0)\n", + "\n", + " z_output = np.matmul(w_output, x_prev)\n", + " x_output = z_output\n", + "\n", + " return x_output[0][0]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Setting up the network using Autograd; The trial solution\n", + "\n", + "The cost function must then iterate through the given arrays\n", + "containing values for $x$ and $t$, defines a point $(x,t)$ the deep\n", + "neural network and the trial solution is evaluated at, and then finds\n", + "the Jacobian of the trial solution.\n", + "\n", + "A possible trial solution for this PDE is\n", + "\n", + "$$\n", + "g_t(x,t) = h_1(x,t) + x(1-x)tN(x,t,P)\n", + "$$\n", + "\n", + "with $A(x,t)$ being a function ensuring that $g_t(x,t)$ satisfies our given conditions, and $N(x,t,P)$ being the output from the deep neural network using weights and biases for each layer from $P$.\n", + "\n", + "To fulfill the conditions, $A(x,t)$ could be:\n", + "\n", + "$$\n", + "h_1(x,t) = (1-t)\\Big(u(x) - \\big((1-x)u(0) + x u(1)\\big)\\Big) = (1-t)u(x) = (1-t)\\sin(\\pi x)\n", + "$$\n", + "since $(0) = u(1) = 0$ and $u(x) = \\sin(\\pi x)$.\n", + "\n", + "\n", + "## Why the jacobian?\n", + "\n", + "The Jacobian is used because the program must find the derivative of\n", + "the trial solution with respect to $x$ and $t$.\n", + "\n", + "This gives the necessity of computing the Jacobian matrix, as we want\n", + "to evaluate the gradient with respect to $x$ and $t$ (note that the\n", + "Jacobian of a scalar-valued multivariate function is simply its\n", + "gradient).\n", + "\n", + "In Autograd, the differentiation is by default done with respect to\n", + "the first input argument of your Python function. Since the points is\n", + "an array representing $x$ and $t$, the Jacobian is calculated using\n", + "the values of $x$ and $t$.\n", + "\n", + "To find the second derivative with respect to $x$ and $t$, the\n", + "Jacobian can be found for the second time. The result is a Hessian\n", + "matrix, which is the matrix containing all the possible second order\n", + "mixed derivatives of $g(x,t)$." + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "# Set up the trial function:\n", + "def u(x):\n", + " return np.sin(np.pi*x)\n", + "\n", + "def g_trial(point,P):\n", + " x,t = point\n", + " return (1-t)*u(x) + x*(1-x)*t*deep_neural_network(P,point)\n", + "\n", + "# The right side of the ODE:\n", + "def f(point):\n", + " return 0.\n", + "\n", + "# The cost function:\n", + "def cost_function(P, x, t):\n", + " cost_sum = 0\n", + "\n", + " g_t_jacobian_func = jacobian(g_trial)\n", + " g_t_hessian_func = hessian(g_trial)\n", + "\n", + " for x_ in x:\n", + " for t_ in t:\n", + " point = np.array([x_,t_])\n", + "\n", + " g_t = g_trial(point,P)\n", + " g_t_jacobian = g_t_jacobian_func(point,P)\n", + " g_t_hessian = g_t_hessian_func(point,P)\n", + "\n", + " g_t_dt = g_t_jacobian[1]\n", + " g_t_d2x = g_t_hessian[0][0]\n", + "\n", + " func = f(point)\n", + "\n", + " err_sqr = ( (g_t_dt - g_t_d2x) - func)**2\n", + " cost_sum += err_sqr\n", + "\n", + " return cost_sum" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Setting up the network using Autograd; The full program\n", + "\n", + "Having set up the network, along with the trial solution and cost function, we can now see how the deep neural network performs by comparing the results to the analytical solution.\n", + "\n", + "The analytical solution of our problem is\n", + "\n", + "$$\n", + "g(x,t) = \\exp(-\\pi^2 t)\\sin(\\pi x)\n", + "$$\n", + "\n", + "A possible way to implement a neural network solving the PDE, is given below.\n", + "Be aware, though, that it is fairly slow for the parameters used.\n", + "A better result is possible, but requires more iterations, and thus longer time to complete.\n", + "\n", + "\n", + "Indeed, the program below is not optimal in its implementation, but rather serves as an example on how to implement and use a neural network to solve a PDE.\n", + "Using TensorFlow results in a much better execution time. Try it!" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "import autograd.numpy as np\n", + "from autograd import jacobian,hessian,grad\n", + "import autograd.numpy.random as npr\n", + "from matplotlib import cm\n", + "from matplotlib import pyplot as plt\n", + "from mpl_toolkits.mplot3d import axes3d\n", + "\n", + "## Set up the network\n", + "\n", + "def sigmoid(z):\n", + " return 1/(1 + np.exp(-z))\n", + "\n", + "def deep_neural_network(deep_params, x):\n", + " # x is now a point and a 1D numpy array; make it a column vector\n", + " num_coordinates = np.size(x,0)\n", + " x = x.reshape(num_coordinates,-1)\n", + "\n", + " num_points = np.size(x,1)\n", + "\n", + " # N_hidden is the number of hidden layers\n", + " N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n", + "\n", + " # Assume that the input layer does nothing to the input x\n", + " x_input = x\n", + " x_prev = x_input\n", + "\n", + " ## Hidden layers:\n", + "\n", + " for l in range(N_hidden):\n", + " # From the list of parameters P; find the correct weigths and bias for this layer\n", + " w_hidden = deep_params[l]\n", + "\n", + " # Add a row of ones to include bias\n", + " x_prev = np.concatenate((np.ones((1,num_points)), x_prev ), axis = 0)\n", + "\n", + " z_hidden = np.matmul(w_hidden, x_prev)\n", + " x_hidden = sigmoid(z_hidden)\n", + "\n", + " # Update x_prev such that next layer can use the output from this layer\n", + " x_prev = x_hidden\n", + "\n", + " ## Output layer:\n", + "\n", + " # Get the weights and bias for this layer\n", + " w_output = deep_params[-1]\n", + "\n", + " # Include bias:\n", + " x_prev = np.concatenate((np.ones((1,num_points)), x_prev), axis = 0)\n", + "\n", + " z_output = np.matmul(w_output, x_prev)\n", + " x_output = z_output\n", + "\n", + " return x_output[0][0]\n", + "\n", + "## Define the trial solution and cost function\n", + "def u(x):\n", + " return np.sin(np.pi*x)\n", + "\n", + "def g_trial(point,P):\n", + " x,t = point\n", + " return (1-t)*u(x) + x*(1-x)*t*deep_neural_network(P,point)\n", + "\n", + "# The right side of the ODE:\n", + "def f(point):\n", + " return 0.\n", + "\n", + "# The cost function:\n", + "def cost_function(P, x, t):\n", + " cost_sum = 0\n", + "\n", + " g_t_jacobian_func = jacobian(g_trial)\n", + " g_t_hessian_func = hessian(g_trial)\n", + "\n", + " for x_ in x:\n", + " for t_ in t:\n", + " point = np.array([x_,t_])\n", + "\n", + " g_t = g_trial(point,P)\n", + " g_t_jacobian = g_t_jacobian_func(point,P)\n", + " g_t_hessian = g_t_hessian_func(point,P)\n", + "\n", + " g_t_dt = g_t_jacobian[1]\n", + " g_t_d2x = g_t_hessian[0][0]\n", + "\n", + " func = f(point)\n", + "\n", + " err_sqr = ( (g_t_dt - g_t_d2x) - func)**2\n", + " cost_sum += err_sqr\n", + "\n", + " return cost_sum /( np.size(x)*np.size(t) )\n", + "\n", + "## For comparison, define the analytical solution\n", + "def g_analytic(point):\n", + " x,t = point\n", + " return np.exp(-np.pi**2*t)*np.sin(np.pi*x)\n", + "\n", + "## Set up a function for training the network to solve for the equation\n", + "def solve_pde_deep_neural_network(x,t, num_neurons, num_iter, lmb):\n", + " ## Set up initial weigths and biases\n", + " N_hidden = np.size(num_neurons)\n", + "\n", + " ## Set up initial weigths and biases\n", + "\n", + " # Initialize the list of parameters:\n", + " P = [None]*(N_hidden + 1) # + 1 to include the output layer\n", + "\n", + " P[0] = npr.randn(num_neurons[0], 2 + 1 ) # 2 since we have two points, +1 to include bias\n", + " for l in range(1,N_hidden):\n", + " P[l] = npr.randn(num_neurons[l], num_neurons[l-1] + 1) # +1 to include bias\n", + "\n", + " # For the output layer\n", + " P[-1] = npr.randn(1, num_neurons[-1] + 1 ) # +1 since bias is included\n", + "\n", + " print('Initial cost: ',cost_function(P, x, t))\n", + "\n", + " cost_function_grad = grad(cost_function,0)\n", + "\n", + " # Let the update be done num_iter times\n", + " for i in range(num_iter):\n", + " cost_grad = cost_function_grad(P, x , t)\n", + "\n", + " for l in range(N_hidden+1):\n", + " P[l] = P[l] - lmb * cost_grad[l]\n", + "\n", + " print('Final cost: ',cost_function(P, x, t))\n", + "\n", + " return P\n", + "\n", + "if __name__ == '__main__':\n", + " ### Use the neural network:\n", + " npr.seed(15)\n", + "\n", + " ## Decide the vales of arguments to the function to solve\n", + " Nx = 10; Nt = 10\n", + " x = np.linspace(0, 1, Nx)\n", + " t = np.linspace(0,1,Nt)\n", + "\n", + " ## Set up the parameters for the network\n", + " num_hidden_neurons = [100, 25]\n", + " num_iter = 250\n", + " lmb = 0.01\n", + "\n", + " P = solve_pde_deep_neural_network(x,t, num_hidden_neurons, num_iter, lmb)\n", + "\n", + " ## Store the results\n", + " g_dnn_ag = np.zeros((Nx, Nt))\n", + " G_analytical = np.zeros((Nx, Nt))\n", + " for i,x_ in enumerate(x):\n", + " for j, t_ in enumerate(t):\n", + " point = np.array([x_, t_])\n", + " g_dnn_ag[i,j] = g_trial(point,P)\n", + "\n", + " G_analytical[i,j] = g_analytic(point)\n", + "\n", + " # Find the map difference between the analytical and the computed solution\n", + " diff_ag = np.abs(g_dnn_ag - G_analytical)\n", + " print('Max absolute difference between the analytical solution and the network: %g'%np.max(diff_ag))\n", + "\n", + " ## Plot the solutions in two dimensions, that being in position and time\n", + "\n", + " T,X = np.meshgrid(t,x)\n", + "\n", + " fig = plt.figure(figsize=(10,10))\n", + " ax = fig.gca(projection='3d')\n", + " ax.set_title('Solution from the deep neural network w/ %d layer'%len(num_hidden_neurons))\n", + " s = ax.plot_surface(T,X,g_dnn_ag,linewidth=0,antialiased=False,cmap=cm.viridis)\n", + " ax.set_xlabel('Time $t$')\n", + " ax.set_ylabel('Position $x$');\n", + "\n", + "\n", + " fig = plt.figure(figsize=(10,10))\n", + " ax = fig.gca(projection='3d')\n", + " ax.set_title('Analytical solution')\n", + " s = ax.plot_surface(T,X,G_analytical,linewidth=0,antialiased=False,cmap=cm.viridis)\n", + " ax.set_xlabel('Time $t$')\n", + " ax.set_ylabel('Position $x$');\n", + "\n", + " fig = plt.figure(figsize=(10,10))\n", + " ax = fig.gca(projection='3d')\n", + " ax.set_title('Difference')\n", + " s = ax.plot_surface(T,X,diff_ag,linewidth=0,antialiased=False,cmap=cm.viridis)\n", + " ax.set_xlabel('Time $t$')\n", + " ax.set_ylabel('Position $x$');\n", + "\n", + " ## Take some slices of the 3D plots just to see the solutions at particular times\n", + " indx1 = 0\n", + " indx2 = int(Nt/2)\n", + " indx3 = Nt-1\n", + "\n", + " t1 = t[indx1]\n", + " t2 = t[indx2]\n", + " t3 = t[indx3]\n", + "\n", + " # Slice the results from the DNN\n", + " res1 = g_dnn_ag[:,indx1]\n", + " res2 = g_dnn_ag[:,indx2]\n", + " res3 = g_dnn_ag[:,indx3]\n", + "\n", + " # Slice the analytical results\n", + " res_analytical1 = G_analytical[:,indx1]\n", + " res_analytical2 = G_analytical[:,indx2]\n", + " res_analytical3 = G_analytical[:,indx3]\n", + "\n", + " # Plot the slices\n", + " plt.figure(figsize=(10,10))\n", + " plt.title(\"Computed solutions at time = %g\"%t1)\n", + " plt.plot(x, res1)\n", + " plt.plot(x,res_analytical1)\n", + " plt.legend(['dnn','analytical'])\n", + "\n", + " plt.figure(figsize=(10,10))\n", + " plt.title(\"Computed solutions at time = %g\"%t2)\n", + " plt.plot(x, res2)\n", + " plt.plot(x,res_analytical2)\n", + " plt.legend(['dnn','analytical'])\n", + "\n", + " plt.figure(figsize=(10,10))\n", + " plt.title(\"Computed solutions at time = %g\"%t3)\n", + " plt.plot(x, res3)\n", + " plt.plot(x,res_analytical3)\n", + " plt.legend(['dnn','analytical'])\n", + "\n", + " plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Example: Solving the wave equation with Neural Networks\n", + "\n", + "The wave equation is" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "\\frac{\\partial^2 g(x,t)}{\\partial t^2} = c^2\\frac{\\partial^2 g(x,t)}{\\partial x^2}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "with $c$ being the specified wave speed.\n", + "\n", + "Here, the chosen conditions are" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "\\begin{align*}\n", + "\tg(0,t) &= 0 \\\\\n", + "\tg(1,t) &= 0 \\\\\n", + "\tg(x,0) &= u(x) \\\\\n", + "\t\\frac{\\partial g(x,t)}{\\partial t} \\Big |_{t = 0} &= v(x)\n", + "\\end{align*}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "where $\\frac{\\partial g(x,t)}{\\partial t} \\Big |_{t = 0}$ means the derivative of $g(x,t)$ with respect to $t$ is evaluated at $t = 0$, and $u(x)$ and $v(x)$ being given functions.\n", + "\n", + "\n", + "## The problem to solve for\n", + "\n", + "The wave equation to solve for, is" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "
\n", + "\n", + "$$\n", + "\\begin{equation} \\label{wave} \\tag{19}\n", + "\\frac{\\partial^2 g(x,t)}{\\partial t^2} = c^2 \\frac{\\partial^2 g(x,t)}{\\partial x^2}\n", + "\\end{equation}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "where $c$ is the given wave speed.\n", + "The chosen conditions for this equation are" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "
\n", + "\n", + "$$\n", + "\\begin{aligned}\n", + "g(0,t) &= 0, &t \\geq 0 \\\\\n", + "g(1,t) &= 0, &t \\geq 0 \\\\\n", + "g(x,0) &= u(x), &x\\in[0,1] \\\\\n", + "\\frac{\\partial g(x,t)}{\\partial t}\\Big |_{t = 0} &= v(x), &x \\in [0,1]\n", + "\\end{aligned} \\label{condwave} \\tag{20}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In this example, let $c = 1$ and $u(x) = \\sin(\\pi x)$ and $v(x) = -\\pi\\sin(\\pi x)$.\n", + "\n", + "\n", + "\n", + "## The trial solution\n", + "Setting up the network is done in similar matter as for the example of solving the diffusion equation.\n", + "The only things we have to change, is the trial solution such that it satisfies the conditions from ([20](#condwave)) and the cost function.\n", + "\n", + "The trial solution becomes slightly different since we have other conditions than in the example of solving the diffusion equation. Here, a possible trial solution $g_t(x,t)$ is\n", + "\n", + "$$\n", + "g_t(x,t) = h_1(x,t) + x(1-x)t^2N(x,t,P)\n", + "$$\n", + "\n", + "where\n", + "\n", + "$$\n", + "h_1(x,t) = (1-t^2)u(x) + tv(x)\n", + "$$\n", + "\n", + "Note that this trial solution satisfies the conditions only if $u(0) = v(0) = u(1) = v(1) = 0$, which is the case in this example.\n", + "\n", + "\n", + "## The analytical solution\n", + "\n", + "The analytical solution for our specific problem, is\n", + "\n", + "$$\n", + "g(x,t) = \\sin(\\pi x)\\cos(\\pi t) - \\sin(\\pi x)\\sin(\\pi t)\n", + "$$\n", + "\n", + "\n", + "## Solving the wave equation - the full program using Autograd" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "import autograd.numpy as np\n", + "from autograd import hessian,grad\n", + "import autograd.numpy.random as npr\n", + "from matplotlib import cm\n", + "from matplotlib import pyplot as plt\n", + "from mpl_toolkits.mplot3d import axes3d\n", + "\n", + "## Set up the trial function:\n", + "def u(x):\n", + " return np.sin(np.pi*x)\n", + "\n", + "def v(x):\n", + " return -np.pi*np.sin(np.pi*x)\n", + "\n", + "def h1(point):\n", + " x,t = point\n", + " return (1 - t**2)*u(x) + t*v(x)\n", + "\n", + "def g_trial(point,P):\n", + " x,t = point\n", + " return h1(point) + x*(1-x)*t**2*deep_neural_network(P,point)\n", + "\n", + "## Define the cost function\n", + "def cost_function(P, x, t):\n", + " cost_sum = 0\n", + "\n", + " g_t_hessian_func = hessian(g_trial)\n", + "\n", + " for x_ in x:\n", + " for t_ in t:\n", + " point = np.array([x_,t_])\n", + "\n", + " g_t_hessian = g_t_hessian_func(point,P)\n", + "\n", + " g_t_d2x = g_t_hessian[0][0]\n", + " g_t_d2t = g_t_hessian[1][1]\n", + "\n", + " err_sqr = ( (g_t_d2t - g_t_d2x) )**2\n", + " cost_sum += err_sqr\n", + "\n", + " return cost_sum / (np.size(t) * np.size(x))\n", + "\n", + "## The neural network\n", + "def sigmoid(z):\n", + " return 1/(1 + np.exp(-z))\n", + "\n", + "def deep_neural_network(deep_params, x):\n", + " # x is now a point and a 1D numpy array; make it a column vector\n", + " num_coordinates = np.size(x,0)\n", + " x = x.reshape(num_coordinates,-1)\n", + "\n", + " num_points = np.size(x,1)\n", + "\n", + " # N_hidden is the number of hidden layers\n", + " N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n", + "\n", + " # Assume that the input layer does nothing to the input x\n", + " x_input = x\n", + " x_prev = x_input\n", + "\n", + " ## Hidden layers:\n", + "\n", + " for l in range(N_hidden):\n", + " # From the list of parameters P; find the correct weigths and bias for this layer\n", + " w_hidden = deep_params[l]\n", + "\n", + " # Add a row of ones to include bias\n", + " x_prev = np.concatenate((np.ones((1,num_points)), x_prev ), axis = 0)\n", + "\n", + " z_hidden = np.matmul(w_hidden, x_prev)\n", + " x_hidden = sigmoid(z_hidden)\n", + "\n", + " # Update x_prev such that next layer can use the output from this layer\n", + " x_prev = x_hidden\n", + "\n", + " ## Output layer:\n", + "\n", + " # Get the weights and bias for this layer\n", + " w_output = deep_params[-1]\n", + "\n", + " # Include bias:\n", + " x_prev = np.concatenate((np.ones((1,num_points)), x_prev), axis = 0)\n", + "\n", + " z_output = np.matmul(w_output, x_prev)\n", + " x_output = z_output\n", + "\n", + " return x_output[0][0]\n", + "\n", + "## The analytical solution\n", + "def g_analytic(point):\n", + " x,t = point\n", + " return np.sin(np.pi*x)*np.cos(np.pi*t) - np.sin(np.pi*x)*np.sin(np.pi*t)\n", + "\n", + "def solve_pde_deep_neural_network(x,t, num_neurons, num_iter, lmb):\n", + " ## Set up initial weigths and biases\n", + " N_hidden = np.size(num_neurons)\n", + "\n", + " ## Set up initial weigths and biases\n", + "\n", + " # Initialize the list of parameters:\n", + " P = [None]*(N_hidden + 1) # + 1 to include the output layer\n", + "\n", + " P[0] = npr.randn(num_neurons[0], 2 + 1 ) # 2 since we have two points, +1 to include bias\n", + " for l in range(1,N_hidden):\n", + " P[l] = npr.randn(num_neurons[l], num_neurons[l-1] + 1) # +1 to include bias\n", + "\n", + " # For the output layer\n", + " P[-1] = npr.randn(1, num_neurons[-1] + 1 ) # +1 since bias is included\n", + "\n", + " print('Initial cost: ',cost_function(P, x, t))\n", + "\n", + " cost_function_grad = grad(cost_function,0)\n", + "\n", + " # Let the update be done num_iter times\n", + " for i in range(num_iter):\n", + " cost_grad = cost_function_grad(P, x , t)\n", + "\n", + " for l in range(N_hidden+1):\n", + " P[l] = P[l] - lmb * cost_grad[l]\n", + "\n", + "\n", + " print('Final cost: ',cost_function(P, x, t))\n", + "\n", + " return P\n", + "\n", + "if __name__ == '__main__':\n", + " ### Use the neural network:\n", + " npr.seed(15)\n", + "\n", + " ## Decide the vales of arguments to the function to solve\n", + " Nx = 10; Nt = 10\n", + " x = np.linspace(0, 1, Nx)\n", + " t = np.linspace(0,1,Nt)\n", + "\n", + " ## Set up the parameters for the network\n", + " num_hidden_neurons = [50,20]\n", + " num_iter = 1000\n", + " lmb = 0.01\n", + "\n", + " P = solve_pde_deep_neural_network(x,t, num_hidden_neurons, num_iter, lmb)\n", + "\n", + " ## Store the results\n", + " res = np.zeros((Nx, Nt))\n", + " res_analytical = np.zeros((Nx, Nt))\n", + " for i,x_ in enumerate(x):\n", + " for j, t_ in enumerate(t):\n", + " point = np.array([x_, t_])\n", + " res[i,j] = g_trial(point,P)\n", + "\n", + " res_analytical[i,j] = g_analytic(point)\n", + "\n", + " diff = np.abs(res - res_analytical)\n", + " print(\"Max difference between analytical and solution from nn: %g\"%np.max(diff))\n", + "\n", + " ## Plot the solutions in two dimensions, that being in position and time\n", + "\n", + " T,X = np.meshgrid(t,x)\n", + "\n", + " fig = plt.figure(figsize=(10,10))\n", + " ax = fig.gca(projection='3d')\n", + " ax.set_title('Solution from the deep neural network w/ %d layer'%len(num_hidden_neurons))\n", + " s = ax.plot_surface(T,X,res,linewidth=0,antialiased=False,cmap=cm.viridis)\n", + " ax.set_xlabel('Time $t$')\n", + " ax.set_ylabel('Position $x$');\n", + "\n", + "\n", + " fig = plt.figure(figsize=(10,10))\n", + " ax = fig.gca(projection='3d')\n", + " ax.set_title('Analytical solution')\n", + " s = ax.plot_surface(T,X,res_analytical,linewidth=0,antialiased=False,cmap=cm.viridis)\n", + " ax.set_xlabel('Time $t$')\n", + " ax.set_ylabel('Position $x$');\n", + "\n", + "\n", + " fig = plt.figure(figsize=(10,10))\n", + " ax = fig.gca(projection='3d')\n", + " ax.set_title('Difference')\n", + " s = ax.plot_surface(T,X,diff,linewidth=0,antialiased=False,cmap=cm.viridis)\n", + " ax.set_xlabel('Time $t$')\n", + " ax.set_ylabel('Position $x$');\n", + "\n", + " ## Take some slices of the 3D plots just to see the solutions at particular times\n", + " indx1 = 0\n", + " indx2 = int(Nt/2)\n", + " indx3 = Nt-1\n", + "\n", + " t1 = t[indx1]\n", + " t2 = t[indx2]\n", + " t3 = t[indx3]\n", + "\n", + " # Slice the results from the DNN\n", + " res1 = res[:,indx1]\n", + " res2 = res[:,indx2]\n", + " res3 = res[:,indx3]\n", + "\n", + " # Slice the analytical results\n", + " res_analytical1 = res_analytical[:,indx1]\n", + " res_analytical2 = res_analytical[:,indx2]\n", + " res_analytical3 = res_analytical[:,indx3]\n", + "\n", + " # Plot the slices\n", + " plt.figure(figsize=(10,10))\n", + " plt.title(\"Computed solutions at time = %g\"%t1)\n", + " plt.plot(x, res1)\n", + " plt.plot(x,res_analytical1)\n", + " plt.legend(['dnn','analytical'])\n", + "\n", + " plt.figure(figsize=(10,10))\n", + " plt.title(\"Computed solutions at time = %g\"%t2)\n", + " plt.plot(x, res2)\n", + " plt.plot(x,res_analytical2)\n", + " plt.legend(['dnn','analytical'])\n", + "\n", + " plt.figure(figsize=(10,10))\n", + " plt.title(\"Computed solutions at time = %g\"%t3)\n", + " plt.plot(x, res3)\n", + " plt.plot(x,res_analytical3)\n", + " plt.legend(['dnn','analytical'])\n", + "\n", + " plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Resources on differential equations and deep learning\n", + "\n", + "1. [Artificial neural networks for solving ordinary and partial differential equations by I.E. Lagaris et al](https://pdfs.semanticscholar.org/d061/df393e0e8fbfd0ea24976458b7d42419040d.pdf)\n", + "\n", + "2. [Neural networks for solving differential equations by A. Honchar](https://becominghuman.ai/neural-networks-for-solving-differential-equations-fa230ac5e04c)\n", + "\n", + "3. [Solving differential equations using neural networks by M.M Chiaramonte and M. Kiener](http://cs229.stanford.edu/proj2013/ChiaramonteKiener-SolvingDifferentialEquationsUsingNeuralNetworks.pdf)\n", + "\n", + "4. [Introduction to Partial Differential Equations by A. Tveito, R. Winther](https://www.springer.com/us/book/9783540225515)" + ] + } + ], + "metadata": {}, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/doc/src/LectureNotes/_build/html/_sources/chapter11.ipynb b/doc/src/LectureNotes/_build/html/_sources/chapter11.ipynb new file mode 100644 index 000000000..200cdd664 --- /dev/null +++ b/doc/src/LectureNotes/_build/html/_sources/chapter11.ipynb @@ -0,0 +1,1030 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "# Data Analysis and Machine Learning: \n", + "\n", + " \n", + "**Christian Forssén**, Department of Physics, Chalmers University of Technology, Sweden \n", + "\n", + " **Morten Hjorth-Jensen**, Department of Physics, University of Oslo and Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University\n", + "\n", + "Date: **Dec 23, 2020**\n", + "\n", + "Copyright 1999-2020, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license\n", + "\n", + "\n", + "\n", + "\n", + "# Elements of Bayesian theory and Bayesian Neural Networks\n", + "\n", + "\n", + "## Why Bayesian Statistics?\n", + "\n", + "We have already made ourselves familiar with elements of a statistical\n", + "data analysis via quantities like the bias-variance tradeoff as well\n", + "as some central distribution functions such as the Normal\n", + "distribution, the binomial distribution and other probability\n", + "distribution functions. \n", + "\n", + "In essentially all the Machine Learning algorithms we have studied,\n", + "our focus has been on a so-called **frequentist approach**, where\n", + "knowledge of an underlying likelihood function has not been\n", + "emphasized. Our data, whether we had a classification or a regression\n", + "problem, have been our central points of departure.\n", + "\n", + "Here we wish to merge this approach with the derivation of a\n", + "likelihood function which can be used to make prediction on how our\n", + "system under study evolves. We will venture into the realm of what is\n", + "called Bayesian Neural Networks. To get an overarching view on what\n", + "this entails, the following figure conveys the essential differences\n", + "between a standard Neural network that we have met earlier and a\n", + "Bayesian Neural Network. In order to get there, we need to present\n", + "some of the basic elements of Bayesian statistics, starting with the\n", + "product rule and Bayes' theorem.\n", + "\n", + "\n", + "\n", + "\n", + "## Inference\n", + "Inference:\n", + " : \n", + " \"the act of passing from one proposition, statement or judgment considered as true to another whose truth is believed to follow from that of the former\" (Webster) \n", + " Do premises $A, B, \\ldots \\to$ hypothesis, $H$? \n", + "\n", + "Deductive inference:\n", + " : \n", + " Premises allow definite determination of truth/falsity of H (syllogisms, symbolic logic, Boolean algebra) \n", + " $B(H|A,B,...) = 0$ or $1$\n", + "\n", + "Inductive inference:\n", + " : \n", + " Premises bear on truth/falsity of H, but don’t allow its definite determination (weak syllogisms, analogies)\n", + " $A, B, C, D$ share properties $x, y, z$; $E$ has properties $x, y$\n", + " $\\to$ $E$ probably has property $z$.\n", + "\n", + "\n", + "\n", + "\n", + "## Statistical Inference\n", + "* Quantify the strength of inductive inferences from facts, in the form of data ($D$), and other premises, e.g. models, to hypotheses about the phenomena producing the data.\n", + "\n", + "* Quantify via probabilities, or averages calculated using probabilities. Frequentists ($\\mathcal{F}$) and Bayesians ($\\mathcal{B}$) use probabilities very differently for this.\n", + "\n", + "* To the pioneers such as Bernoulli, Bayes and Laplace, a probability represented a *degree-of-belief* or plausability: how much they thought that something as true based on the evidence at hand. This is the Bayesian approach.\n", + "\n", + "* To the 19th century scholars, this seemed too vague and subjective. They redefined probability as the *long run relative frequency* with which an event occurred, given (infinitely) many repeated (experimental) trials.\n", + "\n", + "\n", + "\n", + "\n", + "## Some history\n", + "Adapted from D.S. Sivia[^Sivia]:\n", + "\n", + "[^Sivia]: Sivia, Devinderjit, and John Skilling. Data Analysis : A Bayesian Tutorial, OUP Oxford, 2006\n", + "\n", + "> Although the frequency definition appears to be more objective, its range of validity is also far more limited. For example, Laplace used (his) probability theory to estimate the mass of Saturn, given orbital data that were available to him from various astronomical observatories. In essence, he computed the posterior pdf for the mass M , given the data and all the relevant background information I (such as a knowledge of the laws of classical mechanics): prob(M|{data},I); this is shown schematically in the figure [Fig. 1.2].\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "> To Laplace, the (shaded) area under the posterior pdf curve between $m_1$ and $m_2$ was a measure of how much he believed that the mass of Saturn lay in the range $m_1 \\le M \\le m_2$. As such, the position of the maximum of the posterior pdf represents a best estimate of the mass; its width, or spread, about this optimal value gives an indication of the uncertainty in the estimate. Laplace stated that: ‘ . . . it is a bet of 11,000 to 1 that the error of this result is not 1/100th of its value.’ He would have won the bet, as another 150 years’ accumulation of data has changed the estimate by only 0.63%!\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "> According to the frequency definition, however, we are not permitted to use probability theory to tackle this problem. This is because the mass of Saturn is a constant and not a random variable; therefore, it has no frequency distribution and so probability theory cannot be used.\n", + "> \n", + "> If the pdf [of Fig. 1.2] had to be interpreted in terms of the frequency definition, we would have to imagine a large ensemble of universes in which everything remains constant apart from the mass of Saturn.\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "> As this scenario appears quite far-fetched, we might be inclined to think of [Fig. 1.2] in terms of the distribution of the measurements of the mass in many repetitions of the experiment. Although we are at liberty to think about a problem in any way that facilitates its solution, or our understanding of it, having to seek a frequency interpretation for every data analysis problem seems rather perverse.\n", + "> For example, what do we mean by the ‘measurement of the mass’ when the data consist of orbital periods? Besides, why should we have to think about many repetitions of an experiment that never happened? What we really want to do is to make the best inference of the mass given the (few) data that we actually have; this is precisely the Bayes and Laplace view of probability.\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "> Faced with the realization that the frequency definition of probability theory did not permit most real-life scientific problems to be addressed, a new subject was invented — statistics! To estimate the mass of Saturn, for example, one has to relate the mass to the data through some function called the statistic; since the data are subject to ‘random’ noise, the statistic becomes the random variable to which the rules of probability the- ory can be applied. But now the question arises: How should we choose the statistic? The frequentist approach does not yield a natural way of doing this and has, therefore, led to the development of several alternative schools of orthodox or conventional statis- tics. The masters, such as Fisher, Neyman and Pearson, provided a variety of different principles, which has merely resulted in a plethora of tests and procedures without any clear underlying rationale. This lack of unifying principles is, perhaps, at the heart of the shortcomings of the cook-book approach to statistics that students are often taught even today.\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "## The Bayesian recipe\n", + "Assess hypotheses by calculating their probabilities $p(H_i | \\ldots)$ conditional on known and/or presumed information using the rules of probability theory.\n", + "\n", + "\n", + "Probability Theory Axioms:\n", + "Product (AND) rule :\n", + " : \n", + " $p(A, B | I) = p(A|I) p(B|A, I) = p(B|I)p(A|B,I)$\n", + " Should read $p(A,B|I)$ as the probability for propositions $A$ AND $B$ being true given that $I$ is true.\n", + "\n", + "Sum (OR) rule:\n", + " : \n", + " $p(A + B | I) = p(A | I) + p(B | I) - p(A, B | I)$\n", + " $p(A+B|I)$ is the probability that proposition $A$ OR $B$ is true given that $I$ is true.\n", + "\n", + "Normalization:\n", + " : \n", + " $p(A|I) + p(\\bar{A}|I) = 1$\n", + " $\\bar{A}$ denotes the proposition that $A$ is false.\n", + "\n", + "\n", + "\n", + "\n", + "## Bayes' theorem\n", + "Bayes' theorem follows directly from the product rule" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "$$\n", + "p(A|B,I) = \\frac{p(B|A,I) p(A|I)}{p(B|I)}.\n", + "$$\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The importance of this property to data analysis becomes apparent if we replace $A$ and $B$ by hypothesis($H$) and data($D$):" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "
\n", + "\n", + "$$\n", + "\\begin{equation}\n", + "p(H|D,I) = \\frac{p(D|H,I) p(H|I)}{p(D|I)}.\n", + "\\label{eq:bayes} \\tag{1}\n", + "\\end{equation}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The power of Bayes’ theorem lies in the fact that it relates the quantity of interest, the probability that the hypothesis is true given the data, to the term we have a better chance of being able to assign, the probability that we would have observed the measured data if the hypothesis was true.\n", + "\n", + "\n", + "\n", + "\n", + "The various terms in Bayes’ theorem have formal names. \n", + "* The quantity on the far right, $p(H|I)$, is called the *prior* probability; it represents our state of knowledge (or ignorance) about the truth of the hypothesis before we have analysed the current data. \n", + "\n", + "* This is modified by the experimental measurements through $p(D|H,I)$, the *likelihood* function, \n", + "\n", + "* The denominator $p(D|I)$ is called the *evidence*. It does not depend on the hypothesis and can be regarded as a normalization constant.\n", + "\n", + "* Together, these yield the *posterior* probability, $p(H|D, I )$, representing our state of knowledge about the truth of the hypothesis in the light of the data. \n", + "\n", + "In a sense, Bayes’ theorem encapsulates the process of learning.\n", + "\n", + "\n", + "\n", + "\n", + "## The friends of Bayes' theorem\n", + "Normalization:\n", + " : \n", + " $\\sum_i p(H_i|\\ldots) = 1$.\n", + "\n", + "Marginalization:\n", + " : \n", + " $\\sum_i p(A,H_i|I) = \\sum_i p(H_i|A,I) p(A|I) = p(A|I)$.\n", + "\n", + "Marginalization (continuum limit):\n", + " : \n", + " $\\int dx p(A,H(x)|I) = p(A|I)$.\n", + "\n", + "In the above, $H_i$ is an exclusive and exhaustive list of hypotheses. For example,let’s imagine that there are five candidates in a presidential election; then $H_1$ could be the proposition that the first candidate will win, and so on. The probability that $A$ is true, for example that unemployment will be lower in a year’s time (given all relevant information $I$, but irrespective of whoever becomes president) is then given by $\\sum_i p(A,H_i|I)$.\n", + "\n", + "In the continuum limit of propositions we must understand $p(\\ldots)$ as a pdf (probability density function).\n", + "\n", + "Marginalization is a very powerful device in data analysis because it enables us to deal with nuisance parameters; that is, quantities which necessarily enter the analysis but are of no intrinsic interest. The unwanted background signal present in many experimental measurements are examples of nuisance parameters.\n", + "\n", + "\n", + "\n", + "\n", + "## Inference With Parametric Models\n", + "Inductive inference with parametric models is a very important tool in the natural sciences.\n", + "* Consider $N$ different models $M_i$ ($i = 1, \\ldots, N$), each with parameters $\\boldsymbol{\\alpha}_i$. Each of them implies a sampling distribution (conditional predictive distribution for possible data)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "$$\n", + "p(D|\\boldsymbol{\\alpha}_i, M_i)\n", + "$$\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "* The $\\boldsymbol{\\alpha}_i$ dependence when we fix attention on the actual, observed data ($D_\\mathrm{obs}$) is the likelihood function:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "$$\n", + "\\mathcal{L}_i (\\boldsymbol{\\alpha}_i) \\equiv p(D_\\mathrm{obs}|\\boldsymbol{\\alpha}_i, M_i)\n", + "$$\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "* We may be uncertain about $i$ (model uncertainty),\n", + "\n", + "* or uncertain about $\\boldsymbol{\\alpha}_i$ (parameter uncertainty).\n", + "\n", + "\n", + "\n", + "\n", + "Parameter Estimation:\n", + " : \n", + " Premise = choice of model (pick specific $i$)\n", + " $\\Rightarrow$ What can we say about $\\boldsymbol{\\alpha}_i$?\n", + "\n", + "Model comparison:\n", + " : \n", + " Premise = $\\{M_i\\}$\n", + " $\\Rightarrow$ What can we say about $i$?\n", + "\n", + "Model adequacy:\n", + " : \n", + " Premise = $M_1$\n", + " $\\Rightarrow$ Is $M_1$ adequate?\n", + "\n", + "Hybrid Uncertainty:\n", + " : \n", + " Models share some common params: $\\boldsymbol{\\alpha}_1 = \\{ \\boldsymbol{\\varphi}, \\boldsymbol{\\eta}_i\\}$\n", + " $\\Rightarrow$ What can we say about $\\boldsymbol{\\varphi}$? (Systematic error is an example)\n", + "\n", + "\n", + "\n", + "\n", + "## Illustrative examples with python code\n", + "* Is this a fair coin? (analytical)\n", + "\n", + "* Flux from a star (single parameter, MCMC)\n", + "\n", + "* The lighthouse problem (two parameters, MCMC)\n", + "\n", + "* Linear fit with outliers (nuisance parameters)\n", + "\n", + "* ...\n", + "\n", + "\n", + "\n", + "\n", + "## Example: Is this a fair coin?\n", + "Let us begin with the analysis of data from a simple coin-tossing experiment. \n", + "Given that we had observed 6 heads in 8 flips, would you think it was a fair coin? By fair, we mean that we would be prepared to lay an even 1 : 1 bet on the outcome of a flip being a head or a tail. If we decide that the coin was fair, the question which follows naturally is how sure are we that this was so; if it was not fair, how unfair do we think it was? Furthermore, if we were to continue collecting data for this particular coin, observing the outcomes of additional flips, how would we update our belief on the fairness of the coin?\n", + "\n", + "A sensible way of formulating this problem is to consider a large number of hypotheses about the range in which the bias-weighting of the coin might lie. If we denote the bias-weighting by $H$, then $H = 0$ and $H = 1$ can represent a coin which produces a tail or a head on every flip, respectively. There is a continuum of possibilities for the value of H between these limits, with $H = 0.5$ indicating a fair coin. Our state of knowledge about the fairness, or the degree of unfairness, of the coin is then completely summarized by specifying how much we believe these various propositions to be true. \n", + "\n", + "Let us perform a computer simulation of a coin-tossing experiment. This provides the data that we will be analysing." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "0\n", + " \n", + "<\n", + "<\n", + "<\n", + "!\n", + "!\n", + "C\n", + "O\n", + "D\n", + "E\n", + "_\n", + "B\n", + "L\n", + "O\n", + "C\n", + "K\n", + " \n", + " \n", + "p\n", + "y\n", + "c\n", + "o\n", + "d" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "np.random.seed(999) # for reproducibility\n", + "a=0.6 # biased coin\n", + "flips=np.random.rand(2**12) # simulates 4096 coin flips\n", + "heads=flips\n", + "
\n", + "\n", + "$$\n", + "\\begin{equation}\n", + "\\int_0^1 p(H|D,I) dH = 1.\n", + "\\label{eq:coin_posterior_norm} \\tag{2}\n", + "\\end{equation}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The prior pdf, $p(H|I)$, represents what we know about the coin given only the information $I$ that we are dealing with a ‘strange coin’. We could keep a very open mind about the nature of the coin; a simple probability assignment which reflects this is a uniform, or flat, prior" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "
\n", + "\n", + "$$\n", + "\\begin{equation}\n", + "p(H|I) = \\left\\{ \\begin{array}{ll}\n", + "1 & 0 \\le H \\le 1, \\\\\n", + "0 & \\mathrm{otherwise}.\n", + "\\end{array} \\right.\n", + "\\label{eq:coin_prior_uniform} \\tag{3}\n", + "\\end{equation}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We will get back later to the choice of prior and its effect on the analysis.\n", + "\n", + "This prior state of knowledge, or ignorance, is modified by the data through the likelihood function $p(D|H,I)$. It is a measure of the chance that we would have obtained the data that we actually observed, if the value of the bias-weighting was given (as known). If, in the conditioning information $I$, we assume that the flips of the coin were independent events, so that the outcome of one did not influence that of another, then the probability of obtaining the data `R heads in N tosses' is given by the binomial distribution (we leave a formal definition of this to a statistics textbook)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "
\n", + "\n", + "$$\n", + "\\begin{equation}\n", + "p(D|H,I) \\propto H^R (1-H)^{N-R}.\n", + "\\label{_auto1} \\tag{4}\n", + "\\end{equation}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "It seems reasonable because $H$ is the chance of obtaining a head on any flip, and there were $R$ of them, and $1-H$ is the corresponding probability for a tail, of which there were $N-R$. We note that this binomial distribution also contains a normalization factor, but we will ignore it since it does not depend explicitly on $H$, the quantity of interest. It will be absorbed by the normalization condition ([2](#eq:coin_posterior_norm)).\n", + "\n", + "We perform the setup of this Bayesian framework on the computer." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "def prior(H):\n", + " p=np.zeros_like(H)\n", + " p[(0<=x)&(x<=1)]=1 # allowed range: 0<=H<=1\n", + " return p # uniform prior\n", + "def likelihood(H,data):\n", + " N = len(data)\n", + " no_of_heads = sum(data)\n", + " no_of_tails = N - no_of_heads\n", + " return H**no_of_heads * (1-H)**no_of_tails\n", + "def posterior(H,data):\n", + " p=prior(H)*likelihood(H,data)\n", + " norm=np.trapz(p,H)\n", + " return p/norm" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The next step is to confront this setup with the simulated data. To get a feel for the result, it is instructive to see how the posterior pdf evolves as we obtain more and more data pertaining to the coin. The results of such an analyses is shown in Fig. [fig:coinflipping](#fig:coinflipping)." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "x=np.linspace(0,1,100)\n", + "fig, axs = plt.subplots(nrows=4,ncols=3,sharex=True,sharey='row')\n", + "axs_vec=np.reshape(axs,-1)\n", + "axs_vec[0].plot(x,prior(x))\n", + "for ndouble in range(11):\n", + " ax=axs_vec[1+ndouble]\n", + " ax.plot(x,posterior(x,heads[:2**ndouble]))\n", + " ax.text(0.1, 0.8, '$N={0}$'.format(2**ndouble), transform=ax.transAxes)\n", + "for row in range(4): axs[row,0].set_ylabel('$p(H|D_\\mathrm{obs},I)$')\n", + "for col in range(3): axs[-1,col].set_xlabel('$H$')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "\n", + "
\n", + "\n", + "

The evolution of the posterior pdf for the bias-weighting of a coin, as the number of data available increases. The figure on the top left-hand corner of each panel shows the number of data included in the analysis.

\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "The panel in the top left-hand corner shows the posterior pdf for $H$ given no data, i.e., it is the same as the prior pdf of Eq. ([3](#eq:coin_prior_uniform)). It indicates that we have no more reason to believe that the coin is fair than we have to think that it is double-headed, double-tailed, or of any other intermediate bias-weighting.\n", + "\n", + "The first flip is obviously tails. At this point we have no evidence that the coin has a side with heads, as indicated by the pdf going to zero as $H \\to 1$. The second flip is obviously heads and we have now excluded both extreme options $H=0$ (double-tailed) and $H=1$ (double-headed). We can note that the posterior at this point has the simple form $p(H|D,I) = H(1-H)$ for $0 \\le H \\le 1$.\n", + "\n", + "The remainder of Fig. [fig:coinflipping](#fig:coinflipping) shows how the posterior pdf evolves as the number of data analysed becomes larger and larger. We see that the position of the maximum moves around, but that the amount by which it does so decreases with the increasing number of observations. The width of the posterior pdf also becomes narrower with more data, indicating that we are becoming increasingly confident in our estimate of the bias-weighting. For the coin in this example, the best estimate of $H$ eventually converges to 0.6, which, of course, was the value chosen to simulate the flips.\n", + "\n", + "\n", + "## A few words on different priors\n", + "* uniform\n", + "\n", + "* Gaussian\n", + "\n", + "* Jeffrey's prior\n", + "\n", + "Repeat the coin flipping experiment with other priors.\n", + "\n", + "\n", + "## Bayesian parameter estimation (single parameter)\n", + "We will now consider the very important task of model parameter estimation using statistical inference. \n", + "[CF 1: maybe stress that model parameters are not random variables, and the meaning of parameter estimation is therefore very different between frequentist and bayesian approaches.]\n", + "\n", + "Throughout this section we will consider a specific example that involves a model with a single parameter: \"Measured flux from a star\".\n", + "\n", + "\n", + "\n", + "\n", + "### Example: Measured flux from a star\n", + "\n", + "Adapted from the blog [Pythonic Perambulations](http://jakevdp.github.io) by Jake VanderPlas.\n", + "\n", + "Imagine that we point our telescope to the sky, and observe the light coming from a single star. For the time being, we'll assume that the star's true flux is constant with time, i.e. that is it has a fixed value $F_\\mathrm{true}$ (we'll also ignore effects like sky noise and other sources of systematic error). We'll assume that we perform a series of $N$ measurements with our telescope, where the ith measurement reports the observed photon flux $F_i$ and error $e_i$[^errors].\n", + "The question is, given this set of measurements $D = \\{F_i, e_i\\}$, what is our best estimate of the true flux $F_\\mathrm{true}$?\n", + "\n", + "[^errors]: We'll make the reasonable assumption that errors are Gaussian. In a Frequentist perspective, $e_i$ is the standard deviation of the results of a single measurement event in the limit of repetitions of *that event*. In the Bayesian perspective, $e_i$ is the standard deviation of the (Gaussian) probability distribution describing our knowledge of that particular measurement given its observed value.\n", + "\n", + "Because the measurements are number counts, a Poisson distribution is a good approximation to the measurement process:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "np.random.seed(1) # for repeatability\n", + "F_true = 1000 # true flux, say number of photons measured in 1 second\n", + "N = 50 # number of measurements\n", + "F = stats.poisson(F_true).rvs(N)\n", + " # N measurements of the flux\n", + "e = np.sqrt(F) # errors on Poisson counts estimated via square root" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now let's make a simple visualization of the \"observed\" data, see Fig. [fig:flux](#fig:flux)." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "fig, ax = plt.subplots()\n", + "ax.errorbar(F, np.arange(N), xerr=e, fmt='ok', ecolor='gray', alpha=0.5)\n", + "ax.vlines([F_true], 0, N, linewidth=5, alpha=0.2)\n", + "ax.set_xlabel(\"Flux\");ax.set_ylabel(\"measurement number\");" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "\n", + "
\n", + "\n", + "

Single photon counts (flux measurements).

\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "These measurements each have a different error $e_i$ which is estimated from Poisson statistics using the standard square-root rule. In this toy example we already know the true flux $F_\\mathrm{true}$, but the question is this: given our measurements and errors, what is our best estimate of the true flux?\n", + "\n", + "Let's take a look at the frequentist and Bayesian approaches to solving this.\n", + "\n", + "### Simple Photon Counts: Frequentist Approach\n", + "\n", + "We'll start with the classical frequentist maximum likelihood approach. Given a single observation $D_i = (F_i, e_i)$, we can compute the probability distribution of the measurement given the true flux Ftrue given our assumption of Gaussian errors" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "
\n", + "\n", + "$$\n", + "\\begin{equation}\n", + "p(D_i | F_\\mathrm{true}, I) = \\frac{1}{\\sqrt{2\\pi e_i^2}} \\exp \\left( \\frac{-(F_i-F_\\mathrm{true})^2}{2e_i^2} \\right).\n", + "\\label{_auto2} \\tag{5}\n", + "\\end{equation}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This should be read \"the probability of $D_i$ given $F_\\mathrm{true}$\n", + "equals ...\". You should recognize this as a normal distribution with mean $F_\\mathrm{true}$ and standard deviation $e_i$.\n", + "\n", + "We construct the *likelihood function* by computing the product of the probabilities for each data point" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "
\n", + "\n", + "$$\n", + "\\begin{equation}\n", + "\\mathcal{L}(D | F_\\mathrm{true}, I) = \\prod_{i=1}^N p(D_i | F_\\mathrm{true}, I),\n", + "\\label{_auto3} \\tag{6}\n", + "\\end{equation}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "here $D = \\{D_i\\}$ represents the entire set of measurements. Because the value of the likelihood can become very small, it is often more convenient to instead compute the log-likelihood. Combining the previous two equations and computing the log, we have" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "
\n", + "\n", + "$$\n", + "\\begin{equation}\n", + "\\log\\mathcal{L} = -\\frac{1}{2} \\sum_{i=1}^N \\left[ \\log(2\\pi e_i^2) + \\frac{(F_i-F_\\mathrm{true})^2}{e_i^2} \\right].\n", + "\\label{_auto4} \\tag{7}\n", + "\\end{equation}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "What we'd like to do is determine $F_\\mathrm{true}$ such that the likelihood is maximized. For this simple problem, the maximization can be computed analytically (i.e. by setting $d\\log\\mathcal{L}/d F_\\mathrm{true} = 0$). This results in the following observed estimate of $F_\\mathrm{true}$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "
\n", + "\n", + "$$\n", + "\\begin{equation}\n", + "F_\\mathrm{est} = \\sum_{i=1}^N w_i F_i; \\quad w_i = 1/e_i^2.\n", + "\\label{_auto5} \\tag{8}\n", + "\\end{equation}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Notice that in the special case of all errors $e_i$ being equal, this reduces to" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "
\n", + "\n", + "$$\n", + "\\begin{equation}\n", + "F_\\mathrm{est} = \\frac{1}{N} \\sum_{i=1} F_i.\n", + "\\label{_auto6} \\tag{9}\n", + "\\end{equation}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "That is, in agreement with intuition, $F_\\mathrm{est}$ is simply the mean of the observed data when errors are equal.\n", + "\n", + "We can go further and ask what the error of our estimate is. In the frequentist approach, this can be accomplished by fitting a Gaussian approximation to the likelihood curve at maximum; in this simple case this can also be solved analytically (the sum of Gaussians is also a Gaussian). It can be shown that the standard deviation of this Gaussian approximation is" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "
\n", + "\n", + "$$\n", + "\\begin{equation}\n", + "\\sigma_\\mathrm{est} = \\sum_{i=1}^N w_i.\n", + "\\label{_auto7} \\tag{10}\n", + "\\end{equation}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "These results are fairly simple calculations; let's evaluate them for our toy dataset:" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "w=1./e**2\n", + "print(\"\"\"\n", + "F_true = {0}\n", + "F_est = {1:.0f} +/- {2:.0f} (based on {3} measurements) \"\"\"\\\n", + " .format(F_true, (w * F).sum() / w.sum(), w.sum() ** -0.5, N))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`F_true = 1000` \n", + "`F_est = 998 +/- 4 (based on 50 measurements)` \n", + "\n", + "We find that for 50 measurements of the flux, our estimate has an error of about 0.4% and is consistent with the input value.\n", + "\n", + "\n", + "### Simple Photon Counts: Bayesian Approach\n", + "\n", + "The Bayesian approach, as you might expect, begins and ends with probabilities. Our hypothesis is that the star has a constant flux $F_\\mathrm{true}$. It recognizes that what we fundamentally want to compute is our knowledge of the parameters in question given the data and other information (such as our knowledge of uncertainties for the observed values), i.e. in this case, $p(F_\\mathrm{true} | D,I)$.\n", + "Note that this formulation of the problem is fundamentally contrary to the frequentist philosophy, which says that probabilities have no meaning for model parameters like $F_\\mathrm{true}$. Nevertheless, within the Bayesian philosophy this is perfectly acceptable.\n", + "\n", + "To compute this result, Bayesians next apply Bayes' Theorem ([1](#eq:bayes)).\n", + "If we set the prior $p(F_\\mathrm{true}|I) \\propto 1$ (a flat prior), we find\n", + "$p(F_\\mathrm{true}|D,I) \\propto p(D | F_\\mathrm{true},I) \\equiv \\mathcal{L}(D | F_\\mathrm{true},I)$\n", + "and the Bayesian probability is maximized at precisely the same value as the frequentist result! So despite the philosophical differences, we see that (for this simple problem at least) the Bayesian and frequentist point estimates are equivalent.\n", + "\n", + "### A note about priors\n", + "\n", + "The prior allows inclusion of other information into the computation, which becomes very useful in cases where multiple measurement strategies are being combined to constrain a single model. The necessity to specify a prior, however, is one of the more controversial pieces of Bayesian analysis.\n", + "A frequentist will point out that the prior is problematic when no true prior information is available. Though it might seem straightforward to use a noninformative prior like the flat prior mentioned above, there are some [surprisingly subtleties](http://normaldeviate.wordpress.com/2013/07/13/lost-causes-in-statistics-ii-noninformative- priors/comment-page-1/) involved. It turns out that in many situations, a truly noninformative prior does not exist! Frequentists point out that the subjective choice of a prior which necessarily biases your result has no place in statistical data analysis.\n", + "A Bayesian would counter that frequentism doesn't solve this problem, but simply skirts the question. Frequentism can often be viewed as simply a special case of the Bayesian approach for some (implicit) choice of the prior: a Bayesian would say that it's better to make this implicit choice explicit, even if the choice might include some subjectivity.\n", + "\n", + "### Simple Photon Counts: Bayesian approach in practice\n", + "\n", + "Leaving these philosophical debates aside for the time being, let's address how Bayesian results are generally computed in practice. For a one parameter problem like the one considered here, it's as simple as computing the posterior probability $p(F_\\mathrm{true} | D,I)$ as a function of $F_\\mathrm{true}$: this is the distribution reflecting our knowledge of the parameter $F_\\mathrm{true}$.\n", + "But as the dimension of the model grows, this direct approach becomes increasingly intractable. For this reason, Bayesian calculations often depend on sampling methods such as Markov Chain Monte Carlo (MCMC). For this practical example, let us apply an MCMC approach using Dan Foreman-Mackey's [emcee](http://dan.iel.fm/emcee/current/) package. Keep in mind here that the goal is to generate a set of points drawn from the posterior probability distribution, and to use those points to determine the answer we seek.\n", + "To perform this MCMC, we start by defining Python functions for the prior $p(F_\\mathrm{true} | I)$, the likelihood $p(D | F_\\mathrm{true},I)$, and the posterior $p(F_\\mathrm{true} | D,I)$, noting that none of these need be properly normalized. Our model here is one-dimensional, but to handle multi-dimensional models we'll define the model in terms of an array of parameters $\\boldsymbol{\\alpha}$, which in this case is $\\boldsymbol{\\alpha} = [F_\\mathrm{true}]$" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "def log_prior(alpha):\n", + " return 0 # flat prior\n", + "\n", + "def log_likelihood(alpha, F, e):\n", + " return -0.5 * np.sum(np.log(2 * np.pi * e ** 2) \\\n", + " + (F - alpha[0]) ** 2 / e ** 2)\n", + " \n", + "def log_posterior(alpha, F, e):\n", + " return log_prior(alpha) + log_likelihood(alpha, F, e)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we set up the problem, including generating some random starting guesses for the multiple chains of points." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "ndim = 1 # number of parameters in the model\n", + "nwalkers = 50 # number of MCMC walkers\n", + "nburn = 1000 # \"burn-in\" period to let chains stabilize\n", + "nsteps = 2000 # number of MCMC steps to take\n", + "# we'll start at random locations between 0 and 2000\n", + "starting_guesses = 2000 * np.random.rand(nwalkers, ndim)\n", + "sampler = emcee.EnsembleSampler(nwalkers, ndim, log_posterior, args=[F,e])\n", + "sampler.run_mcmc(starting_guesses, nsteps)\n", + "# Shape of sampler.chain = (nwalkers, nsteps, ndim)\n", + "# Flatten the sampler chain and discard burn-in points:\n", + "samples = sampler.chain[:, nburn:, :].reshape((-1, ndim))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If this all worked correctly, the array sample should contain a series of 50,000 points drawn from the posterior. Let's plot them and check. See results in Fig. [fig:flux-bayesian](#fig:flux-bayesian)." + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "fig, ax = plt.subplots()\n", + "ax.hist(samples, bins=50, histtype=\"stepfilled\", alpha=0.3, normed=True)\n", + "ax.set_xlabel(r'$F_\\mathrm{est}$')\n", + "ax.set_ylabel(r'$p(F_\\mathrm{est}|D,I)$')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "\n", + "
\n", + "\n", + "

Bayesian posterior pdf (represented by a histogram of MCMC samples) from flux measurements.

\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "### Best estimates and confidence intervals\n", + "\n", + "The posterior distribution from our Bayesian data analysis is the key quantity that encodes our inference about the values of the model parameters, given the data and the relevant background information. Often, however, we wish to summarize this result with just a few numbers: the best estimate and a measure of its reliability. \n", + "\n", + "There are a few different options for this. The choice of the most appropriate one depends mainly on the shape of the posterior distribution:\n", + "\n", + "*Symmetric posterior pdfs*: Since the probability (density) associated with any particular value of the parameter is a measure of how much we believe that it lies in the neighbourhood of that point, our best estimate is given by the maximum of the posterior pdf. If we denote the quantity of interest by $X$, with a posterior pdf $P =p(X|D,I)$, then the best estimate of its value $X_0$ is given by the condition $dP/dX|_{X=X_0}=0$. Strictly speaking, we should also check the sign of the second derivative to ensure that $X_0$ represents a maximum.\n", + "\n", + "To obtain a measure of the reliability of this best estimate, we need to look at the width or spread of the posterior pdf about $X_0$. When considering the behaviour of any function in the neighbourhood of a particular point, it is often helpful to carry out a Taylor series expansion; this is simply a standard tool for (locally) approximating a complicated function by a low-order polynomial. The linear term is zero at the maximum and the quadratic term is often the dominating one determining the width of the posterior pdf. Ignoring all the higher-order terms we arrive at the Gaussian approximation" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "
\n", + "\n", + "$$\n", + "\\begin{equation}\n", + "p(X|D,I) \\approx \\frac{1}{\\sigma\\sqrt{2\\pi}} \\exp \\left[ -\\frac{(x-\\mu)^2}{2\\sigma^2} \\right],\n", + "\\label{_auto8} \\tag{11}\n", + "\\end{equation}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "where the mean $\\mu = X_0$ and the variance $\\sigma = \\left( - \\left. \\frac{d^2L}{dX^2} \\right|_{X_0} \\right)^{-1/2}$, where $L$ is the logarithm of the posterior $P$. Our inference about the quantity of interest is conveyed very concisely, therefore, by the statement $X = X_0 \\pm \\sigma$, and" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "$$\n", + "p(X_0-\\sigma < X < X_0+\\sigma | D,I) = \\int_{X_0-\\sigma}^{X_0+\\sigma} p(X|D,I) dX \\approx 0.67.\n", + "$$\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "*Asymmetric posterior pdfs*: While the maximum of the posterior ($X_0$) can still be regarded as giving the best estimate, the true value is now more likely to be on one side of this rather than the other. Alternatively one can compute the mean value, $\\langle X \\rangle = \\int X p(X|D,I) dX$, although this tends to overemphasise very long tails. The best option is probably a compromise that can be employed when having access to a large sample from the posterior (as provided by an MCMC), namely to give the median of this ensamble.\n", + "\n", + "Furthermore, the concept of an error-bar does not seem appropriate in this case, as it implicitly entails the idea of symmetry. A good way of expressing the reliability with which a parameter can be inferred, for an asymmetric posterior pdf, is rather through a *confidence interval*. Since the area under the posterior pdf between $X_1$ and $X_2$ is proportional to how much we believe that $X$ lies in that range, the shortest interval that encloses 67% of the area represents a sensible measure of the uncertainty of the estimate. Obviously we can choose to provide some other degree-of-belief that we think is relevant for the case at hand. Assuming that the posterior pdf has been normalized, to have unit area, we need to find $X_1$ and $X_2$ such that:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "$$\n", + "p(X_1 < X < X_2 | D,I) = \\int_{X_1}^{X_2} p(X|D,I) dX \\approx 0.67, \n", + "$$\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "where the difference $X_2 - X_1$ is as small as possible. The region $X_1 < X < X_2$ is then called the shortest 67% confidence interval. \n", + "\n", + "*Multimodal posterior pdfs*: We can sometimes obtain posteriors which are multimodal; i.e. contains several disconnected regions with large probabilities. There is no difficulty when one of the maxima is very much larger than the others: we can simply ignore the subsidiary solutions, to a good approximation, and concentrate on the global maximum. The problem arises when there are several maxima of comparable magnitude. What do we now mean by a best estimate, and how should we quantify its reliability? The idea of a best estimate and an error-bar, or even a confidence interval, is merely an attempt to summarize the posterior with just two or three numbers; sometimes this just can’t be done, and so these concepts are not valid. For the bimodal case we might be able to characterize the posterior in terms of a few numbers: two best estimates and their associated error-bars, or disjoint confidence intervals. For a general multimodal pdf, the most honest thing we can do is just display the posterior itself.\n", + "\n", + "### Simple Photon Counts: Best estimates and confidence intervals\n", + "\n", + "To compute these numbers for our example, you would run:" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "sampper=np.percentile(samples, [2.5, 16.5, 50, 83.5, 97.5],axis=0).flatten()\n", + "print(\"\"\"\n", + "F_true = {0}\n", + "Based on {1} measurements the posterior point estimates are:\n", + "...F_est = {2:.0f} +/- {3:.0f}\n", + "or using credible intervals:\n", + "...F_est = {4:.0f} (posterior median) \n", + "...F_est in [{5:.0f}, {6:.0f}] (67% credible interval) \n", + "...F_est in [{7:.0f}, {8:.0f}] (95% credible interval) \"\"\"\\\n", + " .format(F_true, N, np.mean(samples), np.std(samples), \\\n", + " sampper[2], sampper[1], sampper[3], sampper[0], sampper[4]))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`F_true = 1000` \n", + "`Based on 50 measurements the posterior point estimates are:` \n", + "`...F_est = 998 +/- 4` \n", + "`or using credible intervals:` \n", + "`...F_est = 998 (posterior median)` \n", + "`...F_est in [993, 1002] (67% credible interval)` \n", + "`...F_est in [989, 1006] (95% credible interval)` \n", + "\n", + "In this particular example, the posterior pdf is actually a Gaussian (since it is constructed as a product of Gaussians), and the mean and variance from the quadratic approximation will agree exactly with the frequentist approach.\n", + "\n", + "From this final result you might come away with the impression that the Bayesian method is unnecessarily complicated, and in this case it certainly is. Using an MCMC sampler to characterize a one-dimensional normal distribution is a bit like using the Death Star to destroy a beach ball, but we did this here because it demonstrates an approach that can scale to complicated posteriors in many, many dimensions, and can provide nice results in more complicated situations where an analytic likelihood approach is not possible.\n", + "\n", + "Furthermore, as data and models grow in complexity, the two approaches can diverge greatly. \n", + "\n", + "\n", + "## Bayesian parameter estimation (multiple parameters, covariance)\n", + "* multidimensional posterior pdf:s\n", + "\n", + "* nuisance parameters (e.g. background subtraction?)\n", + "\n", + "* corner plots, covariance, correlations\n", + "\n", + "* best example?\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "## Bayesian model selection\n", + "* Bayesian evidence\n", + "\n", + "* Occam's razor\n", + "\n", + "* Best example? How many spectral lines are there?" + ] + } + ], + "metadata": {}, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/doc/src/LectureNotes/_build/html/chapter1.html b/doc/src/LectureNotes/_build/html/chapter1.html index 4455a060b..97c2f7dfd 100644 --- a/doc/src/LectureNotes/_build/html/chapter1.html +++ b/doc/src/LectureNotes/_build/html/chapter1.html @@ -124,6 +124,26 @@ 8. Convolutional Neural Networks +
  • + + 9. Recurrent Neural Networks + +
    • +
  • + + 10. Solving ODEs with Deep Learning + +
    • +
  • + + 11. Data Analysis and Machine Learning: + +
    • +
  • + + 12. Elements of Bayesian theory and Bayesian Neural Networks + +
    • diff --git a/doc/src/LectureNotes/_build/html/chapter10.html b/doc/src/LectureNotes/_build/html/chapter10.html new file mode 100644 index 000000000..c3a6b7869 --- /dev/null +++ b/doc/src/LectureNotes/_build/html/chapter10.html @@ -0,0 +1,9723 @@ + + + + + + + + + 9. Recurrent Neural Networks — Applied Machine Learning and Data Analysis + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    +
    + + + + + + + + +
    + +
    +
    +
    +
    + + + +
    + + + +
    + + + + + + +
    + +
    + + + + + + + + +
    + + +
    +
    + Contents +
    + +
    +
    +
    +
    + +
    + +
    +

    9. Recurrent Neural Networks

    +

    Overview video. +See also lecture on Thursday October 22 and examples from week 42.

    +

    IN5400 at UiO Lecture

    +

    CS231 at Stanford Lecture

    +
    +

    9.1. Recurrent neural networks: Overarching view

    +

    Till now our focus has been, including convolutional neural networks +as well, on feedforward neural networks. The output or the activations +flow only in one direction, from the input layer to the output layer.

    +

    A recurrent neural network (RNN) looks very much like a feedforward +neural network, except that it also has connections pointing +backward.

    +

    RNNs are used to analyze time series data such as stock prices, and +tell you when to buy or sell. In autonomous driving systems, they can +anticipate car trajectories and help avoid accidents. More generally, +they can work on sequences of arbitrary lengths, rather than on +fixed-sized inputs like all the nets we have discussed so far. For +example, they can take sentences, documents, or audio samples as +input, making them extremely useful for natural language processing +systems such as automatic translation and speech-to-text.

    +
    +
    +

    9.2. Set up of an RNN

    +

    Text to come.

    +
    +
    +

    9.3. A simple example

    +
    +
    +
    %matplotlib inline
+
+# Start importing packages
+import pandas as pd
+import numpy as np
+import matplotlib.pyplot as plt
+import tensorflow as tf
+from tensorflow.keras import datasets, layers, models
+from tensorflow.keras.layers import Input
+from tensorflow.keras.models import Model, Sequential 
+from tensorflow.keras.layers import Dense, SimpleRNN, LSTM, GRU
+from tensorflow.keras import optimizers     
+from tensorflow.keras import regularizers           
+from tensorflow.keras.utils import to_categorical 
+
+
+
+# convert into dataset matrix
+def convertToMatrix(data, step):
+ X, Y =[], []
+ for i in range(len(data)-step):
+  d=i+step  
+  X.append(data[i:d,])
+  Y.append(data[d,])
+ return np.array(X), np.array(Y)
+
+step = 4
+N = 1000    
+Tp = 800    
+
+t=np.arange(0,N)
+x=np.sin(0.02*t)+2*np.random.rand(N)
+df = pd.DataFrame(x)
+df.head()
+
+plt.plot(df)
+plt.show()
+
+values=df.values
+train,test = values[0:Tp,:], values[Tp:N,:]
+
+# add step elements into train and test
+test = np.append(test,np.repeat(test[-1,],step))
+train = np.append(train,np.repeat(train[-1,],step))
+ 
+trainX,trainY =convertToMatrix(train,step)
+testX,testY =convertToMatrix(test,step)
+trainX = np.reshape(trainX, (trainX.shape[0], 1, trainX.shape[1]))
+testX = np.reshape(testX, (testX.shape[0], 1, testX.shape[1]))
+
+model = Sequential()
+model.add(SimpleRNN(units=32, input_shape=(1,step), activation="relu"))
+model.add(Dense(8, activation="relu")) 
+model.add(Dense(1))
+model.compile(loss='mean_squared_error', optimizer='rmsprop')
+model.summary()
+
+model.fit(trainX,trainY, epochs=100, batch_size=16, verbose=2)
+trainPredict = model.predict(trainX)
+testPredict= model.predict(testX)
+predicted=np.concatenate((trainPredict,testPredict),axis=0)
+
+trainScore = model.evaluate(trainX, trainY, verbose=0)
+print(trainScore)
+
+index = df.index.values
+plt.plot(index,df)
+plt.plot(index,predicted)
+plt.axvline(df.index[Tp], c="r")
+plt.show()
+
    +
    +
    +
    +_images/chapter10_1_0.png +
    Model: "sequential"
+_________________________________________________________________
+Layer (type)                 Output Shape              Param #   
+=================================================================
+simple_rnn (SimpleRNN)       (None, 32)                1184      
+_________________________________________________________________
+dense (Dense)                (None, 8)                 264       
+_________________________________________________________________
+dense_1 (Dense)              (None, 1)                 9         
+=================================================================
+Total params: 1,457
+Trainable params: 1,457
+Non-trainable params: 0
+_________________________________________________________________
+Epoch 1/100
+
    +
    +
    50/50 - 0s - loss: 2.1500
+
    +
    +
    Epoch 2/100
+50/50 - 0s - loss: 0.4827
+
    +
    +
    Epoch 3/100
+50/50 - 0s - loss: 0.4041
+
    +
    +
    Epoch 4/100
+50/50 - 0s - loss: 0.3986
+
    +
    +
    Epoch 5/100
+50/50 - 0s - loss: 0.3980
+
    +
    +
    Epoch 6/100
+
    +
    +
    50/50 - 0s - loss: 0.3996
+
    +
    +
    Epoch 7/100
+
    +
    +
    50/50 - 0s - loss: 0.3982
+
    +
    +
    Epoch 8/100
+
    +
    +
    50/50 - 0s - loss: 0.3962
+
    +
    +
    Epoch 9/100
+
    +
    +
    50/50 - 0s - loss: 0.3965
+
    +
    +
    Epoch 10/100
+
    +
    +
    50/50 - 0s - loss: 0.3934
+
    +
    +
    Epoch 11/100
+
    +
    +
    50/50 - 0s - loss: 0.3933
+
    +
    +
    Epoch 12/100
+
    +
    +
    50/50 - 0s - loss: 0.3951
+
    +
    +
    Epoch 13/100
+
    +
    +
    50/50 - 0s - loss: 0.3947
+
    +
    +
    Epoch 14/100
+
    +
    +
    50/50 - 0s - loss: 0.3924
+
    +
    +
    Epoch 15/100
+
    +
    +
    50/50 - 0s - loss: 0.3932
+
    +
    +
    Epoch 16/100
+
    +
    +
    50/50 - 0s - loss: 0.3932
+
    +
    +
    Epoch 17/100
+
    +
    +
    50/50 - 0s - loss: 0.3924
+
    +
    +
    Epoch 18/100
+
    +
    +
    50/50 - 0s - loss: 0.3914
+
    +
    +
    Epoch 19/100
+
    +
    +
    50/50 - 0s - loss: 0.3918
+
    +
    +
    Epoch 20/100
+
    +
    +
    50/50 - 0s - loss: 0.3914
+
    +
    +
    Epoch 21/100
+
    +
    +
    50/50 - 0s - loss: 0.3918
+
    +
    +
    Epoch 22/100
+
    +
    +
    50/50 - 0s - loss: 0.3889
+
    +
    +
    Epoch 23/100
+
    +
    +
    50/50 - 0s - loss: 0.3912
+
    +
    +
    Epoch 24/100
+
    +
    +
    50/50 - 0s - loss: 0.3892
+
    +
    +
    Epoch 25/100
+
    +
    +
    50/50 - 0s - loss: 0.3884
+
    +
    +
    Epoch 26/100
+
    +
    +
    50/50 - 0s - loss: 0.3881
+
    +
    +
    Epoch 27/100
+
    +
    +
    50/50 - 0s - loss: 0.3886
+
    +
    +
    Epoch 28/100
+
    +
    +
    50/50 - 0s - loss: 0.3884
+
    +
    +
    Epoch 29/100
+
    +
    +
    50/50 - 0s - loss: 0.3867
+
    +
    +
    Epoch 30/100
+
    +
    +
    50/50 - 0s - loss: 0.3878
+
    +
    +
    Epoch 31/100
+
    +
    +
    50/50 - 0s - loss: 0.3871
+
    +
    +
    Epoch 32/100
+
    +
    +
    50/50 - 0s - loss: 0.3866
+
    +
    +
    Epoch 33/100
+
    +
    +
    50/50 - 0s - loss: 0.3869
+
    +
    +
    Epoch 34/100
+
    +
    +
    50/50 - 0s - loss: 0.3869
+
    +
    +
    Epoch 35/100
+
    +
    +
    50/50 - 0s - loss: 0.3854
+
    +
    +
    Epoch 36/100
+
    +
    +
    50/50 - 0s - loss: 0.3848
+
    +
    +
    Epoch 37/100
+
    +
    +
    50/50 - 0s - loss: 0.3854
+
    +
    +
    Epoch 38/100
+
    +
    +
    50/50 - 0s - loss: 0.3855
+
    +
    +
    Epoch 39/100
+
    +
    +
    50/50 - 0s - loss: 0.3839
+
    +
    +
    Epoch 40/100
+
    +
    +
    50/50 - 0s - loss: 0.3845
+
    +
    +
    Epoch 41/100
+
    +
    +
    50/50 - 0s - loss: 0.3838
+
    +
    +
    Epoch 42/100
+
    +
    +
    50/50 - 0s - loss: 0.3824
+
    +
    +
    Epoch 43/100
+
    +
    +
    50/50 - 0s - loss: 0.3820
+
    +
    +
    Epoch 44/100
+
    +
    +
    50/50 - 0s - loss: 0.3828
+
    +
    +
    Epoch 45/100
+
    +
    +
    50/50 - 0s - loss: 0.3818
+
    +
    +
    Epoch 46/100
+
    +
    +
    50/50 - 0s - loss: 0.3808
+
    +
    +
    Epoch 47/100
+
    +
    +
    50/50 - 0s - loss: 0.3827
+
    +
    +
    Epoch 48/100
+
    +
    +
    50/50 - 0s - loss: 0.3818
+
    +
    +
    Epoch 49/100
+
    +
    +
    50/50 - 0s - loss: 0.3807
+
    +
    +
    Epoch 50/100
+
    +
    +
    50/50 - 0s - loss: 0.3827
+
    +
    +
    Epoch 51/100
+
    +
    +
    50/50 - 0s - loss: 0.3796
+
    +
    +
    Epoch 52/100
+
    +
    +
    50/50 - 0s - loss: 0.3808
+
    +
    +
    Epoch 53/100
+
    +
    +
    50/50 - 0s - loss: 0.3798
+
    +
    +
    Epoch 54/100
+
    +
    +
    50/50 - 0s - loss: 0.3807
+
    +
    +
    Epoch 55/100
+
    +
    +
    50/50 - 0s - loss: 0.3776
+
    +
    +
    Epoch 56/100
+
    +
    +
    50/50 - 0s - loss: 0.3777
+
    +
    +
    Epoch 57/100
+
    +
    +
    50/50 - 0s - loss: 0.3793
+
    +
    +
    Epoch 58/100
+
    +
    +
    50/50 - 0s - loss: 0.3785
+
    +
    +
    Epoch 59/100
+
    +
    +
    50/50 - 0s - loss: 0.3778
+
    +
    +
    Epoch 60/100
+
    +
    +
    50/50 - 0s - loss: 0.3782
+
    +
    +
    Epoch 61/100
+
    +
    +
    50/50 - 0s - loss: 0.3776
+
    +
    +
    Epoch 62/100
+50/50 - 0s - loss: 0.3775
+
    +
    +
    Epoch 63/100
+
    +
    +
    50/50 - 0s - loss: 0.3767
+
    +
    +
    Epoch 64/100
+
    +
    +
    50/50 - 0s - loss: 0.3761
+
    +
    +
    Epoch 65/100
+
    +
    +
    50/50 - 0s - loss: 0.3762
+
    +
    +
    Epoch 66/100
+
    +
    +
    50/50 - 0s - loss: 0.3760
+
    +
    +
    Epoch 67/100
+
    +
    +
    50/50 - 0s - loss: 0.3762
+
    +
    +
    Epoch 68/100
+
    +
    +
    50/50 - 0s - loss: 0.3745
+
    +
    +
    Epoch 69/100
+
    +
    +
    50/50 - 0s - loss: 0.3747
+
    +
    +
    Epoch 70/100
+
    +
    +
    50/50 - 0s - loss: 0.3756
+
    +
    +
    Epoch 71/100
+
    +
    +
    50/50 - 0s - loss: 0.3754
+
    +
    +
    Epoch 72/100
+
    +
    +
    50/50 - 0s - loss: 0.3761
+
    +
    +
    Epoch 73/100
+
    +
    +
    50/50 - 0s - loss: 0.3758
+
    +
    +
    Epoch 74/100
+
    +
    +
    50/50 - 0s - loss: 0.3744
+
    +
    +
    Epoch 75/100
+
    +
    +
    50/50 - 0s - loss: 0.3732
+
    +
    +
    Epoch 76/100
+
    +
    +
    50/50 - 0s - loss: 0.3733
+
    +
    +
    Epoch 77/100
+
    +
    +
    50/50 - 0s - loss: 0.3724
+
    +
    +
    Epoch 78/100
+
    +
    +
    50/50 - 0s - loss: 0.3726
+
    +
    +
    Epoch 79/100
+
    +
    +
    50/50 - 0s - loss: 0.3726
+
    +
    +
    Epoch 80/100
+
    +
    +
    50/50 - 0s - loss: 0.3710
+
    +
    +
    Epoch 81/100
+
    +
    +
    50/50 - 0s - loss: 0.3700
+
    +
    +
    Epoch 82/100
+
    +
    +
    50/50 - 0s - loss: 0.3711
+
    +
    +
    Epoch 83/100
+
    +
    +
    50/50 - 0s - loss: 0.3718
+
    +
    +
    Epoch 84/100
+
    +
    +
    50/50 - 0s - loss: 0.3714
+
    +
    +
    Epoch 85/100
+
    +
    +
    50/50 - 0s - loss: 0.3712
+
    +
    +
    Epoch 86/100
+
    +
    +
    50/50 - 0s - loss: 0.3713
+
    +
    +
    Epoch 87/100
+
    +
    +
    50/50 - 0s - loss: 0.3710
+
    +
    +
    Epoch 88/100
+
    +
    +
    50/50 - 0s - loss: 0.3714
+
    +
    +
    Epoch 89/100
+
    +
    +
    50/50 - 0s - loss: 0.3688
+
    +
    +
    Epoch 90/100
+
    +
    +
    50/50 - 0s - loss: 0.3694
+
    +
    +
    Epoch 91/100
+
    +
    +
    50/50 - 0s - loss: 0.3696
+
    +
    +
    Epoch 92/100
+
    +
    +
    50/50 - 0s - loss: 0.3695
+
    +
    +
    Epoch 93/100
+
    +
    +
    50/50 - 0s - loss: 0.3694
+
    +
    +
    Epoch 94/100
+
    +
    +
    50/50 - 0s - loss: 0.3704
+
    +
    +
    Epoch 95/100
+
    +
    +
    50/50 - 0s - loss: 0.3676
+
    +
    +
    Epoch 96/100
+
    +
    +
    50/50 - 0s - loss: 0.3693
+
    +
    +
    Epoch 97/100
+
    +
    +
    50/50 - 0s - loss: 0.3664
+
    +
    +
    Epoch 98/100
+
    +
    +
    50/50 - 0s - loss: 0.3673
+
    +
    +
    Epoch 99/100
+
    +
    +
    50/50 - 0s - loss: 0.3687
+
    +
    +
    Epoch 100/100
+
    +
    +
    50/50 - 0s - loss: 0.3675
+
    +
    +
    0.36208778619766235
+
    +
    +_images/chapter10_1_197.png +
    +
    +
    +
    +

    9.4. An extrapolation example

    +

    The following code provides an example of how recurrent neural +networks can be used to extrapolate to unknown values of physics data +sets. Specifically, the data sets used in this program come from +a quantum mechanical many-body calculation of energies as functions of the number of particles.

    +
    +
    +
    # For matrices and calculations
+import numpy as np
+# For machine learning (backend for keras)
+import tensorflow as tf
+# User-friendly machine learning library
+# Front end for TensorFlow
+import tensorflow.keras
+# Different methods from Keras needed to create an RNN
+# This is not necessary but it shortened function calls 
+# that need to be used in the code.
+from tensorflow.keras import datasets, layers, models
+from tensorflow.keras.layers import Input
+from tensorflow.keras import regularizers
+from tensorflow.keras.models import Model, Sequential
+from tensorflow.keras.layers import Dense, SimpleRNN, LSTM, GRU
+# For timing the code
+from timeit import default_timer as timer
+# For plotting
+import matplotlib.pyplot as plt
+
+
+# The data set
+datatype='VaryDimension'
+X_tot = np.arange(2, 42, 2)
+y_tot = np.array([-0.03077640549, -0.08336233266, -0.1446729567, -0.2116753732, -0.2830637392, -0.3581341341, -0.436462435, -0.5177783846,
+	-0.6019067271, -0.6887363571, -0.7782028952, -0.8702784034, -0.9649652536, -1.062292565, -1.16231451, 
+	-1.265109911, -1.370782966, -1.479465113, -1.591317992, -1.70653767])
+
    +
    +
    +
    +
    +
    +

    9.5. Formatting the Data

    +

    The way the recurrent neural networks are trained in this program +differs from how machine learning algorithms are usually trained. +Typically a machine learning algorithm is trained by learning the +relationship between the x data and the y data. In this program, the +recurrent neural network will be trained to recognize the relationship +in a sequence of y values. This is type of data formatting is +typically used time series forcasting, but it can also be used in any +extrapolation (time series forecasting is just a specific type of +extrapolation along the time axis). This method of data formatting +does not use the x data and assumes that the y data are evenly spaced.

    +

    For a standard machine learning algorithm, the training data has the +form of (x,y) so the machine learning algorithm learns to assiciate a +y value with a given x value. This is useful when the test data has x +values within the same range as the training data. However, for this +application, the x values of the test data are outside of the x values +of the training data and the traditional method of training a machine +learning algorithm does not work as well. For this reason, the +recurrent neural network is trained on sequences of y values of the +form ((y1, y2), y3), so that the network is concerned with learning +the pattern of the y data and not the relation between the x and y +data. As long as the pattern of y data outside of the training region +stays relatively stable compared to what was inside the training +region, this method of training can produce accurate extrapolations to +y values far removed from the training data set.

    + + + + + +
    +
    +
    # FORMAT_DATA
+def format_data(data, length_of_sequence = 2):  
+    """
+        Inputs:
+            data(a numpy array): the data that will be the inputs to the recurrent neural
+                network
+            length_of_sequence (an int): the number of elements in one iteration of the
+                sequence patter.  For a function approximator use length_of_sequence = 2.
+        Returns:
+            rnn_input (a 3D numpy array): the input data for the recurrent neural network.  Its
+                dimensions are length of data - length of sequence, length of sequence, 
+                dimnsion of data
+            rnn_output (a numpy array): the training data for the neural network
+        Formats data to be used in a recurrent neural network.
+    """
+
+    X, Y = [], []
+    for i in range(len(data)-length_of_sequence):
+        # Get the next length_of_sequence elements
+        a = data[i:i+length_of_sequence]
+        # Get the element that immediately follows that
+        b = data[i+length_of_sequence]
+        # Reshape so that each data point is contained in its own array
+        a = np.reshape (a, (len(a), 1))
+        X.append(a)
+        Y.append(b)
+    rnn_input = np.array(X)
+    rnn_output = np.array(Y)
+
+    return rnn_input, rnn_output
+
+
+# ## Defining the Recurrent Neural Network Using Keras
+# 
+# The following method defines a simple recurrent neural network in keras consisting of one input layer, one hidden layer, and one output layer.
+
+def rnn(length_of_sequences, batch_size = None, stateful = False):
+    """
+        Inputs:
+            length_of_sequences (an int): the number of y values in "x data".  This is determined
+                when the data is formatted
+            batch_size (an int): Default value is None.  See Keras documentation of SimpleRNN.
+            stateful (a boolean): Default value is False.  See Keras documentation of SimpleRNN.
+        Returns:
+            model (a Keras model): The recurrent neural network that is built and compiled by this
+                method
+        Builds and compiles a recurrent neural network with one hidden layer and returns the model.
+    """
+    # Number of neurons in the input and output layers
+    in_out_neurons = 1
+    # Number of neurons in the hidden layer
+    hidden_neurons = 200
+    # Define the input layer
+    inp = Input(batch_shape=(batch_size, 
+                length_of_sequences, 
+                in_out_neurons))  
+    # Define the hidden layer as a simple RNN layer with a set number of neurons and add it to 
+    # the network immediately after the input layer
+    rnn = SimpleRNN(hidden_neurons, 
+                    return_sequences=False,
+                    stateful = stateful,
+                    name="RNN")(inp)
+    # Define the output layer as a dense neural network layer (standard neural network layer)
+    #and add it to the network immediately after the hidden layer.
+    dens = Dense(in_out_neurons,name="dense")(rnn)
+    # Create the machine learning model starting with the input layer and ending with the 
+    # output layer
+    model = Model(inputs=[inp],outputs=[dens])
+    # Compile the machine learning model using the mean squared error function as the loss 
+    # function and an Adams optimizer.
+    model.compile(loss="mean_squared_error", optimizer="adam")  
+    return model
+
    +
    +
    +
    +
    +
    +

    9.6. Predicting New Points With A Trained Recurrent Neural Network

    +
    +
    +
    def test_rnn (x1, y_test, plot_min, plot_max):
+    """
+        Inputs:
+            x1 (a list or numpy array): The complete x component of the data set
+            y_test (a list or numpy array): The complete y component of the data set
+            plot_min (an int or float): the smallest x value used in the training data
+            plot_max (an int or float): the largest x valye used in the training data
+        Returns:
+            None.
+        Uses a trained recurrent neural network model to predict future points in the 
+        series.  Computes the MSE of the predicted data set from the true data set, saves
+        the predicted data set to a csv file, and plots the predicted and true data sets w
+        while also displaying the data range used for training.
+    """
+    # Add the training data as the first dim points in the predicted data array as these
+    # are known values.
+    y_pred = y_test[:dim].tolist()
+    # Generate the first input to the trained recurrent neural network using the last two 
+    # points of the training data.  Based on how the network was trained this means that it
+    # will predict the first point in the data set after the training data.  All of the 
+    # brackets are necessary for Tensorflow.
+    next_input = np.array([[[y_test[dim-2]], [y_test[dim-1]]]])
+    # Save the very last point in the training data set.  This will be used later.
+    last = [y_test[dim-1]]
+
+    # Iterate until the complete data set is created.
+    for i in range (dim, len(y_test)):
+        # Predict the next point in the data set using the previous two points.
+        next = model.predict(next_input)
+        # Append just the number of the predicted data set
+        y_pred.append(next[0][0])
+        # Create the input that will be used to predict the next data point in the data set.
+        next_input = np.array([[last, next[0]]], dtype=np.float64)
+        last = next
+
+    # Print the mean squared error between the known data set and the predicted data set.
+    print('MSE: ', np.square(np.subtract(y_test, y_pred)).mean())
+    # Save the predicted data set as a csv file for later use
+    name = datatype + 'Predicted'+str(dim)+'.csv'
+    np.savetxt(name, y_pred, delimiter=',')
+    # Plot the known data set and the predicted data set.  The red box represents the region that was used
+    # for the training data.
+    fig, ax = plt.subplots()
+    ax.plot(x1, y_test, label="true", linewidth=3)
+    ax.plot(x1, y_pred, 'g-.',label="predicted", linewidth=4)
+    ax.legend()
+    # Created a red region to represent the points used in the training data.
+    ax.axvspan(plot_min, plot_max, alpha=0.25, color='red')
+    plt.show()
+
+# Check to make sure the data set is complete
+assert len(X_tot) == len(y_tot)
+
+# This is the number of points that will be used in as the training data
+dim=12
+
+# Separate the training data from the whole data set
+X_train = X_tot[:dim]
+y_train = y_tot[:dim]
+
+
+# Generate the training data for the RNN, using a sequence of 2
+rnn_input, rnn_training = format_data(y_train, 2)
+
+
+# Create a recurrent neural network in Keras and produce a summary of the 
+# machine learning model
+model = rnn(length_of_sequences = rnn_input.shape[1])
+model.summary()
+
+# Start the timer.  Want to time training+testing
+start = timer()
+# Fit the model using the training data genenerated above using 150 training iterations and a 5%
+# validation split.  Setting verbose to True prints information about each training iteration.
+hist = model.fit(rnn_input, rnn_training, batch_size=None, epochs=150, 
+                 verbose=True,validation_split=0.05)
+
+for label in ["loss","val_loss"]:
+    plt.plot(hist.history[label],label=label)
+
+plt.ylabel("loss")
+plt.xlabel("epoch")
+plt.title("The final validation loss: {}".format(hist.history["val_loss"][-1]))
+plt.legend()
+plt.show()
+
+# Use the trained neural network to predict more points of the data set
+test_rnn(X_tot, y_tot, X_tot[0], X_tot[dim-1])
+# Stop the timer and calculate the total time needed.
+end = timer()
+print('Time: ', end-start)
+
    +
    +
    +
    +
    Model: "functional_1"
+_________________________________________________________________
+Layer (type)                 Output Shape              Param #   
+=================================================================
+input_1 (InputLayer)         [(None, 2, 1)]            0         
+_________________________________________________________________
+RNN (SimpleRNN)              (None, 200)               40400     
+_________________________________________________________________
+dense (Dense)                (None, 1)                 201       
+=================================================================
+Total params: 40,601
+Trainable params: 40,601
+Non-trainable params: 0
+_________________________________________________________________
+Epoch 1/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.1614
+
    +
    +
    
+1/1 [==============================] - 0s 168ms/step - loss: 0.1614 - val_loss: 0.1927
+
    +
    +
    Epoch 2/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0681
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0681 - val_loss: 0.0356
+
    +
    +
    Epoch 3/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0154
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 0.0154 - val_loss: 0.0018
+
    +
    +
    Epoch 4/150
+
+1/1 [==============================] - ETA: 0s - loss: 4.3675e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 4.3675e-04 - val_loss: 0.0539
+
    +
    +
    Epoch 5/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0127
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0127 - val_loss: 0.1238
+
    +
    +
    Epoch 6/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0318
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0318 - val_loss: 0.1571
+
    +
    +
    Epoch 7/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0409
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0409 - val_loss: 0.1457
+
    +
    +
    Epoch 8/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0373
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 0.0373 - val_loss: 0.1070
+
    +
    +
    Epoch 9/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0263
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 0.0263 - val_loss: 0.0618
+
    +
    +
    Epoch 10/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0140
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 0.0140 - val_loss: 0.0252
+
    +
    +
    Epoch 11/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0049
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 0.0049 - val_loss: 0.0047
+
    +
    +
    Epoch 12/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 8.9601e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 8.9601e-04 - val_loss: 1.7706e-04
+
    +
    +
    Epoch 13/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0015
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 0.0015 - val_loss: 0.0066
+
    +
    +
    Epoch 14/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0050
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0050 - val_loss: 0.0169
+
    +
    +
    Epoch 15/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0092
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0092 - val_loss: 0.0252
+
    +
    +
    Epoch 16/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0122
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0122 - val_loss: 0.0280
+
    +
    +
    Epoch 17/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0131
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 0.0131 - val_loss: 0.0249
+
    +
    +
    Epoch 18/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0119
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 0.0119 - val_loss: 0.0178
+
    +
    +
    Epoch 19/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0091
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0091 - val_loss: 0.0095
+
    +
    +
    Epoch 20/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0057
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 0.0057 - val_loss: 0.0029
+
    +
    +
    Epoch 21/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0028
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 0.0028 - val_loss: 5.1024e-05
+
    +
    +
    Epoch 22/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 9.3047e-04
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 9.3047e-04 - val_loss: 0.0015
+
    +
    +
    Epoch 23/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 4.2170e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 4.2170e-04 - val_loss: 0.0064
+
    +
    +
    Epoch 24/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0011
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 0.0011 - val_loss: 0.0127
+
    +
    +
    Epoch 25/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0024
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 0.0024 - val_loss: 0.0182
+
    +
    +
    Epoch 26/150
+
    +
    +
    
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0037
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 0.0037 - val_loss: 0.0210
+
    +
    +
    Epoch 27/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0044
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 0.0044 - val_loss: 0.0205
+
    +
    +
    Epoch 28/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0043
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 0.0043 - val_loss: 0.0170
+
    +
    +
    Epoch 29/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0035
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 0.0035 - val_loss: 0.0118
+
    +
    +
    Epoch 30/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0024
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 0.0024 - val_loss: 0.0066
+
    +
    +
    Epoch 31/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0012
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 0.0012 - val_loss: 0.0026
+
    +
    +
    Epoch 32/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 4.1938e-04
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 4.1938e-04 - val_loss: 4.2108e-04
+
    +
    +
    Epoch 33/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.4696e-04
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.4696e-04 - val_loss: 4.9351e-05
+
    +
    +
    Epoch 34/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 3.3515e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 3.3515e-04 - val_loss: 8.7654e-04
+
    +
    +
    Epoch 35/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 7.8864e-04
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 7.8864e-04 - val_loss: 0.0021
+
    +
    +
    Epoch 36/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0013
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 0.0013 - val_loss: 0.0030
+
    +
    +
    Epoch 37/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0015
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 0.0015 - val_loss: 0.0031
+
    +
    +
    Epoch 38/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0015
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0015 - val_loss: 0.0026
+
    +
    +
    Epoch 39/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0013
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 0.0013 - val_loss: 0.0016
+
    +
    +
    Epoch 40/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 8.4817e-04
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 8.4817e-04 - val_loss: 5.9712e-04
+
    +
    +
    Epoch 41/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 4.1960e-04
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 4.1960e-04 - val_loss: 5.1834e-05
+
    +
    +
    Epoch 42/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.2261e-04
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.2261e-04 - val_loss: 1.0502e-04
+
    +
    +
    Epoch 43/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 2.5307e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 2.5307e-05 - val_loss: 6.7755e-04
+
    +
    +
    Epoch 44/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.1302e-04
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.1302e-04 - val_loss: 0.0015
+
    +
    +
    Epoch 45/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 3.0417e-04
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 3.0417e-04 - val_loss: 0.0022
+
    +
    +
    Epoch 46/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 4.9066e-04
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 4.9066e-04 - val_loss: 0.0025
+
    +
    +
    Epoch 47/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 5.8441e-04
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 5.8441e-04 - val_loss: 0.0023
+
    +
    +
    Epoch 48/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 5.5018e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 5.5018e-04 - val_loss: 0.0017
+
    +
    +
    Epoch 49/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 4.1198e-04
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 4.1198e-04 - val_loss: 9.9305e-04
+
    +
    +
    Epoch 50/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 2.3364e-04
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 2.3364e-04 - val_loss: 3.8335e-04
+
    +
    +
    Epoch 51/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 8.5796e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 8.5796e-05 - val_loss: 4.9130e-05
+
    +
    +
    Epoch 52/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.5314e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.5314e-05 - val_loss: 2.2907e-05
+
    +
    +
    Epoch 53/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 2.9857e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 2.9857e-05 - val_loss: 2.1308e-04
+
    +
    +
    Epoch 54/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.0133e-04
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.0133e-04 - val_loss: 4.6367e-04
+
    +
    +
    Epoch 55/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.8355e-04
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.8355e-04 - val_loss: 6.2971e-04
+
    +
    +
    Epoch 56/150
+
    +
    +
    
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 2.3446e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 2.3446e-04 - val_loss: 6.3475e-04
+
    +
    +
    Epoch 57/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 2.3285e-04
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 2.3285e-04 - val_loss: 4.9020e-04
+
    +
    +
    Epoch 58/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.8382e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 1.8382e-04 - val_loss: 2.7457e-04
+
    +
    +
    Epoch 59/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.1231e-04
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.1231e-04 - val_loss: 8.7585e-05
+
    +
    +
    Epoch 60/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 4.9274e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 4.9274e-05 - val_loss: 2.2688e-06
+
    +
    +
    Epoch 61/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.7310e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.7310e-05 - val_loss: 3.5831e-05
+
    +
    +
    Epoch 62/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 2.1970e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 2.1970e-05 - val_loss: 1.4969e-04
+
    +
    +
    Epoch 63/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 5.1827e-05
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 5.1827e-05 - val_loss: 2.7475e-04
+
    +
    +
    Epoch 64/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 8.6183e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 8.6183e-05 - val_loss: 3.4722e-04
+
    +
    +
    Epoch 65/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.0592e-04
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.0592e-04 - val_loss: 3.3682e-04
+
    +
    +
    Epoch 66/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.0207e-04
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.0207e-04 - val_loss: 2.5532e-04
+
    +
    +
    Epoch 67/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 7.8321e-05
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 7.8321e-05 - val_loss: 1.4419e-04
+
    +
    +
    Epoch 68/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 4.7236e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 4.7236e-05 - val_loss: 5.0422e-05
+
    +
    +
    Epoch 69/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 2.2810e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 2.2810e-05 - val_loss: 3.9370e-06
+
    +
    +
    Epoch 70/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.3631e-05
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 1.3631e-05 - val_loss: 7.0204e-06
+
    +
    +
    Epoch 71/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.9720e-05
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 1.9720e-05 - val_loss: 3.8779e-05
+
    +
    +
    Epoch 72/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 3.3982e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 3.3982e-05 - val_loss: 6.9903e-05
+
    +
    +
    Epoch 73/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 4.6795e-05
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 4.6795e-05 - val_loss: 7.8865e-05
+
    +
    +
    Epoch 74/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 5.1028e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 5.1028e-05 - val_loss: 6.1482e-05
+
    +
    +
    Epoch 75/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 4.5036e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 4.5036e-05 - val_loss: 3.0208e-05
+
    +
    +
    Epoch 76/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 3.2523e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 3.2523e-05 - val_loss: 5.0685e-06
+
    +
    +
    Epoch 77/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.9844e-05
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 1.9844e-05 - val_loss: 1.9610e-06
+
    +
    +
    Epoch 78/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.2452e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.2452e-05 - val_loss: 2.4494e-05
+
    +
    +
    Epoch 79/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.2335e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.2335e-05 - val_loss: 6.3044e-05
+
    +
    +
    Epoch 80/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.7583e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.7583e-05 - val_loss: 1.0059e-04
+
    +
    +
    Epoch 81/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 2.3980e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 2.3980e-05 - val_loss: 1.2157e-04
+
    +
    +
    Epoch 82/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 2.7513e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 2.7513e-05 - val_loss: 1.1890e-04
+
    +
    +
    Epoch 83/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 2.6387e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 2.6387e-05 - val_loss: 9.5984e-05
+
    +
    +
    Epoch 84/150
+
+1/1 [==============================] - ETA: 0s - loss: 2.1564e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 2.1564e-05 - val_loss: 6.3446e-05
+
    +
    +
    Epoch 85/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.5803e-05
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 1.5803e-05 - val_loss: 3.3027e-05
+
    +
    +
    Epoch 86/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.1910e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.1910e-05 - val_loss: 1.2295e-05
+
    +
    +
    Epoch 87/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.1264e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.1264e-05 - val_loss: 2.4889e-06
+
    +
    +
    Epoch 88/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.3337e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.3337e-05 - val_loss: 4.8689e-08
+
    +
    +
    Epoch 89/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.6291e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.6291e-05 - val_loss: 1.6905e-07
+
    +
    +
    Epoch 90/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.8174e-05
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 1.8174e-05 - val_loss: 8.0057e-08
+
    +
    +
    Epoch 91/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.7968e-05
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 1.7968e-05 - val_loss: 2.7600e-07
+
    +
    +
    Epoch 92/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.5971e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.5971e-05 - val_loss: 3.3847e-06
+
    +
    +
    Epoch 93/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.3408e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.3408e-05 - val_loss: 1.1733e-05
+
    +
    +
    Epoch 94/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.1615e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.1615e-05 - val_loss: 2.5258e-05
+
    +
    +
    Epoch 95/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.1290e-05
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 1.1290e-05 - val_loss: 4.0986e-05
+
    +
    +
    Epoch 96/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.2221e-05
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 1.2221e-05 - val_loss: 5.4292e-05
+
    +
    +
    Epoch 97/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.3550e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.3550e-05 - val_loss: 6.1085e-05
+
    +
    +
    Epoch 98/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.4361e-05
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 1.4361e-05 - val_loss: 5.9625e-05
+
    +
    +
    Epoch 99/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.4195e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.4195e-05 - val_loss: 5.1104e-05
+
    +
    +
    Epoch 100/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.3226e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.3226e-05 - val_loss: 3.8767e-05
+
    +
    +
    Epoch 101/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.2059e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.2059e-05 - val_loss: 2.6280e-05
+
    +
    +
    Epoch 102/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.1312e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.1312e-05 - val_loss: 1.6264e-05
+
    +
    +
    Epoch 103/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.1259e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.1259e-05 - val_loss: 9.6858e-06
+
    +
    +
    Epoch 104/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.1740e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.1740e-05 - val_loss: 6.1976e-06
+
    +
    +
    Epoch 105/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.2320e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.2320e-05 - val_loss: 4.9763e-06
+
    +
    +
    Epoch 106/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.2591e-05
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 1.2591e-05 - val_loss: 5.4740e-06
+
    +
    +
    Epoch 107/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.2401e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.2401e-05 - val_loss: 7.6209e-06
+
    +
    +
    Epoch 108/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.1895e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.1895e-05 - val_loss: 1.1510e-05
+
    +
    +
    Epoch 109/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.1385e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.1385e-05 - val_loss: 1.6893e-05
+
    +
    +
    Epoch 110/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.1131e-05
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 1.1131e-05 - val_loss: 2.2888e-05
+
    +
    +
    Epoch 111/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.1203e-05
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 1.1203e-05 - val_loss: 2.8135e-05
+
    +
    +
    Epoch 112/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.1467e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.1467e-05 - val_loss: 3.1299e-05
+
    +
    +
    Epoch 113/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.1704e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.1704e-05 - val_loss: 3.1629e-05
+
    +
    +
    Epoch 114/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.1751e-05
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 1.1751e-05 - val_loss: 2.9233e-05
+
    +
    +
    Epoch 115/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.1592e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.1592e-05 - val_loss: 2.4977e-05
+
    +
    +
    Epoch 116/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.1341e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.1341e-05 - val_loss: 2.0057e-05
+
    +
    +
    Epoch 117/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.1148e-05
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 1.1148e-05 - val_loss: 1.5546e-05
+
    +
    +
    Epoch 118/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.1104e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.1104e-05 - val_loss: 1.2102e-05
+
    +
    +
    Epoch 119/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.1196e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.1196e-05 - val_loss: 9.9551e-06
+
    +
    +
    Epoch 120/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.1329e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.1329e-05 - val_loss: 9.0678e-06
+
    +
    +
    Epoch 121/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.1403e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.1403e-05 - val_loss: 9.3212e-06
+
    +
    +
    Epoch 122/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.1372e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.1372e-05 - val_loss: 1.0588e-05
+
    +
    +
    Epoch 123/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.1265e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.1265e-05 - val_loss: 1.2699e-05
+
    +
    +
    Epoch 124/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.1155e-05
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 1.1155e-05 - val_loss: 1.5344e-05
+
    +
    +
    Epoch 125/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.1104e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.1104e-05 - val_loss: 1.8049e-05
+
    +
    +
    Epoch 126/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.1126e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.1126e-05 - val_loss: 2.0259e-05
+
    +
    +
    Epoch 127/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.1188e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.1188e-05 - val_loss: 2.1508e-05
+
    +
    +
    Epoch 128/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.1237e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.1237e-05 - val_loss: 2.1579e-05
+
    +
    +
    Epoch 129/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.1239e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.1239e-05 - val_loss: 2.0558e-05
+
    +
    +
    Epoch 130/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.1197e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.1197e-05 - val_loss: 1.8797e-05
+
    +
    +
    Epoch 131/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.1140e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.1140e-05 - val_loss: 1.6756e-05
+
    +
    +
    Epoch 132/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.1104e-05
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 1.1104e-05 - val_loss: 1.4866e-05
+
    +
    +
    Epoch 133/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.1104e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.1104e-05 - val_loss: 1.3434e-05
+
    +
    +
    Epoch 134/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.1131e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.1131e-05 - val_loss: 1.2624e-05
+
    +
    +
    Epoch 135/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.1157e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.1157e-05 - val_loss: 1.2484e-05
+
    +
    +
    Epoch 136/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.1164e-05
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 1.1164e-05 - val_loss: 1.2973e-05
+
    +
    +
    Epoch 137/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.1146e-05
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 1.1146e-05 - val_loss: 1.3983e-05
+
    +
    +
    Epoch 138/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.1119e-05
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 1.1119e-05 - val_loss: 1.5332e-05
+
    +
    +
    Epoch 139/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.1098e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.1098e-05 - val_loss: 1.6776e-05
+
    +
    +
    Epoch 140/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.1095e-05
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 1.1095e-05 - val_loss: 1.8043e-05
+
    +
    +
    Epoch 141/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.1106e-05
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 1.1106e-05 - val_loss: 1.8897e-05
+
    +
    +
    Epoch 142/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.1120e-05
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 1.1120e-05 - val_loss: 1.9197e-05
+
    +
    +
    Epoch 143/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.1125e-05
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 1.1125e-05 - val_loss: 1.8938e-05
+
    +
    +
    Epoch 144/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.1118e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.1118e-05 - val_loss: 1.8237e-05
+
    +
    +
    Epoch 145/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.1104e-05
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 1.1104e-05 - val_loss: 1.7295e-05
+
    +
    +
    Epoch 146/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.1093e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.1093e-05 - val_loss: 1.6333e-05
+
    +
    +
    Epoch 147/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.1091e-05
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 1.1091e-05 - val_loss: 1.5537e-05
+
    +
    +
    Epoch 148/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.1096e-05
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 1.1096e-05 - val_loss: 1.5039e-05
+
    +
    +
    Epoch 149/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.1103e-05
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 1.1103e-05 - val_loss: 1.4898e-05
+
    +
    +
    Epoch 150/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.1105e-05
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 1.1105e-05 - val_loss: 1.5105e-05
+
    +
    +_images/chapter10_7_411.png +
    MSE:  9.884935143687282e-05
+
    +
    +_images/chapter10_7_413.png +
    Time:  4.521348709999998
+
    +
    +
    +
    +
    +
    +

    9.7. Other Things to Try

    +

    Changing the size of the recurrent neural network and its parameters +can drastically change the results you get from the model. The below +code takes the simple recurrent neural network from above and adds a +second hidden layer, changes the number of neurons in the hidden +layer, and explicitly declares the activation function of the hidden +layers to be a sigmoid function. The loss function and optimizer can +also be changed but are kept the same as the above network. These +parameters can be tuned to provide the optimal result from the +network. For some ideas on how to improve the performance of a +recurrent neural network.

    +
    +
    +
    def rnn_2layers(length_of_sequences, batch_size = None, stateful = False):
+    """
+        Inputs:
+            length_of_sequences (an int): the number of y values in "x data".  This is determined
+                when the data is formatted
+            batch_size (an int): Default value is None.  See Keras documentation of SimpleRNN.
+            stateful (a boolean): Default value is False.  See Keras documentation of SimpleRNN.
+        Returns:
+            model (a Keras model): The recurrent neural network that is built and compiled by this
+                method
+        Builds and compiles a recurrent neural network with two hidden layers and returns the model.
+    """
+    # Number of neurons in the input and output layers
+    in_out_neurons = 1
+    # Number of neurons in the hidden layer, increased from the first network
+    hidden_neurons = 500
+    # Define the input layer
+    inp = Input(batch_shape=(batch_size, 
+                length_of_sequences, 
+                in_out_neurons))  
+    # Create two hidden layers instead of one hidden layer.  Explicitly set the activation
+    # function to be the sigmoid function (the default value is hyperbolic tangent)
+    rnn1 = SimpleRNN(hidden_neurons, 
+                    return_sequences=True,  # This needs to be True if another hidden layer is to follow
+                    stateful = stateful, activation = 'sigmoid',
+                    name="RNN1")(inp)
+    rnn2 = SimpleRNN(hidden_neurons, 
+                    return_sequences=False, activation = 'sigmoid',
+                    stateful = stateful,
+                    name="RNN2")(rnn1)
+    # Define the output layer as a dense neural network layer (standard neural network layer)
+    #and add it to the network immediately after the hidden layer.
+    dens = Dense(in_out_neurons,name="dense")(rnn2)
+    # Create the machine learning model starting with the input layer and ending with the 
+    # output layer
+    model = Model(inputs=[inp],outputs=[dens])
+    # Compile the machine learning model using the mean squared error function as the loss 
+    # function and an Adams optimizer.
+    model.compile(loss="mean_squared_error", optimizer="adam")  
+    return model
+
+# Check to make sure the data set is complete
+assert len(X_tot) == len(y_tot)
+
+# This is the number of points that will be used in as the training data
+dim=12
+
+# Separate the training data from the whole data set
+X_train = X_tot[:dim]
+y_train = y_tot[:dim]
+
+
+# Generate the training data for the RNN, using a sequence of 2
+rnn_input, rnn_training = format_data(y_train, 2)
+
+
+# Create a recurrent neural network in Keras and produce a summary of the 
+# machine learning model
+model = rnn_2layers(length_of_sequences = 2)
+model.summary()
+
+# Start the timer.  Want to time training+testing
+start = timer()
+# Fit the model using the training data genenerated above using 150 training iterations and a 5%
+# validation split.  Setting verbose to True prints information about each training iteration.
+hist = model.fit(rnn_input, rnn_training, batch_size=None, epochs=150, 
+                 verbose=True,validation_split=0.05)
+
+
+# This section plots the training loss and the validation loss as a function of training iteration.
+# This is not required for analyzing the couple cluster data but can help determine if the network is
+# being overtrained.
+for label in ["loss","val_loss"]:
+    plt.plot(hist.history[label],label=label)
+
+plt.ylabel("loss")
+plt.xlabel("epoch")
+plt.title("The final validation loss: {}".format(hist.history["val_loss"][-1]))
+plt.legend()
+plt.show()
+
+# Use the trained neural network to predict more points of the data set
+test_rnn(X_tot, y_tot, X_tot[0], X_tot[dim-1])
+# Stop the timer and calculate the total time needed.
+end = timer()
+print('Time: ', end-start)
+
    +
    +
    +
    +
    Model: "functional_3"
+_________________________________________________________________
+Layer (type)                 Output Shape              Param #   
+=================================================================
+input_2 (InputLayer)         [(None, 2, 1)]            0         
+_________________________________________________________________
+RNN1 (SimpleRNN)             (None, 2, 500)            251000    
+_________________________________________________________________
+RNN2 (SimpleRNN)             (None, 500)               500500    
+_________________________________________________________________
+dense (Dense)                (None, 1)                 501       
+=================================================================
+Total params: 752,001
+Trainable params: 752,001
+Non-trainable params: 0
+_________________________________________________________________
+Epoch 1/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.4176
+
    +
    +
    
+1/1 [==============================] - 0s 227ms/step - loss: 1.4176 - val_loss: 3.6488
+
    +
    +
    Epoch 2/150
+
+1/1 [==============================] - ETA: 0s - loss: 5.4802
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 5.4802 - val_loss: 0.5738
+
    +
    +
    Epoch 3/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.4330
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 1.4330 - val_loss: 0.8091
+
    +
    +
    Epoch 4/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.2697
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.2697 - val_loss: 3.5191
+
    +
    +
    Epoch 5/150
+
+1/1 [==============================] - ETA: 0s - loss: 2.1554
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 2.1554 - val_loss: 3.2451
+
    +
    +
    Epoch 6/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.9444
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 1.9444 - val_loss: 1.2933
+
    +
    +
    Epoch 7/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.5533
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.5533 - val_loss: 0.1028
+
    +
    +
    Epoch 8/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0521
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0521 - val_loss: 0.1031
+
    +
    +
    Epoch 9/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.5942
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.5942 - val_loss: 0.3703
+
    +
    +
    Epoch 10/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.1041
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 1.1041 - val_loss: 0.2869
+
    +
    +
    Epoch 11/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.9592
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.9592 - val_loss: 0.0406
+
    +
    +
    Epoch 12/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.4307
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.4307 - val_loss: 0.0672
+
    +
    +
    Epoch 13/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0683
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0683 - val_loss: 0.5012
+
    +
    +
    Epoch 14/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.1235
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.1235 - val_loss: 1.0556
+
    +
    +
    Epoch 15/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.4083
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 0.4083 - val_loss: 1.3256
+
    +
    +
    Epoch 16/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.5737
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.5737 - val_loss: 1.1606
+
    +
    +
    Epoch 17/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.4712
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.4712 - val_loss: 0.7278
+
    +
    +
    Epoch 18/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.2276
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.2276 - val_loss: 0.3054
+
    +
    +
    Epoch 19/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0589
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0589 - val_loss: 0.0652
+
    +
    +
    Epoch 20/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0695
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0695 - val_loss: 8.7287e-04
+
    +
    +
    Epoch 21/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.1958
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.1958 - val_loss: 0.0067
+
    +
    +
    Epoch 22/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.2957
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.2957 - val_loss: 0.0046
+
    +
    +
    Epoch 23/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.2815
+
    +
    +
    
+1/1 [==============================] - 0s 20ms/step - loss: 0.2815 - val_loss: 0.0030
+
    +
    +
    Epoch 24/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.1768
+
    +
    +
    
+1/1 [==============================] - 0s 20ms/step - loss: 0.1768 - val_loss: 0.0605
+
    +
    +
    Epoch 25/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0727
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0727 - val_loss: 0.2095
+
    +
    +
    Epoch 26/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0430
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0430 - val_loss: 0.4103
+
    +
    +
    Epoch 27/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0896
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 0.0896 - val_loss: 0.5719
+
    +
    +
    Epoch 28/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.1536
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.1536 - val_loss: 0.6165
+
    +
    +
    Epoch 29/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.1739
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.1739 - val_loss: 0.5322
+
    +
    +
    Epoch 30/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.1365
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 0.1365 - val_loss: 0.3724
+
    +
    +
    Epoch 31/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0773
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0773 - val_loss: 0.2118
+
    +
    +
    Epoch 32/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0432
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0432 - val_loss: 0.0999
+
    +
    +
    Epoch 33/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0528
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0528 - val_loss: 0.0440
+
    +
    +
    Epoch 34/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0865
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 0.0865 - val_loss: 0.0261
+
    +
    +
    Epoch 35/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.1092
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.1092 - val_loss: 0.0315
+
    +
    +
    Epoch 36/150
+
    +
    +
    
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.1012
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.1012 - val_loss: 0.0612
+
    +
    +
    Epoch 37/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0719
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 0.0719 - val_loss: 0.1231
+
    +
    +
    Epoch 38/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0466
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0466 - val_loss: 0.2126
+
    +
    +
    Epoch 39/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0432
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0432 - val_loss: 0.3043
+
    +
    +
    Epoch 40/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0587
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0587 - val_loss: 0.3636
+
    +
    +
    Epoch 41/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0747
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0747 - val_loss: 0.3676
+
    +
    +
    Epoch 42/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0759
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0759 - val_loss: 0.3193
+
    +
    +
    Epoch 43/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0623
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0623 - val_loss: 0.2431
+
    +
    +
    Epoch 44/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0468
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0468 - val_loss: 0.1684
+
    +
    +
    Epoch 45/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0417
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0417 - val_loss: 0.1143
+
    +
    +
    Epoch 46/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0485
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0485 - val_loss: 0.0857
+
    +
    +
    Epoch 47/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0580
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0580 - val_loss: 0.0803
+
    +
    +
    Epoch 48/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0605
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 0.0605 - val_loss: 0.0955
+
    +
    +
    Epoch 49/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0540
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 0.0540 - val_loss: 0.1299
+
    +
    +
    Epoch 50/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0452
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 0.0452 - val_loss: 0.1782
+
    +
    +
    Epoch 51/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0415
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0415 - val_loss: 0.2284
+
    +
    +
    Epoch 52/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0448
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0448 - val_loss: 0.2639
+
    +
    +
    Epoch 53/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0502
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0502 - val_loss: 0.2725
+
    +
    +
    Epoch 54/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0518
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0518 - val_loss: 0.2531
+
    +
    +
    Epoch 55/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0483
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0483 - val_loss: 0.2155
+
    +
    +
    Epoch 56/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0434
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 0.0434 - val_loss: 0.1742
+
    +
    +
    Epoch 57/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0415
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0415 - val_loss: 0.1411
+
    +
    +
    Epoch 58/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0435
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0435 - val_loss: 0.1224
+
    +
    +
    Epoch 59/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0464
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0464 - val_loss: 0.1194
+
    +
    +
    Epoch 60/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0470
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0470 - val_loss: 0.1310
+
    +
    +
    Epoch 61/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0448
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0448 - val_loss: 0.1544
+
    +
    +
    Epoch 62/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0422
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 0.0422 - val_loss: 0.1836
+
    +
    +
    Epoch 63/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0415
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 0.0415 - val_loss: 0.2101
+
    +
    +
    Epoch 64/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0429
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0429 - val_loss: 0.2253
+
    +
    +
    Epoch 65/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0444
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0444 - val_loss: 0.2246
+
    +
    +
    Epoch 66/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0443
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0443 - val_loss: 0.2096
+
    +
    +
    Epoch 67/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0428
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0428 - val_loss: 0.1869
+
    +
    +
    Epoch 68/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0415
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0415 - val_loss: 0.1645
+
    +
    +
    Epoch 69/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0415
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0415 - val_loss: 0.1485
+
    +
    +
    Epoch 70/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0425
+
    +
    +
    
+1/1 [==============================] - 0s 20ms/step - loss: 0.0425 - val_loss: 0.1423
+
    +
    +
    Epoch 71/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0431
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 0.0431 - val_loss: 0.1461
+
    +
    +
    Epoch 72/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0427
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0427 - val_loss: 0.1584
+
    +
    +
    Epoch 73/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0418
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 0.0418 - val_loss: 0.1752
+
    +
    +
    Epoch 74/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0413
+
    +
    +
    
+1/1 [==============================] - 0s 20ms/step - loss: 0.0413 - val_loss: 0.1915
+
    +
    +
    Epoch 75/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0416
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 0.0416 - val_loss: 0.2018
+
    +
    +
    Epoch 76/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0421
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0421 - val_loss: 0.2030
+
    +
    +
    Epoch 77/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0422
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 0.0422 - val_loss: 0.1955
+
    +
    +
    Epoch 78/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0418
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0418 - val_loss: 0.1827
+
    +
    +
    Epoch 79/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0413
+
    +
    +
    
+1/1 [==============================] - 0s 21ms/step - loss: 0.0413 - val_loss: 0.1693
+
    +
    +
    Epoch 80/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0413
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0413 - val_loss: 0.1595
+
    +
    +
    Epoch 81/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0416
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 0.0416 - val_loss: 0.1557
+
    +
    +
    Epoch 82/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0418
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0418 - val_loss: 0.1584
+
    +
    +
    Epoch 83/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0416
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0416 - val_loss: 0.1662
+
    +
    +
    Epoch 84/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0413
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0413 - val_loss: 0.1764
+
    +
    +
    Epoch 85/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0411
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0411 - val_loss: 0.1856
+
    +
    +
    Epoch 86/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0413
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0413 - val_loss: 0.1906
+
    +
    +
    Epoch 87/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0414
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0414 - val_loss: 0.1899
+
    +
    +
    Epoch 88/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0414
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0414 - val_loss: 0.1844
+
    +
    +
    Epoch 89/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0412
+
    +
    +
    
+1/1 [==============================] - 0s 20ms/step - loss: 0.0412 - val_loss: 0.1764
+
    +
    +
    Epoch 90/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0411
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0411 - val_loss: 0.1688
+
    +
    +
    Epoch 91/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0411
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0411 - val_loss: 0.1641
+
    +
    +
    Epoch 92/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0412
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0412 - val_loss: 0.1634
+
    +
    +
    Epoch 93/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0412
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0412 - val_loss: 0.1666
+
    +
    +
    Epoch 94/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0411
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0411 - val_loss: 0.1723
+
    +
    +
    Epoch 95/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0410
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0410 - val_loss: 0.1783
+
    +
    +
    Epoch 96/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0410
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0410 - val_loss: 0.1826
+
    +
    +
    Epoch 97/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0411
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0411 - val_loss: 0.1836
+
    +
    +
    Epoch 98/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0411
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0411 - val_loss: 0.1814
+
    +
    +
    Epoch 99/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0410
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0410 - val_loss: 0.1769
+
    +
    +
    Epoch 100/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0409
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 0.0409 - val_loss: 0.1720
+
    +
    +
    Epoch 101/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0409
+
    +
    +
    
+1/1 [==============================] - 0s 20ms/step - loss: 0.0409 - val_loss: 0.1684
+
    +
    +
    Epoch 102/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0409
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0409 - val_loss: 0.1673
+
    +
    +
    Epoch 103/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0409
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0409 - val_loss: 0.1687
+
    +
    +
    Epoch 104/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0409
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0409 - val_loss: 0.1719
+
    +
    +
    Epoch 105/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0409
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 0.0409 - val_loss: 0.1756
+
    +
    +
    Epoch 106/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0408
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0408 - val_loss: 0.1784
+
    +
    +
    Epoch 107/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0408
+
    +
    +
    
+1/1 [==============================] - 0s 20ms/step - loss: 0.0408 - val_loss: 0.1793
+
    +
    +
    Epoch 108/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0408
+
    +
    +
    
+1/1 [==============================] - 0s 20ms/step - loss: 0.0408 - val_loss: 0.1781
+
    +
    +
    Epoch 109/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0408
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 0.0408 - val_loss: 0.1754
+
    +
    +
    Epoch 110/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0408
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0408 - val_loss: 0.1723
+
    +
    +
    Epoch 111/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0408
+
    +
    +
    
+1/1 [==============================] - 0s 21ms/step - loss: 0.0408 - val_loss: 0.1701
+
    +
    +
    Epoch 112/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0408
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0408 - val_loss: 0.1693
+
    +
    +
    Epoch 113/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0407
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0407 - val_loss: 0.1702
+
    +
    +
    Epoch 114/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0407
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0407 - val_loss: 0.1722
+
    +
    +
    Epoch 115/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0407
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0407 - val_loss: 0.1745
+
    +
    +
    Epoch 116/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0407
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0407 - val_loss: 0.1760
+
    +
    +
    Epoch 117/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0407
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0407 - val_loss: 0.1764
+
    +
    +
    Epoch 118/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0407
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0407 - val_loss: 0.1754
+
    +
    +
    Epoch 119/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0406
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0406 - val_loss: 0.1736
+
    +
    +
    Epoch 120/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0406
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 0.0406 - val_loss: 0.1718
+
    +
    +
    Epoch 121/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0406
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0406 - val_loss: 0.1705
+
    +
    +
    Epoch 122/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0406
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0406 - val_loss: 0.1703
+
    +
    +
    Epoch 123/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0406
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0406 - val_loss: 0.1711
+
    +
    +
    Epoch 124/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0405
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0405 - val_loss: 0.1724
+
    +
    +
    Epoch 125/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0405
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0405 - val_loss: 0.1737
+
    +
    +
    Epoch 126/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0405
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0405 - val_loss: 0.1744
+
    +
    +
    Epoch 127/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0405
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0405 - val_loss: 0.1742
+
    +
    +
    Epoch 128/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0405
+
    +
    +
    
+1/1 [==============================] - 0s 20ms/step - loss: 0.0405 - val_loss: 0.1734
+
    +
    +
    Epoch 129/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0405
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0405 - val_loss: 0.1721
+
    +
    +
    Epoch 130/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0404
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0404 - val_loss: 0.1711
+
    +
    +
    Epoch 131/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0404
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0404 - val_loss: 0.1706
+
    +
    +
    Epoch 132/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0404
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0404 - val_loss: 0.1707
+
    +
    +
    Epoch 133/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0404
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 0.0404 - val_loss: 0.1714
+
    +
    +
    Epoch 134/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0404
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0404 - val_loss: 0.1722
+
    +
    +
    Epoch 135/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0403
+
    +
    +
    
+1/1 [==============================] - 0s 20ms/step - loss: 0.0403 - val_loss: 0.1728
+
    +
    +
    Epoch 136/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0403
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0403 - val_loss: 0.1729
+
    +
    +
    Epoch 137/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0403
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0403 - val_loss: 0.1725
+
    +
    +
    Epoch 138/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0403
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0403 - val_loss: 0.1717
+
    +
    +
    Epoch 139/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0403
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0403 - val_loss: 0.1710
+
    +
    +
    Epoch 140/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0402
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0402 - val_loss: 0.1705
+
    +
    +
    Epoch 141/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0402
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0402 - val_loss: 0.1704
+
    +
    +
    Epoch 142/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0402
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0402 - val_loss: 0.1707
+
    +
    +
    Epoch 143/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0402
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0402 - val_loss: 0.1712
+
    +
    +
    Epoch 144/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0402
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0402 - val_loss: 0.1716
+
    +
    +
    Epoch 145/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0402
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0402 - val_loss: 0.1717
+
    +
    +
    Epoch 146/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0401
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 0.0401 - val_loss: 0.1715
+
    +
    +
    Epoch 147/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0401
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0401 - val_loss: 0.1710
+
    +
    +
    Epoch 148/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0401
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0401 - val_loss: 0.1705
+
    +
    +
    Epoch 149/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0401
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0401 - val_loss: 0.1701
+
    +
    +
    Epoch 150/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0401
+
    +
    +
    
+1/1 [==============================] - 0s 20ms/step - loss: 0.0401 - val_loss: 0.1699
+
    +
    +_images/chapter10_9_421.png +
    MSE:  0.32920351498520983
+
    +
    +_images/chapter10_9_423.png +
    Time:  6.260979576
+
    +
    +
    +
    +
    +
    +

    9.8. Other Types of Recurrent Neural Networks

    +

    Besides a simple recurrent neural network layer, there are two other +commonly used types of recurrent neural network layers: Long Short +Term Memory (LSTM) and Gated Recurrent Unit (GRU). For a short +introduction to these layers see https://medium.com/mindboard/lstm-vs-gru-experimental-comparison-955820c21e8b +and https://medium.com/mindboard/lstm-vs-gru-experimental-comparison-955820c21e8b.

    +

    The first network created below is similar to the previous network, +but it replaces the SimpleRNN layers with LSTM layers. The second +network below has two hidden layers made up of GRUs, which are +preceeded by two dense (feeddorward) neural network layers. These +dense layers “preprocess” the data before it reaches the recurrent +layers. This architecture has been shown to improve the performance +of recurrent neural networks (see the link above and also +https://arxiv.org/pdf/1807.02857.pdf.

    +
    +
    +
    def lstm_2layers(length_of_sequences, batch_size = None, stateful = False):
+    """
+        Inputs:
+            length_of_sequences (an int): the number of y values in "x data".  This is determined
+                when the data is formatted
+            batch_size (an int): Default value is None.  See Keras documentation of SimpleRNN.
+            stateful (a boolean): Default value is False.  See Keras documentation of SimpleRNN.
+        Returns:
+            model (a Keras model): The recurrent neural network that is built and compiled by this
+                method
+        Builds and compiles a recurrent neural network with two LSTM hidden layers and returns the model.
+    """
+    # Number of neurons on the input/output layer and the number of neurons in the hidden layer
+    in_out_neurons = 1
+    hidden_neurons = 250
+    # Input Layer
+    inp = Input(batch_shape=(batch_size, 
+                length_of_sequences, 
+                in_out_neurons)) 
+    # Hidden layers (in this case they are LSTM layers instead if SimpleRNN layers)
+    rnn= LSTM(hidden_neurons, 
+                    return_sequences=True,
+                    stateful = stateful,
+                    name="RNN", use_bias=True, activation='tanh')(inp)
+    rnn1 = LSTM(hidden_neurons, 
+                    return_sequences=False,
+                    stateful = stateful,
+                    name="RNN1", use_bias=True, activation='tanh')(rnn)
+    # Output layer
+    dens = Dense(in_out_neurons,name="dense")(rnn1)
+    # Define the midel
+    model = Model(inputs=[inp],outputs=[dens])
+    # Compile the model
+    model.compile(loss='mean_squared_error', optimizer='adam')  
+    # Return the model
+    return model
+
+def dnn2_gru2(length_of_sequences, batch_size = None, stateful = False):
+    """
+        Inputs:
+            length_of_sequences (an int): the number of y values in "x data".  This is determined
+                when the data is formatted
+            batch_size (an int): Default value is None.  See Keras documentation of SimpleRNN.
+            stateful (a boolean): Default value is False.  See Keras documentation of SimpleRNN.
+        Returns:
+            model (a Keras model): The recurrent neural network that is built and compiled by this
+                method
+        Builds and compiles a recurrent neural network with four hidden layers (two dense followed by
+        two GRU layers) and returns the model.
+    """    
+    # Number of neurons on the input/output layers and hidden layers
+    in_out_neurons = 1
+    hidden_neurons = 250
+    # Input layer
+    inp = Input(batch_shape=(batch_size, 
+                length_of_sequences, 
+                in_out_neurons)) 
+    # Hidden Dense (feedforward) layers
+    dnn = Dense(hidden_neurons/2, activation='relu', name='dnn')(inp)
+    dnn1 = Dense(hidden_neurons/2, activation='relu', name='dnn1')(dnn)
+    # Hidden GRU layers
+    rnn1 = GRU(hidden_neurons, 
+                    return_sequences=True,
+                    stateful = stateful,
+                    name="RNN1", use_bias=True)(dnn1)
+    rnn = GRU(hidden_neurons, 
+                    return_sequences=False,
+                    stateful = stateful,
+                    name="RNN", use_bias=True)(rnn1)
+    # Output layer
+    dens = Dense(in_out_neurons,name="dense")(rnn)
+    # Define the model
+    model = Model(inputs=[inp],outputs=[dens])
+    # Compile the mdoel
+    model.compile(loss='mean_squared_error', optimizer='adam')  
+    # Return the model
+    return model
+
+# Check to make sure the data set is complete
+assert len(X_tot) == len(y_tot)
+
+# This is the number of points that will be used in as the training data
+dim=12
+
+# Separate the training data from the whole data set
+X_train = X_tot[:dim]
+y_train = y_tot[:dim]
+
+
+# Generate the training data for the RNN, using a sequence of 2
+rnn_input, rnn_training = format_data(y_train, 2)
+
+
+# Create a recurrent neural network in Keras and produce a summary of the 
+# machine learning model
+# Change the method name to reflect which network you want to use
+model = dnn2_gru2(length_of_sequences = 2)
+model.summary()
+
+# Start the timer.  Want to time training+testing
+start = timer()
+# Fit the model using the training data genenerated above using 150 training iterations and a 5%
+# validation split.  Setting verbose to True prints information about each training iteration.
+hist = model.fit(rnn_input, rnn_training, batch_size=None, epochs=150, 
+                 verbose=True,validation_split=0.05)
+
+
+# This section plots the training loss and the validation loss as a function of training iteration.
+# This is not required for analyzing the couple cluster data but can help determine if the network is
+# being overtrained.
+for label in ["loss","val_loss"]:
+    plt.plot(hist.history[label],label=label)
+
+plt.ylabel("loss")
+plt.xlabel("epoch")
+plt.title("The final validation loss: {}".format(hist.history["val_loss"][-1]))
+plt.legend()
+plt.show()
+
+# Use the trained neural network to predict more points of the data set
+test_rnn(X_tot, y_tot, X_tot[0], X_tot[dim-1])
+# Stop the timer and calculate the total time needed.
+end = timer()
+print('Time: ', end-start)
+
+
+# ### Training Recurrent Neural Networks in the Standard Way (i.e. learning the relationship between the X and Y data)
+# 
+# Finally, comparing the performace of a recurrent neural network using the standard data formatting to the performance of the network with time sequence data formatting shows the benefit of this type of data formatting with extrapolation.
+
+# Check to make sure the data set is complete
+assert len(X_tot) == len(y_tot)
+
+# This is the number of points that will be used in as the training data
+dim=12
+
+# Separate the training data from the whole data set
+X_train = X_tot[:dim]
+y_train = y_tot[:dim]
+
+# Reshape the data for Keras specifications
+X_train = X_train.reshape((dim, 1))
+y_train = y_train.reshape((dim, 1))
+
+
+# Create a recurrent neural network in Keras and produce a summary of the 
+# machine learning model
+# Set the sequence length to 1 for regular data formatting 
+model = rnn(length_of_sequences = 1)
+model.summary()
+
+# Start the timer.  Want to time training+testing
+start = timer()
+# Fit the model using the training data genenerated above using 150 training iterations and a 5%
+# validation split.  Setting verbose to True prints information about each training iteration.
+hist = model.fit(X_train, y_train, batch_size=None, epochs=150, 
+                 verbose=True,validation_split=0.05)
+
+
+# This section plots the training loss and the validation loss as a function of training iteration.
+# This is not required for analyzing the couple cluster data but can help determine if the network is
+# being overtrained.
+for label in ["loss","val_loss"]:
+    plt.plot(hist.history[label],label=label)
+
+plt.ylabel("loss")
+plt.xlabel("epoch")
+plt.title("The final validation loss: {}".format(hist.history["val_loss"][-1]))
+plt.legend()
+plt.show()
+
+# Use the trained neural network to predict the remaining data points
+X_pred = X_tot[dim:]
+X_pred = X_pred.reshape((len(X_pred), 1))
+y_model = model.predict(X_pred)
+y_pred = np.concatenate((y_tot[:dim], y_model.flatten()))
+
+# Plot the known data set and the predicted data set.  The red box represents the region that was used
+# for the training data.
+fig, ax = plt.subplots()
+ax.plot(X_tot, y_tot, label="true", linewidth=3)
+ax.plot(X_tot, y_pred, 'g-.',label="predicted", linewidth=4)
+ax.legend()
+# Created a red region to represent the points used in the training data.
+ax.axvspan(X_tot[0], X_tot[dim], alpha=0.25, color='red')
+plt.show()
+
+# Stop the timer and calculate the total time needed.
+end = timer()
+print('Time: ', end-start)
+
    +
    +
    +
    +
    Model: "functional_5"
+_________________________________________________________________
+Layer (type)                 Output Shape              Param #   
+=================================================================
+input_3 (InputLayer)         [(None, 2, 1)]            0         
+_________________________________________________________________
+dnn (Dense)                  (None, 2, 125)            250       
+_________________________________________________________________
+dnn1 (Dense)                 (None, 2, 125)            15750     
+_________________________________________________________________
+RNN1 (GRU)                   (None, 2, 250)            282750    
+_________________________________________________________________
+RNN (GRU)                    (None, 250)               376500    
+_________________________________________________________________
+dense (Dense)                (None, 1)                 251       
+=================================================================
+Total params: 675,501
+Trainable params: 675,501
+Non-trainable params: 0
+_________________________________________________________________
+Epoch 1/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.2339
+
    +
    +
    
+1/1 [==============================] - 1s 653ms/step - loss: 0.2339 - val_loss: 0.5383
+
    +
    +
    Epoch 2/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.1652
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 0.1652 - val_loss: 0.3519
+
    +
    +
    Epoch 3/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.1008
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 0.1008 - val_loss: 0.1726
+
    +
    +
    Epoch 4/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0430
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0430 - val_loss: 0.0348
+
    +
    +
    Epoch 5/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0070
+
    +
    +
    
+1/1 [==============================] - 0s 20ms/step - loss: 0.0070 - val_loss: 0.0070
+
    +
    +
    Epoch 6/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0194
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0194 - val_loss: 0.0534
+
    +
    +
    Epoch 7/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0503
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0503 - val_loss: 0.0415
+
    +
    +
    Epoch 8/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0430
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0430 - val_loss: 0.0088
+
    +
    +
    Epoch 9/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0204
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0204 - val_loss: 0.0014
+
    +
    +
    Epoch 10/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0061
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0061 - val_loss: 0.0239
+
    +
    +
    Epoch 11/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0052
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 0.0052 - val_loss: 0.0584
+
    +
    +
    Epoch 12/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0120
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 0.0120 - val_loss: 0.0860
+
    +
    +
    Epoch 13/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0191
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0191 - val_loss: 0.0977
+
    +
    +
    Epoch 14/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0225
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0225 - val_loss: 0.0928
+
    +
    +
    Epoch 15/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0213
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0213 - val_loss: 0.0754
+
    +
    +
    Epoch 16/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0166
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0166 - val_loss: 0.0516
+
    +
    +
    Epoch 17/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0106
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 0.0106 - val_loss: 0.0278
+
    +
    +
    Epoch 18/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0056
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0056 - val_loss: 0.0099
+
    +
    +
    Epoch 19/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0032
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0032 - val_loss: 0.0011
+
    +
    +
    Epoch 20/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0042
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0042 - val_loss: 4.2492e-04
+
    +
    +
    Epoch 21/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0072
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0072 - val_loss: 0.0028
+
    +
    +
    Epoch 22/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0096
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0096 - val_loss: 0.0035
+
    +
    +
    Epoch 23/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0094
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0094 - val_loss: 0.0016
+
    +
    +
    Epoch 24/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0068
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0068 - val_loss: 2.1770e-05
+
    +
    +
    Epoch 25/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0037
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0037 - val_loss: 0.0014
+
    +
    +
    Epoch 26/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0017
+
    +
    +
    
+1/1 [==============================] - 0s 20ms/step - loss: 0.0017 - val_loss: 0.0061
+
    +
    +
    Epoch 27/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0015
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0015 - val_loss: 0.0121
+
    +
    +
    Epoch 28/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0025
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0025 - val_loss: 0.0166
+
    +
    +
    Epoch 29/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0037
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0037 - val_loss: 0.0177
+
    +
    +
    Epoch 30/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0042
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0042 - val_loss: 0.0151
+
    +
    +
    Epoch 31/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0037
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0037 - val_loss: 0.0101
+
    +
    +
    Epoch 32/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0024
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0024 - val_loss: 0.0046
+
    +
    +
    Epoch 33/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0011
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 0.0011 - val_loss: 8.8886e-04
+
    +
    +
    Epoch 34/150
+
+1/1 [==============================] - ETA: 0s - loss: 2.7391e-04
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 2.7391e-04 - val_loss: 6.9638e-05
+
    +
    +
    Epoch 35/150
+
+1/1 [==============================] - ETA: 0s - loss: 3.0805e-04
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 3.0805e-04 - val_loss: 0.0016
+
    +
    +
    Epoch 36/150
+
+1/1 [==============================] - ETA: 0s - loss: 9.6065e-04
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 9.6065e-04 - val_loss: 0.0037
+
    +
    +
    Epoch 37/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0016
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0016 - val_loss: 0.0045
+
    +
    +
    Epoch 38/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0016
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0016 - val_loss: 0.0035
+
    +
    +
    Epoch 39/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0011
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0011 - val_loss: 0.0016
+
    +
    +
    Epoch 40/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 3.6501e-04
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 3.6501e-04 - val_loss: 2.5101e-04
+
    +
    +
    Epoch 41/150
+
+1/1 [==============================] - ETA: 0s - loss: 5.2387e-05
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 5.2387e-05 - val_loss: 4.9809e-05
+
    +
    +
    Epoch 42/150
+
+1/1 [==============================] - ETA: 0s - loss: 2.7319e-04
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 2.7319e-04 - val_loss: 5.0112e-04
+
    +
    +
    Epoch 43/150
+
+1/1 [==============================] - ETA: 0s - loss: 7.0581e-04
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 7.0581e-04 - val_loss: 7.5034e-04
+
    +
    +
    Epoch 44/150
+
+1/1 [==============================] - ETA: 0s - loss: 9.3238e-04
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 9.3238e-04 - val_loss: 5.0648e-04
+
    +
    +
    Epoch 45/150
+
+1/1 [==============================] - ETA: 0s - loss: 8.0866e-04
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 8.0866e-04 - val_loss: 9.4408e-05
+
    +
    +
    Epoch 46/150
+
+1/1 [==============================] - ETA: 0s - loss: 4.7288e-04
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 4.7288e-04 - val_loss: 5.9254e-05
+
    +
    +
    Epoch 47/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.8597e-04
+
    +
    +
    
+1/1 [==============================] - 0s 20ms/step - loss: 1.8597e-04 - val_loss: 6.7079e-04
+
    +
    +
    Epoch 48/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.2584e-04
+
    +
    +
    
+1/1 [==============================] - 0s 20ms/step - loss: 1.2584e-04 - val_loss: 0.0017
+
    +
    +
    Epoch 49/150
+
+1/1 [==============================] - ETA: 0s - loss: 2.6867e-04
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 2.6867e-04 - val_loss: 0.0024
+
    +
    +
    Epoch 50/150
+
+1/1 [==============================] - ETA: 0s - loss: 4.3826e-04
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 4.3826e-04 - val_loss: 0.0025
+
    +
    +
    Epoch 51/150
+
+1/1 [==============================] - ETA: 0s - loss: 4.6733e-04
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 4.6733e-04 - val_loss: 0.0018
+
    +
    +
    Epoch 52/150
+
+1/1 [==============================] - ETA: 0s - loss: 3.3438e-04
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 3.3438e-04 - val_loss: 9.4701e-04
+
    +
    +
    Epoch 53/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.5339e-04
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 1.5339e-04 - val_loss: 2.6460e-04
+
    +
    +
    Epoch 54/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 5.4331e-05
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 5.4331e-05 - val_loss: 6.5156e-06
+
    +
    +
    Epoch 55/150
+
+1/1 [==============================] - ETA: 0s - loss: 7.7712e-05
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 7.7712e-05 - val_loss: 6.1830e-05
+
    +
    +
    Epoch 56/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.6479e-04
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 1.6479e-04 - val_loss: 1.8163e-04
+
    +
    +
    Epoch 57/150
+
+1/1 [==============================] - ETA: 0s - loss: 2.2423e-04
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 2.2423e-04 - val_loss: 1.9299e-04
+
    +
    +
    Epoch 58/150
+
+1/1 [==============================] - ETA: 0s - loss: 2.0515e-04
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 2.0515e-04 - val_loss: 9.4929e-05
+
    +
    +
    Epoch 59/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.2535e-04
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 1.2535e-04 - val_loss: 6.2223e-06
+
    +
    +
    Epoch 60/150
+
+1/1 [==============================] - ETA: 0s - loss: 4.6826e-05
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 4.6826e-05 - val_loss: 3.4402e-05
+
    +
    +
    Epoch 61/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 2.2165e-05
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 2.2165e-05 - val_loss: 1.7489e-04
+
    +
    +
    Epoch 62/150
+
+1/1 [==============================] - ETA: 0s - loss: 5.5691e-05
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 5.5691e-05 - val_loss: 3.1733e-04
+
    +
    +
    Epoch 63/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.0595e-04
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 1.0595e-04 - val_loss: 3.4700e-04
+
    +
    +
    Epoch 64/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.2525e-04
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 1.2525e-04 - val_loss: 2.4500e-04
+
    +
    +
    Epoch 65/150
+
+1/1 [==============================] - ETA: 0s - loss: 9.9924e-05
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 9.9924e-05 - val_loss: 9.6135e-05
+
    +
    +
    Epoch 66/150
+
+1/1 [==============================] - ETA: 0s - loss: 5.5901e-05
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 5.5901e-05 - val_loss: 6.8225e-06
+
    +
    +
    Epoch 67/150
+
+1/1 [==============================] - ETA: 0s - loss: 2.9911e-05
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 2.9911e-05 - val_loss: 1.7879e-05
+
    +
    +
    Epoch 68/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 3.7141e-05
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 3.7141e-05 - val_loss: 8.5801e-05
+
    +
    +
    Epoch 69/150
+
+1/1 [==============================] - ETA: 0s - loss: 6.3071e-05
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 6.3071e-05 - val_loss: 1.3386e-04
+
    +
    +
    Epoch 70/150
+
+1/1 [==============================] - ETA: 0s - loss: 8.0889e-05
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 8.0889e-05 - val_loss: 1.1862e-04
+
    +
    +
    Epoch 71/150
+
+1/1 [==============================] - ETA: 0s - loss: 7.5159e-05
+
    +
    +
    
+1/1 [==============================] - 0s 20ms/step - loss: 7.5159e-05 - val_loss: 5.8102e-05
+
    +
    +
    Epoch 72/150
+
+1/1 [==============================] - ETA: 0s - loss: 5.2129e-05
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 5.2129e-05 - val_loss: 7.2266e-06
+
    +
    +
    Epoch 73/150
+
+1/1 [==============================] - ETA: 0s - loss: 3.1149e-05
+
    +
    +
    
+1/1 [==============================] - 0s 20ms/step - loss: 3.1149e-05 - val_loss: 7.1946e-06
+
    +
    +
    Epoch 74/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 2.6803e-05
+
    +
    +
    
+1/1 [==============================] - 0s 20ms/step - loss: 2.6803e-05 - val_loss: 5.2427e-05
+
    +
    +
    Epoch 75/150
+
+1/1 [==============================] - ETA: 0s - loss: 3.7385e-05
+
    +
    +
    
+1/1 [==============================] - 0s 24ms/step - loss: 3.7385e-05 - val_loss: 1.0039e-04
+
    +
    +
    Epoch 76/150
+
+1/1 [==============================] - ETA: 0s - loss: 4.8700e-05
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 4.8700e-05 - val_loss: 1.1159e-04
+
    +
    +
    Epoch 77/150
+
+1/1 [==============================] - ETA: 0s - loss: 4.8312e-05
+
    +
    +
    
+1/1 [==============================] - 0s 20ms/step - loss: 4.8312e-05 - val_loss: 8.1357e-05
+
    +
    +
    Epoch 78/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 3.6325e-05
+
    +
    +
    
+1/1 [==============================] - 0s 21ms/step - loss: 3.6325e-05 - val_loss: 3.6670e-05
+
    +
    +
    Epoch 79/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 2.3281e-05
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 2.3281e-05 - val_loss: 6.9527e-06
+
    +
    +
    Epoch 80/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.9011e-05
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 1.9011e-05 - val_loss: 6.2134e-08
+
    +
    +
    Epoch 81/150
+
+1/1 [==============================] - ETA: 0s - loss: 2.4047e-05
+
    +
    +
    
+1/1 [==============================] - 0s 20ms/step - loss: 2.4047e-05 - val_loss: 3.3985e-06
+
    +
    +
    Epoch 82/150
+
+1/1 [==============================] - ETA: 0s - loss: 3.0829e-05
+
    +
    +
    
+1/1 [==============================] - 0s 25ms/step - loss: 3.0829e-05 - val_loss: 3.1615e-06
+
    +
    +
    Epoch 83/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 3.1812e-05
+
    +
    +
    
+1/1 [==============================] - 0s 20ms/step - loss: 3.1812e-05 - val_loss: 1.7273e-08
+
    +
    +
    Epoch 84/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 2.6124e-05
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 2.6124e-05 - val_loss: 6.8793e-06
+
    +
    +
    Epoch 85/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.9199e-05
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 1.9199e-05 - val_loss: 3.3150e-05
+
    +
    +
    Epoch 86/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.6832e-05
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 1.6832e-05 - val_loss: 7.2050e-05
+
    +
    +
    Epoch 87/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.9787e-05
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 1.9787e-05 - val_loss: 1.0364e-04
+
    +
    +
    Epoch 88/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 2.3840e-05
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 2.3840e-05 - val_loss: 1.1049e-04
+
    +
    +
    Epoch 89/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 2.4491e-05
+
    +
    +
    
+1/1 [==============================] - 0s 24ms/step - loss: 2.4491e-05 - val_loss: 9.1017e-05
+
    +
    +
    Epoch 90/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 2.1253e-05
+
    +
    +
    
+1/1 [==============================] - 0s 20ms/step - loss: 2.1253e-05 - val_loss: 5.8702e-05
+
    +
    +
    Epoch 91/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.7418e-05
+
    +
    +
    
+1/1 [==============================] - 0s 20ms/step - loss: 1.7418e-05 - val_loss: 2.9923e-05
+
    +
    +
    Epoch 92/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.6259e-05
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 1.6259e-05 - val_loss: 1.2810e-05
+
    +
    +
    Epoch 93/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.7905e-05
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 1.7905e-05 - val_loss: 6.0232e-06
+
    +
    +
    Epoch 94/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.9781e-05
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 1.9781e-05 - val_loss: 5.3134e-06
+
    +
    +
    Epoch 95/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.9548e-05
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 1.9548e-05 - val_loss: 9.3216e-06
+
    +
    +
    Epoch 96/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.7296e-05
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 1.7296e-05 - val_loss: 1.9159e-05
+
    +
    +
    Epoch 97/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.5052e-05
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 1.5052e-05 - val_loss: 3.3857e-05
+
    +
    +
    Epoch 98/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.4498e-05
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 1.4498e-05 - val_loss: 4.7879e-05
+
    +
    +
    Epoch 99/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.5386e-05
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 1.5386e-05 - val_loss: 5.4022e-05
+
    +
    +
    Epoch 100/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.6115e-05
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 1.6115e-05 - val_loss: 4.8949e-05
+
    +
    +
    Epoch 101/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.5549e-05
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 1.5549e-05 - val_loss: 3.5722e-05
+
    +
    +
    Epoch 102/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.4079e-05
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 1.4079e-05 - val_loss: 2.1059e-05
+
    +
    +
    Epoch 103/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.2963e-05
+
    +
    +
    
+1/1 [==============================] - 0s 20ms/step - loss: 1.2963e-05 - val_loss: 1.0307e-05
+
    +
    +
    Epoch 104/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.2911e-05
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 1.2911e-05 - val_loss: 4.7690e-06
+
    +
    +
    Epoch 105/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.3470e-05
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 1.3470e-05 - val_loss: 2.9656e-06
+
    +
    +
    Epoch 106/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.3678e-05
+
    +
    +
    
+1/1 [==============================] - 0s 20ms/step - loss: 1.3678e-05 - val_loss: 3.4672e-06
+
    +
    +
    Epoch 107/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.3126e-05
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 1.3126e-05 - val_loss: 6.0939e-06
+
    +
    +
    Epoch 108/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.2266e-05
+
    +
    +
    
+1/1 [==============================] - 0s 20ms/step - loss: 1.2266e-05 - val_loss: 1.0808e-05
+
    +
    +
    Epoch 109/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.1796e-05
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 1.1796e-05 - val_loss: 1.6232e-05
+
    +
    +
    Epoch 110/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.1902e-05
+
    +
    +
    
+1/1 [==============================] - 0s 22ms/step - loss: 1.1902e-05 - val_loss: 1.9843e-05
+
    +
    +
    Epoch 111/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.2143e-05
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 1.2143e-05 - val_loss: 1.9755e-05
+
    +
    +
    Epoch 112/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.2022e-05
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 1.2022e-05 - val_loss: 1.6197e-05
+
    +
    +
    Epoch 113/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.1506e-05
+
    +
    +
    
+1/1 [==============================] - 0s 20ms/step - loss: 1.1506e-05 - val_loss: 1.1183e-05
+
    +
    +
    Epoch 114/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.0982e-05
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 1.0982e-05 - val_loss: 6.8771e-06
+
    +
    +
    Epoch 115/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.0778e-05
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 1.0778e-05 - val_loss: 4.3037e-06
+
    +
    +
    Epoch 116/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.0826e-05
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 1.0826e-05 - val_loss: 3.4128e-06
+
    +
    +
    Epoch 117/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.0803e-05
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 1.0803e-05 - val_loss: 3.9190e-06
+
    +
    +
    Epoch 118/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.0525e-05
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 1.0525e-05 - val_loss: 5.7593e-06
+
    +
    +
    Epoch 119/150
+
+1/1 [==============================] - ETA: 0s - loss: 1.0114e-05
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 1.0114e-05 - val_loss: 8.7378e-06
+
    +
    +
    Epoch 120/150
+
+1/1 [==============================] - ETA: 0s - loss: 9.8203e-06
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 9.8203e-06 - val_loss: 1.2031e-05
+
    +
    +
    Epoch 121/150
+
+1/1 [==============================] - ETA: 0s - loss: 9.7284e-06
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 9.7284e-06 - val_loss: 1.4340e-05
+
    +
    +
    Epoch 122/150
+
+1/1 [==============================] - ETA: 0s - loss: 9.6966e-06
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 9.6966e-06 - val_loss: 1.4696e-05
+
    +
    +
    Epoch 123/150
+
+1/1 [==============================] - ETA: 0s - loss: 9.5506e-06
+
    +
    +
    
+1/1 [==============================] - 0s 20ms/step - loss: 9.5506e-06 - val_loss: 1.3124e-05
+
    +
    +
    Epoch 124/150
+
+1/1 [==============================] - ETA: 0s - loss: 9.2787e-06
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 9.2787e-06 - val_loss: 1.0547e-05
+
    +
    +
    Epoch 125/150
+
+1/1 [==============================] - ETA: 0s - loss: 9.0185e-06
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 9.0185e-06 - val_loss: 8.0814e-06
+
    +
    +
    Epoch 126/150
+
+1/1 [==============================] - ETA: 0s - loss: 8.8755e-06
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 8.8755e-06 - val_loss: 6.4479e-06
+
    +
    +
    Epoch 127/150
+
+1/1 [==============================] - ETA: 0s - loss: 8.8137e-06
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 8.8137e-06 - val_loss: 5.8674e-06
+
    +
    +
    Epoch 128/150
+
+1/1 [==============================] - ETA: 0s - loss: 8.7187e-06
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 8.7187e-06 - val_loss: 6.2909e-06
+
    +
    +
    Epoch 129/150
+
+1/1 [==============================] - ETA: 0s - loss: 8.5374e-06
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 8.5374e-06 - val_loss: 7.5345e-06
+
    +
    +
    Epoch 130/150
+
+1/1 [==============================] - ETA: 0s - loss: 8.3250e-06
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 8.3250e-06 - val_loss: 9.2095e-06
+
    +
    +
    Epoch 131/150
+
+1/1 [==============================] - ETA: 0s - loss: 8.1651e-06
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 8.1651e-06 - val_loss: 1.0688e-05
+
    +
    +
    Epoch 132/150
+
+1/1 [==============================] - ETA: 0s - loss: 8.0688e-06
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 8.0688e-06 - val_loss: 1.1327e-05
+
    +
    +
    Epoch 133/150
+
+1/1 [==============================] - ETA: 0s - loss: 7.9771e-06
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 7.9771e-06 - val_loss: 1.0835e-05
+
    +
    +
    Epoch 134/150
+
+1/1 [==============================] - ETA: 0s - loss: 7.8467e-06
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 7.8467e-06 - val_loss: 9.4450e-06
+
    +
    +
    Epoch 135/150
+
+1/1 [==============================] - ETA: 0s - loss: 7.6825e-06
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 7.6825e-06 - val_loss: 7.7252e-06
+
    +
    +
    Epoch 136/150
+
+1/1 [==============================] - ETA: 0s - loss: 7.5362e-06
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 7.5362e-06 - val_loss: 6.2324e-06
+
    +
    +
    Epoch 137/150
+
+1/1 [==============================] - ETA: 0s - loss: 7.4299e-06
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 7.4299e-06 - val_loss: 5.2841e-06
+
    +
    +
    Epoch 138/150
+
+1/1 [==============================] - ETA: 0s - loss: 7.3370e-06
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 7.3370e-06 - val_loss: 4.9452e-06
+
    +
    +
    Epoch 139/150
+
+1/1 [==============================] - ETA: 0s - loss: 7.2211e-06
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 7.2211e-06 - val_loss: 5.1307e-06
+
    +
    +
    Epoch 140/150
+
+1/1 [==============================] - ETA: 0s - loss: 7.0810e-06
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 7.0810e-06 - val_loss: 5.6509e-06
+
    +
    +
    Epoch 141/150
+
+1/1 [==============================] - ETA: 0s - loss: 6.9456e-06
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 6.9456e-06 - val_loss: 6.2259e-06
+
    +
    +
    Epoch 142/150
+
+1/1 [==============================] - ETA: 0s - loss: 6.8358e-06
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 6.8358e-06 - val_loss: 6.5445e-06
+
    +
    +
    Epoch 143/150
+
+1/1 [==============================] - ETA: 0s - loss: 6.7413e-06
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 6.7413e-06 - val_loss: 6.4017e-06
+
    +
    +
    Epoch 144/150
+
+1/1 [==============================] - ETA: 0s - loss: 6.6380e-06
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 6.6380e-06 - val_loss: 5.8084e-06
+
    +
    +
    Epoch 145/150
+
+1/1 [==============================] - ETA: 0s - loss: 6.5196e-06
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 6.5196e-06 - val_loss: 4.9715e-06
+
    +
    +
    Epoch 146/150
+
+1/1 [==============================] - ETA: 0s - loss: 6.4007e-06
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 6.4007e-06 - val_loss: 4.1612e-06
+
    +
    +
    Epoch 147/150
+
+1/1 [==============================] - ETA: 0s - loss: 6.2981e-06
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 6.2981e-06 - val_loss: 3.5793e-06
+
    +
    +
    Epoch 148/150
+
+1/1 [==============================] - ETA: 0s - loss: 6.2091e-06
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 6.2091e-06 - val_loss: 3.3095e-06
+
    +
    +
    Epoch 149/150
+
+1/1 [==============================] - ETA: 0s - loss: 6.1165e-06
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 6.1165e-06 - val_loss: 3.3423e-06
+
    +
    +
    Epoch 150/150
+
+1/1 [==============================] - ETA: 0s - loss: 6.0133e-06
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 6.0133e-06 - val_loss: 3.6012e-06
+
    +
    +_images/chapter10_11_326.png +
    MSE:  1.9124180119966666e-05
+
    +
    +_images/chapter10_11_328.png +
    Time:  9.464447478
+
    +
    +
    Model: "functional_7"
+_________________________________________________________________
+Layer (type)                 Output Shape              Param #   
+=================================================================
+input_4 (InputLayer)         [(None, 1, 1)]            0         
+_________________________________________________________________
+RNN (SimpleRNN)              (None, 200)               40400     
+_________________________________________________________________
+dense (Dense)                (None, 1)                 201       
+=================================================================
+Total params: 40,601
+Trainable params: 40,601
+Non-trainable params: 0
+_________________________________________________________________
+Epoch 1/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 1.4102
+
    +
    +
    
+1/1 [==============================] - 0s 154ms/step - loss: 1.4102 - val_loss: 2.1967
+
    +
    +
    Epoch 2/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.9243
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.9243 - val_loss: 1.4521
+
    +
    +
    Epoch 3/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.5465
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.5465 - val_loss: 0.8738
+
    +
    +
    Epoch 4/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.2756
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.2756 - val_loss: 0.4570
+
    +
    +
    Epoch 5/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.1066
+
    +
    +
    
+1/1 [==============================] - 0s 20ms/step - loss: 0.1066 - val_loss: 0.1890
+
    +
    +
    Epoch 6/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0284
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0284 - val_loss: 0.0477
+
    +
    +
    Epoch 7/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0229
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 0.0229 - val_loss: 0.0012
+
    +
    +
    Epoch 8/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0654
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0654 - val_loss: 0.0122
+
    +
    +
    Epoch 9/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.1283
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.1283 - val_loss: 0.0453
+
    +
    +
    Epoch 10/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.1870
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.1870 - val_loss: 0.0747
+
    +
    +
    Epoch 11/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.2252
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.2252 - val_loss: 0.0870
+
    +
    +
    Epoch 12/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.2363
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.2363 - val_loss: 0.0801
+
    +
    +
    Epoch 13/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.2218
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 0.2218 - val_loss: 0.0593
+
    +
    +
    Epoch 14/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.1884
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 0.1884 - val_loss: 0.0332
+
    +
    +
    Epoch 15/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.1445
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.1445 - val_loss: 0.0111
+
    +
    +
    Epoch 16/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0988
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0988 - val_loss: 3.2715e-04
+
    +
    +
    Epoch 17/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0585
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0585 - val_loss: 0.0055
+
    +
    +
    Epoch 18/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0286
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 0.0286 - val_loss: 0.0275
+
    +
    +
    Epoch 19/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0116
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0116 - val_loss: 0.0639
+
    +
    +
    Epoch 20/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0072
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0072 - val_loss: 0.1091
+
    +
    +
    Epoch 21/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0129
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 0.0129 - val_loss: 0.1561
+
    +
    +
    Epoch 22/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0249
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 0.0249 - val_loss: 0.1975
+
    +
    +
    Epoch 23/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0388
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0388 - val_loss: 0.2272
+
    +
    +
    Epoch 24/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0505
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0505 - val_loss: 0.2413
+
    +
    +
    Epoch 25/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0572
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0572 - val_loss: 0.2389
+
    +
    +
    Epoch 26/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0578
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0578 - val_loss: 0.2215
+
    +
    +
    Epoch 27/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0526
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0526 - val_loss: 0.1927
+
    +
    +
    Epoch 28/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0430
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0430 - val_loss: 0.1571
+
    +
    +
    Epoch 29/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0313
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0313 - val_loss: 0.1197
+
    +
    +
    Epoch 30/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0198
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0198 - val_loss: 0.0845
+
    +
    +
    Epoch 31/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0105
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0105 - val_loss: 0.0548
+
    +
    +
    Epoch 32/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0045
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0045 - val_loss: 0.0320
+
    +
    +
    Epoch 33/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0022
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0022 - val_loss: 0.0165
+
    +
    +
    Epoch 34/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0032
+
    +
    +
    
+1/1 [==============================] - 0s 22ms/step - loss: 0.0032 - val_loss: 0.0071
+
    +
    +
    Epoch 35/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0065
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0065 - val_loss: 0.0024
+
    +
    +
    Epoch 36/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0105
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 0.0105 - val_loss: 5.8779e-04
+
    +
    +
    Epoch 37/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0140
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0140 - val_loss: 9.0393e-05
+
    +
    +
    Epoch 38/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0161
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0161 - val_loss: 2.5875e-05
+
    +
    +
    Epoch 39/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0163
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0163 - val_loss: 1.0453e-04
+
    +
    +
    Epoch 40/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0146
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0146 - val_loss: 5.5861e-04
+
    +
    +
    Epoch 41/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0117
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 0.0117 - val_loss: 0.0019
+
    +
    +
    Epoch 42/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0083
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 0.0083 - val_loss: 0.0047
+
    +
    +
    Epoch 43/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0050
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0050 - val_loss: 0.0091
+
    +
    +
    Epoch 44/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0027
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0027 - val_loss: 0.0151
+
    +
    +
    Epoch 45/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0015
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0015 - val_loss: 0.0221
+
    +
    +
    Epoch 46/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0015
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0015 - val_loss: 0.0294
+
    +
    +
    Epoch 47/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0023
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0023 - val_loss: 0.0358
+
    +
    +
    Epoch 48/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0036
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0036 - val_loss: 0.0407
+
    +
    +
    Epoch 49/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0048
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0048 - val_loss: 0.0432
+
    +
    +
    Epoch 50/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0056
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0056 - val_loss: 0.0432
+
    +
    +
    Epoch 51/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0057
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0057 - val_loss: 0.0409
+
    +
    +
    Epoch 52/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0053
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0053 - val_loss: 0.0368
+
    +
    +
    Epoch 53/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0043
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0043 - val_loss: 0.0315
+
    +
    +
    Epoch 54/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0032
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0032 - val_loss: 0.0258
+
    +
    +
    Epoch 55/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0023
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0023 - val_loss: 0.0203
+
    +
    +
    Epoch 56/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0016
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 0.0016 - val_loss: 0.0156
+
    +
    +
    Epoch 57/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0013
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0013 - val_loss: 0.0118
+
    +
    +
    Epoch 58/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0014
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0014 - val_loss: 0.0089
+
    +
    +
    Epoch 59/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0017
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0017 - val_loss: 0.0071
+
    +
    +
    Epoch 60/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0022
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0022 - val_loss: 0.0060
+
    +
    +
    Epoch 61/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0025
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0025 - val_loss: 0.0055
+
    +
    +
    Epoch 62/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0026
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0026 - val_loss: 0.0057
+
    +
    +
    Epoch 63/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0026
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0026 - val_loss: 0.0064
+
    +
    +
    Epoch 64/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0023
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 0.0023 - val_loss: 0.0076
+
    +
    +
    Epoch 65/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0019
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0019 - val_loss: 0.0093
+
    +
    +
    Epoch 66/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0016
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 0.0016 - val_loss: 0.0113
+
    +
    +
    Epoch 67/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0013
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0013 - val_loss: 0.0136
+
    +
    +
    Epoch 68/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0012
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0012 - val_loss: 0.0160
+
    +
    +
    Epoch 69/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0012
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0012 - val_loss: 0.0181
+
    +
    +
    Epoch 70/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0013
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 0.0013 - val_loss: 0.0198
+
    +
    +
    Epoch 71/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0014
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0014 - val_loss: 0.0209
+
    +
    +
    Epoch 72/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0015
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0015 - val_loss: 0.0213
+
    +
    +
    Epoch 73/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0016
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 0.0016 - val_loss: 0.0210
+
    +
    +
    Epoch 74/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0015
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0015 - val_loss: 0.0202
+
    +
    +
    Epoch 75/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0014
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 0.0014 - val_loss: 0.0189
+
    +
    +
    Epoch 76/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0013
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0013 - val_loss: 0.0174
+
    +
    +
    Epoch 77/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0011
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0011 - val_loss: 0.0158
+
    +
    +
    Epoch 78/150
+
+1/1 [==============================] - ETA: 0s - loss: 0.0011
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0011 - val_loss: 0.0143
+
    +
    +
    Epoch 79/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0010
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0010 - val_loss: 0.0130
+
    +
    +
    Epoch 80/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0010
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0010 - val_loss: 0.0120
+
    +
    +
    Epoch 81/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0011
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0011 - val_loss: 0.0114
+
    +
    +
    Epoch 82/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0011
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0011 - val_loss: 0.0110
+
    +
    +
    Epoch 83/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0011
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0011 - val_loss: 0.0110
+
    +
    +
    Epoch 84/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0011
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0011 - val_loss: 0.0113
+
    +
    +
    Epoch 85/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0011
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0011 - val_loss: 0.0118
+
    +
    +
    Epoch 86/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 0.0011
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 0.0011 - val_loss: 0.0125
+
    +
    +
    Epoch 87/150
+
+1/1 [==============================] - ETA: 0s - loss: 9.9906e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 9.9906e-04 - val_loss: 0.0133
+
    +
    +
    Epoch 88/150
+
+1/1 [==============================] - ETA: 0s - loss: 9.5899e-04
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 9.5899e-04 - val_loss: 0.0142
+
    +
    +
    Epoch 89/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 9.3780e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 9.3780e-04 - val_loss: 0.0150
+
    +
    +
    Epoch 90/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 9.3488e-04
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 9.3488e-04 - val_loss: 0.0156
+
    +
    +
    Epoch 91/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 9.4357e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 9.4357e-04 - val_loss: 0.0161
+
    +
    +
    Epoch 92/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 9.5451e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 9.5451e-04 - val_loss: 0.0163
+
    +
    +
    Epoch 93/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 9.5936e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 9.5936e-04 - val_loss: 0.0163
+
    +
    +
    Epoch 94/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 9.5352e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 9.5352e-04 - val_loss: 0.0161
+
    +
    +
    Epoch 95/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 9.3717e-04
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 9.3717e-04 - val_loss: 0.0156
+
    +
    +
    Epoch 96/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 9.1432e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 9.1432e-04 - val_loss: 0.0150
+
    +
    +
    Epoch 97/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 8.9074e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 8.9074e-04 - val_loss: 0.0144
+
    +
    +
    Epoch 98/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 8.7159e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 8.7159e-04 - val_loss: 0.0138
+
    +
    +
    Epoch 99/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 8.5960e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 8.5960e-04 - val_loss: 0.0132
+
    +
    +
    Epoch 100/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 8.5449e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 8.5449e-04 - val_loss: 0.0127
+
    +
    +
    Epoch 101/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 8.5362e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 8.5362e-04 - val_loss: 0.0124
+
    +
    +
    Epoch 102/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 8.5331e-04
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 8.5331e-04 - val_loss: 0.0122
+
    +
    +
    Epoch 103/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 8.5041e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 8.5041e-04 - val_loss: 0.0121
+
    +
    +
    Epoch 104/150
+
+1/1 [==============================] - ETA: 0s - loss: 8.4335e-04
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 8.4335e-04 - val_loss: 0.0121
+
    +
    +
    Epoch 105/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 8.3246e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 8.3246e-04 - val_loss: 0.0122
+
    +
    +
    Epoch 106/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 8.1954e-04
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 8.1954e-04 - val_loss: 0.0124
+
    +
    +
    Epoch 107/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 8.0688e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 8.0688e-04 - val_loss: 0.0127
+
    +
    +
    Epoch 108/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 7.9636e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 7.9636e-04 - val_loss: 0.0129
+
    +
    +
    Epoch 109/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 7.8874e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 7.8874e-04 - val_loss: 0.0131
+
    +
    +
    Epoch 110/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 7.8361e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 7.8361e-04 - val_loss: 0.0132
+
    +
    +
    Epoch 111/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 7.7972e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 7.7972e-04 - val_loss: 0.0133
+
    +
    +
    Epoch 112/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 7.7562e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 7.7562e-04 - val_loss: 0.0132
+
    +
    +
    Epoch 113/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 7.7023e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 7.7023e-04 - val_loss: 0.0131
+
    +
    +
    Epoch 114/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 7.6324e-04
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 7.6324e-04 - val_loss: 0.0129
+
    +
    +
    Epoch 115/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 7.5506e-04
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 7.5506e-04 - val_loss: 0.0126
+
    +
    +
    Epoch 116/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 7.4655e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 7.4655e-04 - val_loss: 0.0123
+
    +
    +
    Epoch 117/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 7.3860e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 7.3860e-04 - val_loss: 0.0121
+
    +
    +
    Epoch 118/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 7.3174e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 7.3174e-04 - val_loss: 0.0118
+
    +
    +
    Epoch 119/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 7.2605e-04
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 7.2605e-04 - val_loss: 0.0115
+
    +
    +
    Epoch 120/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 7.2114e-04
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 7.2114e-04 - val_loss: 0.0114
+
    +
    +
    Epoch 121/150
+
+1/1 [==============================] - ETA: 0s - loss: 7.1643e-04
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 7.1643e-04 - val_loss: 0.0112
+
    +
    +
    Epoch 122/150
+
+1/1 [==============================] - ETA: 0s - loss: 7.1139e-04
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 7.1139e-04 - val_loss: 0.0111
+
    +
    +
    Epoch 123/150
+
+1/1 [==============================] - ETA: 0s - loss: 7.0577e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 7.0577e-04 - val_loss: 0.0111
+
    +
    +
    Epoch 124/150
+
+1/1 [==============================] - ETA: 0s - loss: 6.9964e-04
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 6.9964e-04 - val_loss: 0.0111
+
    +
    +
    Epoch 125/150
+
+1/1 [==============================] - ETA: 0s - loss: 6.9330e-04
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 6.9330e-04 - val_loss: 0.0112
+
    +
    +
    Epoch 126/150
+
+1/1 [==============================] - ETA: 0s - loss: 6.8710e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 6.8710e-04 - val_loss: 0.0112
+
    +
    +
    Epoch 127/150
+
+1/1 [==============================] - ETA: 0s - loss: 6.8134e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 6.8134e-04 - val_loss: 0.0112
+
    +
    +
    Epoch 128/150
+
+1/1 [==============================] - ETA: 0s - loss: 6.7611e-04
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 6.7611e-04 - val_loss: 0.0113
+
    +
    +
    Epoch 129/150
+
+1/1 [==============================] - ETA: 0s - loss: 6.7125e-04
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 6.7125e-04 - val_loss: 0.0113
+
    +
    +
    Epoch 130/150
+
+1/1 [==============================] - ETA: 0s - loss: 6.6659e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 6.6659e-04 - val_loss: 0.0112
+
    +
    +
    Epoch 131/150
+
+1/1 [==============================] - ETA: 0s - loss: 6.6186e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 6.6186e-04 - val_loss: 0.0112
+
    +
    +
    Epoch 132/150
+
+1/1 [==============================] - ETA: 0s - loss: 6.5694e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 6.5694e-04 - val_loss: 0.0111
+
    +
    +
    Epoch 133/150
+
+1/1 [==============================] - ETA: 0s - loss: 6.5184e-04
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 6.5184e-04 - val_loss: 0.0110
+
    +
    +
    Epoch 134/150
+
+1/1 [==============================] - ETA: 0s - loss: 6.4666e-04
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 6.4666e-04 - val_loss: 0.0108
+
    +
    +
    Epoch 135/150
+
+1/1 [==============================] - ETA: 0s - loss: 6.4156e-04
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 6.4156e-04 - val_loss: 0.0107
+
    +
    +
    Epoch 136/150
+
+1/1 [==============================] - ETA: 0s - loss: 6.3666e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 6.3666e-04 - val_loss: 0.0105
+
    +
    +
    Epoch 137/150
+
+1/1 [==============================] - ETA: 0s - loss: 6.3201e-04
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 6.3201e-04 - val_loss: 0.0104
+
    +
    +
    Epoch 138/150
+
+1/1 [==============================] - ETA: 0s - loss: 6.2756e-04
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 6.2756e-04 - val_loss: 0.0103
+
    +
    +
    Epoch 139/150
+
+1/1 [==============================] - ETA: 0s - loss: 6.2321e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 6.2321e-04 - val_loss: 0.0102
+
    +
    +
    Epoch 140/150
+
+1/1 [==============================] - ETA: 0s - loss: 6.1888e-04
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 6.1888e-04 - val_loss: 0.0102
+
    +
    +
    Epoch 141/150
+
+1/1 [==============================] - ETA: 0s - loss: 6.1449e-04
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 6.1449e-04 - val_loss: 0.0101
+
    +
    +
    Epoch 142/150
+
+1/1 [==============================] - ETA: 0s - loss: 6.1007e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 6.1007e-04 - val_loss: 0.0101
+
    +
    +
    Epoch 143/150
+
+1/1 [==============================] - ETA: 0s - loss: 6.0565e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 6.0565e-04 - val_loss: 0.0101
+
    +
    +
    Epoch 144/150
+
+1/1 [==============================] - ETA: 0s - loss: 6.0129e-04
+
    +
    +
    
+1/1 [==============================] - 0s 18ms/step - loss: 6.0129e-04 - val_loss: 0.0101
+
    +
    +
    Epoch 145/150
+
+1/1 [==============================] - ETA: 0s - loss: 5.9706e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 5.9706e-04 - val_loss: 0.0100
+
    +
    +
    Epoch 146/150
+
+1/1 [==============================] - ETA: 0s - loss: 5.9295e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 5.9295e-04 - val_loss: 0.0100
+
    +
    +
    Epoch 147/150
+
+1/1 [==============================] - ETA: 0s - loss: 5.8894e-04
+
    +
    +
    
+1/1 [==============================] - 0s 16ms/step - loss: 5.8894e-04 - val_loss: 0.0100
+
    +
    +
    Epoch 148/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 5.8501e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 5.8501e-04 - val_loss: 0.0100
+
    +
    +
    Epoch 149/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 5.8111e-04
+
    +
    +
    
+1/1 [==============================] - 0s 17ms/step - loss: 5.8111e-04 - val_loss: 0.0099
+
    +
    +
    Epoch 150/150
+
    +
    +
    1/1 [==============================] - ETA: 0s - loss: 5.7722e-04
+
    +
    +
    
+1/1 [==============================] - 0s 19ms/step - loss: 5.7722e-04 - val_loss: 0.0098
+
    +
    +_images/chapter10_11_687.png +_images/chapter10_11_688.png +
    Time:  4.337630193999999
+
    +
    +
    +
    +
    +
    +
    +

    10. Solving ODEs with Deep Learning

    +

    The Universal Approximation Theorem states that a neural network can +approximate any function at a single hidden layer along with one input +and output layer to any given precision.

    +
    +

    10.1. Ordinary Differential Equations

    +

    An ordinary differential equation (ODE) is an equation involving functions having one variable.

    +

    In general, an ordinary differential equation looks like

    + +
    +
    +\[ +\begin{equation} \label{ode} \tag{1} +f\left(x, \, g(x), \, g'(x), \, g''(x), \, \dots \, , \, g^{(n)}(x)\right) = 0 +\end{equation} +\]
    +

    where \(g(x)\) is the function to find, and \(g^{(n)}(x)\) is the \(n\)-th derivative of \(g(x)\).

    +

    The \(f\left(x, g(x), g'(x), g''(x), \, \dots \, , g^{(n)}(x)\right)\) is just a way to write that there is an expression involving \(x\) and \(g(x), \ g'(x), \ g''(x), \, \dots \, , \text{ and } g^{(n)}(x)\) on the left side of the equality sign in (1). +The highest order of derivative, that is the value of \(n\), determines to the order of the equation. +The equation is referred to as a \(n\)-th order ODE. +Along with (1), some additional conditions of the function \(g(x)\) are typically given +for the solution to be unique.

    +
    +
    +

    10.2. The trial solution

    +

    Let the trial solution \(g_t(x)\) be

    + +
    +
    +\[ +\begin{equation} + g_t(x) = h_1(x) + h_2(x,N(x,P)) +\label{_auto1} \tag{2} +\end{equation} +\]
    +

    where \(h_1(x)\) is a function that makes \(g_t(x)\) satisfy a given set +of conditions, \(N(x,P)\) a neural network with weights and biases +described by \(P\) and \(h_2(x, N(x,P))\) some expression involving the +neural network. The role of the function \(h_2(x, N(x,P))\), is to +ensure that the output from \(N(x,P)\) is zero when \(g_t(x)\) is +evaluated at the values of \(x\) where the given conditions must be +satisfied. The function \(h_1(x)\) should alone make \(g_t(x)\) satisfy +the conditions.

    +

    But what about the network \(N(x,P)\)?

    +

    As described previously, an optimization method could be used to minimize the parameters of a neural network, that being its weights and biases, through backward propagation.

    +
    +
    +

    10.3. Minimization process

    +

    For the minimization to be defined, we need to have a cost function at hand to minimize.

    +

    It is given that \(f\left(x, \, g(x), \, g'(x), \, g''(x), \, \dots \, , \, g^{(n)}(x)\right)\) should be equal to zero in (1). +We can choose to consider the mean squared error as the cost function for an input \(x\). +Since we are looking at one input, the cost function is just \(f\) squared. +The cost function \(c\left(x, P \right)\) can therefore be expressed as

    +
    +\[ +C\left(x, P\right) = \big(f\left(x, \, g(x), \, g'(x), \, g''(x), \, \dots \, , \, g^{(n)}(x)\right)\big)^2 +\]
    +

    If \(N\) inputs are given as a vector \(\boldsymbol{x}\) with elements \(x_i\) for \(i = 1,\dots,N\), +the cost function becomes

    + +
    +
    +\[ +\begin{equation} \label{cost} \tag{3} + C\left(\boldsymbol{x}, P\right) = \frac{1}{N} \sum_{i=1}^N \big(f\left(x_i, \, g(x_i), \, g'(x_i), \, g''(x_i), \, \dots \, , \, g^{(n)}(x_i)\right)\big)^2 +\end{equation} +\]
    +

    The neural net should then find the parameters \(P\) that minimizes the cost function in +(3) for a set of \(N\) training samples \(x_i\).

    +
    +
    +

    10.4. Minimizing the cost function using gradient descent and automatic differentiation

    +

    To perform the minimization using gradient descent, the gradient of \(C\left(\boldsymbol{x}, P\right)\) is needed. +It might happen so that finding an analytical expression of the gradient of \(C(\boldsymbol{x}, P)\) from (3) gets too messy, depending on which cost function one desires to use.

    +

    Luckily, there exists libraries that makes the job for us through automatic differentiation. +Automatic differentiation is a method of finding the derivatives numerically with very high precision.

    +
    +
    +

    10.5. Example: Exponential decay

    +

    An exponential decay of a quantity \(g(x)\) is described by the equation

    + +
    +
    +\[ +\begin{equation} \label{solve_expdec} \tag{4} + g'(x) = -\gamma g(x) +\end{equation} +\]
    +

    with \(g(0) = g_0\) for some chosen initial value \(g_0\).

    +

    The analytical solution of (4) is

    + +
    +
    +\[ +\begin{equation} + g(x) = g_0 \exp\left(-\gamma x\right) +\label{_auto2} \tag{5} +\end{equation} +\]
    +

    Having an analytical solution at hand, it is possible to use it to compare how well a neural network finds a solution of (4).

    +
    +
    +

    10.6. The function to solve for

    +

    The program will use a neural network to solve

    + +
    +
    +\[ +\begin{equation} \label{solveode} \tag{6} +g'(x) = -\gamma g(x) +\end{equation} +\]
    +

    where \(g(0) = g_0\) with \(\gamma\) and \(g_0\) being some chosen values.

    +

    In this example, \(\gamma = 2\) and \(g_0 = 10\).

    +
    +
    +

    10.7. The trial solution

    +

    To begin with, a trial solution \(g_t(t)\) must be chosen. A general trial solution for ordinary differential equations could be

    +
    +\[ +g_t(x, P) = h_1(x) + h_2(x, N(x, P)) +\]
    +

    with \(h_1(x)\) ensuring that \(g_t(x)\) satisfies some conditions and \(h_2(x,N(x, P))\) an expression involving \(x\) and the output from the neural network \(N(x,P)\) with \(P \) being the collection of the weights and biases for each layer. For now, it is assumed that the network consists of one input layer, one hidden layer, and one output layer.

    +
    +
    +

    10.8. Setup of Network

    +

    In this network, there are no weights and bias at the input layer, so \(P = \{ P_{\text{hidden}}, P_{\text{output}} \}\). +If there are \(N_{\text{hidden} }\) neurons in the hidden layer, then \(P_{\text{hidden}}\) is a \(N_{\text{hidden} } \times (1 + N_{\text{input}})\) matrix, given that there are \(N_{\text{input}}\) neurons in the input layer.

    +

    The first column in \(P_{\text{hidden} }\) represents the bias for each neuron in the hidden layer and the second column represents the weights for each neuron in the hidden layer from the input layer. +If there are \(N_{\text{output} }\) neurons in the output layer, then \(P_{\text{output}} \) is a \(N_{\text{output} } \times (1 + N_{\text{hidden} })\) matrix.

    +

    Its first column represents the bias of each neuron and the remaining columns represents the weights to each neuron.

    +

    It is given that \(g(0) = g_0\). The trial solution must fulfill this condition to be a proper solution of (6). A possible way to ensure that \(g_t(0, P) = g_0\), is to let \(F(N(x,P)) = x \cdot N(x,P)\) and \(A(x) = g_0\). This gives the following trial solution:

    + +
    +
    +\[ +\begin{equation} \label{trial} \tag{7} +g_t(x, P) = g_0 + x \cdot N(x, P) +\end{equation} +\]
    +
    +
    +

    10.9. Reformulating the problem

    +

    We wish that our neural network manages to minimize a given cost function.

    +

    A reformulation of out equation, (6), must therefore be done, +such that it describes the problem a neural network can solve for.

    +

    The neural network must find the set of weights and biases \(P\) such that the trial solution in (7) satisfies (6).

    +

    The trial solution

    +
    +\[ +g_t(x, P) = g_0 + x \cdot N(x, P) +\]
    +

    has been chosen such that it already solves the condition \(g(0) = g_0\). What remains, is to find \(P\) such that

    + +
    +
    +\[ +\begin{equation} \label{nnmin} \tag{8} +g_t'(x, P) = - \gamma g_t(x, P) +\end{equation} +\]
    +

    is fulfilled as best as possible.

    +
    +
    +

    10.10. More technicalities

    +

    The left hand side and right hand side of (8) must be computed separately, and then the neural network must choose weights and biases, contained in \(P\), such that the sides are equal as best as possible. +This means that the absolute or squared difference between the sides must be as close to zero, ideally equal to zero. +In this case, the difference squared shows to be an appropriate measurement of how erroneous the trial solution is with respect to \(P\) of the neural network.

    +

    This gives the following cost function our neural network must solve for:

    +
    +\[ +\min_{P}\Big\{ \big(g_t'(x, P) - ( -\gamma g_t(x, P) \big)^2 \Big\} +\]
    +

    (the notation \(\min_{P}\{ f(x, P) \}\) means that we desire to find \(P\) that yields the minimum of \(f(x, P)\))

    +

    or, in terms of weights and biases for the hidden and output layer in our network:

    +
    +\[ +\min_{P_{\text{hidden} }, \ P_{\text{output} }}\Big\{ \big(g_t'(x, \{ P_{\text{hidden} }, P_{\text{output} }\}) - ( -\gamma g_t(x, \{ P_{\text{hidden} }, P_{\text{output} }\}) \big)^2 \Big\} +\]
    +

    for an input value \(x\).

    +
    +
    +

    10.11. More details

    +

    If the neural network evaluates \(g_t(x, P)\) at more values for \(x\), say \(N\) values \(x_i\) for \(i = 1, \dots, N\), then the total error to minimize becomes

    + +
    +
    +\[ +\begin{equation} \label{min} \tag{9} +\min_{P}\Big\{\frac{1}{N} \sum_{i=1}^N \big(g_t'(x_i, P) - ( -\gamma g_t(x_i, P) \big)^2 \Big\} +\end{equation} +\]
    +

    Letting \(\boldsymbol{x}\) be a vector with elements \(x_i\) and \(C(\boldsymbol{x}, P) = \frac{1}{N} \sum_i \big(g_t'(x_i, P) - ( -\gamma g_t(x_i, P) \big)^2\) denote the cost function, the minimization problem that our network must solve, becomes

    +
    +\[ +\min_{P} C(\boldsymbol{x}, P) +\]
    +

    In terms of \(P_{\text{hidden} }\) and \(P_{\text{output} }\), this could also be expressed as

    +
    +\[ +\min_{P_{\text{hidden} }, \ P_{\text{output} }} C(\boldsymbol{x}, \{P_{\text{hidden} }, P_{\text{output} }\}) +\]
    +
    +
    +

    10.12. A possible implementation of a neural network

    +

    For simplicity, it is assumed that the input is an array \(\boldsymbol{x} = (x_1, \dots, x_N)\) with \(N\) elements. It is at these points the neural network should find \(P\) such that it fulfills (9).

    +

    First, the neural network must feed forward the inputs. +This means that \(\boldsymbol{x}s\) must be passed through an input layer, a hidden layer and a output layer. The input layer in this case, does not need to process the data any further. +The input layer will consist of \(N_{\text{input} }\) neurons, passing its element to each neuron in the hidden layer. The number of neurons in the hidden layer will be \(N_{\text{hidden} }\).

    +
    +
    +

    10.13. Technicalities

    +

    For the \(i\)-th in the hidden layer with weight \(w_i^{\text{hidden} }\) and bias \(b_i^{\text{hidden} }\), the weighting from the \(j\)-th neuron at the input layer is:

    +
    +\[\begin{split} +\begin{aligned} +z_{i,j}^{\text{hidden}} &= b_i^{\text{hidden}} + w_i^{\text{hidden}}x_j \\ +&= +\begin{pmatrix} +b_i^{\text{hidden}} & w_i^{\text{hidden}} +\end{pmatrix} +\begin{pmatrix} +1 \\ +x_j +\end{pmatrix} +\end{aligned} +\end{split}\]
    +
    +
    +

    10.14. Final technicalities I

    +

    The result after weighting the inputs at the \(i\)-th hidden neuron can be written as a vector:

    +
    +\[\begin{split} +\begin{aligned} +\boldsymbol{z}_{i}^{\text{hidden}} &= \Big( b_i^{\text{hidden}} + w_i^{\text{hidden}}x_1 , \ b_i^{\text{hidden}} + w_i^{\text{hidden}} x_2, \ \dots \, , \ b_i^{\text{hidden}} + w_i^{\text{hidden}} x_N\Big) \\ +&= +\begin{pmatrix} + b_i^{\text{hidden}} & w_i^{\text{hidden}} +\end{pmatrix} +\begin{pmatrix} +1 & 1 & \dots & 1 \\ +x_1 & x_2 & \dots & x_N +\end{pmatrix} \\ +&= \boldsymbol{p}_{i, \text{hidden}}^T X +\end{aligned} +\end{split}\]
    +
    +
    +

    10.15. Final technicalities II

    +

    The vector \(\boldsymbol{p}_{i, \text{hidden}}^T\) constitutes each row in \(P_{\text{hidden} }\), which contains the weights for the neural network to minimize according to (9).

    +

    After having found \(\boldsymbol{z}_{i}^{\text{hidden}} \) for every \(i\)-th neuron within the hidden layer, the vector will be sent to an activation function \(a_i(\boldsymbol{z})\).

    +

    In this example, the sigmoid function has been chosen to be the activation function for each hidden neuron:

    +
    +\[ +f(z) = \frac{1}{1 + \exp{(-z)}} +\]
    +

    It is possible to use other activations functions for the hidden layer also.

    +

    The output \(\boldsymbol{x}_i^{\text{hidden}}\) from each \(i\)-th hidden neuron is:

    +
    +\[ +\boldsymbol{x}_i^{\text{hidden} } = f\big( \boldsymbol{z}_{i}^{\text{hidden}} \big) +\]
    +

    The outputs \(\boldsymbol{x}_i^{\text{hidden} } \) are then sent to the output layer.

    +

    The output layer consists of one neuron in this case, and combines the +output from each of the neurons in the hidden layers. The output layer +combines the results from the hidden layer using some weights \(w_i^{\text{output}}\) +and biases \(b_i^{\text{output}}\). In this case, +it is assumes that the number of neurons in the output layer is one.

    +
    +
    +

    10.16. Final technicalities III

    +

    The procedure of weighting the output neuron \(j\) in the hidden layer to the \(i\)-th neuron in the output layer is similar as for the hidden layer described previously.

    +
    +\[\begin{split} +\begin{aligned} +z_{1,j}^{\text{output}} & = +\begin{pmatrix} +b_1^{\text{output}} & \boldsymbol{w}_1^{\text{output}} +\end{pmatrix} +\begin{pmatrix} +1 \\ +\boldsymbol{x}_j^{\text{hidden}} +\end{pmatrix} +\end{aligned} +\end{split}\]
    +
    +
    +

    10.17. Final technicalities IV

    +

    Expressing \(z_{1,j}^{\text{output}}\) as a vector gives the following way of weighting the inputs from the hidden layer:

    +
    +\[\begin{split} +\boldsymbol{z}_{1}^{\text{output}} = +\begin{pmatrix} +b_1^{\text{output}} & \boldsymbol{w}_1^{\text{output}} +\end{pmatrix} +\begin{pmatrix} +1 & 1 & \dots & 1 \\ +\boldsymbol{x}_1^{\text{hidden}} & \boldsymbol{x}_2^{\text{hidden}} & \dots & \boldsymbol{x}_N^{\text{hidden}} +\end{pmatrix} +\end{split}\]
    +

    In this case we seek a continuous range of values since we are approximating a function. This means that after computing \(\boldsymbol{z}_{1}^{\text{output}}\) the neural network has finished its feed forward step, and \(\boldsymbol{z}_{1}^{\text{output}}\) is the final output of the network.

    +
    +
    +

    10.18. Back propagation

    +

    The next step is to decide how the parameters should be changed such that they minimize the cost function.

    +

    The chosen cost function for this problem is

    +
    +\[ +C(\boldsymbol{x}, P) = \frac{1}{N} \sum_i \big(g_t'(x_i, P) - ( -\gamma g_t(x_i, P) \big)^2 +\]
    +

    In order to minimize the cost function, an optimization method must be chosen.

    +

    Here, gradient descent with a constant step size has been chosen.

    +
    +
    +

    10.19. Gradient descent

    +

    The idea of the gradient descent algorithm is to update parameters in +a direction where the cost function decreases goes to a minimum.

    +

    In general, the update of some parameters \(\boldsymbol{\omega}\) given a cost +function defined by some weights \(\boldsymbol{\omega}\), \(C(\boldsymbol{x}, +\boldsymbol{\omega})\), goes as follows:

    +
    +\[ +\boldsymbol{\omega}_{\text{new} } = \boldsymbol{\omega} - \lambda \nabla_{\boldsymbol{\omega}} C(\boldsymbol{x}, \boldsymbol{\omega}) +\]
    +

    for a number of iterations or until \( \big|\big| \boldsymbol{\omega}_{\text{new} } - \boldsymbol{\omega} \big|\big|\) becomes smaller than some given tolerance.

    +

    The value of \(\lambda\) decides how large steps the algorithm must take +in the direction of \( \nabla_{\boldsymbol{\omega}} C(\boldsymbol{x}, \boldsymbol{\omega})\). +The notation \(\nabla_{\boldsymbol{\omega}}\) express the gradient with respect +to the elements in \(\boldsymbol{\omega}\).

    +

    In our case, we have to minimize the cost function \(C(\boldsymbol{x}, P)\) with +respect to the two sets of weights and biases, that is for the hidden +layer \(P_{\text{hidden} }\) and for the output layer \(P_{\text{output} +}\) .

    +

    This means that \(P_{\text{hidden} }\) and \(P_{\text{output} }\) is updated by

    +
    +\[\begin{split} +\begin{aligned} +P_{\text{hidden},\text{new}} &= P_{\text{hidden}} - \lambda \nabla_{P_{\text{hidden}}} C(\boldsymbol{x}, P) \\ +P_{\text{output},\text{new}} &= P_{\text{output}} - \lambda \nabla_{P_{\text{output}}} C(\boldsymbol{x}, P) +\end{aligned} +\end{split}\]
    +
    +
    +

    10.20. The code for solving the ODE

    +
    +
    +
    import autograd.numpy as np
+from autograd import grad, elementwise_grad
+import autograd.numpy.random as npr
+from matplotlib import pyplot as plt
+
+def sigmoid(z):
+    return 1/(1 + np.exp(-z))
+
+# Assuming one input, hidden, and output layer
+def neural_network(params, x):
+
+    # Find the weights (including and biases) for the hidden and output layer.
+    # Assume that params is a list of parameters for each layer.
+    # The biases are the first element for each array in params,
+    # and the weights are the remaning elements in each array in params.
+
+    w_hidden = params[0]
+    w_output = params[1]
+
+    # Assumes input x being an one-dimensional array
+    num_values = np.size(x)
+    x = x.reshape(-1, num_values)
+
+    # Assume that the input layer does nothing to the input x
+    x_input = x
+
+    ## Hidden layer:
+
+    # Add a row of ones to include bias
+    x_input = np.concatenate((np.ones((1,num_values)), x_input ), axis = 0)
+
+    z_hidden = np.matmul(w_hidden, x_input)
+    x_hidden = sigmoid(z_hidden)
+
+    ## Output layer:
+
+    # Include bias:
+    x_hidden = np.concatenate((np.ones((1,num_values)), x_hidden ), axis = 0)
+
+    z_output = np.matmul(w_output, x_hidden)
+    x_output = z_output
+
+    return x_output
+
+# The trial solution using the deep neural network:
+def g_trial(x,params, g0 = 10):
+    return g0 + x*neural_network(params,x)
+
+# The right side of the ODE:
+def g(x, g_trial, gamma = 2):
+    return -gamma*g_trial
+
+# The cost function:
+def cost_function(P, x):
+
+    # Evaluate the trial function with the current parameters P
+    g_t = g_trial(x,P)
+
+    # Find the derivative w.r.t x of the neural network
+    d_net_out = elementwise_grad(neural_network,1)(P,x)
+
+    # Find the derivative w.r.t x of the trial function
+    d_g_t = elementwise_grad(g_trial,0)(x,P)
+
+    # The right side of the ODE
+    func = g(x, g_t)
+
+    err_sqr = (d_g_t - func)**2
+    cost_sum = np.sum(err_sqr)
+
+    return cost_sum / np.size(err_sqr)
+
+# Solve the exponential decay ODE using neural network with one input, hidden, and output layer
+def solve_ode_neural_network(x, num_neurons_hidden, num_iter, lmb):
+    ## Set up initial weights and biases
+
+    # For the hidden layer
+    p0 = npr.randn(num_neurons_hidden, 2 )
+
+    # For the output layer
+    p1 = npr.randn(1, num_neurons_hidden + 1 ) # +1 since bias is included
+
+    P = [p0, p1]
+
+    print('Initial cost: %g'%cost_function(P, x))
+
+    ## Start finding the optimal weights using gradient descent
+
+    # Find the Python function that represents the gradient of the cost function
+    # w.r.t the 0-th input argument -- that is the weights and biases in the hidden and output layer
+    cost_function_grad = grad(cost_function,0)
+
+    # Let the update be done num_iter times
+    for i in range(num_iter):
+        # Evaluate the gradient at the current weights and biases in P.
+        # The cost_grad consist now of two arrays;
+        # one for the gradient w.r.t P_hidden and
+        # one for the gradient w.r.t P_output
+        cost_grad =  cost_function_grad(P, x)
+
+        P[0] = P[0] - lmb * cost_grad[0]
+        P[1] = P[1] - lmb * cost_grad[1]
+
+    print('Final cost: %g'%cost_function(P, x))
+
+    return P
+
+def g_analytic(x, gamma = 2, g0 = 10):
+    return g0*np.exp(-gamma*x)
+
+# Solve the given problem
+if __name__ == '__main__':
+    # Set seed such that the weight are initialized
+    # with same weights and biases for every run.
+    npr.seed(15)
+
+    ## Decide the vales of arguments to the function to solve
+    N = 10
+    x = np.linspace(0, 1, N)
+
+    ## Set up the initial parameters
+    num_hidden_neurons = 10
+    num_iter = 10000
+    lmb = 0.001
+
+    # Use the network
+    P = solve_ode_neural_network(x, num_hidden_neurons, num_iter, lmb)
+
+    # Print the deviation from the trial solution and true solution
+    res = g_trial(x,P)
+    res_analytical = g_analytic(x)
+
+    print('Max absolute difference: %g'%np.max(np.abs(res - res_analytical)))
+
+    # Plot the results
+    plt.figure(figsize=(10,10))
+
+    plt.title('Performance of neural network solving an ODE compared to the analytical solution')
+    plt.plot(x, res_analytical)
+    plt.plot(x, res[0,:])
+    plt.legend(['analytical','nn'])
+    plt.xlabel('x')
+    plt.ylabel('g(x)')
+    plt.show()
+
    +
    +
    +
    +
    Initial cost: 367.01
+
    +
    +
    Final cost: 0.0666807
+Max absolute difference: 0.0437499
+
    +
    +_images/chapter10_59_2.png +
    +
    +
    +
    +

    10.21. The network with one input layer, specified number of hidden layers, and one output layer

    +

    It is also possible to extend the construction of our network into a more general one, allowing the network to contain more than one hidden layers.

    +

    The number of neurons within each hidden layer are given as a list of integers in the program below.

    +
    +
    +
    import autograd.numpy as np
+from autograd import grad, elementwise_grad
+import autograd.numpy.random as npr
+from matplotlib import pyplot as plt
+
+def sigmoid(z):
+    return 1/(1 + np.exp(-z))
+
+# The neural network with one input layer and one output layer,
+# but with number of hidden layers specified by the user.
+def deep_neural_network(deep_params, x):
+    # N_hidden is the number of hidden layers
+
+    N_hidden = np.size(deep_params) - 1 # -1 since params consists of
+                                        # parameters to all the hidden
+                                        # layers AND the output layer.
+
+    # Assumes input x being an one-dimensional array
+    num_values = np.size(x)
+    x = x.reshape(-1, num_values)
+
+    # Assume that the input layer does nothing to the input x
+    x_input = x
+
+    # Due to multiple hidden layers, define a variable referencing to the
+    # output of the previous layer:
+    x_prev = x_input
+
+    ## Hidden layers:
+
+    for l in range(N_hidden):
+        # From the list of parameters P; find the correct weigths and bias for this layer
+        w_hidden = deep_params[l]
+
+        # Add a row of ones to include bias
+        x_prev = np.concatenate((np.ones((1,num_values)), x_prev ), axis = 0)
+
+        z_hidden = np.matmul(w_hidden, x_prev)
+        x_hidden = sigmoid(z_hidden)
+
+        # Update x_prev such that next layer can use the output from this layer
+        x_prev = x_hidden
+
+    ## Output layer:
+
+    # Get the weights and bias for this layer
+    w_output = deep_params[-1]
+
+    # Include bias:
+    x_prev = np.concatenate((np.ones((1,num_values)), x_prev), axis = 0)
+
+    z_output = np.matmul(w_output, x_prev)
+    x_output = z_output
+
+    return x_output
+
+# The trial solution using the deep neural network:
+def g_trial_deep(x,params, g0 = 10):
+    return g0 + x*deep_neural_network(params, x)
+
+# The right side of the ODE:
+def g(x, g_trial, gamma = 2):
+    return -gamma*g_trial
+
+# The same cost function as before, but calls deep_neural_network instead.
+def cost_function_deep(P, x):
+
+    # Evaluate the trial function with the current parameters P
+    g_t = g_trial_deep(x,P)
+
+    # Find the derivative w.r.t x of the neural network
+    d_net_out = elementwise_grad(deep_neural_network,1)(P,x)
+
+    # Find the derivative w.r.t x of the trial function
+    d_g_t = elementwise_grad(g_trial_deep,0)(x,P)
+
+    # The right side of the ODE
+    func = g(x, g_t)
+
+    err_sqr = (d_g_t - func)**2
+    cost_sum = np.sum(err_sqr)
+
+    return cost_sum / np.size(err_sqr)
+
+# Solve the exponential decay ODE using neural network with one input and one output layer,
+# but with specified number of hidden layers from the user.
+def solve_ode_deep_neural_network(x, num_neurons, num_iter, lmb):
+    # num_hidden_neurons is now a list of number of neurons within each hidden layer
+
+    # The number of elements in the list num_hidden_neurons thus represents
+    # the number of hidden layers.
+
+    # Find the number of hidden layers:
+    N_hidden = np.size(num_neurons)
+
+    ## Set up initial weights and biases
+
+    # Initialize the list of parameters:
+    P = [None]*(N_hidden + 1) # + 1 to include the output layer
+
+    P[0] = npr.randn(num_neurons[0], 2 )
+    for l in range(1,N_hidden):
+        P[l] = npr.randn(num_neurons[l], num_neurons[l-1] + 1) # +1 to include bias
+
+    # For the output layer
+    P[-1] = npr.randn(1, num_neurons[-1] + 1 ) # +1 since bias is included
+
+    print('Initial cost: %g'%cost_function_deep(P, x))
+
+    ## Start finding the optimal weights using gradient descent
+
+    # Find the Python function that represents the gradient of the cost function
+    # w.r.t the 0-th input argument -- that is the weights and biases in the hidden and output layer
+    cost_function_deep_grad = grad(cost_function_deep,0)
+
+    # Let the update be done num_iter times
+    for i in range(num_iter):
+        # Evaluate the gradient at the current weights and biases in P.
+        # The cost_grad consist now of N_hidden + 1 arrays; the gradient w.r.t the weights and biases
+        # in the hidden layers and output layers evaluated at x.
+        cost_deep_grad =  cost_function_deep_grad(P, x)
+
+        for l in range(N_hidden+1):
+            P[l] = P[l] - lmb * cost_deep_grad[l]
+
+    print('Final cost: %g'%cost_function_deep(P, x))
+
+    return P
+
+def g_analytic(x, gamma = 2, g0 = 10):
+    return g0*np.exp(-gamma*x)
+
+# Solve the given problem
+if __name__ == '__main__':
+    npr.seed(15)
+
+    ## Decide the vales of arguments to the function to solve
+    N = 10
+    x = np.linspace(0, 1, N)
+
+    ## Set up the initial parameters
+    num_hidden_neurons = np.array([10,10])
+    num_iter = 10000
+    lmb = 0.001
+
+    P = solve_ode_deep_neural_network(x, num_hidden_neurons, num_iter, lmb)
+
+    res = g_trial_deep(x,P)
+    res_analytical = g_analytic(x)
+
+    plt.figure(figsize=(10,10))
+
+    plt.title('Performance of a deep neural network solving an ODE compared to the analytical solution')
+    plt.plot(x, res_analytical)
+    plt.plot(x, res[0,:])
+    plt.legend(['analytical','dnn'])
+    plt.ylabel('g(x)')
+    plt.show()
+
    +
    +
    +
    +
    +
    +

    10.22. Example: Population growth

    +

    A logistic model of population growth assumes that a population converges toward an equilibrium. +The population growth can be modeled by

    + +
    +
    +\[ +\begin{equation} \label{log} \tag{10} + g'(t) = \alpha g(t)(A - g(t)) +\end{equation} +\]
    +

    where \(g(t)\) is the population density at time \(t\), \(\alpha > 0\) the growth rate and \(A > 0\) is the maximum population number in the environment. +Also, at \(t = 0\) the population has the size \(g(0) = g_0\), where \(g_0\) is some chosen constant.

    +

    In this example, similar network as for the exponential decay using Autograd has been used to solve the equation. However, as the implementation might suffer from e.g numerical instability +and high execution time (this might be more apparent in the examples solving PDEs), +using a library like TensorFlow is recommended. +Here, we stay with a more simple approach and implement for comparison, the simple forward Euler method.

    +
    +
    +

    10.23. Setting up the problem

    +

    Here, we will model a population \(g(t)\) in an environment having carrying capacity \(A\). +The population follows the model

    + +
    +
    +\[ +\begin{equation} \label{solveode_population} \tag{11} +g'(t) = \alpha g(t)(A - g(t)) +\end{equation} +\]
    +

    where \(g(0) = g_0\).

    +

    In this example, we let \(\alpha = 2\), \(A = 1\), and \(g_0 = 1.2\).

    +
    +
    +

    10.24. The trial solution

    +

    We will get a slightly different trial solution, as the boundary conditions are different +compared to the case for exponential decay.

    +

    A possible trial solution satisfying the condition \(g(0) = g_0\) could be

    +
    +\[ +h_1(t) = g_0 + t \cdot N(t,P) +\]
    +

    with \(N(t,P)\) being the output from the neural network with weights and biases for each layer collected in the set \(P\).

    +

    The analytical solution is

    +
    +\[ +g(t) = \frac{Ag_0}{g_0 + (A - g_0)\exp(-\alpha A t)} +\]
    +
    +
    +

    10.25. The program using Autograd

    +

    The network will be the similar as for the exponential decay example, but with some small modifications for our problem.

    +
    +
    +
    import autograd.numpy as np
+from autograd import grad, elementwise_grad
+import autograd.numpy.random as npr
+from matplotlib import pyplot as plt
+
+def sigmoid(z):
+    return 1/(1 + np.exp(-z))
+
+# Function to get the parameters.
+# Done such that one can easily change the paramaters after one's liking.
+def get_parameters():
+    alpha = 2
+    A = 1
+    g0 = 1.2
+    return alpha, A, g0
+
+def deep_neural_network(P, x):
+    # N_hidden is the number of hidden layers
+    N_hidden = np.size(P) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer
+
+    # Assumes input x being an one-dimensional array
+    num_values = np.size(x)
+    x = x.reshape(-1, num_values)
+
+    # Assume that the input layer does nothing to the input x
+    x_input = x
+
+    # Due to multiple hidden layers, define a variable referencing to the
+    # output of the previous layer:
+    x_prev = x_input
+
+    ## Hidden layers:
+
+    for l in range(N_hidden):
+        # From the list of parameters P; find the correct weigths and bias for this layer
+        w_hidden = P[l]
+
+        # Add a row of ones to include bias
+        x_prev = np.concatenate((np.ones((1,num_values)), x_prev ), axis = 0)
+
+        z_hidden = np.matmul(w_hidden, x_prev)
+        x_hidden = sigmoid(z_hidden)
+
+        # Update x_prev such that next layer can use the output from this layer
+        x_prev = x_hidden
+
+    ## Output layer:
+
+    # Get the weights and bias for this layer
+    w_output = P[-1]
+
+    # Include bias:
+    x_prev = np.concatenate((np.ones((1,num_values)), x_prev), axis = 0)
+
+    z_output = np.matmul(w_output, x_prev)
+    x_output = z_output
+
+    return x_output
+
+
+def cost_function_deep(P, x):
+
+    # Evaluate the trial function with the current parameters P
+    g_t = g_trial_deep(x,P)
+
+    # Find the derivative w.r.t x of the trial function
+    d_g_t = elementwise_grad(g_trial_deep,0)(x,P)
+
+    # The right side of the ODE
+    func = f(x, g_t)
+
+    err_sqr = (d_g_t - func)**2
+    cost_sum = np.sum(err_sqr)
+
+    return cost_sum / np.size(err_sqr)
+
+# The right side of the ODE:
+def f(x, g_trial):
+    alpha,A, g0 = get_parameters()
+    return alpha*g_trial*(A - g_trial)
+
+# The trial solution using the deep neural network:
+def g_trial_deep(x, params):
+    alpha,A, g0 = get_parameters()
+    return g0 + x*deep_neural_network(params,x)
+
+# The analytical solution:
+def g_analytic(t):
+    alpha,A, g0 = get_parameters()
+    return A*g0/(g0 + (A - g0)*np.exp(-alpha*A*t))
+
+def solve_ode_deep_neural_network(x, num_neurons, num_iter, lmb):
+    # num_hidden_neurons is now a list of number of neurons within each hidden layer
+
+    # Find the number of hidden layers:
+    N_hidden = np.size(num_neurons)
+
+    ## Set up initial weigths and biases
+
+    # Initialize the list of parameters:
+    P = [None]*(N_hidden + 1) # + 1 to include the output layer
+
+    P[0] = npr.randn(num_neurons[0], 2 )
+    for l in range(1,N_hidden):
+        P[l] = npr.randn(num_neurons[l], num_neurons[l-1] + 1) # +1 to include bias
+
+    # For the output layer
+    P[-1] = npr.randn(1, num_neurons[-1] + 1 ) # +1 since bias is included
+
+    print('Initial cost: %g'%cost_function_deep(P, x))
+
+    ## Start finding the optimal weigths using gradient descent
+
+    # Find the Python function that represents the gradient of the cost function
+    # w.r.t the 0-th input argument -- that is the weights and biases in the hidden and output layer
+    cost_function_deep_grad = grad(cost_function_deep,0)
+
+    # Let the update be done num_iter times
+    for i in range(num_iter):
+        # Evaluate the gradient at the current weights and biases in P.
+        # The cost_grad consist now of N_hidden + 1 arrays; the gradient w.r.t the weights and biases
+        # in the hidden layers and output layers evaluated at x.
+        cost_deep_grad =  cost_function_deep_grad(P, x)
+
+        for l in range(N_hidden+1):
+            P[l] = P[l] - lmb * cost_deep_grad[l]
+
+    print('Final cost: %g'%cost_function_deep(P, x))
+
+    return P
+
+if __name__ == '__main__':
+    npr.seed(4155)
+
+    ## Decide the vales of arguments to the function to solve
+    Nt = 10
+    T = 1
+    t = np.linspace(0,T, Nt)
+
+    ## Set up the initial parameters
+    num_hidden_neurons = [100, 50, 25]
+    num_iter = 1000
+    lmb = 1e-3
+
+    P = solve_ode_deep_neural_network(t, num_hidden_neurons, num_iter, lmb)
+
+    g_dnn_ag = g_trial_deep(t,P)
+    g_analytical = g_analytic(t)
+
+    # Find the maximum absolute difference between the solutons:
+    diff_ag = np.max(np.abs(g_dnn_ag - g_analytical))
+    print("The max absolute difference between the solutions is: %g"%diff_ag)
+
+    plt.figure(figsize=(10,10))
+
+    plt.title('Performance of neural network solving an ODE compared to the analytical solution')
+    plt.plot(t, g_analytical)
+    plt.plot(t, g_dnn_ag[0,:])
+    plt.legend(['analytical','nn'])
+    plt.xlabel('t')
+    plt.ylabel('g(t)')
+
+    plt.show()
+
    +
    +
    +
    +
    +
    +

    10.26. Using forward Euler to solve the ODE

    +

    A straightforward way of solving an ODE numerically, is to use Euler’s method.

    +

    Euler’s method uses Taylor series to approximate the value at a function \(f\) at a step \(\Delta x\) from \(x\):

    +
    +\[ +f(x + \Delta x) \approx f(x) + \Delta x f'(x) +\]
    +

    In our case, using Euler’s method to approximate the value of \(g\) at a step \(\Delta t\) from \(t\) yields

    +
    +\[\begin{split} +\begin{aligned} + g(t + \Delta t) &\approx g(t) + \Delta t g'(t) \\ + &= g(t) + \Delta t \big(\alpha g(t)(A - g(t))\big) +\end{aligned} +\end{split}\]
    +

    along with the condition that \(g(0) = g_0\).

    +

    Let \(t_i = i \cdot \Delta t\) where \(\Delta t = \frac{T}{N_t-1}\) where \(T\) is the final time our solver must solve for and \(N_t\) the number of values for \(t \in [0, T]\) for \(i = 0, \dots, N_t-1\).

    +

    For \(i \geq 1\), we have that

    +
    +\[\begin{split} +\begin{aligned} +t_i &= i\Delta t \\ +&= (i - 1)\Delta t + \Delta t \\ +&= t_{i-1} + \Delta t +\end{aligned} +\end{split}\]
    +

    Now, if \(g_i = g(t_i)\) then

    + +
    +
    +\[\begin{split} +\begin{equation} + \begin{aligned} + g_i &= g(t_i) \\ + &= g(t_{i-1} + \Delta t) \\ + &\approx g(t_{i-1}) + \Delta t \big(\alpha g(t_{i-1})(A - g(t_{i-1}))\big) \\ + &= g_{i-1} + \Delta t \big(\alpha g_{i-1}(A - g_{i-1})\big) + \end{aligned} +\end{equation} \label{odenum} \tag{12} +\end{split}\]
    +

    for \(i \geq 1\) and \(g_0 = g(t_0) = g(0) = g_0\).

    +

    Equation (12) could be implemented in the following way, +extending the program that uses the network using Autograd:

    +
    +
    +
    # Assume that all function definitions from the example program using Autograd
+# are located here.
+
+if __name__ == '__main__':
+    npr.seed(4155)
+
+    ## Decide the vales of arguments to the function to solve
+    Nt = 10
+    T = 1
+    t = np.linspace(0,T, Nt)
+
+    ## Set up the initial parameters
+    num_hidden_neurons = [100,50,25]
+    num_iter = 1000
+    lmb = 1e-3
+
+    P = solve_ode_deep_neural_network(t, num_hidden_neurons, num_iter, lmb)
+
+    g_dnn_ag = g_trial_deep(t,P)
+    g_analytical = g_analytic(t)
+
+    # Find the maximum absolute difference between the solutons:
+    diff_ag = np.max(np.abs(g_dnn_ag - g_analytical))
+    print("The max absolute difference between the solutions is: %g"%diff_ag)
+
+    plt.figure(figsize=(10,10))
+
+    plt.title('Performance of neural network solving an ODE compared to the analytical solution')
+    plt.plot(t, g_analytical)
+    plt.plot(t, g_dnn_ag[0,:])
+    plt.legend(['analytical','nn'])
+    plt.xlabel('t')
+    plt.ylabel('g(t)')
+
+    ## Find an approximation to the funtion using forward Euler
+
+    alpha, A, g0 = get_parameters()
+    dt = T/(Nt - 1)
+
+    # Perform forward Euler to solve the ODE
+    g_euler = np.zeros(Nt)
+    g_euler[0] = g0
+
+    for i in range(1,Nt):
+        g_euler[i] = g_euler[i-1] + dt*(alpha*g_euler[i-1]*(A - g_euler[i-1]))
+
+    # Print the errors done by each method
+    diff1 = np.max(np.abs(g_euler - g_analytical))
+    diff2 = np.max(np.abs(g_dnn_ag[0,:] - g_analytical))
+
+    print('Max absolute difference between Euler method and analytical: %g'%diff1)
+    print('Max absolute difference between deep neural network and analytical: %g'%diff2)
+
+    # Plot results
+    plt.figure(figsize=(10,10))
+
+    plt.plot(t,g_euler)
+    plt.plot(t,g_analytical)
+    plt.plot(t,g_dnn_ag[0,:])
+
+    plt.legend(['euler','analytical','dnn'])
+    plt.xlabel('Time t')
+    plt.ylabel('g(t)')
+
+    plt.show()
+
    +
    +
    +
    +
    +
    +

    10.27. Example: Solving the one dimensional Poisson equation

    +

    The Poisson equation for \(g(x)\) in one dimension is

    + +
    +
    +\[ +\begin{equation} \label{poisson} \tag{13} + -g''(x) = f(x) +\end{equation} +\]
    +

    where \(f(x)\) is a given function for \(x \in (0,1)\).

    +

    The conditions that \(g(x)\) is chosen to fulfill, are

    +
    +\[\begin{split} +\begin{align*} + g(0) &= 0 \\ + g(1) &= 0 +\end{align*} +\end{split}\]
    +

    This equation can be solved numerically using programs where e.g Autograd and TensorFlow are used. +The results from the networks can then be compared to the analytical solution. +In addition, it could be interesting to see how a typical method for numerically solving second order ODEs compares to the neural networks.

    +
    +
    +

    10.28. The specific equation to solve for

    +

    Here, the function \(g(x)\) to solve for follows the equation

    +
    +\[ +-g''(x) = f(x),\qquad x \in (0,1) +\]
    +

    where \(f(x)\) is a given function, along with the chosen conditions

    + +
    +
    +\[ +\begin{aligned} +g(0) = g(1) = 0 +\end{aligned}\label{cond} \tag{14} +\]
    +

    In this example, we consider the case when \(f(x) = (3x + x^2)\exp(x)\).

    +

    For this case, a possible trial solution satisfying the conditions could be

    +
    +\[ +g_t(x) = x \cdot (1-x) \cdot N(P,x) +\]
    +

    The analytical solution for this problem is

    +
    +\[ +g(x) = x(1 - x)\exp(x) +\]
    +
    +
    +

    10.29. Solving the equation using Autograd

    +
    +
    +
    import autograd.numpy as np
+from autograd import grad, elementwise_grad
+import autograd.numpy.random as npr
+from matplotlib import pyplot as plt
+
+def sigmoid(z):
+    return 1/(1 + np.exp(-z))
+
+def deep_neural_network(deep_params, x):
+    # N_hidden is the number of hidden layers
+    N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer
+
+    # Assumes input x being an one-dimensional array
+    num_values = np.size(x)
+    x = x.reshape(-1, num_values)
+
+    # Assume that the input layer does nothing to the input x
+    x_input = x
+
+    # Due to multiple hidden layers, define a variable referencing to the
+    # output of the previous layer:
+    x_prev = x_input
+
+    ## Hidden layers:
+
+    for l in range(N_hidden):
+        # From the list of parameters P; find the correct weigths and bias for this layer
+        w_hidden = deep_params[l]
+
+        # Add a row of ones to include bias
+        x_prev = np.concatenate((np.ones((1,num_values)), x_prev ), axis = 0)
+
+        z_hidden = np.matmul(w_hidden, x_prev)
+        x_hidden = sigmoid(z_hidden)
+
+        # Update x_prev such that next layer can use the output from this layer
+        x_prev = x_hidden
+
+    ## Output layer:
+
+    # Get the weights and bias for this layer
+    w_output = deep_params[-1]
+
+    # Include bias:
+    x_prev = np.concatenate((np.ones((1,num_values)), x_prev), axis = 0)
+
+    z_output = np.matmul(w_output, x_prev)
+    x_output = z_output
+
+    return x_output
+
+def solve_ode_deep_neural_network(x, num_neurons, num_iter, lmb):
+    # num_hidden_neurons is now a list of number of neurons within each hidden layer
+
+    # Find the number of hidden layers:
+    N_hidden = np.size(num_neurons)
+
+    ## Set up initial weigths and biases
+
+    # Initialize the list of parameters:
+    P = [None]*(N_hidden + 1) # + 1 to include the output layer
+
+    P[0] = npr.randn(num_neurons[0], 2 )
+    for l in range(1,N_hidden):
+        P[l] = npr.randn(num_neurons[l], num_neurons[l-1] + 1) # +1 to include bias
+
+    # For the output layer
+    P[-1] = npr.randn(1, num_neurons[-1] + 1 ) # +1 since bias is included
+
+    print('Initial cost: %g'%cost_function_deep(P, x))
+
+    ## Start finding the optimal weigths using gradient descent
+
+    # Find the Python function that represents the gradient of the cost function
+    # w.r.t the 0-th input argument -- that is the weights and biases in the hidden and output layer
+    cost_function_deep_grad = grad(cost_function_deep,0)
+
+    # Let the update be done num_iter times
+    for i in range(num_iter):
+        # Evaluate the gradient at the current weights and biases in P.
+        # The cost_grad consist now of N_hidden + 1 arrays; the gradient w.r.t the weights and biases
+        # in the hidden layers and output layers evaluated at x.
+        cost_deep_grad =  cost_function_deep_grad(P, x)
+
+        for l in range(N_hidden+1):
+            P[l] = P[l] - lmb * cost_deep_grad[l]
+
+    print('Final cost: %g'%cost_function_deep(P, x))
+
+    return P
+
+## Set up the cost function specified for this Poisson equation:
+
+# The right side of the ODE
+def f(x):
+    return (3*x + x**2)*np.exp(x)
+
+def cost_function_deep(P, x):
+
+    # Evaluate the trial function with the current parameters P
+    g_t = g_trial_deep(x,P)
+
+    # Find the derivative w.r.t x of the trial function
+    d2_g_t = elementwise_grad(elementwise_grad(g_trial_deep,0))(x,P)
+
+    right_side = f(x)
+
+    err_sqr = (-d2_g_t - right_side)**2
+    cost_sum = np.sum(err_sqr)
+
+    return cost_sum/np.size(err_sqr)
+
+# The trial solution:
+def g_trial_deep(x,P):
+    return x*(1-x)*deep_neural_network(P,x)
+
+# The analytic solution;
+def g_analytic(x):
+    return x*(1-x)*np.exp(x)
+
+if __name__ == '__main__':
+    npr.seed(4155)
+
+    ## Decide the vales of arguments to the function to solve
+    Nx = 10
+    x = np.linspace(0,1, Nx)
+
+    ## Set up the initial parameters
+    num_hidden_neurons = [200,100]
+    num_iter = 1000
+    lmb = 1e-3
+
+    P = solve_ode_deep_neural_network(x, num_hidden_neurons, num_iter, lmb)
+
+    g_dnn_ag = g_trial_deep(x,P)
+    g_analytical = g_analytic(x)
+
+    # Find the maximum absolute difference between the solutons:
+    max_diff = np.max(np.abs(g_dnn_ag - g_analytical))
+    print("The max absolute difference between the solutions is: %g"%max_diff)
+
+    plt.figure(figsize=(10,10))
+
+    plt.title('Performance of neural network solving an ODE compared to the analytical solution')
+    plt.plot(x, g_analytical)
+    plt.plot(x, g_dnn_ag[0,:])
+    plt.legend(['analytical','nn'])
+    plt.xlabel('x')
+    plt.ylabel('g(x)')
+    plt.show()
+
    +
    +
    +
    +
    +
    +

    10.30. Comparing with a numerical scheme

    +

    The Poisson equation is possible to solve using Taylor series to approximate the second derivative.

    +

    Using Taylor series, the second derivative can be expressed as

    +
    +\[ +g''(x) = \frac{g(x + \Delta x) - 2g(x) + g(x-\Delta x)}{\Delta x^2} + E_{\Delta x}(x) +\]
    +

    where \(\Delta x\) is a small step size and \(E_{\Delta x}(x)\) being the error term.

    +

    Looking away from the error terms gives an approximation to the second derivative:

    + +
    +
    +\[ +\begin{equation} \label{approx} \tag{15} +g''(x) \approx \frac{g(x + \Delta x) - 2g(x) + g(x-\Delta x)}{\Delta x^2} +\end{equation} +\]
    +

    If \(x_i = i \Delta x = x_{i-1} + \Delta x\) and \(g_i = g(x_i)\) for \(i = 1,\dots N_x - 2\) with \(N_x\) being the number of values for \(x\), (15) becomes

    +
    +\[\begin{split} +\begin{aligned} +g''(x_i) &\approx \frac{g(x_i + \Delta x) - 2g(x_i) + g(x_i -\Delta x)}{\Delta x^2} \\ +&= \frac{g_{i+1} - 2g_i + g_{i-1}}{\Delta x^2} +\end{aligned} +\end{split}\]
    +

    Since we know from our problem that

    +
    +\[\begin{split} +\begin{aligned} +-g''(x) &= f(x) \\ +&= (3x + x^2)\exp(x) +\end{aligned} +\end{split}\]
    +

    along with the conditions \(g(0) = g(1) = 0\), +the following scheme can be used to find an approximate solution for \(g(x)\) numerically:

    + +
    +
    +\[\begin{split} +\begin{equation} + \begin{aligned} + -\Big( \frac{g_{i+1} - 2g_i + g_{i-1}}{\Delta x^2} \Big) &= f(x_i) \\ + -g_{i+1} + 2g_i - g_{i-1} &= \Delta x^2 f(x_i) + \end{aligned} +\end{equation} \label{odesys} \tag{16} +\end{split}\]
    +

    for \(i = 1, \dots, N_x - 2\) where \(g_0 = g_{N_x - 1} = 0\) and \(f(x_i) = (3x_i + x_i^2)\exp(x_i)\), which is given for our specific problem.

    +

    The equation can be rewritten into a matrix equation:

    +
    +\[\begin{split} +\begin{aligned} +\begin{pmatrix} +2 & -1 & 0 & \dots & 0 \\ +-1 & 2 & -1 & \dots & 0 \\ +\vdots & & \ddots & & \vdots \\ +0 & \dots & -1 & 2 & -1 \\ +0 & \dots & 0 & -1 & 2\\ +\end{pmatrix} +\begin{pmatrix} +g_1 \\ +g_2 \\ +\vdots \\ +g_{N_x - 3} \\ +g_{N_x - 2} +\end{pmatrix} +&= +\Delta x^2 +\begin{pmatrix} +f(x_1) \\ +f(x_2) \\ +\vdots \\ +f(x_{N_x - 3}) \\ +f(x_{N_x - 2}) +\end{pmatrix} \\ +\boldsymbol{A}\boldsymbol{g} &= \boldsymbol{f}, +\end{aligned} +\end{split}\]
    +

    which makes it possible to solve for the vector \(\boldsymbol{g}\).

    +
    +
    +

    10.31. Setting up the code

    +

    We can then compare the result from this numerical scheme with the output from our network using Autograd:

    +
    +
    +
    import autograd.numpy as np
+from autograd import grad, elementwise_grad
+import autograd.numpy.random as npr
+from matplotlib import pyplot as plt
+
+def sigmoid(z):
+    return 1/(1 + np.exp(-z))
+
+def deep_neural_network(deep_params, x):
+    # N_hidden is the number of hidden layers
+    N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer
+
+    # Assumes input x being an one-dimensional array
+    num_values = np.size(x)
+    x = x.reshape(-1, num_values)
+
+    # Assume that the input layer does nothing to the input x
+    x_input = x
+
+    # Due to multiple hidden layers, define a variable referencing to the
+    # output of the previous layer:
+    x_prev = x_input
+
+    ## Hidden layers:
+
+    for l in range(N_hidden):
+        # From the list of parameters P; find the correct weigths and bias for this layer
+        w_hidden = deep_params[l]
+
+        # Add a row of ones to include bias
+        x_prev = np.concatenate((np.ones((1,num_values)), x_prev ), axis = 0)
+
+        z_hidden = np.matmul(w_hidden, x_prev)
+        x_hidden = sigmoid(z_hidden)
+
+        # Update x_prev such that next layer can use the output from this layer
+        x_prev = x_hidden
+
+    ## Output layer:
+
+    # Get the weights and bias for this layer
+    w_output = deep_params[-1]
+
+    # Include bias:
+    x_prev = np.concatenate((np.ones((1,num_values)), x_prev), axis = 0)
+
+    z_output = np.matmul(w_output, x_prev)
+    x_output = z_output
+
+    return x_output
+
+def solve_ode_deep_neural_network(x, num_neurons, num_iter, lmb):
+    # num_hidden_neurons is now a list of number of neurons within each hidden layer
+
+    # Find the number of hidden layers:
+    N_hidden = np.size(num_neurons)
+
+    ## Set up initial weigths and biases
+
+    # Initialize the list of parameters:
+    P = [None]*(N_hidden + 1) # + 1 to include the output layer
+
+    P[0] = npr.randn(num_neurons[0], 2 )
+    for l in range(1,N_hidden):
+        P[l] = npr.randn(num_neurons[l], num_neurons[l-1] + 1) # +1 to include bias
+
+    # For the output layer
+    P[-1] = npr.randn(1, num_neurons[-1] + 1 ) # +1 since bias is included
+
+    print('Initial cost: %g'%cost_function_deep(P, x))
+
+    ## Start finding the optimal weigths using gradient descent
+
+    # Find the Python function that represents the gradient of the cost function
+    # w.r.t the 0-th input argument -- that is the weights and biases in the hidden and output layer
+    cost_function_deep_grad = grad(cost_function_deep,0)
+
+    # Let the update be done num_iter times
+    for i in range(num_iter):
+        # Evaluate the gradient at the current weights and biases in P.
+        # The cost_grad consist now of N_hidden + 1 arrays; the gradient w.r.t the weights and biases
+        # in the hidden layers and output layers evaluated at x.
+        cost_deep_grad =  cost_function_deep_grad(P, x)
+
+        for l in range(N_hidden+1):
+            P[l] = P[l] - lmb * cost_deep_grad[l]
+
+    print('Final cost: %g'%cost_function_deep(P, x))
+
+    return P
+
+## Set up the cost function specified for this Poisson equation:
+
+# The right side of the ODE
+def f(x):
+    return (3*x + x**2)*np.exp(x)
+
+def cost_function_deep(P, x):
+
+    # Evaluate the trial function with the current parameters P
+    g_t = g_trial_deep(x,P)
+
+    # Find the derivative w.r.t x of the trial function
+    d2_g_t = elementwise_grad(elementwise_grad(g_trial_deep,0))(x,P)
+
+    right_side = f(x)
+
+    err_sqr = (-d2_g_t - right_side)**2
+    cost_sum = np.sum(err_sqr)
+
+    return cost_sum/np.size(err_sqr)
+
+# The trial solution:
+def g_trial_deep(x,P):
+    return x*(1-x)*deep_neural_network(P,x)
+
+# The analytic solution;
+def g_analytic(x):
+    return x*(1-x)*np.exp(x)
+
+if __name__ == '__main__':
+    npr.seed(4155)
+
+    ## Decide the vales of arguments to the function to solve
+    Nx = 10
+    x = np.linspace(0,1, Nx)
+
+    ## Set up the initial parameters
+    num_hidden_neurons = [200,100]
+    num_iter = 1000
+    lmb = 1e-3
+
+    P = solve_ode_deep_neural_network(x, num_hidden_neurons, num_iter, lmb)
+
+    g_dnn_ag = g_trial_deep(x,P)
+    g_analytical = g_analytic(x)
+
+    # Find the maximum absolute difference between the solutons:
+
+    plt.figure(figsize=(10,10))
+
+    plt.title('Performance of neural network solving an ODE compared to the analytical solution')
+    plt.plot(x, g_analytical)
+    plt.plot(x, g_dnn_ag[0,:])
+    plt.legend(['analytical','nn'])
+    plt.xlabel('x')
+    plt.ylabel('g(x)')
+
+    ## Perform the computation using the numerical scheme
+
+    dx = 1/(Nx - 1)
+
+    # Set up the matrix A
+    A = np.zeros((Nx-2,Nx-2))
+
+    A[0,0] = 2
+    A[0,1] = -1
+
+    for i in range(1,Nx-3):
+        A[i,i-1] = -1
+        A[i,i] = 2
+        A[i,i+1] = -1
+
+    A[Nx - 3, Nx - 4] = -1
+    A[Nx - 3, Nx - 3] = 2
+
+    # Set up the vector f
+    f_vec = dx**2 * f(x[1:-1])
+
+    # Solve the equation
+    g_res = np.linalg.solve(A,f_vec)
+
+    g_vec = np.zeros(Nx)
+    g_vec[1:-1] = g_res
+
+    # Print the differences between each method
+    max_diff1 = np.max(np.abs(g_dnn_ag - g_analytical))
+    max_diff2 = np.max(np.abs(g_vec - g_analytical))
+    print("The max absolute difference between the analytical solution and DNN Autograd: %g"%max_diff1)
+    print("The max absolute difference between the analytical solution and numerical scheme: %g"%max_diff2)
+
+    # Plot the results
+    plt.figure(figsize=(10,10))
+
+    plt.plot(x,g_vec)
+    plt.plot(x,g_analytical)
+    plt.plot(x,g_dnn_ag[0,:])
+
+    plt.legend(['numerical scheme','analytical','dnn'])
+    plt.show()
+
    +
    +
    +
    +
    +
    +

    10.32. Partial Differential Equations

    +

    A partial differential equation (PDE) has a solution here the function +is defined by multiple variables. The equation may involve all kinds +of combinations of which variables the function is differentiated with +respect to.

    +

    In general, a partial differential equation for a function \(g(x_1,\dots,x_N)\) with \(N\) variables may be expressed as

    + +
    +
    +\[ +\begin{equation} \label{PDE} \tag{17} + f\left(x_1, \, \dots \, , x_N, \frac{\partial g(x_1,\dots,x_N) }{\partial x_1}, \dots , \frac{\partial g(x_1,\dots,x_N) }{\partial x_N}, \frac{\partial g(x_1,\dots,x_N) }{\partial x_1\partial x_2}, \, \dots \, , \frac{\partial^n g(x_1,\dots,x_N) }{\partial x_N^n} \right) = 0 +\end{equation} +\]
    +

    where \(f\) is an expression involving all kinds of possible mixed derivatives of \(g(x_1,\dots,x_N)\) up to an order \(n\). In order for the solution to be unique, some additional conditions must also be given.

    +
    +
    +

    10.33. Type of problem

    +

    The problem our network must solve for, is similar to the ODE case. +We must have a trial solution \(g_t\) at hand.

    +

    For instance, the trial solution could be expressed as

    +
    +\[ +\begin{align*} + g_t(x_1,\dots,x_N) = h_1(x_1,\dots,x_N) + h_2(x_1,\dots,x_N,N(x_1,\dots,x_N,P)) +\end{align*} +\]
    +

    where \(h_1(x_1,\dots,x_N)\) is a function that ensures \(g_t(x_1,\dots,x_N)\) satisfies some given conditions. +The neural network \(N(x_1,\dots,x_N,P)\) has weights and biases described by \(P\) and \(h_2(x_1,\dots,x_N,N(x_1,\dots,x_N,P))\) is an expression using the output from the neural network in some way.

    +

    The role of the function \(h_2(x_1,\dots,x_N,N(x_1,\dots,x_N,P))\), is to ensure that the output of \(N(x_1,\dots,x_N,P)\) is zero when \(g_t(x_1,\dots,x_N)\) is evaluated at the values of \(x_1,\dots,x_N\) where the given conditions must be satisfied. The function \(h_1(x_1,\dots,x_N)\) should alone make \(g_t(x_1,\dots,x_N)\) satisfy the conditions.

    +
    +
    +

    10.34. Network requirements

    +

    The network tries then the minimize the cost function following the +same ideas as described for the ODE case, but now with more than one +variables to consider. The concept still remains the same; find a set +of parameters \(P\) such that the expression \(f\) in (17) is as +close to zero as possible.

    +

    As for the ODE case, the cost function is the mean squared error that +the network must try to minimize. The cost function for the network to +minimize is

    +
    +\[ +C\left(x_1, \dots, x_N, P\right) = \left( f\left(x_1, \, \dots \, , x_N, \frac{\partial g(x_1,\dots,x_N) }{\partial x_1}, \dots , \frac{\partial g(x_1,\dots,x_N) }{\partial x_N}, \frac{\partial g(x_1,\dots,x_N) }{\partial x_1\partial x_2}, \, \dots \, , \frac{\partial^n g(x_1,\dots,x_N) }{\partial x_N^n} \right) \right)^2 +\]
    +
    +
    +

    10.35. More details

    +

    If we let \(\boldsymbol{x} = \big( x_1, \dots, x_N \big)\) be an array containing the values for \(x_1, \dots, x_N\) respectively, the cost function can be reformulated into the following:

    +
    +\[ +C\left(\boldsymbol{x}, P\right) = f\left( \left( \boldsymbol{x}, \frac{\partial g(\boldsymbol{x}) }{\partial x_1}, \dots , \frac{\partial g(\boldsymbol{x}) }{\partial x_N}, \frac{\partial g(\boldsymbol{x}) }{\partial x_1\partial x_2}, \, \dots \, , \frac{\partial^n g(\boldsymbol{x}) }{\partial x_N^n} \right) \right)^2 +\]
    +

    If we also have \(M\) different sets of values for \(x_1, \dots, x_N\), that is \(\boldsymbol{x}_i = \big(x_1^{(i)}, \dots, x_N^{(i)}\big)\) for \(i = 1,\dots,M\) being the rows in matrix \(X\), the cost function can be generalized into

    +
    +\[ +C\left(X, P \right) = \sum_{i=1}^M f\left( \left( \boldsymbol{x}_i, \frac{\partial g(\boldsymbol{x}_i) }{\partial x_1}, \dots , \frac{\partial g(\boldsymbol{x}_i) }{\partial x_N}, \frac{\partial g(\boldsymbol{x}_i) }{\partial x_1\partial x_2}, \, \dots \, , \frac{\partial^n g(\boldsymbol{x}_i) }{\partial x_N^n} \right) \right)^2. +\]
    +
    +
    +

    10.36. Example: The diffusion equation

    +

    In one spatial dimension, the equation reads

    +
    +\[ +\frac{\partial g(x,t)}{\partial t} = \frac{\partial^2 g(x,t)}{\partial x^2} +\]
    +

    where a possible choice of conditions are

    +
    +\[\begin{split} +\begin{align*} +g(0,t) &= 0 ,\qquad t \geq 0 \\ +g(1,t) &= 0, \qquad t \geq 0 \\ +g(x,0) &= u(x),\qquad x\in [0,1] +\end{align*} +\end{split}\]
    +

    with \(u(x)\) being some given function.

    +
    +
    +

    10.37. Defining the problem

    +

    For this case, we want to find \(g(x,t)\) such that

    + +
    +
    +\[ +\begin{equation} + \frac{\partial g(x,t)}{\partial t} = \frac{\partial^2 g(x,t)}{\partial x^2} +\end{equation} \label{diffonedim} \tag{18} +\]
    +

    and

    +
    +\[\begin{split} +\begin{align*} +g(0,t) &= 0 ,\qquad t \geq 0 \\ +g(1,t) &= 0, \qquad t \geq 0 \\ +g(x,0) &= u(x),\qquad x\in [0,1] +\end{align*} +\end{split}\]
    +

    with \(u(x) = \sin(\pi x)\).

    +

    First, let us set up the deep neural network. +The deep neural network will follow the same structure as discussed in the examples solving the ODEs. +First, we will look into how Autograd could be used in a network tailored to solve for bivariate functions.

    +
    +
    +

    10.38. Setting up the network using Autograd

    +

    The only change to do here, is to extend our network such that +functions of multiple parameters are correctly handled. In this case +we have two variables in our function to solve for, that is time \(t\) +and position \(x\). The variables will be represented by a +one-dimensional array in the program. The program will evaluate the +network at each possible pair \((x,t)\), given an array for the desired +\(x\)-values and \(t\)-values to approximate the solution at.

    +
    +
    +
    def sigmoid(z):
+    return 1/(1 + np.exp(-z))
+
+def deep_neural_network(deep_params, x):
+    # x is now a point and a 1D numpy array; make it a column vector
+    num_coordinates = np.size(x,0)
+    x = x.reshape(num_coordinates,-1)
+
+    num_points = np.size(x,1)
+
+    # N_hidden is the number of hidden layers
+    N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer
+
+    # Assume that the input layer does nothing to the input x
+    x_input = x
+    x_prev = x_input
+
+    ## Hidden layers:
+
+    for l in range(N_hidden):
+        # From the list of parameters P; find the correct weigths and bias for this layer
+        w_hidden = deep_params[l]
+
+        # Add a row of ones to include bias
+        x_prev = np.concatenate((np.ones((1,num_points)), x_prev ), axis = 0)
+
+        z_hidden = np.matmul(w_hidden, x_prev)
+        x_hidden = sigmoid(z_hidden)
+
+        # Update x_prev such that next layer can use the output from this layer
+        x_prev = x_hidden
+
+    ## Output layer:
+
+    # Get the weights and bias for this layer
+    w_output = deep_params[-1]
+
+    # Include bias:
+    x_prev = np.concatenate((np.ones((1,num_points)), x_prev), axis = 0)
+
+    z_output = np.matmul(w_output, x_prev)
+    x_output = z_output
+
+    return x_output[0][0]
+
    +
    +
    +
    +
    +
    +

    10.39. Setting up the network using Autograd; The trial solution

    +

    The cost function must then iterate through the given arrays +containing values for \(x\) and \(t\), defines a point \((x,t)\) the deep +neural network and the trial solution is evaluated at, and then finds +the Jacobian of the trial solution.

    +

    A possible trial solution for this PDE is

    +
    +\[ +g_t(x,t) = h_1(x,t) + x(1-x)tN(x,t,P) +\]
    +

    with \(A(x,t)\) being a function ensuring that \(g_t(x,t)\) satisfies our given conditions, and \(N(x,t,P)\) being the output from the deep neural network using weights and biases for each layer from \(P\).

    +

    To fulfill the conditions, \(A(x,t)\) could be:

    +
    +\[ +h_1(x,t) = (1-t)\Big(u(x) - \big((1-x)u(0) + x u(1)\big)\Big) = (1-t)u(x) = (1-t)\sin(\pi x) +\]
    +

    since \((0) = u(1) = 0\) and \(u(x) = \sin(\pi x)\).

    +
    +
    +

    10.40. Why the jacobian?

    +

    The Jacobian is used because the program must find the derivative of +the trial solution with respect to \(x\) and \(t\).

    +

    This gives the necessity of computing the Jacobian matrix, as we want +to evaluate the gradient with respect to \(x\) and \(t\) (note that the +Jacobian of a scalar-valued multivariate function is simply its +gradient).

    +

    In Autograd, the differentiation is by default done with respect to +the first input argument of your Python function. Since the points is +an array representing \(x\) and \(t\), the Jacobian is calculated using +the values of \(x\) and \(t\).

    +

    To find the second derivative with respect to \(x\) and \(t\), the +Jacobian can be found for the second time. The result is a Hessian +matrix, which is the matrix containing all the possible second order +mixed derivatives of \(g(x,t)\).

    +
    +
    +
    # Set up the trial function:
+def u(x):
+    return np.sin(np.pi*x)
+
+def g_trial(point,P):
+    x,t = point
+    return (1-t)*u(x) + x*(1-x)*t*deep_neural_network(P,point)
+
+# The right side of the ODE:
+def f(point):
+    return 0.
+
+# The cost function:
+def cost_function(P, x, t):
+    cost_sum = 0
+
+    g_t_jacobian_func = jacobian(g_trial)
+    g_t_hessian_func = hessian(g_trial)
+
+    for x_ in x:
+        for t_ in t:
+            point = np.array([x_,t_])
+
+            g_t = g_trial(point,P)
+            g_t_jacobian = g_t_jacobian_func(point,P)
+            g_t_hessian = g_t_hessian_func(point,P)
+
+            g_t_dt = g_t_jacobian[1]
+            g_t_d2x = g_t_hessian[0][0]
+
+            func = f(point)
+
+            err_sqr = ( (g_t_dt - g_t_d2x) - func)**2
+            cost_sum += err_sqr
+
+    return cost_sum
+
    +
    +
    +
    +
    +
    +

    10.41. Setting up the network using Autograd; The full program

    +

    Having set up the network, along with the trial solution and cost function, we can now see how the deep neural network performs by comparing the results to the analytical solution.

    +

    The analytical solution of our problem is

    +
    +\[ +g(x,t) = \exp(-\pi^2 t)\sin(\pi x) +\]
    +

    A possible way to implement a neural network solving the PDE, is given below. +Be aware, though, that it is fairly slow for the parameters used. +A better result is possible, but requires more iterations, and thus longer time to complete.

    +

    Indeed, the program below is not optimal in its implementation, but rather serves as an example on how to implement and use a neural network to solve a PDE. +Using TensorFlow results in a much better execution time. Try it!

    +
    +
    +
    import autograd.numpy as np
+from autograd import jacobian,hessian,grad
+import autograd.numpy.random as npr
+from matplotlib import cm
+from matplotlib import pyplot as plt
+from mpl_toolkits.mplot3d import axes3d
+
+## Set up the network
+
+def sigmoid(z):
+    return 1/(1 + np.exp(-z))
+
+def deep_neural_network(deep_params, x):
+    # x is now a point and a 1D numpy array; make it a column vector
+    num_coordinates = np.size(x,0)
+    x = x.reshape(num_coordinates,-1)
+
+    num_points = np.size(x,1)
+
+    # N_hidden is the number of hidden layers
+    N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer
+
+    # Assume that the input layer does nothing to the input x
+    x_input = x
+    x_prev = x_input
+
+    ## Hidden layers:
+
+    for l in range(N_hidden):
+        # From the list of parameters P; find the correct weigths and bias for this layer
+        w_hidden = deep_params[l]
+
+        # Add a row of ones to include bias
+        x_prev = np.concatenate((np.ones((1,num_points)), x_prev ), axis = 0)
+
+        z_hidden = np.matmul(w_hidden, x_prev)
+        x_hidden = sigmoid(z_hidden)
+
+        # Update x_prev such that next layer can use the output from this layer
+        x_prev = x_hidden
+
+    ## Output layer:
+
+    # Get the weights and bias for this layer
+    w_output = deep_params[-1]
+
+    # Include bias:
+    x_prev = np.concatenate((np.ones((1,num_points)), x_prev), axis = 0)
+
+    z_output = np.matmul(w_output, x_prev)
+    x_output = z_output
+
+    return x_output[0][0]
+
+## Define the trial solution and cost function
+def u(x):
+    return np.sin(np.pi*x)
+
+def g_trial(point,P):
+    x,t = point
+    return (1-t)*u(x) + x*(1-x)*t*deep_neural_network(P,point)
+
+# The right side of the ODE:
+def f(point):
+    return 0.
+
+# The cost function:
+def cost_function(P, x, t):
+    cost_sum = 0
+
+    g_t_jacobian_func = jacobian(g_trial)
+    g_t_hessian_func = hessian(g_trial)
+
+    for x_ in x:
+        for t_ in t:
+            point = np.array([x_,t_])
+
+            g_t = g_trial(point,P)
+            g_t_jacobian = g_t_jacobian_func(point,P)
+            g_t_hessian = g_t_hessian_func(point,P)
+
+            g_t_dt = g_t_jacobian[1]
+            g_t_d2x = g_t_hessian[0][0]
+
+            func = f(point)
+
+            err_sqr = ( (g_t_dt - g_t_d2x) - func)**2
+            cost_sum += err_sqr
+
+    return cost_sum /( np.size(x)*np.size(t) )
+
+## For comparison, define the analytical solution
+def g_analytic(point):
+    x,t = point
+    return np.exp(-np.pi**2*t)*np.sin(np.pi*x)
+
+## Set up a function for training the network to solve for the equation
+def solve_pde_deep_neural_network(x,t, num_neurons, num_iter, lmb):
+    ## Set up initial weigths and biases
+    N_hidden = np.size(num_neurons)
+
+    ## Set up initial weigths and biases
+
+    # Initialize the list of parameters:
+    P = [None]*(N_hidden + 1) # + 1 to include the output layer
+
+    P[0] = npr.randn(num_neurons[0], 2 + 1 ) # 2 since we have two points, +1 to include bias
+    for l in range(1,N_hidden):
+        P[l] = npr.randn(num_neurons[l], num_neurons[l-1] + 1) # +1 to include bias
+
+    # For the output layer
+    P[-1] = npr.randn(1, num_neurons[-1] + 1 ) # +1 since bias is included
+
+    print('Initial cost: ',cost_function(P, x, t))
+
+    cost_function_grad = grad(cost_function,0)
+
+    # Let the update be done num_iter times
+    for i in range(num_iter):
+        cost_grad =  cost_function_grad(P, x , t)
+
+        for l in range(N_hidden+1):
+            P[l] = P[l] - lmb * cost_grad[l]
+
+    print('Final cost: ',cost_function(P, x, t))
+
+    return P
+
+if __name__ == '__main__':
+    ### Use the neural network:
+    npr.seed(15)
+
+    ## Decide the vales of arguments to the function to solve
+    Nx = 10; Nt = 10
+    x = np.linspace(0, 1, Nx)
+    t = np.linspace(0,1,Nt)
+
+    ## Set up the parameters for the network
+    num_hidden_neurons = [100, 25]
+    num_iter = 250
+    lmb = 0.01
+
+    P = solve_pde_deep_neural_network(x,t, num_hidden_neurons, num_iter, lmb)
+
+    ## Store the results
+    g_dnn_ag = np.zeros((Nx, Nt))
+    G_analytical = np.zeros((Nx, Nt))
+    for i,x_ in enumerate(x):
+        for j, t_ in enumerate(t):
+            point = np.array([x_, t_])
+            g_dnn_ag[i,j] = g_trial(point,P)
+
+            G_analytical[i,j] = g_analytic(point)
+
+    # Find the map difference between the analytical and the computed solution
+    diff_ag = np.abs(g_dnn_ag - G_analytical)
+    print('Max absolute difference between the analytical solution and the network: %g'%np.max(diff_ag))
+
+    ## Plot the solutions in two dimensions, that being in position and time
+
+    T,X = np.meshgrid(t,x)
+
+    fig = plt.figure(figsize=(10,10))
+    ax = fig.gca(projection='3d')
+    ax.set_title('Solution from the deep neural network w/ %d layer'%len(num_hidden_neurons))
+    s = ax.plot_surface(T,X,g_dnn_ag,linewidth=0,antialiased=False,cmap=cm.viridis)
+    ax.set_xlabel('Time $t$')
+    ax.set_ylabel('Position $x$');
+
+
+    fig = plt.figure(figsize=(10,10))
+    ax = fig.gca(projection='3d')
+    ax.set_title('Analytical solution')
+    s = ax.plot_surface(T,X,G_analytical,linewidth=0,antialiased=False,cmap=cm.viridis)
+    ax.set_xlabel('Time $t$')
+    ax.set_ylabel('Position $x$');
+
+    fig = plt.figure(figsize=(10,10))
+    ax = fig.gca(projection='3d')
+    ax.set_title('Difference')
+    s = ax.plot_surface(T,X,diff_ag,linewidth=0,antialiased=False,cmap=cm.viridis)
+    ax.set_xlabel('Time $t$')
+    ax.set_ylabel('Position $x$');
+
+    ## Take some slices of the 3D plots just to see the solutions at particular times
+    indx1 = 0
+    indx2 = int(Nt/2)
+    indx3 = Nt-1
+
+    t1 = t[indx1]
+    t2 = t[indx2]
+    t3 = t[indx3]
+
+    # Slice the results from the DNN
+    res1 = g_dnn_ag[:,indx1]
+    res2 = g_dnn_ag[:,indx2]
+    res3 = g_dnn_ag[:,indx3]
+
+    # Slice the analytical results
+    res_analytical1 = G_analytical[:,indx1]
+    res_analytical2 = G_analytical[:,indx2]
+    res_analytical3 = G_analytical[:,indx3]
+
+    # Plot the slices
+    plt.figure(figsize=(10,10))
+    plt.title("Computed solutions at time = %g"%t1)
+    plt.plot(x, res1)
+    plt.plot(x,res_analytical1)
+    plt.legend(['dnn','analytical'])
+
+    plt.figure(figsize=(10,10))
+    plt.title("Computed solutions at time = %g"%t2)
+    plt.plot(x, res2)
+    plt.plot(x,res_analytical2)
+    plt.legend(['dnn','analytical'])
+
+    plt.figure(figsize=(10,10))
+    plt.title("Computed solutions at time = %g"%t3)
+    plt.plot(x, res3)
+    plt.plot(x,res_analytical3)
+    plt.legend(['dnn','analytical'])
+
+    plt.show()
+
    +
    +
    +
    +
    +
    +

    10.42. Example: Solving the wave equation with Neural Networks

    +

    The wave equation is

    +
    +\[ +\frac{\partial^2 g(x,t)}{\partial t^2} = c^2\frac{\partial^2 g(x,t)}{\partial x^2} +\]
    +

    with \(c\) being the specified wave speed.

    +

    Here, the chosen conditions are

    +
    +\[\begin{split} +\begin{align*} + g(0,t) &= 0 \\ + g(1,t) &= 0 \\ + g(x,0) &= u(x) \\ + \frac{\partial g(x,t)}{\partial t} \Big |_{t = 0} &= v(x) +\end{align*} +\end{split}\]
    +

    where \(\frac{\partial g(x,t)}{\partial t} \Big |_{t = 0}\) means the derivative of \(g(x,t)\) with respect to \(t\) is evaluated at \(t = 0\), and \(u(x)\) and \(v(x)\) being given functions.

    +
    +
    +

    10.43. The problem to solve for

    +

    The wave equation to solve for, is

    + +
    +
    +\[ +\begin{equation} \label{wave} \tag{19} +\frac{\partial^2 g(x,t)}{\partial t^2} = c^2 \frac{\partial^2 g(x,t)}{\partial x^2} +\end{equation} +\]
    +

    where \(c\) is the given wave speed. +The chosen conditions for this equation are

    + +
    +
    +\[\begin{split} +\begin{aligned} +g(0,t) &= 0, &t \geq 0 \\ +g(1,t) &= 0, &t \geq 0 \\ +g(x,0) &= u(x), &x\in[0,1] \\ +\frac{\partial g(x,t)}{\partial t}\Big |_{t = 0} &= v(x), &x \in [0,1] +\end{aligned} \label{condwave} \tag{20} +\end{split}\]
    +

    In this example, let \(c = 1\) and \(u(x) = \sin(\pi x)\) and \(v(x) = -\pi\sin(\pi x)\).

    +
    +
    +

    10.44. The trial solution

    +

    Setting up the network is done in similar matter as for the example of solving the diffusion equation. +The only things we have to change, is the trial solution such that it satisfies the conditions from (20) and the cost function.

    +

    The trial solution becomes slightly different since we have other conditions than in the example of solving the diffusion equation. Here, a possible trial solution \(g_t(x,t)\) is

    +
    +\[ +g_t(x,t) = h_1(x,t) + x(1-x)t^2N(x,t,P) +\]
    +

    where

    +
    +\[ +h_1(x,t) = (1-t^2)u(x) + tv(x) +\]
    +

    Note that this trial solution satisfies the conditions only if \(u(0) = v(0) = u(1) = v(1) = 0\), which is the case in this example.

    +
    +
    +

    10.45. The analytical solution

    +

    The analytical solution for our specific problem, is

    +
    +\[ +g(x,t) = \sin(\pi x)\cos(\pi t) - \sin(\pi x)\sin(\pi t) +\]
    +
    +
    +

    10.46. Solving the wave equation - the full program using Autograd

    +
    +
    +
    import autograd.numpy as np
+from autograd import hessian,grad
+import autograd.numpy.random as npr
+from matplotlib import cm
+from matplotlib import pyplot as plt
+from mpl_toolkits.mplot3d import axes3d
+
+## Set up the trial function:
+def u(x):
+    return np.sin(np.pi*x)
+
+def v(x):
+    return -np.pi*np.sin(np.pi*x)
+
+def h1(point):
+    x,t = point
+    return (1 - t**2)*u(x) + t*v(x)
+
+def g_trial(point,P):
+    x,t = point
+    return h1(point) + x*(1-x)*t**2*deep_neural_network(P,point)
+
+## Define the cost function
+def cost_function(P, x, t):
+    cost_sum = 0
+
+    g_t_hessian_func = hessian(g_trial)
+
+    for x_ in x:
+        for t_ in t:
+            point = np.array([x_,t_])
+
+            g_t_hessian = g_t_hessian_func(point,P)
+
+            g_t_d2x = g_t_hessian[0][0]
+            g_t_d2t = g_t_hessian[1][1]
+
+            err_sqr = ( (g_t_d2t - g_t_d2x) )**2
+            cost_sum += err_sqr
+
+    return cost_sum / (np.size(t) * np.size(x))
+
+## The neural network
+def sigmoid(z):
+    return 1/(1 + np.exp(-z))
+
+def deep_neural_network(deep_params, x):
+    # x is now a point and a 1D numpy array; make it a column vector
+    num_coordinates = np.size(x,0)
+    x = x.reshape(num_coordinates,-1)
+
+    num_points = np.size(x,1)
+
+    # N_hidden is the number of hidden layers
+    N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer
+
+    # Assume that the input layer does nothing to the input x
+    x_input = x
+    x_prev = x_input
+
+    ## Hidden layers:
+
+    for l in range(N_hidden):
+        # From the list of parameters P; find the correct weigths and bias for this layer
+        w_hidden = deep_params[l]
+
+        # Add a row of ones to include bias
+        x_prev = np.concatenate((np.ones((1,num_points)), x_prev ), axis = 0)
+
+        z_hidden = np.matmul(w_hidden, x_prev)
+        x_hidden = sigmoid(z_hidden)
+
+        # Update x_prev such that next layer can use the output from this layer
+        x_prev = x_hidden
+
+    ## Output layer:
+
+    # Get the weights and bias for this layer
+    w_output = deep_params[-1]
+
+    # Include bias:
+    x_prev = np.concatenate((np.ones((1,num_points)), x_prev), axis = 0)
+
+    z_output = np.matmul(w_output, x_prev)
+    x_output = z_output
+
+    return x_output[0][0]
+
+## The analytical solution
+def g_analytic(point):
+    x,t = point
+    return np.sin(np.pi*x)*np.cos(np.pi*t) - np.sin(np.pi*x)*np.sin(np.pi*t)
+
+def solve_pde_deep_neural_network(x,t, num_neurons, num_iter, lmb):
+    ## Set up initial weigths and biases
+    N_hidden = np.size(num_neurons)
+
+    ## Set up initial weigths and biases
+
+    # Initialize the list of parameters:
+    P = [None]*(N_hidden + 1) # + 1 to include the output layer
+
+    P[0] = npr.randn(num_neurons[0], 2 + 1 ) # 2 since we have two points, +1 to include bias
+    for l in range(1,N_hidden):
+        P[l] = npr.randn(num_neurons[l], num_neurons[l-1] + 1) # +1 to include bias
+
+    # For the output layer
+    P[-1] = npr.randn(1, num_neurons[-1] + 1 ) # +1 since bias is included
+
+    print('Initial cost: ',cost_function(P, x, t))
+
+    cost_function_grad = grad(cost_function,0)
+
+    # Let the update be done num_iter times
+    for i in range(num_iter):
+        cost_grad =  cost_function_grad(P, x , t)
+
+        for l in range(N_hidden+1):
+            P[l] = P[l] - lmb * cost_grad[l]
+
+
+    print('Final cost: ',cost_function(P, x, t))
+
+    return P
+
+if __name__ == '__main__':
+    ### Use the neural network:
+    npr.seed(15)
+
+    ## Decide the vales of arguments to the function to solve
+    Nx = 10; Nt = 10
+    x = np.linspace(0, 1, Nx)
+    t = np.linspace(0,1,Nt)
+
+    ## Set up the parameters for the network
+    num_hidden_neurons = [50,20]
+    num_iter = 1000
+    lmb = 0.01
+
+    P = solve_pde_deep_neural_network(x,t, num_hidden_neurons, num_iter, lmb)
+
+    ## Store the results
+    res = np.zeros((Nx, Nt))
+    res_analytical = np.zeros((Nx, Nt))
+    for i,x_ in enumerate(x):
+        for j, t_ in enumerate(t):
+            point = np.array([x_, t_])
+            res[i,j] = g_trial(point,P)
+
+            res_analytical[i,j] = g_analytic(point)
+
+    diff = np.abs(res - res_analytical)
+    print("Max difference between analytical and solution from nn: %g"%np.max(diff))
+
+    ## Plot the solutions in two dimensions, that being in position and time
+
+    T,X = np.meshgrid(t,x)
+
+    fig = plt.figure(figsize=(10,10))
+    ax = fig.gca(projection='3d')
+    ax.set_title('Solution from the deep neural network w/ %d layer'%len(num_hidden_neurons))
+    s = ax.plot_surface(T,X,res,linewidth=0,antialiased=False,cmap=cm.viridis)
+    ax.set_xlabel('Time $t$')
+    ax.set_ylabel('Position $x$');
+
+
+    fig = plt.figure(figsize=(10,10))
+    ax = fig.gca(projection='3d')
+    ax.set_title('Analytical solution')
+    s = ax.plot_surface(T,X,res_analytical,linewidth=0,antialiased=False,cmap=cm.viridis)
+    ax.set_xlabel('Time $t$')
+    ax.set_ylabel('Position $x$');
+
+
+    fig = plt.figure(figsize=(10,10))
+    ax = fig.gca(projection='3d')
+    ax.set_title('Difference')
+    s = ax.plot_surface(T,X,diff,linewidth=0,antialiased=False,cmap=cm.viridis)
+    ax.set_xlabel('Time $t$')
+    ax.set_ylabel('Position $x$');
+
+    ## Take some slices of the 3D plots just to see the solutions at particular times
+    indx1 = 0
+    indx2 = int(Nt/2)
+    indx3 = Nt-1
+
+    t1 = t[indx1]
+    t2 = t[indx2]
+    t3 = t[indx3]
+
+    # Slice the results from the DNN
+    res1 = res[:,indx1]
+    res2 = res[:,indx2]
+    res3 = res[:,indx3]
+
+    # Slice the analytical results
+    res_analytical1 = res_analytical[:,indx1]
+    res_analytical2 = res_analytical[:,indx2]
+    res_analytical3 = res_analytical[:,indx3]
+
+    # Plot the slices
+    plt.figure(figsize=(10,10))
+    plt.title("Computed solutions at time = %g"%t1)
+    plt.plot(x, res1)
+    plt.plot(x,res_analytical1)
+    plt.legend(['dnn','analytical'])
+
+    plt.figure(figsize=(10,10))
+    plt.title("Computed solutions at time = %g"%t2)
+    plt.plot(x, res2)
+    plt.plot(x,res_analytical2)
+    plt.legend(['dnn','analytical'])
+
+    plt.figure(figsize=(10,10))
+    plt.title("Computed solutions at time = %g"%t3)
+    plt.plot(x, res3)
+    plt.plot(x,res_analytical3)
+    plt.legend(['dnn','analytical'])
+
+    plt.show()
+
    +
    +
    +
    +
    + +
    + + + + +
    + +
    +
    + + +
    + + 8. Convolutional Neural Networks + 11. Data Analysis and Machine Learning: + +
    +
    +
    +

    + + By Morten Hjorth-Jensen
    + + © Copyright 2020.
    +

    +
    +
    +
    + + +
    +
    + + + + + \ No newline at end of file diff --git a/doc/src/LectureNotes/_build/html/chapter11.html b/doc/src/LectureNotes/_build/html/chapter11.html new file mode 100644 index 000000000..da9037a15 --- /dev/null +++ b/doc/src/LectureNotes/_build/html/chapter11.html @@ -0,0 +1,899 @@ + + + + + + + + + 11. Data Analysis and Machine Learning: — Applied Machine Learning and Data Analysis + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    +
    + + + + + + + + +
    + +
    +
    +
    +
    + + + +
    + + + +
    + + + + + + +
    + +
    + + + + + + + + +
    + + +
    +
    + Contents +
    + +
    +
    +
    +
    + +
    + + +
    +

    11. Data Analysis and Machine Learning:

    + + +

    Christian Forssén, Department of Physics, Chalmers University of Technology, Sweden

    + + **Morten Hjorth-Jensen**, Department of Physics, University of Oslo and Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University +

    Date: Dec 23, 2020

    +

    Copyright 1999-2020, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license

    +
    +
    +

    12. Elements of Bayesian theory and Bayesian Neural Networks

    +
    +

    12.1. Why Bayesian Statistics?

    +

    We have already made ourselves familiar with elements of a statistical +data analysis via quantities like the bias-variance tradeoff as well +as some central distribution functions such as the Normal +distribution, the binomial distribution and other probability +distribution functions.

    +

    In essentially all the Machine Learning algorithms we have studied, +our focus has been on a so-called frequentist approach, where +knowledge of an underlying likelihood function has not been +emphasized. Our data, whether we had a classification or a regression +problem, have been our central points of departure.

    +

    Here we wish to merge this approach with the derivation of a +likelihood function which can be used to make prediction on how our +system under study evolves. We will venture into the realm of what is +called Bayesian Neural Networks. To get an overarching view on what +this entails, the following figure conveys the essential differences +between a standard Neural network that we have met earlier and a +Bayesian Neural Network. In order to get there, we need to present +some of the basic elements of Bayesian statistics, starting with the +product rule and Bayes’ theorem.

    +
    +
    +

    12.2. Inference

    +

    Inference: +:
    +“the act of passing from one proposition, statement or judgment considered as true to another whose truth is believed to follow from that of the former” (Webster) +Do premises \(A, B, \ldots \to\) hypothesis, \(H\)?

    +

    Deductive inference: +:
    +Premises allow definite determination of truth/falsity of H (syllogisms, symbolic logic, Boolean algebra) +\(B(H|A,B,...) = 0\) or \(1\)

    +

    Inductive inference: +:
    +Premises bear on truth/falsity of H, but don’t allow its definite determination (weak syllogisms, analogies) +\(A, B, C, D\) share properties \(x, y, z\); \(E\) has properties \(x, y\) +\(\to\) \(E\) probably has property \(z\).

    +
    +
    +

    12.3. Statistical Inference

    +
      +

    • Quantify the strength of inductive inferences from facts, in the form of data (\(D\)), and other premises, e.g. models, to hypotheses about the phenomena producing the data.

      • +

    • Quantify via probabilities, or averages calculated using probabilities. Frequentists (\(\mathcal{F}\)) and Bayesians (\(\mathcal{B}\)) use probabilities very differently for this.

      • +

    • To the pioneers such as Bernoulli, Bayes and Laplace, a probability represented a degree-of-belief or plausability: how much they thought that something as true based on the evidence at hand. This is the Bayesian approach.

      • +

    • To the 19th century scholars, this seemed too vague and subjective. They redefined probability as the long run relative frequency with which an event occurred, given (infinitely) many repeated (experimental) trials.

      • +
    +
    +
    +

    12.4. Some history

    +

    Adapted from D.S. Sivia1:

    +
    +

    Although the frequency definition appears to be more objective, its range of validity is also far more limited. For example, Laplace used (his) probability theory to estimate the mass of Saturn, given orbital data that were available to him from various astronomical observatories. In essence, he computed the posterior pdf for the mass M , given the data and all the relevant background information I (such as a knowledge of the laws of classical mechanics): prob(M|{data},I); this is shown schematically in the figure [Fig. 1.2].

    +
    + + +

    + + +
    +

    To Laplace, the (shaded) area under the posterior pdf curve between \(m_1\) and \(m_2\) was a measure of how much he believed that the mass of Saturn lay in the range \(m_1 \le M \le m_2\). As such, the position of the maximum of the posterior pdf represents a best estimate of the mass; its width, or spread, about this optimal value gives an indication of the uncertainty in the estimate. Laplace stated that: ‘ … it is a bet of 11,000 to 1 that the error of this result is not 1/100th of its value.’ He would have won the bet, as another 150 years’ accumulation of data has changed the estimate by only 0.63%!

    +
    +
    +

    According to the frequency definition, however, we are not permitted to use probability theory to tackle this problem. This is because the mass of Saturn is a constant and not a random variable; therefore, it has no frequency distribution and so probability theory cannot be used.

    +

    If the pdf [of Fig. 1.2] had to be interpreted in terms of the frequency definition, we would have to imagine a large ensemble of universes in which everything remains constant apart from the mass of Saturn.

    +
    +
    +

    As this scenario appears quite far-fetched, we might be inclined to think of [Fig. 1.2] in terms of the distribution of the measurements of the mass in many repetitions of the experiment. Although we are at liberty to think about a problem in any way that facilitates its solution, or our understanding of it, having to seek a frequency interpretation for every data analysis problem seems rather perverse. +For example, what do we mean by the ‘measurement of the mass’ when the data consist of orbital periods? Besides, why should we have to think about many repetitions of an experiment that never happened? What we really want to do is to make the best inference of the mass given the (few) data that we actually have; this is precisely the Bayes and Laplace view of probability.

    +
    +
    +

    Faced with the realization that the frequency definition of probability theory did not permit most real-life scientific problems to be addressed, a new subject was invented — statistics! To estimate the mass of Saturn, for example, one has to relate the mass to the data through some function called the statistic; since the data are subject to ‘random’ noise, the statistic becomes the random variable to which the rules of probability the- ory can be applied. But now the question arises: How should we choose the statistic? The frequentist approach does not yield a natural way of doing this and has, therefore, led to the development of several alternative schools of orthodox or conventional statis- tics. The masters, such as Fisher, Neyman and Pearson, provided a variety of different principles, which has merely resulted in a plethora of tests and procedures without any clear underlying rationale. This lack of unifying principles is, perhaps, at the heart of the shortcomings of the cook-book approach to statistics that students are often taught even today.

    +
    +
    +
    +

    12.5. The Bayesian recipe

    +

    Assess hypotheses by calculating their probabilities \(p(H_i | \ldots)\) conditional on known and/or presumed information using the rules of probability theory.

    +

    Probability Theory Axioms: +Product (AND) rule : +:
    +\(p(A, B | I) = p(A|I) p(B|A, I) = p(B|I)p(A|B,I)\) +Should read \(p(A,B|I)\) as the probability for propositions \(A\) AND \(B\) being true given that \(I\) is true.

    +

    Sum (OR) rule: +:
    +\(p(A + B | I) = p(A | I) + p(B | I) - p(A, B | I)\) +\(p(A+B|I)\) is the probability that proposition \(A\) OR \(B\) is true given that \(I\) is true.

    +

    Normalization: +:
    +\(p(A|I) + p(\bar{A}|I) = 1\) +\(\bar{A}\) denotes the proposition that \(A\) is false.

    +
    +
    +

    12.6. Bayes’ theorem

    +

    Bayes’ theorem follows directly from the product rule

    +
    +\[\]
    +

    p(A|B,I) = \frac{p(B|A,I) p(A|I)}{p(B|I)}. +$\( +\)$

    +

    The importance of this property to data analysis becomes apparent if we replace \(A\) and \(B\) by hypothesis(\(H\)) and data(\(D\)):

    + +
    +
    +\[ +\begin{equation} +p(H|D,I) = \frac{p(D|H,I) p(H|I)}{p(D|I)}. +\label{eq:bayes} \tag{1} +\end{equation} +\]
    +

    The power of Bayes’ theorem lies in the fact that it relates the quantity of interest, the probability that the hypothesis is true given the data, to the term we have a better chance of being able to assign, the probability that we would have observed the measured data if the hypothesis was true.

    +

    The various terms in Bayes’ theorem have formal names.

    +
      +

    • The quantity on the far right, \(p(H|I)\), is called the prior probability; it represents our state of knowledge (or ignorance) about the truth of the hypothesis before we have analysed the current data.

      • +

    • This is modified by the experimental measurements through \(p(D|H,I)\), the likelihood function,

      • +

    • The denominator \(p(D|I)\) is called the evidence. It does not depend on the hypothesis and can be regarded as a normalization constant.

      • +

    • Together, these yield the posterior probability, \(p(H|D, I )\), representing our state of knowledge about the truth of the hypothesis in the light of the data.

      • +
    +

    In a sense, Bayes’ theorem encapsulates the process of learning.

    +
    +
    +

    12.7. The friends of Bayes’ theorem

    +

    Normalization: +:
    +\(\sum_i p(H_i|\ldots) = 1\).

    +

    Marginalization: +:
    +\(\sum_i p(A,H_i|I) = \sum_i p(H_i|A,I) p(A|I) = p(A|I)\).

    +

    Marginalization (continuum limit): +:
    +\(\int dx p(A,H(x)|I) = p(A|I)\).

    +

    In the above, \(H_i\) is an exclusive and exhaustive list of hypotheses. For example,let’s imagine that there are five candidates in a presidential election; then \(H_1\) could be the proposition that the first candidate will win, and so on. The probability that \(A\) is true, for example that unemployment will be lower in a year’s time (given all relevant information \(I\), but irrespective of whoever becomes president) is then given by \(\sum_i p(A,H_i|I)\).

    +

    In the continuum limit of propositions we must understand \(p(\ldots)\) as a pdf (probability density function).

    +

    Marginalization is a very powerful device in data analysis because it enables us to deal with nuisance parameters; that is, quantities which necessarily enter the analysis but are of no intrinsic interest. The unwanted background signal present in many experimental measurements are examples of nuisance parameters.

    +
    +
    +

    12.8. Inference With Parametric Models

    +

    Inductive inference with parametric models is a very important tool in the natural sciences.

    +
      +

    • Consider \(N\) different models \(M_i\) (\(i = 1, \ldots, N\)), each with parameters \(\boldsymbol{\alpha}_i\). Each of them implies a sampling distribution (conditional predictive distribution for possible data)

      • +
    +
    +\[\]
    +

    p(D|\boldsymbol{\alpha}_i, M_i) +$\( +\)$

    +
      +

    • The \(\boldsymbol{\alpha}_i\) dependence when we fix attention on the actual, observed data (\(D_\mathrm{obs}\)) is the likelihood function:

      • +
    +
    +\[\]
    +

    \mathcal{L}_i (\boldsymbol{\alpha}i) \equiv p(D\mathrm{obs}|\boldsymbol{\alpha}_i, M_i) +$\( +\)$

    +
      +

    • We may be uncertain about \(i\) (model uncertainty),

      • +

    • or uncertain about \(\boldsymbol{\alpha}_i\) (parameter uncertainty).

      • +
    +

    Parameter Estimation: +:
    +Premise = choice of model (pick specific \(i\)) +\(\Rightarrow\) What can we say about \(\boldsymbol{\alpha}_i\)?

    +

    Model comparison: +:
    +Premise = \(\{M_i\}\) +\(\Rightarrow\) What can we say about \(i\)?

    +

    Model adequacy: +:
    +Premise = \(M_1\) +\(\Rightarrow\) Is \(M_1\) adequate?

    +

    Hybrid Uncertainty: +:
    +Models share some common params: \(\boldsymbol{\alpha}_1 = \{ \boldsymbol{\varphi}, \boldsymbol{\eta}_i\}\) +\(\Rightarrow\) What can we say about \(\boldsymbol{\varphi}\)? (Systematic error is an example)

    +
    +
    +

    12.9. Illustrative examples with python code

    +
      +

    • Is this a fair coin? (analytical)

      • +

    • Flux from a star (single parameter, MCMC)

      • +

    • The lighthouse problem (two parameters, MCMC)

      • +

    • Linear fit with outliers (nuisance parameters)

      • +

    • …

      • +
    +
    +
    +

    12.10. Example: Is this a fair coin?

    +

    Let us begin with the analysis of data from a simple coin-tossing experiment. +Given that we had observed 6 heads in 8 flips, would you think it was a fair coin? By fair, we mean that we would be prepared to lay an even 1 : 1 bet on the outcome of a flip being a head or a tail. If we decide that the coin was fair, the question which follows naturally is how sure are we that this was so; if it was not fair, how unfair do we think it was? Furthermore, if we were to continue collecting data for this particular coin, observing the outcomes of additional flips, how would we update our belief on the fairness of the coin?

    +

    A sensible way of formulating this problem is to consider a large number of hypotheses about the range in which the bias-weighting of the coin might lie. If we denote the bias-weighting by \(H\), then \(H = 0\) and \(H = 1\) can represent a coin which produces a tail or a head on every flip, respectively. There is a continuum of possibilities for the value of H between these limits, with \(H = 0.5\) indicating a fair coin. Our state of knowledge about the fairness, or the degree of unfairness, of the coin is then completely summarized by specifying how much we believe these various propositions to be true.

    +

    Let us perform a computer simulation of a coin-tossing experiment. This provides the data that we will be analysing.

    +

    0

    +

    < +< +< +! +! +C +O +D +E +_ +B +L +O +C +K

    +

    p +y +c +o +d

    +
    +
    +
    np.random.seed(999)         # for reproducibility
+a=0.6                       # biased coin
+flips=np.random.rand(2**12) # simulates 4096 coin flips
+heads=flips<a               # boolean array, heads[i]=True if flip i is heads
+
    +
    +
    +
    +
    ---------------------------------------------------------------------------
+NameError                                 Traceback (most recent call last)
+<ipython-input-1-3f1158901148> in <module>
+----> 1 np.random.seed(999)         # for reproducibility
+      2 a=0.6                       # biased coin
+      3 flips=np.random.rand(2**12) # simulates 4096 coin flips
+      4 heads=flips<a               # boolean array, heads[i]=True if flip i is heads
+
+NameError: name 'np' is not defined
+
    +
    +
    +
    +

    In the light of this data, our inference about the fairness of this coin is summarized by the conditional pdf: \(p(H|D,I)\). This is, of course, shorthand for the limiting case of a continuum of propositions for the value of \(H\); that is to say, the probability that \(H\) lies in an infinitesimally narrow range is given by \(p(H|D,I) dH\).

    +

    To estimate this posterior pdf, we need to use Bayes’ theorem (1). We will ignore the denominator \(p(D|I)\) as it does not involve bias-weighting explicitly, and it will therefore not affect the shape of the desired pdf. At the end we can evaluate the missing constant subsequently from the normalization condition

    + +
    +
    +\[ +\begin{equation} +\int_0^1 p(H|D,I) dH = 1. +\label{eq:coin_posterior_norm} \tag{2} +\end{equation} +\]
    +

    The prior pdf, \(p(H|I)\), represents what we know about the coin given only the information \(I\) that we are dealing with a ‘strange coin’. We could keep a very open mind about the nature of the coin; a simple probability assignment which reflects this is a uniform, or flat, prior

    + +
    +
    +\[\begin{split} +\begin{equation} +p(H|I) = \left\{ \begin{array}{ll} +1 & 0 \le H \le 1, \\ +0 & \mathrm{otherwise}. +\end{array} \right. +\label{eq:coin_prior_uniform} \tag{3} +\end{equation} +\end{split}\]
    +

    We will get back later to the choice of prior and its effect on the analysis.

    +

    This prior state of knowledge, or ignorance, is modified by the data through the likelihood function \(p(D|H,I)\). It is a measure of the chance that we would have obtained the data that we actually observed, if the value of the bias-weighting was given (as known). If, in the conditioning information \(I\), we assume that the flips of the coin were independent events, so that the outcome of one did not influence that of another, then the probability of obtaining the data `R heads in N tosses’ is given by the binomial distribution (we leave a formal definition of this to a statistics textbook)

    + +
    +
    +\[ +\begin{equation} +p(D|H,I) \propto H^R (1-H)^{N-R}. +\label{_auto1} \tag{4} +\end{equation} +\]
    +

    It seems reasonable because \(H\) is the chance of obtaining a head on any flip, and there were \(R\) of them, and \(1-H\) is the corresponding probability for a tail, of which there were \(N-R\). We note that this binomial distribution also contains a normalization factor, but we will ignore it since it does not depend explicitly on \(H\), the quantity of interest. It will be absorbed by the normalization condition (2).

    +

    We perform the setup of this Bayesian framework on the computer.

    +
    +
    +
    def prior(H):
+    p=np.zeros_like(H)
+    p[(0<=x)&(x<=1)]=1      # allowed range: 0<=H<=1
+    return p                # uniform prior
+def likelihood(H,data):
+    N = len(data)
+    no_of_heads = sum(data)
+    no_of_tails = N - no_of_heads
+    return H**no_of_heads * (1-H)**no_of_tails
+def posterior(H,data):
+    p=prior(H)*likelihood(H,data)
+    norm=np.trapz(p,H)
+    return p/norm
+
    +
    +
    +
    +

    The next step is to confront this setup with the simulated data. To get a feel for the result, it is instructive to see how the posterior pdf evolves as we obtain more and more data pertaining to the coin. The results of such an analyses is shown in Fig. fig:coinflipping.

    +
    +
    +
    x=np.linspace(0,1,100)
+fig, axs = plt.subplots(nrows=4,ncols=3,sharex=True,sharey='row')
+axs_vec=np.reshape(axs,-1)
+axs_vec[0].plot(x,prior(x))
+for ndouble in range(11):
+    ax=axs_vec[1+ndouble]
+    ax.plot(x,posterior(x,heads[:2**ndouble]))
+    ax.text(0.1, 0.8, '$N={0}$'.format(2**ndouble), transform=ax.transAxes)
+for row in range(4): axs[row,0].set_ylabel('$p(H|D_\mathrm{obs},I)$')
+for col in range(3): axs[-1,col].set_xlabel('$H$')
+
    +
    +
    +
    + + +
    +

    The evolution of the posterior pdf for the bias-weighting of a coin, as the number of data available increases. The figure on the top left-hand corner of each panel shows the number of data included in the analysis.

    + + +

    The panel in the top left-hand corner shows the posterior pdf for \(H\) given no data, i.e., it is the same as the prior pdf of Eq. (3). It indicates that we have no more reason to believe that the coin is fair than we have to think that it is double-headed, double-tailed, or of any other intermediate bias-weighting.

    +

    The first flip is obviously tails. At this point we have no evidence that the coin has a side with heads, as indicated by the pdf going to zero as \(H \to 1\). The second flip is obviously heads and we have now excluded both extreme options \(H=0\) (double-tailed) and \(H=1\) (double-headed). We can note that the posterior at this point has the simple form \(p(H|D,I) = H(1-H)\) for \(0 \le H \le 1\).

    +

    The remainder of Fig. fig:coinflipping shows how the posterior pdf evolves as the number of data analysed becomes larger and larger. We see that the position of the maximum moves around, but that the amount by which it does so decreases with the increasing number of observations. The width of the posterior pdf also becomes narrower with more data, indicating that we are becoming increasingly confident in our estimate of the bias-weighting. For the coin in this example, the best estimate of \(H\) eventually converges to 0.6, which, of course, was the value chosen to simulate the flips.

    +
    +
    +

    12.11. A few words on different priors

    +
      +

    • uniform

      • +

    • Gaussian

      • +

    • Jeffrey’s prior

      • +
    +

    Repeat the coin flipping experiment with other priors.

    +
    +
    +

    12.12. Bayesian parameter estimation (single parameter)

    +

    We will now consider the very important task of model parameter estimation using statistical inference. +[CF 1: maybe stress that model parameters are not random variables, and the meaning of parameter estimation is therefore very different between frequentist and bayesian approaches.]

    +

    Throughout this section we will consider a specific example that involves a model with a single parameter: “Measured flux from a star”.

    +
    +

    12.12.1. Example: Measured flux from a star

    +

    Adapted from the blog Pythonic Perambulations by Jake VanderPlas.

    +

    Imagine that we point our telescope to the sky, and observe the light coming from a single star. For the time being, we’ll assume that the star’s true flux is constant with time, i.e. that is it has a fixed value \(F_\mathrm{true}\) (we’ll also ignore effects like sky noise and other sources of systematic error). We’ll assume that we perform a series of \(N\) measurements with our telescope, where the ith measurement reports the observed photon flux \(F_i\) and error \(e_i\)2. +The question is, given this set of measurements \(D = \{F_i, e_i\}\), what is our best estimate of the true flux \(F_\mathrm{true}\)?

    +

    Because the measurements are number counts, a Poisson distribution is a good approximation to the measurement process:

    +
    +
    +
    np.random.seed(1)      # for repeatability
+F_true = 1000          # true flux, say number of photons measured in 1 second
+N = 50                 # number of measurements
+F = stats.poisson(F_true).rvs(N)
+                       # N measurements of the flux
+e = np.sqrt(F)         # errors on Poisson counts estimated via square root
+
    +
    +
    +
    +

    Now let’s make a simple visualization of the “observed” data, see Fig. fig:flux.

    +
    +
    +
    fig, ax = plt.subplots()
+ax.errorbar(F, np.arange(N), xerr=e, fmt='ok', ecolor='gray', alpha=0.5)
+ax.vlines([F_true], 0, N, linewidth=5, alpha=0.2)
+ax.set_xlabel("Flux");ax.set_ylabel("measurement number");
+
    +
    +
    +
    + + +
    +

    Single photon counts (flux measurements).

    + + +

    These measurements each have a different error \(e_i\) which is estimated from Poisson statistics using the standard square-root rule. In this toy example we already know the true flux \(F_\mathrm{true}\), but the question is this: given our measurements and errors, what is our best estimate of the true flux?

    +

    Let’s take a look at the frequentist and Bayesian approaches to solving this.

    +
    +
    +

    12.12.2. Simple Photon Counts: Frequentist Approach

    +

    We’ll start with the classical frequentist maximum likelihood approach. Given a single observation \(D_i = (F_i, e_i)\), we can compute the probability distribution of the measurement given the true flux Ftrue given our assumption of Gaussian errors

    + +
    +
    +\[ +\begin{equation} +p(D_i | F_\mathrm{true}, I) = \frac{1}{\sqrt{2\pi e_i^2}} \exp \left( \frac{-(F_i-F_\mathrm{true})^2}{2e_i^2} \right). +\label{_auto2} \tag{5} +\end{equation} +\]
    +

    This should be read “the probability of \(D_i\) given \(F_\mathrm{true}\) +equals …”. You should recognize this as a normal distribution with mean \(F_\mathrm{true}\) and standard deviation \(e_i\).

    +

    We construct the likelihood function by computing the product of the probabilities for each data point

    + +
    +
    +\[ +\begin{equation} +\mathcal{L}(D | F_\mathrm{true}, I) = \prod_{i=1}^N p(D_i | F_\mathrm{true}, I), +\label{_auto3} \tag{6} +\end{equation} +\]
    +

    here \(D = \{D_i\}\) represents the entire set of measurements. Because the value of the likelihood can become very small, it is often more convenient to instead compute the log-likelihood. Combining the previous two equations and computing the log, we have

    + +
    +
    +\[ +\begin{equation} +\log\mathcal{L} = -\frac{1}{2} \sum_{i=1}^N \left[ \log(2\pi e_i^2) + \frac{(F_i-F_\mathrm{true})^2}{e_i^2} \right]. +\label{_auto4} \tag{7} +\end{equation} +\]
    +

    What we’d like to do is determine \(F_\mathrm{true}\) such that the likelihood is maximized. For this simple problem, the maximization can be computed analytically (i.e. by setting \(d\log\mathcal{L}/d F_\mathrm{true} = 0\)). This results in the following observed estimate of \(F_\mathrm{true}\)

    + +
    +
    +\[ +\begin{equation} +F_\mathrm{est} = \sum_{i=1}^N w_i F_i; \quad w_i = 1/e_i^2. +\label{_auto5} \tag{8} +\end{equation} +\]
    +

    Notice that in the special case of all errors \(e_i\) being equal, this reduces to

    + +
    +
    +\[ +\begin{equation} +F_\mathrm{est} = \frac{1}{N} \sum_{i=1} F_i. +\label{_auto6} \tag{9} +\end{equation} +\]
    +

    That is, in agreement with intuition, \(F_\mathrm{est}\) is simply the mean of the observed data when errors are equal.

    +

    We can go further and ask what the error of our estimate is. In the frequentist approach, this can be accomplished by fitting a Gaussian approximation to the likelihood curve at maximum; in this simple case this can also be solved analytically (the sum of Gaussians is also a Gaussian). It can be shown that the standard deviation of this Gaussian approximation is

    + +
    +
    +\[ +\begin{equation} +\sigma_\mathrm{est} = \sum_{i=1}^N w_i. +\label{_auto7} \tag{10} +\end{equation} +\]
    +

    These results are fairly simple calculations; let’s evaluate them for our toy dataset:

    +
    +
    +
    w=1./e**2
+print("""
+F_true = {0}
+F_est = {1:.0f} +/- {2:.0f} (based on {3} measurements) """\
+          .format(F_true, (w * F).sum() / w.sum(), w.sum() ** -0.5, N))
+
    +
    +
    +
    +

    F_true = 1000 +F_est = 998 +/- 4 (based on 50 measurements)

    +

    We find that for 50 measurements of the flux, our estimate has an error of about 0.4% and is consistent with the input value.

    +
    +
    +

    12.12.3. Simple Photon Counts: Bayesian Approach

    +

    The Bayesian approach, as you might expect, begins and ends with probabilities. Our hypothesis is that the star has a constant flux \(F_\mathrm{true}\). It recognizes that what we fundamentally want to compute is our knowledge of the parameters in question given the data and other information (such as our knowledge of uncertainties for the observed values), i.e. in this case, \(p(F_\mathrm{true} | D,I)\). +Note that this formulation of the problem is fundamentally contrary to the frequentist philosophy, which says that probabilities have no meaning for model parameters like \(F_\mathrm{true}\). Nevertheless, within the Bayesian philosophy this is perfectly acceptable.

    +

    To compute this result, Bayesians next apply Bayes’ Theorem (1). +If we set the prior \(p(F_\mathrm{true}|I) \propto 1\) (a flat prior), we find +\(p(F_\mathrm{true}|D,I) \propto p(D | F_\mathrm{true},I) \equiv \mathcal{L}(D | F_\mathrm{true},I)\) +and the Bayesian probability is maximized at precisely the same value as the frequentist result! So despite the philosophical differences, we see that (for this simple problem at least) the Bayesian and frequentist point estimates are equivalent.

    +
    +
    +

    12.12.4. A note about priors

    +

    The prior allows inclusion of other information into the computation, which becomes very useful in cases where multiple measurement strategies are being combined to constrain a single model. The necessity to specify a prior, however, is one of the more controversial pieces of Bayesian analysis. +A frequentist will point out that the prior is problematic when no true prior information is available. Though it might seem straightforward to use a noninformative prior like the flat prior mentioned above, there are some [surprisingly subtleties](http://normaldeviate.wordpress.com/2013/07/13/lost-causes-in-statistics-ii-noninformative- priors/comment-page-1/) involved. It turns out that in many situations, a truly noninformative prior does not exist! Frequentists point out that the subjective choice of a prior which necessarily biases your result has no place in statistical data analysis. +A Bayesian would counter that frequentism doesn’t solve this problem, but simply skirts the question. Frequentism can often be viewed as simply a special case of the Bayesian approach for some (implicit) choice of the prior: a Bayesian would say that it’s better to make this implicit choice explicit, even if the choice might include some subjectivity.

    +
    +
    +

    12.12.5. Simple Photon Counts: Bayesian approach in practice

    +

    Leaving these philosophical debates aside for the time being, let’s address how Bayesian results are generally computed in practice. For a one parameter problem like the one considered here, it’s as simple as computing the posterior probability \(p(F_\mathrm{true} | D,I)\) as a function of \(F_\mathrm{true}\): this is the distribution reflecting our knowledge of the parameter \(F_\mathrm{true}\). +But as the dimension of the model grows, this direct approach becomes increasingly intractable. For this reason, Bayesian calculations often depend on sampling methods such as Markov Chain Monte Carlo (MCMC). For this practical example, let us apply an MCMC approach using Dan Foreman-Mackey’s emcee package. Keep in mind here that the goal is to generate a set of points drawn from the posterior probability distribution, and to use those points to determine the answer we seek. +To perform this MCMC, we start by defining Python functions for the prior \(p(F_\mathrm{true} | I)\), the likelihood \(p(D | F_\mathrm{true},I)\), and the posterior \(p(F_\mathrm{true} | D,I)\), noting that none of these need be properly normalized. Our model here is one-dimensional, but to handle multi-dimensional models we’ll define the model in terms of an array of parameters \(\boldsymbol{\alpha}\), which in this case is \(\boldsymbol{\alpha} = [F_\mathrm{true}]\)

    +
    +
    +
    def log_prior(alpha):
+    return 0 # flat prior
+
+def log_likelihood(alpha, F, e):
+    return -0.5 * np.sum(np.log(2 * np.pi * e ** 2) \
+                             + (F - alpha[0]) ** 2 / e ** 2)
+                             
+def log_posterior(alpha, F, e):
+    return log_prior(alpha) + log_likelihood(alpha, F, e)
+
    +
    +
    +
    +

    Now we set up the problem, including generating some random starting guesses for the multiple chains of points.

    +
    +
    +
    ndim = 1      # number of parameters in the model
+nwalkers = 50 # number of MCMC walkers
+nburn = 1000  # "burn-in" period to let chains stabilize
+nsteps = 2000 # number of MCMC steps to take
+# we'll start at random locations between 0 and 2000
+starting_guesses = 2000 * np.random.rand(nwalkers, ndim)
+sampler = emcee.EnsembleSampler(nwalkers, ndim, log_posterior, args=[F,e])
+sampler.run_mcmc(starting_guesses, nsteps)
+# Shape of sampler.chain  = (nwalkers, nsteps, ndim)
+# Flatten the sampler chain and discard burn-in points:
+samples = sampler.chain[:, nburn:, :].reshape((-1, ndim))
+
    +
    +
    +
    +

    If this all worked correctly, the array sample should contain a series of 50,000 points drawn from the posterior. Let’s plot them and check. See results in Fig. fig:flux-bayesian.

    +
    +
    +
    fig, ax = plt.subplots()
+ax.hist(samples, bins=50, histtype="stepfilled", alpha=0.3, normed=True)
+ax.set_xlabel(r'$F_\mathrm{est}$')
+ax.set_ylabel(r'$p(F_\mathrm{est}|D,I)$')
+
    +
    +
    +
    + + +
    +

    Bayesian posterior pdf (represented by a histogram of MCMC samples) from flux measurements.

    + + +
    +
    +

    12.12.6. Best estimates and confidence intervals

    +

    The posterior distribution from our Bayesian data analysis is the key quantity that encodes our inference about the values of the model parameters, given the data and the relevant background information. Often, however, we wish to summarize this result with just a few numbers: the best estimate and a measure of its reliability.

    +

    There are a few different options for this. The choice of the most appropriate one depends mainly on the shape of the posterior distribution:

    +

    Symmetric posterior pdfs: Since the probability (density) associated with any particular value of the parameter is a measure of how much we believe that it lies in the neighbourhood of that point, our best estimate is given by the maximum of the posterior pdf. If we denote the quantity of interest by \(X\), with a posterior pdf \(P =p(X|D,I)\), then the best estimate of its value \(X_0\) is given by the condition \(dP/dX|_{X=X_0}=0\). Strictly speaking, we should also check the sign of the second derivative to ensure that \(X_0\) represents a maximum.

    +

    To obtain a measure of the reliability of this best estimate, we need to look at the width or spread of the posterior pdf about \(X_0\). When considering the behaviour of any function in the neighbourhood of a particular point, it is often helpful to carry out a Taylor series expansion; this is simply a standard tool for (locally) approximating a complicated function by a low-order polynomial. The linear term is zero at the maximum and the quadratic term is often the dominating one determining the width of the posterior pdf. Ignoring all the higher-order terms we arrive at the Gaussian approximation

    + +
    +
    +\[ +\begin{equation} +p(X|D,I) \approx \frac{1}{\sigma\sqrt{2\pi}} \exp \left[ -\frac{(x-\mu)^2}{2\sigma^2} \right], +\label{_auto8} \tag{11} +\end{equation} +\]
    +

    where the mean \(\mu = X_0\) and the variance \(\sigma = \left( - \left. \frac{d^2L}{dX^2} \right|_{X_0} \right)^{-1/2}\), where \(L\) is the logarithm of the posterior \(P\). Our inference about the quantity of interest is conveyed very concisely, therefore, by the statement \(X = X_0 \pm \sigma\), and

    +
    +\[\]
    +

    p(X_0-\sigma < X < X_0+\sigma | D,I) = \int_{X_0-\sigma}^{X_0+\sigma} p(X|D,I) dX \approx 0.67. +$\( +\)$

    +

    Asymmetric posterior pdfs: While the maximum of the posterior (\(X_0\)) can still be regarded as giving the best estimate, the true value is now more likely to be on one side of this rather than the other. Alternatively one can compute the mean value, \(\langle X \rangle = \int X p(X|D,I) dX\), although this tends to overemphasise very long tails. The best option is probably a compromise that can be employed when having access to a large sample from the posterior (as provided by an MCMC), namely to give the median of this ensamble.

    +

    Furthermore, the concept of an error-bar does not seem appropriate in this case, as it implicitly entails the idea of symmetry. A good way of expressing the reliability with which a parameter can be inferred, for an asymmetric posterior pdf, is rather through a confidence interval. Since the area under the posterior pdf between \(X_1\) and \(X_2\) is proportional to how much we believe that \(X\) lies in that range, the shortest interval that encloses 67% of the area represents a sensible measure of the uncertainty of the estimate. Obviously we can choose to provide some other degree-of-belief that we think is relevant for the case at hand. Assuming that the posterior pdf has been normalized, to have unit area, we need to find \(X_1\) and \(X_2\) such that:

    +
    +\[\]
    +

    p(X_1 < X < X_2 | D,I) = \int_{X_1}^{X_2} p(X|D,I) dX \approx 0.67, +$\( +\)$

    +

    where the difference \(X_2 - X_1\) is as small as possible. The region \(X_1 < X < X_2\) is then called the shortest 67% confidence interval.

    +

    Multimodal posterior pdfs: We can sometimes obtain posteriors which are multimodal; i.e. contains several disconnected regions with large probabilities. There is no difficulty when one of the maxima is very much larger than the others: we can simply ignore the subsidiary solutions, to a good approximation, and concentrate on the global maximum. The problem arises when there are several maxima of comparable magnitude. What do we now mean by a best estimate, and how should we quantify its reliability? The idea of a best estimate and an error-bar, or even a confidence interval, is merely an attempt to summarize the posterior with just two or three numbers; sometimes this just can’t be done, and so these concepts are not valid. For the bimodal case we might be able to characterize the posterior in terms of a few numbers: two best estimates and their associated error-bars, or disjoint confidence intervals. For a general multimodal pdf, the most honest thing we can do is just display the posterior itself.

    +
    +
    +

    12.12.7. Simple Photon Counts: Best estimates and confidence intervals

    +

    To compute these numbers for our example, you would run:

    +
    +
    +
    sampper=np.percentile(samples, [2.5, 16.5, 50, 83.5, 97.5],axis=0).flatten()
+print("""
+F_true = {0}
+Based on {1} measurements the posterior point estimates are:
+...F_est = {2:.0f} +/- {3:.0f}
+or using credible intervals:
+...F_est = {4:.0f}          (posterior median) 
+...F_est in [{5:.0f}, {6:.0f}] (67% credible interval) 
+...F_est in [{7:.0f}, {8:.0f}] (95% credible interval) """\
+          .format(F_true, N, np.mean(samples), np.std(samples), \
+                      sampper[2], sampper[1], sampper[3], sampper[0], sampper[4]))
+
    +
    +
    +
    +

    F_true = 1000 +Based on 50 measurements the posterior point estimates are: +...F_est = 998 +/- 4 +or using credible intervals: +...F_est = 998          (posterior median)
    +...F_est in [993, 1002] (67% credible interval)
    +...F_est in [989, 1006] (95% credible interval)

    +

    In this particular example, the posterior pdf is actually a Gaussian (since it is constructed as a product of Gaussians), and the mean and variance from the quadratic approximation will agree exactly with the frequentist approach.

    +

    From this final result you might come away with the impression that the Bayesian method is unnecessarily complicated, and in this case it certainly is. Using an MCMC sampler to characterize a one-dimensional normal distribution is a bit like using the Death Star to destroy a beach ball, but we did this here because it demonstrates an approach that can scale to complicated posteriors in many, many dimensions, and can provide nice results in more complicated situations where an analytic likelihood approach is not possible.

    +

    Furthermore, as data and models grow in complexity, the two approaches can diverge greatly.

    +
    +
    +
    +

    12.13. Bayesian parameter estimation (multiple parameters, covariance)

    +
      +

    • multidimensional posterior pdf:s

      • +

    • nuisance parameters (e.g. background subtraction?)

      • +

    • corner plots, covariance, correlations

      • +

    • best example?

      • +
    +
    +
    +

    12.14. Bayesian model selection

    +
      +

    • Bayesian evidence

      • +

    • Occam’s razor

      • +

    • Best example? How many spectral lines are there?

      • +
    +
    +
    +
    1
    +

    Sivia, Devinderjit, and John Skilling. Data Analysis : A Bayesian Tutorial, OUP Oxford, 2006

    +
    +
    2
    +

    We’ll make the reasonable assumption that errors are Gaussian. In a Frequentist perspective, \(e_i\) is the standard deviation of the results of a single measurement event in the limit of repetitions of that event. In the Bayesian perspective, \(e_i\) is the standard deviation of the (Gaussian) probability distribution describing our knowledge of that particular measurement given its observed value.

    +
    +
    +
    +
    + + + + +
    + +
    +
    + + +
    + + 9. Recurrent Neural Networks + +
    +
    +
    +

    + + By Morten Hjorth-Jensen
    + + © Copyright 2020.
    +

    +
    +
    +
    + + +
    +
    + + + + + \ No newline at end of file diff --git a/doc/src/LectureNotes/_build/html/chapter2.html b/doc/src/LectureNotes/_build/html/chapter2.html index 74589ccf1..7d3bd7ab6 100644 --- a/doc/src/LectureNotes/_build/html/chapter2.html +++ b/doc/src/LectureNotes/_build/html/chapter2.html @@ -125,6 +125,26 @@ 8. Convolutional Neural Networks +
  • + + 9. Recurrent Neural Networks + +
    • +
  • + + 10. Solving ODEs with Deep Learning + +
    • +
  • + + 11. Data Analysis and Machine Learning: + +
    • +
  • + + 12. Elements of Bayesian theory and Bayesian Neural Networks + +
    • @@ -1121,27 +1141,27 @@ uncorrelated.

    -
    2.1453055662574343
-[[12.86665228 10.42534378 14.91184964  7.05197852 10.73905189  9.78942024
-   2.99691286  8.42384711 18.16377739 12.166796  ]
- [10.42534378  8.44724724 12.082487    5.71394166  8.70143262  7.93198333
-   2.42828097  6.82551297 14.71739653  9.85827768]
- [14.91184964 12.082487   17.28213796  8.17291407 12.44606006 11.34548129
-   3.47328217  9.76284574 21.05097048 14.10074886]
- [ 7.05197852  5.71394166  8.17291407  3.8650614   5.88587937  5.36540351
-   1.64255353  4.61695766  9.95523663  6.6684    ]
- [10.73905189  8.70143262 12.44606006  5.88587937  8.96326666  8.17066394
-   2.50135015  7.03089888 15.16025642 10.15492225]
- [ 9.78942024  7.93198333 11.34548129  5.36540351  8.17066394  7.44814941
-   2.28016105  6.40917136 13.819667    9.25694395]
- [ 2.99691286  2.42828097  3.47328217  1.64255353  2.50135015  2.28016105
-   0.69804379  1.96209046  4.23072426  2.83390167]
- [ 8.42384711  6.82551297  9.76284574  4.61695766  7.03089888  6.40917136
-   1.96209046  5.51512534 11.89189544  7.96564848]
- [18.16377739 14.71739653 21.05097048  9.95523663 15.16025642 13.819667
-   4.23072426 11.89189544 25.64169777 17.17579438]
- [12.166796    9.85827768 14.10074886  6.6684     10.15492225  9.25694395
-   2.83390167  7.96564848 17.17579438 11.50500702]]
+
    2.599907486910053
+[[ 0.2319431   1.78702425  2.33555576  1.01612686  0.80805005  1.10075065
+   1.70915039  0.60318841  0.62019593  1.49305147]
+ [ 1.78702425 13.76827179 17.99447691  7.82883097  6.22568649  8.48082175
+  13.16828638  4.64731357  4.77834938 11.5033349 ]
+ [ 2.33555576 17.99447691 23.51792615 10.23191002  8.13667637 11.08403099
+  17.21032448  6.07381797  6.24507555 15.03431202]
+ [ 1.01612686  7.82883097 10.23191002  4.45158225  3.54001199  4.82231328
+   7.48767091  2.64252717  2.71703596  6.54095632]
+ [ 0.80805005  6.22568649  8.13667637  3.54001199  2.81510801  3.83482678
+   5.95438729  2.10140515  2.16065644  5.20153566]
+ [ 1.10075065  8.48082175 11.08403099  4.82231328  3.83482678  5.22391907
+   8.11124963  2.86259878  2.94331271  7.08569196]
+ [ 1.70915039 13.16828638 17.21032448  7.48767091  5.95438729  8.11124963
+  12.59444676  4.44479575  4.57012136 11.00204953]
+ [ 0.60318841  4.64731357  6.07381797  2.64252717  2.10140515  2.86259878
+   4.44479575  1.56864447  1.61287402  3.88281152]
+ [ 0.62019593  4.77834938  6.24507555  2.71703596  2.16065644  2.94331271
+   4.57012136  1.61287402  1.65835067  3.9922914 ]
+ [ 1.49305147 11.5033349  15.03431202  6.54095632  5.20153566  7.08569196
+  11.00204953  3.88281152  3.9922914   9.61098936]]
    @@ -1449,15 +1469,15 @@ more practically oriented methods like the blocking technique.

    -
    0.021442867969978376
-4.117524456004779
-0.06378292739195897
-0.9888063524440697 9.919191379006188 17.823611314267136
-2.969136019500522 3.214525177223236 9.842379203904565
-[[ 0.98880635  2.96913602  3.21452518]
- [ 2.96913602  9.91919138  9.8423792 ]
- [ 3.21452518  9.8423792  17.82361131]]
-[25.24964689  0.08489291  3.39706925]
+
    0.09525604390305954
+4.231031843972674
+0.04124568972977915
+0.831883346005529 8.296268771337688 5.337726521457732
+2.4701318674515136 1.7159602029423742 5.229477584776403
+[[0.83188335 2.47013187 1.7159602 ]
+ [2.47013187 8.29626877 5.22947758]
+ [1.7159602  5.22947758 5.33772652]]
+[12.99430696  0.07864499  1.39292669]
    @@ -1864,7 +1884,7 @@ assumption for approximating \(\sigma
    -
    -0.04067115014325762 0.9243683507629109
+
    0.054334900100894076 1.0092227336467725
 
    
 
    
 _images/chapter2_184_1.png
diff --git a/doc/src/LectureNotes/_build/html/chapter3.html b/doc/src/LectureNotes/_build/html/chapter3.html
index f50739533..b206f7591 100644
--- a/doc/src/LectureNotes/_build/html/chapter3.html
+++ b/doc/src/LectureNotes/_build/html/chapter3.html
@@ -125,6 +125,26 @@
    8. Convolutional Neural Networks
   
  
+ 
  • 
+  
+   9. Recurrent Neural Networks
+  
+ 
    • 
+ 
  • 
+  
+   10. Solving ODEs with Deep Learning
+  
+ 
    • 
+ 
  • 
+  
+   11. Data Analysis and Machine Learning:
+  
+ 
    • 
+ 
  • 
+  
+   12. Elements of Bayesian theory and Bayesian Neural Networks
+  
+ 
    • 
 
 
 
@@ -650,8 +670,8 @@ developed in the 1970s, namely EISPACK and LINPACK.  We describe them shortly he
    -
    [-0.36370241  0.57281693  0.31446735 -1.74797657 -1.47723222 -1.24113195
- -0.714415   -1.26679543 -0.27301697  0.6793838 ]
+
    [-2.57325284 -1.11383985  0.52678895 -0.47530983 -0.1807752   0.52810513
+ -1.50448048  0.33629261 -1.41402417 -0.39264297]
    diff --git a/doc/src/LectureNotes/_build/html/chapter4.html b/doc/src/LectureNotes/_build/html/chapter4.html index 4c1ebec41..acc3b9e60 100644 --- a/doc/src/LectureNotes/_build/html/chapter4.html +++ b/doc/src/LectureNotes/_build/html/chapter4.html @@ -125,6 +125,26 @@ 8. Convolutional Neural Networks +
  • + + 9. Recurrent Neural Networks + +
    • +
  • + + 10. Solving ODEs with Deep Learning + +
    • +
  • + + 11. Data Analysis and Machine Learning: + +
    • +
  • + + 12. Elements of Bayesian theory and Bayesian Neural Networks + +
    • @@ -1756,13 +1776,13 @@ but now splitting the data into a training set and a test set.

    Training R2
-0.9999854211447763
+0.9999863998248437
 Training MSE
-6.298065171189058
+6.805697972778246
 Test R2
-0.9999855160996954
+0.9999773431421539
 Test MSE
-7.1872710004915445
+4.962804899252701
    @@ -1893,7 +1913,7 @@ dtype: int64
    -
    <matplotlib.axes._subplots.AxesSubplot at 0x7f9dd7c37eb0>
+
    <matplotlib.axes._subplots.AxesSubplot at 0x7fe6839db970>
 
    
 
    
 _images/chapter4_131_1.png
@@ -2179,27 +2199,27 @@ techniques.
    
 
    MSE before scaling: 0.00
 R2 score before scaling 0.99
 Feature min values before scaling:
- [1.00000000e+00 1.00405815e-03 3.33637629e-03 1.00813277e-06
- 3.34991581e-06 1.11314068e-05 1.01222393e-09 3.36351028e-09
- 1.11765797e-08 3.71385616e-08 1.01633169e-12 3.37715992e-12
- 1.12219360e-11 3.72892756e-11 1.23908217e-10 1.02045611e-15
- 3.39086494e-15 1.12674763e-14 3.74406011e-14 1.24411055e-13
- 4.13404436e-13]
+ [1.00000000e+00 4.73875395e-04 1.62035199e-03 2.24557890e-07
+ 7.67844938e-07 2.62554056e-06 1.06412459e-10 3.63862824e-10
+ 1.24417907e-09 4.25429987e-09 5.04262460e-14 1.72425639e-13
+ 5.89585850e-13 2.01600803e-12 6.89346326e-12 2.38957572e-17
+ 8.17082679e-17 2.79390227e-16 9.55336603e-16 3.26664263e-15
+ 1.11698369e-14]
 Feature max values before scaling:
- [1.         0.99985941 0.99746704 0.99971883 0.99732681 0.99494051
- 0.99957828 0.99718659 0.99480062 0.99242037 0.99943775 0.99704639
- 0.99466076 0.99228084 0.98990661 0.99929723 0.99690622 0.99452092
- 0.99214133 0.98976744 0.98739922]
+ [1.         0.99909628 0.99839582 0.99819339 0.99749355 0.99679421
+ 0.9972913  0.9965921  0.9958934  0.99519518 0.99639003 0.99569147
+ 0.99499339 0.9942958  0.99359871 0.99548958 0.99479165 0.9940942
+ 0.99339724 0.99270078 0.9920048 ]
 Feature min values after scaling:
- [ 0.         -1.79147165 -1.7862625  -1.1584177  -1.16385875 -1.16903529
- -0.9111456  -0.91736698 -0.92355665 -0.92969742 -0.77335697 -0.77892437
- -0.78453895 -0.79019355 -0.79588019 -0.6835654  -0.68819847 -0.69289982
- -0.69766733 -0.70249839 -0.70738995]
+ [ 0.         -1.71950803 -1.74066137 -1.12098766 -1.13008641 -1.13882035
+ -0.88585974 -0.89326197 -0.90038244 -0.90721064 -0.75127414 -0.7579701
+ -0.76446674 -0.77075409 -0.77682281 -0.6609642  -0.66708788 -0.67306513
+ -0.67888673 -0.68454393 -0.69002844]
 Feature max values after scaling:
- [0.         1.702894   1.66035996 2.19865243 2.16709986 2.13442274
- 2.59980117 2.57660557 2.55254863 2.52759568 2.94821185 2.93060502
- 2.91238065 2.89350743 2.8739528  3.26264844 3.24876534 3.23444979
- 3.21967888 3.20442837 3.18867266]
+ [0.         1.75646602 1.64607128 2.30045841 2.19871569 2.09765001
+ 2.75230652 2.65433203 2.55670018 2.45952257 3.14599634 3.05079367
+ 2.95561862 2.86057207 2.76575615 3.49778287 3.40509201 3.31215604
+ 3.21906542 3.12591311 3.03279417]
 MSE after  scaling: 0.00
 R2 score for  scaled data: 0.99
 
    
@@ -2789,10 +2809,10 @@ covariance matrix through the np.linalg.eig() function.
    -
    0.012893004091480358
-4.036145788156587
-[[0.84855665 2.43974599]
- [2.43974599 8.11534493]]
+
    0.13175867934658783
+4.345411944186196
+[[ 0.99370801  2.9978939 ]
+ [ 2.9978939  10.13547884]]
    @@ -2832,10 +2852,10 @@ a more brute force way. Here we scale the mean values for each column of the des
    -
    0.08534074521320528
-1.494502745126961
-[[1.         0.54933577]
- [0.54933577 1.        ]]
+
    0.07991186451576213
+1.435691712612069
+[[1.         0.69854547]
+ [0.69854547 1.        ]]
    @@ -2867,30 +2887,30 @@ this matrix we easily see that it is a positive definite matrix.

    -
    [[ 0.01162555 -0.25149407]
- [ 0.32741231  1.04530907]
- [-0.5034199  -1.57100552]
- [ 1.67655824  6.4270278 ]
- [-2.84730394 -8.72396496]
- [ 0.71213034  4.19282793]
- [ 0.6767474   0.58026128]
- [ 0.05853864 -0.83340221]
- [-0.2484568  -1.90992384]
- [ 0.13616818  1.04436453]]
+
    [[-0.35269644 -0.0359786 ]
+ [ 0.91157022  2.38965729]
+ [-0.68652791 -1.88459927]
+ [ 0.73533968  3.47606953]
+ [-0.59301581 -2.13696995]
+ [-0.37142517 -1.24766758]
+ [-0.12619024 -0.45164224]
+ [-1.74004988 -5.63952222]
+ [ 1.7194963   3.93948143]
+ [ 0.50349923  1.59117161]]
           0         1
-0  0.011626 -0.251494
-1  0.327412  1.045309
-2 -0.503420 -1.571006
-3  1.676558  6.427028
-4 -2.847304 -8.723965
-5  0.712130  4.192828
-6  0.676747  0.580261
-7  0.058539 -0.833402
-8 -0.248457 -1.909924
-9  0.136168  1.044365
-        0       1
-0  1.0000  0.9645
-1  0.9645  1.0000
+0 -0.352696 -0.035979
+1  0.911570  2.389657
+2 -0.686528 -1.884599
+3  0.735340  3.476070
+4 -0.593016 -2.136970
+5 -0.371425 -1.247668
+6 -0.126190 -0.451642
+7 -1.740050 -5.639522
+8  1.719496  3.939481
+9  0.503499  1.591172
+          0         1
+0  1.000000  0.970341
+1  0.970341  1.000000
    @@ -2950,37 +2970,37 @@ this matrix we easily see that it is a positive definite matrix.

         0         1         2         3         4         5         6         7   \
 0   0.0  0.000000  0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   
-1   0.0  0.067129  0.071920  0.066488  0.067965  0.069289  0.058136  0.058890   
-2   0.0  0.071920  0.077903  0.072255  0.074320  0.076177  0.063662  0.064746   
-3   0.0  0.066488  0.072255  0.070120  0.072252  0.074165  0.064037  0.065174   
-4   0.0  0.067965  0.074320  0.072252  0.074718  0.076940  0.066275  0.067613   
-5   0.0  0.069289  0.076177  0.074165  0.076940  0.079448  0.068289  0.069815   
-6   0.0  0.058136  0.063662  0.064037  0.066275  0.068289  0.060371  0.061607   
-7   0.0  0.058890  0.064746  0.065174  0.067613  0.069815  0.061607  0.062970   
-8   0.0  0.059633  0.065798  0.066265  0.068894  0.071275  0.062784  0.064269   
-9   0.0  0.060376  0.066836  0.067327  0.070137  0.072691  0.063919  0.065523   
-10  0.0  0.049985  0.054935  0.056831  0.058951  0.060864  0.054874  0.056079   
-11  0.0  0.050516  0.055671  0.057601  0.059851  0.061888  0.055712  0.057004   
-12  0.0  0.051063  0.056417  0.058372  0.060747  0.062904  0.056542  0.057918   
-13  0.0  0.051629  0.057177  0.059149  0.061648  0.063923  0.057370  0.058831   
-14  0.0  0.052218  0.057957  0.059939  0.062558  0.064951  0.058203  0.059747   
+1   0.0  0.080229  0.080679  0.087554  0.086417  0.085175  0.083141  0.081602   
+2   0.0  0.080679  0.082259  0.089654  0.089067  0.088307  0.086044  0.084862   
+3   0.0  0.087554  0.089654  0.100625  0.100093  0.099390  0.098651  0.097358   
+4   0.0  0.086417  0.089067  0.100093  0.099970  0.099642  0.098705  0.097743   
+5   0.0  0.085175  0.088307  0.099390  0.099642  0.099662  0.098566  0.097913   
+6   0.0  0.083141  0.086044  0.098651  0.098705  0.098566  0.098847  0.098003   
+7   0.0  0.081602  0.084862  0.097358  0.097743  0.097913  0.098003  0.097449   
+8   0.0  0.080151  0.083732  0.096133  0.096822  0.097279  0.097201  0.096917   
+9   0.0  0.078794  0.082667  0.094987  0.095957  0.096680  0.096453  0.096419   
+10  0.0  0.076701  0.079987  0.093012  0.093523  0.093835  0.094703  0.094277   
+11  0.0  0.075255  0.078808  0.091666  0.092450  0.093019  0.093698  0.093521   
+12  0.0  0.073932  0.077731  0.090440  0.091477  0.092285  0.092793  0.092846   
+13  0.0  0.072727  0.076752  0.089333  0.090604  0.091633  0.091986  0.092253   
+14  0.0  0.071634  0.075869  0.088337  0.089826  0.091062  0.091273  0.091739   
 
           8         9         10        11        12        13        14  
 0   0.000000  0.000000  0.000000  0.000000  0.000000  0.000000  0.000000  
-1   0.059633  0.060376  0.049985  0.050516  0.051063  0.051629  0.052218  
-2   0.065798  0.066836  0.054935  0.055671  0.056417  0.057177  0.057957  
-3   0.066265  0.067327  0.056831  0.057601  0.058372  0.059149  0.059939  
-4   0.068894  0.070137  0.058951  0.059851  0.060747  0.061648  0.062558  
-5   0.071275  0.072691  0.060864  0.061888  0.062904  0.063923  0.064951  
-6   0.062784  0.063919  0.054874  0.055712  0.056542  0.057370  0.058203  
-7   0.064269  0.065523  0.056079  0.057004  0.057918  0.058831  0.059747  
-8   0.065687  0.067056  0.057224  0.058233  0.059231  0.060226  0.061224  
-9   0.067056  0.068537  0.058326  0.059418  0.060497  0.061573  0.062652  
-10  0.057224  0.058326  0.050807  0.051634  0.052448  0.053259  0.054070  
-11  0.058233  0.059418  0.051634  0.052522  0.053398  0.054270  0.055142  
-12  0.059231  0.060497  0.052448  0.053398  0.054336  0.055268  0.056202  
-13  0.060226  0.061573  0.053259  0.054270  0.055268  0.056262  0.057257  
-14  0.061224  0.062652  0.054070  0.055142  0.056202  0.057257  0.058313  
+1   0.080151  0.078794  0.076701  0.075255  0.073932  0.072727  0.071634  
+2   0.083732  0.082667  0.079987  0.078808  0.077731  0.076752  0.075869  
+3   0.096133  0.094987  0.093012  0.091666  0.090440  0.089333  0.088337  
+4   0.096822  0.095957  0.093523  0.092450  0.091477  0.090604  0.089826  
+5   0.097279  0.096680  0.093835  0.093019  0.092285  0.091633  0.091062  
+6   0.097201  0.096453  0.094703  0.093698  0.092793  0.091986  0.091273  
+7   0.096917  0.096419  0.094277  0.093521  0.092846  0.092253  0.091739  
+8   0.096634  0.096370  0.093871  0.093345  0.092885  0.092491  0.092163  
+9   0.096370  0.096324  0.093496  0.093186  0.092926  0.092718  0.092564  
+10  0.093871  0.093496  0.091863  0.091208  0.090630  0.090128  0.089699  
+11  0.093345  0.093186  0.091208  0.090769  0.090391  0.090075  0.089820  
+12  0.092885  0.092926  0.090630  0.090391  0.090200  0.090057  0.089963  
+13  0.092491  0.092718  0.090128  0.090075  0.090057  0.090076  0.090133  
+14  0.092163  0.092564  0.089699  0.089820  0.089963  0.090133  0.090330
    @@ -3330,10 +3350,10 @@ number \(i\) is left out. Usin
    -
    Runtime: 0.410282 sec
+
    Runtime: 0.396214 sec
 Jackknife Statistics :
 original           bias      std. error
- 100.139        100.129        0.148994
+ 99.9535        99.9435        0.150002
    @@ -3475,10 +3495,10 @@ theorem.

    -
    Runtime: 2.02233 sec
+
    Runtime: 2.03201 sec
 Bootstrap Statistics :
 original           bias      std. error
- 100.067  14.9691         100.07        0.150582
+  100.07   15.041        100.069        0.149551
 
    
 
    
 
    ---------------------------------------------------------------------------
diff --git a/doc/src/LectureNotes/_build/html/chapter5.html b/doc/src/LectureNotes/_build/html/chapter5.html
index 884f12c3b..2fddb972a 100644
--- a/doc/src/LectureNotes/_build/html/chapter5.html
+++ b/doc/src/LectureNotes/_build/html/chapter5.html
@@ -125,6 +125,26 @@
    8. Convolutional Neural Networks
   
  
+ 
  • 
+  
+   9. Recurrent Neural Networks
+  
+ 
    • 
+ 
  • 
+  
+   10. Solving ODEs with Deep Learning
+  
+ 
    • 
+ 
  • 
+  
+   11. Data Analysis and Machine Learning:
+  
+ 
    • 
+ 
  • 
+  
+   12. Elements of Bayesian theory and Bayesian Neural Networks
+  
+ 
    • 
 
 
 
diff --git a/doc/src/LectureNotes/_build/html/chapter6.html b/doc/src/LectureNotes/_build/html/chapter6.html
index 8edbea2fb..b136b980b 100644
--- a/doc/src/LectureNotes/_build/html/chapter6.html
+++ b/doc/src/LectureNotes/_build/html/chapter6.html
@@ -125,6 +125,26 @@
    8. Convolutional Neural Networks
   
  
+ 
  • 
+  
+   9. Recurrent Neural Networks
+  
+ 
    • 
+ 
  • 
+  
+   10. Solving ODEs with Deep Learning
+  
+ 
    • 
+ 
  • 
+  
+   11. Data Analysis and Machine Learning:
+  
+ 
    • 
+ 
  • 
+  
+   12. Elements of Bayesian theory and Bayesian Neural Networks
+  
+ 
    • 
 
 
 
@@ -2464,6 +2484,14 @@ Accuracy score on test set:  0.07777777777777778
    +
    +
    <ipython-input-5-16b8e3cda33a>:4: RuntimeWarning: overflow encountered in exp
+  return 1/(1 + np.exp(-x))
+
    +
    +_images/chapter6_134_1.png +_images/chapter6_134_2.png +
    @@ -2499,6 +2527,335 @@ performance overall.

    +
    +
    /Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
+  warnings.warn(
+
    +
    +
    Learning rate  =  1e-05
+Lambda =  1e-05
+Accuracy score on test set:  0.18333333333333332
+
    +
    +
    /Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
+  warnings.warn(
+
    +
    +
    Learning rate  =  1e-05
+Lambda =  0.0001
+Accuracy score on test set:  0.18611111111111112
+
    +
    +
    /Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
+  warnings.warn(
+
    +
    +
    Learning rate  =  1e-05
+Lambda =  0.001
+Accuracy score on test set:  0.13055555555555556
+
    +
    +
    /Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
+  warnings.warn(
+
    +
    +
    Learning rate  =  1e-05
+Lambda =  0.01
+Accuracy score on test set:  0.24444444444444444
+
    +
    +
    /Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
+  warnings.warn(
+
    +
    +
    Learning rate  =  1e-05
+Lambda =  0.1
+Accuracy score on test set:  0.23333333333333334
+
    +
    +
    /Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
+  warnings.warn(
+
    +
    +
    Learning rate  =  1e-05
+Lambda =  1.0
+Accuracy score on test set:  0.12777777777777777
+
    +
    +
    /Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
+  warnings.warn(
+
    +
    +
    Learning rate  =  1e-05
+Lambda =  10.0
+Accuracy score on test set:  0.1527777777777778
+
    +
    +
    /Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
+  warnings.warn(
+
    +
    +
    Learning rate  =  0.0001
+Lambda =  1e-05
+Accuracy score on test set:  0.9111111111111111
+
    +
    +
    /Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
+  warnings.warn(
+
    +
    +
    Learning rate  =  0.0001
+Lambda =  0.0001
+Accuracy score on test set:  0.8888888888888888
+
    +
    +
    /Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
+  warnings.warn(
+
    +
    +
    Learning rate  =  0.0001
+Lambda =  0.001
+Accuracy score on test set:  0.8722222222222222
+
    +
    +
    /Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
+  warnings.warn(
+
    +
    +
    Learning rate  =  0.0001
+Lambda =  0.01
+Accuracy score on test set:  0.8305555555555556
+
    +
    +
    /Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
+  warnings.warn(
+
    +
    +
    Learning rate  =  0.0001
+Lambda =  0.1
+Accuracy score on test set:  0.8888888888888888
+
    +
    +
    /Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
+  warnings.warn(
+
    +
    +
    Learning rate  =  0.0001
+Lambda =  1.0
+Accuracy score on test set:  0.8805555555555555
+
    +
    +
    /Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
+  warnings.warn(
+
    +
    +
    Learning rate  =  0.0001
+Lambda =  10.0
+Accuracy score on test set:  0.8944444444444445
+
    +
    +
    /Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
+  warnings.warn(
+
    +
    +
    Learning rate  =  0.001
+Lambda =  1e-05
+Accuracy score on test set:  0.975
+
    +
    +
    /Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
+  warnings.warn(
+
    +
    +
    Learning rate  =  0.001
+Lambda =  0.0001
+Accuracy score on test set:  0.9777777777777777
+
    +
    +
    /Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
+  warnings.warn(
+
    +
    +
    Learning rate  =  0.001
+Lambda =  0.001
+Accuracy score on test set:  0.9805555555555555
+
    +
    +
    /Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
+  warnings.warn(
+
    +
    +
    Learning rate  =  0.001
+Lambda =  0.01
+Accuracy score on test set:  0.9861111111111112
+
    +
    +
    /Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
+  warnings.warn(
+
    +
    +
    Learning rate  =  0.001
+Lambda =  0.1
+Accuracy score on test set:  0.9805555555555555
+
    +
    +
    /Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
+  warnings.warn(
+
    +
    +
    Learning rate  =  0.001
+Lambda =  1.0
+Accuracy score on test set:  0.9777777777777777
+
    +
    +
    /Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
+  warnings.warn(
+
    +
    +
    Learning rate  =  0.001
+Lambda =  10.0
+Accuracy score on test set:  0.9444444444444444
+
    +
    +
    /Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
+  warnings.warn(
+
    +
    +
    Learning rate  =  0.01
+Lambda =  1e-05
+Accuracy score on test set:  0.9861111111111112
+
    +
    +
    Learning rate  =  0.01
+Lambda =  0.0001
+Accuracy score on test set:  0.9888888888888889
+
    +
    +
    Learning rate  =  0.01
+Lambda =  0.001
+Accuracy score on test set:  0.9888888888888889
+
    +
    +
    Learning rate  =  0.01
+Lambda =  0.01
+Accuracy score on test set:  0.9861111111111112
+
    +
    +
    Learning rate  =  0.01
+Lambda =  0.1
+Accuracy score on test set:  0.9888888888888889
+
    +
    +
    Learning rate  =  0.01
+Lambda =  1.0
+Accuracy score on test set:  0.9722222222222222
+
    +
    +
    Learning rate  =  0.01
+Lambda =  10.0
+Accuracy score on test set:  0.9527777777777777
+
    +
    +
    Learning rate  =  0.1
+Lambda =  1e-05
+Accuracy score on test set:  0.8777777777777778
+
    +
    +
    Learning rate  =  0.1
+Lambda =  0.0001
+Accuracy score on test set:  0.8388888888888889
+
    +
    +
    Learning rate  =  0.1
+Lambda =  0.001
+Accuracy score on test set:  0.8916666666666667
+
    +
    +
    Learning rate  =  0.1
+Lambda =  0.01
+Accuracy score on test set:  0.9111111111111111
+
+Learning rate  =  0.1
+Lambda =  0.1
+Accuracy score on test set:  0.9166666666666666
+
    +
    +
    Learning rate  =  0.1
+Lambda =  1.0
+Accuracy score on test set:  0.9083333333333333
+
    +
    +
    Learning rate  =  0.1
+Lambda =  10.0
+Accuracy score on test set:  0.7944444444444444
+
    +
    +
    Learning rate  =  1.0
+Lambda =  1e-05
+Accuracy score on test set:  0.09166666666666666
+
+Learning rate  =  1.0
+Lambda =  0.0001
+Accuracy score on test set:  0.11388888888888889
+
    +
    +
    Learning rate  =  1.0
+Lambda =  0.001
+Accuracy score on test set:  0.07777777777777778
+
+Learning rate  =  1.0
+Lambda =  0.01
+
    +
    +
    Accuracy score on test set:  0.14444444444444443
+
+Learning rate  =  1.0
+Lambda =  0.1
+Accuracy score on test set:  0.11944444444444445
+
    +
    +
    Learning rate  =  1.0
+Lambda =  1.0
+Accuracy score on test set:  0.1361111111111111
+
    +
    +
    Learning rate  =  1.0
+Lambda =  10.0
+Accuracy score on test set:  0.13055555555555556
+
+Learning rate  =  10.0
+Lambda =  1e-05
+Accuracy score on test set:  0.07777777777777778
+
+Learning rate  =  10.0
+Lambda =  0.0001
+Accuracy score on test set:  0.21388888888888888
+
    +
    +
    Learning rate  =  10.0
+Lambda =  0.001
+Accuracy score on test set:  0.10555555555555556
+
+Learning rate  =  10.0
+Lambda =  0.01
+Accuracy score on test set:  0.07777777777777778
+
+Learning rate  =  10.0
+Lambda =  0.1
+Accuracy score on test set:  0.125
+
    +
    +
    Learning rate  =  10.0
+Lambda =  1.0
+Accuracy score on test set:  0.1527777777777778
+
    +
    +
    Learning rate  =  10.0
+Lambda =  10.0
+Accuracy score on test set:  0.17777777777777778
+
    +
    +
    @@ -2542,6 +2899,10 @@ performance overall.

    +
    +_images/chapter6_138_0.png +_images/chapter6_138_1.png +
    @@ -2580,6 +2941,14 @@ how simple solving a machine learning problem can be.

    +
    +
      File "<ipython-input-13-6ea927cc6e88>", line 1
+    pip3 install tensorflow
+         ^
+SyntaxError: invalid syntax
+
    +
    +

    and/or if you use anaconda, just write (or install from the graphical user interface)

    diff --git a/doc/src/LectureNotes/_build/html/chapter7.html b/doc/src/LectureNotes/_build/html/chapter7.html index c6427dccb..c2c669c6d 100644 --- a/doc/src/LectureNotes/_build/html/chapter7.html +++ b/doc/src/LectureNotes/_build/html/chapter7.html @@ -125,6 +125,26 @@ 8. Convolutional Neural Networks +
  • + + 9. Recurrent Neural Networks + +
    • +
  • + + 10. Solving ODEs with Deep Learning + +
    • +
  • + + 11. Data Analysis and Machine Learning: + +
    • +
  • + + 12. Elements of Bayesian theory and Bayesian Neural Networks + +
    • diff --git a/doc/src/LectureNotes/_build/html/chapter8.html b/doc/src/LectureNotes/_build/html/chapter8.html index 6518e1815..dcc3bd69f 100644 --- a/doc/src/LectureNotes/_build/html/chapter8.html +++ b/doc/src/LectureNotes/_build/html/chapter8.html @@ -125,6 +125,26 @@ 8. Convolutional Neural Networks +
  • + + 9. Recurrent Neural Networks + +
    • +
  • + + 10. Solving ODEs with Deep Learning + +
    • +
  • + + 11. Data Analysis and Machine Learning: + +
    • +
  • + + 12. Elements of Bayesian theory and Bayesian Neural Networks + +
    • @@ -548,27 +568,27 @@ techniques.

    MSE before scaling: 0.01
 R2 score before scaling 0.94
 Feature min values before scaling:
- [1.00000000e+00 2.52277631e-04 1.58998839e-03 6.36440033e-08
- 4.01118504e-07 2.52806307e-06 1.60559584e-11 1.01193226e-10
- 6.37773764e-10 4.01959093e-09 4.05055916e-15 2.55287874e-14
- 1.60896055e-13 1.01405288e-12 6.39110289e-12 1.02186547e-18
- 6.44034203e-18 4.05904756e-17 2.55822858e-16 1.61233230e-15
- 1.01617794e-14]
+ [1.00000000e+00 3.86972479e-03 3.66528972e-03 1.49747700e-05
+ 1.41836625e-05 1.34343487e-05 5.79482386e-08 5.48868704e-08
+ 5.19872323e-08 4.92407803e-08 2.24243736e-10 2.12397083e-10
+ 2.01176282e-10 1.90548268e-10 1.80481726e-10 8.67761543e-13
+ 8.21918258e-13 7.78496845e-13 7.37369358e-13 6.98414609e-13
+ 6.61517814e-13]
 Feature max values before scaling:
- [1.         0.99869632 0.999475   0.99739435 0.998172   0.99895027
- 0.99609407 0.99687071 0.99764796 0.99842581 0.99479548 0.99557111
- 0.99634735 0.99712419 0.99790164 0.99349859 0.99427321 0.99504844
- 0.99582426 0.99660069 0.99737773]
+ [1.         0.99791107 0.99418827 0.99582651 0.99211148 0.98841032
+ 0.9937463  0.99003903 0.9863456  0.98266595 0.99167043 0.98797091
+ 0.9842852  0.98061323 0.97695496 0.9895989  0.98590711 0.98222909
+ 0.9785648  0.97491417 0.97127716]
 Feature min values after scaling:
- [ 0.         -1.70327945 -1.70487638 -1.10269333 -1.11031464 -1.11779074
- -0.8675412  -0.87387845 -0.88025384 -0.88666051 -0.73621263 -0.74081963
- -0.74548023 -0.75019465 -0.75496285 -0.65064103 -0.65385832 -0.65711132
- -0.66040159 -0.66373067 -0.66710007]
+ [ 0.         -1.66745992 -1.72823447 -1.09228638 -1.1114811  -1.1313648
+ -0.86506245 -0.87590574 -0.88698187 -0.89829265 -0.73687981 -0.74440021
+ -0.75205856 -0.75985475 -0.76778823 -0.65190491 -0.65755533 -0.66330693
+ -0.66916042 -0.67511627 -0.68117476]
 Feature max values after scaling:
- [0.         1.7648157  1.69590827 2.29614193 2.24159129 2.18630067
- 2.72674862 2.67767959 2.62822832 2.57838874 3.10180456 3.05384136
- 3.0057178  2.95743601 2.90899738 3.44157349 3.39262419 3.3436382
- 3.2946235  3.24558749 3.196537  ]
+ [0.         1.76330702 1.70952132 2.29506297 2.25330559 2.21033632
+ 2.73427311 2.69911557 2.66307432 2.6261167  3.11835914 3.08796041
+ 3.05688927 3.02512212 2.99263434 3.46490261 3.4379033  3.41038453
+ 3.38232999 3.35372259 3.32454443]
 MSE after  scaling: 0.00
 R2 score for  scaled data: 0.97
    @@ -683,9 +703,7 @@ Test set accuracy scaled data with Standar Scaler: 0.96 
    Test set accuracy: 0.95
-
    -
    -
    Test set accuracy scaled data: 0.96
+Test set accuracy scaled data: 0.96
 
    /Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/linear_model/_logistic.py:762: ConvergenceWarning: lbfgs failed to converge (status=1):
@@ -953,10 +971,10 @@ covariance matrix through the np.linalg.eig() function.
    -
    0.07206099142503472
-4.125937788943556
-[[ 1.03889255  3.1254397 ]
- [ 3.1254397  10.77395298]]
+
    0.11455861677725063
+4.414311328416008
+[[1.04460586 2.98508511]
+ [2.98508511 9.60711152]]
    @@ -996,10 +1014,10 @@ a more brute force way. Here we scale the mean values for each column of the des
    -
    0.07865526085644693
-1.4346285243728323
-[[1.         0.63566281]
- [0.63566281 1.        ]]
+
    0.07650519724946082
+1.5077307406836722
+[[1.         0.63582252]
+ [0.63582252 1.        ]]
    @@ -1031,30 +1049,30 @@ this matrix we easily see that it is a positive definite matrix.

    -
    [[ 0.53680052  2.50858953]
- [-0.62753667 -1.74895806]
- [ 0.16570918  0.64706639]
- [-0.27228985 -0.08345099]
- [ 0.85643181  1.55737697]
- [-0.329196   -2.18391809]
- [ 0.92272798  2.91370357]
- [-0.37864859 -0.25719203]
- [-0.55226917 -1.4496257 ]
- [-0.32172921 -1.9035916 ]]
+
    [[-0.21054401 -1.67179046]
+ [-0.27924545 -1.98111498]
+ [-1.24089109 -3.51965915]
+ [ 2.17042981  6.77708985]
+ [-0.36489938 -0.63821317]
+ [-1.61222872 -5.67668711]
+ [-0.67184873 -0.62798419]
+ [ 1.21860807  4.36876959]
+ [ 0.0351748  -0.93789716]
+ [ 0.95544469  3.90748678]]
           0         1
-0  0.536801  2.508590
-1 -0.627537 -1.748958
-2  0.165709  0.647066
-3 -0.272290 -0.083451
-4  0.856432  1.557377
-5 -0.329196 -2.183918
-6  0.922728  2.913704
-7 -0.378649 -0.257192
-8 -0.552269 -1.449626
-9 -0.321729 -1.903592
+0 -0.210544 -1.671790
+1 -0.279245 -1.981115
+2 -1.240891 -3.519659
+3  2.170430  6.777090
+4 -0.364899 -0.638213
+5 -1.612229 -5.676687
+6 -0.671849 -0.627984
+7  1.218608  4.368770
+8  0.035175 -0.937897
+9  0.955445  3.907487
           0         1
-0  1.000000  0.907143
-1  0.907143  1.000000
+0  1.000000  0.972249
+1  0.972249  1.000000
    @@ -1114,37 +1132,37 @@ this matrix we easily see that it is a positive definite matrix.

         0         1         2         3         4         5         6         7   \
 0   0.0  0.000000  0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   
-1   0.0  0.076168  0.080935  0.078268  0.081921  0.085682  0.070727  0.073798   
-2   0.0  0.080935  0.086925  0.081766  0.086005  0.090410  0.072900  0.076302   
-3   0.0  0.078268  0.081766  0.085075  0.088305  0.091557  0.079623  0.082640   
-4   0.0  0.081921  0.086005  0.088305  0.091884  0.095514  0.082116  0.085368   
-5   0.0  0.085682  0.090410  0.091557  0.095514  0.099557  0.084565  0.088068   
-6   0.0  0.070727  0.072900  0.079623  0.082116  0.084565  0.076371  0.078937   
-7   0.0  0.073798  0.076302  0.082640  0.085368  0.088068  0.078937  0.081684   
-8   0.0  0.077019  0.079892  0.085767  0.088751  0.091725  0.081567  0.084509   
-9   0.0  0.080401  0.083686  0.089005  0.092271  0.095547  0.084258  0.087409   
-10  0.0  0.062426  0.063699  0.072024  0.073913  0.075721  0.070361  0.072486   
-11  0.0  0.065031  0.066507  0.074745  0.076802  0.078786  0.072797  0.075066   
-12  0.0  0.067773  0.069474  0.077585  0.079825  0.082002  0.075324  0.077748   
-13  0.0  0.070658  0.072612  0.080551  0.082990  0.085376  0.077944  0.080532   
-14  0.0  0.073697  0.075932  0.083645  0.086302  0.088919  0.080656  0.083422   
+1   0.0  0.062851  0.073533  0.065769  0.068538  0.069947  0.059538  0.060320   
+2   0.0  0.073533  0.087625  0.078561  0.082559  0.084744  0.071705  0.072983   
+3   0.0  0.065769  0.078561  0.072769  0.076696  0.079022  0.068407  0.069803   
+4   0.0  0.068538  0.082559  0.076696  0.081211  0.083977  0.072511  0.074206   
+5   0.0  0.069947  0.084744  0.079022  0.083977  0.087102  0.075127  0.077070   
+6   0.0  0.059538  0.071705  0.068407  0.072511  0.075127  0.066156  0.067782   
+7   0.0  0.060320  0.072983  0.069803  0.074206  0.077070  0.067782  0.069583   
+8   0.0  0.060735  0.073737  0.070715  0.075353  0.078424  0.068928  0.070878   
+9   0.0  0.060967  0.074217  0.071359  0.076188  0.079434  0.069792  0.071870   
+10  0.0  0.052371  0.063245  0.061909  0.065807  0.068404  0.061247  0.062897   
+11  0.0  0.052613  0.063714  0.062491  0.066554  0.069297  0.062003  0.063759   
+12  0.0  0.052756  0.064027  0.062920  0.067118  0.069987  0.062592  0.064441   
+13  0.0  0.052853  0.064260  0.063261  0.067574  0.070552  0.063079  0.065009   
+14  0.0  0.052933  0.064453  0.063553  0.067966  0.071040  0.063502  0.065504   
 
           8         9         10        11        12        13        14  
 0   0.000000  0.000000  0.000000  0.000000  0.000000  0.000000  0.000000  
-1   0.077019  0.080401  0.062426  0.065031  0.067773  0.070658  0.073697  
-2   0.079892  0.083686  0.063699  0.066507  0.069474  0.072612  0.075932  
-3   0.085767  0.089005  0.072024  0.074745  0.077585  0.080551  0.083645  
-4   0.088751  0.092271  0.073913  0.076802  0.079825  0.082990  0.086302  
-5   0.091725  0.095547  0.075721  0.078786  0.082002  0.085376  0.088919  
-6   0.081567  0.084258  0.070361  0.072797  0.075324  0.077944  0.080656  
-7   0.084509  0.087409  0.072486  0.075066  0.077748  0.080532  0.083422  
-8   0.087542  0.090669  0.074644  0.077376  0.080221  0.083181  0.086261  
-9   0.090669  0.094040  0.076827  0.079721  0.082739  0.085887  0.089170  
-10  0.074644  0.076827  0.065759  0.067867  0.070042  0.072284  0.074590  
-11  0.077376  0.079721  0.067867  0.070098  0.072403  0.074782  0.077234  
-12  0.080221  0.082739  0.070042  0.072403  0.074845  0.077371  0.079979  
-13  0.083181  0.085887  0.072284  0.074782  0.077371  0.080053  0.082828  
-14  0.086261  0.089170  0.074590  0.077234  0.079979  0.082828  0.085781  
+1   0.060735  0.060967  0.052371  0.052613  0.052756  0.052853  0.052933  
+2   0.073737  0.074217  0.063245  0.063714  0.064027  0.064260  0.064453  
+3   0.070715  0.071359  0.061909  0.062491  0.062920  0.063261  0.063553  
+4   0.075353  0.076188  0.065807  0.066554  0.067118  0.067574  0.067966  
+5   0.078424  0.079434  0.068404  0.069297  0.069987  0.070552  0.071040  
+6   0.068928  0.069792  0.061247  0.062003  0.062592  0.063079  0.063502  
+7   0.070878  0.071870  0.062897  0.063759  0.064441  0.065009  0.065504  
+8   0.072303  0.073408  0.064109  0.065065  0.065830  0.066473  0.067035  
+9   0.073408  0.074615  0.065054  0.066094  0.066934  0.067645  0.068268  
+10  0.064109  0.065054  0.057780  0.058594  0.059248  0.059799  0.060281  
+11  0.065065  0.066094  0.058594  0.059478  0.060193  0.060800  0.061333  
+12  0.065830  0.066934  0.059248  0.060193  0.060965  0.061622  0.062202  
+13  0.066473  0.067645  0.059799  0.060800  0.061622  0.062326  0.062949  
+14  0.067035  0.068268  0.060281  0.061333  0.062202  0.062949  0.063612
    @@ -1341,10 +1359,10 @@ We can write our own code or simply use either the functionaly of numpy< 
              0         1
-0  3.884269  1.937050
-1  1.937050  1.951359
-[[3.88426936 1.93705016]
- [1.93705016 1.95135877]]
+0  3.963873  1.979586
+1  1.979586  1.976796
+[[3.96387325 1.97958566]
+ [1.97958566 1.97679551]]
    @@ -1371,8 +1389,8 @@ Our own code here is not very elegant and asks for obvious improvements. It is t 
    Centered covariance using own code
-[[3.88426936 1.93705016]
- [1.93705016 1.95135877]]
+[[3.96387325 1.97958566]
+ [1.97958566 1.97679551]]
    _images/chapter8_77_1.png @@ -1432,16 +1450,14 @@ questions.

    Eigenvalues of Covariance matrix
-5.082577137078103
-0.7530509884519185
+5.185256249406441
+0.7554125101643117
 First eigenvector
-[0.85042593 0.5260948 ]
+[0.85104821 0.52508756]
 Second eigenvector
-[-0.5260948   0.85042593]
-
    -
    -
    Eigenvector of largest eigenvalue
-[-0.85042593 -0.5260948 ]
+[-0.52508756  0.85104821]
+Eigenvector of largest eigenvalue
+[-0.85104821 -0.52508756]
    diff --git a/doc/src/LectureNotes/_build/html/chapter9.html b/doc/src/LectureNotes/_build/html/chapter9.html index e90eece7b..8b71ce520 100644 --- a/doc/src/LectureNotes/_build/html/chapter9.html +++ b/doc/src/LectureNotes/_build/html/chapter9.html @@ -38,6 +38,7 @@ + @@ -124,6 +125,26 @@ 8. Convolutional Neural Networks +
  • + + 9. Recurrent Neural Networks + +
    • +
  • + + 10. Solving ODEs with Deep Learning + +
    • +
  • + + 11. Data Analysis and Machine Learning: + +
    • +
  • + + 12. Elements of Bayesian theory and Bayesian Neural Networks + +
    • @@ -731,88 +752,88 @@ labels = (n_inputs) = (1797,)
    -
     1/12 [=>............................] - ETA: 0s - loss: 2.5817 - accuracy: 0.2500
+
     1/12 [=>............................] - ETA: 0s - loss: 3.0150 - accuracy: 0.1250
 
    
 
    
 
    
-12/12 [==============================] - 0s 917us/step - loss: 2.8078 - accuracy: 0.1139
+12/12 [==============================] - 0s 958us/step - loss: 2.9441 - accuracy: 0.0722
 
    
 
    
 
    Learning rate =  1e-05
 Lambda =  1e-05
-Test accuracy: 0.114
+Test accuracy: 0.072
 
    
 
    
-
     1/12 [=>............................] - ETA: 0s - loss: 3.0782 - accuracy: 0.1875
+
     1/12 [=>............................] - ETA: 0s - loss: 3.3024 - accuracy: 0.0312
 
    
 
    
 
    
-12/12 [==============================] - 0s 932us/step - loss: 3.3640 - accuracy: 0.1500
+12/12 [==============================] - 0s 957us/step - loss: 3.0967 - accuracy: 0.1389
 
    
 
    
 
    Learning rate =  1e-05
 Lambda =  0.0001
-Test accuracy: 0.150
+Test accuracy: 0.139
 
    
 
    
-
     1/12 [=>............................] - ETA: 0s - loss: 2.7005 - accuracy: 0.0312
+
     1/12 [=>............................] - ETA: 0s - loss: 2.9008 - accuracy: 0.2188
 
    
 
    
 
    
-12/12 [==============================] - 0s 933us/step - loss: 2.7588 - accuracy: 0.0944
+12/12 [==============================] - 0s 1ms/step - loss: 3.4843 - accuracy: 0.1028
 
    
 
    
 
    Learning rate =  1e-05
 Lambda =  0.001
-Test accuracy: 0.094
+Test accuracy: 0.103
 
    
 
    
-
     1/12 [=>............................] - ETA: 0s - loss: 3.9994 - accuracy: 0.1250
+
     1/12 [=>............................] - ETA: 0s - loss: 3.7695 - accuracy: 0.1250
 
    
 
    
 
    
-12/12 [==============================] - 0s 1ms/step - loss: 4.2041 - accuracy: 0.1028
+12/12 [==============================] - 0s 968us/step - loss: 3.6647 - accuracy: 0.1417
 
    
 
    
 
    Learning rate =  1e-05
 Lambda =  0.01
-Test accuracy: 0.103
+Test accuracy: 0.142
 
    
 
    
-
     1/12 [=>............................] - ETA: 0s - loss: 12.9177 - accuracy: 0.1250
+
     1/12 [=>............................] - ETA: 0s - loss: 12.4141 - accuracy: 0.0938
 
    
 
    
 
    
-12/12 [==============================] - 0s 939us/step - loss: 12.8693 - accuracy: 0.1056
+12/12 [==============================] - 0s 960us/step - loss: 12.3084 - accuracy: 0.1028
 
    
 
    
 
    Learning rate =  1e-05
 Lambda =  0.1
-Test accuracy: 0.106
+Test accuracy: 0.103
 
    
 
    
-
     1/12 [=>............................] - ETA: 0s - loss: 92.4325 - accuracy: 0.1250
+
     1/12 [=>............................] - ETA: 0s - loss: 92.7441 - accuracy: 0.0938
 
    
 
    
 
    
-12/12 [==============================] - 0s 2ms/step - loss: 92.4491 - accuracy: 0.1111
+12/12 [==============================] - 0s 959us/step - loss: 91.7224 - accuracy: 0.1306
 
    
 
    
 
    Learning rate =  1e-05
 Lambda =  1.0
-Test accuracy: 0.111
+Test accuracy: 0.131
 
    
 
    
-
     1/12 [=>............................] - ETA: 0s - loss: 514.5186 - accuracy: 0.0312
+
     1/12 [=>............................] - ETA: 0s - loss: 514.1083 - accuracy: 0.0625
 
    
 
    
 
    
-12/12 [==============================] - 0s 939us/step - loss: 514.2589 - accuracy: 0.1028
+12/12 [==============================] - 0s 958us/step - loss: 513.9800 - accuracy: 0.1111
 
    
 
    
 
    Learning rate =  1e-05
 Lambda =  10.0
-Test accuracy: 0.103
+Test accuracy: 0.111
 
    
 
    
 
    
@@ -1679,6 +1700,7 @@ of recurrent neural networks (see the link above and also
     
    
         
     7. Dimensionality Reduction
+    9. Recurrent Neural Networks
 
     
    
     
    
diff --git a/doc/src/LectureNotes/_build/html/genindex.html b/doc/src/LectureNotes/_build/html/genindex.html
index acf3d8d03..e4499bbc7 100644
--- a/doc/src/LectureNotes/_build/html/genindex.html
+++ b/doc/src/LectureNotes/_build/html/genindex.html
@@ -123,6 +123,26 @@
    8. Convolutional Neural Networks
   
  
+ 
  • 
+  
+   9. Recurrent Neural Networks
+  
+ 
    • 
+ 
  • 
+  
+   10. Solving ODEs with Deep Learning
+  
+ 
    • 
+ 
  • 
+  
+   11. Data Analysis and Machine Learning:
+  
+ 
    • 
+ 
  • 
+  
+   12. Elements of Bayesian theory and Bayesian Neural Networks
+  
+ 
    • 
 
 
 
diff --git a/doc/src/LectureNotes/_build/html/objects.inv b/doc/src/LectureNotes/_build/html/objects.inv
index c7cb8c33a..2022fee48 100644
Binary files a/doc/src/LectureNotes/_build/html/objects.inv and b/doc/src/LectureNotes/_build/html/objects.inv differ
diff --git a/doc/src/LectureNotes/_build/html/reports/chapter10.log b/doc/src/LectureNotes/_build/html/reports/chapter10.log
new file mode 100644
index 000000000..2bc1d3814
--- /dev/null
+++ b/doc/src/LectureNotes/_build/html/reports/chapter10.log
@@ -0,0 +1,39 @@
+Traceback (most recent call last):
+  File "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/nbclient/client.py", line 604, in _async_poll_for_reply
+    msg = await ensure_async(self.kc.shell_channel.get_msg(timeout=new_timeout))
+  File "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/nbclient/util.py", line 85, in ensure_async
+    result = await obj
+  File "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/jupyter_client/asynchronous/channels.py", line 48, in get_msg
+    raise Empty
+_queue.Empty
+
+During handling of the above exception, another exception occurred:
+
+Traceback (most recent call last):
+  File "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/jupyter_cache/executors/utils.py", line 51, in single_nb_execution
+    executenb(
+  File "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/nbclient/client.py", line 1082, in execute
+    return NotebookClient(nb=nb, resources=resources, km=km, **kwargs).execute()
+  File "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/nbclient/util.py", line 74, in wrapped
+    return just_run(coro(*args, **kwargs))
+  File "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/nbclient/util.py", line 53, in just_run
+    return loop.run_until_complete(coro)
+  File "/Users/hjensen/opt/anaconda3/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
+    return future.result()
+  File "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/nbclient/client.py", line 535, in async_execute
+    await self.async_execute_cell(
+  File "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/nbclient/client.py", line 811, in async_execute_cell
+    exec_reply = await self.task_poll_for_reply
+  File "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/nbclient/client.py", line 628, in _async_poll_for_reply
+    await self._async_handle_timeout(timeout, cell)
+  File "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/nbclient/client.py", line 678, in _async_handle_timeout
+    raise CellTimeoutError.error_from_timeout_and_cell(
+nbclient.exceptions.CellTimeoutError: A cell timed out while it was being executed, after 30 seconds.
+The message was: Cell execution timed out.
+Here is a preview of the cell contents:
+-------------------
+['import autograd.numpy as np', 'from autograd import grad, elementwise_grad', 'import autograd.numpy.random as npr', 'from matplotlib import pyplot as plt', '']
+...
+['    plt.plot(x, res[0,:])', "    plt.legend(['analytical','nn'])", "    plt.xlabel('x')", "    plt.ylabel('g(x)')", '    plt.show()']
+-------------------
+
diff --git a/doc/src/LectureNotes/_build/html/reports/chapter11.log b/doc/src/LectureNotes/_build/html/reports/chapter11.log
new file mode 100644
index 000000000..5b616a288
--- /dev/null
+++ b/doc/src/LectureNotes/_build/html/reports/chapter11.log
@@ -0,0 +1,36 @@
+Traceback (most recent call last):
+  File "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/jupyter_cache/executors/utils.py", line 51, in single_nb_execution
+    executenb(
+  File "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/nbclient/client.py", line 1082, in execute
+    return NotebookClient(nb=nb, resources=resources, km=km, **kwargs).execute()
+  File "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/nbclient/util.py", line 74, in wrapped
+    return just_run(coro(*args, **kwargs))
+  File "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/nbclient/util.py", line 53, in just_run
+    return loop.run_until_complete(coro)
+  File "/Users/hjensen/opt/anaconda3/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
+    return future.result()
+  File "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/nbclient/client.py", line 535, in async_execute
+    await self.async_execute_cell(
+  File "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/nbclient/client.py", line 827, in async_execute_cell
+    self._check_raise_for_error(cell, exec_reply)
+  File "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/nbclient/client.py", line 735, in _check_raise_for_error
+    raise CellExecutionError.from_cell_and_msg(cell, exec_reply['content'])
+nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:
+------------------
+np.random.seed(999)         # for reproducibility
+a=0.6                       # biased coin
+flips=np.random.rand(2**12) # simulates 4096 coin flips
+heads=flips in 
+----> 1 np.random.seed(999)         # for reproducibility
+      2 a=0.6                       # biased coin
+      3 flips=np.random.rand(2**12) # simulates 4096 coin flips
+      4 heads=flips<a               # boolean array, heads[i]=True if flip i is heads
+
+NameError: name 'np' is not defined
+NameError: name 'np' is not defined
+
diff --git a/doc/src/LectureNotes/_build/html/reports/chapter6.log b/doc/src/LectureNotes/_build/html/reports/chapter6.log
index f568062dd..002670576 100644
--- a/doc/src/LectureNotes/_build/html/reports/chapter6.log
+++ b/doc/src/LectureNotes/_build/html/reports/chapter6.log
@@ -1,14 +1,3 @@
-Traceback (most recent call last):
-  File "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/nbclient/client.py", line 604, in _async_poll_for_reply
-    msg = await ensure_async(self.kc.shell_channel.get_msg(timeout=new_timeout))
-  File "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/nbclient/util.py", line 85, in ensure_async
-    result = await obj
-  File "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/jupyter_client/asynchronous/channels.py", line 48, in get_msg
-    raise Empty
-_queue.Empty
-
-During handling of the above exception, another exception occurred:
-
 Traceback (most recent call last):
   File "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/jupyter_cache/executors/utils.py", line 51, in single_nb_execution
     executenb(
@@ -22,18 +11,19 @@ Traceback (most recent call last):
     return future.result()
   File "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/nbclient/client.py", line 535, in async_execute
     await self.async_execute_cell(
-  File "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/nbclient/client.py", line 811, in async_execute_cell
-    exec_reply = await self.task_poll_for_reply
-  File "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/nbclient/client.py", line 628, in _async_poll_for_reply
-    await self._async_handle_timeout(timeout, cell)
-  File "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/nbclient/client.py", line 678, in _async_handle_timeout
-    raise CellTimeoutError.error_from_timeout_and_cell(
-nbclient.exceptions.CellTimeoutError: A cell timed out while it was being executed, after 30 seconds.
-The message was: Cell execution timed out.
-Here is a preview of the cell contents:
--------------------
-['eta_vals = np.logspace(-5, 1, 7)', 'lmbd_vals = np.logspace(-5, 1, 7)', '# store the models for later use', 'DNN_numpy = np.zeros((len(eta_vals), len(lmbd_vals)), dtype=object)', '']
-...
-['        ', '        print("Learning rate  = ", eta)', '        print("Lambda = ", lmbd)', '        print("Accuracy score on test set: ", accuracy_score(Y_test, test_predict))', '        print()']
--------------------
+  File "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/nbclient/client.py", line 827, in async_execute_cell
+    self._check_raise_for_error(cell, exec_reply)
+  File "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/nbclient/client.py", line 735, in _check_raise_for_error
+    raise CellExecutionError.from_cell_and_msg(cell, exec_reply['content'])
+nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:
+------------------
+pip3 install tensorflow
+------------------
+
+  File "", line 1
+    pip3 install tensorflow
+         ^
+SyntaxError: invalid syntax
+
+SyntaxError: invalid syntax (, line 1)
 
diff --git a/doc/src/LectureNotes/_build/html/search.html b/doc/src/LectureNotes/_build/html/search.html
index 97a950d46..aa6673f02 100644
--- a/doc/src/LectureNotes/_build/html/search.html
+++ b/doc/src/LectureNotes/_build/html/search.html
@@ -128,6 +128,26 @@
    8. Convolutional Neural Networks
   
  
+ 
  • 
+  
+   9. Recurrent Neural Networks
+  
+ 
    • 
+ 
  • 
+  
+   10. Solving ODEs with Deep Learning
+  
+ 
    • 
+ 
  • 
+  
+   11. Data Analysis and Machine Learning:
+  
+ 
    • 
+ 
  • 
+  
+   12. Elements of Bayesian theory and Bayesian Neural Networks
+  
+ 
    • 
 
 
 
diff --git a/doc/src/LectureNotes/_build/html/searchindex.js b/doc/src/LectureNotes/_build/html/searchindex.js
index bae7a166b..02df29b79 100644
--- a/doc/src/LectureNotes/_build/html/searchindex.js
+++ b/doc/src/LectureNotes/_build/html/searchindex.js
@@ -1 +1 @@
-Search.setIndex({docnames:["chapter1","chapter2","chapter3","chapter4","chapter5","chapter6","chapter7","chapter8","chapter9","testbook/_build/jupyter_execute/chapter1","testbook/_build/jupyter_execute/chapter2","testbook/_build/jupyter_execute/chapter3","testbook/_build/jupyter_execute/chapter4","testbook/_build/jupyter_execute/chapter5","testbook/_build/jupyter_execute/chapter6","testbook/_build/jupyter_execute/chapter7","testbook/_build/jupyter_execute/lecturenotes/lecturenotes/notebooks","testbook/_build/jupyter_execute/notebooks","testbook/chapter1","testbook/chapter2","testbook/chapter3","testbook/chapter4","testbook/chapter5","testbook/chapter6","testbook/chapter7","testbook/content","testbook/intro","testbook/lecturenotes/CONDUCT","testbook/lecturenotes/CONTRIBUTING","testbook/lecturenotes/README","testbook/lecturenotes/lecturenotes/content","testbook/lecturenotes/lecturenotes/intro","testbook/lecturenotes/lecturenotes/markdown","testbook/lecturenotes/lecturenotes/notebooks","testbook/markdown"],envversion:{"sphinx.domains.c":2,"sphinx.domains.changeset":1,"sphinx.domains.citation":1,"sphinx.domains.cpp":3,"sphinx.domains.index":1,"sphinx.domains.javascript":2,"sphinx.domains.math":2,"sphinx.domains.python":2,"sphinx.domains.rst":2,"sphinx.domains.std":1,"sphinx.ext.intersphinx":1,sphinx:56},filenames:["chapter1.ipynb","chapter2.ipynb","chapter3.ipynb","chapter4.ipynb","chapter5.ipynb","chapter6.ipynb","chapter7.ipynb","chapter8.ipynb","chapter9.ipynb","testbook/_build/jupyter_execute/chapter1.ipynb","testbook/_build/jupyter_execute/chapter2.ipynb","testbook/_build/jupyter_execute/chapter3.ipynb","testbook/_build/jupyter_execute/chapter4.ipynb","testbook/_build/jupyter_execute/chapter5.ipynb","testbook/_build/jupyter_execute/chapter6.ipynb","testbook/_build/jupyter_execute/chapter7.ipynb","testbook/_build/jupyter_execute/lecturenotes/lecturenotes/notebooks.ipynb","testbook/_build/jupyter_execute/notebooks.ipynb","testbook/chapter1.ipynb","testbook/chapter2.ipynb","testbook/chapter3.ipynb","testbook/chapter4.ipynb","testbook/chapter5.ipynb","testbook/chapter6.ipynb","testbook/chapter7.ipynb","testbook/content.md","testbook/intro.md","testbook/lecturenotes/CONDUCT.md","testbook/lecturenotes/CONTRIBUTING.md","testbook/lecturenotes/README.md","testbook/lecturenotes/lecturenotes/content.md","testbook/lecturenotes/lecturenotes/intro.md","testbook/lecturenotes/lecturenotes/markdown.md","testbook/lecturenotes/lecturenotes/notebooks.ipynb","testbook/markdown.md"],objects:{},objnames:{},objtypes:{},terms:{"00000000e":[3,7],"000e":7,"00405815e":3,"00578340e":[],"00694659e":[],"00813277e":3,"00pm":26,"01118504e":7,"01193226e":7,"01222393e":3,"012e":7,"01347297e":[],"01405288e":7,"01443312e":[],"01617794e":7,"01633169e":3,"01959093e":7,"01968016e":7,"02004166e":[],"02045611e":3,"02176297e":[],"02186547e":7,"02599545e":[],"03123374e":[],"03233215e":[],"03256632e":5,"03288157e":[],"03446279e":[],"03705850e":[],"04029857e":[],"040e":7,"041e":7,"04719753e":[],"04722875e":[],"04904333e":[],"05055916e":7,"05246505e":[],"058e":7,"05904756e":7,"060e":7,"064e":7,"06648521e":[],"07106781e":3,"07557895e":[],"075e":7,"07968357e":[],"08248290e":3,"08318298e":5,"08678173e":[],"08915360e":[],"09013732e":[],"09284157e":[],"09358934e":[],"09507879e":[],"0_tlsccwai":26,"0_utc9il9i":26,"0x7f9dd7c37eb0":3,"0x7fad47b08730":[],"0x7fb5be1db040":[],"0x7fddb4c436d0":[],"0x7ff4f8410b20":[],"0x7ff6a7303b50":[],"1000s":3,"100m":[11,20],"10368860e":[],"10378326e":5,"10665316e":[],"10687329e":[],"10836834e":[],"10898112e":3,"10th":26,"10x":2,"112453549290521e":[12,21],"11314068e":3,"11487443e":[],"114e":7,"11507992e":5,"115e":7,"11705599e":[],"11765797e":3,"117e":7,"12219360e":3,"12403403e":[],"12471292e":[],"12541909e":[],"12674763e":3,"12868265e":[],"12908684e":[],"12cm":[10,19],"13037918e":7,"13404436e":3,"14549644e":7,"146e":7,"14g":3,"15657183e":[],"15751255e":[],"15g":3,"16214154e":[],"16496581e":3,"16b8e3cda33a":5,"17061481e":[],"17174962e":5,"17314f270d45":7,"17683028e":7,"18442813e":[],"18721828e":7,"19218307e":[],"19614372e":7,"1970s":[2,10,19],"19762810e":[],"198e":7,"1_1":5,"1_2":5,"1_2v2se359":26,"1_2v439nza":26,"1_3":5,"1_azwo1s2r":26,"1_b81t0tta":26,"1_d70czgce":26,"1_e1cs5784":26,"1_ftgzt035":26,"1_hxxec3uc":26,"1_i9hczn21":26,"1_j5ugthfg":26,"1_rz8vr2ht":26,"1_sp7p28vk":26,"1_t1pocrww":26,"1_t2l86862":26,"1_utivxb87":26,"1_w9xc8az7":26,"1_wk9trq9k":26,"1cm":[1,2,6,11,20],"1e4":3,"1kg":[10,19],"1kgm":[10,19],"1ms":8,"202e":7,"20686874e":[],"20745612e":[],"21351358e":[],"21635571e":[],"21814310e":[],"21869563e":[],"21st":26,"22164060e":[],"22421033e":[],"224805627923717e":[12,21],"22674158e":[],"226e":7,"228e":7,"23908217e":3,"23h":[15,24],"24411055e":3,"24623749e":[],"24629399e":[],"24790498e":[],"24h":[15,24],"252e":7,"254e":7,"2572e3a4b38d":5,"26349118e":[],"263e":7,"26519220e":[],"26888965e":[],"268e":7,"27564656e":[],"27881252e":[],"279e":7,"27n_":1,"28119384e":[],"28444646e":[],"28590170e":[],"28kg":[15,24],"29001174e":[],"29127820e":[],"2_1":5,"2_2":5,"2_3":5,"2_i":5,"2_m":1,"2_x":1,"2cm":[6,10,19],"2dimgrav":[14,23],"2dimhovv":[14,23],"2ff97f4bf03b":1,"2k_2":[13,22],"2k_3":[13,22],"2me":[14,23],"2meb":[14,23],"2mr":[14,23],"2mr_":[14,23],"2ms":8,"2nd":[13,22,26],"2v_":[11,20],"2v_x":[11,20],"2v_y":[11,20],"2x_ix_jy_iy_j":6,"2x_j":6,"2y_0":[10,19],"2y_j":6,"2z_":7,"30273573e":[],"30579019e":7,"30pm":26,"31819744e":7,"32218090e":[],"32849325e":[],"33026664e":7,"33066907e":3,"33291949e":[],"33381074e":[],"333e":7,"33583503e":[],"33608388e":[],"33637629e":3,"33851317e":[],"34991581e":3,"35186316e":[],"35938336e":[],"36351028e":3,"36440033e":7,"36648631e":[],"36816390e":[],"37689988e":[],"37715992e":3,"37773764e":7,"379e":7,"38189254e":[],"381e":7,"38483045e":7,"39086494e":3,"39110289e":7,"39377982e":[],"39860289e":7,"3rd":26,"40925064e":[],"41088348e":[],"412252157986472e":[12,21],"41241709e":[],"41511965e":5,"422e":7,"43297505e":[],"435e":7,"43821714e":[],"44034203e":7,"44786221e":7,"447e":7,"44970586e":5,"45035234e":[],"45426079e":[],"47269386e":[],"47580075e":[],"48974653e":[],"49887171e":[],"4ar":[14,23],"4dr":[14,23],"4f_0":[12,21],"4pm":26,"4th":26,"501e":7,"50446329e":[],"50470354e":[],"504e":7,"50565973e":7,"50pm":26,"50x10":5,"512e":7,"51355225e":[],"51502206e":[],"51737928e":[],"52131750e":[],"52177694e":[],"52277631e":7,"52491181e":[],"52533249e":[],"52583599e":[],"52783754e":[],"52806307e":7,"53990135e988":3,"54220685e":[],"54223480e":[],"54686289e":[],"55287874e":7,"55456211e":[],"55726683e":[],"55822858e":7,"55868382e":[],"56282700e":[],"56432255e":7,"565e":7,"56min":[15,24],"57072598e":[],"570e":7,"57425734e":3,"58130740e":[],"58458590e":[],"58473257e":[],"58928789e":[],"58934440e":[],"58977964e":7,"58998839e":7,"59310961e":7,"59766065e":[],"59775278e":[],"59958864e":[],"5cm":[1,13,22],"5th":26,"604e":7,"60505104e":[],"60559584e":7,"60896055e":7,"60996409e":[],"61028757e":7,"61233230e":7,"61445925e":[],"62113986e":[],"62496853e":[],"62790003e":[],"628e":7,"63164705e":[],"63177381e":[],"63748198e":[],"638e":7,"64043119e":[],"64x50":5,"65939208e":3,"66371431e":[],"66517501e":[],"66678115e":[],"67887686e":[],"67901631e":[],"67m":[15,24],"69069n_":1,"69095941e":7,"69238112e":7,"6n_":1,"6pm":26,"6th":26,"70287435e":[],"70865417e":[],"70896266e":7,"710e":7,"71385616e":3,"713e":7,"71833086e":[],"72347840e":[],"72352580e":[],"72892756e":3,"729e":7,"73212847e":[],"73374509e":[],"74406011e":3,"744e":7,"74784357e":[],"75209222e":[],"76071445e":[],"77350269e":3,"77667437e":[],"77804918e":7,"78512013e":[],"78920723e":[],"79318674e":7,"79464371e":[],"7th":26,"80872400e":[],"811e":7,"81402316e":7,"81862750e":[],"82725395e":[],"83131741e":[],"83194721e":7,"83259341e":7,"83286207e":[],"83323451e":[],"84045777e":[],"84355903e":5,"84443254e":5,"84683780e":[],"852e":7,"86195387e":7,"86797194e":[],"873e":7,"882e":7,"885e":7,"89045976e":[],"89354206e":[],"89456494e":[],"8th":26,"8x8":5,"903e":7,"90524667e":[],"90555053e":[],"90838144e":[],"91032829e":[],"91269450e":7,"917u":8,"91811329e":7,"92507116e":5,"92870110e":[],"92983892e":7,"930e":7,"932u":8,"933u":8,"938e":7,"93947492e":[],"939u":8,"94395333e":[],"94466097e":[],"94553659e":[],"948e":7,"95125898e":[],"954e":7,"95572537e":[],"955820c21e8b":8,"96064784e":[],"960e":7,"97624658e":[],"98010863e":[],"981e":7,"984e":7,"98658737e":[],"99126621e":7,"996e":7,"9th":26,"abstract":[5,9,18],"boolean":8,"break":[2,3,7,10,11,14,19,20,23],"byte":[2,10,19],"case":[0,2,3,4,5,7,8,9,11,12,13,14,15,18,20,21,22,23,24],"catch":2,"char":1,"class":[0,1,2,3,5,6,7,8,9,10,13,18,19,22,26],"const":[1,7],"default":[2,3,4,5,8,10,19,32,34],"final":[1,2,3,4,6,9,13,14,15,18,22,23,24],"float":[1,2,3,7,8,9,10,12,18,19,21],"function":[0,2,8,9,10,11,12,13,14,15,18,19,20,21,22,23,24,32,34],"import":[0,3,4,5,6,7,9,11,12,13,14,16,17,18,20,21,22,23,26,29,33],"int":[1,3,5,7,8,11,13,14,20,22,23],"long":[1,2,5,8,11,13,14,15,20,22,23,24],"new":[0,1,2,3,4,5,6,7,9,10,11,12,13,18,19,20,21,22],"null":1,"public":[0,2,10,19,27],"return":[1,2,3,5,6,7,8,9,10,11,12,13,14,18,19,20,21,22,23],"s\u00f8renssen":26,"short":[8,25,30],"super":3,"switch":[2,14,15,23,24],"throw":[1,11,13,15,20,22,24],"true":[1,2,3,4,5,6,7,8,12,13,14,15,21,22,23,24],"try":[0,1,2,4,5,6,7,9,10,11,13,14,18,19,20,22,23],"var":[1,3,7],"while":[0,1,2,3,5,6,7,8,11,12,13,15,20,21,22,24,26],AGE:3,AMS:26,Added:[11,20],Adding:[0,5],And:[0,1,3,8,9,10,18,19],Are:[7,11,20],BPS:26,Being:27,But:[1,2,3,5,7,11,12,15,16,17,20,21,24,33],CAS:[2,10,19],DIS:3,Doing:3,EoS:3,For:[0,1,2,3,4,5,6,7,8,10,11,12,13,14,15,16,17,19,20,21,22,23,24,26,32,33,34],GPS:0,Going:[3,5],I_s:[13,22],Ising:5,Its:[5,7,8],MDS:7,NNs:5,N_s:6,Not:[0,2,3,7],ODE:[11,20],One:[0,1,3,4,5,6,7,8,12,13,14,15,21,22,23,24],PCs:7,RMS:1,Such:[1,3,5,9,11,18,20],That:[1,2,4,5,7,11,15,20,24],The:[0,2,9,10,14,18,19,23,26,27,28,29,32,34],Then:[1,2,3,5,6,7,10,14,15,19,23,24],There:[0,1,2,3,5,6,7,8,9,10,12,13,15,16,17,18,19,21,22,24,25,30,32,33,34],These:[0,1,2,3,5,6,7,8,9,10,11,13,14,15,18,19,20,22,23,24,26],Use:[0,8,14,23],Useful:3,Uses:8,Using:[2,3,6,8,10,11,12,13,14,15,19,20,21,22,23,24,27],With:[1,2,3,5,6,7,9,10,11,12,13,14,15,18,19,20,21,22,23,24],Yes:[0,11,20],__doc__:3,__future__:6,__init__:[3,5],__name__:3,_auto10:[3,5,11,12,13,14,20,21,22,23],_auto11:[3,11,12,13,14,20,21,22,23],_auto12:[11,12,13,14,20,21,22,23],_auto13:[3,11,12,13,14,20,21,22,23],_auto14:[11,12,13,14,20,21,22,23],_auto15:[11,12,13,14,20,21,22,23],_auto16:[11,13,14,20,22,23],_auto17:[11,13,14,20,22,23],_auto18:[11,13,14,20,22,23],_auto19:[11,13,20,22],_auto1:[1,3,4,5,11,12,13,14,15,20,21,22,23,24],_auto20:[11,13,20,22],_auto21:[11,13,20,22],_auto23:[13,22],_auto24:[13,22],_auto25:[13,22],_auto26:[13,22],_auto27:[13,22],_auto28:[13,22],_auto29:[13,22],_auto2:[1,3,5,11,12,13,14,15,20,21,22,23,24],_auto30:[13,22],_auto31:[13,22],_auto32:[13,22],_auto33:[13,22],_auto34:[13,22],_auto3:[3,5,11,12,13,14,15,20,21,22,23,24],_auto4:[3,5,11,12,13,14,15,20,21,22,23,24],_auto5:[3,5,11,12,13,14,20,21,22,23],_auto6:[3,5,11,12,13,14,20,21,22,23],_auto7:[3,5,11,12,13,14,20,21,22,23],_auto8:[3,5,11,12,13,14,20,21,22,23],_auto9:[3,5,11,12,13,14,20,21,22,23],_ax:3,_base:6,_build:29,_check_optimize_result:7,_compon:7,_datafram:[2,10,19],_lambda:3,_logist:7,_ratio:7,_setattr_cm:3,_subplot:3,_update_properti:3,_varianc:7,a_0:[3,12,21],a_1:[13,15,22,24],a_1a:[2,3],a_1x:[12,21],a_2:[13,15,22,24],a_2a:[2,3],a_2a_3:[12,21],a_2x:[12,21],a_3:[2,15,24],a_3a:3,a_3a_2:[12,21],a_4:2,a_4a:3,a_fc1:5,a_fc2:5,a_h:5,a_i:[2,5,10,12,15,19,21,24],a_j:5,a_ja_k:[12,21],a_k:5,a_n:[12,21],a_p:[13,22],a_x:[10,19],a_z:[10,19],aaverag:[10,19],abandon:5,abid:[1,28],abil:[0,2],abl:[1,2,3,4,5,11,13,14,20,22,23,26],abort:1,abotu:[15,24],about:[0,1,3,5,6,7,8,9,11,12,13,14,15,16,17,18,20,21,22,23,24,26,28,32,33,34],abov:[1,2,3,4,5,6,7,8,10,11,12,13,14,15,19,20,21,22,23,24],abovement:[3,9,15,18,24],abs:[2,3,9,12,18,21],absolut:[2,3,9,10,18,19],absorb:[13,14,15,22,23,24],abus:27,acccess:2,accelearatio:[10,19],acceler:[12,13,14,21,22,23,26],acceleromet:[15,24],accept:[0,3,8,11,20,27,32,34],access:[0,1,2,3,7,8],accid:[3,8],accommod:[13,22],accompani:[0,3],accomplish:[6,14,23],accord:[1,2,3,5,10,13,14,15,19,22,23,24],accordingli:7,account:[0,1,2,8,10,15,19,24,27],accumul:[1,5,9,18],accur:[1,3,8,12,21],accuraci:[0,2,3,5,7,8,12,21],accuracy_scor:[2,5],accuracy_score_numpi:5,achiev:[0,2,3,5,6,10,19],aco:[1,11,20],acquaint:0,acquir:[0,5,26],acr:3,across:[2,3,5,8,10,19],act:[5,10,12,13,14,15,19,21,22,23,24,26,27],action:[1,11,12,20,21,27,29],activ:[2,8,26],actual:[2,3,5,6,7,10,11,14,15,19,20,23,24],adam:8,adapt:[3,27],add:[0,1,2,3,5,6,7,10,11,12,13,15,19,20,21,22,24,32,34],add_:[16,17,33],add_graph:5,add_subplot:[4,5],added:[2,3,5,6,7,8,9,11,12,15,18,20,21,24],adding:[3,5,12,13,21,22],addit:[0,1,2,3,4,5,6,7,8,9,10,11,13,14,15,18,19,20,22,23,24,26],addition:5,address:[7,9,12,18,21,27],adjac:5,adjust:[11,13,14,20,22,23],admir:2,adopt:[10,19],advanc:[5,9,18,27],advantag:[3,5,8,11,20],afecionado:2,affect:[3,12,14,21,23],affin:[3,6,7,8],afford:8,aficionado:2,african:3,after:[0,1,2,3,5,7,8,10,12,13,14,19,21,22,23,26],afterward:[2,10,19],again:[0,1,2,3,4,5,6,7,12,13,15,21,22,24,32,34],against:[5,11,13,20,22],age:[0,2,3,27],agre:[1,13,22,28],aid:7,aim:[0,2,4,7,9,10,11,12,14,18,19,20,21,23],air:[12,13,15,21,22,24,26],aircraft:[15,24],airplan:8,akin:[13,22],algebra:[0,2,14,23],algo:[0,1],algorithm:[0,1,2,4,6,8,9,10,11,13,18,19,20,22,26],align:[1,3,4,6,16,17,27,33],all:[0,1,2,3,4,5,7,8,9,10,11,12,13,14,15,18,19,20,21,22,23,24,27,29,32,34],allevi:5,allow:[0,2,3,5,6,7,8,9,10,11,12,13,14,18,19,20,21,22,23,26,32,34],almost:[0,1,2,3,5,6,7],alon:[13,22],along:[2,3,7,8,10,12,14,15,19,21,23,24],alpha:[1,2,3,5,6,7,8,11,12,14,15,20,21,23,24],alpha_n:[13,22],alreadi:[1,2,5,10,11,13,19,20,22,32,34],also:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,26,32,33,34],alter:[1,5],altern:[2,3,5,6,7,10,11,12,19,20,21],although:[3,5,6,11,12,13,20,21,22],alwai:[0,1,2,3,5,8,10,11,12,14,19,20,21,23,26,28],amaz:0,ame2016:[2,3],american:3,among:[0,3,5,8],amongst:[14,23],amount:[3,6,8,12,13,15,21,22,24],amplitud:[11,13,15,20,22,24],an_:1,anaconda3:[3,6,7],anaconda:[2,5,10,19,26],analog:[14,23],analogu:[14,23],analys:[1,3],analysi:[2,4,5,8,9,10,11,14,18,19,20,23],analyt:[2,3,4,5,9,11,12,13,15,18,20,21,22,24,26],analyz:[0,1,2,3,5,8,11,20,26],ander:26,andrew:5,anew:[12,13,14,21,22,23],angl:[1,3,7,10,12,13,14,15,19,21,22,23,24],angmom:[14,23],angular:[11,13,14,15,20,22,23,24,26],anharmon:[13,22],ani:[0,1,2,3,4,5,6,8,11,12,13,14,15,20,21,22,23,24,27,28,32,34],anim:5,ann:5,annot:[2,3,5,6,7,8],anoth:[0,1,2,5,6,7,8,10,11,13,14,15,19,20,22,23,24,32,34],ans:1,ansatz:2,answer:[0,2,3,5,10,11,13,19,20,22],anticip:[8,10,19],antiparallel:[14,23],antisymmetri:[12,21],anymor:6,anyon:6,anyth:[1,5,28],apach:5,apart:7,api:[2,5,10,19],appar:[15,24],appear:[0,1,2,5,8,11,12,13,14,15,20,21,22,23,24,27],append:[1,2,6,8,10,19],appendix:[13,22],appl:0,appli:[1,2,3,4,5,6,7,8,9,10,11,12,13,14,18,19,20,21,22,23,26,27],applic:[0,1,2,3,5,7,8,9,11,15,18,20,24,26],appoint:27,appreci:28,approach:[1,2,5,7,9,10,12,13,14,18,19,21,22,23],appropri:[1,3,5,11,20,27],approx:[1,3,7,11,12,13,14,15,20,21,22,23,24],approxim:[1,3,4,7,8,9,11,12,13,15,18,20,21,22,24],apt:[2,10,19],aragorn:[2,10,19],arang:[2,3,4,5,8,10,19],arbitrari:[1,5,6,8,11,13,14,15,20,22,23,24],arbitrarili:[2,5,7,14,23],architectur:8,arctan:[14,23],arctanh:[10,19],area:[0,2,8,11,12,14,20,21,23],areal:[14,23],arg:3,argc:1,argmax:[5,7],argsort:7,argu:5,argument:[1,3,5,7,8],argv:1,aris:[0,1,2,3,5,15,24],arithmet:2,arma:1,armadillo:1,around:[1,2,3,5,7,10,11,12,13,14,15,19,20,21,22,23,24],arrai:[1,3,5,6,8,9,12,13,14,16,17,18,21,22,23,33],arrang:[8,26],arriv:[1,2,7,9,18],arrow:[5,11,20],arrowprop:6,art:0,articl:[1,3,5,8,28],artifici:[0,2,4],artificialneuron:5,artist:3,arxiv:8,asarrai:2,ascii:[1,2],asid:[12,13,21,22],asin:[11,13,20,22],ask:[3,5,7,11,15,20,24],aspect:[2,9,11,18,20],assembl:[2,8],assert:8,assess:[0,3],assici:8,assign:[0,2,3,4,5,6,14,23,26],assist:26,associ:[1,2,3,5,15,24,32,34],assum:[1,2,3,4,5,6,7,8,10,11,12,13,14,15,19,20,21,22,23,24],assumpt:[0,1,2,7,8,15,24],ast:3,asteroid:[14,23],astronom:[12,21],asymmetr:[12,21],asymmetri:2,asymptot:[3,14,23],atoi:1,atom:[2,11,12,13,14,20,21,22,23],attack:27,attempt:[2,12,21],attent:[2,10,15,19,24,27],attitud:[0,9,18],attract:[2,14,23],attribut:3,attributeerror:3,audi:2,audio:8,auditori:[32,34],australia:[32,34],author:[0,1,2,5],authour:[0,2],auto:[1,7],autocor:1,autocorrelation_tim:1,autocorrelform:1,autocovari:1,autograd:[2,10,19],autom:[0,9,18,29],automag:[0,2,10,19],automat:[2,7,8,10,19,29],automobil:[8,11,20],autonom:8,avail:[0,2,3,5,7,9,10,18,19,26],avdt:[11,20],avenu:0,averag:[1,2,3,5,8,12,14,21,23],avg:1,avoid:[0,2,3,7,8,10,12,19,21],awai:[1,3,13,14,15,22,23,24],aws:26,axel:[11,20],axes:[3,6,7,8,10,11,16,17,19,20,33],axes_grid1:3,axessubplot:3,axhlin:6,axi:[1,2,3,4,5,6,7,8,11,12,14,15,20,21,23,24],axlabel:[2,10,19],axs:[10,19],axvlin:[6,8],axvspan:8,azimuth:[12,14,21,23],b_1:5,b_1e:[13,22],b_2e:[13,22],b_fc1:5,b_fc2:5,b_i:[3,5],b_ia_:3,b_j:5,b_k:5,b_m:5,b_out:5,b_p:[13,22],b_x:[10,19],b_y:[10,19],b_z:[10,19],back:[0,1,3,8,10,11,12,14,15,19,20,21,23,24],backend:8,background:[10,19],backpropag:5,backward:[5,8,14,23],bad:[1,3],badli:1,baggin:[2,10,19],bake:[9,18],balanc:3,ball:[10,11,15,19,20,24],ban:27,band:[2,10,19],bandwidth:[2,10,19],bar:[2,7,10,12,19,21],barrier:26,base:[0,1,2,3,4,5,6,8,9,10,13,18,19,22],basi:[2,3,4,5,6,7,10,14,19,23,26],basic:[0,1,5,6,9,13,15,18,22,24],batch:[5,7,8],batch_i:5,batch_shap:8,batch_siz:[5,8],batch_x:5,bay:4,bayesian:0,bdb:[14,23],becaus:[0,2,3,5,6,8,9,11,12,13,14,15,18,20,21,22,23,24],becom:[0,1,2,3,5,7,10,11,12,13,14,15,19,20,21,22,23,24],been:[0,2,3,5,7,8,9,10,11,13,18,19,20,22,26,29],befor:[0,1,2,3,4,5,6,8,9,11,12,13,14,15,18,20,21,22,23,24],beforehand:[0,1,2],began:[14,23],begin:[1,2,3,4,5,6,7,10,11,12,13,14,15,16,17,19,20,21,22,23,24,26,33],behav:[3,5,14,23],behavior:[0,2,5,9,13,18,22,27],behaviour:5,behind:[0,2,3,5,6,13,22],being:[0,1,2,3,4,5,6,7,8,9,13,14,15,18,22,23,24,32,34],believ:[11,20],belong:[4,6,15,24],below:[1,2,3,4,5,6,7,8,9,10,11,12,13,15,18,19,20,21,22,24,28],benchmark:[9,18],benefici:5,benefit:[2,5,7,8,9,18],bengio:5,benign:7,besid:8,best:[0,1,2,3,5,6,7,8,11,13,14,20,22,23,27,28],beta:[2,4,5,7,14,23],beta_0:[3,4,5,13,22],beta_0x_:3,beta_1:[3,4,5],beta_1x_0:3,beta_1x_1:[3,4],beta_1x_2:3,beta_1x_:3,beta_1x_i:4,beta_2:3,beta_2x_0:3,beta_2x_1:3,beta_2x_2:[3,4],beta_2x_:3,beta_:[3,4],beta_i:3,beta_j:3,beta_n:[13,22],beta_p:4,beta_px_p:4,better:[0,2,5,7,8,9,11,12,13,18,20,21,22],between:[0,1,2,3,4,5,6,7,8,10,11,12,13,14,15,19,20,21,22,23,24],beyond:[0,2,3,5,6,9,13,18,22],bia:[0,1,5,6,8],bias:[3,8],bias_vari:5,bib:[32,34],bibliographi:[32,34],bibtex:[32,34],bicycl:[10,19],big:[0,2,3,5,11,20],bigger:5,bigr:5,bilbo:[2,10,19],billion:5,bin:[1,3,7],binari:[1,2,4,8],binomi:0,binsboot:3,bioinformat:[0,2],biolog:5,bird:[2,8],birth:[2,10,19],bit:[1,2,5,10,11,13,15,19,20,22,24,28],bitwis:1,bla:[2,10,19],black:6,blob:26,block:[0,2,3,10,19,26,32,34],blockeulercrom:[13,22],blockingavg:1,blockingstd:1,blockingvar:1,blocksiz:1,blocksizemax:1,blocksizemin:1,blog:28,blue:[2,8],blue_patch:[10,19],bmatrix:[2,3,4,5,6,7,10,19],bmi:5,bodi:[0,3,5,8,11,12,15,20,21,24,26,27],boldfac:[3,10,19],boldsymbol:[0,2,5,6,7,10,11,12,14,15,19,20,21,23,24],bolt:[11,20],boltzmann:[0,5],bonu:[9,18],book:[8,16,17,26,32,33,34],boost:5,bootavg:1,bootstd:1,bootstrap:[0,5],bootvar:1,bootvec:1,bore:0,borrow:[2,10,19],boston_dataset:3,bot:6,both:[0,1,2,3,5,6,9,10,11,13,14,15,18,19,20,22,23,24,27,32,34],bottl:4,bottom:[3,13,22],bounc:[11,20],bound:[3,5,6,14,23],boundari:[5,6,7,11,20],box:[8,12,21,32,34],boyd:6,bracket:[1,8],brain:[4,5],brake:[15,24],branch:[28,29],brand:0,breast:5,brew:[2,10,19],brg:6,brian:[32,34],brief:[12,21],briefli:[2,9,13,18,22],bring:[2,3,13,15,22,24],brisban:[32,34],broad:[0,2],broken:[11,20],browser:2,brute:[3,7],bte:[13,22],bthe:[14,23],buck:0,bui:8,build:[0,1,2,3,10,19,32,34],built:[3,5,8,13,22,29,32,34],bunch:7,bundl:31,buoyanc:[10,19],bus:[15,24],busi:3,butler:26,c46dd114b2af:6,c_0:1,c_1:[5,15,24],c_1e:[15,24],c_2:[5,15,24],c_2e:[15,24],c_3:5,c_4:5,c_i:5,c_k:1,c_w:[11,20],cal:[5,6],calcul:[0,2,3,5,6,7,8,9,10,11,12,13,14,18,19,20,21,22,23],calculu:26,call:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,18,19,20,21,22,23,24,32,34],callabl:3,came:[11,15,20,24],camera:0,can:[0,1,2,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,26,28,29,32,33,34],cancel:[3,12,21],cancer:5,cancerpd:7,candid:6,cannot:[0,1,2,3,5,6,7,11,12,20,21],canopi:[2,10,19],capabl:[0,2,6,10,19],capita:3,captur:[5,7],car:[0,8,11,15,20,24],card:[0,3,4],cardin:5,care:[7,13,22],carlo:[0,1,2,3],carmak:0,carri:[3,4],cartesian:[10,14,19,23],cast:5,cat:8,categor:[0,2,5,7,8],categori:[0,2,4,5,8],categorical_crossentropi:[5,8],caus:[0,1,3,7,11,15,20,24],causal:3,causat:0,caveat:[12,21],cbook:3,ccc:5,cdf:1,cdot:[1,2,3,5,10,11,12,13,14,15,19,20,21,22,23,24],ceil:[10,12,13,14,19,21,22,23],cell:[32,34],cellphon:26,center:[1,2,3,4,6,11,12,13,15,20,21,22,24,26],central:[0,2,3,6,8,9,10,15,18,19,24,26],centrifug:26,centroid:1,centuri:[8,9,18],certain:[0,1,2,3,4,8,14,23],cha:3,chain:[0,1,12,14,15,21,23,24,26],challeng:[12,21],chanc:[1,5,14,23],chang:[1,2,3,5,6,7,8,9,10,11,12,13,14,15,18,19,20,21,22,23,24,28],channel:8,chapter:[3,7,8,13,22,26],charact:[1,2,3,6,8],character:[1,5,6],characterist:[3,8,13,22],charg:[2,11,14,20,23],charl:3,cheap:3,cheaper:5,check:[2,3,5,7,8,10,11,13,16,17,19,20,22,31,33],checkmark:[13,22],chemic:1,chi:2,chimborazo:[15,24],choic:[1,3,8,10,11,13,14,19,20,22,23],choleski:3,choos:[1,3,7,8,10,11,12,13,14,19,20,21,22,23],chosen:[1,2,3,5,6,10,11,12,13,14,15,19,20,21,22,23,24],chosen_datapoint:5,chpater:[13,22],christoph:[32,34],cifar10:8,cifar:8,cin:1,circ:[5,15,24],circl:[3,5,6,7,11,14,20,23],circuit:[13,22],circul:[14,15,23,24],circular:[11,12,14,20,21,23],circumst:27,circumv:[3,5],cite:[32,34],citizen:0,civita:[12,21],claim:0,clarifi:27,clariti:1,class_nam:8,classic:[2,9,12,14,15,18,21,23,24,34],classif:[0,2,3,4,6,8,10,19],classifi:[0,4,5,7],classificaton:5,clean:5,clear:[5,14,23],clearli:[1,3,6,7,8,11,12,20,21],clever:5,clf3:2,clf:[2,3,6],clf_lasso:3,clf_ridg:3,clip:[1,8],clockwis:[11,15,20,24],clone:[28,29],close:[0,1,2,3,5,6,7,10,11,13,14,15,19,20,22,23,24],closer:[11,20],closest:[6,7,10,14,19,23],closur:[2,10,19],cloud:[2,10,19],cluster:[0,2,3,5,7,8],cmap:[2,3,5,6,8,16,17,33],cmap_arg:3,cmath:1,cmse201:[9,18],cn_:1,cna:[11,20],cnn:5,cnn_kera:8,cntk:[2,10,19],code:[0,2,4,8,9,11,18,20],coef0:6,coef:2,coef_:[2,3,6],coeffici:[2,3,4,6,11,13,15,20,22,24],coerc:[2,3],cognit:[32,34],coin:1,col:[2,3,7,10,19],colab:[2,10,19],cold:[16,17,33],colinear:3,collaps:6,collect:[1,2,3,7,10,11,19,20],collid:[0,14,23],collinear:3,collis:[0,14,23],color:[1,2,3,6,8,10,14,16,17,19,23,33],color_channel:8,color_cod:3,colorbar:3,column:[2,3,5,6,7,10,19],com:[8,26],combin:[1,3,4,9,12,14,15,18,21,23,24],come:[0,2,3,5,8,10,11,13,14,15,19,20,22,23,24],comma:2,command:[0,1,2,5,9,10,18,19,32,34],comment:27,commerci:[2,10,19],commit:[27,28],commod:2,common:[0,1,2,3,4,5,7,8,10,11,12,13,14,19,20,21,22,23],commonli:[3,4,5,8],commonmark:[32,34],commun:[0,2,5,27],commut:[12,21],compact:[3,4,5,7,12,14,21,23],compactli:[12,21],compani:0,compar:[2,3,7,8,9,10,11,13,14,15,18,19,20,22,23,24],comparison:8,compat:4,compdriv:[13,22],compet:[0,2,9,18,26],compil:[0,1,2,5],complaint:27,complement:[12,21],complementari:[13,22],complet:[2,5,8,11,13,20,22],completenn:5,complex:[0,3,5,6,7,13,14,15,22,23,24,32,34],complic:[0,2,3,5,9,11,14,15,18,20,23,24],compon:[3,5,8,11,12,13,14,15,20,21,22,23,24],components_:7,compos:5,compress:[2,3,7,11,20],compris:3,compromis:3,comput:[0,2,3,5,6,8,9,11,12,13,15,18,20,21,22,24,26,32,34],computation:[0,1,8],con:26,concaten:[3,8],concentr:3,concept:[0,2,3,12,21],conceptu:[5,14,15,23,24],concern:[0,4,5,8,12,21],concic:[2,10,19],conclud:2,conclus:[0,5],conda:[2,5,10,19,28],condit:[1,3,6,7,10,11,12,13,14,15,19,20,21,22,23,24],conduct:[0,9,18],confer:[32,34],confid:[3,4,6],confidenti:27,configur:8,confirm:5,confus:3,congruenti:1,conjug:6,connect:[2,3,5,7,8,10,19,26],consequ:[3,5,6,9,14,18,23],conserv:[3,11,13,14,20,22,23,26],consid:[0,1,2,3,4,5,6,8,11,12,13,14,15,20,21,22,23,24,27],consider:[0,3,5,7,11,20],consist:[1,3,5,8,11,13,20,22],constant:[1,2,3,5,6,11,12,13,14,15,20,21,22,23,24],constitu:2,constitut:3,constrain:[5,7,8,14,23],constraint:[3,6,26],construct:[0,1,2,3,5,6,7,9,18,27],consumpt:[11,20],contact:[2,11,14,20,23,27],contain:[0,1,2,3,4,5,6,7,8,9,10,18,19,26],content:[2,10,19,31,32,34],context:3,continu:[0,2,3,4,5,6,9,10,11,15,18,19,20,24],continuosli:0,contourf:6,contrast:[5,11,15,20,24],contribut:[0,1,2,3,8,12,13,14,15,21,22,23,24,27,29],contributor:[0,2,27,28],control:[0,2,5,8],conv2d:8,conv:8,conveni:[2,3,5,9,10,11,12,14,18,19,20,21,23],convent:[5,9,10,18,19],converg:[3,5,6,7],convergencewarn:[6,7],convert:[2,3,7,8,10,14,15,19,23,24],converttomatrix:8,convex:4,convolut:[13,22],cookiecutt:29,coolwarm:[16,17,33],coordin:[3,5,7,11,12,13,14,20,21,22,23,26],coorel:3,copi:[5,28],corel:0,corioli:26,corner:[15,24],corona:26,corr:[3,7],correalt:[3,7],correct:[0,1,2,3,5,8,11,12,13,15,20,21,22,24,27],correct_predict:5,correctli:[3,5],correl:[0,2,5],correlation_matrix:[3,7],correspod:[15,24],correspond:[1,2,3,5,6,7,8,10,12,13,14,15,19,21,22,23,24],cortex:[5,32,34],cos:[3,10,11,12,13,14,15,19,20,21,22,23,24],cosh:[10,19],cosin:[3,13,22],cost:[0,1,2,3,4,6,8],could:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,18,19,20,21,22,23,24,27,28],coulomb:[2,14,23],count:[1,2],counter:[10,13,15,19,22,24],coupl:[3,8,10,11,13,19,20,22],cours:[0,1,3,7,8,9,10,11,12,18,19,20,21,26],cout:1,cov:[1,2,3,7],cov_xi:[3,7],cov_xx:[3,7],cov_yi:[3,7],covari:[0,2],covariance_matrix:[3,7],coven:27,cover:[0,2,13,14,22,23,25,26,30],covert:2,covxi:1,covxx:1,covxz:1,covyi:1,covyz:1,covzz:1,creat:[2,5,7,8,9,10,14,16,17,18,19,23,27,28,29,33],create_accuraci:5,create_biases_and_weight:5,create_convolutional_neural_network_kera:8,create_dnn:5,create_loss:5,create_neural_network_kera:5,create_optimis:5,create_placehold:5,create_x:[3,7],creation:5,credit:[0,3,4,26,28],crim:3,crime:3,criteria:[1,2],critic:[0,12,21,26,27],criticallydamp:[13,22],cromer:[13,22,26],cross:[0,1,4,5,8,10,11,12,13,15,19,20,21,22,24,26],cross_val_scor:3,crossvalid:3,crucial:[1,5],cs231:8,csc:[13,22],csr_matrix:2,cstdlib:1,csv:[2,3,8],ctnk:5,ctrajectori:[14,23],cubic:2,cumsum:[2,7,10,19],cumul:3,current:[8,15,24,29],curs:[3,7],curv:[4,5,11,12,20,21],curvatur:[14,15,23,24],custom_lin:[16,17,33],cvxopt:6,cycl:[1,5,14,23],cycler:[16,17,33],cyclon:[15,24],cyclotron:[11,20],d2l:26,da_d:[14,23],da_i:[15,24],dagger:[2,3,10,19],dai:[5,9,11,15,18,20,24,26],damp:26,dampeddiffyq:[13,22],darkr:1,dat:[2,3,10,19],dat_id:[2,3,7,10,12,13,14,19,21,22,23],data:[4,6,9,10,11,12,13,14,16,17,18,19,20,21,22,23,33],data_id:[2,3,7,10,12,13,14,19,21,22,23],data_indic:5,data_panda:[2,10,19],data_path:[2,3,7,10,12,13,14,19,21,22,23],databas:[2,5],datafil:[2,3,7,10,12,13,14,19,21,22,23],datafram:[2,3,7,8,10,19],datapoint:[3,4,5,7],dataset:[3,4,6,7],datatyp:8,date:[0,2,10,19],dbh:5,dbo:5,ddot:[11,13,14,15,20,22,23,24],ddrive:[13,22],dead:5,deadlin:26,deal:[0,1,2,3,5,6,7,8,10,15,19,24],dearli:0,debt:4,debug:3,decad:[0,2,8],decai:[1,2,11,13,20,22],decid:[0,3,8],decim:[2,3],decis:[0,2,5,6,7],decision_funct:6,decisiontreeregressor:2,declar:[2,3,8,10,19],decomposit:[0,5],decompost:3,decreas:[3,5,7,12,21],deduc:[0,2,11,20],deem:[11,20,27],deep:[2,4,8,11,20],deepen:[9,18],deeper:[0,2,8,9,11,12,18,20,21,26],deer:8,def:[1,2,3,5,6,7,8,9,10,12,13,14,18,19,21,22,23],def_covari:1,default_tim:8,defect:3,defici:3,defin:[1,2,3,4,6,7,8,9,10,12,13,14,15,18,19,21,22,23,24,26,27,32,34],definit:[3,6,7,9,10,11,13,14,15,18,19,20,22,23,24],defint:[1,10,19],deflect:[15,24],degre:[0,1,2,6,9,11,12,13,14,15,18,20,21,22,23,24],delet:[1,3],delimit:8,deliv:2,delta:[1,2,5,6,10,11,12,13,14,15,19,20,21,22,23,24],delta_:[2,5,10,19],delta_h:[2,5],delta_j:5,delta_k:5,delta_l:5,delta_n:[2,13,22],deltat:[10,12,13,14,19,21,22,23],deltax:[14,23],delv:2,demonstr:[2,3,4,5,13,15,22,24],demostr:[15,24],den:8,denomin:[5,13,14,22,23],denot:[1,3,4,5,12,15,21,24],dens:5,densiti:[1,3,11,14,20,23],depart:[15,24],departur:[14,23],depend:[1,2,3,4,5,6,7,10,11,12,13,14,15,19,20,21,22,23,24,26,32,34],depict:1,deploy:[2,10,19],deposit:[14,23],depth:8,deriv:[2,3,4,6,7,9,10,11,15,18,19,20,24,26],derogatori:27,descend:[3,7],descent:[0,3,4,6,8],descr:3,describ:[0,2,3,5,6,7,10,11,13,14,15,19,20,22,23,24],descript:[3,6,15,24],deserv:0,design:[2,4,5,11,13,20,22],designmatrix:3,desir:[0,2,3],despit:[5,11,20],det:3,detail:[2,7,10,12,14,19,21,23,26,27,28],detect:[5,6,15,24],detector:[14,23],determin:[0,1,2,3,5,6,7,8,10,11,12,13,15,19,20,21,22,24,26,27],determinist:[1,4],detial:26,detriment:0,dev:[1,5],develop:[0,2,6,7,8,9,10,11,12,13,14,18,19,20,21,22,23,26,28,29],deviat:[2,3,5,10,19],devic:[9,18],devis:5,df1:[2,10,19],diag:[3,6],diagnost:5,diagon:[1,2,3,4,10,19],diagonaliz:3,diagram:26,diagsvd:3,dial:[13,22],dialect:0,diamet:[15,24],dict:[3,6],dict_kei:3,dictionari:3,did:[3,4,5,7,11,13,15,20,22,24],die:5,died:[13,22],diff:[11,20],diffeent:6,differ:[0,1,2,3,5,7,8,10,11,12,13,14,15,19,20,21,22,23,24,26,27,32,34],differenti:[0,2,8,12,14,15,21,23,24,26],difficult:[0,1,3,5,11,12,15,20,21,24],difficulti:[0,5],digit:[0,2,5,8],dilut:5,dim:[7,8,13,22],dimens:[2,3,5,6,7,8,12,13,15,21,22,24],dimension:[0,2,8,10,12,13,14,19,21,22,23],dimensionless:[2,11,12,13,20,21,22],diment:[10,19],dimless:[11,20],dimnsion:8,diod:[13,22],direct:[0,2,3,5,7,8,10,11,12,13,14,15,19,20,21,22,23,24],directli:[1,3,5,10,14,19,23,32,34],directori:29,disabl:[27,29],disadvantag:2,disappear:[13,14,22,23],discard:[3,7],disciplin:[0,2,5],disclaim:[0,1],discours:[0,9,18],discov:0,discrep:[15,24],discret:[4,5,8,9,13,18,22],discrimin:[4,7],discuss:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,18,19,20,21,22,23,24,26],discussiom:26,diseas:4,disk:2,disord:4,displac:[12,13,15,21,22,24],displai:[1,2,3,5,6,7,8,10,14,19,23,32,34],displaystyl:3,disregard:2,disrupt:0,dissimilar:7,dissip:[11,20],dissipitav:[13,22],distanc:[1,3,6,7,11,12,14,15,20,21,23,24],distant:[14,23],distinct:[4,6,8],distinctli:6,distinguish:[1,2,6,7,11,20],distort:[14,15,23,24],distplot:3,distribuit:[10,19],distribut:[0,2,3,5,7,10,14,15,19,23,24],distrubut:[2,10,19],dive:[2,3,6,13,22],diverg:[5,14,23],divid:[0,1,2,3,5,6,7,8,10,11,13,14,19,20,22,23],divis:[1,3,6],dna:4,dnn1:8,dnn2_gru2:8,dnn:[2,5,8],dnn_kera:5,dnn_numpi:5,dnn_scikit:[2,5],dnn_tf:5,doc:[26,28,32,34],doconc:2,docstr:28,doctest:[9,18],document:[7,8,16,17,32,33,34],doe:[0,1,2,3,5,6,7,8,10,11,12,13,14,15,19,20,21,22,23,24],doesn:[5,8,11,12,20,21,32,34],dog:[5,8],doing:[2,3,7,11,20],dollar:[16,17,33],domain:6,domin:[2,11,14,20,23],don:[1,2,3,5,6,7,10,11,14,15,19,20,23,24],done:[1,2,3,7,8,10,11,12,13,14,15,19,20,21,22,23,24],door:[13,22],dot:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,18,19,20,21,22,23,24],doubl:[1,2,10,11,14,15,19,20,23,24,26],doubli:[5,12,15,21,24],down:[2,3,7,11,13,14,15,20,22,23,24],download:[2,5,8,10,19],downsampl:8,drag:[10,11,13,19,20,22],dramat:7,drastic:8,draw:[3,11,20],drawback:[3,7],drawn:[1,2,3,4,5,7,10,19],dream:0,drive:[0,8,11,13,20,22],driven:[26,28],drivenosc:[13,22],driver:0,drop:[1,2,3,7,15,24],dropna:[2,3],dry:0,dtype:[2,3,5,8,10,19],dub:2,due:[3,5,6,7,10,11,12,13,14,15,19,20,21,22,23,24,26],dummi:3,dure:[0,2,5,6,7,8,13,15,22,24],dv_:[11,20],dv_x:[11,12,15,20,21,24],dv_y:[11,12,15,20,21,24],dv_z:[12,15,21,24],dwell:3,dwh:5,dwo:5,dx_1:1,dx_n:1,dxdy:[13,22],dying:5,e_0:[11,20],each:[0,1,2,3,4,5,6,7,8,10,12,13,14,15,19,21,22,23,24],eager:3,eapprox:[2,3],ear:0,earli:5,earlier:[3,4,5,6,7,12,13,21,22],earth:[11,14,15,20,23,24,26],earthsuneul:[12,21],earthsunvv:[12,21],easi:[0,2,3,4,5,6,7,10,11,12,13,14,19,20,21,22,23],easier:[0,1,2,3,6,14,23,26,28],easili:[0,2,3,4,5,6,7,9,10,11,12,13,18,19,20,21,22],east:[15,24],ebind:[2,3],eccentr:[14,23],econom:0,econometr:[2,10,19],ecosystem:[2,10,19,32,34],edgecolor:3,edit:[2,27],edu:26,educ:[0,2,3,9,13,18,22,26],eff:[1,14,15,23,24],effect:[1,5,10,11,13,19,20,22,26],effic:5,effici:[1,2,8,10,11,19,20],efron:[1,3],eig:[1,2,3,7],eigen:1,eigenpair:[3,7],eigenvalu:[2,3,6,7,10,19],eigenvector:[3,7],eight:[2,10,19],eigval:[1,2],eigvalu:7,eigvec:[1,2],eigvector:7,eispack:[2,10,19],either:[0,1,2,3,4,5,6,7,11,12,13,14,15,20,21,22,23,24],elabor:1,elast:[14,23],electr:[2,5,11,13,14,20,22,23],electrodynam:[14,23],electromagnet:[11,14,20,23],electron:[9,12,14,18,21,23,27],eleg:7,elel:[10,19],element:[0,2,3,4,5,6,7,8,9,11,12,18,20,21],elementari:[2,10,19,26],elementwis:8,elessar:[2,10,19],elimin:[6,13,14,22,23],ellips:[14,15,23,24],ellipt:[15,24,26],els:[1,4,5,11,20],elu:5,elus:0,email:26,emb:[16,17,33],embed:[3,7],embodi:3,emit:[1,12,21],empathi:27,emphas:[0,2,13,22,26],emphasi:[2,10,19],empir:[1,5,7],emploi:[1,2,3,5,7,11,13,20,22],employ:[0,2,3],employe:0,empti:3,emul:5,enabl:[7,9,18],enbodi:[13,22],encapsul:[13,14,22,23],encod:[0,7,8],encompass:[0,1,2,11,20],encount:[0,1,3,4,5,14,23],end:[0,1,2,3,4,5,6,7,8,10,11,12,13,14,15,16,17,19,20,21,22,23,24,26,33],endl:1,endpoint:[1,3,13,22],energi:[8,10,11,13,14,15,19,20,22,23,24,26],enforc:5,engag:[9,18],engin:[0,1,2,5,10,11,19,20,26],enhanc:[9,18,26,28],enorm:8,enough:[0,3,11,14,15,20,23,24],enquiri:[11,20],enscrib:[12,21],ensembl:[1,5],ensur:[1,3,5,7,8,13,14,22,23],entail:2,enter:[0,1,3,13,15,22,24],enthought:[2,10,19],entir:[1,2,4,5,8,10,13,14,19,22,23,32,34],entiti:[2,5,10,19],entitl:0,entri:[2,3,5,6,7],entropi:[2,4,5,8],enumer:[2,3,5,6,8],env:1,environ:[2,9,10,11,18,19,20,27,29],eol:3,eosfit:3,epoch:[2,5,8],epsilon:[3,4,12,21],epsilon_0:[3,14,23],epsilon_1:3,epsilon_2:3,epsilon_:[2,3,11,12,20,21],epsilon_i:3,eqnarrai:[3,11,12,13,14,15,20,21,22,23,24],equador:[15,24],equal:[0,1,2,3,5,6,7,10,11,13,14,15,19,20,22,23,24],equat:[0,1,2,4,7,9,14,15,18,23,24,26],equatori:[15,24],equilibrium:[5,11,20],equiv:[1,12,13,14,15,21,22,23,24],equival:[2,3,5,6,7,10,12,13,14,19,21,22,23],erf:1,eriador:[2,10,19],eridg:3,err:2,err_:3,error:[0,1,2,4,5,7,8,9,13,18,22],error_estimate_corr_tim:1,error_hidden:5,error_output:5,escap:[16,17,33],especi:[5,12,13,14,21,22,23],essenti:[0,1,2,3,5,9,11,15,18,20,24],establish:7,estim:[0,1,2,3,4,5,12,21],estimated_mse_fold:3,estimated_mse_kfold:3,estimated_mse_sklearn:3,eta0:6,eta:[2,5,6,8,13,15,22,24],eta_v:[2,5,8],etc:[0,1,3,4,5,6,7,10,11,16,17,19,20,26,33],ethic:[9,18],ethnic:27,etsim:3,euclidean:[3,7],euler:[11,13,20,22,26],eulerintegr:[10,19],eurler:[10,19],european:0,evalu:[0,1,2,3,9,11,12,13,18,20,21,22,26],even:[0,1,2,3,5,6,7,8,10,11,12,13,14,15,19,20,21,22,23,24,28],evenli:8,event:[0,4,27],eventson:3,eventu:[3,5,7,11,15,20,24],everest:[15,24],everi:[1,2,3,5,7,8,10,11,12,15,19,20,21,24,28],everybodi:[0,26],everyon:27,everyth:[5,11,14,20,23],evid:[11,20,32,34],evolut:[13,14,22,23],evolv:[0,2],exact:[1,2,3,5,7,9,10,11,12,13,18,19,20,21,22],exactli:[3,5,7,8,14,15,23,24],exam:26,examin:3,exampl:[0,4,11,13,15,16,17,20,22,24,26,27,32,33,34],exce:5,excel:[0,2,3,5,29],except:[1,3,6,8,11,15,20,24],excess:2,excit:[0,2,13,14,22,23],exclud:[3,5],exclus:[1,2,3,8],execut:3,executablebook:29,exercis:[0,2,3,9,11,12,18,20,21,26],exert:[12,21],exhaust:3,exhibit:[3,6],exist:[0,2,3,5,6,7,8,10,12,13,14,19,21,22,23,29,32,34],exit:[1,3],exp:[1,3,4,5,6,7,13,22],exp_term:5,expand:[3,4,7,10,13,19,22],expans:[3,5,6,12,13,14,21,22,23],expect:[0,2,5,7,11,13,14,15,20,22,23,24,27],expectation_value_of_h_wrt_p:1,expens:1,experi:[0,2,3,5,6,9,11,14,15,18,20,23,24,27],experiment:[1,2,3,8,11,14,20,23],experimentalist:[14,23],expert:5,explain:[1,2,3,7,15,24,28],explained_variance_ratio_:7,explanatori:3,explicit:[2,3,8,10,12,13,15,19,21,22,24,27],explicitli:[0,2,8,12,21],explod:5,exploit:[3,5,8],explor:[2,3,6,9,10,18,19],expon:5,exponenti:[2,5,11,13,20,22],expos:0,express:[1,2,4,5,8,10,11,13,14,15,19,20,22,23,24,27],exptmean:1,exptvari:1,ext:[13,22],extend:[0,4,7,9,10,18,19],extens:[0,2,3,5,12,14,21,23,32,34],extent:[0,2,3,26],extern:[3,11,12,13,14,15,20,21,22,23,24],extra:[3,5,8,13,22,26],extract:[0,2,3,4,6,7,11,13,14,20,22,23],extrapol:[2,3],extrem:[0,2,3,4,5,6,7,8,9,10,18,19],extrins:7,eye:[0,2],f11:2,f12:2,f13:2,f6d7a289d493:[2,10,19],f_0:[12,13,21,22],f_0b:[12,21],f_0e:[13,22],f_2:5,f_3:5,f_d:1,f_g:[12,21],f_h:[12,21],f_i:[3,5,11,13,20,22],f_m:[13,22],f_n:[13,22],f_r:[14,23],f_x:[11,12,14,20,21,23],f_xv_x:[12,21],f_y:[11,14,20,23],f_yv_i:[12,21],f_zv_z:[12,21],face:27,facecolor:[1,3,6],facil:0,facilit:5,fact:[0,2,3,5,7,8,10,12,13,14,15,19,21,22,23,24],factor:[1,2,3,5,7,11,12,14,20,21,23],fade:3,fail:[3,6,7,14,23],failur:4,fair:27,fairli:1,faith:27,fall:[6,14,23,26],fallingbasebal:[10,19],fals:[2,3,4,5,8,13,22],famili:[0,1,2,4,6,7,10,13,19,22,26],familiar:[0,1,2,6,10,12,14,19,21,23],famou:[0,3,5],fanci:0,far:[0,1,2,3,5,6,7,8,11,14,20,23],fashion:[0,9,18],fast:[1,2,3,5,8,10,15,19,24],fastdriven1:[13,22],fastdriven2:[13,22],faster:7,favor:4,favorit:1,fc1:5,fc2:5,featur:[1,4,5,8,13,22],feature_nam:[3,7],feb:26,februari:[13,22],fed:[0,5],feed:[0,2,7,8,11,20],feed_dict:5,feed_forward:5,feed_forward_out:5,feed_forward_train:5,feeddorward:8,feedforward:[5,8],feel:[2,7,10,12,13,15,19,21,22,24,31],feet:[3,15,24],fellow:0,felt:[11,20],few:[1,2,5,8,25,30],fewer:[2,7],ffedbda27704:[13,22],ffnn:5,fictiti:[15,24],field:[0,2,3,5,12,13,14,15,21,22,23,24,26],fifth:[2,10,19],fig:[2,3,4,5,7,8,10,12,13,14,16,17,19,21,22,23,33],fig_id:[2,3,7,10,12,13,14,19,21,22,23],figaxi:1,figsiz:[2,3,5,6,7,8,10,16,17,19,33],figur:[2,3,4,5,6,7,8,10,11,12,13,14,15,19,20,21,22,23,24],figure_id:[2,3,7,10,12,13,14,19,21,22,23],figurefil:[2,3,7,10,12,13,14,19,21,22,23],file:[1,2,3,5,6,7,10,12,13,14,19,21,22,23,26,28],filenam:[1,2,3,10,19],fileout:1,filewrit:5,fill:3,filter:8,financ:[0,2],find:[0,1,2,4,5,6,7,9,11,12,13,14,15,18,20,21,22,23,24,26],fine:5,finish:[11,20],finit:[1,3,5,14,23],first:[0,1,3,4,7,8,9,12,13,14,15,18,21,22,23,24,26,32,34],firsteigvector:7,fit:[0,1,4,5,6,7,8,11,20],fit_intercept:3,fit_transform:[2,3,6,7],fiti:2,five:[2,3],fix:[2,5,8,11,12,13,14,15,16,17,20,21,22,23,24,33],flat:5,flatten:[5,8],flavor:[32,34],flexibl:[2,3,5,6,9,10,13,18,19,22],flip:[14,23],float32:5,float64:[2,3,8,10,19],floor:[10,19],flop:3,flow:[5,8,11,15,20,24],fly:[7,14,23],flyvbjerg:1,fmax:[13,22],focault:[15,24],foci:[14,23],focoault:[15,24],focu:[0,2,3,8,10,11,14,15,19,20,23,24],focus:[4,5,13,22,27],folder:[2,10,19,26],folk:0,follow:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,18,19,20,21,22,23,24,27,32,34],font:[1,2,10,13,19,22],fontdict:1,fontsiz:[1,3,6],footbal:[14,23],foral:6,forc:[3,7,9,10,12,18,19,21,26],forcast:8,forcedblockeulercrom:[13,22],forcedblockrk4:[13,22],forecast:[5,8],forest:[0,2,5],forget:7,fork:28,form:[0,1,2,3,4,5,6,7,8,10,11,12,13,14,15,19,20,21,22,23,24],formal:[1,8,10,12,15,19,21,24,26],formalim:26,format:[1,2,3,6,7,9,10,12,13,14,18,19,21,22,23],format_data:8,formul:[3,7,11,20],formula:[1,10,12,13,14,19,21,22,23],forth:[5,14,15,23,24],fortran90:1,fortran:[0,2,10,19],fortun:[3,7],forum:26,forward:[2,3,8,10,11,12,13,15,19,20,21,22,24],forwardeul:[13,22],forwardeulercrom:[13,22],foster:27,foucalt:26,foucault:26,found:[3,5,7,11,13,14,20,22,23,29],foundat:[9,18],four:[5,6,8,12,15,21,24],fourier:[3,26],fourierdef1:[13,22],fourierdef2:[13,22],fourierseriessign:[13,22],fouriersolut:[13,22],fourpi2:[12,21],fourth:[2,5,10,13,19,22],frac:[1,2,3,4,5,6,7,9,10,11,12,13,14,15,18,19,20,21,22,23,24],frame:[7,11,14,20,23,26],framework:[0,1,5,6],frankefunct:[3,7],free:[1,2,7,10,19,27],freedom:[9,11,13,14,18,20,22,23],freeli:[0,2],freewai:[11,20],frequenc:[1,3,7,10,11,13,14,15,19,20,22,23,24],frequent:[0,2,6],frib:26,friction:[11,13,20,22,26],frictionless:[11,20],fridai:26,friendli:[8,13,22],frodo:[2,10,19],frog:8,from:[0,1,2,4,6,7,9,10,11,12,13,14,16,17,18,19,20,21,22,23,26,27,33],from_logit:8,front:[0,2,8],frontier:[32,34],frydai:26,fstream:1,ftild:[13,22],fuel:[11,20],fulfil:[3,5,11,20],full:[0,1,2,3,4,10,19],fulli:[1,3,5,8,29],fun:[2,10,19],func:3,functionali:7,fundament:[0,3,11,13,20,22],further:[2,8,9,10,11,12,14,15,18,19,20,21,23,24,26,27],furthermor:[2,3,4,5,7,9,10,11,18,19,20],furthest:[15,24],futur:[0,2,6,8,11,13,20,22],futurist:0,fwhm:[13,22],g_m:[13,22],g_n:[13,22],gain:[0,3,5,9,11,12,13,18,20,21,22,26],galaxi:[11,20],galleri:[0,2],gallon:[14,23],gamge:[2,10,19],gamma1:6,gamma2:6,gamma:[3,6,7,11,13,14,15,20,22,23,24],gamma_:3,gamma_i:[1,3,6],gamma_x:3,gap:6,gate:[5,8],gather:5,gaug:5,gauss:[14,23],gaussian:[3,6,13,22],gaussian_rbf:6,gca:[3,6],gen:1,gender:[3,27],genener:8,gener:[0,2,5,6,7,8,9,10,11,12,13,14,15,18,19,20,21,22,23,24,26],generallay:5,genet:[0,5],geodes:[7,15,24],geometr:[2,11,12,20,21],geq:[3,6],get:[0,1,3,4,5,7,8,9,10,11,12,13,14,15,18,19,20,21,22,23,24,26,31],get_default_graph:5,get_yaxi:6,get_yticklabel:3,getattr:3,ggplot:[10,19],ghp:29,gibb:0,git:[0,26],github:[0,2,26,28,29],gitlab:0,give:[0,1,2,3,4,5,6,10,11,12,13,14,15,19,20,21,22,23,24,26,31],given:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,18,19,20,21,22,23,24,28],global:[1,4,12,13,21,22],global_step:5,global_variables_initi:5,glorot:5,gm_1m_2:[14,23],gm_:[12,14,21,23],gm_am_b:[14,23],gmm:[14,15,23,24],goal:[0,2,3,4,11,20],goe:[2,3,5,12,13,14,21,22,23,26],going:[2,5,6,7,8,11,15,20,24],gold:[14,23],gone:3,gong:5,good:[1,2,3,5,7,8,9,11,13,18,20,22,26,27],googl:[2,5,10,19],got:5,govern:[9,11,18,20],gracefulli:27,grader:26,gradient:[0,2,3,4,6,8,15,24],gradientdescentoptim:5,gradual:[11,13,20,22],graph:[0,5,7],graphic:[2,5,9,11,18,20],grasp:[0,2],gravgauss:[14,23],gravit:[5,10,11,12,13,15,19,20,21,22,24,26],graviti:[10,14,15,19,23,24,26],gray_r:[5,8],grayscal:8,graze:[14,23],great:[11,20],greater:[1,5,7],greatli:28,green:[1,2,8,13,22],greeni:[13,22],greeny2:[13,22],grid:[1,3,4,5,6,8,11,14,20,23],ground:[2,10,15,19,24],group:[0,2,3],groupbi:[2,3],grow:[5,8],growth:2,gru:8,guarante:[0,1,3,7],guess:[5,11,13,15,20,22,24],guid:[0,5,16,17,33],habit:[9,18],had:[0,2,3,5,11,12,14,20,21,23],hadamard:5,half:[5,6,14,23],hamiltonian:[12,21],hand:[0,1,2,3,5,7,8,9,10,11,14,15,18,19,20,23,24,26],handi:[8,12,15,21,24],handl:[5,7],handsid:5,handwrit:5,handwritten:5,handwrittennot:26,happen:[1,5,8,11,14,20,23],harass:27,hard:[4,5,6],hardcopi:[2,10,19],harder:[3,5,7],harm:27,harmon:[11,12,20,21,26],has:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,18,19,20,21,22,23,24,26,29,32,34],hassl:[2,10,19],hast:0,hat:[1,2,3,4,5,7,10,11,12,13,14,15,19,20,21,22,23,24],have:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,18,19,20,21,22,23,24,26,27],haven:5,hbar:[12,21],hdf5:2,hdhpk14:[32,34],head:[1,3,8],header:[2,3],hear:[0,2],heart:[0,2],heatmap:[2,3,5,7,8],heavi:[9,14,18,23],heavili:[0,2],heavisid:5,heer:[32,34],height:[5,8,11,12,14,15,20,21,23,24],helic:[11,20],help:[0,5,8,9,14,18,23,28],hemispher:[15,24],henc:[2,3,5,6,11,20],her:4,here:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,26,28,29,32,33,34],hereaft:[3,5,6,12,15,21,24],hermitian:[2,10,19],hessenberg:[2,10,19],hhint:[12,21],hidden:[5,8],hidden_bia:5,hidden_bias_gradi:5,hidden_layer_s:[2,5],hidden_neuron:8,hidden_weight:5,hidden_weights_gradi:5,high:[0,2,3,5,7,8,10,13,15,19,22,24],higher:[0,2,3,5,6,7,10,11,12,13,14,15,19,20,21,22,23,24],highest:5,highli:[1,2,3,8,10,19],highwai:3,hinder:0,hing:6,hire:[0,2],his:4,hist:[1,3,7,8],histogram:[1,3,7],histor:[4,7],histori:[5,8],histtyp:3,hit:[0,10,11,14,19,20,23],hjensen:[6,7,26],hjorth:26,hobbi:1,hoc:3,hold:[1,3,5,8,11,13,15,20,22,24],holder:3,holdgraf:[32,34],holdgraf_evidence_2014:[32,34],home:[3,26],homework:[12,14,21,23,26],homogen:[5,11,13,15,20,22,24],homogn:[11,20],homogsolut:[13,22],hope:[0,9,18],hopefulli:[1,3,7,11,20],horizont:[7,11,15,20,24],hors:[4,8],hot:[5,16,17,33],hour:[1,5,15,24,26],how:[0,2,4,5,7,8,9,10,11,12,13,14,15,18,19,20,21,22,23,24,25,26,28,30,31],howev:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,18,19,20,21,22,23,24],hspace:[1,2,6,10,11,13,19,20,22],htm:26,html:[2,7,9,16,17,18,26,29,33],http:[2,5,7,8,10,19,26],huang:2,huber:2,huge:[5,8],human:[0,2,5,32,34],hundr:5,hungri:5,hurrican:[15,24],hv_:[12,21],hw10:26,hydrogen:2,hyperbol:8,hyperbola:[14,23],hyperparam:6,hyperparamet:[3,8],hyperplan:7,i_1:3,i_2:3,i_c:[13,22],i_i:[13,22],idea:[0,1,2,3,5,8],ideal:[0,1,2,3,6,9,11,13,18,20,22],idem:3,ident:[3,5,13,22,27],identifi:[0,2,4,5,6,7,9,14,18,23],idum:1,ieor:1,iet:[13,22],ifs:[2,10,19],ignor:[3,7,8,10,12,14,15,19,21,23,24],iii:[2,10,19],ijk:[11,12,20,21],illustr:[0,5,11,14,15,20,23,24],imag:[3,5,7,16,17,33],image_height:8,image_path:[2,3,7,10,12,13,14,19,21,22,23],image_width:8,imageri:27,imagin:[11,12,20,21],imaginari:[13,15,22,24],immedi:[0,2,3,8,13,22,26],impact:[0,9,14,18,23],impart:[11,20],implement:[2,3,6,7,8,9,11,12,13,18,20,21,22,26],impli:[2,3,8,9,10,13,18,19,22],implic:[11,20],implicit:8,implicitli:[1,7],impos:[0,3,5,7],imposs:[0,3],impress:[0,2,5],improv:[0,3,7,8,11,12,13,20,21,22],impuls:[13,22],imshow:[3,5,8],in5400:8,in_out_neuron:8,inact:5,inadequ:0,inappropri:27,incid:[14,23,27],includ:[1,2,3,5,7,8,9,11,13,14,15,16,17,18,20,22,23,24,27,28,32,33,34],include_bia:3,inclus:[9,18,27],incom:[5,14,23],inconsist:[13,22],incorpor:[13,15,22,24],incorrect:5,incoveni:6,increas:[0,1,2,3,5,6,7,8,9,12,13,14,18,21,22,23],increasingli:1,incred:0,increment:[9,18],ind:3,inde:[0,2,3,11,12,13,14,20,21,22,23],indent:[1,13,22],indentationerror:1,independ:[1,3,4,5,6,10,11,12,13,14,15,19,20,21,22,23,24,26],index:[1,2,3,5,8,10,19],index_col:[2,3],indic:[2,3,5,7,8,10,11,19,20],indispens:3,individu:[1,4,5,13,14,22,23,27],indu:3,industri:[9,18],inelast:[14,23],inequ:[0,6],inerti:[11,20,26],inertia:[11,20],inf:3,infam:0,infer:[0,3,5,14,23],infil:[2,3,10,19],infin:[3,7,13,22],infinit:[0,11,13,14,20,22,23],infinitesim:[1,13,22],influenc:[3,10,11,14,19,20,23],influenti:5,info:[2,5,10,19],inform:[0,2,3,5,7,8,12,14,16,17,21,23,27,29,32,33,34],infti:[1,11,13,20,22],ingredi:[0,2],inher:3,inherit:[2,10,19],init:[32,34],initi:[1,2,3,5,10,11,12,13,14,15,19,20,21,22,23,24],initialis:1,inlin:[1,2,3,4,5,6,7,8,9,10,12,13,14,18,19,21,22,23],inner:3,inoccu:0,inp:8,input:[0,1,2,3,4,5,6,7,8,9,10,12,13,18,19,21,22,32,34],input_shap:8,inputs_shuffl:5,insert:[1,5,6,10,12,13,14,15,19,21,22,23,24,32,34],insid:[3,7,8,14,23,32,34],insight:[0,2,3,5,9,11,18,20,26],inspect:[11,13,14,20,22,23],inspir:[0,2,5],instal:[5,26,28,29],instanc:[0,3,5,7,11,20,27],instant:[12,21],instantan:[10,19],instead:[0,1,2,3,5,6,7,8,9,10,11,12,14,18,19,20,21,23],institut:[0,5],instruct:[0,5],instructor:26,insult:27,int32:5,int64:3,int_0:[1,9,10,11,12,13,15,18,19,20,21,22,24],int_:[1,10,11,12,13,19,20,21,22],int_a:[1,9,12,18,21],integ:[1,2,5,10,14,19,23],integer_vector:5,integr:[0,1,2,9,11,13,14,18,20,22,23,26],integraldef:[12,21],integrand:[13,22],intellig:[0,2],intens:[5,9,18],interact:[2,3,5,10,11,13,14,16,17,19,20,22,23,33],intercept:[2,3,6],intercept_:[2,3,6],interchang:5,interest:[0,1,2,3,4,5,6,7,8,10,12,13,15,19,21,22,24,27],interfac:[2,5,9,18],interior:2,intermedi:[13,22],intern:[5,11,12,14,20,21,23,32,34],internet:[15,24],interpol:[5,8,12,21],interpr:3,interpret:[0,1,2,5,10,11,12,19,20,21],interv:[1,3,4,9,11,12,13,18,20,21,22],intimid:2,intract:[3,7],intrins:[1,2,7,8,10,19],intro:[32,34],introduc:[0,1,2,5,6,9,10,11,12,13,15,18,19,20,21,22,24,26],introduct:[3,5,8,26],introductori:[2,3],intuit:[2,3,5,6],inv:3,invalid:[2,5,6,7,10,19],invalu:[2,9,10,18,19],invari:[5,12,21],invd:3,inver:6,invers:[2,3,7,10,11,13,14,19,20,22,23],invers_period:1,inverse_transform:6,invert:[3,4,11,12,20,21],investig:[9,18,27],invis:[14,23],invok:[3,6,11,20],involv:[0,3,4,5,7,10,11,13,14,19,20,22,23],inward:[15,24],iomanip:1,ion:[16,17,33],ios:1,iostream:1,ipca:7,ipynb:[2,10,19,32,34],ipython:[1,2,3,5,6,7,10,13,19,22],iri:6,irreduc:3,irrelev:[3,13,22],irrespect:[0,2],ising:5,ismov:[11,20],isnul:3,isol:[10,12,19,21],isomap:7,isotrop:[15,24],issu:[0,5,27,28],item:[0,2,3],items:[2,10,19],iter:[1,3,5,6,7,8,11,20],its:[0,2,3,4,5,6,7,8,9,10,11,12,13,14,15,18,19,20,21,22,23,24,27,28,32,34],itself:[1,3,5,11,12,13,14,20,21,22,23],j_lasso_sk:3,j_ridge_sk:3,j_sk:3,jackknavg:1,jackknif:0,jackknstd:1,jackknvar:1,jackknvec:1,januari:7,jargon:1,jensen:26,jeremi:26,job:[6,11,20],john:26,join:[2,3,7,10,12,13,14,19,21,22,23],joul:[10,19],journal:1,jrt:26,juli:26,jump:1,jupyt:[0,2,9,10,16,17,18,19,29,32,33,34],jupytext:[32,34],just:[0,1,2,3,4,5,6,7,8,10,11,12,13,14,19,20,21,22,23,26],justif:[0,2],k1v:[13,22],k1x:[13,22],k2v:[13,22],k2x:[13,22],k3v:[13,22],k3x:[13,22],k4v:[13,22],k4x:[13,22],k_1:[13,22],k_2:[13,22],k_3:[13,22],k_4:[13,22],kappa_d:1,karush:6,keep:[0,2,3,5,7,9,10,11,12,13,16,17,18,19,20,21,22,28,33],keepdim:[3,5],kei:[3,5,13,22],kepler:[14,23,26],kept:[8,13,22],kera:[0,2,10,19],kernel:[3,5,32,34],kernel_regular:[5,8],kernelpca:7,kev:[2,3],keyword:[2,10,19],kfold:3,kick:5,kilogram:[10,19],kind:[0,3,5,6,8,11,14,20,23,32,34],kindergarten:0,kinet:[10,11,12,13,14,19,20,21,22,23],kkt:6,knight:[32,34],know:[0,1,2,3,6,10,11,13,14,15,19,20,22,23,24,26],knowledg:[0,2,3,9,18,26],known:[0,1,3,4,5,6,7,8,10,11,12,13,15,19,20,21,22,24,26],kondev:2,kpca:7,kr_:[14,23],krh:[14,23],kuhn:6,kutta2:[13,22],kutta4:[13,22],kutta:26,kwarg:3,kwown:3,l1_l2:8,l2_loss:5,l_1:4,l_2:4,l_j:5,l_z:[12,21],la_:[16,17,33],la_i:5,la_k:5,lab:[0,14,23],label:[1,2,3,4,5,6,8,11,12,13,14,15,20,21,22,23,24],labels:[3,6],labels_shuffl:5,laboratori:[11,14,15,20,23,24],lack:[0,2],lad:0,lagrang:[6,7,12,21,26],lagrangian:[12,21,26],lai:[9,18],lambda:[1,2,4,5,6,8],lambda_0:7,lambda_1:[3,6,7],lambda_2:[6,7],lambda_:7,lambda_i:[6,7],lambda_iy_i:6,lambda_jy_iy_j:6,lambda_k:6,lambda_n:[3,6],land:[3,6],landmark:6,langl:[1,3,7],langrangian:26,languag:[2,5,6,8,10,19,27,32,34],lapack:[2,10,19],larg:[0,1,2,3,5,6,7,9,10,11,13,14,18,19,20,22,23,26],larger:[1,2,3,6,7,8,9,13,18,22],largest:[6,7,8],lass:0,lasso:4,lasso_sk:3,last:[0,1,2,3,5,8,10,11,12,13,14,15,19,20,21,22,23,24],later:[0,1,2,3,4,5,6,8,10,11,12,13,14,19,20,21,22,23],latest:26,latex:[2,9,10,18,19],latitud:[15,24],latter:[0,1,2,3,4,6,7,10,11,12,13,14,15,19,20,21,22,23,24],lattic:5,launch:0,law:[0,2,5,9,10,13,14,15,18,19,22,23,24,26],layer:2,lbfg:7,lbl:3,lcc:3,lda:7,ldot:[1,3,7],lead:[0,1,2,3,4,5,6,7,8,10,11,12,13,14,15,19,20,21,22,23,24],leader:0,leadership:27,leaki:5,learn:[4,6,8,10,19],learnabl:8,learning_r:[5,6],learning_rate_init:[2,5],least:[0,1,2,4,6,7,10,11,19,20],leav:[3,5,7,8,12,14,21,23],lectur:[0,2,3,5,7,10,13,15,19,22,24],lecturenot:28,left:[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,18,19,20,21,22,23,24],leftarrow:[5,6],legend:[2,3,6,7,8,10,16,17,19,33],len:[1,2,3,5,6,7,8,10,19],length:[1,2,3,5,6,7,8,10,11,12,13,19,20,21,22],length_of_sequ:8,lengthi:[15,24],leq:[1,2,3,4,6],less:[0,1,2,3,5,6,9,10,13,14,15,18,19,22,23,24,32,34],let:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,18,19,20,21,22,23,24],letter:[1,2,3,10,19],level:[2,3,5,9,10,15,18,19,24,27],levi:[12,21],lhs:[10,19],lib:[3,6,7],liblinear:6,librari:[0,1,3,5,7,8,9,18],licens:[2,5,10,19],lie:[1,3,7],lies:[6,7,9,18],life:[0,2,5,6,9,15,18,24],light:[2,11,20],lightgrai:[14,23],like:[0,1,2,3,4,5,7,8,9,10,11,12,13,14,15,18,19,20,21,22,23,24,26,29,32,34],likelihood:[0,3,5],lim_:[1,10,19],limit:[3,6,7,9,10,11,12,13,15,18,19,20,21,22,24,26],lin_clf:6,lin_model:3,linalg:[1,2,3,6,7],line1:6,line2:6,line2d:[16,17,33],line3:6,line:[0,1,2,3,6,7,8,9,10,13,15,16,17,18,19,22,24,32,33,34],linear:[0,1,4,5,7,8,10,11,13,19,20,22,26],linear_model:[2,3,6,7],linear_regress:3,linearli:[3,7,11,13,20,22],linearregress:[2,3],linearsvc:6,liner:[5,8],linestyl:[14,23],linewidth:[2,3,6,8],link:[2,5,8,15,24],linpack:[2,10,19],linreg:2,linspac:[1,2,3,6,13,14,16,17,22,23,33],linux:[2,5,10,19],liquid:[2,3],list:[2,3,8,10,13,19,22,28,29],listcomp:3,literatur:[4,5,14,23,26],littl:[5,8,10,14,19,23,28],live:6,lle:3,lmb:3,lmbd:[2,5,8],lmbd_val:[2,5,8],load:[2,3],load_boston:3,load_breast_canc:7,load_data:8,load_digit:[5,8],load_iri:6,loc:[2,3,6,7,8,10,19],local:[3,4,5,7,8,12,13,14,15,21,22,23,24,28,29],locat:[6,8,14,23,26],log10:[3,9,18],log:[2,3,4,5,7,10,19],log_:2,logarithm:[2,4,10,19],logic:[0,2],logist:[2,3,6],logisticregress:7,logit:[4,5],logreg:7,logspac:[2,3,5,8,16,17,33],longer:[1,2,6,8,10,11,12,13,14,19,20,21,22,23],loocv:3,look:[0,1,2,3,5,6,7,8,10,11,12,14,15,19,20,21,23,24,28,32,34],loop:[1,2,3,5,10,11,12,19,20,21],loos:[15,24],lorentz:[15,24],lose:5,loss:[0,2,3,4,5,6,7,8,10,11,12,19,20,21],lost:[13,22],lot:[3,9,16,17,18,33],love:[11,13,20,22],low:[1,3,7,14,15,23,24],lower:[2,3,5,8,10,12,13,14,15,19,21,22,23,24],lowercas:[2,10,19],lowest:1,lstat:3,lstm:8,lstm_2layer:8,lstsq:3,lvert:5,m_0:[12,21],m_1:[14,23],m_1m_2:[14,23],m_2:[14,23],m_e:[12,21],m_h:2,m_i:[12,21],m_iv_i:[12,21],m_l:5,m_n:2,m_p:2,machin:[3,4,5,8,10,19],made:[0,2,4,5,7,8,9,12,13,18,21,22],mae:2,magnitud:[5,7,10,11,12,13,14,15,19,20,21,22,23,24],mai:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,15,18,19,20,21,22,24,27,29],mail:[26,27],main:[0,2,3,4,8,13,22,29],mainli:[2,3,4,11,13,20,22],maintain:[3,11,13,20,22,27],major:[2,3,5,9,10,18,19],make:[0,1,2,3,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,26,27,28,33],make_axes_locat:3,make_moon:6,make_pipelin:[2,3],makedir:[2,3,7,10,12,13,14,19,21,22,23],makeplot:[2,10,19],malign:7,malth:26,manag:[2,5,8,10,19],mani:[0,1,2,3,4,6,7,8,10,11,13,14,15,19,20,22,23,24,25,26,30,32,34],manifestli:[12,21],manifold:7,manner:[8,15,24],manual:[9,18,29],map:[0,1,2,3,4,5,6,7],margin:[0,2,6,12,21],marit:3,mark:[2,10,19],markedli:[32,34],marker:[2,3],market:0,markov:0,markup:[32,34],marsaglia:1,mask:1,mass:[3,5,10,11,12,13,15,19,20,21,22,24,26],massag:2,masses2016:2,masses2016ol:3,masses2016tre:2,masseval2016:[2,3,10,19],massiv:[15,24],master:26,match:[5,14,23],materi:[0,2,3,4,5,7,8,9,10,11,12,13,15,18,19,20,21,22,24],math:[1,2,4,5,9,10,12,13,14,16,17,18,19,21,22,23,33],mathbb:[1,3,4,5,6,7],mathbf:[1,2,3,4,6,10,19],mathcal:[3,4,5],mathemat:[0,1,2,7,9,11,13,18,20,22,26],mathemati:[2,10,19],mathemt:3,mathrm:[1,2,3,4,5,6,7,10,11,12,13,15,19,20,21,22,24],matmul:[3,5],matplotlib:[1,2,3,4,5,6,7,8,9,10,12,13,14,16,17,18,19,21,22,23,33],matric:[3,4,6,7,8],matrix:[4,6,8],matter:[7,8],matur:[9,18],max:[2,3,5,7,8,12,13,14,21,22,23],max_depth:2,max_it:[2,5,6,7],maxdegre:3,maxim:[4,5,6,7],maxima:[14,23],maximum:[2,3,6,7,8,11,12,13,14,20,21,22,23],maxpolydegre:3,maxpooling2d:8,mbox:[3,16,17,33],mcculloch:5,mcint:1,mcintsqr2:1,mdoel:8,mean:[0,2,3,4,5,8,9,10,11,12,13,15,16,17,18,19,20,21,22,24,26,33],mean_absolute_error:[2,3],mean_i:1,mean_squared_error:[2,3,7,8],mean_squared_log_error:2,mean_x:1,meaning:0,meansquarederror:2,meant:[0,4,9,14,18,23,26],meantempvec:1,meanvec:1,measur:[1,2,3,5,7,10,11,13,14,15,19,20,22,23,24],mechan:[0,1,8,9,11,12,13,15,18,20,21,22,24,34],mechanist:0,media:[26,27],median:[2,3,7],mediaspac:26,medicin:[0,5],medium:[6,8,11,13,16,17,20,22,33],medv:3,meet:[9,18],mehta:3,member:27,memori:[5,7,8,13,22],mention:[1,2,5,11,13,14,20,22,23],mere:[14,23],mersienn:1,mesh:[10,11,12,19,20,21],meshgrid:[3,6,7],messi:[13,14,22,23],met:[0,2,3,6,7,12,21],metadata:[32,34],meter:[10,14,19,23],method:[0,2,4,5,6,7,8,9,11,14,18,20,23,26],methodolog:[9,18],metric:[2,3,5,7,8],metropoli:0,mev:[1,2,3],mgh:[14,15,23,24],mgl:[13,22],mglearn:[2,10,19],mgsin:[11,13,20,22],mgx:[15,24],mgy:[15,24],mhjensen:26,michael:8,michigan:34,mid:[5,7,12,21],midel:8,midpoint:[12,13,21,22,26],midterm:26,might:[3,5,7,11,12,14,20,21,23,28],mileag:[11,20],miller:1,million:[2,3,7],mimic:5,min:[3,6,7,14,23],min_:3,mind:[0,2,3],mindboard:8,mine:[2,10,19],mini:[5,7],minibatch:[5,7],minim:[0,1,2,3,4,5,7,13,14,22,23],minima:[2,4,5,14,23],minimum:[3,5,6,7,13,14,22,23],minmaxscal:[3,7],minneapoli:[15,24],minor:1,minut:[14,23],misclassif:6,misclassifi:6,mismatch:5,miss:3,mix:[5,10,11,13,19,20,22],mkdir:[2,3,7,10,12,13,14,19,21,22,23],mlab:[1,3],mle:[1,4],mlk:26,mlpclassifi:5,mlpregressor:2,mnist:[5,7],mod:1,mode:[1,13,22],model:[1,4,6,7,9,14,18,23],model_select:[2,3,5,7,8],modern:[0,2,3,4,9,18],modif:[5,14,23],modifi:[2,3,4,5,6,10,15,19,24],modul:[2,3,5,7,9,10,18,19],modular:[1,9,18],modulo:1,moe:[3,7],molecul:[11,20],molten:[15,24],moment:3,momenta:[10,14,19,23],momentum:[10,11,13,14,19,20,22,23,26],mondai:26,monoton:[1,5],monstertruck:0,mont:[0,1,2,3],montecarlocycl:1,moon:[14,23],moor:3,more:[0,1,5,9,12,14,15,16,17,18,21,23,24,26,28,29,32,33,34],moreoev:34,moreov:[2,8,32],morten:26,most:[0,1,2,3,4,5,6,7,8,9,10,11,12,14,15,18,19,20,21,23,24,32,34],mostli:[5,7],motion:[0,2,9,12,13,15,18,21,22,24,26],motiv:[5,11,13,20,22],mount:[15,24],move:[1,2,3,4,5,11,12,14,15,20,21,23,24,26],mpatch:[10,19],mph:[11,20],mpl:[2,10,13,19,22],mpl_toolkit:3,mplregressor:5,mr_:[14,23],mrv_:[14,23],mse:[2,3,7,8],msle:2,msu:26,mt19937_64:1,mu0:1,mu1:1,mu2:1,mu_:1,mu_k:[11,20],mu_n:7,mu_x:1,much:[0,1,2,3,5,6,7,8,9,10,11,12,13,14,15,18,19,20,21,22,23,24],multi:[2,3,4,8,10,19],multiclass:[4,5],multidimension:[2,5,7,10,19],multinomi:4,multipl:[1,3,4,8,10,11,19,20],multipli:[1,3,7,8,10,11,12,15,19,20,21,24,26],multiplum:6,multitud:2,multivari:[0,1,2,7],multivariate_norm:7,murphi:7,must:[1,3,5,6,8,11,13,14,15,20,22,23,24,32,34],mutat:4,mutual:[3,8],mv_0:[12,21],mv_:[14,23],mv_e:[12,21],mvn:1,mx_:1,mydirectivenam:[32,34],myriad:[2,10,19],mz1:1,mz2:1,n_0:[1,5],n_b:1,n_boostrap:3,n_bootstrap:3,n_categori:[5,8],n_compon:7,n_featur:5,n_filter:8,n_hidden_neuron:[2,5],n_i:1,n_input:[5,8],n_iter_i:7,n_l:[1,5],n_neurons_connect:8,n_neurons_layer1:5,n_neurons_layer2:5,n_sampl:[3,6],n_split:3,nabla:[5,12,14,21,23],nabla_i:[12,21],nabla_jv:[12,21],naimi:2,naiv:4,name:[0,2,3,4,5,6,8,10,11,12,13,15,19,20,21,22,24,28,32,34],name_scop:5,namespac:1,narrow:28,nation:[5,27],nativ:[2,10,19],natur:[0,1,2,5,6,8,9,11,13,18,20,22],navier:5,navig:29,nbconvert:[2,10,19],nboot:1,ndarrai:3,nearest:[3,5,7,8],nearli:[15,24],neat:[2,10,19],neccesari:3,necessari:[0,5,6,8,11,13,20,22],necessarili:[0,1,7,14,23],neck:4,need:[0,1,3,5,6,7,8,11,12,13,14,15,20,21,22,23,24],neg:[1,2,3,4,5,11,13,14,15,20,22,23,24],neg_mean_squared_error:3,neglect:[1,13,14,22,23],neglig:[1,15,24],neighbor:[3,7,8],neq:1,nervou:5,nest:5,net:[5,8,11,12,14,15,20,21,23,24],netlib:[2,10,19],network:[0,3,4,10,19],neural:[0,3,4,10,19],neural_network:[2,5],neuralnetwork:5,neuralnetworktensorflow:5,neurosci:[32,34],neutral:2,neutron:2,never:[1,3,5,13,14,22,23],new_hobbit:[2,10,19],new_sig:3,newaxi:[2,3,8],newdata:[10,19],newli:[0,10,19],newton1:[11,20],newton:[1,4,5,6,10,13,15,19,22,24,26],newtonian:[15,24],next:[1,2,5,6,8,10,12,13,14,15,19,21,22,23,24],next_input:8,nice:[2,3,5,7,10,19,26],nielsen:8,nitric:3,nlambda:3,nldr:7,nm_n:2,nmse:3,node:5,nois:[2,3,6],noisi:[3,5],non:[1,3,4,5,7,8,12,13,21,22,26],none:[1,3,5,8],nonlinear:[3,5,6,7,8],nonneg:3,nonparametr:3,nonsens:1,nonsingular:[2,10,19],nonumb:[4,6,11,12,13,14,15,20,21,22,23,24],nor:[0,5],norm:[2,3,5,6,7,11,12,13,20,21,22],normal:[0,2,3,4,5,6,7,8,9,10,11,12,13,15,18,19,20,21,22,24],normali:[2,10,19],normpdf:3,north:[15,24],northern:[15,24],notat:[1,3,10,12,19,21],note:[1,2,3,4,5,6,7,8,10,11,12,13,14,19,20,21,22,23,28,32,34],notebook:[0,2,5,8,9,10,18,19,32,34],notesapril10:26,notesapril13:26,notesapril15:26,notesapril17:26,notesapril1:26,notesapril20:26,notesapril22:26,notesapril24:26,notesapril3:26,notesapril6:26,notesapril8:26,notesmarch13:26,notesmarch16:26,notesmarch18:26,notesmarch20:26,notesmarch23:26,notesmarch25:26,notesmarch27:26,notesmarch30:26,notesmarch9:26,noth:[1,3,5,6,10,19],notic:[1,2,5,10,12,15,19,21,24],notion:8,novel:[3,8],novemb:5,now:[1,2,4,5,6,8,10,11,12,13,14,15,19,20,21,22,23,24],nowadai:[0,2,5,8,9,18],nox:3,nozzl:[14,23],npt:[9,18],nsampl:3,nscl:26,nspin:1,nuclear:[13,14,22,23],nuclei:[1,2,14,23],nucleon:[2,3],nucleu:[2,14,23],number:[0,2,4,5,6,8,9,10,11,12,13,14,15,18,19,20,21,22,23,24],numer:[0,2,3,5,7,9,10,11,14,18,19,20,23,26],numpi:[1,3,4,5,6,7,8,9,12,13,14,16,17,18,21,22,23,33],nutshel:7,obei:[7,11,14,15,20,23,24],object:[2,3,6,8,10,14,19,23,26],oblig:27,observ:[3,4,5,6,7,8,11,14,15,20,23,24],obstacl:0,obtain:[1,3,4,5,6,10,11,12,13,14,15,19,20,21,22,23,24],obviou:[0,1,3,7,12,21],obviouli:2,obvious:[0,2,3,10,11,12,19,20,21],occupi:3,occur:[1,2,6,11,13,14,15,20,22,23,24],ocean:[15,24],odd:[2,3,4,7,12,13,21,22],odot:[12,21],off:[1,3,5,8,13,14,22,23],offend:2,offens:27,offer:[2,3,7,9,18],offic:26,offici:[27,28],offlin:27,ofil:1,ofstream:1,often:[0,1,2,3,4,5,6,7,9,10,11,13,14,15,18,19,20,22,23,24],ofter:[2,10,19],oint:[14,23],old:[5,12,21],ols_inv:3,ols_sk:3,ols_svd:3,omega02:[14,23],omega:[3,10,11,12,13,14,15,19,20,21,22,23,24],omega_0:[11,14,15,20,23,24],omega_0a:[13,22],omega_0t:[11,13,14,15,20,22,23,24],omega_c:[11,20],omega_ct:[11,20],omega_cv_i:[11,20],omega_i:[15,24],omega_t:[13,22],omega_x:[15,24],omega_xv_i:[15,24],omega_xv_z:[15,24],omega_yv_x:[15,24],omega_yv_z:[15,24],omega_z:[15,24],omega_zt:[15,24],omega_zv_i:[15,24],omega_zv_x:[15,24],omegatild:[13,22],omit:3,omnipres:[13,22],onc:[1,3,5,7,11,13,14,20,22,23,32,34],one:[0,1,2,4,6,7,8,9,12,13,14,15,18,21,22,23,24,32,34],onehot:5,onehot_vector:5,ones:[0,2,3,6,7,10,12,19,21],onli:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,18,19,20,21,22,23,24,32,34],onlin:[0,2,7,27],onto:[7,11,20],open:[0,1,2,3,5,9,10,18,19,27,28,29],oper:[0,1,2,3,5,7,8,10,12,13,19,21,22,28],operation:1,oplu:1,opportun:[0,2],oppos:[3,11,20],opposit:[5,6,11,12,14,15,20,21,23,24],oppotun:[9,18],opt:[0,2,3,5,6,7],optim:[0,4,7,8],optimis:8,option:[0,5,6,7,8],optmiz:5,orang:2,orbit:[12,21,26],order:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,18,19,20,21,22,23,24],ordinari:[4,7,8,11,13,20,22],org:[2,5,7,8,10,19],organ:[3,10,19],orient:[1,3,10,19,27],origin:[1,2,3,5,6,7,8,10,11,12,13,14,15,19,20,21,22,23,24],orthogn:3,orthogon:[2,3,6,7,10,15,19,24],orthonorm:3,oscil:[11,12,15,20,21,24,26],oscillatori:[13,22],osx:[2,10,19],other:[0,2,3,4,6,9,11,12,13,14,15,18,20,21,22,23,24,27],otherwis:[2,3,4,5,12,14,21,23,27],ouput:5,our:[0,4,6,8,9,12,14,15,18,21,23,24],ourselv:[0,2,3,6,7,10,11,13,14,15,19,20,22,23,24],out:[1,2,3,4,5,6,7,10,12,13,14,15,16,17,19,21,22,23,24,31,33],outcom:[1,3,4,5,9,15,18,24],outer:[3,5],outfilenam:1,outgo:[14,23],outlier:[2,3,6,7],outlin:[3,7,9,18],output:[0,2,3,4,5,6,8,10,11,19,20,32,34],output_bia:5,output_bias_gradi:5,output_weight:5,output_weights_gradi:5,outputlayer1:5,outputlayer2:5,outsid:8,outward:[12,14,15,21,23,24],over:[0,1,2,3,5,9,10,11,12,13,14,18,19,20,21,22,23],overal:[5,14,23],overarch:[0,11,20],overcom:[5,11,20],overdetermin:3,overfit:[3,5,8],overflow:5,overhead:5,overlap:[6,7,8],overlin:[2,3,7,10,19],overst:[0,2],overtrain:8,overview:[0,8,26],owe:0,own:[0,2,5,6,8,9,12,13,15,18,21,22,24,26],owner:3,oxid:3,p_0:[14,23],p_0b:[14,23],p_0b_:[14,23],p_2:[12,21],p_f:[14,23],p_fr:[14,23],p_i:1,p_j:1,p_n:1,p_x:[1,12,21],pack:2,packag:[0,1,3,5,6,7,8,13,22],pad:8,page:[2,29,31,32,34],pai:5,painless:2,paint:[14,23],pair:[1,2,3,8,11,20],panda:[8,12,13,14,21,22,23],panel:[2,10,19],paper:[5,9,18,26],paradigm:[0,2],parallel:[2,10,11,12,14,19,20,21,23],paramet:[0,1,2,5,6,7,8,10,14,19,23],parameter:[0,3,11,13,20,22],parametr:[2,3],paramt:8,pari:[15,24],park:1,part:[0,1,2,3,5,7,9,10,13,14,15,18,19,22,23,24,26,28],partform:[13,22],partial:[1,3,4,5,6,7,11,12,20,21],partial_:[12,21],partial_i:[12,21],partial_r:[14,23],partial_ru:[14,23],partial_ru_:[14,23],partial_tv:[12,21],partial_x:[12,13,21,22],partial_xv:[12,13,21,22],partial_yv:[12,21],partial_z:[12,21],partial_zv:[12,21],particip:[14,23,27],particl:[1,2,3,8,11,12,13,14,15,20,21,22,23,24,26],particular:[0,1,2,3,5,7,8,9,10,11,13,14,18,19,20,22,23,26],particularli:[1,3,6,7,9,13,18,22],partit:5,partli:3,paslei:[32,34],pass:[1,8,13,14,22,23],passeng:0,past:1,patch:[1,3,10,19],path:[2,3,7,10,11,12,13,14,15,19,20,21,22,23,24,32,34],patient:4,patter:8,pattern:[0,2,5,8,15,24,32,34],pauli:2,pavement:[11,20],pca:3,pdf:[0,2,3,8,10,19,26],pedagog:[3,9,18],pen:[9,18],penal:3,penalti:3,pencil:[9,18,26],pend1:[11,13,20,22],pend2:[11,13,20,22],pendulum:26,penros:3,peopl:[0,3,5],per:[2,3,9,10,14,15,18,19,23,24],perceiv:0,percentag:[3,7,26],perceptron:[2,4],peregrin:[2,10,19],perfect:[2,3,5,14,23],perfectli:[3,11,20],perform:[1,2,6,7,8,10,11,12,13,14,15,19,20,21,22,23,24],performac:8,perhap:[0,2,3,11,12,20,21],period:[5,11,14,15,20,23,24],perman:27,permiss:27,permut:[7,12,21],perp:[14,23],perpendicular:[11,14,15,20,23,24],person:[4,27],perspect:[12,21],pertain:[10,19],pertin:[5,9,18],perturb:[14,23],petal:6,petersen:1,phantom:1,phase:[3,5,11,13,15,20,22,24],phasespac:[14,23],phenomena:[1,13,22],phi:[6,10,11,12,13,14,19,20,21,22,23],phi_a:[15,24],phi_b:[15,24],phi_k:6,philosophi:[13,22],phone:26,phonon:[13,22],phrase:0,phy321:34,physic:[0,1,2,3,4,5,8,9,10,11,12,13,14,15,18,19,20,21,22,23,24,27],physics321:26,piazza:26,pick:[1,5,7,13,22],pictur:[0,2,13,22],pie:[2,10,19],piec:[5,7,12,14,21,23],pillow:[2,10,19],pinv:3,pip3:[2,5,10,19],pip:[2,5,10,19,29],pipelin:[2,3,6],pippin:[2,10,19],pitfal:[12,21],pitt:5,pixel:[5,8],pixel_height:[5,8],pixel_width:[5,8],place:[2,3,6,10,14,15,19,23,24],placehold:5,plai:[0,2,3,6,8,11,12,13,20,21,22],plain:[5,6],plan:[3,26],plane:[6,10,14,15,19,23,24],planet:[14,23],planetari:[14,23],plateau:1,platform:[2,10,19],plausibl:5,pleas:[7,26,28],plenti:5,plethora:5,plot:[1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,17,18,19,20,21,22,23,33],plot_count:3,plot_dataset:6,plot_max:8,plot_min:8,plot_predict:6,plt:[1,2,3,4,5,6,7,8,9,10,12,13,14,16,17,18,19,21,22,23,33],plu:[2,13,15,22,24],plug:[11,13,14,20,22,23],png:[2,3,7,10,12,13,14,19,21,22,23],point:[0,1,2,3,4,5,6,7,9,10,11,12,13,14,15,18,19,20,21,22,23,24],poisson:0,polar:[12,13,14,15,21,22,23,24],pole:[15,24],poli:[3,6],polici:27,polit:[0,27],poly100_kernel_svm_clf:6,poly3:2,poly3_plot:2,poly_featur:6,poly_kernel_svm_clf:6,polydegre:3,polym:5,polynomi:[0,2,3,4,6,7,12,21],polynomial_featur:3,polynomial_svm_clf:6,polynomialfeatur:[2,3,6],polytrop:3,pool:8,pool_siz:8,poor:5,popul:[2,3],popular:[0,1,2,3,5,6,7,8,10,13,19,22],popularli:2,portion:7,pose:[0,1,7],posit:[1,3,4,5,6,7,10,11,12,13,14,15,19,20,21,22,23,24,27],possess:[15,24],possibl:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,18,19,20,21,22,28],post:[16,17,26,27,28,33],postpon:[3,10,19],potenti:[0,2,3,5,9,10,11,12,13,18,19,20,21,22,26],pott:5,power:[2,3,5,6,7,9,11,13,18,20,22,32,34],practic:[1,3,4,6,14,23],practition:[2,5,8],pre:[32,34],preced:[1,5,7],preceed:8,preceq:6,precess:[15,24],precis:[0,1,2,3,7,10,12,14,19,21,23],precmess:[15,24],pred:3,predict:[0,2,3,4,5,6,7,10,15,19,24,32,34],predict_prob:5,predictor:[2,3,4,7],prefer:[2,5,6,7,13,22],prepar:[2,10,19],preprocess:[2,6,8],prerequisit:[0,2],presenc:[11,13,20,22],present:[0,1,2,3,5,9,12,18,21],preserv:[7,8],pressur:[15,24],pretti:[2,6,10,19],prevent:1,previou:[1,3,5,6,7,8,10,11,13,14,19,20,22,23],previous:[1,8],price:[3,8],primal:6,primari:[0,4],prime:[1,12,13,21,22],princip:[3,13,22],principl:[2,3,4,6,11,15,20,24],print:[1,2,3,5,6,7,8,10,12,14,19,21,23,32,34],print_funct:6,printout:[3,10,19],prior:[0,2,3],prioriti:0,privaci:0,privat:[0,2,27],privileg:[10,19],pro:26,prob:1,probabilist:3,probabl:[2,3,4,5,8,14,23],problem:[0,1,2,4,7,9,10,11,13,15,18,19,20,22,24,26],proce:[0,2,3,6,7,13,14,22,23],procedur:[6,7,15,24],proceed:[11,20],process:[0,1,2,3,9,10,11,18,19,20,26,29],prod_:[4,5],produc:[0,1,2,3,5,7,8,9,10,15,18,19,24],product:[2,3,4,5,6,8,10,12,14,19,21,23],profess:3,profession:27,profound:[12,21],program:[1,2,3,5,6,8,9,10,11,13,18,19,20,22],progress:5,prohibit:3,project:[0,1,2,3,5,7,9,15,18,24,26,27,28,29],project_root_dir:[2,3,7,10,12,13,14,19,21,22,23],promin:5,promis:6,pronounc:[2,10,19],proof:[3,5,12,21],prop:3,prop_cycl:[16,17,33],propag:8,proper:[0,2,3],properli:[5,6,9,18,32,34],properti:[2,5,7,8,10,11,15,19,20,24],proport:[1,2,3,5,7,11,13,20,22],propos:[5,28],propto:[14,23],protect:0,proton:[2,14,23],prove:[8,15,24],provid:[1,2,3,5,6,8,9,10,11,13,14,15,18,19,20,22,23,24],proxi:5,pseudoinvers:3,pseudorandom:1,psycholog:[0,2],ptratio:3,pub:26,publish:27,pull:[14,23,26,28,29],pulsar:5,punish:[0,2,5],pure:[1,13,14,22,23],purpos:[0,2,5,9,11,13,18,20,22,32,34],push:[15,24,28,29],put:[13,14,22,23],puzzl:[11,20],pycod:[2,10,19],pyhton2:[2,10,19],pylab:[2,10,13,19,22],pyplot:[1,2,3,4,5,6,7,8,9,10,12,13,14,16,17,18,19,21,22,23,33],python3:[2,3,6,7,10,19],python:[0,3,5,6,7,9,11,13,18,20,22,26],pytorch:[0,2,5,10,19],qbv_x:[11,20],qbv_y:[11,20],qquad:[2,7,10,19],quad:5,quadrat:[2,6,11,13,14,20,22,23],qualit:1,qualiti:[0,2,9,10,11,13,18,19,20,22],quantifi:[5,14,23],quantit:[0,3],quantiti:[1,2,3,4,5,7,11,12,13,15,20,21,22,24],quantum:[0,5,8,12,13,21,22],quartil:[3,7],question:[0,3,5,7,10,11,13,19,20,22,26],quick:[1,9,18],quickli:[7,8,12,13,14,15,21,22,23,24],quit:[3,5],quitecompl:[14,23],quot:1,r2_score:[2,3],r2score:2,r_1:[13,22],r_1t:[13,22],r_2:[13,14,22,23],r_2t:[13,22],r_h:[12,21],r_i:[12,21],rab:[12,14,21,23],race:27,rad:3,radial:[3,5,6,12,14,15,21,23,24],radialeqofmot:[14,23],radialeqofmotion2:[14,23],radialgvv:[14,23],radialhovv:[14,23],radii:[14,23],radio:[13,22],radioact:1,radiu:[3,7,11,14,15,20,23,24],rais:3,ramsai:[32,34],ran1:1,ran2:1,ran3:1,rand:[1,2,3,8,10,19],rand_max:1,randint:3,randn:[2,3,5,7,10,16,17,19,33],random:[0,2,3,5,6,8,10,14,16,17,19,23,33],random_devic:1,random_indic:[5,8],random_st:[3,6,7],randomli:[1,3,5],randomnumbergener:1,rang:[1,2,3,4,5,7,8,9,10,11,12,13,14,16,17,18,19,20,21,22,23,33],rangl:[1,3,7],rangle_x:1,rank:[3,8],raphson:[5,6],rapidli:[0,2,15,24],rare:[11,20],rate:[3,5,6,8,12,14,15,21,23,24],rather:[0,1,2,3,4,5,6,7,8,11,12,13,14,15,20,21,22,23,24],ratio:[4,7,13,14,22,23],rational:2,ravel:[3,6,7],raw:[1,8],rawdata:[10,19],rbf:[5,6,7],rbf_kernel_svm_clf:6,rbf_pca:7,rcond:3,rcparam:[1,2,5,6,8,10,13,16,17,19,22,33],rdotdot:[14,23],rdr:[13,22],reach:[1,3,5,7,8,11,12,14,20,21,23],reaction:[11,12,20,21],read:[0,1,3,5,6,8,9,10,11,12,13,18,19,20,21,22,26],read_csv:3,read_fwf:[2,3],reader:[0,1,2,10,19],readi:[2,5,6,7,9,10,18,19,28],readili:5,real:[3,4,5,7,9,10,13,14,15,18,19,22,23,24],realist:[9,18],realiti:[1,15,24],realiz:[5,13,14,22,23],realli:[2,5,11,20],reason:[0,3,5,8,9,15,18,24,27],rebenstock:26,recal:[1,2,3,5,7,10,15,19,24],recalcul:1,recast:[11,13,20,22],receiv:[1,5,8,13,22],recent:[0,2,3],recept:[5,8],receptive_field:8,recip:[2,3,4,10,19],recogn:[2,8,14,23,29],recognit:[0,2,5,8],recommend:[1,2,3,6,8,10,12,19,21,26,29],reconstruct:7,record:[11,20],recov:[10,19],recreat:1,rectangl:3,rectifi:[5,8],rectilinear:[15,24],recur:[0,2],recurs:[2,10,19],recycl:1,red:[2,3,6,8],redefin:3,reduc:[2,5,8,11,13,14,20,22,23],reduce_mean:5,reduct:[1,2,3],refactor:[9,18],refer:[0,2,3,5,7,8,9,11,13,14,15,18,20,22,23,24,26,32,34],refin:5,refit:3,reflect:[0,1,2,5,8,9,18],reformul:[11,20],reg:7,regard:[5,27],regardless:[5,14,23,27],region:[5,8,9,15,18,24],regist:1,regr_1:2,regr_2:2,regr_3:2,regress:[0,5,6],regressor:[3,4],regular:[4,15,24,32,34],regularizer_loss:5,regularizer_loss_fc1:5,regularizer_loss_fc2:5,regularizer_loss_out:5,reinforc:[0,2,6],reject:27,rel:[0,1,2,3,4,5,8,9,11,12,13,15,18,20,21,22,24,26,32,34],relat:[0,1,2,3,5,7,8,10,11,12,13,14,15,19,20,21,22,23,24,26],relationship:[0,2,3,7,8,15,24],relatist:[15,24],relativeerror:[3,9,18],relativist:[11,13,20,22],releas:[2,5,10,19,28],relev:[1,2,3,7,9,10,18,19],reli:[0,2,3,6,7,11,20],reliabl:[1,4,12,21],religion:27,relu:8,remain:[1,3,5,8,11,13,20,22],remaind:1,remark:[5,15,24],rememb:[2,6,10,13,19,22,28],remind:[0,1,3,7,9,11,13,14,15,18,20,22,23,24],remov:[3,8,27,29],render:[2,3,7,10,19,29,32,34],reorder:4,reorgan:[2,10,19],repeat:[1,2,3,5,7,8,10,11,12,13,14,15,19,20,21,22,23,24],repeated:0,repeatedli:[1,3,13,22],repel:[14,23],repercuss:27,repetit:3,rephras:[11,20],replac:[2,3,5,8,9,10,12,13,14,18,19,21,22,23,26],replica:3,repo:[28,29],report:[9,18,27],repositori:[0,29],repres:[0,1,2,3,4,5,6,8,9,10,11,12,13,14,15,18,19,20,21,22,23,24,27],represent:[0,1,3,5,7,8,13,22,27],reproduc:[0,1,2,5,9,14,16,17,18,23,26,28,33],repuls:[2,14,23],request:[0,2,28,29],requir:[2,3,5,6,7,8,9,10,11,13,14,18,19,20,22,23,29],resampl:[0,1],rescal:[3,5,7],research:[0,2],resembl:[1,3,15,24],reserv:[1,3,5],reset:1,reshap:[2,3,5,6,8,10,19],residenti:3,residu:3,resist:[12,13,21,22,26],reson:26,respect:[0,1,2,3,4,5,6,7,8,9,10,11,13,15,18,19,20,22,24,27],respond:[5,27],respons:[3,4,5,11,15,20,24],rest:[3,7,10,11,13,14,15,19,20,22,23,24,32,34],restat:[3,5],restor:[11,20],restrict:[0,5,15,24],result:[0,1,2,4,5,6,7,8,9,10,12,13,14,15,18,19,21,22,23,24,26,32,34],ret:3,retail:3,retain:[3,12,21],rethink:3,return_sequ:8,reus:[8,12,13,21,22],reveal:[0,5],revers:[5,14,23],revert:[10,19],review:27,revis:[5,10,19],revisit:[13,22],reward:[0,2],rework:5,rewrit:[1,2,4,5,6,10,11,12,13,14,15,19,20,21,22,23,24],rewritten:[1,3,6,14,23],rgb:8,rho:[3,14,15,23,24],rho_m:[11,20],rho_ma:[11,20],rhs:[7,10,19],rich:[0,2],ridg:[2,4,7],ridge_sk:3,right:[1,2,3,4,5,6,8,9,10,11,12,13,14,15,18,19,20,21,22,23,24,27],rightarrow:[1,3,5,6,7,10,11,12,13,14,15,19,20,21,22,23,24],rigid:[15,24],rigor:[0,2,15,24],ring:[3,14,23],rise:[3,15,24],risk:2,river:3,rk2:[13,22],rk4:[13,22],rmin:[14,23],rminofa:[14,23],rmse:3,rmsprop:8,rnn1:8,rnn2:8,rnn:5,rnn_2layer:8,rnn_input:8,rnn_output:8,rnn_train:8,rntrick1:1,rntrick2:1,rntrick3:1,rntrick4:1,road:0,robert:[32,34],robust:2,robustscal:[3,7],rocket:[12,21],role:[0,2,3,6,12,21],rolenam:[32,34],roll:[3,13,22],room:[0,2,3,26],root:[0,1,2,3,13,22],rot:[2,10,19],rotat:[6,10,11,12,14,19,20,21,23,26],roughli:[5,8,14,23],round:[3,7,15,24],routin:[2,10,19],row:[2,3,5,7,10,19],rp_f:[14,23],rrr:3,rtotheta:[14,23],rule:[3,9,10,14,15,18,19,23,24,26],run:[0,2,3,5,6,7,9,10,11,14,18,19,20,23,29,32,34],rung:26,runner:[10,19],runtim:[3,5],runtimewarn:5,rust:2,ruthtraj:[14,23],rvert:5,rvert_2:5,rwidth:3,s_0:[15,24],s_1:3,s_h:[12,21],s_i:[3,4],s_j:3,s_k:3,safe:1,sai:[0,1,2,3,4,5,6,7,8,10,11,14,15,19,20,23,24],said:3,sake:[2,3,4,7],sale:2,sam:[2,10,19],same:[1,2,5,6,7,8,10,11,12,13,14,15,19,20,21,22,23,24,32,34],sampl:[0,2,3,4,5,6,8,14,15,16,17,23,24,31,33],sampleexptvari:1,sampler:0,samwis:[2,10,19],sanitize_sequ:3,satisfactori:2,satisfi:[1,3,5,6,11,13,14,20,22,23],satur:[3,5],save:[2,3,7,8,9,10,12,13,14,18,19,21,22,23],save_fig:[2,3,7,10,12,13,14,19,21,22,23],savefig:[1,2,3,7,10,12,13,14,19,21,22,23],savetxt:8,saw:[12,13,21,22],scalar:3,scale:[2,3,4,5,6,7,9,10,14,18,19,23,26],scaler:[3,6,7],scan:4,scatt:[14,23],scattangl:[14,23],scatter:[2,3,6,10,19,26],scenario:3,scene:7,scheme:[5,10,19],schrage:1,sci:26,scienc:[0,1,2,5,8,9,10,18,19,26],scientif:[0,2,9,10,18,19,26],scientist:[0,2],scikit:[0,6,8,10,19],scikitlearn:2,scipi:[2,3,10,13,19,22],scl:3,scope:[5,28],score:[2,3,5,7,8,26],scores_kfold:3,sea:[15,24],seaborn:[2,3,5,7,8,10,13,19,22],seamless:[2,10,19],search:[2,5,8,11,14,20,23],seat:[15,24],sec:3,second:[0,1,2,3,4,5,6,7,8,10,12,13,14,15,19,21,22,23,24,26],secondeigvector:7,secondli:[5,13,22],section:[1,8,10,11,13,15,19,20,22,24,25,26,30],sector:[0,2],secur:0,see:[0,1,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,26,29,32,33,34],seed:[1,2,3,5,6,7,8,10,16,17,19,33],seek:[5,6],seem:[5,8,13,14,22,23],seemingli:2,seen:[0,1,2,3,5,7,8,9,11,14,15,18,20,23,24],seismic:3,seldomli:[2,9,18],select:[2,3,5,6,7,10,19],self:[0,1,3,5,9,18],sell:8,semi:6,semilogx:3,send:[5,26,28],sens:[0,3,6,11,20],sensibl:8,sensit:[2,3,7],sensor:0,sent:26,sentenc:[5,8],separ:[1,2,3,5,6,8,10,11,13,14,19,20,22,23,27],septemb:3,sequenc:[1,2,4,5,8,10,19],sequenti:[1,5,8],seri:[0,2,3,5,7,8,10,19,26],serif:[1,2,10,13,19,22],seriou:0,serv:[0,2,4,5,8,32,34],sess:5,session:[5,26],set:[0,1,2,4,7,9,10,12,13,14,15,18,19,21,22,23,24,27,28],set_:3,set_label:3,set_tick:6,set_titl:[2,4,5,7,8],set_xlabel:[2,3,4,5,7,8,10,12,13,14,19,21,22,23],set_xlim:[4,5,10,12,13,19,21,22],set_ylabel:[2,3,5,7,8,10,12,13,14,19,21,22,23],set_ylim:[4,5],set_ytick:7,set_yticklabel:3,setiosflag:1,setminu:3,setosa:6,setosa_or_versicolor:6,setp:3,setprecis:1,setup:[2,3,5,6,10,15,19,24,28],setw:1,sever:[0,1,2,3,4,5,6,7,8,9,10,12,13,18,19,21,22,26],sexist:0,sexual:27,sgd:[5,8],sgd_clf:6,sgdclassifi:6,shape:[2,3,5,6,7,8,9,10,14,15,18,19,23,24],share:[0,2,5,8,10,19],she:4,shift:[1,5],ship:8,shire:[2,10,19],shock:[13,22],shorten:8,shorter:1,shorthand:[2,10,12,19,21],shortli:[2,7,10,11,19,20],should:[0,2,3,6,8,10,11,12,13,14,15,19,20,21,22,23,24,26,29],show:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,18,19,20,21,22,23,24,27],shown:[1,5,6,7,8,11,12,20,21],showpoint:1,shrink:[3,6,8],shrinkag:3,shrunk:7,shuffl:[3,5],side:[3,5,6,10,11,14,15,19,20,23,24],sigh:[2,10,19],sigma0:1,sigma1:1,sigma2:1,sigma:[1,2,3,4,5,7,14,23],sigma_1:3,sigma_2:3,sigma_:[1,2,3,14,23],sigma_fn:[4,5],sigma_i:[2,3],sigma_j:3,sigma_m:1,sigma_n:[1,7],sigma_x:1,sigmoid:[4,6,8],sign:[1,4,5,6,11,14,15,16,17,20,23,24,26,33],signal:[5,13,22],signific:[5,15,24],significantli:[1,5,11,13,20,22],sim:[1,3,14,23],similar:[0,2,3,4,6,7,8,9,13,14,18,22,23,26,32,34],similarli:[1,2,3,5,6,7,10,13,14,15,19,22,23,24],simpl:[0,4,9,11,13,14,15,18,20,22,23,24],simpler:[0,2,5,9,11,12,18,20,21],simplernn:8,simplest:[0,2,3,5,8,10,12,13,15,19,21,22,24,32,34],simpli:[1,2,3,5,6,7,9,10,11,12,13,14,15,18,19,20,21,22,23,24],simplic:[3,4,5,6,7,10,11,14,19,20,23],simplicti:[13,22],simplifi:[0,2,3,10,14,19,23],simplist:1,simpson:26,simul:[0,1,3,11,20],simultan:3,sin:[2,3,5,8,10,11,12,13,14,15,19,20,21,22,23,24],sinc:[0,1,2,3,4,5,6,7,9,10,11,12,15,18,19,20,21,24],sine:[5,13,22],singl:[1,2,4,5,6,8,10,12,13,14,15,19,21,22,23,24,26],singular:0,sit:[13,15,22,24],site:[0,2,3,6,7],situat:[0,2,3,4,9,13,14,15,18,22,23,24],six:1,size:[1,2,5,6,7,8,10,11,12,13,14,19,20,21,22,23,27],sketch:[11,14,20,23],skill:[0,2,9,18,26],skip:7,skl:[2,3,7],sklearn:[2,3,5,6,7,8],slack:6,slice:[2,10,19],slide:[1,3,8,26],slight:[3,32,34],slightli:[1,3,4,5,11,12,15,20,21,24],slope:[5,6,7,10,19],sloppi:[11,20],slow:[3,6,7,11,20],slower:[2,3,10,15,19,24],slowli:5,slp:5,small:[1,2,3,5,6,7,8,11,12,13,14,15,20,21,22,23,24,26,31,32,34],smaller:[1,2,3,5,6,7,12,14,21,23],smallest:[2,8,10,19],smart:3,smooth:3,sne:7,sneak:3,sns:[2,3,5,7,8],soar:3,social:[0,2,27],societ:0,societi:[0,9,18],sofist:0,soft:[4,5],soften:6,softmax:[4,8],softmax_cross_entropy_with_logits_v2:5,softmax_loss:5,softwar:[0,6,9,18,26],sol:6,sole:[0,2,3],solid:[0,2,4,13,14,22,23],solut:[1,3,5,6,7,9,11,12,13,15,18,20,21,22,24,26],solv:[0,2,3,5,7,8,9,12,14,15,18,21,23,24,26],solver:[6,7,11,20],some:[0,1,4,5,6,7,8,9,11,12,13,14,15,16,17,18,20,21,22,23,24,26,32,33,34],someth:[0,1,5,7,11,20],sometim:[3,5,7,14,23],somewher:[14,23],soon:[3,10,13,15,19,22,24],sophist:[0,2],sort:[1,3,7],sound:[0,9,13,18,22],sourc:[0,1,2,3,5,9,10,11,13,14,18,19,20,22,23,29],south:[15,24],southern:[15,24],sovl:[12,21],space:[1,3,5,7,8,11,13,15,20,22,24,26,27],spacetim:[15,24],span:[0,2,3,7,10,19,32,34],spare:5,spars:[2,3,8],sparse_mtx:2,sparsecategoricalcrossentropi:8,spartan:[14,23],spatial:[5,8,10,13,19,22],speak:1,speci:[14,23],special:[1,2,3,4,5,9,10,11,15,18,19,20,24],specif:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,18,19,20,21,22,23,24,27,32,34],specifi:[0,1,2,3,4,7,10,11,15,19,20,24],specifici:2,spectacular:8,speech:[0,2,5,8],speed:[5,11,12,20,21],spell:[12,21],spend:1,sphere:[3,7,14,23],spheric:[14,23],sphinx:[32,34],spin:[3,12,21],spite:[0,2],spline:6,split:[0,1,2,5,6,7,8,12,21],spontan:[1,12,21],sprai:[14,23],spread:[1,3,7,14,23],spring2020:26,spring:[11,13,14,20,22,23,34],springer:26,springforc:[13,22],spuriou:[15,24],sqrsignal:[13,22],sqrt:[1,3,6,7,10,11,12,13,14,15,19,20,21,22,23,24],squar:[0,1,2,4,5,6,7,8,10,11,13,14,19,20,22,23],squash:5,srand:1,stabil:[11,13,20,22],stabl:[2,3,7,8,12,14,21,23],stack:[3,8],stadium:[14,23],stage:[0,11,13,20,22],stai:[2,3,7,8,11,12,13,14,15,20,21,22,23,24],stand:[0,2,3,5,10,15,19,24,32,34],standar:7,standard:[0,2,3,4,5,6,8,10,11,14,15,19,20,23,24],standardscal:[3,6,7],standpoint:[9,18],stanford:8,start:[0,1,3,5,6,7,8,9,10,11,12,13,14,15,18,19,20,21,22,23,24,26,31],startpoint:1,stat:3,state:[0,1,2,4,5,6,7,8,10,11,12,13,14,15,16,17,19,20,21,22,23,24,33,34],statement:[0,2,7,10,19],station:[13,22],statis:3,statist:[0,2,4,5,7,10,11,19,20],statu:[3,4,7],std:[1,2,3,10,19],stddev:5,stdev:1,steadi:[13,22],step:[1,2,4,5,8,10,13,14,15,19,22,23,24,28],step_fn:[4,5],stepsiz:[9,18],stereo:8,sthetover2:[14,23],still:[1,3,7,8,11,14,20,23],stimuli:5,stochast:[0,3,5,6,7],stock:[0,8],stoke:5,stone:[2,4],stop:[1,5,7,8,10,11,19,20],storag:3,store:[1,2,5,7,8,10,19,32,34],str:[5,8],straight:[0,2,3,6,11,15,20,24],straightforward:[3,6],strategi:[0,5,11,12,14,20,21,23],stratifi:3,stream:5,strength:[2,3,13,22],stretch:[7,11,14,20,23],strict:6,strictli:[6,11,20],strike:[3,14,23],string:[1,5],stroke:4,strong:[1,5,13,14,22,23],strongli:[0,2,3,6,8,10,14,19,23],stronli:3,structur:[0,2,3,5,8,9,10,18,19,26,31,32,34],stuck:5,student:[0,2,9,18],studi:[0,2,3,5,6,7,9,10,11,14,15,18,19,20,23,24,26],style:[2,10,13,19,22],sub:5,subdivid:[0,2,10,11,12,19,20,21],subfield:[0,2],subinterv:[12,21],subject:[1,3,6],subplot:[2,3,5,6,7,8,10,12,13,14,16,17,19,21,22,23,33],subplots_adjust:[1,6],subprogram:[2,10,19],subroutin:2,subscript:[5,10,12,15,19,21,24],subsect:[13,22],subsequ:[1,3,5],subset:[2,3,5,10,19],subspac:[3,6,7],substep:7,substitut:[3,5,11,12,13,14,15,20,21,22,23,24],subtend:[14,23],subtl:5,subtract:[1,3,7,8],succeed:2,success:[1,4,8],sucess:1,sudden:[13,22],suddenli:[15,24],sudo:[2,10,19],suffer:[3,5],suffic:[10,19],suffici:[3,5,6,7,13,14,15,22,23,24],suggest:5,suit:[5,6,13,22],suitabl:[1,3,7],sum:[1,2,4,5,6,7,11,12,13,14,20,21,22,23],sum_:[1,2,3,4,5,6,7,11,12,13,14,20,21,22,23],sum_i:[3,6,7,12,13,15,21,22,24],sum_ia_i:[15,24],sum_im_i:[12,21],sum_im_ix_i:[12,21],sum_j:3,sum_k:[2,3,5,6,10,19],sum_n:[13,22],sum_nx_:[13,22],summar:3,summari:[5,8,26],summat:3,sun:[11,14,20,23,26],superfici:8,superscript:5,supervis:[0,2,3,4],supplement:[4,15,24],support:[0,3,5],suppos:[0,2,3,4,5,6,7,10,11,15,19,20,24],supposedli:0,sure:[3,5,8,16,17,33],surfac:[2,11,12,14,15,20,21,23,24],surpass:3,surpris:2,surprisingli:[14,23],surround:[8,14,23],surv:0,survei:[0,2],svc:[6,7],svd:7,svdinv:3,svm:[6,7],svm_clf:6,svr:7,sweep:[11,20],swift:0,swing:[13,22],symbol:[1,2,5,7,9,10,11,12,18,19,20,21,26],symmetr:[0,2,3,5,6,7,10,14,15,19,23,24],symmetri:[3,13,22],sympi:[2,9,10,12,18,19,21],symplect:[12,21],synonim:1,syntax:[2,7,10,19,32,34],syntaxerror:[2,6,7,10,19],system:[0,2,3,4,5,8,9,10,13,14,18,19,22,23,26,28],systemat:[1,3],t_0:[10,19],t_f:[10,11,19,20],t_h:[12,21],t_i:[5,7,10,11,12,13,19,20,21,22],t_j:5,t_n:[12,21],t_x:[15,24],t_y:[15,24],tab:[13,22,29],taberror:[13,22],tabl:[1,2,10,19],tabul:2,tabular:[2,10,19],tag:[1,3,4,5,11,12,13,14,15,20,21,22,23,24,28],taht:2,tail:1,tailor:[2,6,7],taiwan:3,take:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,18,19,20,21,22,23,24,27],taken:[2,3,5,8,10,19],takeoff:[15,24],tan:[13,22],tangent:[5,8,15,24],tanh:[4,5,6,8,10,19],target:[2,3,5,6,7,8,14,23],task:[0,2,3,5,7,8],tau:[1,12,13,21,22],tau_z:[12,21],taught:[0,8,9,18],tax:3,taxpay:0,taylor2:26,taylor:[10,11,12,13,14,19,20,21,22,23,26],tba:26,tdt:[10,19],teach:[9,18],team:[5,27],teaser:2,techniqu:[0,1,2,3,5,6,9,10,11,12,18,19,20,21],technolog:[0,2,5],tediou:[9,14,18,23],tell:[1,3,7,8,11,12,13,20,21,22],temperatur:3,templat:29,tempor:[13,22],temporari:27,temporarili:27,ten:[2,10,19],tend:[3,5,6,8,11,14,15,20,23,24],tendenc:2,tension:[3,15,24],tensor:8,tensorboard:5,tensorflow:[0,2,6,10,19],term1:[3,7],term2:[3,7],term3:[3,7],term4:[3,7],term:[1,2,3,4,6,7,8,9,10,11,12,13,14,15,18,19,20,21,22,23,24,28],termin:[3,11,20],test:[0,1,2,6,7,8,9,11,14,18,20,23,29],test_acc:8,test_accuraci:[5,8],test_error:3,test_imag:8,test_ind:3,test_label:8,test_loss:[5,8],test_pr:5,test_predict:5,test_rnn:8,test_siz:[3,5,7,8],testerror:3,testi:8,testpredict:8,testx:8,tex:[16,17,33],text:[1,2,3,5,6,7,8,10,13,19,22,26,32,34],textbook:[11,20],tfinal:[10,12,13,14,19,21,22,23],than:[1,2,3,4,5,7,8,10,11,13,14,15,19,20,22,23,24],theano:[2,5,10,19],thei:[0,1,2,3,4,5,6,7,8,9,10,11,13,14,15,18,19,20,22,23,24,26,27,28,32,34],them:[0,2,5,6,7,8,9,10,13,15,18,19,22,24],theme:2,themselv:[1,2],thenc:3,theorem:[3,12,21,26],theoret:[3,11,20],theori:[0,2,5,6,11,13,20,22],thereaft:[2,3,5,7,10,13,15,19,22,24],therebi:[0,1,2,4,7,10,13,15,19,22,24],therefor:[1,2,3,4,5,6,7,8,13,14,22,23],therein:7,thereof:[0,2,3],theta:[1,3,5,11,12,13,14,15,20,21,22,23,24],theta_0:[11,14,20,23],theta_:[5,14,23],theta_i:5,theta_k:1,thi:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,27,28,29,30,31,32,33,34],thick:[14,23],thin:[14,23],thing:[1,3,5,7],think:[0,1,3,5,8,9,11,13,14,15,18,20,22,23,24,26],third:[0,2,3,8,10,12,13,14,15,19,21,22,23,24],thirti:7,those:[2,3,6,7,10,11,12,13,14,15,19,20,21,22,23,24,32,34],though:[0,1,5,11,13,14,20,22,23],thought:[1,3,14,15,23,24],thousand:[0,2,3,5,7],threaten:27,three:[0,2,3,5,6,8,10,12,13,15,19,21,22,24],threshold:[5,7,8],through:[0,1,2,3,5,6,7,8,9,10,11,12,14,15,18,19,20,21,23,24,26,28],throughout:[0,1,2,3,10,19],thrown:[11,20],thu:[0,1,2,3,4,5,6,7,10,11,12,13,14,15,19,20,21,22,23,24,26],thumb:3,thursdai:26,tick_param:3,ticker:1,tidal:[15,24],tide:[14,23],tight_layout:[7,10,12,13,14,19,21,22,23],tightli:7,tild:[1,2,3,7,13,22],till:[3,4,5,6,7,8,10,11,15,19,20,24],time:[1,2,3,4,5,6,7,8,9,11,12,13,14,15,18,20,21,22,23,24,26,32,34],timefunct:1,timeit:8,timer:8,tiniti:[10,19],tip:[8,25,30],tire:[11,20],titl:[1,2,3,5,6,8],to_categor:[5,8],to_categorical_numpi:5,to_numer:[2,3],to_str:1,todai:0,togeth:[2,3,6,7,14,15,23,24],toler:[11,20],tolist:8,tomographi:5,too:[1,3,7,11,20],took:[2,6,10,19],tool:[0,1,2,3,5,7,8,9,10,18,19,32,34],toolbox:6,top:[2,3,9,10,15,18,19,24],topic:[0,2,3,4,6,9,18,26],topolog:[5,8],torqu:[11,12,13,20,21,22],tot:[14,23],total:[1,3,4,5,6,7,8,11,12,13,14,20,21,22,23,26],totaldimens:7,totalvari:1,tout:0,toward:[4,5,13,14,15,22,23,24,27],town:3,tqdm:3,traceback:3,track:[5,8,10,14,19,23],tract:3,tractabl:[0,3,7],traction:0,trade:3,tradeoff:0,tradit:[0,2,3,5,8],tradition:[9,18],train:[0,2,6,7,9,11,15,18,20,24],train_accuraci:[2,5,8],train_end:5,train_error:3,train_imag:8,train_ind:3,train_label:8,train_loss:5,train_pr:5,train_siz:[5,8],train_test_split:[2,3,5,7,8],train_test_split_numpi:5,trainabl:5,traini:8,trainingerror:3,trainpredict:8,trainscor:8,trainx:8,trait:[0,2],trajectori:[8,11,12,13,14,20,21,22,23],transcendent:[11,20],transform:[3,4,5,6,7,14,23,26],transit:[3,5],translat:[5,8,12,13,21,22],translation:[12,21],transpos:[3,5,7],transvers:[14,23],trapez1:[12,21],trapez:[9,12,18,21],trapezoid:[9,18,26],trapoezoid:[12,21],travel:[11,14,20,23],treat:[1,3,5,8,13,14,15,22,23,24],tree:[0,5,26],trend:1,tri:[11,20],trial:[0,1,2,3],triangular:[2,10,19],trick:[1,6,7,8,14,15,23,24],trickier:[1,10,19],tridiagon:[2,10,19],trivial:[1,3,5,7,14,23],troll:27,troubl:[3,5,6,7],troubleshoot:28,truck:8,true_divid:5,true_fun:3,truncat:[10,13,19,22],truncated_norm:5,trust:[11,12,20,21],truth:0,tucker:6,tumor:[4,7],tumour:7,tune:[2,8,10,13,19,22],turn:[0,1,2,3,5,6,7,9,11,12,13,15,18,20,21,22,24],tutori:5,tweak:[1,5],twice:[10,14,19,23],twist:7,twister:1,two:[0,1,2,3,5,7,8,10,12,13,15,19,21,22,24,26,32,34],twoard:[15,24],txt:29,type:[0,1,3,6,10,11,12,13,19,20,21,22,26],typeset:[9,18],typic:[0,1,2,3,4,5,8,11,12,13,14,20,21,22,23],u_i:5,ubuntu:[2,10,19],uci:3,ugli:[13,22],unaccept:27,unari:[2,10,19],unbalanc:3,unbias:[1,2,3],unbound:[14,23],uncertainti:[3,11,20],uncertitud:1,unchang:[5,8,12,21],uncov:5,undamp:[13,22],undefin:3,under:[2,3,5,10,11,12,14,19,20,21,23],underdetermin:3,underfit:[3,5],undergo:[9,11,14,18,20,23],undergradu:0,underli:[0,1,2,5,9,18],understand:[0,2,5,8,9,11,18,20,26],understood:[6,11,14,20,23],undesir:6,undetermin:6,undriven:[11,13,20,22],unexpect:[1,3],unexpected:1,unfortun:[5,6,11,20],unicode_liter:6,uniform:[0,2,3,5,7,11,20],uniform_real_distribut:1,uniformli:[1,14,15,23,24],unifrompdf:1,union:3,uniqu:[0,2,3,9,18],unit:[1,2,3,5,8,9,11,12,13,15,18,20,21,22,24],unitari:[2,3,10,19],unitarili:[2,10,19],uniti:[1,13,22],univ:26,univari:1,univers:[0,8,34],unix:5,unknow:3,unknown:[3,5,6,7,8,13,22],unknowwn:5,unlabel:5,unless:[2,3,7,11,12,13,15,20,21,22,24],unlik:[1,5,6,8],unlimit:[12,21],unord:8,unravel:5,unrol:[7,8],unscatt:[14,23],unseen:4,unstabl:[5,14,23],unsupervis:[0,2,5],unsymmetr:[2,10,19],until:[1,5,8,13,22],unusu:5,unwelcom:27,updat:[3,5,7,10,12,13,14,19,21,22,23],upgrad:[11,20],upon:[0,5,7,11,15,20,24],upper:[2,3,6,10,12,19,21],uppercas:[1,2,10,19],ups:2,upward:[11,15,20,24],usag:[0,1,2,3,6,10,19],usain:[11,20],uscibook:26,usd10000:3,usd:3,use:[0,2,3,4,6,7,8,9,10,11,12,13,14,15,18,19,20,21,22,23,24,26,27,28,32,34],use_bia:8,usecol:[2,3],used:[0,1,2,3,4,5,6,7,9,10,11,12,13,14,15,18,19,20,21,22,23,24,26,32,34],useful:[0,1,2,4,5,7,8,10,11,12,13,14,15,19,20,21,22,23,24,26],useless:5,user:[2,4,5,6,7,8,9,10,11,18,19,20],uses:[1,2,3,5,7,8,10,12,13,14,19,21,22,23,32,34],usetex:1,using:[0,3,6,7,8,9,10,11,12,13,14,15,18,19,20,21,22,23,24,26,27,28,29],usr:1,usual:[2,4,5,8,11,14,15,20,23,24],util:[2,3,5,8],utliz:[12,21],utmost:0,v_0:[7,10,11,12,13,19,20,21,22],v_1:[12,21],v_e:[12,21],v_i:[10,11,12,19,20,21],v_jb_k:[11,20],v_n:[12,21],v_t:[10,11,19,20],v_x:[11,12,15,20,21,24],v_xf_x:[12,21],v_y:[11,12,20,21],v_yf_i:[12,21],v_z:[11,12,15,20,21,24],v_zf_z:[12,21],v_zt:[11,20],val_accuraci:8,val_loss:8,valid:[0,1,5,8,9,11,18,20,32,34],validation_data:8,validation_split:8,valu:[0,2,4,5,6,8,9,10,11,12,13,14,18,19,20,21,22,23],valy:8,van:3,vandenbergh:6,vandermond:3,vanilla:7,vanish:[1,5],var_x:1,varabl:6,varepsilon:3,varepsilon_:3,varepsilon_i:3,vari:[3,5,8,10,15,19,24],variabl:[2,3,4,5,6,7,10,12,14,15,19,21,23,24,26],varianc:[0,2,4,5,7],variance_i:[3,7],variance_x:[3,7],variant:[2,3,5,6,7,9,10,18,19],variat:[7,12,21,26,32,34],varieti:[0,2,5,10,19],variou:[0,1,2,4,5,6,7,9,10,12,13,15,18,19,21,22,24,26],vartempvec:1,varvec:1,varydimens:8,vastli:8,vaverag:[10,19],vdt:[11,20],vec:[1,3],vecto:[12,21],vector:[4,8,11,12,14,15,20,21,23,24,26],vehicl:[15,24],veloc:[11,13,14,15,20,22,23,24,26],velocityverlet:[13,22],ventur:[0,2,6,9,10,12,18,19,21],verbos:[5,8],veri:[0,1,2,3,5,6,7,8,10,11,12,13,15,19,20,21,22,24,26,32,34],verif:[9,18,26],verifi:[2,7,9,10,11,13,18,19,20,22],verlet:[13,14,22,23,26],versa:[12,21],versatil:[2,6,9,10,18,19],versicolor:6,version:[0,1,2,10,13,14,19,22,23,27,28,29],versu:[5,14,23],vert:[2,3,4,5,6,7,10,19],vert_1:3,vert_2:[3,7],vert_:[12,21],vertic:3,via:[0,1,2,3,5,6,7,9,10,11,12,13,18,19,20,21,22,27],vibrat:[14,23],vice:[12,21],video:[5,26],view:[0,1,2,5,9,10,12,15,18,19,21,24,26],viewpoint:27,violat:6,viridi:[2,5,8],virtual:[5,29],virtualenv:28,viru:26,viscos:[11,13,20,22],viscou:[11,13,20,22],vision:[0,2,8],visual:[0,2,3,7,13,22],viz:[1,3,6],vmax:3,vmc:1,vmin:3,volcano:[15,24],voltag:[14,23],volum:[2,11,14,20,23],volunt:28,vstack:[1,2,3,7],w_1:6,w_1x_1:6,w_1x_:6,w_2:6,w_2x_2:6,w_2x_:6,w_fc1:5,w_fc2:5,w_i:5,w_ix_i:5,w_out:5,w_px_:6,w_px_p:6,wai:[0,1,2,3,4,5,6,7,8,9,10,12,18,19,21,25,27,28,30],walker:1,wang:2,want:[0,1,2,3,5,6,7,8,10,11,13,15,16,17,19,20,22,24,28,33],warn:6,warrant:3,wast:8,water:[14,15,23,24],wave:[13,22],wavelet:6,wealth:[0,9,18],weather:[5,15,24],web:[2,9,18,26,28],websit:[2,10,19,28],wedg:[1,6],wednesdai:26,weekli:26,weight:[0,1,3,4,8,12,21],weight_vari:5,welcom:[6,27,28,29,32],well:[0,1,2,3,4,5,6,10,11,12,13,14,15,16,17,19,20,21,22,23,24,26,33],wendi:[32,34],went:6,were:[0,1,2,3,4,5,6,7,8,11,12,13,14,15,20,21,22,23,24],wessel:3,what:[4,5,7,8,9,10,11,12,13,14,15,18,19,20,21,22,23,24,26,27],whatev:[0,8],when:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,18,19,20,21,22,23,24,26,27,32,34],whenev:1,where:[0,1,2,4,5,6,7,8,9,10,11,12,13,14,15,18,19,20,21,22,23,24],wherea:[1,3,11,12,14,20,21,23,32,34],wherein:5,whether:[0,1,2,4,11,12,13,14,20,21,22,23,28,32,34],which:[0,2,3,4,6,7,8,9,10,11,12,13,14,15,18,19,20,21,22,23,24,27,32,34],whichev:8,who:[0,2,27],whoever:28,whole:[5,7,8],whose:[1,3,12,13,21,22],whow:[3,7],why:[0,2,11,15,20,24],wide:[2,3,5,8,10,11,13,19,20,22],widehat:3,wider:[0,15,24],width:[2,3,6,8],wieringen:3,wiki:27,willing:0,win:[14,23],wind:[11,13,20,22],wiscons:7,wise:[3,5],wish:[0,2,4,6,7,10,11,13,14,15,19,20,22,23,24,29],wither:3,within:[0,1,2,4,5,8,11,20,27,29],without:[0,2,3,5,6,7,10,11,13,19,20,22,26,27],witth:[13,22],won:[2,3],wonder:6,word:[1,2,3,7,8,15,24],work:[0,1,2,3,4,5,6,7,8,9,10,11,13,14,16,17,18,19,20,22,23,26,28,32,33,34],workflow:29,world:[3,6,7,8,11,20],worldwid:[2,15,24],wors:[2,3,5,8],would:[0,1,2,3,4,5,6,7,8,10,11,12,13,14,15,19,20,21,22,23,24,26,28],wrap:[2,3,10,19,26],write:[0,1,2,3,5,6,9,10,12,13,14,15,18,19,21,22,23,24,25,30,32,34],writer:5,written:[0,1,2,3,5,7,9,10,11,12,13,14,18,19,20,21,22,23,32,34],wrong:[5,6,9,18,26],wrongli:1,wrote:[0,3,7],www:[2,5,10,19,26],wx_1:6,x0s:6,x1_exampl:6,x1d:6,x1s:6,x2d:[6,7],x2d_train:7,x2dsl:7,x2s:6,x3s:6,x_0:[2,3,7,10,11,12,13,19,20,21,22],x_1:[0,1,2,3,4,6,7,10,12,19,21],x_2:[0,1,2,3,4,6,7,10,12,19,21],x_3:[1,6],x_center:7,x_data:5,x_data_ful:5,x_f:[12,21],x_i:[1,2,3,4,5,6,7,11,12,20,21],x_ix_:3,x_iy_i:6,x_j:[1,5,6],x_jy_j:6,x_k:[1,5],x_l:1,x_m:[1,5],x_n:[0,1,3,5,6,7,13,22],x_p:[4,13,22],x_pred:8,x_reduc:7,x_scale:6,x_test:[3,5,7,8],x_test_own:3,x_test_scal:[3,7],x_tot:8,x_train:[2,3,5,7,8],x_train_own:3,x_train_scal:[3,7],xarrai:[2,10,19],xavier:5,xcode:[2,10,19],xf_y:[12,21],xfinal:[14,23],xi_1:6,xi_:6,xi_i:6,xiniti:[14,23],xlabel:[1,2,3,6,8,9,10,18,19],xlim:3,xnew:2,xpd:[3,7],xplot:2,xtest:3,xtick:[3,6,8],xtrain:3,xytext:6,y_0:[2,3,7,10,11,12,19,20,21],y_1:[2,3,6,7,12,13,21,22],y_1y_1:6,y_1y_1k:6,y_1y_2:6,y_1y_2k:6,y_1y_n:6,y_1y_nk:6,y_2:[2,3,6,7,12,21],y_2y_1:6,y_2y_1k:6,y_2y_2:6,y_2y_2k:6,y_3:2,y_data:[3,5],y_data_ful:5,y_decis:6,y_i:[2,3,4,5,6,7,10,12,13,19,21,22],y_ix_:3,y_ix_i:[4,6],y_iy_jk:6,y_j:[3,5,6],y_k:5,y_model:[3,8],y_n:[6,12,21],y_ny_1:6,y_ny_1k:6,y_ny_2:6,y_ny_2k:6,y_ny_n:6,y_ny_nk:6,y_p:[11,20],y_pred:[3,5,6,8],y_test:[3,5,7,8],y_test_onehot:5,y_test_predict:3,y_tot:8,y_train:[2,3,5,7,8],y_train_onehot:5,y_train_predict:3,yanalyt:[10,19],year:[0,2,12,13,21,22],yes:[0,3,4],yet:[0,3,5,6,7,10,11,14,19,20,23],yf_x:[12,21],yield:[1,2,3,5,6,10,12,13,14,19,21,22,23],ylabel:[1,2,3,6,8,9,10,18,19],ylim:[3,8,13,22],yoshua:5,you:[0,1,2,3,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,26,28,29,31,32,33,34],young:[0,2],your:[0,1,2,3,5,6,7,9,10,11,12,13,15,16,17,18,19,20,21,22,24,26,28,29,33],yourself:[2,7,14,15,23,24],ypred:3,ypredict:[2,3],yridg:3,ytest:3,ytick:[3,6,8],ytild:3,ytildenp:3,ytrain:3,z_0:[2,11,20],z_1:2,z_2:2,z_c:5,z_h:5,z_i:5,z_k:5,z_m:5,z_o:5,z_out:5,zaman:1,zero:[1,2,3,5,6,7,8,9,10,11,12,13,14,15,18,19,20,21,22,23,24],zillion:0,zip:3,zm_h:2,zone:3,zze:[14,23]},titles:["Introduction to Applied Data Analysis and Machine Learning","1. Elements of Probability Theory and Statistical Data Analysis","2. Getting started, our first data and Machine Learning encounters","3. Linear Regression and more Advanced Regression Analysis","4. Logistic Regression","5. Neural networks, from the simple perceptron to deep learning","6. Support Vector Machines, overarching aims","7. Dimensionality Reduction","8. Convolutional Neural Networks","Introduction","Space, Time, Motion, Reference Frames  and Reminder on vectors and other mathematical quantities","Basic Steps of Scientific Investigations","Work, Energy, Momentum and Conservation laws","Harmonic Oscillator","Two-body Problems","Non-inertial Frames, Translation and Rotating Coordinate Systems","Content with notebooks","Content with notebooks","Introduction","Space, Time, Motion, Reference Frames  and Reminder on vectors and other mathematical quantities","Basic Steps of Scientific Investigations","Work, Energy, Momentum and Conservation laws","Harmonic Oscillator","Two-body Problems","Non-inertial Frames, Translation and Rotating Coordinate Systems","Content in Jupyter Book","PHY321, Classical Mechanics I, Michigan State University, Spring 2021","Code of Conduct","Contributing","LectureNotes","Content in Jupyter Book","Welcome to your Jupyter Book","Markdown Files","Content with notebooks","Markdown Files"],titleterms:{"break":26,"case":[1,6,10,19],"class":4,"final":[5,7,8,10,11,12,19,20,21,26],"function":[1,3,4,5,6,7],"import":[1,2,8,10,19],"new":8,"transient":[13,22],"try":8,Adding:[3,6,12,21,32,34],And:[2,11,20],Going:[11,20],Ising:3,NNs:8,OLS:3,One:[11,20],The:[1,3,4,5,6,7,8,11,12,13,15,20,21,22,24],Then:[11,20],Useful:[2,10,19],Using:[5,32,34],With:8,about:2,abov:[9,18],acceler:[10,11,15,19,20,24],act:[11,20],activ:5,actual:1,add:8,adjust:5,advanc:3,again:8,aim:[3,6],air:[10,11,19,20],aka:3,algebra:[3,10,15,19,24],algorithm:[3,5,7,12,21],all:6,altern:[13,22],analysi:[0,1,3,7],analyt:[10,19],angular:[12,21],anoth:3,appli:0,approach:[0,3,6,11,20],approxim:5,april:26,architectur:5,arrai:[2,10,19],artifici:5,asap:26,aspect:0,assumpt:3,attempt:[6,10,19],attribut:27,autocorrel:1,averag:[10,19],back:[5,6,7],backend:5,background:3,ballist:[11,20],basebal:[10,19],basic:[2,3,4,7,10,11,12,19,20,21],befor:7,beta:[3,13,22],better:[1,3,6],bia:3,bias:[0,5],binari:5,bind:[2,3],binomi:1,block:[1,11,13,16,17,20,22,33],bodi:[14,23],boldsymbol:3,book:[25,29,30,31],bootstrap:[1,3],boston:3,breast:7,brief:[2,3,8],bring:5,bug:28,build:[5,8,12,21,29],buld:[12,21],calcul:1,can:3,cancer:7,center:[7,14,23],central:1,centrifug:[14,15,23,24],chain:5,chi:3,choic:[0,5],choos:5,cifar01:8,citat:[32,34],classic:[7,11,13,20,22,26],classif:[5,7],classifi:6,cnn:8,code:[1,3,5,6,7,10,12,13,14,16,17,19,21,22,23,27,28,32,33,34],collect:[5,8],compact:[10,19],compil:8,compon:7,comput:[1,7,10,19],computation:3,concept:[1,9,18],conduct:[27,28],conserv:[12,21],constant:[10,19],content:[16,17,25,30,33],continu:[1,7],contn:[11,20],contribut:28,contributor:29,convex:6,convolut:[5,8],coordin:[15,24],corioli:[15,24],correl:[1,3,7,8],cost:5,cote:[12,21],coupl:[12,21],covari:[1,3,7],credit:29,critic:[13,22],cromer:[10,12,19,21],cross:[3,14,23],cumul:1,cython:2,damp:[13,22],dampl:[13,22],data:[0,1,2,3,5,7,8],dataset:[5,8],date:26,decompos:3,decomposit:[3,7],deep:5,defin:[5,11,20],definit:[1,5],degre:[3,7],demonstr:1,dens:[3,8],deriv:[5,12,13,14,21,22,23],descent:5,design:[3,7],detail:[3,6,8,11,20],develop:5,deviat:1,diagon:7,dice:1,differ:6,differenti:[10,11,13,19,20,22],dimens:[10,11,14,19,20,23],dimension:[3,6,7],direct:[32,34],discret:[1,10,12,19,21],disguis:1,distribut:1,document:28,doing:5,domain:1,don:8,down:5,driven:[13,22],earth:[12,21],economi:3,effect:[14,15,23,24],element:[1,10,19],ellipt:[14,23],encount:[2,10,19],energi:[2,3,12,21],enforc:27,equal:[12,21],equat:[3,5,6,10,11,12,13,19,20,21,22],error:[3,12,21],essenti:[],etc:[2,8],ethic:0,euler:[10,12,15,19,21,24],evalu:[5,8],event:1,exampl:[1,2,3,5,6,7,8,9,10,12,14,18,19,21,23],execut:[32,34],expect:[1,3],expens:3,experi:1,exponenti:1,express:[3,12,21],extens:[10,19],extrapol:8,extrem:[],factor:[13,22],fall:[10,11,15,19,20,24],famili:[3,5],famou:[1,2,10,19],fantast:3,featur:[2,3,6,7,10,19,28],februari:26,feed:5,feedback:28,field:[11,20],file:[8,32,34],find:[3,10,19],first:[2,5,6,10,11,19,20],fit:[2,3],fix:[3,28],fold:3,forc:[11,13,14,15,20,22,23,24],format:8,forward:5,foucault:[15,24],fouri:[13,22],fourier:[13,22],frame:[10,15,19,24],frank:[3,7],freedom:[3,7],frequentist:0,from:[3,5,8,15,24],full:[5,8],funtion:5,galilean:[15,24],gaussian:1,gener:[1,3],geometr:7,get:[2,6,28],going:3,good:0,grade:26,gradient:5,gravit:[14,23],handl:[0,2,10,19],happen:3,harmon:[13,14,22,23],height:[10,19],host:29,hous:3,how:[1,3,6],hyperbol:5,hyperparamet:5,hyperplan:6,idea:7,identifi:[11,20],illustr:[9,18],imag:8,implement:[1,5,10,19,28],improv:5,includ:[4,10,19],increment:7,inerti:[15,24],instal:[2,10,19],integr:[10,12,19,21],interpret:[3,7],introduc:[3,7],introduct:[0,2,4,9,18],investig:[11,20],its:1,jackknif:[1,3],januari:26,julia:2,jupyt:[25,30,31],kera:[5,8],kernel:[6,7],known:[9,18],kutta:[13,22],label:[10,19],lagrangian:6,lambda:3,languag:0,lasso:3,last:6,law:[11,12,20,21],layer:[5,8],learn:[0,2,3,5,7,26],least:3,lectur:26,lecturenot:29,librari:[2,10,19],likelihood:4,limit:[1,5],linear:[2,3,6,15,24],link:[3,7,26],list:5,lle:7,logist:[4,5,7],machin:[0,2,6,7],made:3,magnet:[11,20],mai:26,main:1,mani:[5,9,18],march:26,markdown:[16,17,32,33,34],mass:[2,14,23],materi:26,math:3,mathemat:[3,5,6,10,12,19,21],matric:[2,10,19],matrix:[1,2,3,5,7,10,19],matter:3,maximum:4,mean:[1,7],mechan:26,medic:8,meet:[1,2,10,19],mercer:6,mersenn:1,method:[1,3,10,12,13,19,21,22],michigan:26,minim:6,mlp:5,mnist:8,model:[0,2,3,5,8,10,11,19,20],moment:1,momentum:[12,21],moon:6,more:[2,3,4,6,7,8,10,11,13,19,20,22],motion:[10,11,14,19,20,23],multi:5,multidisciplinari:[],multilay:5,multipl:5,multipli:6,myst:[16,17,32,33,34],name:1,need:[2,10,19],network:[2,5,8],neural:[2,5,8],neuron:[5,8],newton:[11,12,20,21],non:[6,15,24],normal:1,notat:5,note:26,notebook:[16,17,33],now:[3,7],nuclear:[2,3],numba:2,number:[1,3,7],numer:[1,12,13,21,22],numpi:[2,10,19],object:[5,11,15,20,24],observ:1,obtain:7,omega_0:[13,22],one:[3,5,10,11,19,20],ones:5,optim:[3,5,6],optmiz:6,orbit:[14,23],ordinari:3,organ:2,orient:5,oscil:[13,14,22,23],other:[1,5,7,8,10,19,26],our:[1,2,3,5,7,10,11,13,19,20,22,27],outcom:[0,26],output:[1,16,17,33],overarch:[3,6,7,8],overdamp:[13,22],own:[3,7],packag:[2,10,19],panda:[2,3,7,10,19],paramet:[3,4],part:[8,11,20],pass:5,pca:7,pdf:1,pendulum:[11,13,15,20,22,24],perceptron:5,perform:[3,5],period:[1,13,22],perspect:[5,15,24],philosophi:[12,21],phy321:26,pledg:27,point:8,poisson:1,possibl:26,potenti:[14,15,23,24],practic:[10,19,26],pre:[5,8],predict:8,preprocess:[3,7],prerequisit:8,princip:7,principl:[13,22],probabl:[0,1],problem:[3,5,6,12,14,21,23],procedur:3,process:[5,8],program:0,projectil:[11,20],proof:7,propag:5,properti:[1,3],pseudo:1,python:[1,2,10,19],quadratur:[12,21],quantiti:[10,19],quick:6,raindrop:[11,20],ran0:1,random:[1,7],read:2,real:2,realist:6,recip:1,rectangl:[12,21],rectangular:[12,21],recurr:[5,8],reduc:[3,7],reduct:[7,8],refer:[10,19],reformul:3,regress:[2,3,4,7],regular:[3,5,8],rel:[14,23],relev:5,relu:5,remind:[2,6,10,19],report:28,repositori:3,resampl:3,resist:[10,11,19,20],reson:[13,22],respons:[13,22,27],result:[3,11,20],rewrit:[3,7],rich:[],ridg:3,rng:1,rnn:8,role:[32,34],rotat:[15,24],rule:[5,12,21],run:8,rung:[13,22],rutherford:[14,23],same:3,sampl:[1,7],scale:[8,11,13,20,22],scan:8,scatter:[14,23],schedul:26,scientif:[11,20],scikit:[2,3,5,7],scope:27,second:[11,20],section:[14,23],see:2,select:1,seri:[13,22],set:[3,5,6,8,11,20],should:[1,5,7],sigmoid:5,simpl:[1,2,3,5,6,7,8,10,12,19,21],simpler:6,simpson:[12,21],singl:[11,20],singular:[3,7],sinusoid:[13,22],situat:1,size:3,slide:[11,13,20,22],small:0,soft:6,softer:[11,20],softmax:5,softwar:[2,10,19],solar:[12,21],solut:[10,14,19,23],solv:[6,10,11,13,19,20,22],some:[2,3,10,19],sound:8,space:[6,10,19],split:[3,11,20],spring:26,squar:3,stabil:[14,23],standard:[1,27],start:[2,28],state:[3,26],statist:[1,3],step:[3,6,7,11,12,20,21],stochast:1,strong:8,studi:[12,13,21,22],submit:28,suit:3,sum:3,summari:2,sun:[12,21],superposit:[13,22],supervis:5,support:[6,7],svd:3,symmetri:[12,21],system:[11,12,15,20,21,24],systemat:8,teach:26,team:26,techniqu:7,tensorflow:[5,8],term:5,test:[3,5],textbook:26,theorem:[1,5,6,7,15,24],theori:1,thi:26,thing:8,think:7,third:[11,20],three:1,tidal:[14,23],tied:[11,20],time:[10,19],togeth:5,top:[5,8],toss:1,toward:7,tradeoff:3,train:[3,5,8],transform:[8,13,15,22,24],translat:[15,24],trapezoid:[12,21],tree:2,truncat:[12,21],two:[4,6,11,14,20,23],type:[2,5,8],uncorrel:1,underdamp:[13,22],understand:3,uniform:1,unit:[10,19],univers:[5,26],updat:26,usag:29,use:[1,5],used:8,useful:3,using:[1,2,5],valid:3,valu:[1,3,7],variabl:1,varianc:[1,3],variou:3,vector:[2,3,5,6,7,10,19],veloc:[10,12,19,21],verifi:8,verlet:[12,21],view:[3,7,8],visual:[5,8],volum:8,wai:[11,20],wall:[11,20],week:26,weight:5,welcom:31,well:[8,9,18],what:[0,1,2,3,6,32,34],where:3,which:[1,5],why:[1,3,5,7,8],width:[13,22],wood:2,work:[12,21],write:[7,28],your:[31,32,34],z_j:5}})
\ No newline at end of file
+Search.setIndex({docnames:["chapter1","chapter10","chapter11","chapter2","chapter3","chapter4","chapter5","chapter6","chapter7","chapter8","chapter9","testbook/_build/jupyter_execute/chapter1","testbook/_build/jupyter_execute/chapter2","testbook/_build/jupyter_execute/chapter3","testbook/_build/jupyter_execute/chapter4","testbook/_build/jupyter_execute/chapter5","testbook/_build/jupyter_execute/chapter6","testbook/_build/jupyter_execute/chapter7","testbook/_build/jupyter_execute/lecturenotes/lecturenotes/notebooks","testbook/_build/jupyter_execute/notebooks","testbook/chapter1","testbook/chapter2","testbook/chapter3","testbook/chapter4","testbook/chapter5","testbook/chapter6","testbook/chapter7","testbook/content","testbook/intro","testbook/lecturenotes/CONDUCT","testbook/lecturenotes/CONTRIBUTING","testbook/lecturenotes/README","testbook/lecturenotes/lecturenotes/content","testbook/lecturenotes/lecturenotes/intro","testbook/lecturenotes/lecturenotes/markdown","testbook/lecturenotes/lecturenotes/notebooks","testbook/markdown"],envversion:{"sphinx.domains.c":2,"sphinx.domains.changeset":1,"sphinx.domains.citation":1,"sphinx.domains.cpp":3,"sphinx.domains.index":1,"sphinx.domains.javascript":2,"sphinx.domains.math":2,"sphinx.domains.python":2,"sphinx.domains.rst":2,"sphinx.domains.std":1,"sphinx.ext.intersphinx":1,sphinx:56},filenames:["chapter1.ipynb","chapter10.ipynb","chapter11.ipynb","chapter2.ipynb","chapter3.ipynb","chapter4.ipynb","chapter5.ipynb","chapter6.ipynb","chapter7.ipynb","chapter8.ipynb","chapter9.ipynb","testbook/_build/jupyter_execute/chapter1.ipynb","testbook/_build/jupyter_execute/chapter2.ipynb","testbook/_build/jupyter_execute/chapter3.ipynb","testbook/_build/jupyter_execute/chapter4.ipynb","testbook/_build/jupyter_execute/chapter5.ipynb","testbook/_build/jupyter_execute/chapter6.ipynb","testbook/_build/jupyter_execute/chapter7.ipynb","testbook/_build/jupyter_execute/lecturenotes/lecturenotes/notebooks.ipynb","testbook/_build/jupyter_execute/notebooks.ipynb","testbook/chapter1.ipynb","testbook/chapter2.ipynb","testbook/chapter3.ipynb","testbook/chapter4.ipynb","testbook/chapter5.ipynb","testbook/chapter6.ipynb","testbook/chapter7.ipynb","testbook/content.md","testbook/intro.md","testbook/lecturenotes/CONDUCT.md","testbook/lecturenotes/CONTRIBUTING.md","testbook/lecturenotes/README.md","testbook/lecturenotes/lecturenotes/content.md","testbook/lecturenotes/lecturenotes/intro.md","testbook/lecturenotes/lecturenotes/markdown.md","testbook/lecturenotes/lecturenotes/notebooks.ipynb","testbook/markdown.md"],objects:{},objnames:{},objtypes:{},terms:{"00000000e":[5,9],"000e":9,"0039e":1,"00405815e":[],"00578340e":[],"0057e":1,"0059e":1,"00694659e":[],"00813277e":[],"00pm":28,"01118504e":[],"0112e":1,"0114e":1,"01176282e":9,"01193226e":[],"01222393e":[],"0129e":1,"012e":9,"0133e":1,"01347297e":[],"01405288e":[],"01443312e":[],"01600803e":5,"01617794e":[],"01633169e":[],"0185e":1,"01959093e":[],"01968016e":9,"02004166e":[],"02045611e":[],"0204e":1,"0207e":1,"0208e":1,"02176297e":[],"02186547e":[],"0232e":1,"02599545e":[],"0259e":1,"0307e":1,"03123374e":[],"03233215e":[],"03256632e":7,"03288157e":[],"03446279e":[],"0364e":1,"03705850e":[],"0393e":1,"04029857e":[],"040e":9,"0417e":1,"041e":9,"0422e":1,"04262460e":5,"0453e":1,"04719753e":[],"04722875e":[],"04904333e":[],"0502e":1,"05055916e":[],"0515e":1,"05246505e":[],"0525e":1,"0547e":1,"0558e":1,"0565e":1,"0577e":1,"0581e":1,"0588e":1,"058e":9,"05904756e":[],"0592e":1,"0595e":1,"060e":9,"06412459e":5,"0648e":1,"064e":9,"06648521e":[],"0678e":1,"0685e":1,"0688e":1,"07106781e":5,"07557895e":[],"075e":9,"0778e":1,"07968357e":[],"0803e":1,"0805e":1,"0808e":1,"0810e":1,"0814e":1,"08248290e":5,"0826e":1,"0829e":1,"08318298e":7,"0835e":1,"0866e":1,"08678173e":[],"0889e":1,"08915360e":[],"09013732e":[],"09284157e":[],"09358934e":[],"0939e":1,"09507879e":[],"0982e":1,"0986e":1,"0_tlsccwai":28,"0_utc9il9i":28,"0x7f9dd7c37eb0":[],"0x7fad47b08730":[],"0x7fb5be1db040":[],"0x7fddb4c436d0":[],"0x7fe6839db970":5,"0x7ff4f8410b20":[],"0x7ff6a7303b50":[],"1000s":5,"1007e":1,"100m":[13,22],"100th":2,"1017e":1,"1024e":1,"1028e":1,"10368860e":[],"10378326e":7,"1049e":1,"1059e":1,"10665316e":[],"10687329e":[],"10836834e":[],"1085e":1,"10898112e":5,"1091e":1,"1093e":1,"1095e":1,"1096e":1,"1098e":1,"10th":28,"10x":4,"1103e":1,"1104e":1,"1105e":1,"1106e":1,"1118e":1,"1119e":1,"1120e":1,"112453549290521e":[14,23],"1125e":1,"1126e":1,"11314068e":[],"1131e":1,"1139e":1,"1140e":1,"1146e":1,"11487443e":[],"1148e":1,"1149e":1,"114e":9,"11507992e":7,"1155e":1,"1157e":1,"1159e":1,"115e":9,"1164e":1,"1165e":1,"11698369e":5,"11705599e":[],"11765797e":[],"117e":9,"1183e":1,"1188e":1,"1196e":1,"1197e":1,"1198e":1,"1203e":1,"12219360e":[],"1231e":1,"1237e":1,"12397083e":9,"1239e":1,"12403403e":[],"12471292e":[],"1253e":1,"12541909e":[],"1259e":1,"1264e":1,"1265e":1,"12674763e":[],"12868265e":[],"12908684e":[],"1290e":1,"1299e":1,"12cm":[12,21],"1302e":1,"13037918e":9,"1307e":1,"1308e":1,"1312e":1,"1327e":1,"1329e":1,"13404436e":[],"1341e":1,"1357e":1,"1372e":1,"1385e":1,"1403e":1,"1432e":1,"1449e":1,"14549644e":9,"1467e":1,"146e":9,"1482e":1,"14g":5,"1506e":1,"1508e":1,"1510e":1,"154m":1,"1564e":1,"15657183e":[],"15751255e":[],"1579e":1,"1592e":1,"15g":5,"1612e":1,"1615e":1,"16214154e":[],"1629e":1,"1643e":1,"16496581e":5,"1651e":1,"168m":1,"16b8e3cda33a":7,"16m":1,"1704e":1,"17061481e":[],"17082679e":5,"17174962e":7,"17314f270d45":9,"1733e":1,"1740e":1,"1751e":1,"17683028e":9,"1770e":1,"1796e":1,"17m":1,"1812e":1,"1827e":1,"1830e":1,"1834e":1,"18442813e":[],"1862e":1,"18721828e":9,"1888e":1,"1890e":1,"1895e":1,"18m":1,"1902e":1,"1910e":1,"19218307e":[],"1938e":1,"1946e":1,"1954e":1,"1960e":1,"19614372e":9,"1970e":1,"1970s":[4,12,21],"19762810e":[],"1976e":1,"19872323e":9,"198e":9,"19m":1,"19th":2,"1_1":7,"1_2":7,"1_2v2se359":28,"1_2v439nza":28,"1_3":7,"1_azwo1s2r":28,"1_b81t0tta":28,"1_d70czgce":28,"1_e1cs5784":28,"1_ftgzt035":28,"1_hxxec3uc":28,"1_i9hczn21":28,"1_j5ugthfg":28,"1_rz8vr2ht":28,"1_sp7p28vk":28,"1_t1pocrww":28,"1_t2l86862":28,"1_utivxb87":28,"1_w9xc8az7":28,"1_wk9trq9k":28,"1cm":[3,4,8,13,22],"1e4":5,"1kg":[12,21],"1kgm":[12,21],"1ms":10,"2022e":1,"202e":9,"2031e":1,"2050e":1,"2059e":1,"20686874e":[],"20745612e":[],"2091e":1,"2095e":1,"20m":1,"2102e":1,"2108e":1,"2114e":1,"2129e":1,"2134e":1,"21351358e":[],"2143e":1,"2157e":1,"21635571e":[],"2165e":1,"2170e":1,"21814310e":[],"21869563e":[],"21918258e":9,"21m":1,"21st":28,"2211e":1,"22164060e":[],"2221e":1,"2223e":1,"22421033e":[],"224805627923717e":[14,23],"2259e":1,"2261e":1,"2266e":1,"22674158e":[],"226e":9,"227m":1,"228e":9,"2295e":1,"22m":1,"2320e":1,"2321e":1,"2324e":1,"2335e":1,"2387e":1,"23908217e":[],"23h":[17,26],"2401e":1,"2423e":1,"24243736e":9,"2427e":1,"24411055e":[],"24417907e":5,"2452e":1,"24557890e":5,"24623749e":[],"24629399e":[],"24790498e":[],"2484e":1,"2492e":1,"24h":[17,26],"24m":1,"2523e":1,"2525e":1,"252e":9,"2535e":1,"25429987e":5,"254e":9,"2572e3a4b38d":7,"2584e":1,"2591e":1,"25m":1,"2605e":1,"2624e":1,"26349118e":[],"263e":9,"26519220e":[],"26664263e":5,"26888965e":[],"2688e":1,"268e":9,"2699e":1,"2715e":1,"27564656e":[],"2756e":1,"2787e":1,"27881252e":[],"279e":9,"27n_":3,"2810e":1,"28119384e":[],"2841e":1,"28444646e":[],"28590170e":[],"2888e":1,"28kg":[17,26],"29001174e":[],"2907e":1,"2909e":1,"2911e":1,"29127820e":[],"2963e":1,"2971e":1,"2973e":1,"2981e":1,"2_1":7,"2_2":7,"2_3":7,"2_i":7,"2_m":3,"2_x":3,"2cm":[8,12,21],"2dimgrav":[16,25],"2dimhovv":[16,25],"2e_i":2,"2ff97f4bf03b":3,"2g_i":1,"2k_2":[15,24],"2k_3":[15,24],"2me":[16,25],"2meb":[16,25],"2mr":[16,25],"2mr_":[16,25],"2ms":[],"2nd":[15,24,28],"2v_":[13,22],"2v_x":[13,22],"2v_y":[13,22],"2x_ix_jy_iy_j":8,"2x_j":8,"2y_0":[12,21],"2y_j":8,"2z_":9,"30273573e":[],"3027e":1,"3037e":1,"3044e":1,"3047e":1,"30579019e":9,"3071e":1,"3095e":1,"30pm":28,"3124e":1,"3126e":1,"3134e":1,"3150e":1,"3174e":1,"31819744e":9,"3201e":1,"3212e":1,"3216e":1,"32218090e":[],"3226e":1,"3238e":1,"3246e":1,"3250e":1,"3281e":1,"32849325e":[],"3285e":1,"33026664e":9,"33066907e":5,"33291949e":[],"3337e":1,"33381074e":[],"333e":9,"33583503e":[],"33608388e":[],"33637629e":[],"3364e":1,"3370e":1,"33851317e":[],"3386e":1,"3408e":1,"3423e":1,"34343487e":9,"3434e":1,"3438e":1,"3446e":1,"3470e":1,"3475e":1,"3488e":1,"34991581e":[],"3515e":1,"35186316e":[],"3550e":1,"35938336e":[],"3631e":1,"36351028e":[],"36440033e":[],"36648631e":[],"3666e":1,"3675e":1,"3678e":1,"36816390e":[],"3682e":1,"3717e":1,"37369358e":9,"37689988e":[],"37715992e":[],"37773764e":[],"3780e":1,"379e":9,"38189254e":[],"381e":9,"3826e":1,"3840e":1,"3847e":1,"38483045e":9,"3857e":1,"3860e":1,"38957572e":5,"39086494e":[],"39110289e":[],"39377982e":[],"3980e":1,"3982e":1,"3983e":1,"3985e":1,"39860289e":9,"3f1158901148":2,"3rd":28,"3x_i":1,"4007e":1,"4017e":1,"4022e":1,"4047e":1,"4079e":1,"40925064e":[],"41088348e":[],"412252157986472e":[14,23],"41241709e":[],"4128e":1,"41511965e":7,"4156e":1,"41836625e":9,"4195e":1,"422e":9,"4292e":1,"4299e":1,"43297505e":[],"4331e":1,"4335e":1,"4340e":1,"4357e":1,"435e":9,"4361e":1,"43821714e":[],"4402e":1,"44034203e":[],"4408e":1,"4419e":1,"4450e":1,"44786221e":9,"4479e":1,"447e":9,"4491e":1,"4494e":1,"44970586e":7,"4498e":1,"4500e":1,"45035234e":[],"45426079e":[],"4655e":1,"4666e":1,"4672e":1,"4696e":1,"4700e":1,"4701e":1,"4722e":1,"47269386e":[],"4740e":1,"47580075e":[],"4817e":1,"4866e":1,"48868704e":9,"4889e":1,"48974653e":[],"4898e":1,"4929e":1,"4969e":1,"49747700e":9,"4977e":1,"49887171e":[],"4ar":[16,25],"4dr":[16,25],"4f_0":[14,23],"4pm":28,"4th":28,"5018e":1,"501e":9,"5034e":1,"5036e":1,"5039e":1,"5041e":1,"50446329e":[],"50470354e":[],"504e":9,"5052e":1,"50565973e":9,"50pm":28,"50x10":7,"5101e":1,"5105e":1,"512e":9,"51355225e":[],"51502206e":[],"5156e":1,"5159e":1,"51737928e":[],"5184e":1,"5196e":1,"52131750e":[],"52177694e":[],"52277631e":[],"52491181e":[],"52533249e":[],"52583599e":[],"5258e":1,"52783754e":[],"52806307e":[],"5307e":1,"5314e":1,"5331e":1,"5332e":1,"5339e":1,"5344e":1,"5345e":1,"5352e":1,"5362e":1,"5374e":1,"5386e":1,"53990135e988":5,"54220685e":[],"54223480e":[],"5445e":1,"5449e":1,"5451e":1,"54686289e":[],"5506e":1,"55287874e":[],"5532e":1,"55336603e":5,"5537e":1,"55456211e":[],"5546e":1,"5549e":1,"55726683e":[],"55822858e":[],"55868382e":[],"56282700e":[],"56432255e":9,"565e":9,"5691e":1,"5694e":1,"56min":[17,26],"57072598e":[],"570e":9,"5722e":1,"57425734e":5,"5793e":1,"5796e":1,"5801e":1,"5803e":1,"58130740e":[],"5831e":1,"58458590e":[],"58473257e":[],"5861e":1,"5875e":1,"58928789e":[],"58934440e":[],"58977964e":9,"58998839e":[],"5899e":1,"5901e":1,"59310961e":9,"5936e":1,"5960e":1,"5971e":1,"59766065e":[],"59775278e":[],"5984e":1,"59958864e":[],"5cm":[3,15,24],"5th":28,"6012e":1,"604e":9,"60505104e":[],"60559584e":[],"6065e":1,"60896055e":[],"60996409e":[],"61028757e":9,"6115e":1,"61233230e":[],"6124e":1,"6135e":1,"61445925e":[],"61517814e":9,"6183e":1,"6186e":1,"6197e":1,"62035199e":5,"6209e":1,"62113986e":[],"6232e":1,"62496853e":[],"62554056e":5,"6259e":1,"6264e":1,"62790003e":[],"6280e":1,"628e":9,"6291e":1,"63164705e":[],"63177381e":[],"6324e":1,"6325e":1,"6333e":1,"6367e":1,"63748198e":[],"6380e":1,"63862824e":5,"6387e":1,"638e":9,"64043119e":[],"6460e":1,"6479e":1,"64x50":7,"6501e":1,"6509e":1,"653m":1,"65939208e":5,"66371431e":[],"66517501e":[],"66528972e":9,"6659e":1,"66678115e":[],"6670e":1,"6733e":1,"6756e":1,"67761543e":9,"6776e":1,"67844938e":5,"67887686e":[],"67901631e":[],"6795e":1,"67m":[17,26],"6803e":1,"6825e":1,"6826e":1,"6832e":1,"6858e":1,"6867e":1,"6893e":1,"6905e":1,"69069n_":3,"69095941e":9,"69238112e":9,"6966e":1,"6ea927cc6e88":7,"6n_":3,"6pm":28,"6th":28,"7023e":1,"70287435e":[],"7079e":1,"70865417e":[],"70896266e":9,"710e":9,"7125e":1,"71385616e":[],"713e":9,"7141e":1,"7159e":1,"71833086e":[],"7187e":1,"72347840e":[],"72352580e":[],"7236e":1,"72425639e":5,"7252e":1,"7273e":1,"7284e":1,"7287e":1,"7288e":1,"72892756e":[],"7295e":1,"7296e":1,"729e":9,"7310e":1,"7319e":1,"73212847e":[],"73374509e":[],"7378e":1,"7385e":1,"73875395e":5,"7391e":1,"7413e":1,"7418e":1,"74406011e":[],"744e":9,"7457e":1,"7475e":1,"74784357e":[],"7489e":1,"7513e":1,"75209222e":[],"7562e":1,"7583e":1,"7585e":1,"7593e":1,"7600e":1,"76071445e":[],"7611e":1,"7654e":1,"7690e":1,"7706e":1,"7712e":1,"7722e":1,"77350269e":5,"7755e":1,"77667437e":[],"77804918e":9,"78496845e":9,"78512013e":[],"7879e":1,"78920723e":[],"7905e":1,"79318674e":9,"79390227e":5,"79464371e":[],"79482386e":9,"7968e":1,"7972e":1,"7th":28,"8043e":1,"80481726e":9,"8049e":1,"8084e":1,"80872400e":[],"8102e":1,"8111e":1,"811e":9,"8134e":1,"8135e":1,"8137e":1,"81402316e":9,"8163e":1,"8174e":1,"81862750e":[],"8203e":1,"8225e":1,"8237e":1,"82725395e":[],"8312e":1,"83131741e":[],"83194721e":9,"8321e":1,"83259341e":9,"83286207e":[],"83323451e":[],"8335e":1,"8355e":1,"8358e":1,"8361e":1,"8382e":1,"84045777e":[],"84355903e":7,"8441e":1,"84443254e":7,"8467e":1,"84683780e":[],"8501e":1,"852e":9,"8597e":1,"86195387e":9,"8674e":1,"86797194e":[],"8689e":1,"86972479e":9,"8700e":1,"8702e":1,"8710e":1,"873e":9,"8755e":1,"8767e":1,"8771e":1,"8779e":1,"8793e":1,"8797e":1,"882e":9,"884935143687282e":1,"885e":9,"8864e":1,"8865e":1,"8874e":1,"8886e":1,"8894e":1,"8897e":1,"89045976e":[],"89346326e":5,"89354206e":[],"8938e":1,"89456494e":[],"8949e":1,"89585850e":5,"8th":28,"8x8":7,"9011e":1,"9020e":1,"903e":9,"90524667e":[],"90548268e":9,"90555053e":[],"9066e":1,"9074e":1,"90838144e":[],"91032829e":[],"9124180119966666e":1,"91269450e":9,"9130e":1,"9159e":1,"917u":[],"91811329e":9,"9190e":1,"9197e":1,"9199e":1,"9233e":1,"92407803e":9,"92507116e":7,"9254e":1,"9274e":1,"92870110e":[],"9295e":1,"92983892e":9,"9299e":1,"9305e":1,"930e":9,"932u":[],"9330e":1,"933u":[],"9351e":1,"9370e":1,"938e":9,"93947492e":[],"939u":[],"94395333e":[],"94466097e":[],"9452e":1,"94553659e":[],"9456e":1,"948e":9,"95125898e":[],"9527e":1,"9548e":1,"954e":9,"9551e":1,"95572537e":[],"955820c21e8b":[1,10],"957u":10,"958u":10,"959u":10,"9601e":1,"96064784e":[],"960e":9,"960u":10,"9610e":1,"9625e":1,"9636e":1,"9638e":1,"9656e":1,"968u":10,"9706e":1,"9712e":1,"9715e":1,"9720e":1,"9755e":1,"97624658e":[],"9763e":1,"9771e":1,"9781e":1,"9787e":1,"98010863e":[],"9809e":1,"981e":9,"98414609e":9,"9843e":1,"9844e":1,"984e":9,"9857e":1,"98658737e":[],"9903e":1,"9906e":1,"9911e":1,"99126621e":9,"9923e":1,"9924e":1,"9964e":1,"996e":9,"9th":28,"abstract":[7,11,20],"boolean":[1,2,10],"break":[4,5,9,12,13,16,21,22,25],"byte":[4,12,21],"case":[0,1,2,4,5,6,7,9,10,11,13,14,15,16,17,20,22,23,24,25,26],"catch":4,"char":3,"class":[0,3,4,5,7,8,9,10,11,12,15,20,21,24,28],"const":[3,9],"default":[1,4,5,6,7,10,12,21,34,36],"final":[2,3,4,5,6,8,11,15,16,17,20,24,25,26],"float":[1,3,4,5,9,10,11,12,14,20,21,23],"forss\u00e9n":2,"function":[0,2,4,10,11,12,13,14,15,16,17,20,21,22,23,24,25,26,34,36],"import":[0,1,2,5,6,7,8,9,11,13,14,15,16,18,19,20,22,23,24,25,28,31,35],"int":[1,2,3,5,7,9,10,13,15,16,22,24,25],"long":[1,2,3,4,7,10,13,15,16,17,22,24,25,26],"new":[0,2,3,4,5,6,7,8,9,11,12,13,14,15,20,21,22,23,24],"null":3,"public":[0,4,12,21,29],"return":[1,2,3,4,5,7,8,9,10,11,12,13,14,15,16,20,21,22,23,24,25],"s\u00f8renssen":28,"short":[1,10,27,32],"super":5,"switch":[4,16,17,25,26],"throw":[3,13,15,17,22,24,26],"true":[1,2,3,4,5,6,7,8,9,10,14,15,16,17,23,24,25,26],"try":[0,3,4,6,7,8,9,11,12,13,15,16,20,21,22,24,25],"var":[3,5,9],"while":[0,1,2,3,4,5,7,8,9,10,13,14,15,17,22,23,24,26,28],AGE:5,AMS:28,AND:[1,2],Added:[13,22],Adding:[0,7],And:[0,3,5,10,11,12,20,21],Are:[9,13,22],BPS:28,Being:29,But:[1,2,3,4,5,7,9,13,14,17,18,19,22,23,26,35],CAS:[4,12,21],DIS:5,Doing:5,EoS:5,For:[0,1,2,3,4,5,6,7,8,9,10,12,13,14,15,16,17,18,19,21,22,23,24,25,26,28,34,35,36],GPS:0,Going:[5,7],I_s:[15,24],Ising:7,Its:[1,7,9,10],MDS:9,NNs:7,N_s:8,Not:[0,4,5,9],ODE:[13,22],One:[0,3,5,6,7,8,9,10,14,15,16,17,23,24,25,26],PCs:9,RMS:3,Such:[3,5,7,11,13,20,22],That:[2,3,4,6,7,9,13,17,22,26],The:[0,4,11,12,16,20,21,25,28,29,30,31,34,36],Then:[3,4,5,7,8,9,12,16,17,21,25,26],There:[0,2,3,4,5,7,8,9,10,11,12,14,15,17,18,19,20,21,23,24,26,27,32,34,35,36],These:[0,1,2,3,4,5,7,8,9,10,11,12,13,15,16,17,20,21,22,24,25,26,28],Use:[0,1,10,16,25],Useful:5,Uses:[1,10],Using:[2,4,5,8,10,12,13,14,15,16,17,21,22,23,24,25,26,29],With:[3,4,5,7,8,9,11,12,13,14,15,16,17,20,21,22,23,24,25,26],Yes:[0,13,22],_________________________________________________________________:1,__doc__:5,__future__:8,__init__:[5,7],__main__:1,__name__:[1,5],_auto10:[5,7,13,14,15,16,22,23,24,25],_auto11:[5,13,14,15,16,22,23,24,25],_auto12:[13,14,15,16,22,23,24,25],_auto13:[5,13,14,15,16,22,23,24,25],_auto14:[13,14,15,16,22,23,24,25],_auto15:[13,14,15,16,22,23,24,25],_auto16:[13,15,16,22,24,25],_auto17:[13,15,16,22,24,25],_auto18:[13,15,16,22,24,25],_auto19:[13,15,22,24],_auto1:[1,2,3,5,6,7,13,14,15,16,17,22,23,24,25,26],_auto20:[13,15,22,24],_auto21:[13,15,22,24],_auto23:[15,24],_auto24:[15,24],_auto25:[15,24],_auto26:[15,24],_auto27:[15,24],_auto28:[15,24],_auto29:[15,24],_auto2:[1,2,3,5,7,13,14,15,16,17,22,23,24,25,26],_auto30:[15,24],_auto31:[15,24],_auto32:[15,24],_auto33:[15,24],_auto34:[15,24],_auto3:[2,5,7,13,14,15,16,17,22,23,24,25,26],_auto4:[2,5,7,13,14,15,16,17,22,23,24,25,26],_auto5:[2,5,7,13,14,15,16,22,23,24,25],_auto6:[2,5,7,13,14,15,16,22,23,24,25],_auto7:[2,5,7,13,14,15,16,22,23,24,25],_auto8:[2,5,7,13,14,15,16,22,23,24,25],_auto9:[5,7,13,14,15,16,22,23,24,25],_ax:5,_base:8,_build:31,_check_optimize_result:9,_compon:9,_datafram:[4,12,21],_lambda:5,_logist:9,_multilayer_perceptron:7,_ratio:9,_setattr_cm:5,_subplot:5,_update_properti:5,_varianc:9,a_0:[5,14,23],a_1:[15,17,24,26],a_1a:[4,5],a_1x:[14,23],a_2:[15,17,24,26],a_2a:[4,5],a_2a_3:[14,23],a_2x:[14,23],a_3:[4,17,26],a_3a:5,a_3a_2:[14,23],a_4:4,a_4a:5,a_fc1:7,a_fc2:7,a_h:7,a_i:[1,4,7,12,14,17,21,23,26],a_j:7,a_ja_k:[14,23],a_k:7,a_n:[14,23],a_p:[15,24],a_x:[12,21],a_z:[12,21],aaverag:[12,21],abandon:7,abid:[3,30],abil:[0,4],abl:[2,3,4,5,6,7,13,15,16,22,24,25,28],abort:3,abotu:[17,26],about:[0,1,3,5,7,8,9,10,11,13,14,15,16,17,18,19,20,22,23,24,25,26,28,30,34,35,36],abov:[1,2,3,4,5,6,7,8,9,10,12,13,14,15,16,17,21,22,23,24,25,26],abovement:[5,11,17,20,26],abs:[1,4,5,11,14,20,23],absolut:[1,4,5,11,12,20,21],absorb:[2,15,16,17,24,25,26],abus:29,acccess:4,accelearatio:[12,21],acceler:[14,15,16,23,24,25,28],acceleromet:[17,26],accept:[0,2,5,10,13,22,29,34,36],access:[0,2,3,4,5,9,10],accid:[1,5,10],accommod:[15,24],accompani:[0,5],accomplish:[2,8,16,25],accord:[1,2,3,4,5,7,12,15,16,17,21,24,25,26],accordingli:9,account:[0,3,4,10,12,17,21,26,29],accumul:[2,3,7,11,20],accur:[1,3,5,10,14,23],accuraci:[0,4,5,7,9,10,14,23],accuracy_scor:[4,7],accuracy_score_numpi:7,achiev:[0,4,5,7,8,12,21],aco:[3,13,22],acquaint:0,acquir:[0,7,28],acr:5,across:[4,5,7,10,12,21],act:[2,7,12,14,15,16,17,21,23,24,25,26,28,29],action:[3,13,14,22,23,29,31],activ:[1,4,10,28],actual:[2,4,5,7,8,9,12,13,16,17,21,22,25,26],adam:[1,10],adapt:[2,5,29],add:[0,1,3,4,5,7,8,9,12,13,14,15,17,21,22,23,24,26,34,36],add_:[18,19,35],add_graph:7,add_subplot:[6,7],added:[4,5,7,8,9,10,11,13,14,17,20,22,23,26],adding:[5,7,14,15,23,24],addit:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,15,16,17,20,21,22,24,25,26,28],addition:7,address:[2,9,11,14,20,23,29],adequ:2,adequaci:2,adjac:7,adjust:[13,15,16,22,24,25],admir:4,adopt:[12,21],advanc:[7,11,20,29],advantag:[5,7,10,13,22],afecionado:4,affect:[2,5,14,16,23,25],affin:[5,8,9,10],afford:10,aficionado:4,african:5,after:[0,1,3,4,5,7,9,10,12,14,15,16,21,23,24,25,28],afterward:[4,12,21],ag_0:1,again:[0,3,4,5,6,7,8,9,14,15,17,23,24,26,34,36],against:[7,13,15,22,24],age:[0,4,5,29],agre:[2,3,15,24,30],agreement:2,aid:9,aim:[0,4,6,9,11,12,13,14,16,20,21,22,23,25],air:[14,15,17,23,24,26,28],aircraft:[17,26],airplan:10,akin:[15,24],algebra:[0,2,4,16,25],algo:[0,3],algorithm:[0,1,2,3,4,6,8,10,11,12,13,15,20,21,22,24,28],align:[1,3,5,6,8,18,19,29,35],all:[0,1,2,3,4,5,6,7,9,10,11,12,13,14,15,16,17,20,21,22,23,24,25,26,29,31,34,36],allevi:7,allow:[0,1,2,4,5,7,8,9,10,11,12,13,14,15,16,20,21,22,23,24,25,28,34,36],almost:[0,3,4,5,7,8,9],alon:[1,15,24],along:[1,4,5,9,10,12,14,16,17,21,23,25,26],alpha:[1,2,3,4,5,7,8,9,10,13,14,16,17,22,23,25,26],alpha_n:[15,24],alreadi:[1,2,3,4,7,12,13,15,21,22,24,34,36],also:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,28,34,35,36],alter:[3,7],altern:[2,4,5,7,8,9,12,13,14,21,22,23],although:[2,5,7,8,13,14,15,22,23,24],alwai:[0,3,4,5,7,10,12,13,14,16,21,22,23,25,28,30],amaz:0,ame2016:[4,5],american:5,among:[0,5,7,10],amongst:[16,25],amount:[2,5,8,10,14,15,17,23,24,26],amplitud:[13,15,17,22,24,26],an_:3,anaconda3:[5,7,8,9],anaconda:[4,7,12,21,28],analog:[2,16,25],analogu:[16,25],analys:[2,3,5],analysi:[4,6,7,10,11,12,13,16,20,21,22,25],analyt:[2,4,5,6,7,11,13,14,15,17,20,22,23,24,26,28],analyz:[0,1,3,4,5,7,10,13,22,28],ander:28,andrew:7,anew:[14,15,16,23,24,25],angl:[3,5,9,12,14,15,16,17,21,23,24,25,26],angmom:[16,25],angular:[13,15,16,17,22,24,25,26,28],anharmon:[15,24],ani:[0,1,2,3,4,5,6,7,8,10,13,14,15,16,17,22,23,24,25,26,29,30,34,36],anim:7,ann:7,annot:[4,5,7,8,9,10],anoth:[0,1,2,3,4,7,8,9,10,12,13,15,16,17,21,22,24,25,26,34,36],ans:3,ansatz:4,answer:[0,2,4,5,7,12,13,15,21,22,24],antialias:1,anticip:[1,10,12,21],antiparallel:[16,25],antisymmetri:[14,23],anymor:8,anyon:8,anyth:[3,7,30],apach:7,apart:[2,9],api:[4,7,12,21],appar:[1,2,17,26],appear:[0,2,3,4,7,10,13,14,15,16,17,22,23,24,25,26,29],append:[1,3,4,8,10,12,21],appendix:[15,24],appl:0,appli:[2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,20,21,22,23,24,25,28,29],applic:[0,1,3,4,5,7,9,10,11,13,17,20,22,26,28],appoint:29,appreci:30,approach:[1,3,4,7,9,11,12,14,15,16,20,21,23,24,25],appropri:[1,2,3,5,7,13,22,29],approx:[1,2,3,5,9,13,14,15,16,17,22,23,24,25,26],approxim:[1,2,3,5,6,9,10,11,13,14,15,17,20,22,23,24,26],apt:[4,12,21],aragorn:[4,12,21],arang:[1,2,4,5,6,7,10,12,21],arbitrari:[1,3,7,8,10,13,15,16,17,22,24,25,26],arbitrarili:[4,7,9,16,25],architectur:[1,10],arctan:[16,25],arctanh:[12,21],area:[0,2,4,10,13,14,16,22,23,25],areal:[16,25],arg:[2,5],argc:3,argmax:[7,9],argsort:9,argu:7,argument:[1,3,5,7,9,10],argv:3,aris:[0,2,3,4,5,7,17,26],arithmet:4,arma:3,armadillo:3,around:[2,3,4,5,7,9,12,13,14,15,16,17,21,22,23,24,25,26],arrai:[1,2,3,5,7,8,10,11,14,15,16,18,19,20,23,24,25,35],arrang:[10,28],arriv:[2,3,4,9,11,20],arrow:[7,13,22],arrowprop:8,art:0,articl:[3,5,7,10,30],artifici:[0,1,4,6],artificialneuron:7,artist:5,arxiv:[1,10],asarrai:4,ascii:[3,4],asid:[2,14,15,23,24],asin:[13,15,22,24],ask:[2,5,7,9,13,17,22,26],aspect:[4,11,13,20,22],assembl:[4,10],assert:[1,10],assess:[0,2,5],assici:[1,10],assign:[0,2,4,5,6,7,8,16,25,28],assist:28,associ:[2,3,4,5,7,17,26,34,36],assum:[1,2,3,4,5,6,7,8,9,10,12,13,14,15,16,17,21,22,23,24,25,26],assumpt:[0,2,3,4,9,10,17,26],ast:5,asteroid:[16,25],astronom:[2,14,23],astronomi:2,asymmetr:[2,14,23],asymmetri:4,asymptot:[5,16,25],atoi:3,atom:[4,13,14,15,16,22,23,24,25],attack:29,attempt:[2,4,14,23],attent:[2,4,12,17,21,26,29],attitud:[0,11,20],attract:[4,16,25],attribut:[2,5],attributeerror:5,audi:4,audio:[1,10],auditori:[34,36],australia:[34,36],author:[0,3,4,7],authour:[0,4],auto:[3,9],autocor:3,autocorrelation_tim:3,autocorrelform:3,autocovari:3,autograd:[4,12,21],autom:[0,11,20,31],automag:[0,4,12,21],automat:[4,9,10,12,21,31],automobil:[10,13,22],autonom:[1,10],avail:[0,2,4,5,7,9,11,12,20,21,28],avdt:[13,22],avenu:0,averag:[2,3,4,5,7,10,14,16,23,25],avg:3,avoid:[0,1,4,5,9,10,12,14,21,23],awai:[1,2,3,5,15,16,17,24,25,26],awar:1,aws:28,axel:[13,22],axes3d:1,axes:[5,8,9,10,12,13,18,19,21,22,35],axes_grid1:5,axessubplot:5,axhlin:8,axi:[1,2,3,4,5,6,7,8,9,10,13,14,16,17,22,23,25,26],axiom:2,axlabel:[4,12,21],axs:[2,12,21],axs_vec:2,axvlin:[1,8,10],axvspan:[1,10],azimuth:[14,16,23,25],b_1:[1,7],b_1e:[15,24],b_2e:[15,24],b_fc1:7,b_fc2:7,b_i:[1,5,7],b_ia_:5,b_j:7,b_k:7,b_m:7,b_out:7,b_p:[15,24],b_x:[12,21],b_y:[12,21],b_z:[12,21],back:[0,2,3,5,10,12,13,14,16,17,21,22,23,25,26],backend:[1,10],background:[2,12,21],backpropag:7,backward:[1,7,10,16,25],bad:[3,5],badli:3,baggin:[4,12,21],bake:[11,20],balanc:5,ball:[2,12,13,17,21,22,26],ban:29,band:[4,12,21],bandwidth:[4,12,21],bar:[2,4,9,12,14,21,23],barrier:28,base:[0,1,2,3,4,5,6,7,8,10,11,12,15,20,21,24],basi:[4,5,6,7,8,9,12,16,21,25,28],basic:[0,2,3,7,8,11,15,17,20,24,26],batch:[7,9,10],batch_i:7,batch_shap:[1,10],batch_siz:[1,7,10],batch_x:7,bay:6,bayesian:0,bdb:[16,25],beach:2,bear:2,becaus:[0,1,2,4,5,7,8,10,11,13,14,15,16,17,20,22,23,24,25,26],becom:[0,1,2,3,4,5,7,9,12,13,14,15,16,17,21,22,23,24,25,26],been:[0,1,2,4,5,7,9,10,11,12,13,15,20,21,22,24,28,31],befor:[0,1,2,3,4,5,6,7,8,10,11,13,14,15,16,17,20,22,23,24,25,26],beforehand:[0,3,4],began:[16,25],begin:[1,2,3,4,5,6,7,8,9,12,13,14,15,16,17,18,19,21,22,23,24,25,26,28,35],behav:[5,7,16,25],behavior:[0,4,7,11,15,20,24,29],behaviour:[2,7],behind:[0,4,5,7,8,15,24],being:[0,1,2,3,4,5,6,7,8,9,10,11,15,16,17,20,24,25,26,34,36],belief:2,believ:[2,13,22],belong:[6,8,17,26],below:[1,3,4,5,6,7,8,9,10,11,12,13,14,15,17,20,21,22,23,24,26,30],benchmark:[11,20],benefici:7,benefit:[1,4,7,9,10,11,20],bengio:7,benign:9,bernoulli:2,besid:[1,2,10],best:[0,1,3,4,5,7,8,9,10,13,15,16,22,24,25,29,30],bet:2,beta:[4,6,7,9,16,25],beta_0:[5,6,7,15,24],beta_0x_:5,beta_1:[5,6,7],beta_1x_0:5,beta_1x_1:[5,6],beta_1x_2:5,beta_1x_:5,beta_1x_i:6,beta_2:5,beta_2x_0:5,beta_2x_1:5,beta_2x_2:[5,6],beta_2x_:5,beta_:[5,6],beta_i:5,beta_j:5,beta_n:[15,24],beta_p:6,beta_px_p:6,better:[0,1,2,4,7,9,10,11,13,14,15,20,22,23,24],between:[0,1,2,3,4,5,6,7,8,9,10,12,13,14,15,16,17,21,22,23,24,25,26],beyond:[0,4,5,7,8,11,15,20,24],bia:[0,1,2,3,7,8,10],bias:[1,2,5,10],bias_vari:7,bib:[34,36],bibliographi:[34,36],bibtex:[34,36],bicycl:[12,21],big:[0,1,4,5,7,13,22],bigger:7,bigr:7,bilbo:[4,12,21],billion:7,bimod:2,bin:[2,3,5,9],binari:[3,4,6,10],binomi:[0,2],binsboot:5,bioinformat:[0,4],biolog:7,bird:[4,10],birth:[4,12,21],bit:[2,3,4,7,12,13,15,17,21,22,24,26,30],bitwis:3,bivari:1,bla:[4,12,21],black:8,blob:28,block:[0,4,5,12,21,28,34,36],blockeulercrom:[15,24],blockingavg:3,blockingstd:3,blockingvar:3,blocksiz:3,blocksizemax:3,blocksizemin:3,blog:[2,30],blue:[4,10],blue_patch:[12,21],bmatrix:[4,5,6,7,8,9,12,21],bmi:7,bodi:[0,1,5,7,10,13,14,17,22,23,26,28,29],boldfac:[5,12,21],boldsymbol:[0,1,2,4,7,8,9,12,13,14,16,17,21,22,23,25,26],bolt:[13,22],boltzmann:[0,7],bonu:[11,20],book:[2,10,18,19,28,34,35,36],boost:7,bootavg:3,bootstd:3,bootstrap:[0,7],bootvar:3,bootvec:3,bore:0,borrow:[4,12,21],boston_dataset:5,bot:8,both:[0,2,3,4,5,7,8,11,12,13,15,16,17,20,21,22,24,25,26,29,34,36],bottl:6,bottom:[5,15,24],bounc:[13,22],bound:[5,7,8,16,25],boundari:[1,7,8,9,13,22],box:[1,10,14,23,34,36],boyd:8,bracket:[1,3,10],brain:[6,7],brake:[17,26],branch:[30,31],brand:0,breast:7,brew:[4,12,21],brg:8,brian:[34,36],brief:[14,23],briefli:[4,11,15,20,24],bring:[4,5,15,17,24,26],brisban:[34,36],broad:[0,4],broken:[13,22],browser:4,brute:[5,9],bte:[15,24],bthe:[16,25],buck:0,bui:[1,10],build:[0,1,3,4,5,12,21,34,36],built:[1,5,7,10,15,24,31,34,36],bunch:9,bundl:33,buoyanc:[12,21],burn:2,bus:[17,26],busi:5,butler:28,c46dd114b2af:8,c_0:3,c_1:[7,17,26],c_1e:[17,26],c_2:[7,17,26],c_2e:[17,26],c_3:7,c_4:7,c_i:7,c_k:3,c_w:[13,22],cal:[7,8],calcul:[0,1,2,4,5,7,8,9,10,11,12,13,14,15,16,20,21,22,23,24,25],calculu:28,call:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,20,21,22,23,24,25,26,34,36],callabl:5,came:[13,17,22,26],camera:0,can:[0,1,2,3,4,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,28,30,31,34,35,36],cancel:[5,14,23],cancer:7,cancerpd:9,candid:[2,8],cannot:[0,2,3,4,5,7,8,9,13,14,22,23],canopi:[4,12,21],capabl:[0,4,8,12,21],capac:1,capita:5,captur:[7,9],car:[0,1,10,13,17,22,26],card:[0,5,6],cardin:7,care:[9,15,24],carlo:[0,2,3,4,5],carmak:0,carri:[1,2,5,6],cartesian:[12,16,21,25],cast:7,cat:10,categor:[0,4,7,9,10],categori:[0,4,6,7,10],categorical_crossentropi:[7,10],caus:[0,2,3,5,9,13,17,22,26],causal:5,causat:0,caveat:[14,23],cbook:5,ccc:7,cdf:3,cdot:[1,3,4,5,7,12,13,14,15,16,17,21,22,23,24,25,26],ceil:[12,14,15,16,21,23,24,25],cell:[34,36],cellphon:28,center:[3,4,5,6,8,13,14,15,17,22,23,24,26,28],central:[0,2,4,5,8,10,11,12,17,20,21,26,28],centrifug:28,centroid:3,centuri:[2,10,11,20],certain:[0,3,4,5,6,10,16,25],certainli:2,cha:5,chain:[0,2,3,14,16,17,23,25,26,28],challeng:[14,23],chalmer:2,chanc:[2,3,7,16,25],chang:[1,2,3,4,5,7,8,9,10,11,12,13,14,15,16,17,20,21,22,23,24,25,26,30],channel:10,chapter:[5,9,10,15,24,28],charact:[3,4,5,8,10],character:[2,3,7,8],characterist:[5,10,15,24],charg:[4,13,16,22,25],charl:5,cheap:5,cheaper:7,check:[1,2,4,5,7,9,10,12,13,15,18,19,21,22,24,33,35],checkmark:[15,24],chemic:3,chi:4,chiaramont:1,chimborazo:[17,26],choic:[1,2,3,5,10,12,13,15,16,21,22,24,25],choleski:5,choos:[1,2,3,5,9,10,12,13,14,15,16,21,22,23,24,25],chosen:[1,2,3,4,5,7,8,12,13,14,15,16,17,21,22,23,24,25,26],chosen_datapoint:7,chpater:[15,24],christian:2,christoph:[34,36],cifar10:10,cifar:10,cin:3,circ:[7,17,26],circl:[5,7,8,9,13,16,22,25],circuit:[15,24],circul:[16,17,25,26],circular:[13,14,16,22,23,25],circumst:29,circumv:[5,7],cite:[34,36],citizen:0,civita:[14,23],claim:0,clarifi:29,clariti:3,class_nam:10,classic:[2,4,11,14,16,17,20,23,25,26,36],classif:[0,2,4,5,6,8,10,12,21],classifi:[0,6,7,9],classificaton:7,clean:7,clear:[2,7,16,25],clearli:[3,5,8,9,10,13,14,22,23],clever:7,clf3:4,clf:[4,5,8],clf_lasso:5,clf_ridg:5,clip:[3,10],clockwis:[13,17,22,26],clone:[30,31],close:[0,1,3,4,5,7,8,9,12,13,15,16,17,21,22,24,25,26],closer:[13,22],closest:[8,9,12,16,21,25],closur:[4,12,21],cloud:[4,12,21],cluster:[0,1,4,5,7,9,10],cmap:[1,4,5,7,8,10,18,19,35],cmap_arg:5,cmath:3,cmse201:[11,20],cn_:3,cna:[13,22],cnn:7,cnn_kera:10,cntk:[4,12,21],code:[0,4,6,10,11,13,20,22],coef0:8,coef:4,coef_:[4,5,8],coeffici:[4,5,6,8,13,15,17,22,24,26],coerc:[4,5],cognit:[34,36],coin:3,coin_posterior_norm:2,coin_prior_uniform:2,coinflip:2,coinflipping_fig_1:2,col:[2,4,5,9,12,21],colab:[4,12,21],cold:[18,19,35],colinear:5,collaps:8,collect:[1,2,3,4,5,9,12,13,21,22],collid:[0,16,25],collinear:5,collis:[0,16,25],color:[1,3,4,5,8,10,12,16,18,19,21,25,35],color_channel:10,color_cod:5,colorbar:5,column:[1,4,5,7,8,9,12,21],com:[1,2,10,28],combin:[1,2,3,5,6,11,14,16,17,20,23,25,26],come:[0,1,2,4,5,7,10,12,13,15,16,17,21,22,24,25,26],comma:4,command:[0,3,4,7,11,12,20,21,34,36],comment:[2,29],commerci:[4,12,21],commit:[29,30],commod:4,common:[0,2,3,4,5,6,7,9,10,12,13,14,15,16,21,22,23,24,25],commonli:[1,5,6,7,10],commonmark:[34,36],commun:[0,4,7,29],commut:[14,23],compact:[5,6,7,9,14,16,23,25],compactli:[14,23],compani:0,compar:[2,4,5,9,10,11,12,13,15,16,17,20,21,22,24,25,26],comparison:[1,2,10],compat:6,compdriv:[15,24],compet:[0,4,11,20,28],compil:[0,1,3,4,7],complaint:29,complement:[14,23],complementari:[15,24],complet:[1,2,4,7,10,13,15,22,24],completenn:7,complex:[0,2,5,7,8,9,15,16,17,24,25,26,34,36],complic:[0,2,4,5,7,11,13,16,17,20,22,25,26],compon:[1,5,7,10,13,14,15,16,17,22,23,24,25,26],components_:9,compos:7,compress:[4,5,9,13,22],compris:5,compromis:[2,5],comput:[0,1,2,4,5,7,8,10,11,13,14,15,17,20,22,23,24,26,28,34,36],computation:[0,3,10],con:28,concaten:[1,5,10],concentr:[2,5],concept:[0,1,2,4,5,14,23],conceptu:[7,16,17,25,26],concern:[0,1,6,7,10,14,23],concic:[4,12,21],concis:2,conclud:4,conclus:[0,7],cond:1,conda:[4,7,12,21,30],condit:[1,2,3,5,8,9,12,13,14,15,16,17,21,22,23,24,25,26],conduct:[0,11,20],condwav:1,confer:[34,36],confid:[5,6,8],confidenti:29,configur:10,confirm:7,confront:2,confus:5,congruenti:3,conjug:8,connect:[1,4,5,7,9,10,12,21,28],consequ:[5,7,8,11,16,20,25],conserv:[5,13,15,16,22,24,25,28],consid:[0,1,2,3,4,5,6,7,8,10,13,14,15,16,17,22,23,24,25,26,29],consider:[0,5,7,9,13,22],consist:[1,2,3,5,7,10,13,15,22,24],constant:[1,2,3,4,5,7,8,13,14,15,16,17,22,23,24,25,26],constitu:4,constitut:[1,5],constrain:[2,7,9,10,16,25],constraint:[5,8,28],construct:[0,1,2,3,4,5,7,8,9,11,20,29],consumpt:[13,22],contact:[4,13,16,22,25,29],contain:[0,1,2,3,4,5,6,7,8,9,10,11,12,20,21,28],content:[4,12,21,33,34,36],context:5,continu:[0,1,2,4,5,6,7,8,11,12,13,17,20,21,22,26],continuosli:0,continuum:2,contourf:8,contrari:2,contrast:[7,13,17,22,26],contribut:[0,3,4,5,10,14,15,16,17,23,24,25,26,29,31],contributor:[0,4,29,30],control:[0,4,7,10],controversi:2,conv2d:10,conv:10,convei:2,conveni:[2,4,5,7,11,12,13,14,16,20,21,22,23,25],convent:[2,7,11,12,20,21],converg:[1,2,5,7,8,9],convergencewarn:[7,8,9],convert:[1,4,5,9,10,12,16,17,21,25,26],converttomatrix:[1,10],convex:6,convolut:[1,15,24],cook:2,cookiecutt:31,coolwarm:[18,19,35],coordin:[5,7,9,13,14,15,16,22,23,24,25,28],coorel:5,copi:[7,30],copyright:2,corel:0,corioli:28,corner:[2,17,26],corona:28,corr:[5,9],correalt:[5,9],correct:[0,1,3,4,5,7,10,13,14,15,17,22,23,24,26,29],correct_predict:7,correctli:[1,2,5,7],correl:[0,2,4,7],correlation_matrix:[5,9],correspod:[17,26],correspond:[2,3,4,5,7,8,9,10,12,14,15,16,17,21,23,24,25,26],cortex:[7,34,36],cos:[1,5,12,13,14,15,16,17,21,22,23,24,25,26],cosh:[12,21],cosin:[5,15,24],cost:[0,3,4,5,6,8,10],cost_deep_grad:1,cost_funct:1,cost_function_deep:1,cost_function_deep_grad:1,cost_function_grad:1,cost_grad:1,cost_sum:1,could:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,20,21,22,23,24,25,26,29,30],coulomb:[4,16,25],count:[3,4],counter:[2,12,15,17,21,24,26],coupl:[1,5,10,12,13,15,21,22,24],cours:[0,2,3,5,9,10,11,12,13,14,20,21,22,23,28],cout:3,cov:[3,4,5,9],cov_xi:[5,9],cov_xx:[5,9],cov_yi:[5,9],covari:[0,4],covariance_matrix:[5,9],coven:29,cover:[0,4,15,16,24,25,27,28,32],covert:4,covxi:3,covxx:3,covxz:3,covyi:3,covyz:3,covzz:3,creat:[1,4,7,9,10,11,12,16,18,19,20,21,25,29,30,31,35],create_accuraci:7,create_biases_and_weight:7,create_convolutional_neural_network_kera:10,create_dnn:7,create_loss:7,create_neural_network_kera:7,create_optimis:7,create_placehold:7,create_x:[5,9],creation:7,credibl:2,credit:[0,5,6,28,30],crim:5,crime:5,criteria:[3,4],critic:[0,14,23,28,29],criticallydamp:[15,24],cromer:[15,24,28],cross:[0,3,6,7,10,12,13,14,15,17,21,22,23,24,26,28],cross_val_scor:5,crossvalid:5,crucial:[3,7],cs231:[1,10],csc:[15,24],csr_matrix:4,cstdlib:3,csv:[1,4,5,10],ctnk:7,ctrajectori:[16,25],cubic:4,cumsum:[4,9,12,21],cumul:5,current:[1,2,10,17,26,31],curs:[5,9],curv:[2,6,7,13,14,22,23],curvatur:[16,17,25,26],custom_lin:[18,19,35],cvxopt:8,cycl:[3,7,16,25],cycler:[18,19,35],cyclon:[17,26],cyclotron:[2,13,22],d2_g_t:1,d2l:28,d_g_t:1,d_i:2,d_net_out:1,da_d:[16,25],da_i:[17,26],dagger:[4,5,12,21],dai:[7,11,13,17,20,22,26,28],damp:28,dampeddiffyq:[15,24],dan:2,darkr:3,dat:[4,5,12,21],dat_id:[4,5,9,12,14,15,16,21,23,24,25],data:[6,8,11,12,13,14,15,16,18,19,20,21,22,23,24,25,35],data_id:[4,5,9,12,14,15,16,21,23,24,25],data_indic:7,data_panda:[4,12,21],data_path:[4,5,9,12,14,15,16,21,23,24,25],databas:[4,7],datafil:[4,5,9,12,14,15,16,21,23,24,25],datafram:[1,4,5,9,10,12,21],datapoint:[5,6,7,9],dataset:[1,2,5,6,8,9],datatyp:[1,10],date:[0,2,4,12,21],dbh:7,dbo:7,ddot:[1,13,15,16,17,22,24,25,26],ddrive:[15,24],dead:7,deadlin:28,deal:[0,2,3,4,5,7,8,9,10,12,17,21,26],dearli:0,death:2,debat:2,debt:6,debug:5,dec:2,decad:[0,4,10],decai:[3,4,13,15,22,24],decid:[0,1,2,5,10],decim:[4,5],decis:[0,4,7,8,9],decision_funct:8,decisiontreeregressor:4,declar:[1,4,5,10,12,21],decomposit:[0,7],decompost:5,decreas:[1,2,5,7,9,14,23],deduc:[0,4,13,22],deduct:2,deem:[13,22,29],deep:[4,6,10,13,22],deep_neural_network:1,deep_param:1,deepen:[11,20],deeper:[0,4,10,11,13,14,20,22,23,28],deer:10,def:[1,2,3,4,5,7,8,9,10,11,12,14,15,16,20,21,23,24,25],def_covari:3,default_tim:[1,10],defect:5,defici:5,defin:[2,3,4,5,6,8,9,10,11,12,14,15,16,17,20,21,23,24,25,26,28,29,34,36],definit:[1,2,5,8,9,11,12,13,15,16,17,20,21,22,24,25,26],defint:[3,12,21],deflect:[17,26],degre:[0,2,3,4,8,11,13,14,15,16,17,20,22,23,24,25,26],delet:[3,5],delimit:[1,10],deliv:4,delta:[1,3,4,7,8,12,13,14,15,16,17,21,22,23,24,25,26],delta_:[4,7,12,21],delta_h:[4,7],delta_j:7,delta_k:7,delta_l:7,delta_n:[4,15,24],deltat:[12,14,15,16,21,23,24,25],deltax:[16,25],delv:4,demonstr:[2,4,5,6,7,15,17,24,26],demostr:[17,26],den:[1,10],denomin:[2,7,15,16,24,25],denot:[1,2,3,5,6,7,14,17,23,26],dens:[1,7],dense_1:1,densiti:[1,2,3,5,13,16,22,25],depart:[2,17,26],departur:[2,16,25],depend:[1,2,3,4,5,6,7,8,9,12,13,14,15,16,17,21,22,23,24,25,26,28,34,36],depict:3,deploy:[4,12,21],deposit:[16,25],depth:10,deriv:[1,2,4,5,6,8,9,11,12,13,17,20,21,22,26,28],derogatori:29,descend:[5,9],descent:[0,5,6,8,10],descr:5,describ:[0,1,2,4,5,7,8,9,12,13,15,16,17,21,22,24,25,26],descript:[5,8,17,26],deserv:0,design:[4,6,7,13,15,22,24],designmatrix:5,desir:[0,1,2,4,5],despit:[2,7,13,22],destroi:2,det:5,detail:[4,9,12,14,16,21,23,25,28,29,30],detect:[7,8,17,26],detector:[16,25],determin:[0,1,2,3,4,5,7,8,9,10,12,13,14,15,17,21,22,23,24,26,28,29],determinist:[3,6],detial:28,detriment:0,dev:[3,7],develop:[0,2,4,8,9,10,11,12,13,14,15,16,20,21,22,23,24,25,28,30,31],deviat:[1,2,4,5,7,12,21],devic:[2,11,20],devinderjit:2,devis:7,df1:[4,12,21],diag:[5,8],diagnost:7,diagon:[3,4,5,6,12,21],diagonaliz:5,diagram:28,diagsvd:5,dial:[15,24],dialect:0,diamet:[17,26],dict:[5,8],dict_kei:5,dictionari:5,did:[2,5,6,7,9,13,15,17,22,24,26],die:7,died:[15,24],diff1:1,diff2:1,diff:[1,13,22],diff_ag:1,diffeent:8,differ:[0,1,3,4,5,7,9,10,12,13,14,15,16,17,21,22,23,24,25,26,28,29,34,36],differenti:[0,4,10,14,16,17,23,25,26,28],difficult:[0,3,5,7,13,14,17,22,23,26],difficulti:[0,2,7],diffonedim:1,digit:[0,4,7,10],dilut:7,dim:[1,9,10,15,24],dimens:[1,2,4,5,7,8,9,10,14,15,17,23,24,26],dimension:[0,2,4,10,12,14,15,16,21,23,24,25],dimensionless:[4,13,14,15,22,23,24],diment:[12,21],dimless:[13,22],dimnsion:[1,10],diod:[15,24],direct:[0,1,2,4,5,7,9,10,12,13,14,15,16,17,21,22,23,24,25,26],directli:[2,3,5,7,12,16,21,25,34,36],directori:31,disabl:[29,31],disadvantag:4,disappear:[15,16,24,25],discard:[2,5,9],disciplin:[0,4,7],disclaim:[0,3],disconnect:2,discours:[0,11,20],discov:0,discrep:[17,26],discret:[6,7,10,11,15,20,24],discrimin:[6,9],discuss:[0,1,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,20,21,22,23,24,25,26,28],discussiom:28,diseas:6,disjoint:2,disk:4,disord:6,displac:[14,15,17,23,24,26],displai:[1,2,3,4,5,7,8,9,10,12,16,21,25,34,36],displaystyl:5,disregard:4,disrupt:0,dissimilar:9,dissip:[13,22],dissipitav:[15,24],distanc:[3,5,8,9,13,14,16,17,22,23,25,26],distant:[16,25],distinct:[6,8,10],distinctli:8,distinguish:[3,4,8,9,13,22],distort:[16,17,25,26],distplot:5,distribuit:[12,21],distribut:[0,2,4,5,7,9,12,16,17,21,25,26],distrubut:[4,12,21],dive:[4,5,8,15,24],diverg:[2,7,16,25],divid:[0,3,4,5,7,8,9,10,12,13,15,16,21,22,24,25],divis:[3,5,8],dna:6,dnn1:[1,10],dnn2_gru2:[1,10],dnn:[1,4,7,10],dnn_kera:7,dnn_numpi:7,dnn_scikit:[4,7],dnn_tf:7,doc:[28,30,34,36],doconc:4,docstr:30,doctest:[11,20],document:[1,9,10,18,19,34,35,36],doe:[0,1,2,3,4,5,7,8,9,10,12,13,14,15,16,17,21,22,23,24,25,26],doesn:[2,7,10,13,14,22,23,34,36],dog:[7,10],doing:[2,4,5,9,13,22],dollar:[18,19,35],dom:2,domain:8,domin:[2,4,13,16,22,25],don:[2,3,4,5,7,8,9,12,13,16,17,21,22,25,26],done:[1,2,3,4,5,9,10,12,13,14,15,16,17,21,22,23,24,25,26],door:[15,24],dot:[0,1,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,20,21,22,23,24,25,26],doubl:[2,3,4,12,13,16,17,21,22,25,26,28],doubli:[7,14,17,23,26],down:[4,5,9,13,15,16,17,22,24,25,26],download:[4,7,10,12,21],downsampl:10,drag:[12,13,15,21,22,24],dramat:9,drastic:[1,10],draw:[5,13,22],drawback:[5,9],drawn:[2,3,4,5,6,7,9,12,21],dream:0,drive:[0,1,10,13,15,22,24],driven:[28,30],drivenosc:[15,24],driver:0,drop:[3,4,5,9,17,26],dropna:[4,5],dry:0,dtype:[1,4,5,7,10,12,21],dub:4,due:[1,5,7,8,9,12,13,14,15,16,17,21,22,23,24,25,26,28],dummi:5,dure:[0,4,7,8,9,10,15,17,24,26],dv_:[13,22],dv_x:[13,14,17,22,23,26],dv_y:[13,14,17,22,23,26],dv_z:[14,17,23,26],dwell:5,dwh:7,dwo:7,dx_1:3,dx_n:3,dxdy:[15,24],dying:7,e_0:[13,22],e_i:2,each:[0,1,2,3,4,5,6,7,8,9,10,12,14,15,16,17,21,23,24,25,26],eager:5,eapprox:[4,5],ear:0,earli:7,earlier:[2,5,6,7,8,9,14,15,23,24],earth:[13,16,17,22,25,26,28],earthsuneul:[14,23],earthsunvv:[14,23],easi:[0,4,5,6,7,8,9,12,13,14,15,16,21,22,23,24,25],easier:[0,3,4,5,8,16,25,28,30],easili:[0,1,4,5,6,7,8,9,11,12,13,14,15,20,21,22,23,24],east:[17,26],ebind:[4,5],eccentr:[16,25],ecolor:2,econom:0,econometr:[4,12,21],ecosystem:[4,12,21,34,36],edgecolor:5,edit:[4,29],edu:28,educ:[0,4,5,11,15,20,24,28],eff:[3,16,17,25,26],effect:[2,3,7,12,13,15,21,22,24,28],effic:7,effici:[3,4,10,12,13,21,22],efron:[3,5],eig:[3,4,5,9],eigen:3,eigenpair:[5,9],eigenvalu:[4,5,8,9,12,21],eigenvector:[5,9],eight:[4,12,21],eigval:[3,4],eigvalu:9,eigvec:[3,4],eigvector:9,eispack:[4,12,21],either:[0,3,4,5,6,7,8,9,13,14,15,16,17,22,23,24,25,26],elabor:3,elast:[16,25],elect:2,electr:[4,7,13,15,16,22,24,25],electrodynam:[16,25],electromagnet:[13,16,22,25],electron:[11,14,16,20,23,25,29],eleg:9,elel:[12,21],element:[0,1,4,5,6,7,8,9,10,11,13,14,20,22,23],elementari:[4,12,21,28],elementwis:10,elementwise_grad:1,elessar:[4,12,21],elimin:[8,15,16,24,25],ellips:[16,17,25,26],ellipt:[17,26,28],els:[3,6,7,13,22],elu:7,elus:0,email:28,emb:[18,19,35],embed:[5,9],embodi:5,emce:2,emit:[3,14,23],empathi:29,emphas:[0,2,4,15,24,28],emphasi:[4,12,21],empir:[3,7,9],emploi:[2,3,4,5,7,9,13,15,22,24],employ:[0,4,5],employe:0,empti:5,emul:7,enabl:[2,9,11,20],enbodi:[15,24],encapsul:[2,15,16,24,25],enclos:2,encod:[0,2,9,10],encompass:[0,3,4,13,22],encount:[0,3,5,6,7,16,25],end:[0,1,2,3,4,5,6,7,8,9,10,12,13,14,15,16,17,18,19,21,22,23,24,25,26,28,35],endl:3,endpoint:[3,5,15,24],energi:[1,10,12,13,15,16,17,21,22,24,25,26,28],enforc:7,engag:[11,20],engin:[0,3,4,7,12,13,21,22,28],enhanc:[11,20,28,30],enorm:10,enough:[0,5,13,16,17,22,25,26],enquiri:[13,22],ensambl:2,enscrib:[14,23],ensembl:[2,3,7],ensemblesampl:2,ensur:[1,2,3,5,7,9,10,15,16,24,25],entail:[2,4],enter:[0,2,3,5,15,17,24,26],enthought:[4,12,21],entir:[2,3,4,6,7,10,12,15,16,21,24,25,34,36],entiti:[4,7,12,21],entitl:0,entri:[4,5,7,8,9],entropi:[4,6,7,10],enumer:[1,4,5,7,8,10],env:3,environ:[1,4,11,12,13,20,21,22,29,31],eol:5,eosfit:5,epoch:[1,4,7,10],epsilon:[5,6,14,23],epsilon_0:[5,16,25],epsilon_1:5,epsilon_2:5,epsilon_:[4,5,13,14,22,23],epsilon_i:5,eqnarrai:[5,13,14,15,16,17,22,23,24,25,26],equador:[17,26],equal:[0,1,2,3,4,5,7,8,9,12,13,15,16,17,21,22,24,25,26],equat:[0,2,3,4,6,9,11,16,17,20,25,26,28],equatori:[17,26],equilibrium:[1,7,13,22],equiv:[2,3,14,15,16,17,23,24,25,26],equival:[2,4,5,7,8,9,12,14,15,16,21,23,24,25],erf:3,eriador:[4,12,21],eridg:5,err:4,err_:5,err_sqr:1,erron:1,error:[0,1,2,3,4,6,7,9,10,11,15,20,24],error_estimate_corr_tim:3,error_hidden:7,error_output:7,errorbar:2,escap:[18,19,35],especi:[7,14,15,16,23,24,25],essenc:2,essenti:[0,2,3,4,5,7,11,13,17,20,22,26],est:2,establish:9,estim:[0,3,4,5,6,7,14,23],estimated_mse_fold:5,estimated_mse_kfold:5,estimated_mse_sklearn:5,eta0:8,eta:[1,2,4,7,8,10,15,17,24,26],eta_v:[4,7,10],etc:[0,3,5,6,7,8,9,12,13,18,19,21,22,28,35],ethic:[11,20],ethnic:29,etsim:5,euclidean:[5,9],euler:[13,15,22,24,28],eulerintegr:[12,21],eurler:[12,21],european:0,evalu:[0,1,2,3,4,5,11,13,14,15,20,22,23,24,28],even:[0,2,3,4,5,7,8,9,10,12,13,14,15,16,17,21,22,23,24,25,26,30],evenli:[1,10],event:[0,2,6,29],eventson:5,eventu:[2,5,7,9,13,17,22,26],everest:[17,26],everi:[1,2,3,4,5,7,9,10,12,13,14,17,21,22,23,26,30],everybodi:[0,28],everyon:29,everyth:[2,7,13,16,22,25],evid:[2,13,22,34,36],evolut:[2,15,16,24,25],evolv:[0,2,4],exact:[3,4,5,7,9,11,12,13,14,15,20,21,22,23,24],exactli:[2,5,7,9,10,16,17,25,26],exam:28,examin:5,exampl:[0,6,13,15,17,18,19,22,24,26,28,29,34,35,36],exce:7,excel:[0,4,5,7,31],except:[1,3,5,8,10,13,17,22,26],excess:4,excit:[0,4,15,16,24,25],exclud:[2,5,7],exclus:[2,3,4,5,10],execut:[1,5],executablebook:31,exercis:[0,4,5,11,13,14,20,22,23,28],exert:[14,23],exhaust:[2,5],exhibit:[5,8],exist:[0,1,2,4,5,7,8,9,10,12,14,15,16,21,23,24,25,31,34,36],exit:[3,5],exp:[1,2,3,5,6,7,8,9,15,24],exp_term:7,expand:[5,6,9,12,15,21,24],expans:[2,5,7,8,14,15,16,23,24,25],expect:[0,2,4,7,9,13,15,16,17,22,24,25,26,29],expectation_value_of_h_wrt_p:3,expens:3,experi:[0,2,4,5,7,8,11,13,16,17,20,22,25,26,29],experiment:[1,2,3,4,5,10,13,16,22,25],experimentalist:[16,25],expert:7,explain:[3,4,5,9,17,26,30],explained_variance_ratio_:9,explanatori:5,explicit:[2,4,5,10,12,14,15,17,21,23,24,26,29],explicitli:[0,1,2,4,10,14,23],explod:7,exploit:[5,7,10],explor:[4,5,8,11,12,20,21],expon:7,exponenti:[4,7,13,15,22,24],expos:0,express:[1,2,3,4,6,7,10,12,13,15,16,17,21,22,24,25,26,29],exptmean:3,exptvari:3,ext:[15,24],extend:[0,1,6,9,11,12,20,21],extens:[0,4,5,7,14,16,23,25,34,36],extent:[0,4,5,28],extern:[5,13,14,15,16,17,22,23,24,25,26],extra:[5,7,10,15,24,28],extract:[0,4,5,6,8,9,13,15,16,22,24,25],extrapol:[4,5],extrem:[0,1,2,4,5,6,7,8,9,10,11,12,20,21],extrins:9,eye:[0,4],f11:4,f12:4,f13:4,f6d7a289d493:[4,12,21],f_0:[14,15,23,24],f_0b:[14,23],f_0e:[15,24],f_2:7,f_3:7,f_d:3,f_est:2,f_g:[14,23],f_h:[14,23],f_i:[2,5,7,13,15,22,24],f_m:[15,24],f_n:[15,24],f_r:[16,25],f_true:2,f_vec:1,f_x:[13,14,16,22,23,25],f_xv_x:[14,23],f_y:[13,16,22,25],f_yv_i:[14,23],f_zv_z:[14,23],face:[2,29],facecolor:[3,5,8],facil:0,facilit:[2,7],fact:[0,2,4,5,7,9,10,12,14,15,16,17,21,23,24,25,26],factor:[2,3,4,5,7,9,13,14,16,22,23,25],fade:5,fail:[5,8,9,16,25],failur:6,fair:29,fairli:[1,2,3],faith:29,fall:[8,16,25,28],fallingbasebal:[12,21],fals:[1,2,4,5,6,7,10,15,24],falsiti:2,famili:[0,3,4,6,8,9,12,15,21,24,28],familiar:[0,2,3,4,8,12,14,16,21,23,25],famou:[0,5,7],fanci:0,far:[0,1,2,3,4,5,7,8,9,10,13,16,22,25],fashion:[0,11,20],fast:[3,4,5,7,10,12,17,21,26],fastdriven1:[15,24],fastdriven2:[15,24],faster:9,favor:6,favorit:3,fc1:7,fc2:7,featur:[3,6,7,10,15,24],feature_nam:[5,9],feb:28,februari:[15,24],fed:[0,7],feed:[0,1,4,9,10,13,22],feed_dict:7,feed_forward:7,feed_forward_out:7,feed_forward_train:7,feeddorward:[1,10],feedforward:[1,7,10],feel:[2,4,9,12,14,15,17,21,23,24,26,33],feet:[5,17,26],fellow:0,felt:[13,22],fetch:2,few:[3,4,7,10,27,32],fewer:[4,9],ffedbda27704:[15,24],ffnn:7,fictiti:[17,26],field:[0,4,5,7,14,15,16,17,23,24,25,26,28],fifth:[4,12,21],fig:[1,2,4,5,6,7,9,10,12,14,15,16,18,19,21,23,24,25,35],fig_id:[4,5,9,12,14,15,16,21,23,24,25],figaxi:3,figsiz:[1,4,5,7,8,9,10,12,18,19,21,35],figur:[1,2,4,5,6,7,8,9,10,12,13,14,15,16,17,21,22,23,24,25,26],figure_id:[4,5,9,12,14,15,16,21,23,24,25],figurefil:[4,5,9,12,14,15,16,21,23,24,25],file:[1,3,4,5,7,8,9,12,14,15,16,21,23,24,25,28,30],filenam:[3,4,5,12,21],fileout:3,filewrit:7,fill:5,filter:10,financ:[0,4],find:[0,1,2,3,4,6,7,8,9,11,13,14,15,16,17,20,22,23,24,25,26,28],fine:7,finish:[1,13,22],finit:[3,5,7,16,25],first:[0,1,2,3,5,6,9,10,11,14,15,16,17,20,23,24,25,26,28,34,36],firsteigvector:9,fisher:2,fit:[0,1,2,3,6,7,8,9,10,13,22],fit_intercept:5,fit_transform:[4,5,8,9],fiti:4,five:[2,4,5],fix:[1,2,4,7,10,13,14,15,16,17,18,19,22,23,24,25,26,35],flat:[2,7],flatten:[1,2,7,10],flavor:[34,36],flexibl:[4,5,7,8,11,12,15,20,21,24],flip:[2,16,25],float32:7,float64:[1,4,5,10,12,21],floor:[12,21],flop:5,flow:[1,7,10,13,17,22,26],fly:[9,16,25],flyvbjerg:3,fmax:[15,24],fmt:2,focault:[17,26],foci:[16,25],focoault:[17,26],focu:[0,1,2,4,5,10,12,13,16,17,21,22,25,26],focus:[6,7,15,24,29],folder:[4,12,21,28],folk:0,follow:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,20,21,22,23,24,25,26,29,34,36],font:[3,4,12,15,21,24],fontdict:3,fontsiz:[3,5,8],footbal:[16,25],foral:8,forc:[5,9,11,12,14,20,21,23,28],forcast:[1,10],forcedblockeulercrom:[15,24],forcedblockrk4:[15,24],forecast:[1,7,10],foreman:2,forest:[0,4,7],forget:9,fork:30,form:[0,1,2,3,4,5,6,7,8,9,10,12,13,14,15,16,17,21,22,23,24,25,26],formal:[2,3,10,12,14,17,21,23,26,28],formalim:28,format:[2,3,4,5,8,9,11,12,14,15,16,20,21,23,24,25],format_data:[1,10],former:2,formul:[2,5,9,13,22],formula:[3,12,14,15,16,21,23,24,25],forth:[7,16,17,25,26],fortran90:3,fortran:[0,4,12,21],fortun:[5,9],forum:28,forward:[4,5,10,12,13,14,15,17,21,22,23,24,26],forwardeul:[15,24],forwardeulercrom:[15,24],foster:29,foucalt:28,foucault:28,found:[1,5,7,9,13,15,16,22,24,25,31],foundat:[11,20],four:[1,7,8,10,14,17,23,26],fourier:[5,28],fourierdef1:[15,24],fourierdef2:[15,24],fourierseriessign:[15,24],fouriersolut:[15,24],fourpi2:[14,23],fourth:[4,7,12,15,21,24],frac:[1,2,3,4,5,6,7,8,9,11,12,13,14,15,16,17,20,21,22,23,24,25,26],frame:[9,13,16,22,25,28],framework:[0,2,3,7,8],frankefunct:[5,9],free:[3,4,9,12,21,29],freedom:[11,13,15,16,20,22,24,25],freeli:[0,4],freewai:[13,22],frequenc:[2,3,5,9,12,13,15,16,17,21,22,24,25,26],frequent:[0,2,4,8],frib:28,friction:[13,15,22,24,28],frictionless:[13,22],fridai:28,friendli:[1,10,15,24],frodo:[4,12,21],frog:10,from:[0,1,3,4,6,8,9,11,12,13,14,15,16,18,19,20,21,22,23,24,25,28,29,35],from_logit:10,front:[0,1,4,10],frontier:[34,36],frydai:28,fstream:3,ftild:[15,24],ftrue:2,fuel:[13,22],fulfil:[1,5,7,13,22],full:[0,3,4,5,6,12,21],fulli:[3,5,7,10,31],fun:[4,12,21],func:[1,5],functional_1:1,functional_3:1,functional_5:1,functional_7:1,functionali:9,fundament:[0,2,5,13,15,22,24],funtion:1,further:[1,2,4,10,11,12,13,14,16,17,20,21,22,23,25,26,28,29],furthermor:[2,4,5,6,7,9,11,12,13,20,21,22],furthest:[17,26],futur:[0,1,4,8,10,13,15,22,24],futurist:0,fwhm:[15,24],g_0:1,g_1:1,g_2:1,g_analyt:1,g_dnn_ag:1,g_euler:1,g_i:1,g_m:[15,24],g_n:[15,24],g_re:1,g_t:1,g_t_d2t:1,g_t_d2x:1,g_t_dt:1,g_t_hessian:1,g_t_hessian_func:1,g_t_jacobian:1,g_t_jacobian_func:1,g_trial:1,g_trial_deep:1,g_vec:1,gain:[0,5,7,11,13,14,15,20,22,23,24,28],galaxi:[13,22],galleri:[0,4],gallon:[16,25],gamge:[4,12,21],gamma1:8,gamma2:8,gamma:[1,5,8,9,13,15,16,17,22,24,25,26],gamma_:5,gamma_i:[3,5,8],gamma_x:5,gap:8,gate:[1,7,10],gather:7,gaug:7,gauss:[16,25],gaussian:[2,5,8,15,24],gaussian_rbf:8,gca:[1,5,8],gen:3,gender:[5,29],genener:[1,10],gener:[0,1,2,4,7,8,9,10,11,12,13,14,15,16,17,20,21,22,23,24,25,26,28],generallay:7,genet:[0,7],geodes:[9,17,26],geometr:[4,13,14,22,23],geq:[1,5,8],get:[0,1,2,3,5,6,7,9,10,11,12,13,14,15,16,17,20,21,22,23,24,25,26,28,33],get_default_graph:7,get_paramet:1,get_yaxi:8,get_yticklabel:5,getattr:5,ggplot:[12,21],ghp:31,gibb:0,git:[0,28],github:[0,4,28,30,31],gitlab:0,give:[0,1,2,3,4,5,6,7,8,12,13,14,15,16,17,21,22,23,24,25,26,28,33],given:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,20,21,22,23,24,25,26,30],global:[2,3,6,14,15,23,24],global_step:7,global_variables_initi:7,glorot:7,gm_1m_2:[16,25],gm_:[14,16,23,25],gm_am_b:[16,25],gmm:[16,17,25,26],goal:[0,2,4,5,6,13,22],goe:[1,4,5,7,14,15,16,23,24,25,28],going:[2,4,7,8,9,10,13,17,22,26],gold:[16,25],gone:5,gong:7,good:[2,3,4,5,7,9,10,11,13,15,20,22,24,28,29],googl:[4,7,12,21],got:7,govern:[11,13,20,22],gracefulli:29,grad:1,grader:28,gradient:[0,4,5,6,8,10,17,26],gradientdescentoptim:7,gradual:[13,15,22,24],grai:2,graph:[0,7,9],graphic:[4,7,11,13,20,22],grasp:[0,4],gravgauss:[16,25],gravit:[7,12,13,14,15,17,21,22,23,24,26,28],graviti:[12,16,17,21,25,26,28],gray_r:[7,10],grayscal:10,graze:[16,25],great:[13,22],greater:[3,7,9],greatli:[2,30],green:[3,4,10,15,24],greeni:[15,24],greeny2:[15,24],grid:[3,5,6,7,8,10,13,16,22,25],ground:[4,12,17,21,26],group:[0,4,5],groupbi:[4,5],grow:[2,7,10],growth:4,gru:[1,10],guarante:[0,3,5,9],guess:[2,7,13,15,17,22,24,26],guid:[0,7,18,19,35],h_1:[1,2],h_2:1,h_i:2,habit:[11,20],had:[0,2,4,5,7,13,14,16,22,23,25],hadamard:7,half:[7,8,16,25],hamiltonian:[14,23],hand:[0,1,2,3,4,5,7,9,10,11,12,13,16,17,20,21,22,25,26,28],handi:[10,14,17,23,26],handl:[1,2,7,9],handsid:7,handwrit:7,handwritten:7,handwrittennot:28,happen:[1,2,3,7,10,13,16,22,25],harass:29,hard:[6,7,8],hardcopi:[4,12,21],harder:[5,7,9],harm:29,harmon:[13,14,22,23,28],has:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,20,21,22,23,24,25,26,28,31,34,36],hasn:7,hassl:[4,12,21],hast:0,hat:[3,4,5,6,7,9,12,13,14,15,16,17,21,22,23,24,25,26],have:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,20,21,22,23,24,25,26,28,29],haven:7,hbar:[14,23],hdf5:4,hdhpk14:[34,36],head:[1,2,3,5,10],header:[4,5],hear:[0,4],heart:[0,2,4],heatmap:[4,5,7,9,10],heavi:[11,16,20,25],heavili:[0,4],heavisid:7,heer:[34,36],height:[7,10,13,14,16,17,22,23,25,26],helic:[13,22],help:[0,1,2,7,10,11,16,20,25,30],hemispher:[17,26],henc:[4,5,7,8,13,22],her:6,here:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,28,30,31,34,35,36],hereaft:[5,7,8,14,17,23,26],hermitian:[4,12,21],hessenberg:[4,12,21],hessian:1,hhint:[14,23],hidden:[7,10],hidden_bia:7,hidden_bias_gradi:7,hidden_layer_s:[4,7],hidden_neuron:[1,10],hidden_weight:7,hidden_weights_gradi:7,high:[0,1,4,5,7,9,10,12,15,17,21,24,26],higher:[0,2,4,5,7,8,9,12,13,14,15,16,17,21,22,23,24,25,26],highest:[1,7],highli:[3,4,5,10,12,21],highwai:5,him:2,hinder:0,hing:8,hire:[0,4],his:[2,6],hist:[1,2,3,5,9,10],histogram:[2,3,5,9],histor:[6,9],histori:[1,7,10],histtyp:[2,5],hit:[0,12,13,16,21,22,25],hjensen:[7,8,9,28],hjorth:[2,28],hobbi:3,hoc:5,hold:[3,5,7,10,13,15,17,22,24,26],holder:5,holdgraf:[34,36],holdgraf_evidence_2014:[34,36],home:[5,28],homework:[14,16,23,25,28],homogen:[7,13,15,17,22,24,26],homogn:[13,22],homogsolut:[15,24],honchar:1,honest:2,hope:[0,11,20],hopefulli:[3,5,9,13,22],horizont:[9,13,17,22,26],hors:[6,10],hot:[7,18,19,35],hour:[3,7,17,26,28],how:[0,1,2,4,6,7,9,10,11,12,13,14,15,16,17,20,21,22,23,24,25,26,27,28,30,32,33],howev:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,20,21,22,23,24,25,26],hspace:[3,4,8,12,13,15,21,22,24],htm:28,html:[4,9,11,18,19,20,28,31,35],http:[1,2,4,7,9,10,12,21,28],huang:4,huber:4,huge:[7,10],human:[0,4,7,34,36],hundr:7,hungri:7,hurrican:[17,26],hv_:[14,23],hw10:28,hybrid:2,hydrogen:4,hyperbol:[1,10],hyperbola:[16,25],hyperparam:8,hyperparamet:[5,10],hyperplan:9,hypothes:2,hypothesi:2,i_1:5,i_2:5,i_c:[15,24],i_i:[15,24],idea:[0,1,2,3,4,5,7,10],ideal:[0,1,3,4,5,8,11,13,15,20,22,24],idem:5,ident:[5,7,15,24,29],identifi:[0,4,6,7,8,9,11,16,20,25],idum:3,ieor:3,iet:[15,24],ifs:[4,12,21],ignor:[2,5,9,10,12,14,16,17,21,23,25,26],iii:[4,12,21],ijk:[13,14,22,23],illustr:[0,7,13,16,17,22,25,26],imag:[5,7,9,18,19,35],image_height:10,image_path:[4,5,9,12,14,15,16,21,23,24,25],image_width:10,imageri:29,imagin:[2,13,14,22,23],imaginari:[15,17,24,26],immedi:[0,1,4,5,10,15,24,28],impact:[0,11,16,20,25],impart:[13,22],implement:[4,5,8,9,10,11,13,14,15,20,22,23,24,28],impli:[2,4,5,10,11,12,15,20,21,24],implic:[13,22],implicit:[2,10],implicitli:[2,3,9],impos:[0,5,7,9],imposs:[0,5],impress:[0,2,4,7],improv:[0,1,5,9,10,13,14,15,22,23,24],impuls:[15,24],imshow:[5,7,10],in5400:[1,10],in_out_neuron:[1,10],inact:7,inadequ:0,inappropri:29,incid:[16,25,29],inclin:2,includ:[1,2,3,4,5,7,9,10,11,13,15,16,17,18,19,20,22,24,25,26,29,30,34,35,36],include_bia:5,inclus:[2,11,20,29],incom:[7,16,25],inconsist:[15,24],incorpor:[15,17,24,26],incorrect:7,incoveni:8,increas:[0,1,2,3,4,5,7,8,9,10,11,14,15,16,20,23,24,25],increasingli:[2,3],incred:0,increment:[11,20],ind:5,inde:[0,1,4,5,13,14,15,16,22,23,24,25],indent:[3,15,24],indentationerror:3,independ:[2,3,5,6,7,8,12,13,14,15,16,17,21,22,23,24,25,26,28],index:[1,3,4,5,7,10,12,21],index_col:[4,5],indic:[2,4,5,7,9,10,12,13,21,22],indispens:5,individu:[3,6,7,15,16,24,25,29],indu:5,induct:2,industri:[11,20],indx1:1,indx2:1,indx3:1,inelast:[16,25],inequ:[0,8],inerti:[13,22,28],inertia:[13,22],inf:5,infam:0,infer:[0,5,7,16,25],infil:[4,5,12,21],infin:[5,9,15,24],infinit:[0,2,13,15,16,22,24,25],infinitesim:[2,3,15,24],influenc:[2,5,12,13,16,21,22,25],influenti:7,info:[4,7,12,21],inform:[0,1,2,4,5,7,9,10,14,16,18,19,23,25,29,31,34,35,36],infti:[3,13,15,22,24],ingredi:[0,4],inher:5,inherit:[4,12,21],init:[34,36],initi:[1,3,4,5,7,12,13,14,15,16,17,21,22,23,24,25,26],initialis:3,inlin:[1,3,4,5,6,7,8,9,10,11,12,14,15,16,20,21,23,24,25],inner:5,inoccu:0,inp:[1,10],input:[0,2,3,4,5,6,7,8,9,10,11,12,14,15,20,21,23,24,34,36],input_1:1,input_2:1,input_3:1,input_4:1,input_shap:[1,10],inputlay:1,inputs_shuffl:7,insert:[3,7,8,12,14,15,16,17,21,23,24,25,26,34,36],insid:[1,5,9,10,16,25,34,36],insight:[0,4,5,7,11,13,20,22,28],inspect:[13,15,16,22,24,25],inspir:[0,4,7],instabl:1,instal:[7,28,30,31],instanc:[0,1,5,7,9,13,22,29],instant:[14,23],instantan:[12,21],instead:[0,1,2,3,4,5,7,8,9,10,11,12,13,14,16,20,21,22,23,25],institut:[0,7],instruct:[0,2,7],instructor:28,insult:29,int32:7,int64:5,int_0:[2,3,11,12,13,14,15,17,20,21,22,23,24,26],int_:[2,3,12,13,14,15,21,22,23,24],int_a:[3,11,14,20,23],integ:[1,3,4,7,12,16,21,25],integer_vector:7,integr:[0,3,4,11,13,15,16,20,22,24,25,28],integraldef:[14,23],integrand:[15,24],intellig:[0,4],intens:[7,11,20],interact:[4,5,7,12,13,15,16,18,19,21,22,24,25,35],intercept:[4,5,8],intercept_:[4,5,8],interchang:7,interest:[0,1,2,3,4,5,6,7,8,9,10,12,14,15,17,21,23,24,26,29],interfac:[4,7,11,20],interior:4,intermedi:[2,15,24],intern:[7,13,14,16,22,23,25,34,36],internet:[17,26],interpol:[7,10,14,23],interpr:5,interpret:[0,2,3,4,7,12,13,14,21,22,23],interv:[3,5,6,11,13,14,15,20,22,23,24],intimid:4,intract:[2,5,9],intrins:[2,3,4,9,10,12,21],intro:[34,36],introduc:[0,3,4,7,8,11,12,13,14,15,17,20,21,22,23,24,26,28],introduct:[1,5,7,10,28],introductori:[4,5],intuit:[2,4,5,7,8],inv:5,invalid:[4,7,8,9,12,21],invalu:[4,11,12,20,21],invari:[7,14,23],invd:5,invent:2,inver:8,invers:[4,5,9,12,13,15,16,21,22,24,25],invers_period:3,inverse_transform:8,invert:[5,6,13,14,22,23],investig:[11,20,29],invis:[16,25],invok:[5,8,13,22],involv:[0,1,2,5,6,7,9,12,13,15,16,21,22,24,25],inward:[17,26],iomanip:3,ion:[18,19,35],ios:3,iostream:3,ipca:9,ipynb:[4,12,21,34,36],ipython:[2,3,4,5,7,8,9,12,15,21,24],iri:8,irreduc:5,irrelev:[5,15,24],irrespect:[0,2,4],ising:7,ismov:[13,22],isnul:5,isol:[12,14,21,23],isomap:9,isotrop:[17,26],issu:[0,7,29,30],item:[0,4,5],items:[4,12,21],iter:[1,3,5,7,8,9,10,13,22],ith:2,its:[0,1,2,4,5,6,7,8,9,10,11,12,13,14,15,16,17,20,21,22,23,24,25,26,29,30,34,36],itself:[2,3,5,7,13,14,15,16,22,23,24,25],j_lasso_sk:5,j_ridge_sk:5,j_sk:5,jackknavg:3,jackknif:0,jackknstd:3,jackknvar:3,jackknvec:3,jake:2,januari:9,jargon:3,jeffrei:2,jensen:[2,28],jeremi:28,job:[1,8,13,22],john:[2,28],join:[4,5,9,12,14,15,16,21,23,24,25],joul:[12,21],journal:3,jrt:28,judgment:2,juli:28,jump:3,jupyt:[0,4,11,12,18,19,20,21,31,34,35,36],jupytext:[34,36],just:[0,1,2,3,4,5,6,7,8,9,10,12,13,14,15,16,21,22,23,24,25,28],justif:[0,4],k1v:[15,24],k1x:[15,24],k2v:[15,24],k2x:[15,24],k3v:[15,24],k3x:[15,24],k4v:[15,24],k4x:[15,24],k_1:[15,24],k_2:[15,24],k_3:[15,24],k_4:[15,24],kappa_d:3,karush:8,keep:[0,2,4,5,7,9,11,12,13,14,15,18,19,20,21,22,23,24,30,35],keepdim:[5,7],kei:[2,5,7,15,24],kepler:[16,25,28],kept:[1,10,15,24],kera:[0,1,4,12,21],kernel:[5,7,34,36],kernel_regular:[7,10],kernelpca:9,kev:[4,5],keyword:[4,12,21],kfold:5,kick:7,kiener:1,kilogram:[12,21],kind:[0,1,5,7,8,10,13,16,22,25,34,36],kindergarten:0,kinet:[12,13,14,15,16,21,22,23,24,25],kkt:8,knight:[34,36],know:[0,1,2,3,4,5,8,12,13,15,16,17,21,22,24,25,26,28],knowledg:[0,2,4,5,11,20,28],known:[0,1,2,3,5,6,7,8,9,10,12,13,14,15,17,21,22,23,24,26,28],kondev:4,kpca:9,kr_:[16,25],krh:[16,25],kuhn:8,kutta2:[15,24],kutta4:[15,24],kutta:28,kwarg:5,kwown:5,l1_l2:10,l2_loss:7,l_1:6,l_2:6,l_j:7,l_z:[14,23],la_:[18,19,35],la_i:7,la_k:7,lab:[0,16,25],label:[1,2,3,4,5,6,7,8,10,13,14,15,16,17,22,23,24,25,26],labels:[5,8],labels_shuffl:7,laboratori:[2,13,16,17,22,25,26],lack:[0,2,4],lad:0,lagari:1,lagrang:[8,9,14,23,28],lagrangian:[14,23,28],lai:[2,11,20],lambda:[1,3,4,6,7,8,10],lambda_0:9,lambda_1:[5,8,9],lambda_2:[8,9],lambda_:9,lambda_i:[8,9],lambda_iy_i:8,lambda_jy_iy_j:8,lambda_k:8,lambda_n:[5,8],land:[5,8],landmark:8,langl:[2,3,5,9],langrangian:28,languag:[1,4,7,8,10,12,21,29,34,36],lapack:[4,12,21],laplac:2,larg:[0,1,2,3,4,5,7,8,9,11,12,13,15,16,20,21,22,24,25,28],larger:[2,3,4,5,8,9,10,11,15,20,24],largest:[1,8,9,10],lass:0,lasso:6,lasso_sk:5,last:[0,1,2,3,4,5,7,10,12,13,14,15,16,17,21,22,23,24,25,26],later:[0,1,2,3,4,5,6,7,8,10,12,13,14,15,16,21,22,23,24,25],latest:28,latex:[4,11,12,20,21],latitud:[17,26],latter:[0,3,4,5,6,8,9,12,13,14,15,16,17,21,22,23,24,25,26],lattic:7,launch:0,law:[0,2,4,7,11,12,15,16,17,20,21,24,25,26,28],layer:4,lbfg:9,lbl:5,lcc:5,lda:9,ldot:[2,3,5,9],lead:[0,3,4,5,6,7,8,9,10,12,13,14,15,16,17,21,22,23,24,25,26],leader:0,leadership:29,leaki:7,learn:[6,8,10,12,21],learnabl:10,learning_r:[7,8],learning_rate_init:[4,7],least:[0,2,3,4,6,8,9,12,13,21,22],leav:[2,5,7,9,10,14,16,23,25],lectur:[0,1,4,5,7,9,12,15,17,21,24,26],lecturenot:30,led:2,left:[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,20,21,22,23,24,25,26],leftarrow:[7,8],legend:[1,4,5,8,9,10,12,18,19,21,35],len:[1,2,3,4,5,7,8,9,10,12,21],length:[1,3,4,5,7,8,9,10,12,13,14,15,21,22,23,24],length_of_sequ:[1,10],lengthi:[17,26],leq:[3,4,5,6,8],less:[0,3,4,5,7,8,11,12,15,16,17,20,21,24,25,26,34,36],let:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,20,21,22,23,24,25,26],letter:[3,4,5,12,21],level:[4,5,7,11,12,17,20,21,26,29],levi:[14,23],lhs:[12,21],lib:[5,7,8,9],liberti:2,liblinear:8,librari:[0,1,3,5,7,9,10,11,20],licens:[2,4,7,12,21],lie:[2,3,5,9],lies:[2,8,9,11,20],life:[0,2,4,7,8,11,17,20,26],light:[2,4,13,22],lightgrai:[16,25],lighthous:2,like:[0,1,2,3,4,5,6,7,9,10,11,12,13,14,15,16,17,20,21,22,23,24,25,26,28,31,34,36],likelihood:[0,2,5,7],lim_:[3,12,21],limit:[2,5,8,9,11,12,13,14,15,17,20,21,22,23,24,26,28],lin_clf:8,lin_model:5,linalg:[1,3,4,5,8,9],line1:8,line2:8,line2d:[18,19,35],line3:8,line:[0,2,3,4,5,7,8,9,10,11,12,15,17,18,19,20,21,24,26,34,35,36],linear:[0,2,3,6,7,9,10,12,13,15,21,22,24,28],linear_model:[4,5,8,9],linear_regress:5,linearli:[5,9,13,15,22,24],linearregress:[4,5],linearsvc:8,liner:[7,10],linestyl:[16,25],linewidth:[1,2,4,5,8,10],link:[1,4,7,10,17,26],linpack:[4,12,21],linreg:4,linspac:[1,2,3,4,5,8,15,16,18,19,24,25,35],linux:[4,7,12,21],liquid:[4,5],list:[1,2,4,5,10,12,15,21,24,30,31],listcomp:5,literatur:[6,7,16,25,28],littl:[7,10,12,16,21,25,30],live:8,lle:5,lmb:[1,5],lmbd:[4,7,10],lmbd_val:[4,7,10],load:[4,5],load_boston:5,load_breast_canc:9,load_data:10,load_digit:[7,10],load_iri:8,loc:[4,5,8,9,10,12,21],local:[2,5,6,7,9,10,14,15,16,17,23,24,25,26,30,31],locat:[1,2,8,10,16,25,28],log10:[5,11,20],log:[1,2,4,5,6,7,9,12,21],log_:4,log_likelihood:2,log_posterior:2,log_prior:2,logarithm:[2,4,6,12,21],logic:[0,2,4],logist:[1,4,5,8],logisticregress:9,logit:[6,7],logreg:9,logspac:[4,5,7,10,18,19,35],longer:[1,3,4,8,10,12,13,14,15,16,21,22,23,24,25],loocv:5,look:[0,1,2,3,4,5,7,8,9,10,12,13,14,16,17,21,22,23,25,26,30,34,36],loop:[3,4,5,7,12,13,14,21,22,23],loos:[17,26],lorentz:[17,26],lose:7,loss:[0,1,4,5,6,7,8,9,10,12,13,14,21,22,23],lost:[2,15,24],lot:[5,11,18,19,20,35],love:[13,15,22,24],low:[2,3,5,9,16,17,25,26],lower:[2,4,5,7,10,12,14,15,16,17,21,23,24,25,26],lowercas:[4,12,21],lowest:3,lstat:5,lstm:[1,10],lstm_2layer:[1,10],lstsq:5,luckili:1,lvert:7,m_0:[14,23],m_1:[2,16,25],m_1m_2:[16,25],m_2:[2,16,25],m_e:[14,23],m_h:4,m_i:[2,14,23],m_iv_i:[14,23],m_l:7,m_n:4,m_p:4,machin:[1,5,6,7,10,12,21],mackei:2,made:[0,1,2,4,6,7,9,10,11,14,15,20,23,24],mae:4,magnitud:[2,7,9,12,13,14,15,16,17,21,22,23,24,25,26],mai:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,17,20,21,22,23,24,26,29,31],mail:[28,29],main:[0,4,5,6,10,15,24,31],mainli:[2,4,5,6,13,15,22,24],maintain:[5,13,15,22,24,29],major:[4,5,7,11,12,20,21],make:[0,1,2,3,4,5,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,28,29,30,35],make_axes_locat:5,make_moon:8,make_pipelin:[4,5],makedir:[4,5,9,12,14,15,16,21,23,24,25],makeplot:[4,12,21],malign:9,malth:28,manag:[1,4,7,10,12,21],mani:[0,1,2,3,4,5,6,8,9,10,12,13,15,16,17,21,22,24,25,26,27,28,32,34,36],manifestli:[14,23],manifold:9,manner:[10,17,26],manual:[11,20,31],map:[0,1,3,4,5,6,7,8,9],margin:[0,2,4,8,14,23],marit:5,mark:[4,12,21],markedli:[34,36],marker:[4,5],market:0,markov:[0,2],markup:[34,36],marsaglia:3,mask:3,mass:[2,5,7,12,13,14,15,17,21,22,23,24,26,28],massag:4,masses2016:4,masses2016ol:5,masses2016tre:4,masseval2016:[4,5,12,21],massiv:[17,26],master:[2,28],match:[7,16,25],materi:[0,4,5,6,7,9,10,11,12,13,14,15,17,20,21,22,23,24,26],math:[3,4,6,7,11,12,14,15,16,18,19,20,21,23,24,25,35],mathbb:[3,5,6,7,8,9],mathbf:[3,4,5,6,8,12,21],mathcal:[2,5,6,7],mathemat:[0,3,4,9,11,13,15,20,22,24,28],mathemati:[4,12,21],mathemt:5,mathrm:[2,3,4,5,6,7,8,9,12,13,14,15,17,21,22,23,24,26],matmul:[1,5,7],matplotlib:[1,3,4,5,6,7,8,9,10,11,12,14,15,16,18,19,20,21,23,24,25,35],matric:[1,5,6,8,9,10],matrix:[1,6,8,10],matter:[1,9,10],matur:[11,20],max:[1,4,5,7,9,10,14,15,16,23,24,25],max_depth:4,max_diff1:1,max_diff2:1,max_diff:1,max_it:[4,7,8,9],maxdegre:5,maxim:[2,6,7,8,9],maxima:[2,16,25],maximum:[1,2,4,5,7,8,9,10,13,14,15,16,22,23,24,25],maxpolydegre:5,maxpooling2d:10,mayb:2,mbox:[5,18,19,35],mcculloch:7,mcint:3,mcintsqr2:3,mcmc:2,mdoel:[1,10],mean:[0,1,2,4,5,6,7,10,11,12,13,14,15,17,18,19,20,21,22,23,24,26,28,35],mean_absolute_error:[4,5],mean_i:3,mean_squared_error:[1,4,5,9,10],mean_squared_log_error:4,mean_x:3,meaning:0,meansquarederror:4,meant:[0,6,11,16,20,25,28],meantempvec:3,meanvec:3,measur:[1,3,4,5,7,9,12,13,15,16,17,21,22,24,25,26],mechan:[0,1,2,3,10,11,13,14,15,17,20,22,23,24,26,36],mechanist:0,media:[28,29],median:[2,4,5,9],mediaspac:28,medicin:[0,7],medium:[1,8,10,13,15,18,19,22,24,35],medv:5,meet:[11,20],mehta:5,member:29,memori:[1,7,9,10,15,24],mention:[2,3,4,7,13,15,16,22,24,25],mere:[2,16,25],merg:2,mersienn:3,mesh:[12,13,14,21,22,23],meshgrid:[1,5,8,9],messi:[1,15,16,24,25],met:[0,2,4,5,8,9,14,23],metadata:[34,36],meter:[12,16,21,25],method:[0,1,2,4,6,7,8,9,10,11,13,16,20,22,25,28],methodolog:[11,20],metric:[4,5,7,9,10],metropoli:0,mev:[3,4,5],mgh:[16,17,25,26],mgl:[15,24],mglearn:[4,12,21],mgsin:[13,15,22,24],mgx:[17,26],mgy:[17,26],mhjensen:28,michael:10,michigan:[2,36],mid:[7,9,14,23],midel:[1,10],midpoint:[14,15,23,24,28],midterm:28,might:[1,2,5,7,9,13,14,16,22,23,25,30],mileag:[13,22],miller:3,million:[4,5,9],mimic:7,min:[1,5,8,9,16,25],min_:[1,5],mind:[0,2,4,5],mindboard:[1,10],mine:[4,12,21],mini:[7,9],minibatch:[7,9],minim:[0,3,4,5,6,7,9,15,16,24,25],minima:[4,6,7,16,25],minimum:[1,5,7,8,9,15,16,24,25],minmaxscal:[5,9],minneapoli:[17,26],minor:3,minut:[16,25],misclassif:8,misclassifi:8,mismatch:7,miss:[2,5],mix:[1,7,12,13,15,21,22,24],mkdir:[4,5,9,12,14,15,16,21,23,24,25],mlab:[3,5],mle:[3,6],mlk:28,mlpclassifi:7,mlpregressor:4,mnist:[7,9],mod:3,mode:[3,15,24],model:[1,3,6,8,9,11,16,20,25],model_select:[4,5,7,9,10],modern:[0,4,5,6,11,20],modif:[1,7,16,25],modifi:[2,4,5,6,7,8,12,17,21,26],modul:[2,4,5,7,9,11,12,20,21],modular:[3,11,20],modulo:3,moe:[5,9],molecul:[13,22],molten:[17,26],moment:5,momenta:[12,16,21,25],momentum:[12,13,15,16,21,22,24,25,28],mondai:28,monoton:[3,7],monstertruck:0,mont:[0,2,3,4,5],montecarlocycl:3,moon:[16,25],moor:5,more:[0,2,3,7,11,14,16,17,18,19,20,23,25,26,28,30,31,34,35,36],moreoev:36,moreov:[4,10,34],morten:[2,28],most:[0,2,3,4,5,6,7,8,9,10,11,12,13,14,16,17,20,21,22,23,25,26,34,36],mostli:[7,9],motion:[0,4,11,14,15,17,20,23,24,26,28],motiv:[7,13,15,22,24],mount:[17,26],move:[2,3,4,5,6,7,13,14,16,17,22,23,25,26,28],mpatch:[12,21],mph:[13,22],mpl:[4,12,15,21,24],mpl_toolkit:[1,5],mplot3d:1,mplregressor:7,mr_:[16,25],mrv_:[16,25],mse:[1,4,5,9,10],msle:4,msu:28,mt19937_64:3,mu0:3,mu1:3,mu2:3,mu_:3,mu_k:[13,22],mu_n:9,mu_x:3,much:[0,1,2,3,4,5,7,8,9,10,11,12,13,14,15,16,17,20,21,22,23,24,25,26],multi:[2,4,5,6,10,12,21],multiclass:[6,7],multidimension:[2,4,7,9,12,21],multimod:2,multinomi:6,multipl:[1,3,5,6,10,12,13,21,22],multipli:[3,5,9,10,12,13,14,17,21,22,23,26,28],multiplum:8,multitud:4,multivari:[0,1,3,4,9],multivariate_norm:9,murphi:9,must:[1,2,3,5,7,8,10,13,15,16,17,22,24,25,26,34,36],mutat:6,mutual:[5,10],mv_0:[14,23],mv_:[16,25],mv_e:[14,23],mvn:3,mx_:3,mydirectivenam:[34,36],myriad:[4,12,21],mz1:3,mz2:3,n_0:[3,7],n_b:3,n_boostrap:5,n_bootstrap:5,n_categori:[7,10],n_compon:9,n_featur:7,n_filter:10,n_hidden:1,n_hidden_neuron:[4,7],n_i:3,n_input:[7,10],n_iter_i:9,n_l:[3,7],n_neurons_connect:10,n_neurons_layer1:7,n_neurons_layer2:7,n_sampl:[5,8],n_split:5,n_t:1,n_x:1,nabla:[7,14,16,23,25],nabla_:1,nabla_i:[14,23],nabla_jv:[14,23],naimi:4,naiv:6,name:[0,1,2,4,5,6,7,8,10,12,13,14,15,17,21,22,23,24,26,30,34,36],name_scop:7,nameerror:2,namespac:3,narrow:[2,30],nation:[2,7,29],nativ:[4,12,21],natur:[0,1,2,3,4,7,8,10,11,13,15,20,22,24],navier:7,navig:31,nbconvert:[4,12,21],nboot:3,nburn:2,ncol:2,ndarrai:5,ndim:2,ndoubl:2,nearest:[5,7,9,10],nearli:[17,26],neat:[4,12,21],neccesari:5,necess:[1,2],necessari:[0,1,7,8,10,13,15,22,24],necessarili:[0,2,3,9,16,25],neck:6,need:[0,1,2,3,5,7,8,9,10,13,14,15,16,17,22,23,24,25,26],neg:[3,4,5,6,7,13,15,16,17,22,24,25,26],neg_mean_squared_error:5,neglect:[3,15,16,24,25],neglig:[3,17,26],neighbor:[5,9,10],neighbourhood:2,neq:3,nervou:7,nest:7,net:[1,7,10,13,14,16,17,22,23,25,26],netlib:[4,12,21],network:[0,5,6,12,21],neural:[0,5,6,12,21],neural_network:[1,4,7],neuralnetwork:7,neuralnetworktensorflow:7,neuron:1,neurosci:[34,36],neutral:4,neutron:4,never:[2,3,5,7,15,16,24,25],nevertheless:2,new_hobbit:[4,12,21],new_sig:5,newaxi:[4,5,10],newdata:[12,21],newli:[0,12,21],newton1:[13,22],newton:[3,6,7,8,12,15,17,21,24,26,28],newtonian:[17,26],next:[1,2,3,4,7,8,10,12,14,15,16,17,21,23,24,25,26],next_input:[1,10],neyman:2,nice:[2,4,5,7,9,12,21,28],nielsen:10,nitric:5,nlambda:5,nldr:9,nm_n:4,nmse:5,nnmin:1,no_of_head:2,no_of_tail:2,node:7,nois:[2,4,5,8],noisi:[5,7],non:[1,3,5,6,7,9,10,14,15,23,24,28],noncommerci:2,none:[1,2,3,5,7,10],noninform:2,nonlinear:[5,7,8,9,10],nonneg:5,nonparametr:5,nonsens:3,nonsingular:[4,12,21],nonumb:[6,8,13,14,15,16,17,22,23,24,25,26],nor:[0,7],norm:[2,4,5,7,8,9,13,14,15,22,23,24],normal:[0,2,4,5,6,7,8,9,10,11,12,13,14,15,17,20,21,22,23,24,26],normaldevi:2,normali:[4,12,21],normpdf:5,north:[17,26],northern:[17,26],notat:[1,3,5,12,14,21,23],note:[1,3,4,5,6,7,8,9,10,12,13,14,15,16,21,22,23,24,25,30,34,36],notebook:[0,4,7,10,11,12,20,21,34,36],notesapril10:28,notesapril13:28,notesapril15:28,notesapril17:28,notesapril1:28,notesapril20:28,notesapril22:28,notesapril24:28,notesapril3:28,notesapril6:28,notesapril8:28,notesmarch13:28,notesmarch16:28,notesmarch18:28,notesmarch20:28,notesmarch23:28,notesmarch25:28,notesmarch27:28,notesmarch30:28,notesmarch9:28,noth:[1,3,5,7,8,12,21],notic:[2,3,4,7,12,14,17,21,23,26],notion:10,novel:[5,10],novemb:7,now:[1,2,3,4,6,7,8,10,12,13,14,15,16,17,21,22,23,24,25,26],nowadai:[0,4,7,10,11,20],nox:5,nozzl:[16,25],npr:1,npt:[11,20],nrow:2,nsampl:5,nscl:28,nspin:3,nstep:2,nuclear:[15,16,24,25],nuclei:[3,4,16,25],nucleon:[4,5],nucleu:[4,16,25],nuisanc:2,num_coordin:1,num_hidden_neuron:1,num_it:1,num_neuron:1,num_neurons_hidden:1,num_point:1,num_valu:1,number:[0,2,4,6,7,8,10,11,12,13,14,15,16,17,20,21,22,23,24,25,26],numer:[0,4,5,7,9,11,12,13,16,20,21,22,25,28],numpi:[1,3,5,6,7,8,9,10,11,14,15,16,18,19,20,23,24,25,35],nutshel:9,nwalker:2,obei:[9,13,16,17,22,25,26],object:[2,4,5,8,10,12,16,21,25,28],oblig:29,obs:2,observ:[2,5,6,7,8,9,10,13,16,17,22,25,26],observatori:2,obstacl:0,obtain:[2,3,5,6,7,8,12,13,14,15,16,17,21,22,23,24,25,26],obviou:[0,3,5,9,14,23],obviouli:4,obvious:[0,2,4,5,12,13,14,21,22,23],occam:2,occupi:5,occur:[2,3,4,8,13,15,16,17,22,24,25,26],ocean:[17,26],octob:1,odd:[4,5,6,9,14,15,23,24],ode:1,odenum:1,odesi:1,odot:[14,23],off:[3,5,7,10,15,16,24,25],offend:4,offens:29,offer:[4,5,9,11,20],offic:28,offici:[29,30],offlin:29,ofil:3,ofstream:3,often:[0,2,3,4,5,6,7,8,9,11,12,13,15,16,17,20,21,22,24,25,26],ofter:[4,12,21],oint:[16,25],old:[7,14,23],ols_inv:5,ols_sk:5,ols_svd:5,omega02:[16,25],omega:[1,5,12,13,14,15,16,17,21,22,23,24,25,26],omega_0:[13,16,17,22,25,26],omega_0a:[15,24],omega_0t:[13,15,16,17,22,24,25,26],omega_c:[13,22],omega_ct:[13,22],omega_cv_i:[13,22],omega_i:[17,26],omega_t:[15,24],omega_x:[17,26],omega_xv_i:[17,26],omega_xv_z:[17,26],omega_yv_x:[17,26],omega_yv_z:[17,26],omega_z:[17,26],omega_zt:[17,26],omega_zv_i:[17,26],omega_zv_x:[17,26],omegatild:[15,24],omit:5,omnipres:[15,24],onc:[3,5,7,9,13,15,16,22,24,25,34,36],one:[0,2,3,4,6,8,9,10,11,14,15,16,17,20,23,24,25,26,34,36],onehot:7,onehot_vector:7,ones:[0,1,4,5,8,9,12,14,21,23],onli:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,20,21,22,23,24,25,26,34,36],onlin:[0,4,9,29],onto:[9,13,22],open:[0,2,3,4,5,7,11,12,20,21,29,30,31],oper:[0,3,4,5,7,9,10,12,14,15,21,23,24,30],operation:3,oplu:3,opportun:[0,4],oppos:[5,13,22],opposit:[7,8,13,14,16,17,22,23,25,26],oppotun:[11,20],opt:[0,4,5,7,8,9],optim:[0,1,2,6,9,10],optimis:10,option:[0,2,7,8,9,10],optmiz:7,orang:4,orbit:[2,14,23,28],order:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,20,21,22,23,24,25,26],ordinari:[6,9,10,13,15,22,24],org:[1,4,7,9,10,12,21],organ:[5,12,21],ori:2,orient:[3,5,12,21,29],origin:[3,4,5,7,8,9,10,12,13,14,15,16,17,21,22,23,24,25,26],orthodox:2,orthogn:5,orthogon:[4,5,8,9,12,17,21,26],orthonorm:5,oscil:[13,14,17,22,23,26,28],oscillatori:[15,24],oslo:2,osx:[4,12,21],other:[0,2,4,5,6,8,11,13,14,15,16,17,20,22,23,24,25,26,29],otherwis:[2,4,5,6,7,14,16,23,25,29],oup:2,ouput:7,our:[0,1,2,6,8,10,11,14,16,17,20,23,25,26],ourselv:[0,2,4,5,8,9,12,13,15,16,17,21,22,24,25,26],out:[1,2,3,4,5,6,7,8,9,12,14,15,16,17,18,19,21,23,24,25,26,33,35],outcom:[2,3,5,6,7,11,17,20,26],outer:[5,7],outfilenam:3,outgo:[16,25],outlier:[2,4,5,8,9],outlin:[5,9,11,20],output:[0,4,5,6,7,8,10,12,13,21,22,34,36],output_bia:7,output_bias_gradi:7,output_weight:7,output_weights_gradi:7,outputlayer1:7,outputlayer2:7,outsid:[1,10],outward:[14,16,17,23,25,26],over:[0,3,4,5,7,11,12,13,14,15,16,20,21,22,23,24,25],overal:[7,16,25],overarch:[0,2,13,22],overcom:[7,13,22],overdetermin:5,overemphasis:2,overfit:[5,7,10],overflow:7,overhead:7,overlap:[8,9,10],overlin:[4,5,9,12,21],overst:[0,4],overtrain:[1,10],overview:[0,1,10,28],owe:0,own:[0,1,4,7,8,10,11,14,15,17,20,23,24,26,28],owner:5,oxford:2,oxid:5,p_0:[16,25],p_0b:[16,25],p_0b_:[16,25],p_2:[14,23],p_f:[16,25],p_fr:[16,25],p_hidden:1,p_i:3,p_j:3,p_n:3,p_output:1,p_x:[3,14,23],pack:4,packag:[0,1,2,3,5,7,8,9,10,15,24],pad:10,page:[2,4,31,33,34,36],pai:7,painless:4,paint:[16,25],pair:[1,3,4,5,10,13,22],panda:[1,10,14,15,16,23,24,25],panel:[2,4,12,21],paper:[7,11,20,28],paradigm:[0,4],parallel:[4,12,13,14,16,21,22,23,25],param:[1,2],paramat:1,paramet:[0,1,3,4,7,8,9,10,12,16,21,25],parameter:[0,5,13,15,22,24],parametr:[4,5],paramt:10,pari:[17,26],park:3,part:[0,3,4,5,7,9,11,12,15,16,17,20,21,24,25,26,28,30],partform:[15,24],partial:[3,5,6,7,8,9,13,14,22,23],partial_:[14,23],partial_i:[14,23],partial_r:[16,25],partial_ru:[16,25],partial_ru_:[16,25],partial_tv:[14,23],partial_x:[14,15,23,24],partial_xv:[14,15,23,24],partial_yv:[14,23],partial_z:[14,23],partial_zv:[14,23],particip:[16,25,29],particl:[1,3,4,5,10,13,14,15,16,17,22,23,24,25,26,28],particular:[0,1,2,3,4,5,7,9,10,11,12,13,15,16,20,21,22,24,25,28],particularli:[3,5,8,9,11,15,20,24],partit:7,partli:5,paslei:[34,36],pass:[1,2,3,10,15,16,24,25],passeng:0,past:3,patch:[3,5,12,21],path:[4,5,9,12,13,14,15,16,17,21,22,23,24,25,26,34,36],patient:6,patter:[1,10],pattern:[0,1,4,7,10,17,26,34,36],pauli:4,pavement:[13,22],pca:5,pde:1,pdf:[0,1,2,4,5,10,12,21,28],pearson:2,pedagog:[5,11,20],pen:[11,20],penal:5,penalti:5,pencil:[11,20,28],pend1:[13,15,22,24],pend2:[13,15,22,24],pendulum:28,penros:5,peopl:[0,5,7],per:[4,5,11,12,16,17,20,21,25,26],perambul:2,perceiv:0,percentag:[5,9,28],percentil:2,perceptron:[4,6],peregrin:[4,12,21],perfect:[4,5,7,16,25],perfectli:[2,5,13,22],perform:[1,2,3,4,8,9,10,12,13,14,15,16,17,21,22,23,24,25,26],performac:[1,10],perhap:[0,2,4,5,13,14,22,23],period:[2,7,13,16,17,22,25,26],perman:29,permiss:29,permit:2,permut:[9,14,23],perp:[16,25],perpendicular:[13,16,17,22,25,26],person:[6,29],perspect:[2,14,23],pertain:[2,12,21],pertin:[7,11,20],perturb:[16,25],pervers:2,petal:8,petersen:3,phantom:3,phase:[5,7,13,15,17,22,24,26],phasespac:[16,25],phenomena:[2,3,15,24],phi:[8,12,13,14,15,16,21,22,23,24,25],phi_a:[17,26],phi_b:[17,26],phi_k:8,philosoph:2,philosophi:[2,15,24],phone:28,phonon:[15,24],phrase:0,phy321:36,physic:[0,1,2,3,4,5,6,7,10,11,12,13,14,15,16,17,20,21,22,23,24,25,26,29],physics321:28,piazza:28,pick:[2,3,7,9,15,24],pictur:[0,4,15,24],pie:[4,12,21],piec:[2,7,9,14,16,23,25],pillow:[4,12,21],pinv:5,pioneer:2,pip3:[4,7,12,21],pip:[4,7,12,21,31],pipelin:[4,5,8],pippin:[4,12,21],pitfal:[14,23],pitt:7,pixel:[7,10],pixel_height:[7,10],pixel_width:[7,10],place:[2,4,5,8,12,16,17,21,25,26],placehold:7,plai:[0,4,5,8,10,13,14,15,22,23,24],plain:[7,8],plan:[5,28],plane:[8,12,16,17,21,25,26],planet:[16,25],planetari:[16,25],plateau:3,platform:[4,12,21],plausabl:2,plausibl:7,pleas:[9,28,30],plenti:7,plethora:[2,7],plot:[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,18,19,20,21,22,23,24,25,35],plot_count:5,plot_dataset:8,plot_max:[1,10],plot_min:[1,10],plot_predict:8,plot_surfac:1,plt:[1,2,3,4,5,6,7,8,9,10,11,12,14,15,16,18,19,20,21,23,24,25,35],plu:[4,15,17,24,26],plug:[13,15,16,22,24,25],pmatrix:1,png:[2,4,5,9,12,14,15,16,21,23,24,25],point:[0,2,3,4,5,6,7,8,9,11,12,13,14,15,16,17,20,21,22,23,24,25,26],poisson:[0,2],polar:[14,15,16,17,23,24,25,26],pole:[17,26],poli:[5,8],polici:29,polit:[0,29],poly100_kernel_svm_clf:8,poly3:4,poly3_plot:4,poly_featur:8,poly_kernel_svm_clf:8,polydegre:5,polym:7,polynomi:[0,2,4,5,6,8,9,14,23],polynomial_featur:5,polynomial_svm_clf:8,polynomialfeatur:[4,5,8],polytrop:5,pool:10,pool_siz:10,poor:7,popul:[4,5],popular:[0,3,4,5,7,8,9,10,12,15,21,24],popularli:4,portion:9,pose:[0,3,9],posit:[1,2,3,5,6,7,8,9,12,13,14,15,16,17,21,22,23,24,25,26,29],possess:[17,26],possibl:[0,2,3,4,5,6,7,8,9,10,11,12,13,14,15,20,21,22,23,24,30],post:[18,19,28,29,30,35],posterior:2,postpon:[5,12,21],potenti:[0,4,5,7,11,12,13,14,15,20,21,22,23,24,28],pott:7,power:[2,4,5,7,8,9,11,13,15,20,22,24,34,36],practic:[3,5,6,8,16,25],practition:[4,7,10],pre:[34,36],preced:[3,7,9],preceed:[1,10],preceq:8,precess:[17,26],precis:[0,1,2,3,4,5,9,12,14,16,21,23,25],precmess:[17,26],pred:5,predict:[0,2,4,5,6,7,8,9,12,17,21,26,34,36],predict_prob:7,predictor:[4,5,6,9],prefer:[4,7,8,9,15,24],premis:2,prepar:[2,4,12,21],preprocess:[1,4,8,10],prerequisit:[0,4],presenc:[13,15,22,24],present:[0,2,3,4,5,7,11,14,20,23],preserv:[9,10],presid:2,presidenti:2,pressur:[17,26],presum:2,pretti:[4,8,12,21],prevent:3,previou:[1,2,3,5,7,8,9,10,12,13,15,16,21,22,24,25],previous:[1,3,10],price:[1,5,10],primal:8,primari:[0,6],prime:[3,14,15,23,24],princip:[5,15,24],principl:[2,4,5,6,8,13,17,22,26],print:[1,2,3,4,5,7,8,9,10,12,14,16,21,23,25,34,36],print_funct:8,printout:[5,12,21],prior:[0,4,5],prioriti:0,privaci:0,privat:[0,4,29],privileg:[12,21],pro:28,prob:[2,3],probabilist:5,probabl:[2,4,5,6,7,10,16,25],problem:[0,2,3,4,6,9,11,12,13,15,17,20,21,22,24,26,28],problemat:2,proce:[0,4,5,8,9,15,16,24,25],procedur:[1,2,8,9,17,26],proceed:[13,22],process:[0,2,3,4,5,11,12,13,20,21,22,28,31],prod_:[2,6,7],produc:[0,1,2,3,4,5,7,9,10,11,12,17,20,21,26],product:[2,4,5,6,7,8,10,12,14,16,21,23,25],profess:5,profession:29,profound:[14,23],program:[3,4,5,7,8,10,11,12,13,15,20,21,22,24],progress:7,prohibit:5,project:[0,1,3,4,5,7,9,11,17,20,26,28,29,30,31],project_root_dir:[4,5,9,12,14,15,16,21,23,24,25],promin:7,promis:8,pronounc:[4,12,21],proof:[5,7,14,23],prop:5,prop_cycl:[18,19,35],propag:10,proper:[0,1,4,5],properli:[2,7,8,11,20,34,36],properti:[2,4,7,9,10,12,13,17,21,22,26],proport:[2,3,4,5,7,9,13,15,22,24],propos:[7,30],proposit:2,propto:[2,16,25],protect:0,proton:[4,16,25],prove:[10,17,26],provid:[1,2,3,4,5,7,8,10,11,12,13,15,16,17,20,21,22,24,25,26],proxi:7,pseudoinvers:5,pseudorandom:3,psycholog:[0,4],ptratio:5,pub:28,publish:29,pull:[16,25,28,30,31],pulsar:7,punish:[0,4,7],pure:[3,15,16,24,25],purpos:[0,4,7,11,13,15,20,22,24,34,36],push:[17,26,30,31],put:[15,16,24,25],puzzl:[13,22],pycod:[4,12,21],pyhton2:[4,12,21],pylab:[4,12,15,21,24],pyplot:[1,3,4,5,6,7,8,9,10,11,12,14,15,16,18,19,20,21,23,24,25,35],python3:[4,5,7,8,9,12,21],python:[0,1,5,7,8,9,11,13,15,20,22,24,28],pytorch:[0,4,7,12,21],qbv_x:[13,22],qbv_y:[13,22],qquad:[1,4,9,12,21],quad:[2,7],quadrat:[2,4,8,13,15,16,22,24,25],qualit:3,qualiti:[0,4,11,12,13,15,20,21,22,24],quantifi:[2,7,16,25],quantit:[0,5],quantiti:[1,2,3,4,5,6,7,9,13,14,15,17,22,23,24,26],quantum:[0,1,7,10,14,15,23,24],quartil:[5,9],question:[0,2,5,7,9,12,13,15,21,22,24,28],quick:[3,11,20],quickli:[9,10,14,15,16,17,23,24,25,26],quit:[2,5,7],quitecompl:[16,25],quot:3,r2_score:[4,5],r2score:4,r_1:[15,24],r_1t:[15,24],r_2:[15,16,24,25],r_2t:[15,24],r_h:[14,23],r_i:[14,23],rab:[14,16,23,25],race:29,rad:5,radial:[5,7,8,14,16,17,23,25,26],radialeqofmot:[16,25],radialeqofmotion2:[16,25],radialgvv:[16,25],radialhovv:[16,25],radii:[16,25],radio:[15,24],radioact:3,radiu:[5,9,13,16,17,22,25,26],rais:5,ramsai:[34,36],ran1:3,ran2:3,ran3:3,rand:[1,2,3,4,5,10,12,21],rand_max:3,randint:5,randn:[1,4,5,7,9,12,18,19,21,35],random:[0,1,2,4,5,7,8,10,12,16,18,19,21,25,35],random_devic:3,random_indic:[7,10],random_st:[5,8,9],randomli:[3,5,7],randomnumbergener:3,rang:[1,2,3,4,5,6,7,9,10,11,12,13,14,15,16,18,19,20,21,22,23,24,25,35],rangl:[2,3,5,9],rangle_x:3,rank:[5,10],raphson:[7,8],rapidli:[0,4,17,26],rare:[13,22],rate:[1,5,7,8,10,14,16,17,23,25,26],rather:[0,1,2,3,4,5,6,7,8,9,10,13,14,15,16,17,22,23,24,25,26],ratio:[6,9,15,16,24,25],rational:[2,4],ravel:[5,8,9],raw:[3,10],rawdata:[12,21],razor:2,rbf:[7,8,9],rbf_kernel_svm_clf:8,rbf_pca:9,rcond:5,rcparam:[3,4,7,8,10,12,15,18,19,21,24,35],rdotdot:[16,25],rdr:[15,24],reach:[1,3,5,7,9,10,13,14,16,22,23,25],reaction:[13,14,22,23],read:[0,1,2,3,5,7,8,10,11,12,13,14,15,20,21,22,23,24,28],read_csv:5,read_fwf:[4,5],reader:[0,3,4,12,21],readi:[4,7,8,9,11,12,20,21,30],readili:7,real:[2,5,6,7,9,11,12,15,16,17,20,21,24,25,26],realist:[11,20],realiti:[3,17,26],realiz:[2,7,15,16,24,25],realli:[2,4,7,13,22],realm:2,reason:[0,1,2,5,7,10,11,17,20,26,29],rebenstock:28,recal:[3,4,5,7,9,12,17,21,26],recalcul:3,recast:[13,15,22,24],receiv:[3,7,10,15,24],recent:[0,2,4,5],recept:[7,10],receptive_field:10,recip:[4,5,6,12,21],recogn:[1,2,4,10,16,25,31],recognit:[0,4,7,10],recommend:[1,3,4,5,8,10,12,14,21,23,28,31],reconstruct:9,record:[13,22],recov:[12,21],recreat:3,rectangl:5,rectifi:[7,10],rectilinear:[17,26],recur:[0,4],recurs:[4,12,21],recycl:3,red:[1,4,5,8,10],redefin:[2,5],reduc:[2,4,7,10,13,15,16,22,24,25],reduce_mean:7,reduct:[3,4,5],refactor:[11,20],refer:[0,1,4,5,7,9,10,11,13,15,16,17,20,22,24,25,26,28,34,36],referenc:1,refin:7,refit:5,reflect:[0,1,2,3,4,7,10,11,20],reformul:[13,22],reg:9,regard:[2,7,29],regardless:[7,16,25,29],region:[1,2,7,10,11,17,20,26],regist:3,regr_1:4,regr_2:4,regr_3:4,regress:[0,2,7,8],regressor:[5,6],regular:[1,6,17,26,34,36],regularizer_loss:7,regularizer_loss_fc1:7,regularizer_loss_fc2:7,regularizer_loss_out:7,reinforc:[0,4,8],reject:29,rel:[0,1,2,3,4,5,6,7,10,11,13,14,15,17,20,22,23,24,26,28,34,36],relat:[0,1,2,3,4,5,7,9,10,12,13,14,15,16,17,21,22,23,24,25,26,28],relationship:[0,1,4,5,9,10,17,26],relatist:[17,26],relativeerror:[5,11,20],relativist:[13,15,22,24],releas:[2,4,7,12,21,30],relev:[2,3,4,5,9,11,12,20,21],reli:[0,4,5,8,9,13,22],reliabl:[2,3,6,14,23],religion:29,relu:[1,10],remain:[1,2,3,5,7,10,13,15,22,24],remaind:[2,3],reman:1,remark:[7,17,26],rememb:[4,8,12,15,21,24,30],remind:[0,3,5,9,11,13,15,16,17,20,22,24,25,26],remov:[1,5,10,29,31],render:[4,5,9,12,21,31,34,36],reorder:6,reorgan:[4,12,21],repeat:[1,2,3,4,5,7,9,10,12,13,14,15,16,17,21,22,23,24,25,26],repeated:0,repeatedli:[3,5,15,24],repel:[16,25],repercuss:29,repetit:[2,5],rephras:[13,22],replac:[1,2,4,5,7,10,11,12,14,15,16,20,21,23,24,25,28],replica:5,repo:[30,31],report:[2,11,20,29],repositori:[0,31],repres:[0,1,2,3,4,5,6,7,8,10,11,12,13,14,15,16,17,20,21,22,23,24,25,26,29],represent:[0,3,5,7,9,10,15,24,29],reproduc:[0,2,3,4,7,11,16,18,19,20,25,28,30,35],repuls:[4,16,25],request:[0,4,30,31],requir:[4,5,7,8,9,10,11,12,13,15,16,20,21,22,24,25,31],res1:1,res2:1,res3:1,res:1,res_analyt:1,res_analytical1:1,res_analytical2:1,res_analytical3:1,resampl:[0,3],rescal:[5,7,9],research:[0,4],resembl:[3,5,17,26],reserv:[3,5,7],reset:3,reshap:[1,2,4,5,7,8,10,12,21],residenti:5,residu:5,resist:[14,15,23,24,28],reson:28,respect:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,15,17,20,21,22,24,26,29],respond:[7,29],respons:[5,6,7,13,17,22,26],rest:[5,9,12,13,15,16,17,21,22,24,25,26,34,36],restat:[5,7],restor:[13,22],restrict:[0,7,17,26],result:[0,1,2,3,4,6,7,8,9,10,11,12,14,15,16,17,20,21,23,24,25,26,28,34,36],ret:5,retail:5,retain:[5,14,23],rethink:5,return_sequ:[1,10],reus:[10,14,15,23,24],reveal:[0,7],revers:[7,16,25],revert:[12,21],review:29,revis:[7,12,21],revisit:[15,24],reward:[0,4],rework:7,rewrit:[3,4,6,7,8,12,13,14,15,16,17,21,22,23,24,25,26],rewritten:[1,3,5,8,16,25],rgb:10,rho:[5,16,17,25,26],rho_m:[13,22],rho_ma:[13,22],rhs:[9,12,21],rich:[0,4],ridg:[4,6,9],ridge_sk:5,right:[1,2,3,4,5,6,7,8,10,11,12,13,14,15,16,17,20,21,22,23,24,25,26,29],right_sid:1,rightarrow:[2,3,5,7,8,9,12,13,14,15,16,17,21,22,23,24,25,26],rigid:[17,26],rigor:[0,4,17,26],ring:[5,16,25],rise:[5,17,26],risk:4,river:5,rk2:[15,24],rk4:[15,24],rmin:[16,25],rminofa:[16,25],rmse:5,rmsprop:[1,10],rnn1:[1,10],rnn2:[1,10],rnn:7,rnn_2layer:[1,10],rnn_input:[1,10],rnn_output:[1,10],rnn_train:[1,10],rntrick1:3,rntrick2:3,rntrick3:3,rntrick4:3,road:0,robert:[34,36],robust:4,robustscal:[5,9],rocket:[14,23],role:[0,1,4,5,8,14,23],rolenam:[34,36],roll:[5,15,24],room:[0,4,5,28],root:[0,2,3,4,5,15,24],rot:[4,12,21],rotat:[8,12,13,14,16,21,22,23,25,28],roughli:[7,10,16,25],round:[5,9,17,26],routin:[4,12,21],row:[1,2,4,5,7,9,12,21],rp_f:[16,25],rrr:5,rtotheta:[16,25],rule:[2,5,11,12,16,17,20,21,25,26,28],run:[0,1,2,4,5,7,8,9,11,12,13,16,20,21,22,25,31,34,36],run_mcmc:2,rung:28,runner:[12,21],runtim:[5,7],runtimewarn:7,rust:4,ruthtraj:[16,25],rvert:7,rvert_2:7,rvs:2,rwidth:5,s_0:[17,26],s_1:5,s_h:[14,23],s_i:[5,6],s_j:5,s_k:5,safe:3,sai:[0,1,2,3,4,5,6,7,8,9,10,12,13,16,17,21,22,25,26],said:5,sake:[4,5,6,9],sale:4,sam:[4,12,21],same:[1,2,3,4,7,8,9,10,12,13,14,15,16,17,21,22,23,24,25,26,34,36],sampl:[0,1,2,4,5,6,7,8,10,16,17,18,19,25,26,33,35],sampleexptvari:3,sampler:[0,2],sampper:2,samwis:[4,12,21],sanitize_sequ:5,satisfactori:4,satisfi:[1,3,5,7,8,13,15,16,22,24,25],satur:[5,7],saturn:2,save:[1,4,5,9,10,11,12,14,15,16,20,21,23,24,25],save_fig:[4,5,9,12,14,15,16,21,23,24,25],savefig:[3,4,5,9,12,14,15,16,21,23,24,25],savetxt:[1,10],saw:[14,15,23,24],scalar:[1,5],scale:[2,4,5,6,7,8,9,11,12,16,20,21,25,28],scaler:[5,8,9],scan:6,scatt:[16,25],scattangl:[16,25],scatter:[4,5,8,12,21,28],scenario:[2,5],scene:9,schemat:2,scheme:[7,12,21],scholar:2,school:2,schrage:3,sci:28,scienc:[0,2,3,4,7,10,11,12,20,21,28],scientif:[0,2,4,11,12,20,21,28],scientist:[0,4],scikit:[0,8,10,12,21],scikitlearn:4,scipi:[4,5,12,15,21,24],scl:5,scope:[7,30],score:[4,5,7,9,10,28],scores_kfold:5,sea:[17,26],seaborn:[4,5,7,9,10,12,15,21,24],seamless:[4,12,21],search:[4,7,10,13,16,22,25],seat:[17,26],sec:5,second:[0,1,2,3,4,5,6,7,8,9,10,12,14,15,16,17,21,23,24,25,26,28],secondeigvector:9,secondli:[7,15,24],section:[1,2,3,10,12,13,15,17,21,22,24,26,27,28,32],sector:[0,4],secur:0,see:[0,1,2,3,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,28,31,34,35,36],seed:[1,2,3,4,5,7,8,9,10,12,18,19,21,35],seek:[1,2,7,8],seem:[2,7,10,15,16,24,25],seemingli:4,seen:[0,3,4,5,7,9,10,11,13,16,17,20,22,25,26],seismic:5,seldomli:[4,11,20],select:[4,5,7,8,9,12,21],self:[0,3,5,7,11,20],sell:[1,10],semi:8,semilogx:5,send:[7,28,30],sens:[0,2,5,8,13,22],sensibl:[2,10],sensit:[4,5,9],sensor:0,sent:[1,28],sentenc:[1,7,10],separ:[1,3,4,5,7,8,10,12,13,15,16,21,22,24,25,29],septemb:5,sequenc:[1,3,4,6,7,10,12,21],sequenti:[1,3,7,10],seri:[0,1,2,4,5,7,9,10,12,21,28],serif:[3,4,12,15,21,24],seriou:0,serv:[0,1,4,6,7,10,34,36],sess:7,session:[7,28],set:[0,2,3,4,6,9,11,12,14,15,16,17,20,21,23,24,25,26,29,30],set_:5,set_label:5,set_tick:8,set_titl:[1,4,6,7,9,10],set_xlabel:[1,2,4,5,6,7,9,10,12,14,15,16,21,23,24,25],set_xlim:[6,7,12,14,15,21,23,24],set_ylabel:[1,2,4,5,7,9,10,12,14,15,16,21,23,24,25],set_ylim:[6,7],set_ytick:9,set_yticklabel:5,setiosflag:3,setminu:5,setosa:8,setosa_or_versicolor:8,setp:5,setprecis:3,setup:[2,4,5,7,8,12,17,21,26,30],setw:3,sever:[0,2,3,4,5,6,7,8,9,10,11,12,14,15,20,21,23,24,28],sexist:0,sexual:29,sgd:[7,10],sgd_clf:8,sgdclassifi:8,shade:2,shape:[1,2,4,5,7,8,9,10,11,12,16,17,20,21,25,26],share:[0,2,4,7,10,12,21],sharei:2,sharex:2,she:6,shift:[3,7],ship:10,shire:[4,12,21],shock:[15,24],shortcom:2,shorten:[1,10],shorter:3,shortest:2,shorthand:[2,4,12,14,21,23],shortli:[4,9,12,13,21,22],should:[0,1,2,4,5,8,10,12,13,14,15,16,17,21,22,23,24,25,26,28,31],show:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,20,21,22,23,24,25,26,29],shown:[1,2,3,7,8,9,10,13,14,22,23],showpoint:3,shrink:[5,8,10],shrinkag:5,shrunk:9,shuffl:[5,7],side:[1,2,5,7,8,12,13,16,17,21,22,25,26],sigh:[4,12,21],sigma0:3,sigma1:3,sigma2:3,sigma:[2,3,4,5,6,7,9,16,25],sigma_1:5,sigma_2:5,sigma_:[2,3,4,5,16,25],sigma_fn:[6,7],sigma_i:[4,5],sigma_j:5,sigma_m:3,sigma_n:[3,9],sigma_x:3,sigmoid:[1,6,8,10],sign:[1,2,3,6,7,8,13,16,17,18,19,22,25,26,28,35],signal:[2,7,15,24],signific:[7,17,26],significantli:[3,7,13,15,22,24],sim:[3,5,16,25],similar:[0,1,4,5,6,8,9,10,11,15,16,20,24,25,28,34,36],similarli:[3,4,5,7,8,9,12,15,16,17,21,24,25,26],simpl:[0,6,11,13,15,16,17,20,22,24,25,26],simple_rnn:1,simpler:[0,4,7,11,13,14,20,22,23],simplernn:[1,10],simplest:[0,4,5,7,10,12,14,15,17,21,23,24,26,34,36],simpli:[1,2,3,4,5,7,8,9,11,12,13,14,15,16,17,20,21,22,23,24,25,26],simplic:[1,5,6,7,8,9,12,13,16,21,22,25],simplicti:[15,24],simplifi:[0,4,5,12,16,21,25],simplist:3,simpson:28,simul:[0,2,3,5,13,22],simultan:5,sin:[1,4,5,7,10,12,13,14,15,16,17,21,22,23,24,25,26],sinc:[0,1,2,3,4,5,6,7,8,9,11,12,13,14,17,20,21,22,23,26],sine:[7,15,24],singl:[1,3,4,6,7,8,10,12,14,15,16,17,21,23,24,25,26,28],singlephotoncount_fig_1:2,singlephotoncount_fig_2:2,singular:0,sit:[15,17,24,26],site:[0,4,5,7,8,9],situat:[0,2,4,5,6,11,15,16,17,20,24,25,26],sivia:2,six:3,size:[1,3,4,7,8,9,10,12,13,14,15,16,21,22,23,24,25,29],sketch:[13,16,22,25],skill:[0,2,4,11,20,28],skip:9,skirt:2,skl:[4,5,9],sklearn:[4,5,7,8,9,10],sky:2,slack:8,slice:[1,4,12,21],slide:[3,5,10,28],slight:[5,34,36],slightli:[1,3,5,6,7,13,14,17,22,23,26],slope:[7,8,9,12,21],sloppi:[13,22],slow:[1,5,8,9,13,22],slower:[4,5,12,17,21,26],slowli:7,slp:7,small:[1,2,3,4,5,7,8,9,10,13,14,15,16,17,22,23,24,25,26,28,33,34,36],smaller:[1,3,4,5,7,8,9,14,16,23,25],smallest:[1,4,10,12,21],smart:5,smooth:5,sne:9,sneak:5,sns:[4,5,7,9,10],soar:5,social:[0,4,29],societ:0,societi:[0,11,20],sofist:0,soft:[6,7],soften:8,softmax:[6,10],softmax_cross_entropy_with_logits_v2:7,softmax_loss:7,softwar:[0,8,11,20,28],sol:8,sole:[0,4,5],solid:[0,4,6,15,16,24,25],solut:[2,3,5,7,8,9,11,13,14,15,17,20,22,23,24,26,28],soluton:1,solv:[0,2,4,5,7,9,10,11,14,16,17,20,23,25,26,28],solve_expdec:1,solve_ode_deep_neural_network:1,solve_ode_neural_network:1,solve_pde_deep_neural_network:1,solveod:1,solveode_popul:1,solver:[1,8,9,13,22],some:[0,1,3,6,7,8,9,10,11,13,14,15,16,17,18,19,20,22,23,24,25,26,28,34,35,36],someth:[0,2,3,7,9,13,22],sometim:[2,5,7,9,16,25],somewher:[16,25],soon:[5,12,15,17,21,24,26],sophist:[0,4],sort:[3,5,9],sound:[0,11,15,20,24],sourc:[0,2,3,4,5,7,11,12,13,15,16,20,21,22,24,25,31],south:[17,26],southern:[17,26],sovl:[14,23],space:[1,3,5,7,9,10,13,15,17,22,24,26,28,29],spacetim:[17,26],span:[0,4,5,9,12,21,34,36],spare:7,spars:[4,5,10],sparse_mtx:4,sparsecategoricalcrossentropi:10,spartan:[16,25],spatial:[1,7,10,12,15,21,24],speak:[2,3],speci:[16,25],special:[2,3,4,5,6,7,11,12,13,17,20,21,22,26],specif:[0,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,20,21,22,23,24,25,26,29,34,36],specifi:[0,2,3,4,5,6,9,12,13,17,21,22,26],specifici:4,spectacular:10,spectral:2,speech:[0,1,4,7,10],speed:[1,7,13,14,22,23],spell:[14,23],spend:3,sphere:[5,9,16,25],spheric:[16,25],sphinx:[34,36],spin:[5,14,23],spite:[0,4],spline:8,split:[0,1,3,4,7,8,9,10,14,23],spontan:[3,14,23],sprai:[16,25],spread:[2,3,5,9,16,25],spring2020:28,spring:[13,15,16,22,24,25,36],springer:28,springforc:[15,24],spuriou:[17,26],sqrsignal:[15,24],sqrt:[2,3,5,8,9,12,13,14,15,16,17,21,22,23,24,25,26],squar:[0,1,2,3,4,6,7,8,9,10,12,13,15,16,21,22,24,25],squash:7,srand:3,stabil:[2,13,15,22,24],stabl:[1,4,5,9,10,14,16,23,25],stack:[5,10],stadium:[16,25],stage:[0,13,15,22,24],stai:[1,4,5,9,10,13,14,15,16,17,22,23,24,25,26],stand:[0,4,5,7,12,17,21,26,34,36],standar:9,standard:[0,1,2,4,5,6,7,8,10,12,13,16,17,21,22,25,26],standardscal:[5,8,9],standpoint:[11,20],stanford:[1,10],start:[0,1,2,3,5,7,8,9,10,11,12,13,14,15,16,17,20,21,22,23,24,25,26,28,33],starting_guess:2,startpoint:3,stat:[2,5],state:[0,1,2,3,4,6,7,8,9,10,12,13,14,15,16,17,18,19,21,22,23,24,25,26,35,36],statement:[0,2,4,9,12,21],stati:2,station:[15,24],statis:5,statist:[0,4,6,7,9,12,13,21,22],statu:[5,6,9],std:[2,3,4,5,12,21],stddev:7,stdev:3,steadi:[15,24],step:[1,2,3,4,6,7,10,12,15,16,17,21,24,25,26,30],step_fn:[6,7],stepfil:2,stepsiz:[11,20],stereo:10,sthetover2:[16,25],still:[1,2,3,5,9,10,13,16,22,25],stimuli:7,stochast:[0,5,7,8,9],stock:[0,1,10],stoke:7,stone:[4,6],stop:[1,3,7,9,10,12,13,21,22],storag:5,store:[1,3,4,7,9,10,12,21,34,36],str:[1,7,10],straight:[0,4,5,8,13,17,22,26],straightforward:[1,2,5,8],strang:2,strategi:[0,2,7,13,14,16,22,23,25],stratifi:5,stream:7,strength:[2,4,5,15,24],stress:2,stretch:[9,13,16,22,25],strict:8,strictli:[2,8,13,22],strike:[5,16,25],string:[3,7],stroke:6,strong:[3,7,15,16,24,25],strongli:[0,4,5,8,10,12,16,21,25],stronli:5,structur:[0,1,4,5,7,10,11,12,20,21,28,33,34,36],stuck:7,student:[0,2,4,11,20],studi:[0,2,4,5,7,8,9,11,12,13,16,17,20,21,22,25,26,28],style:[4,12,15,21,24],sub:7,subdivid:[0,4,12,13,14,21,22,23],subfield:[0,4],subinterv:[14,23],subject:[2,3,5,8],subplot:[1,2,4,5,7,8,9,10,12,14,15,16,18,19,21,23,24,25,35],subplots_adjust:[3,8],subprogram:[4,12,21],subroutin:4,subscript:[7,12,14,17,21,23,26],subsect:[15,24],subsequ:[2,3,5,7],subset:[4,5,7,12,21],subsidiari:2,subspac:[5,8,9],substep:9,substitut:[5,7,13,14,15,16,17,22,23,24,25,26],subtend:[16,25],subtl:7,subtleti:2,subtract:[1,2,3,5,9,10],succeed:4,success:[3,6,10],sucess:3,sudden:[15,24],suddenli:[17,26],sudo:[4,12,21],suffer:[1,5,7],suffic:[12,21],suffici:[5,7,8,9,15,16,17,24,25,26],suggest:7,suit:[7,8,15,24],suitabl:[3,5,9],sum:[1,2,3,4,6,7,8,9,13,14,15,16,22,23,24,25],sum_:[1,2,3,4,5,6,7,8,9,13,14,15,16,22,23,24,25],sum_i:[1,2,5,8,9,14,15,17,23,24,26],sum_ia_i:[17,26],sum_im_i:[14,23],sum_im_ix_i:[14,23],sum_j:5,sum_k:[4,5,7,8,12,21],sum_n:[15,24],sum_nx_:[15,24],summar:[2,5],summari:[1,7,10,28],summat:5,sun:[13,16,22,25,28],superconduct:2,superfici:10,superscript:7,supervis:[0,4,5,6],supplement:[6,17,26],support:[0,5,7],suppos:[0,4,5,6,7,8,9,12,13,17,21,22,26],supposedli:0,sure:[1,2,5,7,10,18,19,35],surfac:[4,13,14,16,17,22,23,25,26],surpass:5,surpris:4,surprisingli:[2,16,25],surround:[10,16,25],surv:0,survei:[0,4],svc:[8,9],svd:9,svdinv:5,svm:[8,9],svm_clf:8,svr:9,sweden:2,sweep:[13,22],swift:0,swing:[15,24],syllog:2,symbol:[2,3,4,7,9,11,12,13,14,20,21,22,23,28],symmetr:[0,2,4,5,7,8,9,12,16,17,21,25,26],symmetri:[2,5,15,24],sympi:[4,11,12,14,20,21,23],symplect:[14,23],synonim:3,syntax:[4,7,9,12,21,34,36],syntaxerror:[4,7,8,9,12,21],system:[0,1,2,4,5,6,7,10,11,12,15,16,20,21,24,25,28,30],systemat:[2,3,5],t_0:[1,12,21],t_f:[12,13,21,22],t_h:[14,23],t_i:[1,7,9,12,13,14,15,21,22,23,24],t_j:7,t_n:[14,23],t_x:[17,26],t_y:[17,26],tab:[15,24,31],taberror:[15,24],tabl:[3,4,12,21],tabul:4,tabular:[4,12,21],tackl:2,tag:[1,2,3,5,6,7,13,14,15,16,17,22,23,24,25,26,30],taht:4,tail:[2,3],tailor:[1,4,8,9],taiwan:5,take:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,20,21,22,23,24,25,26,29],taken:[4,5,7,10,12,21],takeoff:[17,26],tan:[15,24],tangent:[1,7,10,17,26],tanh:[1,6,7,8,10,12,21],target:[4,5,7,8,9,10,16,25],task:[0,2,4,5,7,9,10],tau:[3,14,15,23,24],tau_z:[14,23],taught:[0,2,10,11,20],tax:5,taxpay:0,taylor2:28,taylor:[1,2,12,13,14,15,16,21,22,23,24,25,28],tba:28,tdt:[12,21],teach:[11,20],team:[7,29],teaser:4,techniqu:[0,3,4,5,7,8,11,12,13,14,20,21,22,23],technolog:[0,2,4,7],tediou:[11,16,20,25],telescop:2,tell:[1,3,5,9,10,13,14,15,22,23,24],temperatur:5,templat:31,tempor:[15,24],temporari:29,temporarili:29,ten:[4,12,21],tend:[2,5,7,8,10,13,16,17,22,25,26],tendenc:4,tension:[5,17,26],tensor:10,tensorboard:7,tensorflow:[0,1,4,8,12,21],term1:[5,9],term2:[5,9],term3:[5,9],term4:[5,9],term:[1,2,3,4,5,6,8,9,10,11,12,13,14,15,16,17,20,21,22,23,24,25,26,30],termin:[5,13,22],test:[0,1,2,3,4,8,9,10,11,13,16,20,22,25,31],test_acc:10,test_accuraci:[7,10],test_error:5,test_imag:10,test_ind:5,test_label:10,test_loss:[7,10],test_pr:7,test_predict:7,test_rnn:[1,10],test_siz:[5,7,9,10],testerror:5,testi:[1,10],testpredict:[1,10],testx:[1,10],tex:[18,19,35],text:[1,2,3,4,5,7,8,9,10,12,15,21,24,28,34,36],textbook:[2,13,22],tfinal:[12,14,15,16,21,23,24,25],than:[1,2,3,4,5,6,7,9,10,12,13,15,16,17,21,22,24,25,26],theano:[4,7,12,21],thei:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,15,16,17,20,21,22,24,25,26,28,29,30,34,36],them:[0,1,2,4,7,8,9,10,11,12,15,17,20,21,24,26],theme:4,themselv:[3,4],thenc:5,theorem:[1,5,14,23,28],theoret:[5,13,22],theori:[0,4,7,8,13,15,22,24],thereaft:[4,5,7,9,12,15,17,21,24,26],therebi:[0,3,4,6,9,12,15,17,21,24,26],therefor:[1,2,3,4,5,6,7,8,9,10,15,16,24,25],therein:9,thereof:[0,4,5],theta:[3,5,7,13,14,15,16,17,22,23,24,25,26],theta_0:[13,16,22,25],theta_:[7,16,25],theta_i:7,theta_k:3,thi:[0,1,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,29,30,31,32,33,34,35,36],thick:[16,25],thin:[16,25],thing:[2,3,5,7,9],think:[0,2,3,5,7,10,11,13,15,16,17,20,22,24,25,26,28],third:[0,4,5,10,12,14,15,16,17,21,23,24,25,26],thirti:9,those:[2,4,5,8,9,12,13,14,15,16,17,21,22,23,24,25,26,34,36],though:[0,1,2,3,7,13,15,16,22,24,25],thought:[2,3,5,16,17,25,26],thousand:[0,4,5,7,9],threaten:29,three:[0,2,4,5,7,8,10,12,14,15,17,21,23,24,26],threshold:[7,9,10],through:[0,1,2,3,4,5,7,8,9,10,11,12,13,14,16,17,20,21,22,23,25,26,28,30],throughout:[0,2,3,4,5,12,21],thrown:[13,22],thu:[0,1,3,4,5,6,7,8,9,12,13,14,15,16,17,21,22,23,24,25,26,28],thumb:5,thursdai:[1,28],tic:2,tick_param:5,ticker:3,tidal:[17,26],tide:[16,25],tight_layout:[9,12,14,15,16,21,23,24,25],tightli:9,tild:[3,4,5,9,15,24],till:[1,5,6,7,8,9,10,12,13,17,21,22,26],time:[1,2,3,4,5,6,7,8,9,10,11,13,14,15,16,17,20,22,23,24,25,26,28,34,36],timefunct:3,timeit:[1,10],timer:[1,10],tiniti:[12,21],tip:[10,27,32],tire:[13,22],titl:[1,3,4,5,7,8,10],to_categor:[1,7,10],to_categorical_numpi:7,to_numer:[4,5],to_str:3,todai:[0,2],togeth:[2,4,5,8,9,16,17,25,26],toi:2,toler:[1,13,22],tolist:[1,10],tomographi:7,too:[1,2,3,5,9,13,22],took:[4,8,12,21],tool:[0,2,3,4,5,7,9,10,11,12,20,21,34,36],toolbox:8,top:[2,4,5,11,12,17,20,21,26],topic:[0,4,5,6,8,11,20,28],topolog:[7,10],torqu:[13,14,15,22,23,24],toss:2,tot:[16,25],total:[1,3,5,6,7,8,9,10,13,14,15,16,22,23,24,25,28],totaldimens:9,totalvari:3,tout:0,toward:[1,6,7,15,16,17,24,25,26,29],town:5,tqdm:5,traceback:[2,5],track:[7,10,12,16,21,25],tract:5,tractabl:[0,5,9],traction:0,trade:5,tradeoff:[0,2],tradit:[0,1,4,5,7,10],tradition:[11,20],train:[0,4,8,9,11,13,17,20,22,26],train_accuraci:[4,7,10],train_end:7,train_error:5,train_imag:10,train_ind:5,train_label:10,train_loss:7,train_pr:7,train_siz:[7,10],train_test_split:[4,5,7,9,10],train_test_split_numpi:7,trainabl:[1,7],traini:[1,10],trainingerror:5,trainpredict:[1,10],trainscor:[1,10],trainx:[1,10],trait:[0,4],trajectori:[1,10,13,14,15,16,22,23,24,25],transax:2,transcendent:[13,22],transform:[2,5,6,7,8,9,16,25,28],transit:[5,7],translat:[1,7,10,14,15,23,24],translation:[14,23],transpos:[5,7,9],transvers:[16,25],trapez1:[14,23],trapez:[11,14,20,23],trapezoid:[11,20,28],trapoezoid:[14,23],trapz:2,travel:[13,16,22,25],treat:[3,5,7,10,15,16,17,24,25,26],tree:[0,7,28],trend:3,tri:[1,13,22],trial:[0,2,3,4,5],triangular:[4,12,21],trick:[3,8,9,10,16,17,25,26],trickier:[3,12,21],tridiagon:[4,12,21],trivial:[3,5,7,9,16,25],troll:29,troubl:[5,7,8,9],troubleshoot:30,truck:10,true_divid:7,true_fun:5,truli:2,truncat:[12,15,21,24],truncated_norm:7,trust:[13,14,22,23],truth:[0,2],tucker:8,tumor:[6,9],tumour:9,tune:[1,4,10,12,15,21,24],turn:[0,2,3,4,5,7,8,9,11,13,14,15,17,20,22,23,24,26],tutori:[2,7],tveito:1,tweak:[3,7],twice:[12,16,21,25],twist:9,twister:3,two:[0,1,2,3,4,5,7,9,10,12,14,15,17,21,23,24,26,28,34,36],twoard:[17,26],txt:31,type:[0,3,5,8,12,13,14,15,21,22,23,24,28],typeset:[11,20],typic:[0,1,3,4,5,6,7,10,13,14,15,16,22,23,24,25],u_i:7,ubuntu:[4,12,21],uci:5,ugli:[15,24],uio:1,unaccept:29,unari:[4,12,21],unbalanc:5,unbias:[3,4,5],unbound:[16,25],uncertain:2,uncertainti:[2,5,13,22],uncertitud:3,unchang:[7,10,14,23],uncov:7,undamp:[15,24],undefin:5,under:[2,4,5,7,12,13,14,16,21,22,23,25],underdetermin:5,underfit:[5,7],undergo:[11,13,16,20,22,25],undergradu:0,underli:[0,2,3,4,7,11,20],understand:[0,2,4,7,10,11,13,20,22,28],understood:[8,13,16,22,25],undesir:8,undetermin:8,undriven:[13,15,22,24],unemploy:2,unexpect:[3,5],unexpected:3,unfair:2,unfortun:[7,8,13,22],unicode_liter:8,unifi:2,uniform:[0,2,4,5,7,9,13,22],uniform_real_distribut:3,uniformli:[3,16,17,25,26],unifrompdf:3,union:5,uniqu:[0,1,4,5,11,20],unit:[1,2,3,4,5,7,10,11,13,14,15,17,20,22,23,24,26],unitari:[4,5,12,21],unitarili:[4,12,21],uniti:[3,15,24],univ:28,univari:3,univers:[0,1,2,10,36],unix:7,unknow:5,unknown:[1,5,7,8,9,10,15,24],unknowwn:7,unlabel:7,unless:[4,5,9,13,14,15,17,22,23,24,26],unlik:[3,7,8,10],unlimit:[14,23],unnecessarili:2,unord:10,unravel:7,unrol:[9,10],unscatt:[16,25],unseen:6,unstabl:[7,16,25],unsupervis:[0,4,7],unsymmetr:[4,12,21],until:[1,3,7,10,15,24],unusu:7,unwant:2,unwelcom:29,updat:[1,2,5,7,9,12,14,15,16,21,23,24,25],upgrad:[13,22],upon:[0,7,9,13,17,22,26],upper:[4,5,8,12,14,21,23],uppercas:[3,4,12,21],ups:4,upward:[13,17,22,26],usag:[0,3,4,5,8,12,21],usain:[13,22],uscibook:28,usd10000:5,usd:5,use:[0,1,2,4,5,6,8,9,10,11,12,13,14,15,16,17,20,21,22,23,24,25,26,28,29,30,34,36],use_bia:[1,10],usecol:[4,5],used:[0,1,2,3,4,5,6,7,8,9,11,12,13,14,15,16,17,20,21,22,23,24,25,26,28,34,36],useful:[0,1,2,3,4,6,7,9,10,12,13,14,15,16,17,21,22,23,24,25,26,28],useless:7,user:[1,4,6,7,8,9,10,11,12,13,20,21,22],uses:[1,3,4,5,7,9,10,12,14,15,16,21,23,24,25,34,36],usetex:3,using:[0,2,5,8,9,10,11,12,13,14,15,16,17,20,21,22,23,24,25,26,28,29,30,31],usr:3,usual:[1,4,6,7,10,13,16,17,22,25,26],util:[1,4,5,7,10],utliz:[14,23],utmost:0,v_0:[9,12,13,14,15,21,22,23,24],v_1:[14,23],v_e:[14,23],v_i:[12,13,14,21,22,23],v_jb_k:[13,22],v_n:[14,23],v_t:[12,13,21,22],v_x:[13,14,17,22,23,26],v_xf_x:[14,23],v_y:[13,14,22,23],v_yf_i:[14,23],v_z:[13,14,17,22,23,26],v_zf_z:[14,23],v_zt:[13,22],vagu:2,val_accuraci:10,val_loss:[1,10],vale:1,valid:[0,1,2,3,7,10,11,13,20,22,34,36],validation_data:10,validation_split:[1,10],valu:[0,1,2,4,6,7,8,10,11,12,13,14,15,16,20,21,22,23,24,25],valy:[1,10],van:5,vandenbergh:8,vandermond:5,vanderpla:2,vanilla:9,vanish:[3,7],var_x:3,varabl:8,varepsilon:5,varepsilon_:5,varepsilon_i:5,vari:[5,7,10,12,17,21,26],variabl:[1,2,4,5,6,7,8,9,12,14,16,17,21,23,25,26,28],varianc:[0,2,4,6,7,9],variance_i:[5,9],variance_x:[5,9],variant:[4,5,7,8,9,11,12,20,21],variat:[9,14,23,28,34,36],varieti:[0,2,4,7,12,21],variou:[0,2,3,4,6,7,8,9,11,12,14,15,17,20,21,23,24,26,28],varphi:2,vartempvec:3,varvec:3,varydimens:[1,10],vastli:10,vaverag:[12,21],vdot:1,vdt:[13,22],vec:[3,5],vecto:[14,23],vector:[1,6,10,13,14,16,17,22,23,25,26,28],vehicl:[17,26],veloc:[13,15,16,17,22,24,25,26,28],velocityverlet:[15,24],ventur:[0,2,4,8,11,12,14,20,21,23],verbos:[1,7,10],veri:[0,1,2,3,4,5,7,8,9,10,12,13,14,15,17,21,22,23,24,26,28,34,36],verif:[11,20,28],verifi:[4,9,11,12,13,15,20,21,22,24],verlet:[15,16,24,25,28],versa:[14,23],versatil:[4,8,11,12,20,21],versicolor:8,version:[0,3,4,12,15,16,21,24,25,29,30,31],versu:[7,16,25],vert:[4,5,6,7,8,9,12,21],vert_1:5,vert_2:[5,9],vert_:[14,23],vertic:5,via:[0,2,3,4,5,7,8,9,11,12,13,14,15,20,21,22,23,24,29],vibrat:[16,25],vice:[14,23],video:[1,7,28],view:[0,2,3,4,7,11,12,14,17,20,21,23,26,28],viewpoint:29,violat:8,viridi:[1,4,7,10],virtual:[7,31],virtualenv:30,viru:28,viscos:[13,15,22,24],viscou:[13,15,22,24],vision:[0,4,10],visual:[0,2,4,5,9,15,24],viz:[3,5,8],vline:2,vmax:5,vmc:3,vmin:5,volcano:[17,26],voltag:[16,25],volum:[4,13,16,22,25],volunt:30,vstack:[3,4,5,9],w_1:8,w_1x_1:8,w_1x_:8,w_2:8,w_2x_2:8,w_2x_:8,w_fc1:7,w_fc2:7,w_hidden:1,w_i:[1,2,7],w_ix_i:7,w_out:7,w_output:1,w_px_:8,w_px_p:8,wai:[0,1,2,3,4,5,6,7,8,9,10,11,12,14,20,21,23,27,29,30,32],walker:[2,3],wang:4,want:[0,1,2,3,4,5,7,8,9,10,12,13,15,17,18,19,21,22,24,26,30,35],warn:[7,8],warrant:5,wast:10,water:[16,17,25,26],wave:[15,24],wavelet:8,weak:2,wealth:[0,11,20],weather:[7,17,26],web:[4,11,20,28,30],websit:[4,12,21,30],webster:2,wedg:[3,8],wednesdai:28,week:1,weekli:28,weight:[0,1,2,3,5,6,10,14,23],weight_vari:7,weigth:1,welcom:[8,29,30,31,34],well:[0,1,2,3,4,5,6,7,8,12,13,14,15,16,17,18,19,21,22,23,24,25,26,28,35],wendi:[34,36],went:8,were:[0,2,3,4,5,6,7,8,9,10,13,14,15,16,17,22,23,24,25,26],wessel:5,what:[1,2,6,7,9,10,11,12,13,14,15,16,17,20,21,22,23,24,25,26,28,29],whatev:[0,10],when:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,20,21,22,23,24,25,26,28,29,34,36],whenev:3,where:[0,1,2,3,4,6,7,8,9,10,11,12,13,14,15,16,17,20,21,22,23,24,25,26],wherea:[3,5,13,14,16,22,23,25,34,36],wherein:7,whether:[0,2,3,4,6,13,14,15,16,22,23,24,25,30,34,36],which:[0,1,2,4,5,6,8,9,10,11,12,13,14,15,16,17,20,21,22,23,24,25,26,29,34,36],whichev:10,who:[0,4,29],whoever:[2,30],whole:[1,7,9,10],whose:[2,3,5,14,15,23,24],whow:[5,9],why:[0,4,13,17,22,26],wide:[4,5,7,10,12,13,15,21,22,24],widehat:5,wider:[0,17,26],width:[2,4,5,8,10],wieringen:5,wiki:29,willing:0,win:[2,16,25],wind:[13,15,22,24],winther:1,wiscons:9,wise:[5,7],wish:[0,1,2,4,6,8,9,12,13,15,16,17,21,22,24,25,26,31],wither:5,within:[0,1,2,3,4,6,7,10,13,22,29,31],without:[0,2,4,5,7,8,9,12,13,15,21,22,24,28,29],witth:[15,24],won:[2,4,5],wonder:8,word:[3,4,5,9,10,17,26],wordpress:2,work:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,15,16,18,19,20,21,22,24,25,28,30,34,35,36],workflow:31,world:[5,8,9,10,13,22],worldwid:[4,17,26],wors:[4,5,7,10],would:[0,2,3,4,5,6,7,8,9,10,12,13,14,15,16,17,21,22,23,24,25,26,28,30],wrap:[4,5,12,21,28],write:[0,1,3,4,5,7,8,11,12,14,15,16,17,20,21,23,24,25,26,27,32,34,36],writer:7,written:[0,1,3,4,5,7,9,11,12,13,14,15,16,20,21,22,23,24,25,34,36],wrong:[7,8,11,20,28],wrongli:3,wrote:[0,5,9],www:[4,7,12,21,28],wx_1:8,x0s:8,x1_exampl:8,x1d:8,x1s:8,x2d:[8,9],x2d_train:9,x2dsl:9,x2s:8,x3s:8,x_0:[2,4,5,9,12,13,14,15,21,22,23,24],x_1:[0,1,2,3,4,5,6,8,9,12,14,21,23],x_2:[0,1,2,3,4,5,6,8,9,12,14,21,23],x_3:[3,8],x_center:9,x_data:7,x_data_ful:7,x_f:[14,23],x_hidden:1,x_i:[1,3,4,5,6,7,8,9,13,14,22,23],x_input:1,x_ix_:5,x_iy_i:8,x_j:[1,3,7,8],x_jy_j:8,x_k:[3,7],x_l:3,x_m:[3,7],x_n:[0,1,3,5,7,8,9,15,24],x_output:1,x_p:[6,15,24],x_pred:[1,10],x_prev:1,x_reduc:9,x_scale:8,x_test:[5,7,9,10],x_test_own:5,x_test_scal:[5,9],x_tot:[1,10],x_train:[1,4,5,7,9,10],x_train_own:5,x_train_scal:[5,9],xarrai:[4,12,21],xavier:7,xcode:[4,12,21],xerr:2,xf_y:[14,23],xfinal:[16,25],xi_1:8,xi_:8,xi_i:8,xiniti:[16,25],xlabel:[1,3,4,5,8,10,11,12,20,21],xlim:5,xnew:4,xpd:[5,9],xplot:4,xtest:5,xtick:[5,8,10],xtrain:5,xytext:8,y_0:[4,5,9,12,13,14,21,22,23],y_1:[4,5,8,9,14,15,23,24],y_1y_1:8,y_1y_1k:8,y_1y_2:8,y_1y_2k:8,y_1y_n:8,y_1y_nk:8,y_2:[4,5,8,9,14,23],y_2y_1:8,y_2y_1k:8,y_2y_2:8,y_2y_2k:8,y_3:4,y_data:[5,7],y_data_ful:7,y_decis:8,y_i:[4,5,6,7,8,9,12,14,15,21,23,24],y_ix_:5,y_ix_i:[6,8],y_iy_jk:8,y_j:[5,7,8],y_k:7,y_model:[1,5,10],y_n:[8,14,23],y_ny_1:8,y_ny_1k:8,y_ny_2:8,y_ny_2k:8,y_ny_n:8,y_ny_nk:8,y_p:[13,22],y_pred:[1,5,7,8,10],y_test:[1,5,7,9,10],y_test_onehot:7,y_test_predict:5,y_tot:[1,10],y_train:[1,4,5,7,9,10],y_train_onehot:7,y_train_predict:5,yanalyt:[12,21],year:[0,2,4,14,15,23,24],yes:[0,5,6],yet:[0,5,7,8,9,12,13,16,21,22,25],yf_x:[14,23],yield:[1,2,3,4,5,7,8,12,14,15,16,21,23,24,25],ylabel:[1,3,4,5,8,10,11,12,20,21],ylim:[5,10,15,24],yoshua:7,you:[0,1,2,3,4,5,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,28,30,31,33,34,35,36],young:[0,4],your:[0,1,2,3,4,5,7,8,9,11,12,13,14,15,17,18,19,20,21,22,23,24,26,28,30,31,35],yourself:[4,9,16,17,25,26],ypred:5,ypredict:[4,5],yridg:5,ytest:5,ytick:[5,8,10],ytild:5,ytildenp:5,ytrain:5,z_0:[4,13,22],z_1:4,z_2:4,z_c:7,z_h:7,z_hidden:1,z_i:7,z_k:7,z_m:7,z_o:7,z_out:7,z_output:1,zaman:3,zero:[1,2,3,4,5,7,8,9,10,11,12,13,14,15,16,17,20,21,22,23,24,25,26],zeros_lik:2,zillion:0,zip:5,zm_h:4,zone:5,zze:[16,25]},titles:["Introduction to Applied Data Analysis and Machine Learning","9. Recurrent Neural Networks","11. Data Analysis and Machine Learning:","1. Elements of Probability Theory and Statistical Data Analysis","2. Getting started, our first data and Machine Learning encounters","3. Linear Regression and more Advanced Regression Analysis","4. Logistic Regression","5. Neural networks, from the simple perceptron to deep learning","6. Support Vector Machines, overarching aims","7. Dimensionality Reduction","8. Convolutional Neural Networks","Introduction","Space, Time, Motion, Reference Frames  and Reminder on vectors and other mathematical quantities","Basic Steps of Scientific Investigations","Work, Energy, Momentum and Conservation laws","Harmonic Oscillator","Two-body Problems","Non-inertial Frames, Translation and Rotating Coordinate Systems","Content with notebooks","Content with notebooks","Introduction","Space, Time, Motion, Reference Frames  and Reminder on vectors and other mathematical quantities","Basic Steps of Scientific Investigations","Work, Energy, Momentum and Conservation laws","Harmonic Oscillator","Two-body Problems","Non-inertial Frames, Translation and Rotating Coordinate Systems","Content in Jupyter Book","PHY321, Classical Mechanics I, Michigan State University, Spring 2021","Code of Conduct","Contributing","LectureNotes","Content in Jupyter Book","Welcome to your Jupyter Book","Markdown Files","Content with notebooks","Markdown Files"],titleterms:{"break":28,"case":[3,8,12,21],"class":6,"final":[1,7,9,10,12,13,14,21,22,23,28],"function":[1,3,5,6,7,8,9],"import":[3,4,10,12,21],"new":[1,10],"transient":[15,24],"try":[1,10],Adding:[5,8,14,23,34,36],And:[4,13,22],Going:[13,22],Ising:5,NNs:10,ODE:1,ODEs:1,OLS:5,One:[13,22],The:[1,2,3,5,6,7,8,9,10,13,14,15,17,22,23,24,26],Then:[13,22],Useful:[4,12,21],Using:[1,7,34,36],With:[1,2,10],about:[2,4],abov:[11,20],acceler:[12,13,17,21,22,26],act:[13,22],activ:7,actual:3,add:10,adjust:7,advanc:5,again:10,aim:[5,8],air:[12,13,21,22],aka:5,algebra:[5,12,17,21,26],algorithm:[5,7,9,14,23],all:8,altern:[15,24],analysi:[0,2,3,5,9],analyt:[1,12,21],angular:[14,23],anoth:5,appli:0,approach:[0,2,5,8,13,22],approxim:7,april:28,architectur:7,arrai:[4,12,21],artifici:7,asap:28,aspect:0,assumpt:5,attempt:[8,12,21],attribut:29,autocorrel:3,autograd:1,automat:1,averag:[12,21],back:[1,7,8,9],backend:7,background:5,ballist:[13,22],basebal:[12,21],basic:[4,5,6,9,12,13,14,21,22,23],bay:2,bayesian:2,befor:9,best:2,beta:[5,15,24],better:[3,5,8],bia:5,bias:[0,7],binari:7,bind:[4,5],binomi:3,block:[3,13,15,18,19,22,24,35],bodi:[16,25],boldsymbol:5,book:[27,31,32,33],bootstrap:[3,5],boston:5,breast:9,brief:[4,5,10],bring:7,bug:30,build:[7,10,14,23,31],buld:[14,23],calcul:3,can:5,cancer:9,center:[9,16,25],central:3,centrifug:[16,17,25,26],chain:7,chi:5,choic:[0,7],choos:7,cifar01:10,citat:[34,36],classic:[9,13,15,22,24,28],classif:[7,9],classifi:8,cnn:10,code:[1,2,3,5,7,8,9,12,14,15,16,18,19,21,23,24,25,29,30,34,35,36],coin:2,collect:[7,10],compact:[12,21],compar:1,compil:10,compon:9,comput:[3,9,12,21],computation:5,concept:[3,11,20],conduct:[29,30],confid:2,conserv:[14,23],constant:[12,21],content:[18,19,27,32,35],continu:[3,9],contn:[13,22],contribut:30,contributor:31,convex:8,convolut:[7,10],coordin:[17,26],corioli:[17,26],correl:[3,5,9,10],cost:[1,7],cote:[14,23],count:2,coupl:[14,23],covari:[2,3,5,9],credit:31,critic:[15,24],cromer:[12,14,21,23],cross:[5,16,25],cumul:3,cython:4,damp:[15,24],dampl:[15,24],data:[0,1,2,3,4,5,7,9,10],dataset:[7,10],date:28,decai:1,decompos:5,decomposit:[5,9],deep:[1,7],defin:[1,7,13,22],definit:[3,7],degre:[5,9],demonstr:3,dens:[5,10],deriv:[7,14,15,16,23,24,25],descent:[1,7],design:[5,9],detail:[1,5,8,10,13,22],develop:7,deviat:3,diagon:9,dice:3,differ:[2,8],differenti:[1,12,13,15,21,22,24],diffus:1,dimens:[12,13,16,21,22,25],dimension:[1,5,8,9],direct:[34,36],discret:[3,12,14,21,23],disguis:3,distribut:3,document:30,doing:7,domain:3,don:10,down:7,driven:[15,24],earth:[14,23],economi:5,effect:[16,17,25,26],element:[2,3,12,21],ellipt:[16,25],encount:[4,12,21],energi:[4,5,14,23],enforc:29,equal:[14,23],equat:[1,5,7,8,12,13,14,15,21,22,23,24],error:[5,14,23],essenti:[],estim:2,etc:[4,10],ethic:0,euler:[1,12,14,17,21,23,26],evalu:[7,10],event:3,exampl:[1,2,3,4,5,7,8,9,10,11,12,14,16,20,21,23,25],execut:[34,36],expect:[3,5],expens:5,experi:3,exponenti:[1,3],express:[5,14,23],extens:[12,21],extrapol:[1,10],extrem:[],factor:[15,24],fair:2,fall:[12,13,17,21,22,26],famili:[5,7],famou:[3,4,12,21],fantast:5,featur:[4,5,8,9,12,21,30],februari:28,feed:7,feedback:30,few:2,field:[13,22],file:[10,34,36],find:[5,12,21],first:[4,7,8,12,13,21,22],fit:[4,5],fix:[5,30],flux:2,fold:5,forc:[13,15,16,17,22,24,25,26],format:[1,10],forward:[1,7],foucault:[17,26],fouri:[15,24],fourier:[15,24],frame:[12,17,21,26],frank:[5,9],freedom:[5,9],frequentist:[0,2],friend:2,from:[2,5,7,10,17,26],full:[1,7,10],funtion:7,galilean:[17,26],gaussian:3,gener:[3,5],geometr:9,get:[4,8,30],going:5,good:0,grade:28,gradient:[1,7],gravit:[16,25],growth:1,handl:[0,4,12,21],happen:5,harmon:[15,16,24,25],height:[12,21],hidden:1,histori:2,host:31,hous:5,how:[3,5,8],hyperbol:7,hyperparamet:7,hyperplan:8,idea:9,identifi:[13,22],iii:1,illustr:[2,11,20],imag:10,implement:[1,3,7,12,21,30],improv:7,includ:[6,12,21],increment:9,inerti:[17,26],infer:2,input:1,instal:[4,12,21],integr:[12,14,21,23],interpret:[5,9],interv:2,introduc:[5,9],introduct:[0,4,6,11,20],investig:[13,22],its:3,jackknif:[3,5],jacobian:1,januari:28,julia:4,jupyt:[27,32,33],kera:[7,10],kernel:[8,9],known:[11,20],kutta:[15,24],label:[12,21],lagrangian:8,lambda:5,languag:0,lasso:5,last:8,law:[13,14,22,23],layer:[1,7,10],learn:[0,1,2,4,5,7,9,28],least:5,lectur:28,lecturenot:31,librari:[4,12,21],likelihood:6,limit:[3,7],linear:[4,5,8,17,26],link:[5,9,28],list:7,lle:9,logist:[6,7,9],machin:[0,2,4,8,9],made:5,magnet:[13,22],mai:28,main:3,mani:[7,11,20],march:28,markdown:[18,19,34,35,36],mass:[4,16,25],materi:28,math:5,mathemat:[5,7,8,12,14,21,23],matric:[4,12,21],matrix:[3,4,5,7,9,12,21],matter:5,maximum:6,mean:[3,9],measur:2,mechan:28,medic:10,meet:[3,4,12,21],mercer:8,mersenn:3,method:[3,5,12,14,15,21,23,24],michigan:28,minim:[1,8],mlp:7,mnist:10,model:[0,2,4,5,7,10,12,13,21,22],moment:3,momentum:[14,23],moon:8,more:[1,4,5,6,8,9,10,12,13,15,21,22,24],motion:[12,13,16,21,22,25],multi:7,multidisciplinari:[],multilay:7,multipl:[2,7],multipli:8,myst:[18,19,34,35,36],name:3,need:[4,12,21],network:[1,2,4,7,10],neural:[1,2,4,7,10],neuron:[7,10],newton:[13,14,22,23],non:[8,17,26],normal:3,notat:7,note:[2,28],notebook:[18,19,35],now:[5,9],nuclear:[4,5],numba:4,number:[1,3,5,9],numer:[1,3,14,15,23,24],numpi:[4,12,21],object:[7,13,17,22,26],observ:3,obtain:9,omega_0:[15,24],one:[1,5,7,12,13,21,22],ones:7,optim:[5,7,8],optmiz:8,orbit:[16,25],ordinari:[1,5],organ:4,orient:7,oscil:[15,16,24,25],other:[1,3,7,9,10,12,21,28],our:[3,4,5,7,9,12,13,15,21,22,24,29],outcom:[0,28],output:[1,3,18,19,35],overarch:[1,5,8,9,10],overdamp:[15,24],own:[5,9],packag:[4,12,21],panda:[4,5,9,12,21],paramet:[2,5,6],parametr:2,part:[10,13,22],partial:1,pass:7,pca:9,pdf:3,pendulum:[13,15,17,22,24,26],perceptron:7,perform:[5,7],period:[3,15,24],perspect:[7,17,26],philosophi:[14,23],photon:2,phy321:28,pledg:29,point:[1,10],poisson:[1,3],popul:1,possibl:[1,28],potenti:[16,17,25,26],practic:[2,12,21,28],pre:[7,10],predict:[1,10],preprocess:[5,9],prerequisit:10,princip:9,principl:[15,24],prior:2,probabl:[0,3],problem:[1,5,7,8,14,16,23,25],procedur:5,process:[1,7,10],program:[0,1],projectil:[13,22],proof:9,propag:[1,7],properti:[3,5],pseudo:3,python:[2,3,4,12,21],quadratur:[14,23],quantiti:[12,21],quick:8,raindrop:[13,22],ran0:3,random:[3,9],read:4,real:4,realist:8,recip:[2,3],rectangl:[14,23],rectangular:[14,23],recurr:[1,7,10],reduc:[5,9],reduct:[9,10],refer:[12,21],reformul:[1,5],regress:[4,5,6,9],regular:[5,7,10],rel:[16,25],relev:7,relu:7,remind:[4,8,12,21],report:30,repositori:5,requir:1,resampl:5,resist:[12,13,21,22],reson:[15,24],resourc:1,respons:[15,24,29],result:[5,13,22],rewrit:[5,9],rich:[],ridg:5,rng:3,rnn:[1,10],role:[34,36],rotat:[17,26],rule:[7,14,23],run:10,rung:[15,24],rutherford:[16,25],same:5,sampl:[3,9],scale:[10,13,15,22,24],scan:10,scatter:[16,25],schedul:28,scheme:1,scientif:[13,22],scikit:[4,5,7,9],scope:29,second:[13,22],section:[16,25],see:4,select:[2,3],seri:[15,24],set:[1,5,7,8,10,13,22],setup:1,should:[3,7,9],sigmoid:7,simpl:[1,2,3,4,5,7,8,9,10,12,14,21,23],simpler:8,simpson:[14,23],singl:[2,13,22],singular:[5,9],sinusoid:[15,24],situat:3,size:5,slide:[13,15,22,24],small:0,soft:8,softer:[13,22],softmax:7,softwar:[4,12,21],solar:[14,23],solut:[1,12,16,21,25],solv:[1,8,12,13,15,21,22,24],some:[2,4,5,12,21],sound:10,space:[8,12,21],specif:1,specifi:1,split:[5,13,22],spring:28,squar:5,stabil:[16,25],standard:[3,29],star:2,start:[4,30],state:[5,28],statist:[2,3,5],step:[5,8,9,13,14,22,23],stochast:3,strong:10,studi:[14,15,23,24],submit:30,suit:5,sum:5,summari:4,sun:[14,23],superposit:[15,24],supervis:7,support:[8,9],svd:5,symmetri:[14,23],system:[13,14,17,22,23,26],systemat:10,teach:28,team:28,technic:1,techniqu:9,tensorflow:[7,10],term:7,test:[5,7],textbook:28,theorem:[2,3,7,8,9,17,26],theori:[2,3],thi:[2,28],thing:[1,10],think:9,third:[13,22],three:3,tidal:[16,25],tied:[13,22],time:[12,21],togeth:7,top:[7,10],toss:3,toward:9,tradeoff:5,train:[1,5,7,10],transform:[10,15,17,24,26],translat:[17,26],trapezoid:[14,23],tree:4,trial:1,truncat:[14,23],two:[6,8,13,16,22,25],type:[1,4,7,10],uncorrel:3,underdamp:[15,24],understand:5,uniform:3,unit:[12,21],univers:[7,28],updat:28,usag:31,use:[3,7],used:10,useful:5,using:[1,3,4,7],valid:5,valu:[3,5,9],variabl:3,varianc:[3,5],variou:5,vector:[4,5,7,8,9,12,21],veloc:[12,14,21,23],verifi:10,verlet:[14,23],view:[1,5,9,10],visual:[7,10],volum:10,wai:[13,22],wall:[13,22],wave:1,week:28,weight:7,welcom:33,well:[10,11,20],what:[0,3,4,5,8,34,36],where:5,which:[3,7],why:[1,2,3,5,7,9,10],width:[15,24],wood:4,word:2,work:[14,23],write:[9,30],your:[33,34,36],z_j:7}})
\ No newline at end of file
diff --git a/doc/src/LectureNotes/_build/html/testbook/_build/jupyter_execute/chapter1.html b/doc/src/LectureNotes/_build/html/testbook/_build/jupyter_execute/chapter1.html
index 643e65252..86e67674f 100644
--- a/doc/src/LectureNotes/_build/html/testbook/_build/jupyter_execute/chapter1.html
+++ b/doc/src/LectureNotes/_build/html/testbook/_build/jupyter_execute/chapter1.html
@@ -123,6 +123,26 @@
    8. Convolutional Neural Networks
   
  
+ 
  • 
+  
+   9. Recurrent Neural Networks
+  
+ 
    • 
+ 
  • 
+  
+   10. Solving ODEs with Deep Learning
+  
+ 
    • 
+ 
  • 
+  
+   11. Data Analysis and Machine Learning:
+  
+ 
    • 
+ 
  • 
+  
+   12. Elements of Bayesian theory and Bayesian Neural Networks
+  
+ 
    • 
 
 
 
diff --git a/doc/src/LectureNotes/_build/html/testbook/_build/jupyter_execute/chapter2.html b/doc/src/LectureNotes/_build/html/testbook/_build/jupyter_execute/chapter2.html
index 2a8c644e3..fd278804d 100644
--- a/doc/src/LectureNotes/_build/html/testbook/_build/jupyter_execute/chapter2.html
+++ b/doc/src/LectureNotes/_build/html/testbook/_build/jupyter_execute/chapter2.html
@@ -123,6 +123,26 @@
    8. Convolutional Neural Networks
   
  
+ 
  • 
+  
+   9. Recurrent Neural Networks
+  
+ 
    • 
+ 
  • 
+  
+   10. Solving ODEs with Deep Learning
+  
+ 
    • 
+ 
  • 
+  
+   11. Data Analysis and Machine Learning:
+  
+ 
    • 
+ 
  • 
+  
+   12. Elements of Bayesian theory and Bayesian Neural Networks
+  
+ 
    • 
 
 
 
@@ -1553,8 +1573,8 @@ developed in the 1970s, namely EISPACK and LINPACK.  We describe them shortly he
 
    
 
    
 
    
-
    [-1.17029256  1.16440383 -0.52324284  0.91883189 -0.05688344 -0.80125589
-  1.85759035 -1.11063616  0.67761635  0.09188807]
+
    [ 1.8413241  -1.44931127  0.91189378  1.82553587  0.69940044 -0.65726179
+  1.72526513  1.82825747 -1.34003257 -0.5817835 ]
 
    
 
    
 
    
diff --git a/doc/src/LectureNotes/_build/html/testbook/_build/jupyter_execute/chapter3.html b/doc/src/LectureNotes/_build/html/testbook/_build/jupyter_execute/chapter3.html
index 26cb92774..fc7200445 100644
--- a/doc/src/LectureNotes/_build/html/testbook/_build/jupyter_execute/chapter3.html
+++ b/doc/src/LectureNotes/_build/html/testbook/_build/jupyter_execute/chapter3.html
@@ -123,6 +123,26 @@
    8. Convolutional Neural Networks
   
  
+ 
  • 
+  
+   9. Recurrent Neural Networks
+  
+ 
    • 
+ 
  • 
+  
+   10. Solving ODEs with Deep Learning
+  
+ 
    • 
+ 
  • 
+  
+   11. Data Analysis and Machine Learning:
+  
+ 
    • 
+ 
  • 
+  
+   12. Elements of Bayesian theory and Bayesian Neural Networks
+  
+ 
    • 
 
 
 
diff --git a/doc/src/LectureNotes/_build/html/testbook/_build/jupyter_execute/chapter4.html b/doc/src/LectureNotes/_build/html/testbook/_build/jupyter_execute/chapter4.html
index 5a8a0e5a1..391128f41 100644
--- a/doc/src/LectureNotes/_build/html/testbook/_build/jupyter_execute/chapter4.html
+++ b/doc/src/LectureNotes/_build/html/testbook/_build/jupyter_execute/chapter4.html
@@ -123,6 +123,26 @@
    8. Convolutional Neural Networks
   
  
+ 
  • 
+  
+   9. Recurrent Neural Networks
+  
+ 
    • 
+ 
  • 
+  
+   10. Solving ODEs with Deep Learning
+  
+ 
    • 
+ 
  • 
+  
+   11. Data Analysis and Machine Learning:
+  
+ 
    • 
+ 
  • 
+  
+   12. Elements of Bayesian theory and Bayesian Neural Networks
+  
+ 
    • 
 
 
 
diff --git a/doc/src/LectureNotes/_build/html/testbook/_build/jupyter_execute/chapter5.html b/doc/src/LectureNotes/_build/html/testbook/_build/jupyter_execute/chapter5.html
index 335a300c2..e31cecfb7 100644
--- a/doc/src/LectureNotes/_build/html/testbook/_build/jupyter_execute/chapter5.html
+++ b/doc/src/LectureNotes/_build/html/testbook/_build/jupyter_execute/chapter5.html
@@ -123,6 +123,26 @@
    8. Convolutional Neural Networks
   
  
+ 
  • 
+  
+   9. Recurrent Neural Networks
+  
+ 
    • 
+ 
  • 
+  
+   10. Solving ODEs with Deep Learning
+  
+ 
    • 
+ 
  • 
+  
+   11. Data Analysis and Machine Learning:
+  
+ 
    • 
+ 
  • 
+  
+   12. Elements of Bayesian theory and Bayesian Neural Networks
+  
+ 
    • 
 
 
 
diff --git a/doc/src/LectureNotes/_build/html/testbook/_build/jupyter_execute/chapter6.html b/doc/src/LectureNotes/_build/html/testbook/_build/jupyter_execute/chapter6.html
index 71426d615..d6d910bbd 100644
--- a/doc/src/LectureNotes/_build/html/testbook/_build/jupyter_execute/chapter6.html
+++ b/doc/src/LectureNotes/_build/html/testbook/_build/jupyter_execute/chapter6.html
@@ -123,6 +123,26 @@
    8. Convolutional Neural Networks
   
  
+ 
  • 
+  
+   9. Recurrent Neural Networks
+  
+ 
    • 
+ 
  • 
+  
+   10. Solving ODEs with Deep Learning
+  
+ 
    • 
+ 
  • 
+  
+   11. Data Analysis and Machine Learning:
+  
+ 
    • 
+ 
  • 
+  
+   12. Elements of Bayesian theory and Bayesian Neural Networks
+  
+ 
    • 
 
 
 
diff --git a/doc/src/LectureNotes/_build/html/testbook/_build/jupyter_execute/chapter7.html b/doc/src/LectureNotes/_build/html/testbook/_build/jupyter_execute/chapter7.html
index 135392f9b..6940fe2db 100644
--- a/doc/src/LectureNotes/_build/html/testbook/_build/jupyter_execute/chapter7.html
+++ b/doc/src/LectureNotes/_build/html/testbook/_build/jupyter_execute/chapter7.html
@@ -123,6 +123,26 @@
    8. Convolutional Neural Networks
   
  
+ 
  • 
+  
+   9. Recurrent Neural Networks
+  
+ 
    • 
+ 
  • 
+  
+   10. Solving ODEs with Deep Learning
+  
+ 
    • 
+ 
  • 
+  
+   11. Data Analysis and Machine Learning:
+  
+ 
    • 
+ 
  • 
+  
+   12. Elements of Bayesian theory and Bayesian Neural Networks
+  
+ 
    • 
 
 
 
diff --git a/doc/src/LectureNotes/_build/html/testbook/_build/jupyter_execute/lecturenotes/lecturenotes/notebooks.html b/doc/src/LectureNotes/_build/html/testbook/_build/jupyter_execute/lecturenotes/lecturenotes/notebooks.html
index 5d794f596..2c3e21ad8 100644
--- a/doc/src/LectureNotes/_build/html/testbook/_build/jupyter_execute/lecturenotes/lecturenotes/notebooks.html
+++ b/doc/src/LectureNotes/_build/html/testbook/_build/jupyter_execute/lecturenotes/lecturenotes/notebooks.html
@@ -123,6 +123,26 @@
    8. Convolutional Neural Networks
   
  
+ 
  • 
+  
+   9. Recurrent Neural Networks
+  
+ 
    • 
+ 
  • 
+  
+   10. Solving ODEs with Deep Learning
+  
+ 
    • 
+ 
  • 
+  
+   11. Data Analysis and Machine Learning:
+  
+ 
    • 
+ 
  • 
+  
+   12. Elements of Bayesian theory and Bayesian Neural Networks
+  
+ 
    • 
 
 
 
diff --git a/doc/src/LectureNotes/_build/html/testbook/_build/jupyter_execute/notebooks.html b/doc/src/LectureNotes/_build/html/testbook/_build/jupyter_execute/notebooks.html
index 7962b9748..bd8c407a7 100644
--- a/doc/src/LectureNotes/_build/html/testbook/_build/jupyter_execute/notebooks.html
+++ b/doc/src/LectureNotes/_build/html/testbook/_build/jupyter_execute/notebooks.html
@@ -123,6 +123,26 @@
    8. Convolutional Neural Networks
   
  
+ 
  • 
+  
+   9. Recurrent Neural Networks
+  
+ 
    • 
+ 
  • 
+  
+   10. Solving ODEs with Deep Learning
+  
+ 
    • 
+ 
  • 
+  
+   11. Data Analysis and Machine Learning:
+  
+ 
    • 
+ 
  • 
+  
+   12. Elements of Bayesian theory and Bayesian Neural Networks
+  
+ 
    • 
 
 
 
diff --git a/doc/src/LectureNotes/_build/html/testbook/chapter1.html b/doc/src/LectureNotes/_build/html/testbook/chapter1.html
index ccae0322d..9e69a6d47 100644
--- a/doc/src/LectureNotes/_build/html/testbook/chapter1.html
+++ b/doc/src/LectureNotes/_build/html/testbook/chapter1.html
@@ -123,6 +123,26 @@
    8. Convolutional Neural Networks
   
  
+ 
  • 
+  
+   9. Recurrent Neural Networks
+  
+ 
    • 
+ 
  • 
+  
+   10. Solving ODEs with Deep Learning
+  
+ 
    • 
+ 
  • 
+  
+   11. Data Analysis and Machine Learning:
+  
+ 
    • 
+ 
  • 
+  
+   12. Elements of Bayesian theory and Bayesian Neural Networks
+  
+ 
    • 
 
 
 
diff --git a/doc/src/LectureNotes/_build/html/testbook/chapter2.html b/doc/src/LectureNotes/_build/html/testbook/chapter2.html
index 0b6aa8244..66fbb90bb 100644
--- a/doc/src/LectureNotes/_build/html/testbook/chapter2.html
+++ b/doc/src/LectureNotes/_build/html/testbook/chapter2.html
@@ -123,6 +123,26 @@
    8. Convolutional Neural Networks
   
  
+ 
  • 
+  
+   9. Recurrent Neural Networks
+  
+ 
    • 
+ 
  • 
+  
+   10. Solving ODEs with Deep Learning
+  
+ 
    • 
+ 
  • 
+  
+   11. Data Analysis and Machine Learning:
+  
+ 
    • 
+ 
  • 
+  
+   12. Elements of Bayesian theory and Bayesian Neural Networks
+  
+ 
    • 
 
 
 
@@ -1553,8 +1573,8 @@ developed in the 1970s, namely EISPACK and LINPACK.  We describe them shortly he
 
    
 
    
 
    
-
    [-0.15903306 -1.03373868  0.31582494 -0.65395262  0.96308377 -0.89760255
-  1.62466988  1.27946914 -0.71331923 -0.2179017 ]
+
    [-1.04569783  1.68833083  1.59472197 -0.22027464 -1.12477388  0.78480985
+  0.8799874  -0.43301111 -0.53140431  0.2976373 ]
 
    
 
    
 
    
diff --git a/doc/src/LectureNotes/_build/html/testbook/chapter3.html b/doc/src/LectureNotes/_build/html/testbook/chapter3.html
index 927ff395d..5de2bc5ba 100644
--- a/doc/src/LectureNotes/_build/html/testbook/chapter3.html
+++ b/doc/src/LectureNotes/_build/html/testbook/chapter3.html
@@ -123,6 +123,26 @@
    8. Convolutional Neural Networks
   
  
+ 
  • 
+  
+   9. Recurrent Neural Networks
+  
+ 
    • 
+ 
  • 
+  
+   10. Solving ODEs with Deep Learning
+  
+ 
    • 
+ 
  • 
+  
+   11. Data Analysis and Machine Learning:
+  
+ 
    • 
+ 
  • 
+  
+   12. Elements of Bayesian theory and Bayesian Neural Networks
+  
+ 
    • 
 
 
 
diff --git a/doc/src/LectureNotes/_build/html/testbook/chapter4.html b/doc/src/LectureNotes/_build/html/testbook/chapter4.html
index 0801201a6..72cd947b4 100644
--- a/doc/src/LectureNotes/_build/html/testbook/chapter4.html
+++ b/doc/src/LectureNotes/_build/html/testbook/chapter4.html
@@ -123,6 +123,26 @@
    8. Convolutional Neural Networks
   
  
+ 
  • 
+  
+   9. Recurrent Neural Networks
+  
+ 
    • 
+ 
  • 
+  
+   10. Solving ODEs with Deep Learning
+  
+ 
    • 
+ 
  • 
+  
+   11. Data Analysis and Machine Learning:
+  
+ 
    • 
+ 
  • 
+  
+   12. Elements of Bayesian theory and Bayesian Neural Networks
+  
+ 
    • 
 
 
 
diff --git a/doc/src/LectureNotes/_build/html/testbook/chapter5.html b/doc/src/LectureNotes/_build/html/testbook/chapter5.html
index 901060374..513936694 100644
--- a/doc/src/LectureNotes/_build/html/testbook/chapter5.html
+++ b/doc/src/LectureNotes/_build/html/testbook/chapter5.html
@@ -123,6 +123,26 @@
    8. Convolutional Neural Networks
   
  
+ 
  • 
+  
+   9. Recurrent Neural Networks
+  
+ 
    • 
+ 
  • 
+  
+   10. Solving ODEs with Deep Learning
+  
+ 
    • 
+ 
  • 
+  
+   11. Data Analysis and Machine Learning:
+  
+ 
    • 
+ 
  • 
+  
+   12. Elements of Bayesian theory and Bayesian Neural Networks
+  
+ 
    • 
 
 
 
diff --git a/doc/src/LectureNotes/_build/html/testbook/chapter6.html b/doc/src/LectureNotes/_build/html/testbook/chapter6.html
index 5df638df7..f49b0d517 100644
--- a/doc/src/LectureNotes/_build/html/testbook/chapter6.html
+++ b/doc/src/LectureNotes/_build/html/testbook/chapter6.html
@@ -123,6 +123,26 @@
    8. Convolutional Neural Networks
   
  
+ 
  • 
+  
+   9. Recurrent Neural Networks
+  
+ 
    • 
+ 
  • 
+  
+   10. Solving ODEs with Deep Learning
+  
+ 
    • 
+ 
  • 
+  
+   11. Data Analysis and Machine Learning:
+  
+ 
    • 
+ 
  • 
+  
+   12. Elements of Bayesian theory and Bayesian Neural Networks
+  
+ 
    • 
 
 
 
diff --git a/doc/src/LectureNotes/_build/html/testbook/chapter7.html b/doc/src/LectureNotes/_build/html/testbook/chapter7.html
index acdf58c52..065a34b70 100644
--- a/doc/src/LectureNotes/_build/html/testbook/chapter7.html
+++ b/doc/src/LectureNotes/_build/html/testbook/chapter7.html
@@ -123,6 +123,26 @@
    8. Convolutional Neural Networks
   
  
+ 
  • 
+  
+   9. Recurrent Neural Networks
+  
+ 
    • 
+ 
  • 
+  
+   10. Solving ODEs with Deep Learning
+  
+ 
    • 
+ 
  • 
+  
+   11. Data Analysis and Machine Learning:
+  
+ 
    • 
+ 
  • 
+  
+   12. Elements of Bayesian theory and Bayesian Neural Networks
+  
+ 
    • 
 
 
 
diff --git a/doc/src/LectureNotes/_build/html/testbook/content.html b/doc/src/LectureNotes/_build/html/testbook/content.html
index 8b3bcc931..68436a41a 100644
--- a/doc/src/LectureNotes/_build/html/testbook/content.html
+++ b/doc/src/LectureNotes/_build/html/testbook/content.html
@@ -123,6 +123,26 @@
    8. Convolutional Neural Networks
   
  
+ 
  • 
+  
+   9. Recurrent Neural Networks
+  
+ 
    • 
+ 
  • 
+  
+   10. Solving ODEs with Deep Learning
+  
+ 
    • 
+ 
  • 
+  
+   11. Data Analysis and Machine Learning:
+  
+ 
    • 
+ 
  • 
+  
+   12. Elements of Bayesian theory and Bayesian Neural Networks
+  
+ 
    • 
 
 
 
diff --git a/doc/src/LectureNotes/_build/html/testbook/intro.html b/doc/src/LectureNotes/_build/html/testbook/intro.html
index 4ebb9e5bc..4dee964ee 100644
--- a/doc/src/LectureNotes/_build/html/testbook/intro.html
+++ b/doc/src/LectureNotes/_build/html/testbook/intro.html
@@ -123,6 +123,26 @@
    8. Convolutional Neural Networks
   
  
+ 
  • 
+  
+   9. Recurrent Neural Networks
+  
+ 
    • 
+ 
  • 
+  
+   10. Solving ODEs with Deep Learning
+  
+ 
    • 
+ 
  • 
+  
+   11. Data Analysis and Machine Learning:
+  
+ 
    • 
+ 
  • 
+  
+   12. Elements of Bayesian theory and Bayesian Neural Networks
+  
+ 
    • 
 
 
 
diff --git a/doc/src/LectureNotes/_build/html/testbook/lecturenotes/CONDUCT.html b/doc/src/LectureNotes/_build/html/testbook/lecturenotes/CONDUCT.html
index 59ba705a2..99541d9d1 100644
--- a/doc/src/LectureNotes/_build/html/testbook/lecturenotes/CONDUCT.html
+++ b/doc/src/LectureNotes/_build/html/testbook/lecturenotes/CONDUCT.html
@@ -123,6 +123,26 @@
    8. Convolutional Neural Networks
   
  
+ 
  • 
+  
+   9. Recurrent Neural Networks
+  
+ 
    • 
+ 
  • 
+  
+   10. Solving ODEs with Deep Learning
+  
+ 
    • 
+ 
  • 
+  
+   11. Data Analysis and Machine Learning:
+  
+ 
    • 
+ 
  • 
+  
+   12. Elements of Bayesian theory and Bayesian Neural Networks
+  
+ 
    • 
 
 
 
diff --git a/doc/src/LectureNotes/_build/html/testbook/lecturenotes/CONTRIBUTING.html b/doc/src/LectureNotes/_build/html/testbook/lecturenotes/CONTRIBUTING.html
index 3b241a3d1..a5ce3a0c7 100644
--- a/doc/src/LectureNotes/_build/html/testbook/lecturenotes/CONTRIBUTING.html
+++ b/doc/src/LectureNotes/_build/html/testbook/lecturenotes/CONTRIBUTING.html
@@ -123,6 +123,26 @@
    8. Convolutional Neural Networks
   
  
+ 
  • 
+  
+   9. Recurrent Neural Networks
+  
+ 
    • 
+ 
  • 
+  
+   10. Solving ODEs with Deep Learning
+  
+ 
    • 
+ 
  • 
+  
+   11. Data Analysis and Machine Learning:
+  
+ 
    • 
+ 
  • 
+  
+   12. Elements of Bayesian theory and Bayesian Neural Networks
+  
+ 
    • 
 
 
 
diff --git a/doc/src/LectureNotes/_build/html/testbook/lecturenotes/README.html b/doc/src/LectureNotes/_build/html/testbook/lecturenotes/README.html
index 4189370a6..49dd40798 100644
--- a/doc/src/LectureNotes/_build/html/testbook/lecturenotes/README.html
+++ b/doc/src/LectureNotes/_build/html/testbook/lecturenotes/README.html
@@ -123,6 +123,26 @@
    8. Convolutional Neural Networks
   
  
+ 
  • 
+  
+   9. Recurrent Neural Networks
+  
+ 
    • 
+ 
  • 
+  
+   10. Solving ODEs with Deep Learning
+  
+ 
    • 
+ 
  • 
+  
+   11. Data Analysis and Machine Learning:
+  
+ 
    • 
+ 
  • 
+  
+   12. Elements of Bayesian theory and Bayesian Neural Networks
+  
+ 
    • 
 
 
 
diff --git a/doc/src/LectureNotes/_build/html/testbook/lecturenotes/lecturenotes/content.html b/doc/src/LectureNotes/_build/html/testbook/lecturenotes/lecturenotes/content.html
index 5cbfd6ce8..95e4b43c0 100644
--- a/doc/src/LectureNotes/_build/html/testbook/lecturenotes/lecturenotes/content.html
+++ b/doc/src/LectureNotes/_build/html/testbook/lecturenotes/lecturenotes/content.html
@@ -123,6 +123,26 @@
    8. Convolutional Neural Networks
   
  
+ 
  • 
+  
+   9. Recurrent Neural Networks
+  
+ 
    • 
+ 
  • 
+  
+   10. Solving ODEs with Deep Learning
+  
+ 
    • 
+ 
  • 
+  
+   11. Data Analysis and Machine Learning:
+  
+ 
    • 
+ 
  • 
+  
+   12. Elements of Bayesian theory and Bayesian Neural Networks
+  
+ 
    • 
 
 
 
diff --git a/doc/src/LectureNotes/_build/html/testbook/lecturenotes/lecturenotes/intro.html b/doc/src/LectureNotes/_build/html/testbook/lecturenotes/lecturenotes/intro.html
index e7c7c7138..47b359168 100644
--- a/doc/src/LectureNotes/_build/html/testbook/lecturenotes/lecturenotes/intro.html
+++ b/doc/src/LectureNotes/_build/html/testbook/lecturenotes/lecturenotes/intro.html
@@ -123,6 +123,26 @@
    8. Convolutional Neural Networks
   
  
+ 
  • 
+  
+   9. Recurrent Neural Networks
+  
+ 
    • 
+ 
  • 
+  
+   10. Solving ODEs with Deep Learning
+  
+ 
    • 
+ 
  • 
+  
+   11. Data Analysis and Machine Learning:
+  
+ 
    • 
+ 
  • 
+  
+   12. Elements of Bayesian theory and Bayesian Neural Networks
+  
+ 
    • 
 
 
 
diff --git a/doc/src/LectureNotes/_build/html/testbook/lecturenotes/lecturenotes/markdown.html b/doc/src/LectureNotes/_build/html/testbook/lecturenotes/lecturenotes/markdown.html
index 6f35fa6c4..70507d92b 100644
--- a/doc/src/LectureNotes/_build/html/testbook/lecturenotes/lecturenotes/markdown.html
+++ b/doc/src/LectureNotes/_build/html/testbook/lecturenotes/lecturenotes/markdown.html
@@ -123,6 +123,26 @@
    8. Convolutional Neural Networks
   
  
+ 
  • 
+  
+   9. Recurrent Neural Networks
+  
+ 
    • 
+ 
  • 
+  
+   10. Solving ODEs with Deep Learning
+  
+ 
    • 
+ 
  • 
+  
+   11. Data Analysis and Machine Learning:
+  
+ 
    • 
+ 
  • 
+  
+   12. Elements of Bayesian theory and Bayesian Neural Networks
+  
+ 
    • 
 
 
 
diff --git a/doc/src/LectureNotes/_build/html/testbook/lecturenotes/lecturenotes/notebooks.html b/doc/src/LectureNotes/_build/html/testbook/lecturenotes/lecturenotes/notebooks.html
index 2912b35e5..80c431b7f 100644
--- a/doc/src/LectureNotes/_build/html/testbook/lecturenotes/lecturenotes/notebooks.html
+++ b/doc/src/LectureNotes/_build/html/testbook/lecturenotes/lecturenotes/notebooks.html
@@ -123,6 +123,26 @@
    8. Convolutional Neural Networks
   
  
+ 
  • 
+  
+   9. Recurrent Neural Networks
+  
+ 
    • 
+ 
  • 
+  
+   10. Solving ODEs with Deep Learning
+  
+ 
    • 
+ 
  • 
+  
+   11. Data Analysis and Machine Learning:
+  
+ 
    • 
+ 
  • 
+  
+   12. Elements of Bayesian theory and Bayesian Neural Networks
+  
+ 
    • 
 
 
 
diff --git a/doc/src/LectureNotes/_build/html/testbook/markdown.html b/doc/src/LectureNotes/_build/html/testbook/markdown.html
index 9432c7af8..760066aa4 100644
--- a/doc/src/LectureNotes/_build/html/testbook/markdown.html
+++ b/doc/src/LectureNotes/_build/html/testbook/markdown.html
@@ -123,6 +123,26 @@
    8. Convolutional Neural Networks
   
  
+ 
  • 
+  
+   9. Recurrent Neural Networks
+  
+ 
    • 
+ 
  • 
+  
+   10. Solving ODEs with Deep Learning
+  
+ 
    • 
+ 
  • 
+  
+   11. Data Analysis and Machine Learning:
+  
+ 
    • 
+ 
  • 
+  
+   12. Elements of Bayesian theory and Bayesian Neural Networks
+  
+ 
    • 
 
 
 
diff --git a/doc/src/LectureNotes/_build/jupyter_execute/chapter1.ipynb b/doc/src/LectureNotes/_build/jupyter_execute/chapter1.ipynb
index bc7a8ae19..5a48fe5ec 100644
--- a/doc/src/LectureNotes/_build/jupyter_execute/chapter1.ipynb
+++ b/doc/src/LectureNotes/_build/jupyter_execute/chapter1.ipynb
@@ -416,6 +416,8 @@
     "chapter7.ipynb\n",
     "chapter8.ipynb\n",
     "chapter9.ipynb\n",
+    "chapter10.ipynb\n",
+    "chapter11.ipynb\n",
     "```\n"
    ]
   }
diff --git a/doc/src/LectureNotes/_build/jupyter_execute/chapter1.txt b/doc/src/LectureNotes/_build/jupyter_execute/chapter1.txt
index db9b269da..4711f18d7 100644
--- a/doc/src/LectureNotes/_build/jupyter_execute/chapter1.txt
+++ b/doc/src/LectureNotes/_build/jupyter_execute/chapter1.txt
@@ -405,4 +405,6 @@ chapter6.ipynb
 chapter7.ipynb
 chapter8.ipynb
 chapter9.ipynb
+chapter10.ipynb
+chapter11.ipynb
 ```
diff --git a/doc/src/LectureNotes/_build/jupyter_execute/chapter10.ipynb b/doc/src/LectureNotes/_build/jupyter_execute/chapter10.ipynb
new file mode 100644
index 000000000..627851844
--- /dev/null
+++ b/doc/src/LectureNotes/_build/jupyter_execute/chapter10.ipynb
@@ -0,0 +1,17610 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Recurrent Neural Networks\n",
+    "\n",
+    "[Overview video](https://www.youtube.com/watch?v=SEnXr6v2ifU&ab_channel=AlexanderAmini).\n",
+    "See also lecture on Thursday October 22 and examples from [week 42](https://compphysics.github.io/MachineLearning/doc/pub/week42/html/week42.html).\n",
+    "\n",
+    "[IN5400 at UiO Lecture](https://www.uio.no/studier/emner/matnat/ifi/IN5400/v20/material/week10/in5400_2020_week10_recurrent_neural_network.pdf)\n",
+    "\n",
+    "[CS231 at Stanford Lecture](https://www.youtube.com/watch?v=6niqTuYFZLQ&list=PLzUTmXVwsnXod6WNdg57Yc3zFx_f-RYsq&index=10&ab_channel=StanfordUniversitySchoolofEngineering)\n",
+    "\n",
+    "## Recurrent neural networks: Overarching view\n",
+    "\n",
+    "Till now our focus has been, including convolutional neural networks\n",
+    "as well, on feedforward neural networks. The output or the activations\n",
+    "flow only in one direction, from the input layer to the output layer.\n",
+    "\n",
+    "A recurrent neural network (RNN) looks very much like a feedforward\n",
+    "neural network, except that it also has connections pointing\n",
+    "backward. \n",
+    "\n",
+    "RNNs are used to analyze time series data such as stock prices, and\n",
+    "tell you when to buy or sell. In autonomous driving systems, they can\n",
+    "anticipate car trajectories and help avoid accidents. More generally,\n",
+    "they can work on sequences of arbitrary lengths, rather than on\n",
+    "fixed-sized inputs like all the nets we have discussed so far. For\n",
+    "example, they can take sentences, documents, or audio samples as\n",
+    "input, making them extremely useful for natural language processing\n",
+    "systems such as automatic translation and speech-to-text.\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "## Set up of an RNN\n",
+    "\n",
+    "\n",
+    "Text to come.\n",
+    "\n",
+    "\n",
+    "## A simple example"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD4CAYAAADvsV2wAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO2dd7wU1dnHf8/ewuXSO0hHQLAhiAgW7AZBgzGaaBLLGxNi7DGJsUSjMUZjTDRGI9ZYYomxoiIoNgSp0nsvl3oBufQL9+55/9iZ3TNnz/Sys7vn+/nA3Z2ZnXNm5sxznvOc5zwPMcagUCgUisInkesKKBQKhSIalMBXKBSKIkEJfIVCoSgSlMBXKBSKIkEJfIVCoSgSSnNdAStat27NunXrlutqKBQKRd7wzTffbGOMtZHti7XA79atG2bOnJnraigUCkXeQERrzfYpk45CoVAUCUrgKxQKRZGgBL5CoVAUCUrgKxQKRZGgBL5CoVAUCb4FPhFVENF0IppLRAuJ6F7JMUREjxHRCiKaR0QD/JarUCgUCncE4ZZZC+BMxtgeIioDMImIPmKMTeWOOQ9AL+3fiQCe1P4qFAqFIiJ8a/gsxR7ta5n2T4y5PBLAS9qxUwE0J6IOfstWFB/vzK7C3tq6XFdDochLArHhE1EJEc0BsBXAJ4yxacIhHQGs575Xadtk5xpFRDOJaGZ1dXUQ1VMUCN+s3YFf/Xcufv5SajHe69PXYcfegzmulUKRPwQi8Blj9Yyx4wB0AjCIiI4WDiHZz0zO9TRjbCBjbGCbNtLVwYoiZfeBlGb/9crtWLF1D257ez5ufG12jmtVPMxZvxMHDtXnuhoKHwTqpcMY2wngCwDDhF1VADpz3zsB2Bhk2Yri4mBdEgCwbU9tjmtSHGyuOYALn5iM296ah10HDuFfX6xAMqmy5eUbQXjptCGi5trnhgDOBrBEOGwMgCs0b53BAGoYY5v8lh02ySTDHe/Mx9LNu3NdFYUAkw8QFQHyxoz1eGVaKizL4Ac+BQDM31CDP76/CA+NW4rPlmzNZfUUHgjCS6cDgBeJqASpDuQNxtgHRHQNADDGRgMYC2A4gBUA9gH4vwDKDZ313+7Dq9PWYdLybZh46xkAgL21ddi4cz96tWuS49oVHzIRTySzFiqC4Na35gEAfjSoS3rbyuq9SGj3/FB9Mif1UnjHt8BnjM0D0F+yfTT3mQG4zm9ZuWLdjn3Yue8gmleWY9TLMzF5xXasfmC4EjYRsX7HPqzZvjfX1Sha6gXTzfKte0yOVMQdtdLWIb9/dwEAYPKK7QCyXwJFMKzbvg+jv1yJlI6Q4oyHv8Dlz003HMckt3/jzv045p7xWL5FmeCCpE619YJBCXyH7Dto9E5Q70A4/OI/3+DBj5Zgw8796W1WAmfxpl14aFxqyuijBZux+0AdznlkYuj1LCaSst5VkZcoge8R9RIEz4FD9Vi8aRcAYNd+yeIqk1v+ry9WpnarZxIKdqPZ6t21qPp2X0S1Ufgh1hmv4oQoTJTADx7exfKu9xagtq4eH9xwanqbnWeOeibh8PJU0wRKAIAT7p8AAFjz4IgoqqPwgdLwHSKKElHrOVSfxAo1mRUY36z9Fgs27DJsO1Sfuecy4a7kvXemr95hOkJ6aNzSiGujCAsl8B0ijmrF7/d/uBhn//1LbORszwp36IupzODdAGWySc2reGPcgs34wVNT8Mq0dbmuiiJklMC3gBcqWSYdQbpMXZXy3vl2n4rt4pUz//al5f46TsOXyXZl0vHGWs3lda1yfS14lMD3SL2JcEko3/zQOMhp+Eq4B4felhMJf2139rpvg6iOIkSUwHeIKGDMvit5Hx68yUcm75WXjjf00WqJz8b7vX99HUR1FCGiBL6EGWt24PfvzjdsE2VJUjA36/tJGhhU4QddDhmX8mcL96SFCU4hp3p3LbbsSnlHlfrU8BXxR7llSrhk9BQAwM9P7ZHeliXwhQ36N/GdOViXRF0yicpydau9UpogHKpnBoEvTtAyxgzPJMmAEiW/bNFdKgH/Jh1F/FEavgW8TBcF/B3vGEcAZiadS56agiPvHh9K/YqF0kSqmfJumdkdsHFbnTgEU9ji16SjiD9K4FvAC3kGoI7TML9YWo3Ln5uG+VU16Hbbh1hVLfdwmLt+J4CUn7Mim/lVNdhjk7KwVFPVjTb87DkUftvKrcrjxC1Kwy98lMC3gAlf7h+72LD/q+XbMHnlNsM2swiaP3hqSsC1y38O1SdxweOT8LMXZ1ja3PVMV1YmnSQzrsMd/thXQVa1KFA2/MJHCXyOHz87Ff+dkVl8wvvaMzB8rUXK5BHfEfXKOEcfQU1dtcN0ley0VZl7Xm+w0QtzKEy5avqlRAn8SFm6ebft6DZolMDnmLxiO373VsY2nzTY8OW/Ef3uVYx85xgWtpkcs3Z7JiiX1ZxKkjG10tYnXtaQqFDU3vj35NX4zqMT8dN/zzBsr6tP4h8TlmNvSB1BECkOOxPR50S0mIgWEtFNkmNOJ6IaIpqj/bvbb7lRwBsJzEwObgT83to63PfBIpUIWsNqJbOOwfOGk+hiLCNx0lbhHrcaPhEchaJ+6suV6Hbbh6rdc9z7/iIAwPQ1xrm9t2dvwCMTluGRT5aFUm4QvoJ1AH7NGJtFRE0AfENEnzDGFgnHfcUYOz+A8iKDd/QwkyXiO2Jli37yi5V4btJqtG9agZ8P7WF6XLEgTorLqDc5RqbhK997f4Q1aTv6y1T46n0H61FRVhJKGYVCreaYsD+kztG3hs8Y28QYm6V93g1gMYCOfs8bBwwCyaFJx0rk6KEBzMIyFBv8XVhmYhowLqbitosL35LZncD4hZt91rCwETvIsEz4einK2JlN1NMmgdrwiagbUvltp0l2DyGiuUT0EREdZXGOUUQ0k4hmVldXB1k914gaqCwee7aGn1qp+96cDVnH6i+Yavgp+Ps74rFJ0mM+W7wl/Zm//2KnWc9YVqf8i5e/CaCWhQu/rgEIxiS2dfeBrDDh6VXoquFnkTVvErIyGNjyTyJqDOAtADczxnYJu2cB6MoY20NEwwG8C6CX7DyMsacBPA0AAwcODF0VXrChBrsP1GHI4a2y9hnMxI5t+AxXvzADuw7UYdEm423QT6ECrKVw0rY/X5rp9C2jl6pJW9ccrDcOk9yaxGSpJwfd/ykAYzIUvWOfsnI7TujeEq0bN3Bb1YKlLslw4FC2qSssERGIhk9EZUgJ+1cYY2+L+xljuxhje7TPYwGUEVHrIMr2y/n/nITLnpmKq1+YkbVPXKovI8ukw4Bdmt/4U1+uEs6X+qvkfQq3AoY/XpBVmsBXEt8N4sS3XSpDkW/3HXJ2oHbaX74yC8P/odZHiNzx9vysbWE15SC8dAjAcwAWM8b+bnJMe+04ENEgrdxsp/Yc8umSrVnbRD98WWC0LJOORRl2KfqKDbeNmj9cNOkoWe8eMacD/3U7l27SjLveXeCoHL6Urbvtz1tszOLDSoesDQZh0jkZwOUA5hPRHG3bHQC6AABjbDSAiwH8kojqAOwHcCnLA5cKswlDHpmGb0bGlqlU/GSSufZE4DX4tdv2Zu3LgyYVK8ROk7+/Qx74LLBy1HPJULPvUJbiV+dE0ASEb4HPGJsEm3lIxtjjAB73W1YQbNy5HzX7D6Fvh6a2x/JD3IUbd0ln1EXZLYZVlqHEPfD79xbgVZcp9fh34W+Cn3KSWY+uFNlY5XgQ7ft+UM8lQ78/fpy1rerb/ThYl0R5acbgEpZOWHQxe096MKW58JNKZmS/ENnHiBr+jDXmWX/SXjpK4rsW9oC14JixeocKUOcS0bU1rElvNbdiz6F6o8AP65YVncB3g5OG+uv/zXV8Pv1syksHaFCaSC8ycYrV47j5v3PMdyqkWJl0gkTJe3uiEglK4Fvg1mvBDpUGEZi4rBrb99a6FvYAMEcLNa0IBnHSNizBrOQ98P7cjejUoqHp/qWbdxvmtJRJJwBqBDcyu8mkoDWeTBrE4uWK56d7/u3iTeLyDud8tbwaJ3RrqZb2c5i5ZYodgW+UxMcNr8223K/nA75vpOma1EAoqmiZ4oSJXbsOOmlSurhiVvFzwNLNu3H5c9Nxz5iFua5KrDAz6dgJJ7cod+T4UFQCn6fbbR9izwHrEKRBx7xRGn5u2FObGtmZxespVrJXK6cmDz+cvyngcsz3TV6xDV8LSYQU4VG0Ah8ANu86YLk/8KEtlA0/anYfOIR/T14DAJi1bieue2VWbiuUQ/YfrMe/J69OL6rKWq2cZPj1G86dEJww+suV0hAMOj9+dhp+9Iws9JYiDIpa4NtNygZuyjSJpXPtK9/gtrfmBVuYAgDwx/cX4YN5GY01aO01n/hqeTXufX8R/jp+KQBZTgGGMXM3Blrmgx8tCfR8Cn8UtcC3m5QNetI27aUjbB87fzNen7E+0LIUKf73TVWuqxAbdE27Zn/KxOVknUlYJJMMXyzNDmdS7IT9CJTA97HfLSpMrCKXiM1ZpuFHxYtT1uCqf2cHLFSES5ELfLv9AQt87a+KpZN7ijHdHhPmkLI0/AhV/PU79kdWVj4RtmQoaoFvZ8MPMJwIAKOXzrY9tZi6KlYBQ0MnSoFixarqPehz1zi8WWTmHlF/CdOkM3f9ThU0zQPKpBMikZt0uMc58vHJuPTpqYGeP+784j/xyEClZ2Qat6C4JnAzc0iEFyavxvefnCLdHwQjn5iM9+cV1/0NElko9iAoeIG/snqPqaZhp3GGteKQiLBhZ/ENaT9ZtMX+oAgoK0k1ezHFXzFxz/uLsrYFreDcGPACrmIirMVqBS3wv1pejbP+9iXempWdXxawX1gV9MIr/YWKOnGxwsiXy1JpE+uCXkodMx77dDnOfPiL9Hc7gW7lL68oDApa4C/fkhq6L9hQI91vJ8/DCimi5mxzywtfrwFQ+Br+3z9ZhlVcoph0/2bS/g56CGjnFfUOWBNbkw4RdSaiz4loMREtJKKbJMcQET1GRCuIaB4RDfBbrhPsXmfbhVcBS/zMpK1q7XHgUNCz8jHHrjV7iWCqyC+C0PDrAPyaMdYXwGAA1xHRkcIx5wHopf0bBeDJAMp1jJk2YWvSiSA8svJkyB0rtuzBjDXFkzTFbOGfTm0RuqrGldja8Bljmxhjs7TPuwEsBtBROGwkgJdYiqkAmhNRB79l+8V20jYkGz5/WiXvc8fu2jpcMnqK/YGFgk1bUxp+sMxcsyN2Cl2gNnwi6gagPwAxGlJHAHzsgCpkdwr6OUYR0UwimlldXe2pHntq6/DSlDUO4t1bnyfoZ7Vya8qeyvfeKv2bIioyI0y5jl9bpzT8oBi/cDMuHj0Fr053n8oTiLENX4eIGgN4C8DNjDExU4Ws9lJJxxh7mjE2kDE2sE2bNp7qcs+Yhbj7vYWYvGKbVrj85kXth79UC89r0PADLUHhhWJZdats+NGxdntKuVvDTZrHgUAEPhGVISXsX2GMvS05pApAZ+57JwDBhuXj2LnvIABg30HrF9nOpBO0W6aOMunEiz53jcO3ew/muhqhY2/DVwI/KHR/ALf5q8OWB0F46RCA5wAsZoz93eSwMQCu0Lx1BgOoYYyFuAzP2U22E+hhhQLgz6pMOvFgmxYjvpCxa85RmXS+Xrmt4P3U7MxnuSIIDf9kAJcDOJOI5mj/hhPRNUR0jXbMWACrAKwA8AyAawMo1xY7UWofPC2wqhiwm1v4esW2ojEzxIWi6HYlXmI8ByNyU63eXZid69RV2/HE5ysAZN7xEpcSNuz+wXcSc8bYJNio1Cx19df5Lcsp6Ztmm7M2WrdMHSuTzsKNNfjRs9Nw5ZCuuHfk0aGUHzULNtTE3ue9GAZaenOOg0nn2UmrIysrKvTYWNed0TNt0ikpNJNOHHEas8U+41VYJh1zL52tu1Laz5rt+0IpOxec/89J+N6/vsZRhzXNdVVMKYRE2+/P3Wg5MuRHlrLwHltjoHkzxvDqtHV5PcJ98ouVaXNxwmMclbA0/YIU+DqyhU48tjb8KCZthX26p0R5aeE9mjZNGgAASmIYTCjfNfwZa3bghtdm448fZAdF00lr+EQoTcSzfY1fuBl3vDMff/t4aa6r4pm/jFuSth64nbTVCas9xvOpBwR/z2R286jj4afrwn8W6qXbUfNN4DPG8MzEVbj8uWl4zcT3+IulqXUVYZnK/JDPAr+2rh7jF2wGAFR9ax6Flb/EOHa6QCb94s59h3JcE38k0zZ8itXiK982/DjDu6HJZEzOTDrcacUq6AGsGpQksGzLbvRu1ySUOgTN3Koa3D92MQDgq+XbsGbbXvxuWB/PQ9qoyWeTzn0fLMJ/pqY62ToLLYVx70MuBb7Va5W2fedJuzGjnrMuuBEjYXvt5Zca6ZLZ63amP3vR8MNzyzS36eiucW/P3oBzH5mIr7XFY3FHnJR9auIqzFr3bY5q4x69efxl3JK8MyfoUWEB6xDH/CsQV3nq1/YdF/R7XULkSpXImN0CrxKAAhf4PN40/HDqMofviITmIIaoXRWzlXpuyCedWX9Bn/xiJf752YrcVsYlvCC3atNp7ZHiK1B1xSym1XNMPWfDd2PS0Y9VNnyfyIbsuTLp/I/LpWpm0tHJF6Epe0Hz6Z0Na1V1FPDt1ErDj+HUSRb6O+nWnTFuJLmRipv7rtwyA0J2I+0y/OyIYLl91qStGM8kbwRRfr+gcZxIdgov8Oeu32l6XFzmKazqkdaM81zFz3jpuLvvdp6FfikKgf/spNWYI3kR7DT4MXNDC/eTRqyBuNoxHq+oNdW7a6XL8vNJSZMJ/AUbarC3ti4HtXGH075Kb+7b9hyMrRcMb/vOZ/QRY0mCsHjTbse/C/t9L2gvHR59FRxPHLQ6sdMR65QPCv4J909ARVm27vD9J6fgkuM75aBG7qlPMsME+cG6JM7/5ySc3LMVXvnZ4BzWzB6npkd9NDlxmbew42HCGAMRFcykrf4aJ4hw4ROTHf8ubJlUFBq+GXFI2jx1lTHjklilOPnwWnHAZFk+P1/hll5tG3v+rVvqkwz/+HR5+rsuRGesjr+nkVOBb9fcj+4YzUpoWXX1bfxkZz7jdeHVX8eH6yFW1AI/LLdLN9z42mzDd1HA576GxYE4aat/PZSMdwwggEtOboNdv1CSw9W3mWxwuikkZ1UJhGRMryNm1YmWOGj4IqK2FsMqGghzCBqlkid2/vpEG2NAt9s+xOdLtkZXGZfINPy731vg6Die0hyaUfQO12sc+VzxwuTV+GZt9ihQv45CDI+ct9THUHsT38m4m3SCiILZqUVDDOzaIoDaeKcuafSlEPuxcVrogjgiayIvTVmbfZzNeeKw+jZtw4+ZoDTjnvcX4ftPfp21PT1Sidl1FLnAz3UNsom7Ri8ShMB/6vLj0SvHISTEkUo+JaaxqysTzCVmlJXkTjhlm3QIE5dV48Q/T8Dpf/0c63fEL3rsfouMenrHFbd2VNQCP24PA8iuUwyraOBQvf8KEijnLpxJxgwqMBP6sVzXzwqn7TjOK4h182o9579+65vzsGVXLdZs34cXv16Tw9rJ2aGlUi2XGOp1/SFuYZ6Dymn7PBFtJaJsw2Fq/+lEVMNlxLo7iHL9UheySUcPB+yG7EnbeEt8q2BdTomDME2ZdDL3Oo7KAM8XS7emFwbaVdXppUR1ybJy6usZnv1qFZZvTcUFEt0y4/g09DzIzSvLsvbpc0Jm3mu5Iig//BcAPA7gJYtjvmKMnR9QeYEQ5oTjHcP7oH2zhlleOHZku2UGWKmAeHriShzdsRlOOrx1IGnxShIkXacr3xoO4qSt6J0Th05J51B9Elf9ewYqyhJYct95tmEhnDahqDo5WTn7D9XjTx8uTn8Xbd9xfA/0xYYVZSX43ZvzDPuSPk06YTW3QDR8xthEADtsD4wZYQr85pXlKPMwCZZl0gmqQgHy57FL8KNnpgEA6gIw6Yi3qXPLhrhsUBff53XDtj21WFWdCVS3t1YcisdH4tcL2qNTG74dUc0fyYoR21E+TNryt/W/M9dLj3FyT288s2f2ub1WyoYobfhDiGguEX1EREeZHUREo4hoJhHNrK4Od0VgkJO2p/ZqbfjuNea4+KDjqNnwBDFpS2S04Y+7aSgeuOgY3+d1w58+XIztXOykKSu3R1q+G8Q2IVomPa/liKityda/iObVRIIMJjb985pte2PjuabXQtY36VV0ouEf3bFZcJWyISqBPwtAV8ZYPwD/BPCu2YGMsacZYwMZYwPbtGkTaqWCdMu85Zzehu8JIm8CP89s+EFM2oraXByyfemZl3QO1SfxxdJ4+OJnT+zbafjOtPzoTDrZ28R2VELAll3GHLtz1u/E6Q9/EZsJXCe3y8niTpmciLVJxw7G2C7G2B7t81gAZUTU2uZnoROArEojLrBIJLxp+Nnamp9ahU8QGr54m3K5AEhHnIx+85sqXPXvGZhXZR6NMirEJmE377N40y5H5kuvTe2+kdkD9lFDe5g+R1nHIrajvRKXxzVabojZFhFBo8RZJ2p/nijNV5EIfCJqT5pEJKJBWrk5HzPv2Ftrf5BDZI/Mk8AXGlHYsTX8EoSnU4LIMEGrd565NOOadWRXPj894ppkw7eRr5ZXZ7WZbUK7HvnEZEeryu00/P9dMwSz7zona3u31o2ytrVt0gBn920nPY9MUIr3e99BY5TSOCo+8zfUmO7TR+ZO8ixE2c4D8dIhotcAnA6gNRFVAfgDgDIAYIyNBnAxgF8SUR2A/QAuZTEwxE1eEV6fw5hXgZ+9beuuA2jbtCKAWgXPwboA/PBzr9BncchEQH4bg7DC/Jtz+XPT0bqx0f335tfnZP3GiT+4XZ9wQreWWYIYMDFJkPnaClk5TjqkOJk3l2zeZfAqMsOJmMs7DZ8xdhljrANjrIwx1okx9hxjbLQm7MEYe5wxdhRjrB9jbDBjLHstcp5xbCfriRbGvC2rljWQIFwfw8Jtvy3rAxMWwgEA7hze12Wt/GO1vkAW+z9KsuZ5hO+yWPejXv7G9rw3ndUTRMBfLz42a99zVw4EIBdOZZKFRwkyduTtOYVFNpJYsXWP4btVs4qDfrBjj3VyJDeTtrJ7WgheOgVDv07N0KlFQ8tjGIBSD0vVZQ8692Mhc9xWTS4crO/T4B6tXJbiH6vJ6KysZBEjKsOiUJGFSJi+2t5r+sw+7bD6gRFo1jB7IdFZmnlG9qzMJh35ea2y0sxnmTJ/+9vzDd+zo5fG6yUodRgG04muJlOCVE7bmCE+EPE9SDLmaajm1FPiyS9W4q/jl7g+v1fM4oa4bZgyDxw7y1cuTD5Wk9HLBW00ChhjabOMKPzECVm/QdCsIjzKzl0mCatMZFw2x39zIrytYubrHKxLRpKGVIadMqdX1cm1yu53WOYrJfC9QGQv6BhQ6iG+uMybQlbWX8YtwROfr8waCofBpOXb0PfucZi2KnvOw23DlMUdSXArbds1dR+OIgysBP5F//oa70eQ/pLnP1PXos9d47Bx535brxy/At/q57J9chu+UZA9eulx6c9e3Blfnb4Ot7wxVzt36rzXvzoLA+77xP5kISDr5HjcmXSyt+05EE5qTSXwPSIKuvbNjJOqScbAtwmnkQhlDYSB4ctl1eh224fYXHPAYFL4+UszXdTaG1NWpVL/zViTbRYIRsPP3JtrTjs8a38uNHy7FcTPfrUK86tqIjPvjJ2fCs+8ZtverLa3O+C8u1b3W6aNVpaXZB+HjK390R8ehwFdMuGvnQhBcRJXZmL7eNEW2/OEBf9uW7m8OnLLlEj8d+dsDKVtKYHvEbHNtm1SgQX3fgc/GJjK4cqQ0fDLSxNoUVlue87xCzenX2yxrJe1+OZzq3ZahmUNA6v30+3AU2bmSpC1GSHKmDo6dhPlc6tqcMHjk3DP+wsjqY+uRdclmW0n6/d+uf19l5aV2R05UVpzFTsoJ0IwDvmmgdRI440Z67Mm8flnYKUcOLkOmYbfqlF5KAsQlcD3iOwxNm5Qmn4xU26Z2rEO1eBXp60zLUtfFVyaIOw7lNHoohSFRGS4ln0H6wKZTOOFvex0cbPh88xwMBkaBHq7qmcs/BWxLu83EbKS1fOduLhU4x+fLrM9Z1yy0f135nrc+tY8PDdptWE7L8jlbSW134sN/5ZzemPirWe4r6wDlMDX6Njc2utGxPw5ao2csXSO0PqkM0u3mZBhjKVfgNKShDGwV4TCkDGjdvnO7A2uNXyZ8LYzOfuRb9ecdrgnTclpULioXGbTAr/egYbvs024HUHKRmf8Ujqxuk5CBluFPRFLC9OD51st5r24/oLvdK06JycLr8RRb6cWDdGoQVCBjIWyQjlrHqLb5H5zbm/rA9PIHyRRZq/uh+9UWTET+EmWEUClCTK8kFHIe7498g09QeTapiMX+OFdRZOKUrRp7H4i2Kkgj8qGr9+juiTD5zYxffzeTpkfv0hpgvCjEzMRTcVmQJwR34tAtupw65IMs9dl8shG5bG5pzYzouXfadmajQmLt2YdZ4ao8Mhcl4MinG4kD9FfKJkPsgyzRpZ+eIyhhJuoddIozYUHS4cwKE2QdLVjVPANmODeS0dmH+YFftDvrtfOxLGGX5fEsi27ce4jE3HpCZ3x4PezFy0FgR6X5n8z1+NTm4TqTSqctWEzdK3WjNE/OR592jdBl5aVuPe78sC3hMy99/JMrbTmMXM3YgznJRWGvN9TW4dD3PtYvbsWR/9hPG4ddgQuHtAJ93JzN1ZavJeVtmGmmlQavobbxml2HCGj1RtX2tqf+aCJkGEMBpPOPm6ZvNVkZxiIGr5b7UpWXaLw7PRez+s0RtDBuiTu+2ARAOD1GfKY6EGgm3TWOcjt+olP7xUxBs5PT+5u+D7s6Pbo1roREglKa6OydSnpW+9BIruZtA1jTuOUv3yG/vd9kr6uLbsOAAAeGrcUg/78KeZVZeLoWBX/2nT7NpEt8MMTy0rga6RNMQ7bjlnPTdwwlvdP1g9vUmE+qDLr2RkyGmdZCRm0zyjEvek9IffDaVl9E5QRHEEHyvR6Oqdhnw8lk4bnccpfPvNYojX8pK1XTunpLEDtEe2b4L3rTgYAHNOxGe6+4EjXZfF5ihfACHwAACAASURBVM1GgUcd1tT0926isIZh0hHNWlaKg9/yRZd+p6t4PZUV2pnzjLSGzxjm3XNultcBYAzba/aM+ZGCbEGK1aIY2aKkVJ0yL0CCyKD9hKngpyZp+bIoW8N3eU7ZiCRBwA1n9sRVJ3WTZrrys+rQ6+jBqcApTSQMz6Pq2/3uC3OA3m6cxFc3o2urSvflOZZmoopvHO3KsBJsudbw3eB3Vawy6eQA3qOgaUVZejHJrcOOwC9O6wEA+MVpPXDx8Z3wyA/62fbqSWYU7vrhVwzpZvobs6EcAzPYNPkGHpaPOmMM3W8fmxURMMuG7/JlM9Pwm1SU4Z7vHoWKsswiHrFzMOsQw8CxwC+hQEJE28H74XvFTZ4BXQh59YdPEKU1V7MmYlUfN9f5zuwNeParVW6q5xor06lfD9IoJ22LSuBbLxmXayMNy0pQWVaaPubhS/qhR5vG5jZ8E5OOztl925rWwcwVi7GMJwBjiETD119S0f/YoOEn3JtnZS6STq7h0R8eh3E3n2p5zOifDMArPzvReG6PHeKyLc5CVpQmKJJFQiU+BTCAtJuws2PdlSe2deL+N9OArUa7bjrR29+e7yhUsR1fLqvG+IXGhY+6QhOm6VTsTJTADwirBqYvbxY1VoLcvm+m2fLDM4MJSDveymuECOjbIduuyU/aMkSw8MYCxr2HBHeTtoO6t8Stw47I2u5k4vnC/h3Ro01j9O3QFBcN6Jje/tTlx6c/Dzu6AwZ1b+m8QgFQmkgEkubRDt3jy5eG78JUoDddp21N9G7jH6nZKaxMF069pILkyuen4xcmYaStmqjfDl88dZgZ34pK4FsJ24Qg1Cs1bbtBWQln7nGwak77axYt03ryh8lDpYKlXwDGjC9DWF46sitlwirPPbV10vg6Ztw5vC+aNbQPMWHFRzedir//IBOI6ztHtTfsF++Gn0lOJ5SYaPg79x10nQj9wicmG9wNDeUEouG7EPhpG76z45sLz5WIf6fkJ7EKLpjL0Ar8Ohe96lFq+GHmdA7kzET0PBFtJaIFJvuJiB4johVENI+IBgRRrlssNfz0ZGvqCd94Zi/8+pzeuPj4TunJJd7NsrGJ+UXXxAgkneS16nSSzCQZgkHDdyfEJi6rxguTV9sfmFWmvAxe4P/+3QVZJh8riLxpL7KqnNmnLY5o10RShvH8dfXJUCe2y0xs+Fc8Px2XPTPVlbfJnPU7ceNrs7O2b6rZj5enrgVgnZjFDjf3PrNo0KGGXylo+AYvHff1Mcs4FgW3vz0va1uY7s9RavhBLbx6AcDjAF4y2X8egF7avxMBPKn9jRSrDFSi2aZheQluOKsXAOCqk7phc81+jOIiOT5w0TE4tlNz/GWcMSb9dWf0xK79dfjx4C7SKHjWAl+u4QMZmyZjzOCpsXjTLuzcdxAVZSVgLFVvniu0HKxXCb7Udshet5SXjqvTGH8PynJBsz7enOevOsHRb8KOySJ6TQGpZ7Ro4y7ts7Pz8B3stj21hrSFo79Ymf7sR/N1E67brQ3/ZMHlkzgvHdNJWwuTjlVohbCJOt+BKBLC7FyCSnE4EYDV2H4kgJdYiqkAmhNRhyDKdoNMAOtQWsPPpmF5Ce4debRBq29eWY5fnn44Xh81GB//amh6e9OKMjxw0TGoLDf2pXqjt+q8k8wkGQLLxMeWafh3vrsAJ9w/AX3vHmd+cgfU1SfTnYnZS+rXRdJvrHYZ719/SjoFn3j7QrcFU7bPPj8iu++DRY5i0/BydeCfJhj28ZN4fkxUrmz4Lt1AOzZviF+fkwlLQkToriU3F0OHZ+pjLn5yYcPX4Ts5/VOYJh1RCQwzNlBUNvyOAPglZ1XatiyIaBQRzSSimdXV1YFWwkrYtNSGpG7v9eAerdBbYloQSc/2W/TeE5dVSx/2BY9Pyph0WPZLWLPvEHYHkDCh550f4da3UsNZM8H+l4+W+iojjOHqMZ2apVPwZZl0IjANiFrwks27059fnroWr0xba3sOq5e8jLPpRmbD1w5108HwRxJSI+NXfnYizj2ynfT4Mov6BGnDr6tP4v4PF2H7nlrD9k8Xb8HmmgOOyo5yQXvsbfgOkN0u6RNljD3NGBvIGBvYpk2bQCth1r5eHzUYR7RvqlUqbBMAcGqv1qgok9/6udySbflCGZYlxCat2BZY/d78pspy/1uzrPdbkSByJXR+PDi1CMttJNMBXZqnP9fVJ0OPp58d7936O8/6Hfvw6IRllqYyXsOPyg/fbeA/EaLUKOHknq1NlRwrDd/N3IeO2ST5p0u24pmvVuOe9xcZtl/94kx871+Ts46PwqRz2aAu6bU+4u3p1ML5Ajm3RCXwqwB05r53AhBtjjiYaziDe7RyHVrBLfyk7ctXn4j/jhpi/xtJXUQ/fD98vXIbut32IdZu32s459CHPjckvQ7qlohp7+z48YldsebBEWjRyLtnDy8gHwopsJldgDar/f/3wgw8OmE51n9rHiOH14T9tE8vXjpWWrgIX7fjOjc3P1CjgYUm6ySEsshlz0yVbtfNQ/y8gD6i2iTR8Hky1xSs0nBa7zY4UnPBPsDFxpp/z7mBliMSlcAfA+AKzVtnMIAaxtimsAt9mvPRBqwnbe1cyIJCf/mdvHyjhvbI2sZgrTH+Y8Jyx3V5e9YGAMC0VTsMkTrX7diHy5+b7vg8TslJqkLuJe/Uwt1IwQkEe1FgJfD3aukJrY4pC2iI70bDb9WoHL86uzde/plz34qMh1tPR1pqg9Ls1Ig6B+r8Z3XTRwn6+8KP9HKZX+WUnq0x7Oj2GH358fjtd47A4W0ap/f5jXRqR1Buma8BmALgCCKqIqKriegaIrpGO2QsgFUAVgB4BsC1QZRrx7mij7bwUo2/eShG/yTVKdjF/fDKrcOOwGs/H5xWk/UqOBF+HSUCKqXhm//mkQn22YR0dAFQl2SRJPKg0I0r2fCTf1aT9l5hyG5X4rO1XlGqL8gzLyOoeQ/ZStsfn5gduwhIXdNNZ/cyCCM7Tu2V8tQ5pZczU2wDE7Mm4CwmvxULNtSg150f4fMlWyEq6Q+PX4rD7xjr8ozBCQZ9NN26cQNcd0bPSCPeBuKWyRi7zGY/A3BdEGX5QXzxjmjfBEe0T024hmXSufb0nqnzat91oeNEw5dOfASY4i7jepeMJJFHVO2af4F4k04YHkJAdrRDsVuzKtaJeS6opfZix9GnfRPc/71j8IpJak23HN+1JVb9ebjjjjXM2Ej6gsC3Z2/A6m0pm7w+inrexboUfdQSpFwIezGgFUW10tbqhdf3hDVpq5uKdLOSlXlJR7oIC/ZCYlW1s0kn3tfaSsP3KyYPb9MokPM45dEfHocmmgstv/AqDHlPyH5OWa6hFs9LNztE4U0ktn8rG7pX3IyiwvRG0W/n+3M3YsGG1JoIvWYtuTkhu9cwDNkcttnYiqIS+JZabDo8crh10N8HJ8M42SG8j7cZZ/7tS0d14SMwHoooVd9hLj1uvNC5ZSUe+1F/AEArbgFTWKkU7c5ba3Fv69PutubPNKjOQPTDt7KhR0EYHY6ObP2A/u7xbcKJ4gUEq5WbKWy6YhQmRZXicFONeaxyfZl+z7bObZZuSC/gcDFpK7N4M6RW2ibI3XzDll0HUFaSMGg3/PJ5Kw3fb1PnO7fy0gSaVpRiVwDrBqw4vXcbPHxJP5x/bAdMWJzKABWWScfurLUWHie6MLeaQglq1akoYMPUsJ0QZlRImYDW2yEf6M2us9bP8sXS4NYEydaUffKroWjbRL5ALUiKSuBf2L8jkkmGd+dke4SOOLYDurc+FUdaZOEJAl3mONEspIewVGMuSRCSLlYjnvjnT9GxeUNMvu3M9DY+AmNUybijgohwsZDEJiwNXzytOMdysN7c40TXRK3CAQc1ny4K2FwL/DDLly6e0v7yAx03oT6CQjaa6+Vg8WYQFJVJp3nDcjx6aX/T/WEK+0xohVRrcyJ7zFarpTR898Jrw07jCEefxEva2PCrd9ea7rPj/etPydoWdR5evbSwihWfhSi7ZWECdu47iGVbdqc1fPE33W77EHPW7wQQnIYvCtgoE8rw6Gk+wyxfGhIiPZeTeV5hKQFW5DISaFFp+CEGoXNRB+cmHZnE1xdeBWGe0N307DT89Q4SZ5txTKdmpvbpMdefHKk/dBgmHSLKFvjC9cou8buPTzYkJJdp+F8s3YrjOjcPzIYvavgdmodvQpCh356g1hcYz81Mg/xllC0XAj+ESb1cCvyi0vBzsfBHR/f+Ie2O+7Hh1wUl8LkY61bBqoJyAxXvf+cWlY5WZPrlogEp004bbrIuKFICxrgtS+BL7t86oROV3WP9+QclIHgN/5enH47fDesTyHndol+rruG7ybVrx2vTUyG7pDZ87S//6uRCCcxlAqMi0/Bzr+K7MumYaPhJk/SJbtG9NuqT1r79QSskdnHSg+aGM3ti1NAeoZgQVlbvzdom3i8n77fMoqbfJzsNv1Wjcmzfe9C2jDLOYP2jQV3S+YP/celxoXSGIu9cexKqvt2P3745F0BmErlpRRlaVJbhW5+LrQBg8opt+NGJXaQmHZKZdHIg8XO5yrfINPzcCXwxPLKjSVvZeaCZdAK4Fj5RtZXAD0rDFLMHReWPTESoKCuJ7OXOio/v4TcAMHf9Tlz+3DRDrBUZR3Vs5qhepSWERpKAXSOP64iThHj2YdC/Swtc0O+wtMDTRxxBKTBAZiQta88yc6pTL50gcRpyOgwKXuCfdHir9OfcmnRSuLHhyzqoa17+BgfrkoF4OJRyC6+smiAvmCvL3ftu57KjFfnrxeEEUOP5wVNTDN+d9GsyAfXpkq34avk2rJKMInicjlxKE4RGJpnaokRvTxmBH9zoW7+NcrfM1N+EQeBn4hmFyf+d3C39OZcmnYIX+C9ffSJO7pkS+nGYtNXxuvBq/6F67D5Ql/Z08AJjzDDxW5dklqoM//IMdRgnRSyPR0/UHsaknR2XDOyMQd2cJTr/wwVHBlKmk9XbTlbjmuF0AVNJgtJJfOxGDWGS1vC1jooFqeEzYHPNAdRL56RSZfBFbdtzEEf9YTz+O0MeXiIo2XzZoC646/xUe1KhFUKkJEHpFYX6JNj0O8/CtDvOirYiwjP2GksHAHbXHjLNqeuExz5dgcPvGJt2xbS14XPyxk9j1Tuw0Zcfj9d+PhhNQ44MaMYzVwxMf/7jyKNMj+MX6PjBkYZvIfDtsj91a+1s0rM0kcC1Z6RiO7WJYJGPGfooUfcaSqX2DEbgj1u4GYMf+BSLN+/K2qe/crKyPlm0RXq+oEKtJIjSiWBymL2x8AU+wD/o1N+2TSrQrmm0Df4/PzsRF/XvmNbGHHllmhyT0vDdCyPdhPPq9LUAMkPZvQfrsGa7uesl3xkEYX9sWlGGIZypLWr4hNtXDOmW/ix2wlFO8j/2mXlY61obDb9hWYlhBbUZpSWpxWhrHhwRWGfmhXeuPQm/H9E37TSQZMG7zK7Qkpjw53UTqTZo+PSeyg8/ZHRNIpe25EHdW2JQ94wpwZkwkR+z+0AdurR078pmJtDenrUhHRtfhi7w/zp+KU7s3hIVZQlPCSrijvgiJhKEJfcNw8tT1uL+sYsBpF7cMEbks9ftNN1Xa2N+sZvPGXFMBzSvLEP7iJUcM3q2bYKebZtgxdZUKsggJ2119FERf1a9vcvevQmLtwZavgif7U2ZdEKmYZk8lVgucTZpK9++dvteTyYdXcMnmDd8GbwgnLZ6B0o9rkfPYTvPwsmlJwioKCvxPUHu1xtpv43Ab1BaYhnPp2+HVBjkXLggWpFWwFjw82v6nAgfME7mh29HUG2WkHnfchktsyg0/AaawI+DH76Ok7qYHZFkKa3OrbYpdjKyKhzdsWk6nCxfHk/M5IYnJtxyGhZuzLbz8mQ0wsw2gntXPb+v9/6D/jT8uKLf3yRjhjUCQVCnmcE6t6hM56h1E7gwaBJEaUeLnwzuGnn5OoEIfCIaBuAfAEoAPMsYe1DYfzqA9wCs1ja9zRj7YxBlOyGt4UdVoAOs2ty4m09Fo/JSy8U0XrRsMWm0rLOQTaSKE7pWyad5BvcwesPEqL/F4W0aZ2Vzat+0AsOP6ZBOkCELZZ0gcu1W51ehs9Pw7dwy4+QWy6Pf33rG0CDgOh5KZxIznnfXgUM5GWmSNlpc8+CI6Avn8C3wiagEwBMAzkEqWfkMIhrDGFskHPoVY+x8v+V5ocIilVqusHoJ+7RPuS3usBD4ZSVkqW3qMUV4xFW+Mg8EmfYjCjgno5P/XH0iTtFS3g3t3QYrq/eiWUPvycijgIHh7guOxCeLN2P9jv1Sm68XueTX08POhTLvNfxk8CtedQ2ff151ySSOvedjV+cJqm+IS58bREsZBGAFY2wVY+wggNcBjAzgvJ555WcnGhbY6EvIrRJRxBGrRiImsxCROQKIKe5kmo5U4Au3zUmO1Zr9mWXydw7vi0m/OwNtmoS/fN8PojOMLAyGl6y8fjXKQzZumTKB//xVAyVHxov2zSpwZIemeOCiYwI3E+rtn++scxkCPC6jrCAEfkcA67nvVdo2kSFENJeIPiIiU+dnIhpFRDOJaGZ1tbekAyf3bI1LBnZOf9c1fLuhcdyw0qTtkkfIzA6iMJeJEZkwF8/lxAbKR38sLUmgU4vgAmSFhTiZplvNDJfr4b19Zdo6TFwWXAINEdkzi9MEuRllJQmMvelUDO3dJpL5NS+ddXBlx4MgBL7sWsTmNgtAV8ZYPwD/BPCu2ckYY08zxgYyxga2aeN+VacMXcPP5erCoLET+FbaO6WPcWbS2VRzwPYYkRHHdLA9Jm7o7nLpmD+ScLpeueWNub7PYYbMHJIPAp8nLA8iXlnxMi8c1H3U5xFzTRACvwpAZ+57JwCGlFKMsV2MsT3a57EAyogo/GhNGhmBX0AmnQRZCiJpuF3hcKcmHVnZVvxiaA/HE7txQl9Upt+XEokN3zvhSWBZIL0kY7jmtMMBxMd+bEVYjjPGNu7BHBfQc2uaw4VuPEG8lTMA9CKi7kRUDuBSAGP4A4ioPWnSiYgGaeVuD6BsR+Srhm81BLUTqF4DdpU4UIPstLFcBofyg1jttA2f2+ZVLoW5uLIkQdmdOYITVlEQlkln6Zbd6c9eVok/9eWqQOqRC1dQGb69dBhjdUR0PYDxSLllPs8YW0hE12j7RwO4GMAviagOwH4Al7IIVx80zCOBf97R7dOfrd4B3UvHDNnLrh9vNTJwMiFrd0yeynvOpKO59Ok2fK4P9CqXwmzuMmHJF5dL27VTggj3bUe+ejMFSSB++JqZZqywbTT3+XEAjwdRlhfyadL2n5dlcu7y78D3+nfEO7Mz4Q/sJ23ty5Id4kQTyUUM8SjQl+PrqzR1/3aDW6ZH4RnmPZE/szx7ChH0SXaebcVAUXR5cTXpzPz92VnbeFMNL1xEGWunZW+u2Y+tu4yTraJmLzXpONC07F6cfNXw+eihgDwGk9eReZj3RNb359sziMLikat7Mv+ec3NTsISiEPgNYzpp29omrRwve0WtuqwkYWleOPvvEzHoz59anl86aetAC7LzWskn27EMMQ6LwYbv0fSQZAzffXwSRn+50m/1skgQ4Uou6ieg6fdpb6PAiwycKGzcuYpS6SWybVgUhcDv0aYRyksTuOnsXrmuiiv4V0DUvL0MT8VfyF4AJzZ8u+iN+aZdXjTAuGyk3tKk4xEGzKuqwYMfLfF6BlNKEoQbzuqFf191QnpbkmW63TyQ95H44eerM0GQFIXArywvxbI/nYfvHNXe/uAYYdDwhSdVmkh4tifr57XK+1lM/OF84zpAfdFYWVrgczu9mnS8/cwR6WfG1S3fZFsUK1FzmTw8LhSFwI8z3/z+bEy93Sz7ljFoF095qfsXZNW2vXhh8ur0d5lQcKLh25HL8K9eEL036kWTTgDCKMx7Ii6oA/JuyhZuB6x3n+8+/WRUycNP6NbCkFUtTiiBn2NaNW6A9s3kiSmsbPheY9Lf834mpp100jYAT4Z8EzaiwBe9dIyxdLyx1ybEsR/SAp+rKGMsrzpetyPLn57S3dXxJQn3UU69cmafdjhHS2cYN5TAjzEGG76geftxMdM1WK9eOnbkkZwBIIkxpNW/TGbDj6HJS7ZAjCeGVc4i7PuaoNxmmooLSuDHGDIIGuO+spKEZ3VzlxbJ0klETbt6ych3Lx0dvVM1JECJofDMaPiZbcOObp9XHS/vWtq5ZcPAz58gysn9uKi/LI5k7lACP8bwsiXbpONd8ujmhUWSjE9OQivYlezV3BQ3ZBp+HClJa/ipvx2aVaBBaQkuHdQFjcpLMDwPAtnxDgi3fqdP+nNleYmt+7ITEkSRu2WueXAE/v7D4yIt046iSHGYD/Rp3wRDDm9l2GZlOy4rSfh2t5uzPjtxtt/ViKOG9sANZ/b0dY5ccVpvY3TW9Gpmi7mUOKD3r6KzTs+2jbHwj8NyUie36Nfw+I/646w+Gfv3jDvPRkmC0Oeucb7OH6UNP84ogR8Txt08NGsbr/WIYSHsFl55xe8CmDuG9w2oJtGy/P7zsuYv9HvRk0uFGD9xn+2lE8d5Bjsy+W2Nik6jBikR9e51J+PCJyZ7Pj8hurmlOAeKVQI/xvjJeOUVJ5O2eShPbOFjE4nJ4Tu3rDTsixslomqfh2TSHTLpKMqvuzBRdCttrxBWPceJGPdFCr7di8PRspDs5HEJ45pLPr55KB6+pJ/tcaIJyAu92zW2P8iGRFrDz568zRf6dkjlcW7frEJaf7/XRB6Sz3ulIibJTmQoDT/G8ENzUTsp87DwyglORg75KFDc0KtdE/Rq18Rkb+bif3Zqd3zpM3Vh4wb+X8H0pK2u6Ofh8/nF0B4Y3KMl+ndpIdXE/YZ4rtl/CB8v2uLrHIWA0vBjDN/ExQTbfkIrWOFEw8+H+OphwQvTynL/wvrojs18nyMh2vDz8PkkEoT+XVqkPoeg4StSKIEfYyxNOjm04Rcz/N0JIk+p7G4fdVhTV+fQO+mw8sJGjWzSWTXLYAhE4BPRMCJaSkQriOg2yX4iose0/fOIaEAQ5RY6vKZWJwxzS2PqpVPo8PfcSzyj7PNln+M/V5/o6hx6J91My5taoy2sKyTycdQSR3wLfCIqAfAEgPMAHAngMiISIxudB6CX9m8UgCf9llsMGDR8UeCHJJiVDd8aXvCEtcCsYbm7kYNejbZNUguUClHgK4IhiBY7CMAKxtgqxthBAK8DGCkcMxLASyzFVADNiSj+y/9yjNGGL5p0whE2cVxYFCee/ElmcFoWUo5Uq0ewWLKQStTwC/ERFuI15YIgvHQ6AljPfa8CII5JZcd0BLBJPBkRjUJqFIAuXboEUL08hmvkYuAnmemlsrwE+3xGZXSitRbzu3dc5+bpz2UeRlktG5XbHmNlvpBp/3y0zL9efCyOOsz/RHDcKOY2FyRBCHzZsxD9qpwck9rI2NMAngaAgQMHFvVaaP7F500671x7krY/eBzZ8JW6BcCYf9gpAzRPFB3ZrXTbj/DzAJcM7Oy6TnHkhjN7GtY5qCYXDEEI/CoAfCvrBGCjh2MUAgc5X0x+0lbXwsUJvyDWlTgR+FcO6YpbJHF4CpkLjzsMS7fsMdxzb55Sxjj1Mm3eTWiEAV2a2x+Uh/z63COELfGX+A9dfCy6t26U62pYEoQRcgaAXkTUnYjKAVwKYIxwzBgAV2jeOoMB1DDGssw5CiO82x/vlmkmD4IIS2w3GdykQWleRF8Mmkcv7Y+PbjrVsM3LPEqSAbv211keoxylsjFr82E5L7jlkR/2ww8GdsYJ3VrmuiqW+Bb4jLE6ANcDGA9gMYA3GGMLiegaIrpGO2wsgFUAVgB4BsC1fsstBlo2KsddWio3Xns3C50SiYZPanit40XgM8awY9/B9Hd5GAHnN7hji0r7gwoA2R1Z/cBwjLn+lMjrIqNrq3hr9jqBhFZgjI1FSqjz20ZznxmA64Ioq9i4fHBXrKzeg6tO6oZzH5kIIFyfZEcJUPJgeB0FXtYsJBlwgIt86udOPvnjATg1gHg++QoRxWbdSDxqYY+KpRNzyksT+PP3jsFOG60QCCaXrKPQCvnSumMIgzy1pBfOK0LTmkhc2mK+uDOr0Ap5Aq9Vm7atAOSIk+X5siMm3HIaxkti+iuMpJKLZ77niZzIOWZN26mC/8BFxwRWFxn58hyVwM8XZA1K2BZFLlmC0cZ8/Rk90bCsBD3bNsYR7c0iTCp0GBPnY/JEUuQY80GRs/sXdoyofDFzKoGfJxjTHcobl/5SNK3wbqkTQzhI68J9/s13jsDi+/IjjV4cYGAq1Z4n5PfMqYYfdmC5fOm3lcDPE/j2ZJfgqDykJf+psilvGncc+XbvIYz0kapPYcTpCCnOaQejRN2GPMTOhO8nzo4T3VOZITI89P1jXR2/aNMuw3d1J51hNihyrOGH3Gb5NJhxRgn8PMGJINZXcPoS+C6sDXpaumJmQNcW9gcFQJeWlRgU80U9UTHl9jPTn53azsOIbHq89uw/vPGUdOC6uKPcMvMYM03bT4JzpxO/b/1yCA5v4z8fa77j2w/c4c8fvqQfBnUvXoGvt8qebRujQ7OG6e1OFfcwTDov/XQQqnfXolvMwynwKIGfJzhx5dMPKffTum3kvV728V2LV/jw+Jf3Tm3QxW380du/eBecCvwwTDqNGpSiUQA5iaNEmXTyBYMg1sPhyg/1M2mr/EfcEdWCm7BSWuY7Tu9/sXeYOkrg5wlNOFfLw5pXAABuG9bHcIyuBfkJKKU8Bt3hV947N0kUt8DSTY3i/XKs4Rf5/dNRAj9PSCQIax4cgTUPjkBleUr4XzqoC9Y8OCLr2KtO7u65nHZNG3j+bTHiVxA7/XVYGc7yhYxJx3jHnGr4QUfVbFGZH5O0IsXdigqUC47tgP+OGuzpt73aNcFVJ3Uz3a/0JCNRmXSKXsPXkVvt7wAADNhJREFUBb6o4Tv8fZArbX/7nSMwVgiVnS8ogV9APHxJP3RtVQki8jWEVSESnBOVSacspITp+Y7TNSFBmnS+2+8wg6dQPqFaUQFx8fGd8OVvzwDgT/O0suOrRVdGnN7nIT1aSbc79tIp8knbTi1TAvbqU4zmylzMgTTLU3MOoAR+weKnfYu++LcOE9PNKXREgW8mgG45t7evcrwkTC8kmlaUYc2DI7Jy9jrtcIMwvZ3Wuw16tGmExuX55YrJ40vgE1FLIvqEiJZrf6XLDoloDRHNJ6I5RDTTT5kKZ3hp4MOOai/dfu3pPdOfH7+sv+c6FSKibdhsctDsaZg9pq9uPcNYDnfeW87pbUjwXcw4tuEH0GGOOLYDPvv16Xnt8eNXw78NwKeMsV4APtW+m3EGY+w4xthAn2UqHOClgevCx8qkc1LP1h5rVDj887L++PhXqdj/JLxBZh2tmWA3e0pibJZSzkvnxrN64cWfDnJU10LHr5fOf64+0fDdykstf8V8Br8CfySAF7XPLwK40Of5FAGhTO3hcUG/w9C7XWpiWxQ45u5//h6IWnhlgs+Vtqf0MiowVtHBC2H+yq/Ab8cY2wQA2t+2JscxAB8T0TdENMrqhEQ0iohmEtHM6upqn9UrXryYdNIafsB1KWRE+W423DfvB9RKUT84vS1B3L9CeAK2Ap+IJhDRAsm/kS7KOZkxNgDAeQCuIyLTXHiMsacZYwMZYwPbtFF2Sie0bFSOX5zWw7Ctrt692E57jKjlto4RO1YzweJXO1RumXL0+1qSoLSZTYbTdWuF3vRtbwNj7GzG2NGSf+8B2EJEHQBA+7vV5Bwbtb9bAbwDQBkgA2TWXefg9vP6Grbtrj3k+Pd/HHmU4XuBt/lAcWrS8WvoyeeJwjBJpHUUZhk00PmIl+GzX5+Gk3sa3WiPaNcEZ/U1M2DkD37VhjEArtQ+XwngPfEAImpERE30zwDOBbDAZ7kKG9wsDGlRWZ76YPNOWK3ALVayTDpuJ20t7vl3+x2GU3u1NsR/Vxjh1zE0KDMXZ07j4TMG9GjTGM0blhu2j//VUDSvLDf5Vf7g16H0QQBvENHVANYBuAQAiOgwAM8yxoYDaAfgHW3oVQrgVcbYOJ/lKmzo3roRZt11Du7/cDHemlWV3j6kRytMWbXdcKyo0cuGtbKYPQo3Gr57Df0x5QJrCz/vZKnhO1Rt9XzD+lqUX53dGz84oZOfKsYKXwKfMbYdwFmS7RsBDNc+rwLQz085Cm+0bFSepUFWWGhBCveI99fM9FIADh6xhL+vVmHB3U7aJpOpv73bNc7bMAoy1Ntf4IjautXkob6HFfrMVYCI99NUwzf1w1c9gR/0ERZjNgLfYY+rt3xd0y+0jloJfEWWgFfi3jumGr4S7KHA31Vrk45Dga81/mQ6OmdhPTcl8Asc3RbZVEugYtV8idOWFO7p3a5xlreUjpdJW4U9/BwKL5w/vPEUw3FW8fD/cMGR6c+68qP/DTKschxQAr/Q0YT3XecfiRX3n1dwGkscGH5Me4z+yfH4+FenoV+nZtJjTL13wqxYESDe1haVZbjlnN446jDjc7DS8P+PSxhU6Cad/A37pnBFgkiLx5KtvosJopWC745//fj49GezDtVMcLRvVhFGlYoGIkJFWSKd7nP23edKj3OsqTPDn8gS3ESF0vCLDOtY99HVo1AxUyQJxrzEAHD/947GxccXjstfrlhy33m2aT2deulkNPzU30J7J5TALzJk8l6Mf9+qUf4vMMkV5qEVgM9/c7ohCNo5R7ZTJraIcKqpizZ8peEr8gpdlGdCH5ur+HrTHnncYfjThUeHWq9CxVyAE1o3boCebZtwWwpLmMQZp0nM9bfjN+cege6tG6F/l+bhVSoHKBt+gcOEySephi/x1f/J4K4Y3KMVDhyqD7eCBYaVhg8YTT7ioYc1q8DGmgMh1ay4kU3a/vzUbDOQ3jH069wcn//m9LCrFTlK4CvSiNppz7aNc1ST/MVMkWygLQoig8A3Hvz5b09XLrERcueII7O2vfnLk3JQk+hQAr/AcRInZ/gxHfDV8m24/bw+kdSpkDGz+VZK8qCKxzYoLQmlTgrn6IltChVlwy9wumip8lo1SqVukymQFWUleOSHx6FtU+UiGBaV5Slhztvt9fSID1x0DLq2qpT9TKEIFKXhFzg3ntUL/bs0x1At6bWKkxMuZjZ83aTD79Y/XjaoCy4b1CXkmimsuOeCIzGvqibX1QgdpeEXOGUlCZzZp136O7+MXBE8ZSUJ/O2Sfvjq1jPw0MXHprfr8yMNyzNmm0Jz+Ysbg7q1NHzvYLHI7aqTu+PvPzwu7CrlHCXwiwzeLVARDt8/vhM6t6zEDwZ2ztr36A8zMe6VwA+XN64ZYvh+XOeMi+UHN5wiHl4UKJNOEfLDgZ1BBFw+pCvW79if6+oUFXwoBSXvw6drq0qs3b4PAHDniL74aMFmnNWnLY7uKI95VOj4EvhEdAmAewD0BTCIMTbT5LhhAP4BoASpTFgP+ilX4Y+/cKYGMciUIljGXH8yqnfXGrY1ryzDzn3Ocw4rvDPhltPSgdA6tags+sxtfjX8BQAuAvCU2QFEVALgCQDnAKgCMIOIxjDGFvksW6GIPcd2yl6p+eY1J+GTRVtQUabcMMOmzCJGfjHiN8XhYsA2ScAgACu0VIcgotcBjASgBL6iKOnZtrFa1KbICVF0fx0BrOe+V2nbpBDRKCKaSUQzq6urQ6+cQqFQFAu2Gj4RTQDQXrLrTsbYew7KkKn/ps7gjLGnATwNAAMHDlRO4wqFQhEQtgKfMXa2zzKqAPD+aZ0AbPR5ToVCoVC4JAqTzgwAvYioOxGVA7gUwJgIylUoFAoFhy+BT0TfI6IqAEMAfEhE47XthxHRWABgjNUBuB7AeACLAbzBGFvor9oKhUKhcItfL513ALwj2b4RwHDu+1gAY/2UpVAoFAp/KCdVhUKhKBKUwFcoFIoigeIcLpeIqgGs9fjz1gC2BVidfEBdc3Ggrrnw8XO9XRljbWQ7Yi3w/UBEMxljA3NdjyhR11wcqGsufMK6XmXSUSgUiiJBCXyFQqEoEgpZ4D+d6wrkAHXNxYG65sInlOstWBu+QqFQKIwUsoavUCgUCg4l8BUKhaJIKDiBT0TDiGgpEa0gottyXZ+gIKLORPQ5ES0mooVEdJO2vSURfUJEy7W/Lbjf3K7dh6VE9J3c1d4fRFRCRLOJ6APte0FfMxE1J6I3iWiJ9ryHFME1/0pr1wuI6DUiqii0ayai54loKxEt4La5vkYiOp6I5mv7HiObDFQGGGMF8w+pnLkrAfQAUA5gLoAjc12vgK6tA4AB2ucmAJYBOBLAQwBu07bfBuAv2ucjtetvAKC7dl9Kcn0dHq/9FgCvAvhA+17Q1wzgRQA/0z6XA2heyNeMVEKk1QAaat/fAHBVoV0zgKEABgBYwG1zfY0ApiMVsJIAfATgPKd1KDQNP51OkTF2EICeTjHvYYxtYozN0j7vRiryaEekru9F7bAXAVyofR4J4HXGWC1jbDWAFUjdn7yCiDoBGAHgWW5zwV4zETVFSjA8BwCMsYOMsZ0o4GvWKAXQkIhKAVQilTOjoK6ZMTYRwA5hs6trJKIOAJoyxqawlPR/ifuNLYUm8F2lU8xXiKgbgP4ApgFoxxjbBKQ6BQBttcMK5V48CuBWAEluWyFfcw8A1QD+rZmxniWiRijga2aMbQDwMIB1ADYBqGGMfYwCvmYOt9fYUfssbndEoQl8V+kU8xEiagzgLQA3M8Z2WR0q2ZZX94KIzgewlTH2jdOfSLbl1TUjpekOAPAkY6w/gL1IDfXNyPtr1uzWI5EyXRwGoBER/cTqJ5JteXXNDjC7Rl/XXmgCv6DTKRJRGVLC/hXG2Nva5i3aMA/a363a9kK4FycD+C4RrUHKPHcmEf0HhX3NVQCqGGPTtO9vItUBFPI1nw1gNWOsmjF2CMDbAE5CYV+zjttrrNI+i9sdUWgCv2DTKWoz8c8BWMwY+zu3awyAK7XPVwJ4j9t+KRE1IKLuAHohNdmTNzDGbmeMdWKMdUPqWX7GGPsJCvuaNwNYT0RHaJvOArAIBXzNSJlyBhNRpdbOz0JqjqqQr1nH1TVqZp/dRDRYu1dXcL+xJ9cz1yHMhA9HyoNlJYA7c12fAK/rFKSGbvMAzNH+DQfQCsCnAJZrf1tyv7lTuw9L4WImP47/AJyOjJdOQV8zgOMAzNSe9bsAWhTBNd8LYAmABQBeRso7paCuGcBrSM1RHEJKU7/ayzUCGKjdp5UAHocWMcHJPxVaQaFQKIqEQjPpKBQKhcIEJfAVCoWiSFACX6FQKIoEJfAVCoWiSFACX6FQKIoEJfAVCoWiSFACX6FQKIqE/wd5oWTiM/MhLgAAAABJRU5ErkJggg==\n",
+      "text/plain": [
+       "
    "
+      ]
+     },
+     "metadata": {
+      "filenames": {
+       "image/png": "/Users/hjensen/Teaching/FYS-STK4150/doc/src/LectureNotes/_build/jupyter_execute/chapter10_1_0.png"
+      },
+      "needs_background": "light"
+     },
+     "output_type": "display_data"
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Model: \"sequential\"\n",
+      "_________________________________________________________________\n",
+      "Layer (type)                 Output Shape              Param #   \n",
+      "=================================================================\n",
+      "simple_rnn (SimpleRNN)       (None, 32)                1184      \n",
+      "_________________________________________________________________\n",
+      "dense (Dense)                (None, 8)                 264       \n",
+      "_________________________________________________________________\n",
+      "dense_1 (Dense)              (None, 1)                 9         \n",
+      "=================================================================\n",
+      "Total params: 1,457\n",
+      "Trainable params: 1,457\n",
+      "Non-trainable params: 0\n",
+      "_________________________________________________________________\n",
+      "Epoch 1/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 2.1500\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 2/100\n",
+      "50/50 - 0s - loss: 0.4827\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 3/100\n",
+      "50/50 - 0s - loss: 0.4041\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 4/100\n",
+      "50/50 - 0s - loss: 0.3986\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 5/100\n",
+      "50/50 - 0s - loss: 0.3980\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 6/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3996\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 7/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3982\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 8/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3962\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 9/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3965\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 10/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3934\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 11/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3933\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 12/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3951\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 13/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3947\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 14/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3924\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 15/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3932\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 16/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3932\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 17/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3924\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 18/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3914\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 19/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3918\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 20/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3914\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 21/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3918\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 22/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3889\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 23/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3912\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 24/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3892\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 25/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3884\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 26/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3881\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 27/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3886\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 28/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3884\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 29/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3867\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 30/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3878\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 31/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3871\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 32/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3866\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 33/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3869\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 34/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3869\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 35/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3854\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 36/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3848\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 37/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3854\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 38/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3855\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 39/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3839\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 40/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3845\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 41/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3838\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 42/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3824\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 43/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3820\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 44/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3828\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 45/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3818\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 46/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3808\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 47/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3827\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 48/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3818\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 49/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3807\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 50/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3827\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 51/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3796\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 52/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3808\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 53/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3798\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 54/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3807\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 55/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3776\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 56/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3777\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 57/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3793\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 58/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3785\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 59/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3778\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 60/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3782\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 61/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3776\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 62/100\n",
+      "50/50 - 0s - loss: 0.3775\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 63/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3767\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 64/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3761\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 65/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3762\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 66/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3760\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 67/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3762\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 68/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3745\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 69/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3747\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 70/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3756\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 71/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3754\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 72/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3761\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 73/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3758\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 74/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3744\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 75/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3732\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 76/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3733\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 77/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3724\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 78/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3726\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 79/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3726\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 80/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3710\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 81/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3700\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 82/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3711\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 83/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3718\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 84/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3714\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 85/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3712\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 86/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3713\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 87/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3710\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 88/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3714\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 89/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3688\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 90/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3694\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 91/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3696\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 92/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3695\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 93/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3694\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 94/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3704\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 95/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3676\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 96/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3693\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 97/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3664\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 98/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3673\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 99/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3687\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 100/100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "50/50 - 0s - loss: 0.3675\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "0.36208778619766235\n"
+     ]
+    },
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD4CAYAAADvsV2wAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOxdd5zcxNl+Xml37869g7GNbcCYZpqNMYGAaaEGEkijf0AgJiSUEBJDIIXqBEiosem9hhKKcQAb4447NrhiG/d2btdvizTfH9JIo9FIu3e3e1XP72ffrjSSZqXRM++8lRhjiBAhQoQIrR9aU3cgQoQIESI0DiLCjxAhQoQ2gojwI0SIEKGNICL8CBEiRGgjiAg/QoQIEdoIYk3dgTD06NGDDRgwoKm7ESFChKbEihXW38GDm7YfLQTz58/fwRjrqdrXrAl/wIABmDdvXlN3I0KECE2JkSOtv1980ZS9aDEgonVB+yKVToQIESK0EUSEHyFChAhtBBHhR4gQIUIbQUT4ESJEiNBGEBF+hAgRIrQRNJjwiaiYiOYQ0SIiWkJEf1O0ISJ6lIhWEdFiIjq6odeNECFChAh1Qz7cMpMATmGMVRJRHMB0IprAGPtSaHMWgEH2v2MBjLX/RogQIUKERkKDJXxmodL+Grf/yTmXzwfwkt32SwBdiKh3Q68doe3hvYUbUZXMNHU3IkRokciLDp+IdCL6CsB2AJ8xxmZLTfoA2CB832hvU53rWiKaR0TzSktL89G9CK0E89ftws1vLsL/PT8HAPDyrLXYUZls2k5FiNCCkBfCZ4wZjLEjAfQFMJyIDpOakOqwgHM9xRgbxhgb1rOnMjo4QhtFRa0l2c9duxurtlfgzveX4LevLWziXrUdTP92R7S6auHIq5cOY2wPgC8AnCnt2gign/C9L4DN+bx2hLaFZMYEAOyuTjVxT9oG1u2swqXPzsYtby3Czsok7vloKTKG2dTdilBH5MNLpycRdbE/lwA4DcByqdkHAC63vXVGAChjjG1p6LULjbRh4oePTcfMVTuauisRAkCkWjxGyAfGTFiOuz5cCgA46YEvAADfbq/AXz9cimemf4eJy7Y3Ye8i1Af5kPB7A5hMRIsBzIWlw/+IiEYR0Si7zccA1gBYBeBpAL/Ow3ULjs17avD1pjKMfvdrZ9uOyiSmrIxsCxFaP8ZNWY3nZnwH03S1r6tLqzBr9U4AQFQPu+WhwW6ZjLHFAI5SbB8nfGYArm/otZoK63dVY93OKvTv3h6/eOpLrNpeiTX3nQ1Ni6TLxsCiDXvw9aYy9Ola4myLuKbxYEg3OzKUt1xEkbY54u6PrKXtqu2WB6oZMU5BMG/tLlzx3ByPfvj8J2bgjv9+ozTzi1PuN5vKMGD0eLz/1abCd7QNwTCjsd5aEBF+jpD5XZZ6IuQHV74wF1NWlmLTnpqc2i/dUo6hd38GAPhyjaVquPGNrwrWv7aISLhpPYgIP0fIQz56B/KP8tq043q5ozKJ8tp0TsftrIo8dQqJbBL+gvW78cWKyIDbEtCsK141J8hSTrTMzT/Ka1yCv3DsLADA2jHnONvEZ6CacCNJtDA465Fpofsv+PdMAN5nFaF5IpLwc0Q2lU5pRRLPTv8u8lwoINKGQPgKhX40B9cPyYyB+z9ehsqAoKqNu3NTr0Vo/ogIP0f4VDpSzMlvX1+Auz9aipXbKhGhfpj+bXi8Q1ow5Krm1WiurR9en70eT05dgycmr2rqrkQoMCLCzxGy5C5L+HuqLXVEpOqpP8R4BxVSGYHwFfsjlU79UGvf12jstn5EhJ8jfCqd6OVodHgl/Oj+5wt8LOsNjCv549uL89GdCAVERPgKpA0Te6QcLbL0KBMO/6pFd7RgEAlfNd+a0SRcLziE38A0FW/O25C9UYQmRURPCvzurUU48q7PPFJ9NqMtnxBImRg0QkPAJc+UId7zcKNtRP654b2FG/H2/I0AEEWOtwFEhK/Ah4usRJ5eevESyIZdXs8Fvld+Zxas34137BcqQv0Qs29qmNHWNJlnFRYFxuWGm99chPW7qgG49zlC60VE+CFgIX7fP3tyFr7ZVIa0YeKpqatRmzYAAPKq+IJ/z8Qt/1lU6K62asR1a5iKRltZgDeZd0qObCx1R0N1+BGaPyLCD4HIGQzA0s3lnv3nPjYdb8xZj/s+Xu74Kgel6z1+zOeF6maLhWky/PCx6ZjwdXimbEel4yF8WaXmnaB/+3pUGKWu0KJU060eEeELmLd2F7aX1wpbvIx/85v+HC01tmTPEfTK5Jobpi0hbZr4elMZfvP6wkCvm91VKZTZEbhpM4zwvSqdz5ZuK0CPWzf0OrJBND+0PESEL+An42bhjIenOt89RsAAQpKNtFFBjtzBb6lhssCgqc+WCcTtMcr6zxWp7RuGukr40f2uPzbsqsaA0ePxlsKzaXtFreKI/CAfFa/6EdFkIlpGREuI6EZFm5FEVEZEX9n//tzQ6xYKu6vdfC5MUumoIL8jYa8MYwwrtlbUu2+tDbncX48dRdiu8pKK1PYNQ32MtiMfmJy1zdodVVEUr4Tv/8O6b3+QYhfmr9uF4fdOKliK73xI+BkAtzDGDgYwAsD1RHSIot00xtiR9r+78nDdgkM0AwapHGSpKIxz3v9qM854eGqkbrCRy/0Vy6aKbWS3S4MxZX6dCLmjrkZbImDtzuqs7S55ZjYe+GQFdkVZTbNi6RZLIJzz3a6CnL/BhM8Y28IYW2B/rgCwDECfhp63OUD09AiSHuV3JCy8f+kWy+i7ujTKtwN472naCCB8pn4GshcOM/0qhshuUjfUVR2Zq0qnKmUlZYuUnX40tgY4rzp8IhoAq9zhbMXu44hoERFNIKJDQ85xLRHNI6J5paVNWzs2F5WDHKzCGLCzMomNu/2SD5dQo4FvQZTY+RJXRpWQwdH0kL9CpSNNApFnVDjEqmJA4ciHP6rIvOVHY3tG5Y3wiagDgHcA3MQYK5d2LwDQnzF2BIDHAPw36DyMsacYY8MYY8N69uyZr+4FoqwmjZ0BNTpFKTJI5eCXihhGvTIfJ/x9Mmas8mZ/jAa+FyI/B9VJHTNhufNZfAQqwo8UOnVDSiL8ukYnq1pf8dwcDBg93tvOflZpwz8pt3U0NJ1FXZEXwieiOCyyf5Ux9q68nzFWzhirtD9/DCBORD3yce2GYtg9n2HoPRPx1ly/tVxUJwQtX2WVDmPA3LW7AVi6S88++2+UfsFGHd99cdKVuAomi7Jl1hUZU54063a8GBfBMWWlf1XOH8sx907Er19dULeLtHKkDBPPTFvjbijwGM6Hlw4BeBbAMsbYPwPa7G23AxENt6+7s6HXzge47vgP7/gz/Xk9RNRGwboYbSMJ34u6ErTYWpXMLuL7ukGWtj0qsxzY/yZFXIoK4pn+t2RrTse0Jbw6e32jXSsfJQ6PB3AZgK+JiI+A2wHsCwCMsXEAfgLgOiLKAKgB8AvWAvLbej1E1G3kJVnYr3ISrEWMD6DOAr7n3soBbwZjUcrkOkI2fIvfLxw3M2/XiZ5LODJiUEmBuaHBhM8Ym44sdkjG2OMAHm/otfKBymQGtWkDPToUZW0rvgBLNpeja7u4r438fF6YuTbreSO6B17+ch3+Lujnc4Eogcr+y5ZKJy9dazOQYxnErwvX78nbdaLH4uKUh75AbcorrGzYVQPDZJZbbIEnxzZXxPzkB79AaUUyp4LLstpADMrikFU6r88JXp45XjoR4+PO/35T52PCXoVV2yuxZkfk7loXyNHKhcowGgn4LtaUVim316QNdChy6bhQHNHmCL+0Qu0NokIuOua6ZMJ0jbYRiuMaatN+o18YwlQDVzw3p6FdanMIqumQb0T+U9nh8/Ur0C1rc4RfF+Q7xa5bFavtUv78dbuxpzpVZ7IHgG+jAvF5hWyYLRTJRKo2y3tp707Fgfs37K5GOlP4G9WmCL86lfF8z2ZMyrfE41bFaru4cGz9jYHz1u2u97Hz1+3CkD5dkIhF+QI5woy2eUVE+FlXoGc+PA0AcPf5VkxqoVQ6bWr0H/LnTzzfs41vWcfZUDiXi5T4jYpV2ytw4dhZuGf80qbuSrOCKngNAG55K78FeyKVTt1RqNVWmyJ8EQNGj0dlMhPaJt9GLMcPP69njZANPJ/+N5vKmrgnzQuqIjKpjIl3FuS3JGfYazRj1Q7MXL0juEGEvKLNEj4AbCkLT66lCj75hf45ztJUqYJyQeSl09goq0lj3BQrknHB+j341cvzmrhHTYeqZAZjv1jtFPnxRSubDDe/lVswVa54dNK3voheEZc8MxsXP13f96n1IlLpFADZdJaq3WPiz2Bs4pF6Xc8x2kpP86KnvsSol+fX65wRwvG3D5d40lF/sqTtpqaevmoH/v6/5Xjw0xUA/OPfZAzjF4eXm6wr/vnZyryer7XgMFqD87QZvu2FVn61acLPprFpLKPtrDU7o5DzHNAB1fgscSsOozXZG9t4d0FhCkm0RHCCr6i1VJkqlU5jIWOYeHt+flVHLQkfFd2BRxNPNPp12zThZyP0fBN+lEsnOw6mdTiI1MFrw7XlGKRtwi2x/zRyr1oH5OGskvAbC09NW4Pf1yGGpa2g0NTQpgk/u0onz4Rv/42yZQZjQtFt+F/RaOW+OCzJNAV/iguOHihDMbIH1327raLNpeplkg3JF3iV97iT4PPtqIiqX6kQqXQKiGyELhu1GgrmMj5Wl1bizbmNlyWvOaC81p+aAgAIJq7QP0ERwkkgYRN+OiR8ZF7xdXg78bfQ83yzqQyn/2sqxk1dnaXHrQsyn6vKROaKDqjGSC3YwPvugo1RSoVmiDYVeCVDfgFiyGCEtgzTzSHW/hCJZ37Rr3BO8j5sRfecr8clLI0Ip/1zChgDfn7MvnXveAvF+Y/7jVQAcLY2B3+Lv4g+FO6exwk/KQzbgbQF5awddqKzs+0wbW3oebaUWV4q89fWP5CrJUKUuH/35ld4d+EmaX/u5/pXfCxO1+djRO1jynfgd28twtod6rwxEZoOrVrCN02Gt+ZtQDpAVJdVOjfF3sErifsxnJZZx4e8Ad2pAmfqc+vUH9EPvy1KP98FEEA7sgi4G1WEHh8nW8JnLuFPLroF04turFM/Yrql00i3cpVOKmN6Yk3c8Uc+sgeCx3s3lOPu2HOeFdh+tBkAUELBq7JHP19Vn25HKCBaNeH/96tN+MPbizHuC/XSXZbgB5H1EnS1icdgDAmk8UBsHHor6rVkoCvPu7b4Ytwce9u3PcqWqQazbRqURYMZD1DpcNIh5KaDu/tDK+I2rajY1JpwxXNzcNhf3OjybBGvQYLRX+Mv4rLYRPxM/wKadI8JDJ1QBR2G8liOODLoBDcXUvQOhKNQdr58VLzqR0STiWgZES0hIp+4RRYeJaJVRLSYiI5u6HVzAU9nvLNKLYXIAh4fzKZ9W0yT4WRtIX4am4q/xl/0Ha8mfOukN8Z8lR5do2002BFDxr3fLDfCT8B6npfFJmJR0S/xTdFVnv3tUZvTtdfYK41MvnNnNDPMWuMVUpyfGzD+kgETYH+yYhfujr+AW2JvefYRGBYXX4N/xJ8K7cvY+L+wuPjarH2OUFjkQ8LPALiFMXYwgBEArieiQ6Q2ZwEYZP+7FsDYPFy3wZCNVDFbSuFEbjLRUOgnd0Nx+4qgNkwC3iV1W8eq4svxH9u4yifYw+g7Z/9e2IXH4o+iRCDxIrjqic5UjQ7kJfh/x+sWEMfLW7YVyHEgZ2hzsbb4Yuxtr16DCN8UxvnpmhUgyFdlnWFNnhfq00KvfZq+sN79bt1o3DHYYMJnjG1hjC2wP1cAWAagj9TsfAAvMQtfAuhCRL0beu2GQtZZ6pKEb5jMNwmIEHXJZ2pz0Au7URziaWIoVDptufzbUO1bAIBpk8eBmqtXHh1/HT/Uv8SZmmsn6Uzh6ZFP1L+u0/W/2rAHd3/UdhKqySPtp/oXAICrYv/DHbGXkQxIWZ0SVGhcyOGE353KG9iLtg2tpRG+CCIaAOAoAHJyjD4ANgjfN8I/KfBzXEtE84hoXmlpaZ76pd4u6/BdlY51gMkYYsQJP4Yu8BoV+eCPI4NxiYfxeuIelIT4gBu2RClyfCu3G9YbXL3TjpK4Tv8Ag2k9elLuyc9kXXMQnp3+XfZGrQSuDcka34YtxFwbG49fxiYgmVYnEzSYSxOmRBm3x16tUx9iWXT9rQU1KQO3v/e1k7gvCHqO4zRfyBvhE1EHAO8AuIkxJk/7KspVUh1j7CnG2DDG2LCePXvWqy+z1+zEgNHjsasqPABH9tLR7S5xA5SWqcYDtm7SZISvin/lac9/FPde2F/bguIQr4XdSydjIG3xGM/akoSvwcTp2jyfcTWuIIEf6Vbe/Hvjz+GP8TfwSdFo9ERwndULtame7yuKrshDj1sX5KGWkV7/TNpLTsdrX+NEbZFnTBvQoMNwxvxAzdLvZ1huVBJHeIba1oJXZ6/Da7PX4/HPvw1tJwsmhaaDvBA+EcVhkf2rjDG/tdKS6PsJ3/sC2JyPa6vw1FQ7O+K68ELMsnSt29I8J6BE7S5nX4L8MzWfGES9/UnaYl87jjeL7sbkols8D7Xt0D1wtf4xnk78Exdo051tXxVdg76U20quS4hK56HEOM/3OOUuSYbVIW5NMB0bkgVDUlOytDd77KuJ+/FS4u+eVevh2nd4P3En9tW8z8zM0S6lmtxbI3iGUMriodHiJHyyftGzAJYxxv4Z0OwDAJfb3jojAJQxxvKbls/TJ+tvNjc00zRwvPY14Ej21s3ny05xEKuMsVzdI+rt/6bw5vFd16PSaTuUz7099tXcjJVdqArn6l/mdPyQLAFVYeiF3YFum7e9+3WbCBIyJRuS7IhgZpL4MHE7XoyPwT5wg+DaSWrKbIFtYYgjg/LadKt3W5DvdRD60XaPS2uhPfjyIeEfD+AyAKcQ0Vf2v7OJaBQRjbLbfAxgDYBVAJ4G8Os8XDcEXAfPv6nvYv+VL+LVxP04VVsAwHX741KIaboPQkX4vF2RQvrnuEb/CC/E/+7ZJqpx2hDfO8vXEsmw3Zv8MQ75RF/ajjnF1+PX+geBbYJ80FsT5KHmK5ydSWKIthYn6Ysxs/gGZ3tYcBVHIscVVRwZTF6+vVWubGvTBnbZLuDcPqhnYfBPikbjtthrzvdmr9JhjE1njBFj7HDG2JH2v48ZY+MYY+PsNowxdj1jbH/G2BDGWEGrUDgSfpZZtn2FpfrpTbuQQNqRIGNcz2i6RN6B/MVSDiGrfZAr5nBahj/FX8NI3ZsVUHym8gOuqE1jxH2TMG/tLrQ2cBtJO8lfXnavzDd62br/U0JcA1sjAcmQ7UW+uIWM+jmUIImNrAduS1+NZWZwKpBcDOV8VbxhV3XWti0Nlz4zG0ff/RkAV9jUtewi+8khOYnyjVYZaSsWvAiH64Z5FLlh4HxQkuESeVf49ceXxiahCCmH8Neae3n2P554TPimVtzLaqeF6/dga3ktHp4YbuxpSfjZk7Nw5sNTXRVYyIqooeBBXCK4S22YwbAtrLTE30gEtIdXiEmn1ITfkWrwnnECXjdO9Rl6RXSDOjXGEKF+AY9r+TTnd7TlYN46KzfTpj01jkOISocvqxbFgMOWoNJptsha4MR+KAxuPhfAVdX87BvXK6drQJ6XMfGnHcJ/2jjHs68XuUZj0R3N66XjPR8PfimKtZ5HM+e7XVi+tQL7dLTSGrej7OmL6wuN3Bs6Sv8AkxK3CIQfrHZo6baUbeW1GDB6fKiwIwZe6URoLz0HMxksddewIgDhdpQTNXV++w+L7nA+Z3PLXLW9EgNGj8ekZS13Qjj7kWnOakql0klIgsd+2lasLb4YB1DhM4y2HlZRwKs6UdxJ/lBgegyvfFCWZFzv0q4B0sux2jLHba2MtffsW226sWUDya1oFWa0TdmEn2iBhL+zMon563Zj8x51reCU7fbXAX5iWW72823juClVP5PP6Pgb2F/b4rxgrVnCX7rZGquvzl4X2MaxaRFB18inWgtLXleNosB996YvBgD8MzEO+2AHDrVVnRa8NzaOTOi9XmBLyRO+abkV4Mpq0k6QpUqjEzQOT9EWFjzfVstjlTqACRINH2RxZLC2+GJcpn8KxixyHaEt9XnaDCJv+TVdkBznmIOdz0VIOz7Je9DB2f4/4xiPMXd84jahX0IfpT6nDNsltIUR/sbd1Rh6z0RcOHYmvjfmcyzb4o/ArKq1JMouCvVYbUhRk53o1KC+vZS4H4A7kR9NK3E4eRPqZfPoas743zdbceULVkRyJiRdhCj06BqhvWQ76Y7gqNkwwi9lXZzPM4tvwPii26HDAMHET/UpnraydCvDDJGMWxL45Kpp5BM2r9YnKI8pR3v8bMqpeC1+T8EEkFadD3/Belelwu8fJ5sbYu+hnJ0IADhXn+3zn/+5PjnwvKLvbBdUOn7Ku1hHZ/su1sEjQYleDPGU0C/JzsUl/DWlVRj0p48x9Q8no3fnksC+NBds3uMlj7MemYZJt5yE/Xu6k2BPW8WlJnw/oVycuh29sQvLzP4N6lsn2+DO0yu/W/RXAMCAWr93xOKNe6AR4bA+ndFS8MJMN1o4LCGcmJ5bJ/IZbfemYEeBWpYI3KeaDFYXXwYA2MK6ebZnU+k4knHLknd84OpijchH3jfH31EeU87aoV1qJ76n74R6Smg4WvhtzR1ccngm8SAAKz+InnFVCx1tUng5cxoAYJ8QV8H3je85n5ezfR1d6E7mSqIVaOdzP+TotsfN+SJLllyH//WmMqQNhknLtmf5Zc0DKoFsZ6X39x9KlrqhE/lVOknmSvh3pK/EVanfY6Z5GN4xT/TkcgnDGckxOCX5YOD+Pjm4f573+Ayc+9j0rO2aE0QVYVjZTnGsaQqVzkAtWI2iiiD/v9St+NAYEVpysrc0iYS5MANeomzJ4M9B13K3D9UIE2ek0mkg+E0/QrM8BlIshljGH2xzT+ZSAMDZ+hxnm6iLP6D2Jbxk/AADal/DJOModKYq3Bd/FgBQCVcSr0FRoP/yc7PctEI+o62UwKqlKBpUukp50HJjbUeFDl/0VHjFOB2fm24G7WQIoXDsZB2xgu2LNax+Ofmy1TduzhBVBpmQ3+HsIiCONIrIq14ZSMGxkO8bx/u2fWEehd+mb8h5QgZEV1AG1eh2ibKFEz4TJHxh+9G0MvAY0b27e22wLaYhaBOET+Qn1jRiiGe8qgWTkZJcxCVrBjHwkJUaJNBXKMsnztDcq0EF0WjjM9rKAUAtxpoY/IKepC3COZobTSvaQ1abvfGRcaxD1A9nLvAdryKU7yQXWDdNAOFDY4SzfbZ5kNTLgOpnLeY++yFOVjWp7J5IBL90DwADKFjCT8Kr0plkHOV8zjB1ISAVOqAGDAxriy/BuPjDvv3cBNHSJXxTmLjEd5yrE0WcnbwPgDfz6NE7xxekX22C8Ccu246tZd4BbhG+V9K0yN4/0HiGwPHGcKl9QtkOAHYLBlwZcY+LphdyTvKWS0PunXwx8Xc8kXhU2WYR2x+/Sd/oEHYF89srmDRMpxpDcHLqX55tYm0CMQBI9pwKcs1USfipjNkiktuJXV++NdjTpvfu+dgHO5AyTBg1VrvHMj9y9ndXeOnMNwfhpKQ/Y8qo9M3OZ1Xq8LcyJyn7IAYwqkqEOu6MLVzCFwOvKmvDDdV87IpxEVWxrgXpV5sg/O92VOEn42Z5thGAuKTSCfIG4e/TInN/z/YahSHrotSfcHLyIWyVjFUiEsg4romyhG+YjZs9Lx8YeNt4PPjJCt/2JyavwvjF4SmTUnZNge/Y3gCAjaxXaPvzk3fh8rTl8XRR6k94JnMWAJnw3Zu2h3kn3iCjoWEybClzX7iMYeLAOybg3vHLQvvTHJDrpPTjRddiYtGt+HDRZke9tlyKnBXVlwDwljES6+xnI0IsM6lagVWgnbtfWAF0QI1nUBdLeXoMR4ef9ec0a7i1LwhD75kY3tYeu6JKpzpWGKeBVkn4XVGOm2JveyS9HZVJTz77GDLQmVfHLkuDHIttoo9J6oBa+Al/lnkovmO9Qwn/1tib+Kb4l+iEKt/KQxY0W4KEyZi/nB4ATF5RiutfWxB6LDf4vWKchp8n78Qn5jBn34DuLmksMA8A4FWHzTIPxWzzYADenO2iF1W5QDyAkDZDgmEy3Pof11OL68JfmlUYXWo+kZP5wR5HnOg72NJkleRhs515Jcv/GF5Jfa55oO/UGQXh/1fQ+R+YfAk32rEUHagGmumSPE8rwFUgrpdOy2b8sMArAHgtc7Lb1l4LiwZtk3JXk9UFrZLwFxaPwk2xd52kaAAwgLZ48tnHkQGZBl7JnIoXMj8A4Bpdr0u5ZXlvSY1CGayJQC7UvBftdj5/Zgz17CtnXqIRMcDOFtmNynHe4zM8+2SJv/nTfcPgSoeE2exgiCq1uO4OT05EMfJOunzCCFLpiK6ygH/S5jBMpkyg1hLq3pqMYYS2FMMpZDWS8UrSvNpVGeuAY2qfwD/SPwcADNTcFdke1t5X8OSy1G04tvZxzzY56+bzmTPwNdsPpyYfwDWp3wEgvG+egDLWDsVIIZZ2V9a7bM82Pu5zTTrW3JFtpSJ6NqkkfJ0Vpm5AqyR8DtEI8nz8H559A7VtKDIqUYsE5tiGPa5eECX3DDTngcSkjID72V4NP0/eiVHpmzz7akICVThU+mRZoG/uziN19W6Rc92sYsrCZz7clr4a/0pf6DPCljJr6dtVyJUvqnSqUILNwmorTKXj7adNQAwYMHo8Fm8Mr63QlDBMhjcS9+CtorudbW/OlXL8p7zqS24X+Yrtj1J0xQTTsk+Jvvmqcp21KMI2eFevcm1n7uSwmvXBZ8KKzSqeYiKWdp9VF6pEJ1Q545zPuS1Fh//Jkq1Ytd0fVyIGXl2oTcWr8Xs9DgNiLQIV4WusMHUDWjXhd4I7yFXl8TSYyCDmeOZw45NI+AZ0TDaOBABMNQ73HD82cx7STMcitp+vmIRK3SOjA/wpCOSyi81dpVPXtMJbYUnqM41DcFHqT3jDsJa2fbv6jbWikLcbnfCIcaHPgPst6wsAniyOokonhZhHSg0KazdMhgHpbzE+cRtKUOubaE3Qj/IAACAASURBVF/5svmqdlRD5I/vSPV9U16DbBwZbGbdnPu5ifUAAMw0D3Xa5JrkjkmODvJ3DgM6YjAQM1xniScT/8Li4muECTY46VhzxK9eno8zH57q2y7GEzyUGIfj9SWeydQE4T3jeCw0D4BpqyMHa667dqEIv1VH2nYhl/BlKYQjA81Z5juEz0QJX8cidoAnKpPjI/M4fJQ8TnneXHzHO1CNT2fTvOndj7oS/jbWDfvQLmSgY5ZNLuNvOAGvfLm+XpWn0ojhB8m/Y6ugexZVOikWt1YVNn/IqzSOjMlwWfnTOFRbhyO11T7VWlBNheaAnAJ7JAk/RgbSTDS8xnFLr2fw8XoNWpphSfHVOV+fchy1BjRoMKGZ/olEJnydCF9vLMMjk77FrdsqsF+P9jm8UY2L2rRd71qxyuW2CFFgEyuHxWDg5vT1AIC9Ydm/jtWWO/u1SKVTd4il89axvZRtDOhYwqzQ/XcMK9UC19lb++t7i7IThC8fOdwBX4wkfqmPB8zCzPT5gip3y5qiS3Ct/qGyPVfBiPeVQA2KLFzJ+qFccIMVJfw0YljM9nO+B6l0RNIcqX3lS3nRnAXObITPGPMRfgIZj6cNAGxN9EMNikPz5qhQIdmrgiV8S6VDpp/MOGmKEarXvjwPE5dtw+6qlC91R3MAL3YS1xWu3PYj4ZMC4M0SK640VRwj2wvzhXzVtH2OiLYT0TcB+0cSUZlQEevP+bhuNpyiLXSkvSApJMM0bGS9MKD2NfzP1mNuF5JBqXyM84WOcoqBTBJ/mf89XK2Px+uJe3FH/FUM2B7u0tXUkCV8DSY0Yrg9/rqyvaw+A/JPppqU6O7W9K/wrO2+GeSHL0ppv4qN90v4zYzwp31bit024WQT8BkDkBJtHCbiyPhLHNqPkqt5ns+ckVNfdqIz3syMzNrOZBo0UhO+aTK8MOM7rC61+il76TTHlS8n/C7t/OpbrtKpFSLnxVKR4jiUDeNA85fwXwBwZpY204SKWHfl6bqh6EC1uDn2NgC32pIMlUuZmCIhlyo+Kvz+B6772jpT7Vvuk/BrLSPz9bH3cZRmFWQxCzjh1BdPTlmNmautCGM5MjghVf86ScqRzoleHOS6plaY1FeNwp/Zz5N3Yiu6owbF+NJ231QZIgG/7STdjL1z0oaJy56dgxP+/jmA7FHCDPBI+ANpCwbSFp+EL05yA2pfw98yV+Tcp0XMjVEJ6g2X8KEg/OqUgb9+uBSfLLE82HRF0rHmhmTGTqMe1zH6HW/yRVlFBQBnCIFmMcoi4Tdnoy1jbCqAZlmTb6iduyKIuNUVfMKLl+eCXp2KAQArzT64Nv07ZRuf0TatSCoWC47YbSrcP2E5Ln56NgC/SocTvmF747yY8Nbz5cZt8b7LDhn79WiPS0cEl9LLBu6lYwjPkUuzJcKyWhwT28prUZ10XzJ/dGTzEfG52qPKTqOQk0on6Ur4k4puxYHaJp8jQ0P4tToklQiHo9JRSK/yOGoJqRXE2/7G3A3KNqIccVPsXeezKOFfc+IBvuOau4SfC44jokVENIGIDg1qRETXEtE8IppXWloa1KzO0GFiqjHEE/AAwOddw/GeHTiSK+GfdrBXiicAEy9YjLNT9weqhXx1cm0pTBzqhtbcTFVeyCqdIls3qVqmAm6AlLifyKvD//C3J+CeHw2pd5/eNb4PAJ4IUb6Seyg+1tl2Z+xl5/P9E5Z7XDMXLF2Je2PPYn/aZPex3t3JO3yuu770S4pYjpTfdVDOZNkQj7BcbF2OhG/43yl5RaVpJGWStT4v2rDHtxprKoT1gt/KoMlY1OEf3MefRqGlu2UuANCfMXYEgMcA/DeoIWPsKcbYMMbYsJ49ezb4wvw91WCiDO1xe+Yaz/6gGp0pO11vgnKbaX9zyiDPd40IWqIYGcRQbkfwjsuci6mGS2Q+Cd8mfI9apJmva4NUOoGEb28XJ0FZmmtoecfXjFMwsPYVlMJvi9lHILnz9JmB5+i8dRYuiU3Cr/SPAADVyQz+M29Ds3CTlUmkxJR87H3R2gBL+TPDymjITxOfdzajrcoRQZbwdQK2lXuDxeZ8twvnPzEDT05dg+YAp75AiDBgmswJwpxuuHKu6DxAml8gbNGBV4yxcsZYpf35YwBxIurRGNfm0GEqpRCVDh+Ak71xT0C6hWzQNJfIStEFx9c+gn9kfuF5MTr6JHxLChPz6JtGYR58vsBf1P1oM/4Se9HJm6661y9lTlfq8DXy6utjekOHJfn89TPMf87uVIGDSe1fzwzbA8Oe8P/71Wbc+vZizF+3W9m+MSHz8n9T14bun7l6B8ykX8KXkYt7pyog6pFfHAlTIvnfnHwAenTwJxfUYIIUbpnySrFckXCMl85cvjW4MldjQp78R8dex+Iiy52Vr05M5nrcdBbcxMXKXyrCb9ESPhHtTXYkBRENt6+bvRpFHhFM+Opb8LRxDn6ZugWfmMfU+5oxoWzPJvSECc2Tr91nQLQJX/QyeWf++mYhVQaBv6hPxv+FK2Of4CCydJkGNFyhf+Jp+4k5zJXwWbCEz5FPNYpsoOQ4TZuvvradikBeqchJ+JoCIjG/Nnu9r0bwup1eaf7/np8LbF/mi3KWkW2UTfvDyfjqz6f7tndrn/BEln9qDEPX9gkM6++PyNVzJHxuEPX2r3m9Bx8u3uz5Pir2oVNdjcNgzJHmxQLwotcgKcp7ac3cLfN1ALMADCaijUR0NRGNIqJRdpOfAPiGiBYBeBTAL1gjsRgRj3gznYg2EUZALm8TGiaaQ5GrsU5uZZrqMm1jMhfh2tTNWGTuhzgy6EfbcE/sWcuAqFh26zCxtbz5+SBzpG0JnxtAuUqHgfC3+IuetgZ0R8IXDapBibLyOULkpGAcQ7Vv3euJOwyL8A3FmGlqiDECt7/3tW//DW8s9G0zty3FLPOQ0PNmU43369ZOKeHHNA1TzMNxX/oiDKl9BkvYQBD8E7YZotJJh9TildEczCkL1+/GK19mDxRkpql0Bb4980vnM2l+YWRE6X8a1sEA5MtL5yLGWG/GWJwx1pcx9ixjbBxjbJy9/3HG2KGMsSMYYyMYY8HK0zyDE5AOU6lXDpLww9CjQwLfHxSukWLwSvgcKcTxqXmMlY8fGTwafwKXxibhcFqjNKxpMJt1NSY+b8sZ/1RlDDNMc7IAGh4dfrg0/8gvjmxwP7ehK9KKyX2kvgg94E+7QZzwFeNDDKZpCtRH0t29Zw+2ITzH+kM/PRznHbEPnrj4aN++j357AgD1aiymWyq0p4wfOmmRibxt+3UrQQa6VdxcIeG/t3Cj53szHvIAgJosY8Ax2poZz4odAJaZ/bCRufZJTVDpbGHdsNA8ANV6J6UA2FA0P/Elz+BDzlLp+AdrkA4/DH26tkOHovDjGGOhCaDSLIYEZZyESrKvNIcGppR0DZM1C28FuQeyH74IAzqYQ/iiDj9cZhMLodcXJjTsQkflvlcS9/m2aYZapQMoqpI1MrI99gHmRjwUH+tJBV3Ean3pv+9LX+T5fkCvjnj0oqOUkaO8qLtqTKu2kfOfC0ulw5SBV6/P8bo1NmMtJgBvJtcwqH6r7Bko3r5drCMuSP0V9w+ZACTqZz8MQ6sk/F+k7sBt6auRZrqTflfLo4QP5BDdiPCMf2noVopmcWOASkeFcx6dhtP/NSV7R/MAxhgmL9+utCXwTY6EH0L4GeiAvXz1umWql+n5doUMSqtwkOb3oeYSvqok4ksz19Y5h1BDUZs2MHOVFeyWzbh6Y8U/caE+DYcIBukSJH0ZXJ8yfqg8PmwCVu2LK1ayRN6wuR4ditxI2xw8UOTfaDJg8UbvSmxbeS2mrMyf63ZdEMuSzdPpvYLwNUlMEhPFpaGDWWJeQ7uoRKsk/C/NQ/C6cSqmmofjWG05ipCCBqZcngf54WdDtgfCGAsdFClbpePdqJbwTWZJ89w7gTGG5VsrsLq0yp8GtwB4fc4GXPnCXLyzYJNvn3wfwgjfhAbmEL6gwxcG/BmHqnMe5QOqEn6AFRxn9UMwpHEvHcUk8eCnK/Hc9O8K0MNg3Dt+GS5+ZjaWbi7PLmyQNc65Wi2GDBJkeCq0/S41SnksED7RqoZ0TLEisNR01vbj9uuOJy8bCgMEHSb23/Jx+A+AP1319vJaPD9jrWfbBf+eiSuem5P1XIWAKOGHPQ/K+CO75SBQ8Z5yPpq4dHtBVLmtkvA5TtUt49Vl+meBOvzvFOXbcoH8kAfv3RH792zvVGlizEtk3dt7XdQsHb7hWOsZCGatn5A0mGAMGDtlNc58eBoWb9yDZNrAPrCkvbFfrK5X/+uCTXss4tha5k/n7Er4FvYRirpzcKJhAMgmI5EiNCKHHEbs1913fKGiLp/PnIFl5r5OUitxBaDbEn6MDKWUf/+E5Xh44kpfxbJCgeeY2VWV8qy0ztDkurAMaTtfUW+yHOG4m6+YFO1d88TAa4XdblXa4u4dFKnAiRwi+9kxfdGrY7FjtO1VsST4AjZyIbtNe/zjsTFxlT4BZ2pzlH11fPQVqxkf4QuMz50aCuWo0aoJf71pGUa4/69Kwl+dYwEOGfIjLo7rmHTLSIewGFzJJxHTfOodbrTlhE9gWL5hq89fXIcJBssrALCCUVIrJ2Fm8Q04X5ve5HnD+XR1oGZJ/xfHJvvabGb8nhBM8ts+xFvTmLrbl4wfYK45GO3snEYi4WumRZI/1mfgu+JL0QUVGKV/APHJPzzxW/wmSwnHfIGPH4N511RDtZWedhoYiuwkXTxZF68bW5tjFsy6jqmeHYpw8mBvkCTBndT5M81AD8lN5X3wYZXGGnPMb6+oxVmPTPNNLowBf46/jHGJhxXqPYYe6U04XZvnGKiTzI2Yl9W0GhEeSv8EgOuufN3I/QtSBKZVE/7PUlZSzmIklX74YsHmbEZYD5jakAq40pEpGG1N068ASiHuqWGpw0RlVTV2w2ugtIy2zHFbi2kEY7tVMPxUfWGju6iJ0uWSzWVgjHkKzajApRYGUq6ymmrSSkNHNYrR3iZEsYZB3PC+4P+IP4XR8TcwQvOWEdxdrU7Glm84hG+aHv22XEA8jgz6Ziw1H4/z4CuYXPLdAHVPsUBE6CMVsCFBpcNPZ0ALtKPIeu3m4pn2zvxNWLalHC/NWuvZLj4DOR9+XyrFmI2X4+nEPwGb8MXawXLmXo2AzXYBGgMa/vGTw/HHM72V3fKFVk34W9EdlawY3alCabR90xjpfO5cYr3s+3QuzvHsQQPSHeS8Lqcqm2GK6R4dfhwGdDOFMiYTviXhc4knphPMcivgIwO9UZySxe6LY3vRhrI6mZYYAKYg90KWs7v1jMHo08VfTQuwpKkqVoQiSkOH4YTAV7EidDS8EbVdbf2/TFjJTOMYb2MO4XufgWyD6kfbUcysFQu3p3CVTi5lNwGgvKbu0d3yENeE/Eh8lwkN+9J2bGQ98I5xgqe9fF/7l83D2uKL0Qv+yGZ5tBQypCfIVie+02lpDAwi19bFVYPVcHlFJeHz5H4Z6A1OLRKGVk34AFCBduiAGqWE/5RxrvOZk851I/dHLggaY5owyPk5VW1dlY6FGGWgsbTjx+z0y9bhuxK+BlZt5YPpiopGlfCJyPNyaQSAhc85D6V/4rFT8NailONR6eSxv0C4PtqwJXzASlXNCTKNGDpmvEQTlEc/lTFhmgxz1+7C9gIGyHE7xtbyWlzwb7fwfUcpyvYYbYXzeYhmGZa5yqoGRTg1+QDOSd4beq1cVi3nDOmN968/Hr852cr0KD8360nz8W/tPUOfh45Ug760w8lVxSET/vDdVgGd4zV/iY1Ne2pwkxBcVsjFgKOLB+H5Gd9hR2USjDHUpNz+yhK+SOi6Yd37KiYQPnkJX9fImYzjyCjjd/KFVl3iELAMhiWURIxUOnzRU8T6m9PYoeCibo5UI/nhy6QvG21jMKGbadRIj0QjEwBzlrgxnWCmLamhG1U0ujpEHNtElgQUtExfZe6Dx4wLcLZueVKYUPtfitJgvhGWU98EOWqcBDKOik2DiU6mt2h5UAWilGFi7JTVeOATi2jXjjknH932gduDXpm1zpNUrJ1UU0H0kjpdnw89bTj5jWpYIiebVTYj+Yp7zkRMs+xSR/TromxDQjCd6l2RM8h2RYVHCt6jWWkZepG/ePzs71RZPvM7gKasLPVM4Cu3VWDclNX4YNFmnHRgTzw88Vustbsru5CKKs7duy3BoSZEpUPkOo/0o1KsVHg95QutXsKvRZHzUvDUCsfWPo7jah/ztNMoWBpXIWgZSYJKx+uW6W2fRswTpBRDBjGWRopJhG8HXmVsw1BMIzA7z0sR0o2uwxcHN9lFKoJiBfjA/lX6ZozLnItVAtmIxjuRX5S+/g2Q+0XikWEKNgUNpkOWOkzEmTdTI5/U5J4k06ZTDAYA5q8rTFkI3Zb6ZGOmnM1VdostQdIx3uZauvDiY/fFj49yn9VnN3s9eopiuk8NJz82EqZa1TNNI4ZHMhdgvGFVmdtbStWcsifiIGHCc+2sLeqOK56bg1vfdouacMPswvV78PDEb4MOAwD8MzHO+bx2y3YAXvuJSqXzHeuNScZR+EP62pyDuuqDVk/4NUigA1mEzyX8beiGLfC6/4mS+b0/Pgy9Ooa/HEGDTBPOoykk/E7FFqFnoCMmxP7GYCBB/jqjXIcvqnR4UJAGs1HztDPJWE2wJ7aAl3KcHdiznu2FMZmLARA2UF8AwC7WyWmnEeHYgZZExyM6gfwYc8UzbGLeZ16NYmdM6BLhyxWHYkGTGgGZjImusGIkLhxbmORqXOiT1RdyZLP8vR2S6E9bAQCnDBmQ07Xiuoarjh8IADh0n04YtJc6QjkUFC5EpaFj4l5X4+GM5Z2yD3lzKerMLt+Yg0iTUxH3gsLE6Ji6pCe3/VRl0eEb0HF1+lbMYQdHhN8Q1LAE2tt554NytAPC4ARwybH9cdZh1hJr+IBuOLi3RU5c6jnhgO4hXjq2Z45PwrfQt6ulo88wHToxx8c7ZtcZTSGOu9OX4q70ZQBcHT5X6RABZHDVA6t3GcBc8OLMtVhdWumZVEyPDt+S4TXyk+GA2tfwH8EozjE3PhSXpG7D84ZbEVMjwpmH9ca8O05T+uE3BGLfz07e73weVjsWSSScADCdTIcsNZg+FQ7/LhcNiWmEMyvfw8LiUehH2/Lad8/1AyT8Iil4T/T8AoDuVI47468CAMyY2nitAlcj564fl9QU8Kt0RqVucvanEUNMJ2xl1kTfWyZ8W6jJpZj3jsqUkzq5UPALH+7vHczWYVTsQ+VxD8SfAuA12tYyb9yCTBOq1Bb5QusnfBQ5tWNNyk748gA/e8jeOPNQW7/WtQQzRp+C350+OOsyUjTa8u8A8JcfWhkLuQ6TS47702b0z6xFb9qJZ42z8V+74pYl4TOnIhBjANk+4oWU8Blj+MsHS/Cjx2d4tpum6UygmmZHFNchlSsBmGEO8Rin+W3q0SF8VfWrE/cL3V8U03B4386ebeKEWAYrN8lE4yjsgNWOZ8PUYDpkGYMJHSZmm65rHP+ND8af9Jw/pmv4fq1VW7Yb1JG8+QAX+oyAkpIcskpnQtFtzmdTz9UDTXgf6mkRFauYcRlhI3MTDmagI6YRKtAOFazEP5HaYzymECZkHD/mc3xvzOf16qeIymQG5bXe+8fVUfJrJhrxc0llXCWodC5N3+bZJ08mDa8HEYxWT/i1gkpHLoohQlTpBIEB6NOlBLpGwTr8AKMtR7uEpbLhqgSe4OpCfSoA4AhtjWe/BgZmmjg+OQNkkz8P+5d9l/MJ/vMqkhnPaiY2dxyWFF+NPigF2Vp6eYn6ROa8wPOecZg/sjkX1c3aMefgtrMP9m0XXS5X3HMW3rnue9K5RSmfMKL2MVyfvtHZ71Xp2PfVTrEw1Tgcz2fOAGBF3HIUCXUMYhqhk2nleBGX7fmGK+HLhJ/BbPMgHFP7b7tvFmFtZt5c9ACQSqgNrOrrBbsUq9A+4VVFkvA/XxWK6soUizneKNtYV/Qiy7hpGTwZ4qb1zuYi4ecLR/7tUxz+10+V++QhKrpU5+IWWmOPjcXmQE/pTcAv4Sciwq8/aliRU0qQp+ZVIcwzQZZUwiDqLb1eOq5KBnCLanOpUgz6AVzdpQ4TnZe+grtT/8DP9ClWygbDlfALBdVPJSLEl7wNADhQ22h56TCgk+QaKLqgifjv9cf7av/WFZ/cdCKm//Fk5/uM0ad4+yi1NyW7w1Z0RxLukpqr+YZry9GTvJWUDGiYaw4G4NXh96Ptzue4rqGTWa68dl0xcek2bAtw7XQkfInwiyiFFIuh1v5NnPA/NYZ52r2cOQ2xWO5Oee6KNzfC79LOO35VXjopYYxnoDueR2Voj06oRg+UYXHxNbhefx8xW4evsp1MS9yItcUXO3aTfEGeTEX4JXxBlZbDPeKBVyohTRZ44rFmrtIhoueIaDsR+Z1mrf1ERI8S0SoiWkxE/oTbBcJudHAiDVmYSsfexZ/dBUdbxsWRg3u53gbCw8rmh28y5gReeffbkpNN+O0UUZ7WflfVEK+wAjm6wwp04mH/GrGs0vH2ilp8u63uqoZsUosOE09OWYNnX30V7xX9xbMvyFZCcCXV+mLw3h0dO4jyGtL9CHuJAfc+f19b7NtnhWNZz0mUNAfaRlDAmtQT8E/A//x0BQaMHl+niNFfvjQPF471l4rIGCY+/tq6pkrCTyHu/I6ELUCslaTISpRkzfAoQowSzwVd2nn10lY+fPuLQsLPIOYYJytYO3SkanQna6V0nj7TIXyVhN9PszJk8lxZ+cYbc9yEhEGvwVWxCc7nXO5qjS0EKQlf+l5IP/x8nfkFAGeG7D8LwCD737UAxubpulmxU/AGyZAiyZMN12hrPZAj+nXB2jHnYECP9jj5IEsqPe1gN5PjqJPUAVo/sPX9xx/QQ6nD588y4xCJaf/1DgRuTNSE/OEZ6Jb3j034BIZlW8pDSeV793+O0/81NXB/EALPaPAX0cTSLeVOcI/3WPUroBEpJ8H6YMboUzD59yN92+Wzy8WxZXBC5xPuKnMfzz6ePlvU2YrF50USFV/msVOspHa5Ej6fYDfu9hsfn5q2BruqUs75NJgYSpbffwJpJAXC5xL+etbLUUcBQAUrqVNEc1iUuAqD9/Z68mjkOmbyW5AUXI7Ttg4fACpQgk6odt6JBNKImdyeIhO+2x9VFG4+8OpsfwZaWZC4Mfae8zmXlbbpKLlUEr73eyEjz/NV8WoqgDAH5PMBvMQsfAmgCxH1DmmfN+xkrhGvnIILach5P0Qc1qcz1o45B0ft61YMOmFQD6wdcw6uPXE/3HW+W43+mAHdsHbMOTisT2fPIOHn5RMLH9x8adjFdt96uOMtAFwp2SoJl7aPiVkZJ03XfRAAHp30LcZMWI4/KcrdZZNwgxD4nmcsQuIDdzfz39NAv3yq32BW9aVPlxIM7GEZYW86bZCTVll+edJGuGGbE2VPlCHNdPzHOMnZl4HuFMgRJU2dLC+e/Wiz5X5o3wv+V8x9NH1VaU46XrHJlc97U/7urHRtBhnTxHX6B3in6G+4N/asLeHHnPHCCd+A7lFdrWN710nCd3JC5ag1PGZAN5xykFddl7BTBHD3ZFHCr2DtHJVOOWuPjlTtkHuCMo6EL5OpOLa4bS7fUL0zDaVgLlC0h39Clz3tCpkqorF0+H0AiFUmNtrbfCCia4loHhHNKy1teHGDPXCrxlRQsD9xXPNKI7ni9rMPxuXHDcjajj9EV6Vj3fpiewnOPTy+SRzh2U9gWLjWug8ZaJYO3/QabRes341xU1YrJZO6YPQ7i/G/b7aGtknXWgPWWZkovCiC45DVudODkGvLm047EE9eZums66vS6UoVKEN7Se3gSviiLlmHiTtir+Dzot+ju+m6E3IX2w273Jf6qhfm4YNF3mLXKoi68skrvONe9Ms2TIYDNasc4CWxSSiiNFIs7vSTG5QtG5G9amA9MMEcXifvDzdZW+4vxJFC1C0R4abTBuHK4wfgp0Mt9aiow//QPM5RXVSgBB1Rg4Qt/HRGFeIBOnxR4ud2sFwwecV2pKScN9vLa5XlKg1hluO/PrRGQA4S/geG5Uywr+bnNPnchYwqaCzCV90u5e9ijD3FGBvGGBvWs2dPVZM6QRxkFZqa8H9z8gEY2r+r3anCBnFwISsj1VflxL9gk2UAFSX80jIrVNuSNhl0g6t0rIE27Vt/Dvr64I25GzDqlfkAJHuF0Ka6ptrpFwDsq/A9z5eEf3R/i0Bkg2DW4/Z1iSeTpTIVv8/tkEQScQ+JGNAclY9YLlCHiVM1Ky2y6LHDJ2Du2ne29iUOp9WhOfMratOYurI0dNSJftkZk3mKx3RDOXaho+OBxp0ADGF8vZI5DSa0+unw6yBtykF5HYvj+MsPD0Vx3OoLz+z5nnE8DMFoy4MQuYtpB6pFIkCHLxpLDekdCsqPP3vNTlz5/Fw89NkKz/bh903C9a/601ur1Gph4kc2b7m70pdhFzphhnEork/dAAA4/ZC9lMVkAGBg9/yXNuRoLMLfCKCf8L0vgOxiTx4gSmxVASqd358x2Fl2Fmo15ejwibDozz/ANSMHK9vxl0IMCOL64zR0pYRfX2zcXY3Lnp2Nitq0xzh3/4RlWL/TX4QccFUGfLK5IfZfXxuV1A9Yv70uxUz+fO6h+PiG74caabMhbbjBaRcc7V9UGgJRplnMk+MlzWJOfQJdqIalw0B7W50gGqjl5/HvxKP4oOjO0EnuhtcX4vLn5gR65wD+6kriNYsog+3MElYyTHOkZJX0W5fJtq5eOjIGKEgrgxiOqf03bk3/CoCVogGw1E8xMj1pItoxS8iR/fBFwpcD/o4P8MXfYavExDHNV9yTlm/3ta9OZXMF9d4TnqsoCPydviT9J4w3RwAAjhnQFf3teySuMqbcOtIToZ9vNBbhVbnoAgAAIABJREFUfwDgcttbZwSAMsbYlkJf9Ncj9/fkpklrwX7SrkNBoSV8Qud2cbQvUQcZuRMUwYQOHYbjA55hlmlXZ26krYxPloSrZEQ8PPFbTPt2ByZ8sxUpw8SfYy/h8fijeHLKmkBDL598gqR43nPl9jqO40RMwyH7dMreMASiSuJC2/PKs5+r1pBCGjGPgGBAUxa512E6wXzicp5/ln9nmCfViq2WKk9WN4iQQ+0NqUjOdtbF6S+fkDPQpSyldVOndW0Xx3492uPeHw/J+Ri+KvzRkftgiBQAx1GKLs495WmA07akXgI3fxHXdcsSvqcqWY5uybxforCRq6bKyZYp3Dqxn9Z3P+HXMq8Lqoi9OxXjquMH4rGLjsLJg3s6xA/A87kQyJdb5usAZgEYTEQbiehqIhpFRLxw5scA1gBYBeBpAL/Ox3Wz4Q9nHuQN9iAv4b941XDceoYladc1eVquOKJvZ/zp7IMdoYAPHNKtfpWzEsw1D3Tai/1legwxGBhGVlUjAxqYYUBjwaT7q5fn59y3mKCnTZdtwVWx/+Fc/UtfO8mkZF07JAIyaOVBPvNUYSBeXUxFoFpd8IR6xUghhZhDPoD1LFRF7nUYjgpOJKQg20UYz3IbQ5j0LYfay1lfK1Fib9cdFZOqVrPqGscFpLKI6Ro+//1InHFo7iVAD9rbmpxzPaYobv0O3tcf6m4eIk6qspdOPEfC37CrGgNGj8c3m8pccrd//vtfbcLIByfn1Ec+WYi80EEyvBZLEwDgfY/TUkLEgT3aI6ZrOKxPZzx/5XDHuN0YyEt6ZMbYRVn2MwDX5+NadYXHCKfFAeHhnHRgT5x0oGUncCtV5ff67//GKvTw6CQrwx5frvHBOsM8DF3sdKoZpnmW64ysFMrc71iH6bhFAtZL8VT8IdyduRQbWN2Lf/O+ZEwGc4vfw0cF3u8ww2yYDr+xIbplqgiPp6/rQLXoxKo9Er2owxfhyXfORAmfh+F7rxNG5rl4UcV1Dedqs/AVOwAbWU+f2yv3ADGgOTEnScWrHZf8u7u2i+P5K4/BQXf+L2sfcsGZh+2Nib87CQf0CvaGE8FVOnxS/bE+w9dGHktixHNYFO6kZZZt6YWZa1Fh21T4hP/HdxajNp3j6sB5PO5zkr2DShQqnY5Ug1oWRzGlfeq1pkz21uojbcUScLoePL+pgqvyCVeHb/+1iTsF18NCzpTJtLjXWEimk0fHYIQ4GfiBPh93xF71HJdrYW1HwjdMZDK5VTnipMZfxFIhzmGdabnlyZPB/j3b29sbh/TvPPcQ53PGZM41VbwrEno/rdSrw4eu1IUnJAMuh0P40nXCPF24UTmM9zUYeDzxGN5K/M3us/e15WpLE+REPVexEmFM232VbsC+3do5BtV8IVeyB1yVjqzyEOGX8NX3Xgaf59+evxGfLLHIn//67u2F3PQ5jscwCV9W8XDMMq1xKL/XEeEXECLhhxVGRoFUOhyyW6Zm+9KnBEOhXJ+UaTEpSZMJyvCC1K6PtSzxjbh/Uk59cvOlIHfCJy/hi25zM81DPftkEAF7dSpcvhmOo/ftiicvGwoAaJdwyURlDJPJ0+uloyslfG8tYuH5kFqHH1YGkU8GRsjYPG/6jwEA3W3XXfl5pwQJn0ubVYrc97IOvzFVCSrwnDFhhC+PpVxVOqr7yZ9L9w7uu5OrE4FICx3IS/g8BbIM/rvkMRYkAPTMkpI9H2j1hC/qzzYp3a0sDOxheYLs263+HiFhEL10ADfft6UnDiZ8n5HKlvC9hJ8dOyuTKKvxZgJ0oilNE0YmrTos8PwkSfoAUG5nwAx6ERkDiuO6UxOgkDj94L3wp7MP9iRcU0X5+glfTPAVV+rwiz2umH6jrSw0hBF+xiH8wCboXL0OgEvi8jjh38XfUq1I5CYn5WpywrevH5bzXpbwxRWvuNKSoeJUfhVevxpQjwkR/DSfC948soSv8lQDXG8q2aal6ts71x2H8Tec4N+RZ7R+whdejvOO2MeTHkHEj47sgzevHaF03csnuJBJuvXy7kQnh/CVKh3yEj5XBSUlCf/J+D/xRuLuwOsOvWcifvyEV0eq65zwc5fwxb4AXsIrZ1x14x3RjV2GEbCk+WtO3A8digSVnkLCl/P+iNJmJUp8vt4AcE3sY/ecnhWY9bvlJXsyE6xr5tJe6OrTBifxuER0roRvp4lgcWQQ83npyATPdehNhVwmHFlPL5L8JbFJ6AC1+7BKiubClhiPUJ+hKZeVDIIbtOf9DSqVztD+3dCrY+FXv22K8Lu1L8IzVwxTtiMiHLtf94KRk5xaofrgn+C+9EV4PPMjl/Ala76pxT0DXBOMtt6Mj4Qz9HkYoS0L7cOaHVWe766XjgnDEHWj3gG6o8Kvo3RzALlExb1FxElATlcMNP4EwK+muqxPwmci4RdnjebUFEZbmbtV+Xx2V6WwbEu5I+GbJnAgbcCL8THohCoMGD0eX23w1nPlWUh9hM+8En5lQJpmmWALmYY3DB3tFR6/flikapyC3TIBq66zCsqkb44txx0I9Un1IfcpCHwCln9fXaKX841WX8RcXP4WMJ4hZzgqnVgcT9klAIMlfL9KhxSEX99MH2KO9YzpqnTiyHh01+t3+aUoLj2Kg3mKeTgA4H27eAsADO3fNTC24YPfHJ93r6gwqL10ZAnffQaVrCRriT0xMIjHH8gSnOonnvvYdE9kaMY0cYE+DSfpi/EK3YfzUvfiixXbPekK+PhISITDJXyTaQB5C2aLkP35+3bNvQJWPsFvTzzG04cEox9tRw9tM9bbXmhxqYavvJpkzMogKyZ964xKlKOd8+6JAkdWHb5i7Obq/+8WOZIl/JwOLwjalIRfyAi2bODePzxDs0g+A+z0BAdo3uBj2UtHg+nUs01S3XT4Koh++MxwB6VcRUm1BFVJ+GtZbwyofQ0L2IG+9oBfwu7XtZ2H0AqFi4/dFwCwl2LJLKt00pJKR6XDF0Esu0pHNeHJaQBMxhxS40FdBAJMvzQpS/hJSYfPS+jxovHr7POKEv5Npw3CrWeqo70LDX5/uITfo706dYbBCPvQLiRg4ADajOO0JT79uUymz89Ya1/D+t4dZVhUfC1ujL3resgJ47A+lJBrhbcyW8UpB+/Vt4pYPtDqJXwTGh5K/wSfm0eh8CaR7HClDHfbgbQhoHHM5wLI69mmQrx0coUu+OEzJhK+WiIRyc3x0iETK8y++MgYkfV6qqjFxsC1J+6Pa0/cX/miiRL+b1K/9QgIScSzSnPby6rAH0Uw4auP3Qc7cEv8Ldye/iUMU4jUFd07DXfy5aeRJ+ReXTuhfLfp/Bbul/+mMRIrzb5YyAZZxwkS/oVH93V0+M9cPgxd2wenDs8XJtz4fWwpq8Gv7fw13C2zOEZQBKtiNzqih1DkpDd2emxagN9wO2/dLlx1wkDnWfe3hamTtMV4g6crEVNa14Pxc63C9UDm5yhlnfGR6X03crHXFAqtXsIHgMeMC7CEDWwS4yGHq8O3/oreAbIqxzmGYh4XwBhcP3zZaFsfeIpcmKL3g5dQnALqAuET3ALs440ReMy4IPA6Qfe9seUc1cstSvgfmcehnIleWuSZECYbR/jPqfDSkb2hgn7nX+Iv4UJ9OkZqi2CYTDCEW0d8unQrht3lGoj5c5Yl/D49rFUSV1+6xXTIIXvAet7ciC0+ktMO2ctJHlhIHNy7E045aC/HxsEJH0xNgFxC5jCh+aqryTV8HRi1iCODLlQJANjD2gvxGLmrdFTPjrsiX576Y+ix1SjGv40f+VaRTeiG3/oJny/ngaaJ9OTwuWVqIuGrDYNMj3uCOv4QfxMPf7LEOkZz9bT1HT980smYDBAlfFKrdHTmXW3wgS/rwWXIKo3GylukwotXDfd8T0mG8q3ojmpWhE8Ny49f9NpR1awV4xC4ZH7VC/M8bbL/TOYhfP73m03lMNJ+g7ko1dawBBLxuKevYoZYT18Fwm9KOCodR8WkvkG74c1ua0JDZ6qCwdx3RyZ8fq9/P+dETEz8Hl1hEz46uGlNJJXO0s3ldVKzcAlfdZ+vTN2qPObGU92JN9eiMoVAqyf8+348BCcPttIncI7tXBKvU6rYvEB6xqK0qQruASyjbbG01uUvO9Nd8gkqKRiEZMbAmtJKT85zEqSsw+k7rC2+GN/TrIqV/AX1qHTIdCTaXK/PXzReQSzfUZ654KQDe+L8I92qVnJpSQA4MvkUrkvfhAd/eoRnMqtifiOnKtLWAhM+qV9wLrFrYDCYu2ISs0CKgUYlSOGtxN9wmFBlrBztUBL3Gv2TLIDwdQ0dbA+Z7BkhCwd+N7gRmQIIcI8k4ScojaG0EhVwV2FFUlqDtGFi6korFUl/bbuTybKGFTkrTVHY2lGZwtmPTsPjk1eF9vkvsRdxV+x5AO4kLwsLADDZPArnJe/G+cm7PNt/eERvPHO55SHYlF46rZ7wAVelwKXruX86DUvuOiPskLyjm60jVS0rg/LSMC3uy8TnEH6s7hI+v+Rt73yNUx6aguqUXTrRNMEE4+BxmrWKOF2zErHxoCBNUPv0wm7cGHvX2l9HldKYCw7H9D+ejPZNJG3+4yeHO5/HXuF3G7VqxOq2gECOq2a1wvtF5Ycvfw4S6PjmvWkXDNP0qXQAeFIGD9I2Ybi2At1sNQVgVY4aYFf+SvtUOl7ENMLfLxyCo/ftgv7dCxNgmAsG2ekX3MCrAJUOvGka7o09h+/pSz2TsCzhT1y2HZc/51YMc9Wd7idVsNXijWXKPvBnd2XsE1we+8w63n7mQfd5Mdsfi9gB0lbCQb07es7ZFGj69V0jwAl2sr83RYThf0Ydh+mrdjiGMnHQBVr9tRiKybukd9QtMVG9kBvh8lXNjNVWwRQeAbp5Ty2WV+8C99mQo2idtBCCSueimJttMJtKh4MP9ERMa1CO+4ZCDDj63mBX2tfI6zKna4QTD+wJc50GwFCqdHqSSxQqfX4YOAX9Jf4yBrxwFq7T/dHLor4+wzRffvhytENRTEP39gmkU976vDKKYhqG9u+Gd399vHJ/Y+HVXx6LZVsqnPG4WDtU2U4un8ntWd0F3/uwaFsAiGvuA3VVOt73pQOqsVd6AwC/4ZqB+VM0U7BKJwgaCdXuIgm/sOAveFMabft1a4eLhrv2BDFxYTyA8FUSvkMAcVe94DXahmSxtF8wkrwVPl++HUs2+QtCc+LhA/TpL1YqzxukkmoR0Ny+y+8hEeGlq4Y7pKFS6VwX+8BtHyjhq5+JKU3UfJLoRXswgKxyESLhy2QPWBK+k1fekfC9xHXFcf3xr58fgV6NkMcoF3TvUIQTBvVw3scl+mD8oP3bvnblkkpH1N1zlFASRUgFRr/ywDiCO+5lbe4biXtw78b/w4+06bhM/9Sz78xVd+EGeyXL0Y+sNAvy8wsDEQm5qyLCLyi4rrgu1ZYKDbEvPOnY7emrPW2YFg/U4SPmko8oTYb5CMek9LieiEOBoPhWR0dvD9Cgyj65SvjN6Pbj8IACHSIcv217ZaNKSCZCVxhwgeApWJ4oxef4u5hFgEHCAMd61kvIOqnW4ffoUIQfH+Uv/tLU4PeXMcDQ4vh16gbcnb7U2S+vqDYyf8nT9qjF9KIbsLT4KuU1SuKuncT1wyf0xG4cRmsAAIdpawEADyf+jbvjL3iOH1I6HjfG3vNsu1CfDiA86ZsMUcJvSj/8fBVAOZOIVhDRKiIardg/kojKiOgr+9+f83HdXFGSsI1DzYhwRJXOjenrcUryQbxmnOpsO2jvjoAeDyzxxnRXihOXtbLLnueaIYZqlW+x7hC+9b1jQN6SuhqNmwNev2YEpv3hZM82OdWArOutYiW4MfPbwHOKUr1I/kECHQ+qW2r2t6/nHjNUW4mL9EmhzxMAVrB+jooyFZCErzmNexFiGUVdI3xsjsCrwjtQK61U+P25I32ls60TqtCTyhGEjgn3mTq2PI3wWdEf8FHRHcpjOqNSuV2GKOjMNwfhhOQjgW0JhO7tEzhnSG8nk2tToMFvKhHpAJ4AcBaAQwBcRESHKJpOY4wdaf+7S7G/YCiOcQm/Ma8aDtFLJ4kE1jBXl7z6vrMx/obvg5H74q41rWhJLvFrgrQuSvXOhKBgGa4zdYu9qAnK0eHz0oq247ScFpZDJeFfNqI/xl5ytLJ9c0D7ohj6SZlRu7VP4JWrj0V3x8AuET6KnfqxKojeNbno8NvZLreOd45wTB/aifvjz2Yl/CpW7BB+JkCl05SqzDCIhO/otwWpWZag+9IOlLLOeMU43SH9jgFjkoMLTASrLsKHizZjR2USXagq8JigdMcAcIrmFj03hVKTq819lCsQDiLrnX/ikqMxbEC30D4XEvkQzYYDWMUYW8MYSwF4A8D5eThvvSGXhCuxc6LnUl2oOUDXLH0fEwq2fMessnG8olERc3WWZ+pznc/9yDLIqqRKWcIX23xPX+J8lo22PFCmI3In/FMO6oWzhvQGANxx7iHYq1NRkxpqc4HJGE4Y1APtitQCQjWKfEFyIiEH6vADlDr8WcYk/3vP+Smc8GtQJFSOco22Pzik7hXQGhvFdonD3p1LHMIXU1moxhVfvbxinI5S1jlw1cnBi88TGA7f+i72/Oe3+GzpNme/amKWg7tEPJd4UNm/bGrN5jLn5oPw+wAQcwNstLfJOI6IFhHRBCJSm+XzhNm3n4bpf3SX61yHn2tZs2YDzdXFrmSWDrY9apBiOjoYe5SHcB9tkzHANPBT/QtHXeNI+HZbkYaO1Zb7zqVLOvyOFKDSYf5hJEabnjy4F2bfflqT+N3XBbI8IE+QlazER/iiW+DdsRdwOK0G4CX/DbtqlEXKeVCdpnDH5MjmhVKLBOK6ZhOKdXySxfGL4f1Cj2sO6NWpGE9cfDTGXTrUcWJgAiVlFKmpxftaztoFjkkOx2hLwI83PYjLYhM9+1XVqk7SFuFiPXsRIbMOhN9c7If5IHzVL5FH7gIA/RljRwB4DIC6YgAAIrqWiOYR0bzS0tJ6dahb+4RHmnQJv+mCTeoFgfC5KqE9JZFCHB0yfq8aADhRWwSCaT2Aha/ggfhTuEqfYJ1OFlmzeAvIRls5cRWHKsEY9w1vCTg3eQ9OTT7g/E45lTVHNYqREVzxkizuqW9aRGl8UHQnAK+0Pn/lOtz2rr9mcDupULdK2sym0qlhRc5EzieMJOJN6utdF5xzeG90a59QEqKKRPehXc7nCpRgP9oaen6uLguKdWmnIPxb4m/jvvizoeeV+5eN8OuThrkQyAfhbwQgihN9AXjSPjLGyhljlfbnjwHEiaiH6mSMsacYY8MYY8N69gzWidUFJS2U8Jnmel1wA1Y71CKNGFb3+oHymHP0Ofil/rFFXknLmLUXWZODPOh+N3M4Hok/7jsHf/e45862cluNZEuzb8bO87SXjbY/G9a3UbJg5gvfsP2wmvVx3E+DkrxVoQgZ0j3fgyBK698U/xJTVm73tSmxVTo6meiJPTiE1vnaZCP8WsSdiVwkfJMBPToUPiFavqCyM2Tzgqlg7XCI5r9n8KjWLMLvCfWKuITU9WhzgUjy2RwXOjZClbdckA/CnwtgEBENJKIEgF8A+EBsQER7k/1EiWi4fd2debh2TuC6wpqWRvi6JU3WIu7k22mPWqQQw7reZwWG0B9IGy3SslcIMmmIL9f5+kzf8aryhYBL+LWaVxcvv5hd2rUcohEhZ7mUJ8gqVoKMYEhXlRHkIOneqSRuLl3qMDGhaDS+r3/ja8Of3SuZU337AGAb6+p4E/FLWBI+w0+G/n97Zx7mRnUl+t+R1Lt3u721bWzAKxiw8QIGzGIWs9owLIaQGAh7gAHCFpYJeSTfOEAmCQ9eGAJhmYlheCwDYXh4gJDHMCGACUMwYzCbg202E2KzuVst6c4fVSXdKlVJpZa6Wyrd3/f116WrK9W9papTp849i6WHVYk1oSBxnzEW05r1FAs6+s22K2k5OSyM5z9hQS4VdU8oxaTTUiXmzLIFvlIqBZwHrALWAvcrpV4XkbNF5Gy727HAGhF5FbgJWKb6MHNWrWr42Bp+kkS2GlabWBp+Q1wCc7WL2MLLXvQt5ss9Q9a7d+vjNRInTbMk6VIJUuIW6Fs8IfD9kRStEnj9o71mhq9oplsz6TgVqLwM43PGaKYH8PfFdzyu4mQYYbsWeoOLnNQKv83McrXf0H08i7pu4COGE4tZN5RtynriaCSFInixuBrxM+mE0fD90L3WOpP+2TSddBneOJdS0K+/YmOtFk+pijxn2Gaaxz1tt2rbNwP5toM+otn20tlWA4u2rkLGtq99igTj2gfDVpgTW8dGNYJEPBYY4SpkLI3S/rxzAQjA1k1sl9nAJnIWtZn2Qu/qzBTmxNZxdNyqfatr+O80fxOAz1ULaXE/WfxFDXK9rhFnqDxyBcWt/94Ee100kNIyiW7zCcUHeLTpasbZ3lIOfkVkHO19pOTMDY5XSa6P9dtt8aQZ6KSBd+wCJ46G7zxxtNDluulKCRGh/YW/Db+IwMcd+dxCJxPkE9ex8vN8minvZssUHqJ5uHnxPqXpbFONLg2/VmJRamOUZeL44Xf2Y4ZAP645Ij9cYaexuQhQx4afJsbAQTn/73HyKY3xGA9k9vP9XsEWMLZJx1U04qczWJm8wNXfETzrMu5oTEfDPzn+ZLYtSQOpmFvg/1W509jWqIKfFfjO/4a8nEviql4UlEvFK+wh/5jEyPimSvDiBGdtxb0IrotHx/TkrCm00Wntr4Z+h5iPJCpWbcyr4f+k4VZWNV3BU025FMV6riOH32gBV2cnHgv8/kJPxjekTggdYf7zZbuF6tcX1IfAt234nanqEvjf3ntS4Q62YO1WcZIeu3kiLqzIfMP3Ywq44qHX+NdXLG/ZRrq5MPEA7SpnZtC1F8f1z1sL1TEn/LDhzmxbFw1kYjnN9u3MWP6CW8OvJVOCTjqr4VvHxom8/f3ed7GiexngLlZTSuEZpRTPvPkJ79mF5Istxjo4/bwavistc0wYP6yVJ9JWrv9nMru5foEqsSYUxKvhv59p99Xwb+w+LrvtteHPFOtJVQ/GOjb+bI/HVChNiaBcAr9QPMCS3fy81PuH6lg67mWcpFE7jR1UpGd1IXGnsEWCZMKt4SXiMZfP8trMBKbH3gegiRT/9qcPGRj/mKUNMCv2NkfIC+zf+Wa2/6LYK9ltZzHWm/53j9javDF1qQaStv1zbWYChyZX5PWpVQ3fIavh2wL/s5HzuTWdM6/58Zka4Epb7EUBp95pmQ/Wrzg8K8jTSvLMODpOuckvPeYLVxoHEe5YPofrnxjIxNUrrf3V2G+gL5DP7fw/fE0T0328lm5OH53d9pp0xsd65sYdRLGbsn79fUZtyJa60PA7hrTw2Pl78/0jezXeq+KorMCPkfII/EaPW8MrmR2y244Ad07YDrEcogZr9UFvbMgusTDafn+byrdJL4+vcr3uooEuewG5p6UVq43/f+l+rteptCPw3XUUABB/u3Lxql/u185v480X48UpN+kN+HItqMeE4QOaWDxzdLYto2rrOUtf1NzMEL6ipagN//OARdtKcXB8dfFONjenlvbiSCpHXQh8gJ07BldlpOc9p83j5pNm+b6nErkTOhn3mHRiMc9iXG67MSvw3Y+kEzO5gGj9cXW5Xdih08evfL5Hy0/SkNVygwRKrXnpjBjgnnfao+HH/A+zC1XkUhqjPkY/YqcmngDyzWheGqWbjJK8RUFX7n1xR1CD+7ephduyX1xSIc+X9oFNgW6ZleKGhtsKvOs+x7035PmT+i9fTiHqRuBXKwuntHPELmN931N2zvsYiu48k44nL45L4DveH/7RuOD/uOpNR5tWkheh2EUDXbZbZlCCsNoS91oxbRsnWVy2BJ+rOpk/xXKjPxk7nwvidprdj17jgoQVbF5cw0/51jzWM2tm6xxo41RK1dSNN2ykrcOLVy7KK3JeLVy+eBr/ctae/T0MX4zAr2KUlvNexXOa4CXdZ1kLito1ooBDulawIdNOo3TTweaCHgh+C1KfetwrNzOEQz1ua1tVG1vEiqINylRYQ3IGsNZDdBy3Uj+TjohwVfdpHNV1neszYTw2Dou/YG2sfy7bNpLgmzJYT2vFTBvO8F0avgqOGK5Gwmj4GeX+HbzuwIbi1MWibc3SaGkwgnIJnWfSu3G8V0gR4001gbdUB+2ypWjaWL+FwveVO8Piu5kxjI67BdJWBrAlbrmIDsI/xWxtWY+DcdwyXRYdgV+nD8zrG2Y9Y4TjIvh1Lsi8UQp7jjWSyron3pvanwZJk1QJbk8dlu0Tj+XXe5g3aRhrNm21x1/9El8/vzuGtLBpyzbXTfQTNYTzk+5SG5VeKH05M5kO+ZTRBZ6MS2XCsOrKEGs0/CrG0fAF9wWxhQE+Jh2L8bKZmbH1TJENhGFNZmJ2+1M1yGUrfjazS17/LaqNrbaG/3rG36201jT8IBy3TN1HXD/qejK5jE/5PS9ONC3JwhkeXWOQ7uxv8r3UGVzSfTZXpk7nS81+Hc/a8K3/owc1M3ZIC3tPtoLrZm8XnMO/WtDP7yW7WSbOr7VI5nfVGF5Q012fKaWmbBgeT8/v0ULwO5kxvu3PXro/vzl/b9/3+gsj8KuImR3usnti11sVFLpCnyZOYzzmWaSzXk2ObQLgpsZbQu1TrzD0OW0oye3I618PlkknLQkO7/oRZ3VflPf+uKEtnFYsvqBKGTPYvYbha8PXtocVKJRRkO7g4hsrUwe4Xk+QT4qG7Ts3pFyRbuv/flNH8sZ1i9m9BgS+sw6x4piZnHfAjgB8yHDWL3mIzo49+mQMd6YXh/Y8c3rt0/VTliav8+0zYXgrg1sqe1MqFyPwq4S1/2sxD527wN2YsATQx2poXi7/Qou2OqckL/Vtd0hqCdjSxF3eJn420q9s3+fX1SQ+J3/R7LnLD2Ao7Ld4AAAZt0lEQVSH9gF57dXOs5fuzxN/u9DV5gihsYPzi5cDbNAqHJVkNNE0/A+V25vjytTprtdzY+uKF/nw1DnQx1KNnml+ODep1qaES9ufOGsRzQ0Jpo72N998M5lXUbVH+HlChWGDGtXr3kKVxAj8KqGlMZ7VKB3SQyZyafeZnN99PgrFLamjuCtlpUVOxGKhFuO6aGRi50pWpecEvO/RQDQN3y85VRcNNbEIWCoThrcyuNVfG5s6Opc6Qp/6Wcn8JxydzoBspnTnBPil3WcVHVtTkapX2dq7WQ2/9n4gR8grpXw9drxxJw7/kdmFFzLTKrD/ytkhj929+grGO5hF2yrn/6b3Ayzf8BtSy7Lt3oLbjoZ/Qtc1/EtT7hHTqRoU5DboLXittIvtPTWGTtVAs5YwrFM1RlLg6zx4zgL+8+38fDjgXhhNDGzHKXglPgLD33NHQTJn0hkwdBQBa9+hiWU1fMc9s7zv6w/GD7Oeoga1BCgUBeYUpn5wX9I+sHBsRX9iNPwqRj/x054UlEEmnXeU26ffETpBJh9vwIjSIkm7iXN16jTX+8X8xqPA7tsN5YJFkwPezR3Hnxy3a8Hvud++WevEybg0/ExD+eaA7KKtx4ZfS1x44BRuPXk2+01pDygHGDwpv9KQXpKecolnJy8sdYiRwAj8KkY/8dMeJSYRF5e7nXPKe1P2egW6F68WqlzfKaQ99WpfzgQJwvpAl0VtTcH28amdd3FH+rC89gRpSOdysE/uGMkZyYvLGlMsz4ZfexK/IR5j8c5jEBFfn/xCfK/7dJ5K+0erA7yS2dF1HTyX3onfZQrfrIsRVDKx2jECv4rRhYs3n7pj0vlR90lATlB7Q/UdD4+g07PLmz9Hs+FnEJcp6Pru4/mI4TUpUCqFPvOWhgQnJq/ihu7j8wRAkkTezRLs6GeVu3unYs08mXGvr9zfeLT3YwVxFm3zahbXKH5rEIVmtk6N5/TuYOeEB9P7uAT+ejW6aDBbVKmIwBeRxSLypoi8LSJ5y+ZicZP9/p9EZHYl9ht1dMGayjPpWIu2jqBxBH6GGL9P5/Lsd2cFvv8l01lg0TZDjCcy83o+gQiiy6LGhPB8ZiduSecnzlLEfG34/9F0EXz4avZ1dyy/atYR86bntRXCMek4LoBbt/lXeapn9PQUKeJFc+0Xo241fBGJA7cAhwIzgBNFxFvZ41Bgsv13JvCLcvdbD7g0fI/Ad8L+nS66Jn5S99V8aQetOBp+UJBKl8cEpDwafheN3JI6yrWvWrQRVwr9Jpzwq9qhEcbNLxPPXxNpCPBICcIZxqiB1m9uBH4+uoafwZ1avJ6oxKznAW8rpd5VSiWB+4Alnj5LgHuUxR+AISLiH55myKJf9t5F2wb7Ks8tWLmFhHMDcAT+M2n/qjuFNHznqcD7dFDH8p67Tpub3dYrYr3o4xpYLKGazteqiX9KWSkbnOI0/9sn5e66Hx6a1+Zo+INaLKHW2lif5opCODWhIXdN3JU6uMeLt7WaGrwSAr8D0OP4N9ptpfYBQETOFJHVIrJ68+bKFjSoObRzymvSicUsXTNn0nFf6M4J6bhl/iazgKu7T83bRafHhq9rPo6G+oeM9cC2OjPVHldtnuyVYOqonE9+g2Yzv6L7DA7u+rGrb6GEar9L78rCrp9mD+WMrju5xvGIstdr/G4YjXllF93ZMm89efeqC+evBrwaPsC1qVN4KlPYuvwXT/lOh7o16eCv8HmPRpg+VqNStyml5iil5rS3t/t1qUv0RdtHvrOX9o7bhp/tb/+0eoWm5zP5NXS9gVfbBozXvsP6zucyM5nR+au8XCb1iL6gqAfKddHIOjXe1beQFriVtrxkddl9NFspNraq4hHLMXGPafHOo2sy0tnLD47aiccv2Cf7uic6xm+1p1pd4Ov2+6AKZg67d/2jb/saVZvpQyoh8DcC+pk+DvigB30MHrq0dAq6SUfX6JzrQOFOWuZs6lqinwDy+tW/ttfN2W39s19rufIvOXhK2ClEhssXT+PEeRNcbflFzt3oGv5LmSm+7/l6PM09nb/rXs496YOKjmu38UOK9qlFli+YyIwyS5Lqnjh6gGHY4uNBzO+8md9nds5rv+vUuay6cKHPJ6qHSkTavgRMFpFJwCZgGXCSp8+jwHkich8wH9iqlPqwAvuONEO0UH+vW6aD7qWjpyX+UA1nmHzpEtp+JgLvom2mdXh22+8GMbApwdyJ1VnNpzc5Z78d8toSRdwgdcHyYHohc2Prsq+dJzA/zTWWaOCe9CGhxlWbhoXKsOf2w0mmM7z8Z/90xromr2v4aZW/xvHHjJWwzbmezkuezzMZ/3Wvj8k//+88dS77TR0ZfvD9RNkavlIqBZwHrALWAvcrpV4XkbNF5Gy72+PAu8DbwC+Bc8vdbz0wsLmBO0+1FgmHtuYEsx6Q5WguXlv8KcnLuDh5NlvI2SD9vEa8bYkQ9fzq2ITvwpv7COD+1L58oaw0AfoN0xsNWiidcim5cOZVaSm9vuDeM/fgh0vzNW2Hn6aO5Zn0rjyS3su1aOvV8Cd33sOxyWsBOxIa+ILWbKLAY7v+jkuK5DwaUmVZMYOoSC4dpdTjWEJdb7tV21bAdyqxr3pjvyntrDhmJguntPPoq5YVTJcHd6cPYZB8zS/Th7vk82aG8lDG/XgZxmskXix4R2ozkrM38DtWl6XO4rKUJRx0weLN91KuWQHgt9/dt+oKbPQ1/mkYLDaodk7tvhxw++F7lRxd+3cEvr62tVpNY3V6Gq9nJuYnG7SplYR1JnlalSMiLJs3gS1fJ/PfwzLJ3Jg6AaBolhsVokhHIhbjhcw05sfeKDCm/LbmhlheCud6p5BQz5p0inzHKcnLAotybB+BxdlyKSRn9diTRGMTTlXPQkFXCbsCWVLli8a1arvgcRQZZ7VgBH6NoGvV2ZPce5YVMeiGCQQSgeXJy2h3yvHljcP/5H7lmoMjU9qwUugmHa8bX1gN/3cBdmSDRaEHUv0Yjx8xGD62tgtdB34afhhqRME3Ar9mEH2zZ2dXGJOOUrCNZt5X+SH/2f1rZ/eD5yzg5T9/RosJ9vGhgA2/hlMZVxOFTSmah1o8J8ALVRBL2I8BpZZPrBUzpxH4NUIYweBo2INbGnzD68MJ/OJauv4tu283tCZK6PU3Xht+zkunNgRFtRL26KlYzuBZ6OkqnhX4pYnGWvkZjcCvEVy+MwEnlyOr/aIx878lxwld12TTKocxytTKyV1NfIX7iSkd0oZvKEyhRVsdFQur4Vs35mRQtbIA/Dy2qpHaGKXBJYiLmfC91bAcgjT8F9R0/qQsP/NiCr6IGK1U48mLCgfa3Nh9HEd3/YAH0+5+lfDSqSf08/KYWbmsLKFPxbheu7m4hl+KDX/UoCYmj6yNBXSj4dcwQYI3KNtiKJNOCQuvly8uv5ZorVMsB/3N6Vxu+yfTszko/kdAW9ANKbCuPGwa08eUF3kaBSaPHMA/nJBbyA6t4WtZSf3qFDhkNfwQovH6v9mFe/6wnsfO36do32rBCPwaQddwgk06Vqegx8swXjph5f36FYeH6xhx4iU87ejHv2BqBR9mTxjKnDqMcK4UXw7JKSfhbPjFNfzj547n+Lnji/arJsxzZa3gEsT+QsLpEiTwdTfB47uu4aCu6wvvxgdjzXETVsMEt/93qSadogFxdYL3cIet8vX5iFxWzEKKzx1pK/10qYu2tYIR+DWCvhDruEBu397m6uM8BQQl9dJNOi+q6bylxuX1CeGkY9Ao5QaoJ/NSJbpl1sqiYF8T9vCnBozNbv+nT+Izh79PncTEzl+X8M21RTRvYxGkpTHOQ+cuYNNft9ExxMrx8avlc3llw1857a7Vrr7bDWvl1Q1b8r4jjEnHBE+VRil1ZI+KP5/dduzIYT+dKLEKVlTxmsD8nrA2qhEMYJu7sbGVL1QLt6aO5DMKrYVE+zgbgV9DzJ4wlNkTcj7vQ9saOWBafk71Hx29M21Nce59cYOrPcyi7R7bDy/ax5CjFBu+TqkmnWKZOaNOkBrid/j36fpZXltchJldd1RsPN8/Mr+2RC1gnhMjxIIdLGE9oCnB0bN8zDUhBH5DPMbfHzMz8P36Fjv5lCKHN6nczTTUArpGsfq59YJXwPsJfOVTs7aSayBPXbwvp+5VvwVQDFXCL781h6cuXoiI4GfyDStkjB0/PKXEJFyYzCWMLTW1Qr2bdByBPWWUu+SgbtJZl/Gtmur6fCXQ61TUGkbgR4i2pgQ7jrQuCD9BVEpRbR29jqsJunITVpDcf9ae7iIcJbpl1ruG35yIsfKM+fz4b3ZxtetH79jktezf9RPfz1dyzXtQc50KfBEZJiJPishb9n/fpCoisl5EXhOR/xKR1X59DJXFz7ZcTOBPHG6l4fUu3D5wzp7Z7aNnBWtR9YhX3gfdAGLiX0g7iJ8vc2fJ1DX88cNaShxlNFiww4i8JH26hv85bbynxvh+thT32SCuW7ozz12+f4HUJdVPuSO/AnhaKTUZeNp+HcT+SqndlFJzytynIQR+J3gxG/600Zb3gtekM1DTaK46zBQynzdxGNfai3beJ56gRVwRdxGOrIYf8JMs2c19Y23QNPxVFy7kj9cUr3dbD4Q2iQU8IS3zBE6NGNAU+B3NiRjjhtZ2wZlyBf4S4G57+25gaZnfZ6gQ/hdC4avD+UwhE34pbohR5f6z9+QUe9EurIYP4qvhh04Fo2n4rY0JhrUVK3dTH4Q1MQZZxFZ4TESFTu8omDPLFfijnGLk9v+gKr4K+HcReVlEziz0hSJypoisFpHVmzdvLnN49UtPFqkicD73Od7jHOQ+KSWadLzUu1tmEGHP2bDXQ9T9FYqedSLylIis8flbUsJ+9lJKzQYOBb4jIoEpBpVStyml5iil5rS3t5ewi/pm7kT38knGuNr0CV7TWTzAmyYm4iqbl/XDDymxTKStP/rxf1Bba/IS9oZZ6LKJwi23aOCVUurAoPdE5GMRGaOU+lBExgCfBHzHB/b/T0TkYWAe8GwPx2zw8PLVB9LW5P4pv+xMBfZ/Oj3L9fqshdvzj8++m/MYMTeL0HjldaCGj7eQtmTbw2AUfH+cwxKPCcPagu3v4RdtFbd/aw7Xr3qDdR9/6XonCoV+ylUbHgWW29vLgUe8HUSkTUQGOtvAwcCaMvdr0Bg+oInmBrf3woBm/3v57M5bOaf7Qlfbzh2DrY0iNvyFU8wTlxfvIm2QYMkz6RRI0eswdrBVNGXlGfMjYT/uDZzjrZQq6D0T2qSj4MAZo5js8fdfv+JwJo5oC/hU7VBuaoUVwP0i8m3gfeA4ABEZC9yulDoMGAU8bJ+wCWClUuqJMvdrKMJOYwfz0LkL+Ofn/8xDr2zKto/tGMeaTZ+7+noFvJ+C/8o1B9HaZOrWevEK+GANX/z98AvIoecuPwCFyZRZCP34NRUQ+KHz5jv/7YvgxHkT+OYe2/V0eFVHWQJfKfUXYJFP+wfAYfb2u8Cu5ezH0DNmTxjKyhfed7W1F3A7cy4Jv7q2Q41XiC9h0/UGLdoWCrwyHlHF0T3LCmn4YSOVnXM/Y5cg3mfyCGaMjU7hGbMSFHG8stuYBiqL93gW8tLxK4BiKI+cSSe4tCeET3KX1fDtrajdc81ZF3HCpDv2avRmybbnBGr4tia/Km3FHfY0zYXBjX4UCwn8sE9LzqWQtjX8qClIRuDXCa12SHqh01c0bclQOiMHNvHdg6b6vufIDbFvp5kQNnxDcXTbvC7UHzp3gatfIbfMKw7NlT90lB/nf0/TX1crRuBHHVt4X3vUTrxx3eJQGouR96Vx8IxR/OIbs3nxqgPz4iEcHMGUE/iluWUa/PGezoNbGrjowCmuuhFQWMM/e98dstvOue/EsURM3psCKPVCXMR23cwX545GH7Fzu8+47Vu59FBBN1Svhu/kNRpXp4nQKoWIMLytkYsPngLAq98/2LdfaE1duf5VJOlaNWE0/DqjYCRhtM7tfiHIhVKAMYObszfVDMJPjtuVpbuZ7KPl8vI1B/GN+YVdJ0tNrZBxlKCIXRNG4EecPB973z7u1o4hzb02nqgTJFdE4IkLF+Jk91UI+0wZEblFwWolfOCV24ZvNHxDTZL1V/ZR8b0mnUN2Gs2dp8ztm4FFjGA/fGFwSwMttq+4InrCpJop1S3zR0tnctSuY5m//bDeG1Q/YGz4EacnLpciwv7TRvLilYvoSmV6Z2ARJTC1gvNfVLbF23PfKe28++mXGCqP343Yr3bz6EHW0+2E4a3cdOKsvPdrHSPwI05WvBTIi+a4bA5qcZduGznImHZKJUiTzOY6UjkvHe/N4e7T5vXq2AxuTpw3Ia9t5Rl79MNI+g4j8OuEQiH8B88YzdWHT+ek+fkXgKE0giwHzk01JjkvHWPSqT5GD462kmNs+BFnrx1GALDjyAGAv0knFhNO32d7WhvN/b9cggW+dWx1t0yxr77Hzt+bS2y3QoOhNzFXeMQ5bs44Dpg+Mlurc1hrQ5FPGMrBqZ06pLWBLV93Z9udxF7ZBHXk7P07dwzOpag29AsPn7uAjX/d1t/D6HWMhh9xRMRVmPm6pTv342iiTzwmPP3dfXn+ikU8e+n+ee8PsE07GWKRS8xVbVx/rLterV7PwZtKedaEoRy569g+GVd/YgR+nTGwOafhdwwxUZ69wQ7tA2hpjDNheGvee5OGW8fc2PB7n+PnjHe9nmybNQFeujqwkF+kMSadOmTl6fNBYJdxQ/i6K7gUoqF8xg5u5oOtndnXsay3lBH2fcG1R87go8+7ALhs8VS6UmkOnD6KQc31adosS+CLyHHAtcB0YJ5SanVAv8XAz4E4ViWsFeXs11AeC3Yckd0e0GTu+b3JkxfvS1KPZXAiOTGBV33BKXtNym43JeL8cGm+7309Ua5JZw1wDAUKkotIHLgFOBSYAZwoIjPK3K/BUBO0NSXc1cKUJfx/duLsghWaDIbeoNwSh2uhaJGAecDbdqlDROQ+YAnw3+Xs22CoTSwNf+xgs35i6Hv6QsXoADZorzfabb6IyJkislpEVm/evLnXB2cw9CkqomkYDTVBUQ1fRJ4CRvu8dZVS6pEQ+/A7swNTuiilbgNuA5gzZ46pxWGIGI7AN+YcQ99TVOArpcr1X9oI6P5R44APyvxOg6E2Sdih+xLv33EY6pK+cNF4CZgsIpOATcAy4KQ+2K/BUH0ccxusvhM6Zvf3SAx1SFnPlSJytIhsBPYE/k1EVtntY0XkcQClVAo4D1gFrAXuV0q9Xt6wDYYaZdBYOOAqY8M39Avleuk8DDzs0/4BcJj2+nHg8XL2ZTAYDIbyMCtHBoPBUCcYgW8wGAx1ghH4BoPBUCcYgW8wGAx1ghH4BoPBUCcYgW8wGAx1ghH4BoPBUCeIUtWbrkZENgN/7uHHRwCfVnA4tYCZc31g5hx9ypnvdkqpdr83qlrgl4OIrFZKzenvcfQlZs71gZlz9Omt+RqTjsFgMNQJRuAbDAZDnRBlgX9bfw+gHzBzrg/MnKNPr8w3sjZ8g8FgMLiJsoZvMBgMBg0j8A0Gg6FOiJzAF5HFIvKmiLwtIlf093gqhYiMF5FnRGStiLwuIn9rtw8TkSdF5C37/1DtM9+zj8ObInJI/42+PEQkLiKviMhj9utIz1lEhojIAyLyhv1771kHc77IPq/XiMi9ItIctTmLyK9E5BMRWaO1lTxHEdldRF6z37tJpIRqOkqpyPwBceAdYHugEXgVmNHf46rQ3MYAs+3tgcA6YAZwPXCF3X4F8GN7e4Y9/yZgkn1c4v09jx7O/WJgJfCY/TrScwbuBk63txuBIVGeM9ABvAe02K/vB06J2pyBhcBsYI3WVvIcgRexqgwK8P+AQ8OOIWoa/jzgbaXUu0qpJHAfsKSfx1QRlFIfKqX+aG9/gVUusgNrfnfb3e4GltrbS4D7lFJdSqn3gLexjk9NISLjgMOB27XmyM5ZRAZhCYY7AJRSSaXUFiI8Z5sE0CIiCaAV+ICIzVkp9Szwmae5pDmKyBhgkFLqeWVJ/3u0zxQlagK/A9igvd5ot0UKEZkIzAJeAEYppT4E66YAjLS7ReVY/Ay4DMhobVGe8/bAZuBO24x1u4i0EeE5K6U2ATcC7wMfAluVUv9OhOesUeocO+xtb3sooibw/WxZkfI7FZEBwIPAhUqpzwt19WmrqWMhIkcAnyilXg77EZ+2mpozlqY7G/iFUmoW8BXWo34QNT9n2269BMt0MRZoE5GTC33Ep62m5hyCoDmWNfeoCfyNwHjt9TisR8NIICINWML+10qph+zmj+3HPOz/n9jtUTgWewFHich6LPPcASLyz0R7zhuBjUqpF+zXD2DdAKI85wOB95RSm5VS3cBDwAKiPWeHUue40d72tociagL/JWCyiEwSkUZgGfBoP4+pItgr8XcAa5VS/6C99Siw3N5eDjyitS8TkSYRmQRMxlrsqRmUUt9TSo1TSk3E+i1/q5Q6mWjP+SNgg4hMtZsWAf9NhOeMZcrZQ0Ra7fN8EdYaVZTn7FDSHG2zzxcisod9rL6lfaY4/b1y3Qsr4YdhebC8A1zV3+Op4Lz2xnp0+xPwX/bfYcBw4GngLfv/MO0zV9nH4U1KWMmvxj9gP3JeOpGeM7AbsNr+rf8VGFoHc/4B8AawBvgnLO+USM0ZuBdrjaIbS1P/dk/mCMyxj9M7wM3YGRPC/JnUCgaDwVAnRM2kYzAYDIYAjMA3GAyGOsEIfIPBYKgTjMA3GAyGOsEIfIPBYKgTjMA3GAyGOsEIfIPBYKgT/gcJHQRp0wxPogAAAABJRU5ErkJggg==\n",
+      "text/plain": [
+       "
    "
+      ]
+     },
+     "metadata": {
+      "filenames": {
+       "image/png": "/Users/hjensen/Teaching/FYS-STK4150/doc/src/LectureNotes/_build/jupyter_execute/chapter10_1_197.png"
+      },
+      "needs_background": "light"
+     },
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "%matplotlib inline\n",
+    "\n",
+    "# Start importing packages\n",
+    "import pandas as pd\n",
+    "import numpy as np\n",
+    "import matplotlib.pyplot as plt\n",
+    "import tensorflow as tf\n",
+    "from tensorflow.keras import datasets, layers, models\n",
+    "from tensorflow.keras.layers import Input\n",
+    "from tensorflow.keras.models import Model, Sequential \n",
+    "from tensorflow.keras.layers import Dense, SimpleRNN, LSTM, GRU\n",
+    "from tensorflow.keras import optimizers     \n",
+    "from tensorflow.keras import regularizers           \n",
+    "from tensorflow.keras.utils import to_categorical \n",
+    "\n",
+    "\n",
+    "\n",
+    "# convert into dataset matrix\n",
+    "def convertToMatrix(data, step):\n",
+    " X, Y =[], []\n",
+    " for i in range(len(data)-step):\n",
+    "  d=i+step  \n",
+    "  X.append(data[i:d,])\n",
+    "  Y.append(data[d,])\n",
+    " return np.array(X), np.array(Y)\n",
+    "\n",
+    "step = 4\n",
+    "N = 1000    \n",
+    "Tp = 800    \n",
+    "\n",
+    "t=np.arange(0,N)\n",
+    "x=np.sin(0.02*t)+2*np.random.rand(N)\n",
+    "df = pd.DataFrame(x)\n",
+    "df.head()\n",
+    "\n",
+    "plt.plot(df)\n",
+    "plt.show()\n",
+    "\n",
+    "values=df.values\n",
+    "train,test = values[0:Tp,:], values[Tp:N,:]\n",
+    "\n",
+    "# add step elements into train and test\n",
+    "test = np.append(test,np.repeat(test[-1,],step))\n",
+    "train = np.append(train,np.repeat(train[-1,],step))\n",
+    " \n",
+    "trainX,trainY =convertToMatrix(train,step)\n",
+    "testX,testY =convertToMatrix(test,step)\n",
+    "trainX = np.reshape(trainX, (trainX.shape[0], 1, trainX.shape[1]))\n",
+    "testX = np.reshape(testX, (testX.shape[0], 1, testX.shape[1]))\n",
+    "\n",
+    "model = Sequential()\n",
+    "model.add(SimpleRNN(units=32, input_shape=(1,step), activation=\"relu\"))\n",
+    "model.add(Dense(8, activation=\"relu\")) \n",
+    "model.add(Dense(1))\n",
+    "model.compile(loss='mean_squared_error', optimizer='rmsprop')\n",
+    "model.summary()\n",
+    "\n",
+    "model.fit(trainX,trainY, epochs=100, batch_size=16, verbose=2)\n",
+    "trainPredict = model.predict(trainX)\n",
+    "testPredict= model.predict(testX)\n",
+    "predicted=np.concatenate((trainPredict,testPredict),axis=0)\n",
+    "\n",
+    "trainScore = model.evaluate(trainX, trainY, verbose=0)\n",
+    "print(trainScore)\n",
+    "\n",
+    "index = df.index.values\n",
+    "plt.plot(index,df)\n",
+    "plt.plot(index,predicted)\n",
+    "plt.axvline(df.index[Tp], c=\"r\")\n",
+    "plt.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## An extrapolation example\n",
+    "\n",
+    "The following code provides an example of how recurrent neural\n",
+    "networks can be used to extrapolate to unknown values of physics data\n",
+    "sets.  Specifically, the data sets used in this program come from\n",
+    "a quantum mechanical many-body calculation of energies as functions of the number of particles."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "\n",
+    "# For matrices and calculations\n",
+    "import numpy as np\n",
+    "# For machine learning (backend for keras)\n",
+    "import tensorflow as tf\n",
+    "# User-friendly machine learning library\n",
+    "# Front end for TensorFlow\n",
+    "import tensorflow.keras\n",
+    "# Different methods from Keras needed to create an RNN\n",
+    "# This is not necessary but it shortened function calls \n",
+    "# that need to be used in the code.\n",
+    "from tensorflow.keras import datasets, layers, models\n",
+    "from tensorflow.keras.layers import Input\n",
+    "from tensorflow.keras import regularizers\n",
+    "from tensorflow.keras.models import Model, Sequential\n",
+    "from tensorflow.keras.layers import Dense, SimpleRNN, LSTM, GRU\n",
+    "# For timing the code\n",
+    "from timeit import default_timer as timer\n",
+    "# For plotting\n",
+    "import matplotlib.pyplot as plt\n",
+    "\n",
+    "\n",
+    "# The data set\n",
+    "datatype='VaryDimension'\n",
+    "X_tot = np.arange(2, 42, 2)\n",
+    "y_tot = np.array([-0.03077640549, -0.08336233266, -0.1446729567, -0.2116753732, -0.2830637392, -0.3581341341, -0.436462435, -0.5177783846,\n",
+    "\t-0.6019067271, -0.6887363571, -0.7782028952, -0.8702784034, -0.9649652536, -1.062292565, -1.16231451, \n",
+    "\t-1.265109911, -1.370782966, -1.479465113, -1.591317992, -1.70653767])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Formatting the Data\n",
+    "\n",
+    "The way the recurrent neural networks are trained in this program\n",
+    "differs from how machine learning algorithms are usually trained.\n",
+    "Typically a machine learning algorithm is trained by learning the\n",
+    "relationship between the x data and the y data.  In this program, the\n",
+    "recurrent neural network will be trained to recognize the relationship\n",
+    "in a sequence of y values.  This is type of data formatting is\n",
+    "typically used time series forcasting, but it can also be used in any\n",
+    "extrapolation (time series forecasting is just a specific type of\n",
+    "extrapolation along the time axis).  This method of data formatting\n",
+    "does not use the x data and assumes that the y data are evenly spaced.\n",
+    "\n",
+    "For a standard machine learning algorithm, the training data has the\n",
+    "form of (x,y) so the machine learning algorithm learns to assiciate a\n",
+    "y value with a given x value.  This is useful when the test data has x\n",
+    "values within the same range as the training data.  However, for this\n",
+    "application, the x values of the test data are outside of the x values\n",
+    "of the training data and the traditional method of training a machine\n",
+    "learning algorithm does not work as well.  For this reason, the\n",
+    "recurrent neural network is trained on sequences of y values of the\n",
+    "form ((y1, y2), y3), so that the network is concerned with learning\n",
+    "the pattern of the y data and not the relation between the x and y\n",
+    "data.  As long as the pattern of y data outside of the training region\n",
+    "stays relatively stable compared to what was inside the training\n",
+    "region, this method of training can produce accurate extrapolations to\n",
+    "y values far removed from the training data set.\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    ""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "# FORMAT_DATA\n",
+    "def format_data(data, length_of_sequence = 2):  \n",
+    "    \"\"\"\n",
+    "        Inputs:\n",
+    "            data(a numpy array): the data that will be the inputs to the recurrent neural\n",
+    "                network\n",
+    "            length_of_sequence (an int): the number of elements in one iteration of the\n",
+    "                sequence patter.  For a function approximator use length_of_sequence = 2.\n",
+    "        Returns:\n",
+    "            rnn_input (a 3D numpy array): the input data for the recurrent neural network.  Its\n",
+    "                dimensions are length of data - length of sequence, length of sequence, \n",
+    "                dimnsion of data\n",
+    "            rnn_output (a numpy array): the training data for the neural network\n",
+    "        Formats data to be used in a recurrent neural network.\n",
+    "    \"\"\"\n",
+    "\n",
+    "    X, Y = [], []\n",
+    "    for i in range(len(data)-length_of_sequence):\n",
+    "        # Get the next length_of_sequence elements\n",
+    "        a = data[i:i+length_of_sequence]\n",
+    "        # Get the element that immediately follows that\n",
+    "        b = data[i+length_of_sequence]\n",
+    "        # Reshape so that each data point is contained in its own array\n",
+    "        a = np.reshape (a, (len(a), 1))\n",
+    "        X.append(a)\n",
+    "        Y.append(b)\n",
+    "    rnn_input = np.array(X)\n",
+    "    rnn_output = np.array(Y)\n",
+    "\n",
+    "    return rnn_input, rnn_output\n",
+    "\n",
+    "\n",
+    "# ## Defining the Recurrent Neural Network Using Keras\n",
+    "# \n",
+    "# The following method defines a simple recurrent neural network in keras consisting of one input layer, one hidden layer, and one output layer.\n",
+    "\n",
+    "def rnn(length_of_sequences, batch_size = None, stateful = False):\n",
+    "    \"\"\"\n",
+    "        Inputs:\n",
+    "            length_of_sequences (an int): the number of y values in \"x data\".  This is determined\n",
+    "                when the data is formatted\n",
+    "            batch_size (an int): Default value is None.  See Keras documentation of SimpleRNN.\n",
+    "            stateful (a boolean): Default value is False.  See Keras documentation of SimpleRNN.\n",
+    "        Returns:\n",
+    "            model (a Keras model): The recurrent neural network that is built and compiled by this\n",
+    "                method\n",
+    "        Builds and compiles a recurrent neural network with one hidden layer and returns the model.\n",
+    "    \"\"\"\n",
+    "    # Number of neurons in the input and output layers\n",
+    "    in_out_neurons = 1\n",
+    "    # Number of neurons in the hidden layer\n",
+    "    hidden_neurons = 200\n",
+    "    # Define the input layer\n",
+    "    inp = Input(batch_shape=(batch_size, \n",
+    "                length_of_sequences, \n",
+    "                in_out_neurons))  \n",
+    "    # Define the hidden layer as a simple RNN layer with a set number of neurons and add it to \n",
+    "    # the network immediately after the input layer\n",
+    "    rnn = SimpleRNN(hidden_neurons, \n",
+    "                    return_sequences=False,\n",
+    "                    stateful = stateful,\n",
+    "                    name=\"RNN\")(inp)\n",
+    "    # Define the output layer as a dense neural network layer (standard neural network layer)\n",
+    "    #and add it to the network immediately after the hidden layer.\n",
+    "    dens = Dense(in_out_neurons,name=\"dense\")(rnn)\n",
+    "    # Create the machine learning model starting with the input layer and ending with the \n",
+    "    # output layer\n",
+    "    model = Model(inputs=[inp],outputs=[dens])\n",
+    "    # Compile the machine learning model using the mean squared error function as the loss \n",
+    "    # function and an Adams optimizer.\n",
+    "    model.compile(loss=\"mean_squared_error\", optimizer=\"adam\")  \n",
+    "    return model"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Predicting New Points With A Trained Recurrent Neural Network"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Model: \"functional_1\"\n",
+      "_________________________________________________________________\n",
+      "Layer (type)                 Output Shape              Param #   \n",
+      "=================================================================\n",
+      "input_1 (InputLayer)         [(None, 2, 1)]            0         \n",
+      "_________________________________________________________________\n",
+      "RNN (SimpleRNN)              (None, 200)               40400     \n",
+      "_________________________________________________________________\n",
+      "dense (Dense)                (None, 1)                 201       \n",
+      "=================================================================\n",
+      "Total params: 40,601\n",
+      "Trainable params: 40,601\n",
+      "Non-trainable params: 0\n",
+      "_________________________________________________________________\n",
+      "Epoch 1/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.1614"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 168ms/step - loss: 0.1614 - val_loss: 0.1927\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 2/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0681"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0681 - val_loss: 0.0356\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 3/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0154"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 0.0154 - val_loss: 0.0018\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 4/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 4.3675e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 4.3675e-04 - val_loss: 0.0539\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 5/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0127"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0127 - val_loss: 0.1238\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 6/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0318"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0318 - val_loss: 0.1571\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 7/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0409"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0409 - val_loss: 0.1457\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 8/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0373"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 0.0373 - val_loss: 0.1070\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 9/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0263"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 0.0263 - val_loss: 0.0618\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 10/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0140"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 0.0140 - val_loss: 0.0252\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 11/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0049"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 0.0049 - val_loss: 0.0047\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 12/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 8.9601e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 8.9601e-04 - val_loss: 1.7706e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 13/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0015"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 0.0015 - val_loss: 0.0066\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 14/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0050"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0050 - val_loss: 0.0169\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 15/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0092"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0092 - val_loss: 0.0252\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 16/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0122"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0122 - val_loss: 0.0280\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 17/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0131"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 0.0131 - val_loss: 0.0249\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 18/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0119"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 0.0119 - val_loss: 0.0178\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 19/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0091"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0091 - val_loss: 0.0095\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 20/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0057"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 0.0057 - val_loss: 0.0029\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 21/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0028"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 0.0028 - val_loss: 5.1024e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 22/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 9.3047e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 9.3047e-04 - val_loss: 0.0015\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 23/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 4.2170e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 4.2170e-04 - val_loss: 0.0064\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 24/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0011"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 0.0011 - val_loss: 0.0127\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 25/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0024"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 0.0024 - val_loss: 0.0182\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 26/150"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0037"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 0.0037 - val_loss: 0.0210\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 27/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0044"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 0.0044 - val_loss: 0.0205\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 28/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0043"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 0.0043 - val_loss: 0.0170\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 29/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0035"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 0.0035 - val_loss: 0.0118\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 30/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0024"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 0.0024 - val_loss: 0.0066\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 31/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0012"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 0.0012 - val_loss: 0.0026\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 32/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 4.1938e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 4.1938e-04 - val_loss: 4.2108e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 33/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.4696e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.4696e-04 - val_loss: 4.9351e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 34/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 3.3515e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 3.3515e-04 - val_loss: 8.7654e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 35/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 7.8864e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 7.8864e-04 - val_loss: 0.0021\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 36/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0013"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 0.0013 - val_loss: 0.0030\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 37/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0015"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 0.0015 - val_loss: 0.0031\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 38/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0015"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0015 - val_loss: 0.0026\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 39/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0013"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 0.0013 - val_loss: 0.0016\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 40/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 8.4817e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 8.4817e-04 - val_loss: 5.9712e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 41/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 4.1960e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 4.1960e-04 - val_loss: 5.1834e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 42/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.2261e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.2261e-04 - val_loss: 1.0502e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 43/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 2.5307e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 2.5307e-05 - val_loss: 6.7755e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 44/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1302e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.1302e-04 - val_loss: 0.0015\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 45/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 3.0417e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 3.0417e-04 - val_loss: 0.0022\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 46/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 4.9066e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 4.9066e-04 - val_loss: 0.0025\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 47/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 5.8441e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 5.8441e-04 - val_loss: 0.0023\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 48/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 5.5018e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 5.5018e-04 - val_loss: 0.0017\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 49/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 4.1198e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 4.1198e-04 - val_loss: 9.9305e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 50/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 2.3364e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 2.3364e-04 - val_loss: 3.8335e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 51/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 8.5796e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 8.5796e-05 - val_loss: 4.9130e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 52/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.5314e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.5314e-05 - val_loss: 2.2907e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 53/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 2.9857e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 2.9857e-05 - val_loss: 2.1308e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 54/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.0133e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.0133e-04 - val_loss: 4.6367e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 55/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.8355e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.8355e-04 - val_loss: 6.2971e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 56/150"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 2.3446e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 2.3446e-04 - val_loss: 6.3475e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 57/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 2.3285e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 2.3285e-04 - val_loss: 4.9020e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 58/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.8382e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 1.8382e-04 - val_loss: 2.7457e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 59/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1231e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.1231e-04 - val_loss: 8.7585e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 60/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 4.9274e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 4.9274e-05 - val_loss: 2.2688e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 61/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.7310e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.7310e-05 - val_loss: 3.5831e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 62/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 2.1970e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 2.1970e-05 - val_loss: 1.4969e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 63/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 5.1827e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 5.1827e-05 - val_loss: 2.7475e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 64/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 8.6183e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 8.6183e-05 - val_loss: 3.4722e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 65/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.0592e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.0592e-04 - val_loss: 3.3682e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 66/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.0207e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.0207e-04 - val_loss: 2.5532e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 67/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 7.8321e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 7.8321e-05 - val_loss: 1.4419e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 68/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 4.7236e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 4.7236e-05 - val_loss: 5.0422e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 69/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 2.2810e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 2.2810e-05 - val_loss: 3.9370e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 70/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.3631e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 1.3631e-05 - val_loss: 7.0204e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 71/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.9720e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 1.9720e-05 - val_loss: 3.8779e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 72/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 3.3982e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 3.3982e-05 - val_loss: 6.9903e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 73/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 4.6795e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 4.6795e-05 - val_loss: 7.8865e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 74/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 5.1028e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 5.1028e-05 - val_loss: 6.1482e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 75/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 4.5036e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 4.5036e-05 - val_loss: 3.0208e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 76/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 3.2523e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 3.2523e-05 - val_loss: 5.0685e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 77/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.9844e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 1.9844e-05 - val_loss: 1.9610e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 78/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.2452e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.2452e-05 - val_loss: 2.4494e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 79/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.2335e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.2335e-05 - val_loss: 6.3044e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 80/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.7583e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.7583e-05 - val_loss: 1.0059e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 81/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 2.3980e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 2.3980e-05 - val_loss: 1.2157e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 82/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 2.7513e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 2.7513e-05 - val_loss: 1.1890e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 83/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 2.6387e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 2.6387e-05 - val_loss: 9.5984e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 84/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 2.1564e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 2.1564e-05 - val_loss: 6.3446e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 85/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.5803e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 1.5803e-05 - val_loss: 3.3027e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 86/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1910e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.1910e-05 - val_loss: 1.2295e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 87/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1264e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.1264e-05 - val_loss: 2.4889e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 88/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.3337e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.3337e-05 - val_loss: 4.8689e-08\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 89/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.6291e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.6291e-05 - val_loss: 1.6905e-07\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 90/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.8174e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 1.8174e-05 - val_loss: 8.0057e-08\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 91/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.7968e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 1.7968e-05 - val_loss: 2.7600e-07\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 92/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.5971e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.5971e-05 - val_loss: 3.3847e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 93/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.3408e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.3408e-05 - val_loss: 1.1733e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 94/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1615e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.1615e-05 - val_loss: 2.5258e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 95/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1290e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 1.1290e-05 - val_loss: 4.0986e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 96/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.2221e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 1.2221e-05 - val_loss: 5.4292e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 97/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.3550e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.3550e-05 - val_loss: 6.1085e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 98/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.4361e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 1.4361e-05 - val_loss: 5.9625e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 99/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.4195e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.4195e-05 - val_loss: 5.1104e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 100/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.3226e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.3226e-05 - val_loss: 3.8767e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 101/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.2059e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.2059e-05 - val_loss: 2.6280e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 102/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1312e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.1312e-05 - val_loss: 1.6264e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 103/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1259e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.1259e-05 - val_loss: 9.6858e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 104/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1740e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.1740e-05 - val_loss: 6.1976e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 105/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.2320e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.2320e-05 - val_loss: 4.9763e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 106/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.2591e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 1.2591e-05 - val_loss: 5.4740e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 107/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.2401e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.2401e-05 - val_loss: 7.6209e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 108/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1895e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.1895e-05 - val_loss: 1.1510e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 109/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1385e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.1385e-05 - val_loss: 1.6893e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 110/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1131e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 1.1131e-05 - val_loss: 2.2888e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 111/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1203e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 1.1203e-05 - val_loss: 2.8135e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 112/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1467e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.1467e-05 - val_loss: 3.1299e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 113/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1704e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.1704e-05 - val_loss: 3.1629e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 114/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1751e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 1.1751e-05 - val_loss: 2.9233e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 115/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1592e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.1592e-05 - val_loss: 2.4977e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 116/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1341e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.1341e-05 - val_loss: 2.0057e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 117/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1148e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 1.1148e-05 - val_loss: 1.5546e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 118/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1104e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.1104e-05 - val_loss: 1.2102e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 119/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1196e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.1196e-05 - val_loss: 9.9551e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 120/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1329e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.1329e-05 - val_loss: 9.0678e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 121/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1403e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.1403e-05 - val_loss: 9.3212e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 122/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1372e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.1372e-05 - val_loss: 1.0588e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 123/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1265e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.1265e-05 - val_loss: 1.2699e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 124/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1155e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 1.1155e-05 - val_loss: 1.5344e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 125/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1104e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.1104e-05 - val_loss: 1.8049e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 126/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1126e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.1126e-05 - val_loss: 2.0259e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 127/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1188e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.1188e-05 - val_loss: 2.1508e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 128/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1237e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.1237e-05 - val_loss: 2.1579e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 129/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1239e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.1239e-05 - val_loss: 2.0558e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 130/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1197e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.1197e-05 - val_loss: 1.8797e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 131/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1140e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.1140e-05 - val_loss: 1.6756e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 132/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1104e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 1.1104e-05 - val_loss: 1.4866e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 133/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1104e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.1104e-05 - val_loss: 1.3434e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 134/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1131e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.1131e-05 - val_loss: 1.2624e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 135/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1157e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.1157e-05 - val_loss: 1.2484e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 136/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1164e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 1.1164e-05 - val_loss: 1.2973e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 137/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1146e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 1.1146e-05 - val_loss: 1.3983e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 138/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1119e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 1.1119e-05 - val_loss: 1.5332e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 139/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1098e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.1098e-05 - val_loss: 1.6776e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 140/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1095e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 1.1095e-05 - val_loss: 1.8043e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 141/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1106e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 1.1106e-05 - val_loss: 1.8897e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 142/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1120e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 1.1120e-05 - val_loss: 1.9197e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 143/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1125e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 1.1125e-05 - val_loss: 1.8938e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 144/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1118e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.1118e-05 - val_loss: 1.8237e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 145/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1104e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 1.1104e-05 - val_loss: 1.7295e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 146/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1093e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.1093e-05 - val_loss: 1.6333e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 147/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1091e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 1.1091e-05 - val_loss: 1.5537e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 148/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1096e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 1.1096e-05 - val_loss: 1.5039e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 149/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1103e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 1.1103e-05 - val_loss: 1.4898e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 150/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1105e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 1.1105e-05 - val_loss: 1.5105e-05\n"
+     ]
+    },
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEWCAYAAABxMXBSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd5zcVb3/8dd7ZrakbCrpHQiEEkg0BLBEBJEiGlHQICAiiqiIehHBgiLqlYvd3+WCqAhIV+CSiwgoIIhSUggJIZQYEtjUJW2TbLbO5/fH98xmdnbaJjvsMnyej8c8ZuZbzvd8Z3bn8z3le47MDOecc65YsZ7OgHPOuTcXDxzOOee6xAOHc865LvHA4Zxzrks8cDjnnOsSDxzOOee6xAPHbpB0maSbuimtPpL+T9JWSX+UdLqkB7sp7ZWS3tcdaWWka5L2Da+vkXRpMdvuxnG67bPISPcoSbXdna5zbxUeOLKQtD3tkZS0M+396d18uFOAEcBQMzvVzG42s/d38zFKxszOM7Pv72k6kiaGIJNIS/tN9VkUS9L3JS2R1CrpsgLbXiapJeNvcu9i05L0CUmrJO2Q9L+ShmTZZoikOkmPZyy3sF/quL9NWydJP5C0Olz0/F3SQWnrz5c0X1KTpOsz0k191+nndGlG2v8laWN4XClJYd34jP22h7QuDOvfGz6PLWHfuyWNSUv7SkmvSaoPn8u3MvI2TdICSQ3heVraurPCsnpJtSGtRNr6AyQ9HD6P5ZJOLvacuyp8Z3eH72eVpE+U6ljZeODIwsz6px7Aq8AH05bd3M2HmwC8ZGat3Zyu672WA18H/lzk9ren/02a2Ypi0go/5L8GziS6OGkA/idL+v8FLMtx7EPTjvuZtOWnAp8G3g0MAZ4A/pC2fg3wA+C6POc1KC3t9IuPc4EPA4cChwAnAZ8DMLNXM/4/pwJJ4M6w7/PAcWY2CBgNvAxcnZb274ApZjYAeAfwCUkfAZBUCdwD3AQMBm4A7gnLAfoCXwH2Ag4HjgG+FvZNhH3vDZ/HucBNkvYr8py76iqgmeh7PR24Oj1wd/OxOvHAsfsqJd0oaZukpZJmpFZIGi3pznAV94qkC7IlIOl7wHeAj4ergnMkfSr9yi9cOZwn6WVJmyVdlXb1tU+4wtko6XVJN0saVCjjko6QtE5SPG3ZyZIWh9czJT0RrtrWSvrvtH+ezLSul/SDtPcXhX3WSPp0xrYfkPRMuGJ7TR2vkB8Lz1vCZ3Fkls/iHZLmhSu6eZLekbbu74quvv8ZvpMHJe1V6LMI+x4Q9t8SvssPpa07UdLzIc3VklI/FHtJujfss0nSPyQV9f9kZjeY2V+AbcVsvwdpnQ78n5k9ZmbbgUuBj0iqSTu/I4GDgd938dCTgMfNbIWZtRH92B6Ylq+7zOx/gY1dTBfgLOCnZlZrZquBnwKfyrHtJ4HHzGxlOO56M1uTtr4NaK8qNbMXzWxH2vpk2vqjgATwCzNrMrNfAQKODvtebWb/MLPmkK+bgXeGfacQBaqfm1mbmT0M/JMoaBckqUrSTyS9Kmm9oirgPjm27Qd8FLjUzLab2ePA3GKP1R08cOy+DwG3AYOIvrT/Bgg/Hv8HPAuMIboq+Yqk4zITMLPvAv/JrivK3+U41knAYURXYB8DUmkJ+BHRH+wBwDjgskIZN7MngR2Ef4jgE8At4XUb8FWiK6sjwzl8oVC6ko4nugI7FpgMZLav7CD6Rx8EfAD4vKQPh3WzwnPqKumJjLSHEF1V/woYCvwM+LOkoRnncDYwHKgMeSmU5wqi7+vBsN+XgJsl7R82+R3wOTOrIfqBfTgsvxCoBYYRXfV9E7CQ5v9IynZlv7s+GILTUkmf78J+BxH9HQJgZv8mukrdL+QzTnTlej4h71k8Fi4y7pI0MW35bcC+kvYLn+FZwP1dyBvAKkVVPr/PCPId8h1eZ15Np3ySqGTQTlF11hZgJ9HfwJUZ6y+RtJ3o++vHrr/7g4DF1nEcpsV5jj0LWJpKNst6Ef3NpMt1zv9F9L1MIwpkY4guKrPZD2gzs5fSlmX7jHIda4954Nh9j5vZfeFq6w9EP+oQ/cAPM7PLw5XJCuA3wJw9ONYVZrbFzF4FHiH648LMlpvZX8PVUR3Rj+l7ikzzVuA0gHAFemJYhpktMLMnzaw1XMn9ush0Pwb83syeC1d1l6WvNLO/m9kSM0ua2eJwvGLz+wHgZTP7Q8jXrcALwAfTtvm9mb1kZjuBOwifUwFHAP2JPuPmcKV4L+GzAVqAAyUNMLPNZrYwbfkoYIKZtYQrUQvn+QUzKxhoi3QH0UXBMOCzwHcknZZ/l3b9ga0Zy7YCqRLHBcBTZrYgx/7vASYSXU2vAe7Vrjr9tcA/gBeJfqBPJbrYKMbrRP8nE4C3h/ykVwFn5nsr0D9V0k6R9G6ioP2n9OWhOmsQ0YXPt4n+TtLXXxGO+Tai/93UsQp9XunHPhuYAfwkLHoB2ABcJKlC0vuJPr++hc45nNdnga+a2SYz20Z0QZnrN6NQPgt9vnvMA8fuW5f2ugGoDv9UE4DRoQpjS7jy+SbRH3h3Has/gKThkm4LVSj1RNUFxV5Z3EJUbVEFfARYaGarQrr7hWqYdSHd/ywy3dHAa2nvV6WvlHS4pEcUVeFtBc7rQn5HZ6YX3o9Je5/1cyomz2aWzJHuR4mC6ipJj4aqHYAfE7UvPChphaRLijuNrjGz581sTaj++BfwS6IOFcXYDgzIWDYA2CZpNFHg+FanvXYd+7EQTLcAXyaqnjogrP4u0Y/TOKAa+B7wsKS+WRPrmO52M5sfLgDWE5V43i8pldfMfA8AtmeUBCAq5dwZquGyHWcTu9opEhnrzMyeIQp638tx3NSxO1QDhlLyFcAJZvZ6SK+FqF3mA0R/hxcSBf3aIs55GFGAWZD2m3F/WI6kv6hj55y8+Szi891jHji632vAK2Y2KO1RY2YnluBYPyKqYjgkNPadQfYicydm9jzRD+QJdKymgqgx8QVgckj3m0Wmu5bohyRlfMb6W4iq9caZ2UDgmrR0Cw3TvIYoKKcbD6wuIl+F0h2X0T7Rnq6ZzTOz2UTVWP9L9GOAmW0zswvNbG+iUs9/SDpmD/NSDKPI75ioGiVVEkZRb6wq4CVgJlGJ6XlJ64gC0kxltH3lOfahRFWsteEH6nqiBuUDc+ybT+q7T6XdId/h9dL0HUL9/6lkVFNlkSD67nL9aCaAfdKOe0hGyeaQ9GOH6tjfEHWYWdLhJMwWm9l7zGyomR0H7A08neO46ef8OlEAOyjtN2OgRY3/mNkJ1rFzzktAQtLktPQ6fUY5jtUtPHB0v6eBekkXK7pHIy7pYEmHleBYNURXH1sUdTm8qIv730J01TkL+GNGuvXAdklTgGLr1e8APiXpwHDl+d0s+d1kZo2SZhIFrJQ6oobKvcnuPmA/Rd1LE5I+TvQjdW+RecvlKaK2l6+HKoajiALBbZIqFd1LMjBcUdYTtf8g6SRJ+4YfmdTytmIOGI5TTfT/l5BUnevHWtJsSYMVmUn0fd1TZFo3E7WPvFtRg+rlwF2hKuQvRNVQ08LjO8AzwDQza5N0kKKuqXFJ/YkaqFezq/fVPOBUSSMkxSSdCVQQlcII31E1EAfiIV+JsO5wSfuH/YYStVv93cxS1S83EgXiMaFkdCFwfcZHczKwhajqNv3z+kha2sOIqm+fMbNNYdnnMj7PLwIPhd3/TvQdXqCosfr8sPzhkPbR4TP9qJl1CgiSDgnn2VdRJ4pRqXznO+dQ2v0N8HNJw8P2Y5SlXRQgVAPfBVwuqZ+kdwKzCb3aivh895yZ+SPPA1gJvC9j2WXATWnvJxJF9UR4P5qo/n4dsBl4MjONPGl9iqj9JPXegH3T3l8P/CC8PghYQBQ8FhEabPPlPePY44l+rP+csXwWUYljO1E99uW58pSen/D+knDea4i6a6ZvewpRKWcb0Q/+f2ec++VEAWQLUdtD5mfxrnC+W8Pzu9LW/R34TK7PMeP8jsr4nA4CHg3pPg+cHJZXElUZbCYKDvNSxySqz19JFHRqiXq4pNK7Brgmz+d+ffhc0h+fCuveTVQtk9r2VqKeSdvDd3JBsWmF9Z8g6lK+gyjgDMmRp8zP+mii9osdRHX3/0tUAk2tryZqWF8bPpuFwPEZf9eZ+bosrDsNeCWkvZYoUIxM21dEDdqbwuNKQBn5fQD4fpbz+FJa2uuIGvEnhHWx8H1uCp/nS4TSdNr+08Pf1s5wTtPT1j0CtIZ9U4+/pK3/cfhb2U4UmNP/bwudczVRlfCK8Hkuy/yuM85zSPhOdoTv9xPFHqs7HgoHcs4554riVVXOOee6xAOHc865LvHA4Zxzrks8cDjnnOuSROFNdl/o8/xLoi55v7Xojs309acDF4e324HPm9mz+fZVNPTE7UQ9mVYCHzOzzfnysddee9nEiRO756Scc+4tYsGCBa+b2bDM5SXrVRX6kr9ENG5RLVFXxtMsuvEstc07gGVmtlnSCUTd9Q7Pt6+kK4nuBbgi3K072MwuJo8ZM2bY/PnzS3GazjlXtiQtMLMZmctLWVU1E1hu0eiZzUT9qWenb2Bm/0orLTwJjC1i39nsulv0BqLb/J1zzr1BShk4xtBx3KJaOo4rlOkcoptmCu07wszWAoTn4dkSk3Suoolk5tfV1e1G9p1zzmVTysCRbVyUrPVikt5LFDhSVU5F75uLmV1rZjPMbMawYZ2q6Jxzzu2mUjaO19JxwLuxRMNQdCDpEOC3RCNNbixi3/WSRpnZWkmjiIZDcM65TlpaWqitraWxsbGns9KrVVdXM3bsWCoqKoravpSBYx4wWdIkosHR5tBxUDskjScarOtM6zgpSb595xINp3xFeL4H55zLora2lpqaGiZOnEjHQW9dipmxceNGamtrmTRpUlH7lCxwmFlrGF3yAaIutdeZ2VJJ54X11xCNyDkU+J/wpbaG6qWs+4akrwDukHQO0eBep5bqHJxzb26NjY0eNAqQxNChQ+lKW3BJ7+Mws/uIhsNOX3ZN2uvPAJ8pdt+wfCPRVKbOOVeQB43CuvoZ+Z3j+bx4P/zjZz2dC+ec61U8cOSz/K/wr//X07lwzr1J9e9fzOzFbz4eOPKJJcCKmtTNOefeMjxw5KM4JD1wOOf2jJlx0UUXcfDBBzN16lRuv/12ANauXcusWbOYNm0aBx98MP/4xz9oa2vjU5/6VPu2P//5z3s4952VtHH8TS8W88DhXJn43v8t5fk19d2a5oGjB/DdDx5UcLu77rqLRYsW8eyzz/L6669z2GGHMWvWLG655RaOO+44vvWtb9HW1kZDQwOLFi1i9erVPPfccwBs2bKlW/PcHbzEkY/iXlXlnNtjjz/+OKeddhrxeJwRI0bwnve8h3nz5nHYYYfx+9//nssuu4wlS5ZQU1PD3nvvzYoVK/jSl77E/fffz4ABA3o6+514iSOfWMJLHM6ViWJKBqWSaxTyWbNm8dhjj/HnP/+ZM888k4suuohPfvKTPPvsszzwwANcddVV3HHHHVx33XVvcI7z8xJHPrFQ4ijR0PPOubeGWbNmcfvtt9PW1kZdXR2PPfYYM2fOZNWqVQwfPpzPfvaznHPOOSxcuJDXX3+dZDLJRz/6Ub7//e+zcOHCns5+J17iyEfx6NmSu14751wXnXzyyTzxxBMceuihSOLKK69k5MiR3HDDDfz4xz+moqKC/v37c+ONN7J69WrOPvtskskkAD/60Y96OPedlWwip95ktydyeuzH8PAP4Nt1kKjs/ow550pq2bJlHHDAAT2djTeFbJ9VT0zk9OYXCwUybyB3zrl2HjjySVVPJVt7Nh/OOdeLeODIJ5YKHF7icM65FA8c+aQ3jjvnnAM8cOTnJQ7nnOvEA0c+MW/jcM65TCUNHJKOl/SipOWSLsmyfoqkJyQ1Sfpa2vL9JS1Ke9RL+kpYd5mk1WnrTizdCaSqqrzE4ZxzKSULHJLiwFXACcCBwGmSDszYbBNwAfCT9IVm9qKZTTOzacDbgQbg7rRNfp5aH2YKLA2vqnLOvYHyzd+xcuVKDj744DcwN7mVssQxE1huZivMrBm4DZidvoGZbTCzeUBLnnSOAf5tZqtKl9UcvMThnHOdlHLIkTHAa2nva4HDdyOdOcCtGcvOl/RJYD5woZltztxJ0rnAuQDjx4/fjcOy6wZAL3E49+b3l0tg3ZLuTXPkVDjhipyrL774YiZMmMAXvvAFAC677DIk8dhjj7F582ZaWlr4wQ9+wOzZs3OmkU1jYyOf//znmT9/PolEgp/97Ge8973vZenSpZx99tk0NzeTTCa58847GT16NB/72Meora2lra2NSy+9lI9//ON7dNqlLHFkm/28S+ObSKoEPgT8MW3x1cA+wDRgLfDTbPua2bVmNsPMZgwbNqwrh90lFj4eDxzOud0wZ86c9kmbAO644w7OPvts7r77bhYuXMgjjzzChRdemHP03FyuuuoqAJYsWcKtt97KWWedRWNjI9dccw1f/vKXWbRoEfPnz2fs2LHcf//9jB49mmeffZbnnnuO448/fo/Pq5QljlpgXNr7scCaLqZxArDQzNanFqS/lvQb4N49yWReXlXlXPnIUzIolenTp7NhwwbWrFlDXV0dgwcPZtSoUXz1q1/lscceIxaLsXr1atavX8/IkSOLTvfxxx/nS1/6EgBTpkxhwoQJvPTSSxx55JH88Ic/pLa2lo985CNMnjyZqVOn8rWvfY2LL76Yk046iXe/+917fF6lLHHMAyZLmhRKDnOAuV1M4zQyqqkkjUp7ezLw3B7lMh9vHHfO7aFTTjmFP/3pT9x+++3MmTOHm2++mbq6OhYsWMCiRYsYMWIEjY2NXUozVwnlE5/4BHPnzqVPnz4cd9xxPPzww+y3334sWLCAqVOn8o1vfIPLL798j8+pZCUOM2uVdD7wABAHrjOzpZLOC+uvkTSSqJ1iAJAMXW4PNLN6SX2BY4HPZSR9paRpRNVeK7Os7z7tbRx+H4dzbvfMmTOHz372s7z++us8+uij3HHHHQwfPpyKigoeeeQRVq3qer+fWbNmcfPNN3P00Ufz0ksv8eqrr7L//vuzYsUK9t57by644AJWrFjB4sWLmTJlCkOGDOGMM86gf//+XH/99Xt8TiWdjyN0lb0vY9k1aa/XEVVhZdu3ARiaZfmZ3ZzN3HzIEefcHjrooIPYtm0bY8aMYdSoUZx++ul88IMfZMaMGUybNo0pU6Z0Oc0vfOELnHfeeUydOpVEIsH1119PVVUVt99+OzfddBMVFRWMHDmS73znO8ybN4+LLrqIWCxGRUUFV1999R6fk8/Hkc/yv8FNH4VPPwjjd6dDmHOuJ/l8HMXz+Ti6izeOO+dcJz51bD7exuGce4MtWbKEM8/sWCNfVVXFU0891UM56swDRz7eq8q5Nz0zQ8p2W1nvNHXqVBYtWvSGHrOrTRZeVZWPV1U596ZWXV3Nxo0bu/zD+FZiZmzcuJHq6uqi9/ESRz7tJQ7vVeXcm9HYsWOpra2lrq6up7PSq1VXVzN2bNYOrll54Mgn5iUO597MKioqmDRpUk9no+x4VVUeS9Zsj15447hzzrXzwJHHoy9vil5447hzzrXzwJFHLB5q8ryqyjnn2nngyCOeChzeOO6cc+08cOQRS/gNgM45l8kDRx7xeNSrKlkocKz8J/x4X9ix8Q3IlXPO9SwPHHmkqqra2goEjvVLYUcdrHnmDciVc871LA8ceaSqqtraCjSON22NnjcsLXGOnHOu53ngyCMerwAg2dqSf8PG+uh5/fMlzpFzzvU8Dxx5xBNFVlU1eonDOffWUdLAIel4SS9KWi7pkizrp0h6QlKTpK9lrFspaYmkRZLmpy0fIumvkl4Oz4NLlf9EqnG8YFVVKHHUvQSFgoxzzr3JlSxwSIoDVwEnAAcCp0k6MGOzTcAFwE9yJPNeM5uWMQPVJcBDZjYZeCi8L4l4IlRVFQocqaqqtibYtKJU2XHOuV6hlCWOmcByM1thZs3AbcDs9A3MbIOZzQMKNCJ0MBu4Iby+Afhwd2Q2m11VVQWy11QPfcP06F5d5Zwrc6UMHGOA19Le14ZlxTLgQUkLJJ2btnyEma0FCM/Ds+0s6VxJ8yXN390hlStC4LBi2jjGzADFvIHcOVf2Shk4sk251ZXZVN5pZm8jqur6oqRZXTm4mV1rZjPMbMawYcO6smu79l5VxVRV9R8OQ/aBDR44nHPlrZSBoxYYl/Z+LLCm2J3NbE143gDcTVT1BbBe0iiA8LyhW3KbRUWiyDvHm+qheiCMONADh3Ou7JUycMwDJkuaJKkSmAPMLWZHSf0k1aReA+8Hngur5wJnhddnAfd0a67TVCTitFqMZL6qqrYWaGmIAsfwg2DTK9C8o1RZcs65HleyGQDNrFXS+cADQBy4zsyWSjovrL9G0khgPjAASEr6ClEPrL2Au8ME8wngFjO7PyR9BXCHpHOAV4FTS3UOFfEYbcSwfPNxpHpUVQ2Iqqsw2LwSRhxUqmw551yPKunUsWZ2H3BfxrJr0l6vI6rCylQPHJojzY3AMd2YzZwScZGkQIkjNdxI9QDoMygs21b6zDnnXA/xO8fzqIjFaCVefImjsiZ63bS99Jlzzrke4oEjj4qESKL83XFTd41XD4Sq/tHrZi9xOOfKlweOPBKxYto40qqqKkPg8BKHc66MeeDIozI0jieLrapKlTi8jcM5V8Y8cOSRahwnX+BIr6pKtXE0e4nDOVe+PHDkUREPjeP5xqpKVVVVDYB4AhLVXuJwzpU1Dxx5VMRF0mKQTObeqLEeKvpFQQOgqsZLHM65suaBI4/2GwAtX1XV1qhhPKWyvzeOO+fKmgeOPBJx0VaojaMxjFOVUtXfSxzOubLmgSOPitAdl3yDHDZujdo3UiprvI3DOVfWPHDkEYsV2asqvaqqqr8HDudcWfPAUUBSccjXxtFYn1Hi8Koq51x588BRgBW6c7wpSxuHN44758qYB44C2hRHeUscGb2qqgZ4icM5V9Y8cBRgytPG0dIIbc3Zq6ry3fvhnHNvYh44CjDiyHIEgfThRlJS41W1+CyAzrny5IGjAFMsd+N4Y5bAUekDHTrnyltJA4ek4yW9KGm5pEuyrJ8i6QlJTZK+lrZ8nKRHJC2TtFTSl9PWXSZptaRF4XFiKc8hqTgxy3EfR/o4VSlVPpmTc668lWzqWElx4CrgWKAWmCdprpk9n7bZJuAC4MMZu7cCF5rZQkk1wAJJf03b9+dm9pNS5T2dKQ7WlH1l+rSxKZU+mZNzrryVssQxE1huZivMrBm4DZidvoGZbTCzeUBLxvK1ZrYwvN4GLAPGlDCvOZliuXtVZauqqvLJnJxz5a2UgWMM8Fra+1p248df0kRgOvBU2uLzJS2WdJ2kwTn2O1fSfEnz6+rqunrYtITyNI7nq6ryLrnOuTJVysChLMusSwlI/YE7ga+YWbi852pgH2AasBb4abZ9zexaM5thZjOGDRvWlcN2TCfffRwtO6Pnyn67lvn0sc65MlfKwFELjEt7PxZYU+zOkiqIgsbNZnZXarmZrTezNjNLAr8hqhIrnVicWK7A0RbaPuKVu5a1N47Xd97eOefKQCkDxzxgsqRJkiqBOcDcYnaUJOB3wDIz+1nGulFpb08Gnuum/GZl+aqqWkPgSFTvWtbeOO4lDudceSpZryoza5V0PvAAEAeuM7Olks4L66+RNBKYDwwAkpK+AhwIHAKcCSyRtCgk+U0zuw+4UtI0omqvlcDnSnUOQFTiIE/gUGzX7H8AFX2iZV5V5ZwrUyULHADhh/6+jGXXpL1eR1SFlelxsreRYGZndmceC8nbxtHWBPGqjsukaE4OL3E458qU3zlegGJx4uQIHK1NkKjsvNxHyHXOlTEPHAUUbONIb99IqarxGwCdc2XLA0cBytfG0dbcuaoKogZyL3E458qUB45CCjWO56yq8hKHc648eeAoJJYgnqtxPFdVlU8f65wrYx44ClAslqeqqqnjzX8pVTVeVeWcK1seOAqQEsTzVlXlaOPwxnHnXJnywFFIPEGMJG3JLMNsteYqcXjjuHOufHngKECxOAnaaGnLUupoy9MdN9mya0gS55wrIx44ClAsTlxGa7bA0dqcvVdVZWqgQ6+ucs6VHw8cBSgWB6ClJcv0sa2N2e/jqPJ5x51z5csDRwEKAxi2tLV0XtnWnLs7LniXXOdcWfLAUUAsHpU4WluyBI58NwACNO8oYc6cc65neOAoIBaLShytrVmqqnINOVIRZgT0wOGcK0MeOApQqsTRmq3E0ZjjPo6+0bMHDudcGfLAUUAslgocGcOOmIU2jmwljhA4WhpKnDvnnHvjlTRwSDpe0ouSlku6JMv6KZKekNQk6WvF7CtpiKS/Sno5PA8u5TnE4hUAtLY0d1zRFt5nuwGw0ts4nHPlq6jAIenLkgYo8jtJCyW9v8A+ceAq4ASi6WBPk3RgxmabgAuAn3Rh30uAh8xsMvBQeF8y7VVVbRkljtbG6NmrqpxzbzHFljg+bWb1wPuBYcDZwBUF9pkJLDezFWbWDNwGzE7fwMw2mNk8ILMBId++s4EbwusbgA8XeQ67Jd7eOJ6RxdZQ4sjWHderqpxzZazYwJGa//tE4Pdm9iw55gRPMwZ4Le19bVhWjHz7jjCztQDheXjWDEvnSpovaX5dXV2Rh+0sFu7jaMts42gLw4lkq6qKxSHRx0sczrmyVGzgWCDpQaLA8YCkGsg1ZGy7bIEly0iB3b5vtLHZtWY2w8xmDBs2rCu7dhBLpAJHZokjBI5sVVUQVVd5icM5V4YSRW53DjANWGFmDZKGEFVX5VMLjEt7PxZYU+Tx8u27XtIoM1sraRSwocg0d0s8lDhaM+8cLxQ4Kvp5icM5V5aKLXEcCbxoZlsknQF8G9haYJ95wGRJkyRVAnOAuUUeL9++c4GzwuuzgHuKTHO3pAJHMrNxvL2qKk+JwwOHc64MFRs4rgYaJB0KfB1YBdyYbwczawXOBx4AlgF3mNlSSedJOg9A0khJtcB/AN+WVCtpQK59Q9JXAMdKehk4lsKN9HukfciRzDvH2xvHs7RxAFT288UrQlYAACAASURBVKoq51xZKraqqtXMTNJs4Jdm9jtJZxXayczuA+7LWHZN2ut1RNVQRe0blm8Ejiky33sskYju40i2ZQaOVHfcLL2qIOpZ1eyBwzlXfootcWyT9A3gTODP4T6LitJlq/fY1asqI3C03wCYq6qqn4+O65wrS8UGjo8DTUT3c6wj6hr745LlqheJJ1JtHLkax3NUVVV4ryrnXHkqKnCEYHEzMFDSSUCjmeVt4ygXifZeVTnuHM9b4vDA4ZwrP8UOOfIx4GngVOBjwFOSTillxnqL1H0cllniSFVV5byPox+0eK8q51z5KbZx/FvAYWa2AUDSMOBvwJ9KlbHeIhEGOWzrVOIodB+Hd8d1zpWnYts4YqmgEWzswr5vaolE1B23U6+qgo3jfSHZuqvbrnPOlYliSxz3S3oAuDW8/zhZusqWI4VBDjuXOPKMjgu7hlZv2ZG7Ad05596EigocZnaRpI8C7yQaR+paM7u7pDnrLULgsGSuGwDzVFVBVF3Vp6RThjjn3Buq2BIHZnYncGcJ89I7xaIauWSn+ziaALUHlk4qU/OOe88q51x5yRs4JG0j+6i0AszMBpQkV72JQhtHpxJHmG9cOUaXTwUO71nlnCszeQOHmdW8URnptcKc49apjSPHfOMp7VVVXuJwzpWXt0TPqD0SqqKSySyj4+bqUQVpVVVe4nDOlRcPHIWEqio6VVU1FVfi8Koq51yZ8cBRSKpxPNsNgPkChzeOO+fKlAeOQkKJwzpVVTUXV1XlAx0658qMB45Cct7H0ZT/xr72xnEfWt05V148cBQSy9XG0Zh7EieAij6AvKrKOVd2Sho4JB0v6UVJyyVdkmW9JP0qrF8s6W1h+f6SFqU96iV9Jay7TNLqtHUnlvIc2quqOs053gzxPCUOyaePdc6VpaLvHO+qMEvgVUTzgtcC8yTNNbPn0zY7AZgcHocTzW1+uJm9CExLS2c1kD7Eyc/N7CelynsHqfs4LEvjePXA/PtW9PWqKudc2SlliWMmsNzMVphZM3AbMDtjm9nAjRZ5EhgkaVTGNscA/zazVSXMa26xHI3jrU35SxwQjZDrVVXOuTJTysAxBngt7X1tWNbVbeawa1TelPND1dZ1krKOICjpXEnzJc2vq6vreu7bE4oCh7LdAJivjQOiEXK9qso5V2ZKGTiyDeKUOe5V3m0kVQIfAv6Ytv5qYB+iqqy1wE+zHdzMrjWzGWY2Y9iwYV3Jd0ftJY4so+Pmu48DfDIn51xZKmXgqAXGpb0fC6zp4jYnAAvNbH1qgZmtN7M2M0sCvyGqEiud1J3jluy4vK3YqioPHM658lLKwDEPmCxpUig5zAHmZmwzF/hk6F11BLDVzNamrT+NjGqqjDaQk4Hnuj/radq742a7c7xAVVWF96pyzpWfkvWqMrNWSecDDwBx4DozWyrpvLD+GqJZBE8ElgMNwNmp/SX1JeqR9bmMpK+UNI2oSmtllvXdSyG2dvUGQIi643qJwzlXZkoWOADM7D4yppgNASP12oAv5ti3ARiaZfmZ3ZzN/CTaiKP07rhmhUfHhaiqyksczrky43eOF8EU69jG0VZg2tiUCi9xOOfKjweOIphiHe/jaG2KngsFjlTjuGWbRNE5596cPHAUwZQgZm20tIVSR6rEUbCqqh9g0bhWzjlXJjxwFMEUI0aSxpZQ6kgFgkKN4xU+C6Bzrvx44CiCKU6cJI0tocTRXlVV6M7x1NDqHjicc+XDA0cxFAuBI5Q42quqCpU4UtPHes8q51z58MBRjFiCOG3sbK+qKrZxvH/07AMdOufKiAeOYsTiHUscXelVBT60unOurHjgKIbixJRkZ3OqqioEjkK9qqpqomcPHM65MuKBoxihxLGrqqrIGwBTgaNpW+ny5pxzbzAPHEVQLEGiQ6+q0B23UON41YDo2QOHc66MeOAogmLxjvdxtBXZHbe9xFFfusw559wbzANHERTPVVVVoMSRqIpKJV7icM6VEQ8cRchZ4ijUOA5RqcMDh3OujHjgKEIsXkEi630cBaqqwAOHc67seOAoglL3cTRnBo4CVVXggcM5V3ZKGjgkHS/pRUnLJV2SZb0k/SqsXyzpbWnrVkpaImmRpPlpy4dI+qukl8Pz4FKeA0SBoyJmNLZm9KpK9Cm8c9UADxzOubJSssAhKQ5cBZwAHAicJunAjM1OACaHx7nA1Rnr32tm08xsRtqyS4CHzGwy8FB4X1qKk5DtugGwpQFiFRAvYgLFqhrvVeWcKyulLHHMBJab2QozawZuA2ZnbDMbuNEiTwKDJI0qkO5s4Ibw+gbgw92Z6axicSqU1quqZeeuAQwL8aoq51yZKWXgGAO8lva+NiwrdhsDHpS0QNK5aduMMLO1AOF5eLfmOpsQONp7VbU0QEUR1VTggcM5V3aKqGvZbcqyLHMO1XzbvNPM1kgaDvxV0gtm9ljRB4+CzbkA48ePL3a3HIlFVVWNHUocHjicc29NpSxx1ALj0t6PBdYUu42ZpZ43AHcTVX0BrE9VZ4XnDdkObmbXmtkMM5sxbNiwPTuTWJy4bPerqlobd9006Jxzb3KlDBzzgMmSJkmqBOYAczO2mQt8MvSuOgLYamZrJfWTVAMgqR/wfuC5tH3OCq/PAu4p4TlEYgkSShurqmUnVBRxDwdApY+Q65wrLyWrqjKzVknnAw8AceA6M1sq6byw/hrgPuBEYDnQAJwddh8B3C0plcdbzOz+sO4K4A5J5wCvAqeW6hzaKRbdANi8myUOiHpW9R1Smvw559wbqJRtHJjZfUTBIX3ZNWmvDfhilv1WAIfmSHMjcEz35rSAUFXVoXG8f5Ft8j60unOuzPid48VQ5gyAjV1rHAcPHM65suGBoxixRMfRcVsaulBV5XNyOOfKiweOYsTixNMHOWzZWdwAh+AlDudc2fHAUYz2qqokZpazcXzVxh185ob5vL69addCn8zJOVdmPHAUIxYjRtQVt6mlLeed4/ctWcfflq3nF397addCL3E458qMB45ixBLtgWNnYyNYMmvgeObVzQDc+vRrLN8Q7tuo7AfIA4dzrmx44CiG4sQsat9o2hkCQkbgMDOeeW0L79lvGH0q4lzxlxfCvvKh1Z1zZcUDRzFicWIWqqpyBI7VW3ZSt62J9x0wnM8ftQ9/W7aeF9aFdo2qGmjyO8edc+XBA0cxFEdEJY7mnTuiZRmN48+8ugWA6eMHc+LUaGT4xa9tjVb6nBzOuTJS0jvHy0a8gliyBYCWxlTg6FjieObVLVRXxNh/ZA0xiT4VcZZ1KHF4VZVzrjx44ChGVQ2xZAtVNNPS1BAtyyxxvLaZqWMGUhGPCnH7j6zhhbXb2vencesbmWPnnCsZr6oqRvVAAGrYSWuqxJF2A2BTaxtLV9czffyu6c8PGFXDsnX10X0fXuJwzpURDxzFCMOG1KiBtqbObRzPr6mnuS3J9HGD2pdNGTmALQ0trK9v8sDhnCsrHjiKUR0CBw20tVdV7WrjeGFdFBQOHjOwfdmUkdGNf8vW1Xt3XOdcWfHAUYy0EkeypXPgWLe1EQlGDtxVfTVlZLTPC2u3RSWO5m2QTL5xeXbOuRLxwFGM9hLHTqx5Z7QsrapqfX0jQ/tVtTeMAwzsW8HogdXRvRxVPgugc658eOAoRlqJw5pTJY5dpYv19Y2MGFDVabcDRg3YVeKArlVXLfkT1L1UeDvnnHuDlTRwSDpe0ouSlku6JMt6SfpVWL9Y0tvC8nGSHpG0TNJSSV9O2+cySaslLQqPE0t5DkB7iWNQbGc0Mi50KHGsq29i5IDOw6xPGVXDv+u201LRL1pQbOB4fi7ceQ7c8jFIBSrnnOslShY4JMWBq4ATgAOB0yQdmLHZCcDk8DgXuDosbwUuNLMDgCOAL2bs+3MzmxYeHaamLYlQ4hgc3wmtOyGWgHhF++r19Y2MGJglcIwcQGvSWN0Qti0mcNSvhf+7AAZNgM2vwCM/7JZTcM657lLKEsdMYLmZrTCzZuA2YHbGNrOBGy3yJDBI0igzW2tmCwHMbBuwDBhTwrzmF4tDZX8GxRqJtXaci6OptY1NO5qzljj2Dz2rXt0RDxsXcRPg3POhpRFO/xO8/VPw5P9A7YLuOAvnnOsWpQwcY4DX0t7X0vnHv+A2kiYC04Gn0hafH6q2rpM0mCwknStpvqT5dXV1u3cG6aoHMlA7UcZ84xvqo0mbsrVxTBzaj3hMvLwt3KDfsDn/MerXwPK/wbsvhGH7wbGXQ9+h8M9f7Hn+nXOum5QycCjLMuvKNpL6A3cCXzGz1CiBVwP7ANOAtcBPsx3czK41sxlmNmPYsGFdzXtnVQMYqIaoxJHo2DAOMCJLiaMyEWPC0L4s3hLWbV+X/xir/hU9T35f9Fw9EPY7HlY8Cm2te3wKzjnXHUoZOGqBcWnvxwJrit1GUgVR0LjZzO5KbWBm682szcySwG+IqsRKr3oANWog3taY0TAeBY6RWdo4ACYP78+S1y0KNtuKCByV/WHE1F3L9n1fVMW1ev4en4JzznWHUgaOecBkSZMkVQJzgLkZ28wFPhl6Vx0BbDWztZIE/A5YZmY/S99B0qi0tycDz5XuFNJUDaA/DSSSHauq1qeqqmpyBY4aVm3aifUfAdvX5z/Gqn/BuMMhnjb25N7vAcWiKqw9Ub8WFv8xenbOuT1QstFxzaxV0vnAA0AcuM7Mlko6L6y/BrgPOBFYDjQAZ4fd3wmcCSyRtCgs+2boQXWlpGlEVVorgc+V6hw6qB5AP2tgZ1sVVOwak2p9fSOViRiD+lZk3W3f4f1pSxqN1cPok6/E0bAJ6pbB1FM6Lu8zGMYeBssfgqO/vXt5b2uB206DNc9E7ye+G07/Y9bpb51zrpCSDqsefujvy1h2TdprA76YZb/Hyd7+gZmd2c3ZLE7VAPpaAwn6dhpuZOSAaqJCUmf7Du8PwJbYEPpsX5k7/VefiJ4nvKPzun2Ogb//CHZshH5Du573R/8rChrHXwE7N0fvn7gKZn2t62k5597y/M7xYlUPoG9yO5XJpqLuGk/ZZ1h/JFhvg2BbnqqqVf+CeBWMflvndfu+DzBY8UjX8/3qU/CPn8K00+GIz8N7vwlTToLHf164zcU557LwwFGsqgEkrIUa29ZpnKpsPapS+lTGGTu4D68210SN3Kk7zzOt+heMeXuHoNRu9LSoymr5Q13P9z9+Av2GR6WNlGMvh9YmePj7XU/POfeW54GjWGEyp8G2tb2qysxYV9+Y9ea/dJOH17C8IQw7kq2BvHkHrH0WJhyZPYFYHCa8E1b9s2t53r4hCjbTTmsfNgWAofvA4Z+DZ26GTa90LU3n3FueB45ihWFHEkq2lzjqd7bS2JLM2RU3ZfLw/jy/PbSLZKuuWr8UrC17NVXKhHfAllXRTYLFeu7OKN1D5nRed8Tno+dFtxSfnnPO4YGjeGlX7I1UArB+W3QPx/ACJY59h/dnTWuY5CnbTYBrn42eRx2SO5FUo3nqJsFiLL4dRh0Kw6d0XjdwLOxzdBQ4km3Fp+mce8vzwFGsql2BY2tL1Blt3dZw81+hqqoRNWywMDJKthLHusVRG8bAcR0WmxnL1tZTt60puimwsn/xgaPupagnVbbSRsr0M6C+Flb8vbg0nXOOEnfHLStpJY5NzXFGkHbXeIHAMWVkDfXxASSJE8ta4lgMIw+BtC69j7ywgV889DLPvraF/lUJLj5+f84YOxOluu0Wsvj26MbBgz+aJ2MfiALWM3+AfY8pLl3n3FuelziKlVbiqGuKRrtdtXEHiZgKtnFUV8Q5YPRgtsSydMlta4ENz3eoplpcu4XP3DifLQ3NXHrSgUwbN4hL71nKffWTom0bNhXO77K50Y1+NSNyb5OogkM+Di/8ubg0nXMODxzFC72qANbvjEoGL6/fzsS9+lGZKPwxTh83iLVtA0huyxjyo+4FaGuGUdOAaJj2i/64mL36VzL3/Hdxzrsm8YdzZnLB0ftyw+rR0T6vPpn/YHUvwesvwQEfLHxe08+Ijr/kj4W3dc45PHAULzX9K7CuIQocyzdsZ99h/Yvaffr4QaxLDqJpS0ZV1drF0fPIqMRx1cPLeXH9Nn70kakM7BMNYyKJL79vPyonHEazJdj64qP5D/bin6Pn/YuYHHHk1ChoPfOHos7DOec8cBQrFofKKHjUbo9KBqs2NTB5RHGB423jB7PBBnW+W3vd4qh779B9WLNlJ1c/+m9Onj6Go6d0rGKKx8SVc2byrPZj85IHiEZryWHZvVHX3oFFzn01/QxYtwTWLCq8rXPuLc8DR1eEBvLV2+HfG3bQlrT2sagKGTu4D9srhlLVvKnj3BprF0dX/bE41z62AjO48P37ZU1j9KA+JKacwMTWV/jrE/OyH6h+bTQE+5QPFH9eU0+Jhjt55qbi93HOvWV54OiK0EC+PVnJP16OZhUsNnBIos/QMcQw2BFmJEwmoxLHyEPYuL2J2+a9yoenj2Hs4L450znkmNMAePZvt7GtsaXzBqlqqmLaN1L6DIYDPwRL7sg9JIpzzgUeOLoilDh2UslDL2xAigYxLNbQEdF9GvV1tdGCtYugeTuMncHv/7mSptYk571nn7xpxIdNpnHg3hze8jS//NvLnTdY8icYOhn2yl5qyentn4LGrbDQ2zqcc/l54OiKUOJopJIFqzYzbnBfqiviRe8+dtwkAJavCD/4L/wZFGPLmKO44YmVnHDwyKJKMNUHfYB3JJZxxz+XsmBV2jzm65+Phmd/2yc73BOS7tWNDdyzaDX3P7eOBas2k0yGtpIJ74TxR0aj5rY2FX1Ozrm3Hr8BsCtCiaNFVbQljclFVlOl7Hfw22i8v5LNC++BYz8RBY4J7+QX/9rIjqZWvnxMkaWE/U4g8a//xwf7v8hFfxzGny94N30q4zD/uqitYtrpnXZ55MUNXPXwcuanBxpgwtC+fGLmeD555ET6vOdi+MOHox5Wh32m+BNra+04a6Fzrqz5f3tXhBLHkEEDWbMJ9i2yR1X77v2H8OKYD/LO2rk899TfOLhuGXX7fY8/PLKKOTPHs//ImsKJQDS9bJ/BfHXYc9z80nQuv/d5/vMDE9Gzt8FBH+4w2dPmHc18/97nueuZ1UwY2pevH78/791/OEkzXl6/nVuefpUf/eUFbnxiFd86cQonjDsc/ePnMO2M7EO8AzTWRw3pC2+ErbXQvC2qHpv8fph+Oow4qEufC8k22LwSNi6PRgruPyIawbdmZNfScc69IUoaOCQdD/ySaOrY35rZFRnrFdafSDR17KfMbGG+fSUNAW4HJhJNHfsxM+t4GV0qocSx15DBsGlr0fdwpJv4gQupuvZOhj/4BQD+c8Uk+lbE+Y9ju9AmEU/A289mr8d/xpWHvI+vPw0zN93Dyc3bYMY57Zvdt2Qt37nnObY0tHDB0fvyxaP3pSqxq2rtoNED+fD0MTy5YiOXzV3KF255hnPGnMyl9ZfAXZ+BU2+IuiGntLXC/N/Bwz+Epq00jz6M1/c+lY2tlQzZvJiRT/+G+JNXUT/xeKqOuYSqcdNzn4MZrF4Ii2+DpXfv6jCQbsTBsN9xcODsTkOyOOd6TskCh6Q4cBVwLFALzJM018yeT9vsBGByeBwOXA0cXmDfS4CHzOwKSZeE9xeX6jw62P9EaNrOmOYBwFYmjyiyhJCmavRB1A45krGbnmBpcgJ3v5Lg0pP2Y6/+uWcRzOqoS+ClBzh19X8xfL+PceSqX7OiYl8eemUvtGoF9y5ey6LXtnDwmAHc+OnDOXD0gJxJHbH3UO790ru4dd5r/PTBCpKtZ/LdZX/gtRs+C8dcyojhI2l+7h7iT/yKPhuX8kLfGfy48mM8tGJsWiqzGMh2zk7cz6dfuZ+q393P0xUzeXHsKdRMOYr9xo1i0l796LNtJbz8ICy6GdYtwRLVNE56H5tHvYeNfSayNVlJy5Z11GxZxviNj7PX478g9o+f0lgzkYbJJ5E44CT6jz+EWFW/XYdOJmHnJti2NrpPZtta2FGHJdtoM9FaPZi2PntB/+EkakYQ7zeEeJ+BKJalic8supO+aXtUkmreES2LV0KiMjxXR/feJKo8mLm3JOW9kWxPEpaOBC4zs+PC+28AmNmP0rb5NfB3M7s1vH8ROIqoNJF139Q2ZrZW0qiw//758jJjxgybP39+t53bzU+t4od/XsbT33of/au6Hnublt1P1e0f519jP0PN8d9h6tiBhXfKZt1z8Jv3QlsztXu9my/t/CzPbIzyc+CoAZw6YyxnHjGBRLz4PhBbGpq57vFXGPzklZyd/FOUX0tQpVZWJkdwZevHWdBvFjMmDeXt4wczZWQN44f2pW9lgraksb6+kdXr1tPnmd8yffUt1Ng2mi3OVvpTRQsD1ADAS0zkprZjuKvlSLaTu/vxYOo5Lj6fE2NP8Y7YUhJKkjTxugbRQgUJ2hjCVipozZlGNq0WYxv9qKcfcSWpppk+NFFFMwmKG2a+jRiNVNJINTupolW7/haEB5R0ojS/M2+EN3PeAba//2cccOQJu7WvpAVmNqPT8hIGjlOA483sM+H9mcDhZnZ+2jb3AleY2ePh/UNEpYeJufaVtMXMBqWlsdksNWZ5h+OfC5wLMH78+LevWrWq286ttS3JpoZmhtfkH9wwJzNYfAfsf3yHMbB2y7J7oWEjTD8TYjHWbW2kpS3JuCG5f4yL0draxssLH6Zx5TzYvIq6kbNonXQUU8cOZuzgPqiYK+2WRtpWPcnWpQ+wbfPrbGkWq2NjeLbPYWytHsugvpUM6lvBoD4VDOxTset93woq4zEamtvY2dJGQ3Mb2xpb2L5pPZW1T1C9+SX6NKwGayVpYmtiKFvjQ2moGkZD1XAaq4fRXD2UeEUl1XGoSW6lpmUTVU2vU9m4kYrmrVS0bKWyuZ7K1npaLU6zKmmkiiZV0qQ+NMf70hTvQ0usL0gkrI0ELSSslQprpjK5k8pkIxXJRiqSO6lo20ncWjGirxd4k//cdL839+fx5r0QGPj+r7Pv1ByzixaQK3CUso0j2yed+beTa5ti9s3LzK4FroWoxNGVfQtJxGO7HzQgqt449OPdk5kDTurwttBIvcVKJOIcMPNYmHns7idSUU1836MYsu9RDAEmAIcSNWgVY2inJcPg8IN3Pz/OuW5Ryvs4aoH0mYnGApnznubaJt++60MVFeF5Qzfm2TnnXAGlDBzzgMmSJkmqBOYAczO2mQt8UpEjgK1mtrbAvnOBs8Lrs4B7SngOzjnnMpSsqsrMWiWdDzxA1KX2OjNbKum8sP4a4D6imovlRN1xz863b0j6CuAOSecArwKnluocnHPOdVayxvHepLt7VTnn3FtBrsZxH6vKOedcl3jgcM451yUeOJxzznWJBw7nnHNd8pZoHJdUB+zureN7Aa93Y3ZKwfPYPTyPe6635w88j10xwcyGZS58SwSOPSFpfrZeBb2J57F7eB73XG/PH3geu4NXVTnnnOsSDxzOOee6xANHYdf2dAaK4HnsHp7HPdfb8weexz3mbRzOOee6xEsczjnnusQDh3POuS7xwJGHpOMlvShpeZjfvKfzM07SI5KWSVoq6cth+RBJf5X0cnjuNCNiD+Q1LumZMMtjr8ujpEGS/iTphfB5HtkL8/jV8D0/J+lWSdU9nUdJ10naIOm5tGU58yTpG+H/50VJx/VgHn8cvuvFku6WlD6LaK/IY9q6r0kySXv1ZB7z8cCRg6Q4cBVwAnAgcJqkA3s2V7QCF5rZAcARwBdDni4BHjKzycBD4X1P+zKwLO19b8vjL4H7zWwK0cSEy+hFeZQ0BrgAmGFmBxNNLzCnF+TxeuD4jGVZ8xT+NucAB4V9/if8X/VEHv8KHGxmhwAvAd/ohXlE0jjgWKIpI1LLeiqPOXngyG0msNzMVphZM3AbMLsnM2Rma81sYXi9jejHbkzI1w1hsxuAD/dMDiOSxgIfAH6btrjX5FHSAGAW8DsAM2s2sy30ojwGCaCPpATQl2gWzB7No5k9BmzKWJwrT7OB28ysycxeIZp3Z2ZP5NHMHjSz1vD2SaJZRXtVHoOfA1+n41TZPZLHfDxw5DYGeC3tfW1Y1itImghMB54CRoSZEwnPw3suZwD8guiPP5m2rDflcW+gDvh9qE77raR+vSmPZrYa+AnRledaotkxH+xNeUyTK0+99X/o08Bfwutek0dJHwJWm9mzGat6TR5TPHDkpizLekXfZUn9gTuBr5hZfU/nJ52kk4ANZragp/OSRwJ4G3C1mU0HdtDzVWcdhHaC2cAkYDTQT9IZPZurLut1/0OSvkVU5XtzalGWzd7wPErqC3wL+E621VmW9ejn6IEjt1pgXNr7sURVBT1KUgVR0LjZzO4Ki9dLGhXWjwI29FT+gHcCH5K0kqh672hJN9G78lgL1JrZU+H9n4gCSW/K4/uAV8yszsxagLuAd/SyPKbkylOv+h+SdBZwEnC67bqBrbfkcR+ii4Rnw//OWGChpJH0njy288CR2zxgsqRJkiqJGqfm9mSGJImoXn6Zmf0sbdVc4Kzw+izgnjc6bylm9g0zG2tmE4k+s4fN7Ax6Vx7XAa9J2j8sOgZ4nl6UR6IqqiMk9Q3f+zFEbVq9KY8pufI0F5gjqUrSJGAy8HQP5A9JxwMXAx8ys4a0Vb0ij2a2xMyGm9nE8L9TC7wt/K32ijx2YGb+yPEATiTqgfFv4Fu9ID/vIiqiLgYWhceJwFCi3iwvh+chPZ3XkN+jgHvD616VR2AaMD98lv8LDO6Fefwe8ALwHPAHoKqn8wjcStTm0kL043ZOvjwRVb/8G3gROKEH87icqJ0g9X9zTW/LY8b6lcBePZnHfA8fcsQ551yXeFWVc865LvHA4Zxzrks8cDjnnOsSDxzOOee6xAOHc865LvHA4VwvJ+mo1CjDzvUGHjicc851iQcO57qJpDMkPS1paarGjQAAAY9JREFUkaRfhzlJtkv6qaSFkh6SNCxsO03Sk2nzQwwOy/eV9DdJz4Z99gnJ99eu+UNuDneTO9cjPHA41w0kHQB8HHinmU0D2oDTgX7AQjN7G/Ao8N2wy43AxRbND7EkbfnNwFVmdijR2FRrw/LpwFeI5obZm2hMMOd6RKKnM+BcmTgGeDswLxQG+hAN9pcEbg/b3ATcJWkgMMjMHg3LbwD+KKkGGGNmdwOYWSNASO9pM6sN7xcBE4HHS39aznXmgcO57iHgBjP7RoeF0qUZ2+Ub4ydf9VNT2us2/H/X9SCvqnKuezwEnCJpOLTPwz2B6H/slLDNJ4DHzWwrsFnSu8PyM4FHLZpbpVbSh0MaVWGeBud6Fb9qca4bmNnzkr4NPCgpRjTq6ReJJok6SNICYCtROwhEw49fEwLDCuDssPxM4NeSLg9pnPoGnoZzRfHRcZ0rIUnbzax/T+fDue7kVVXOOee6xEsczjnnusRLHM4557rEA4dzzrku8cDhnHOuSzxwOOec6xIPHM4557rk/wPdK6ABV4KOAgAAAABJRU5ErkJggg==\n",
+      "text/plain": [
+       "
    "
+      ]
+     },
+     "metadata": {
+      "filenames": {
+       "image/png": "/Users/hjensen/Teaching/FYS-STK4150/doc/src/LectureNotes/_build/jupyter_execute/chapter10_7_411.png"
+      },
+      "needs_background": "light"
+     },
+     "output_type": "display_data"
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "MSE:  9.884935143687282e-05\n"
+     ]
+    },
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD4CAYAAADhNOGaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deVhV5drH8e/NJKACgoizqDnmlOAUWo5lapmWpb6WZWWmVp5G63TKhnMytcksOw45lGU2aulJzSE1R1RUnHHGeUgFZd7P+wdbAgEF94a1gftzXfvaa6/nWXv9WAW3a3qWGGNQSilVcrlZHUAppZS1tBAopVQJp4VAKaVKOC0ESilVwmkhUEqpEs7D6gA3onz58iY0NPTGFr54EVJTnZpHqULj4QF+flanUEXUxo0bzxhjgq+eXyQLQWhoKJGRkTe28OzZULGicwMpVVhOnIC+fa1OoYooETmU03w9NKSUUiWcUwqBiHQVkd0iEiMiI3NoFxEZb2/fKiLN87qsUkqpguVwIRARd+BT4C6gIdBPRBpe1e0uoI79NRiYmI9llVJKFSBnnCNoCcQYY/YDiMhsoCewI1OfnsBMkz6exVoRCRCRSkBoHpZVSpUAKSkpxMbGkpiYaHWUIs/b25uqVavi6emZp/7OKARVgCOZPscCrfLQp0oel1VKlQCxsbGULVuW0NBQRMTqOEWWMYazZ88SGxtLzZo187SMM84R5PRf7OqR7HLrk5dl079AZLCIRIpI5OnTp/MZUSnl6hITEwkKCtIi4CARISgoKF97Vs4oBLFAtUyfqwLH8tgnL8sCYIyZZIwJN8aEBwdnuww2Tz5Zspfvz59iyrH5xKVevqHvUEoVHC0CzpHf7eiMQrABqCMiNUXEC+gLzLuqzzzgYfvVQ62BC8aY43lc1iliTsXx/uI9PBa7nCf2jiN49X303/Eeqy9Eo0NxK6VKMocLgTEmFRgOLAR2AnOMMdtFZIiIDLF3WwDsB2KAycDQay3raKaczImMxZBCvMdSAJJMIt+c/o2IqKcJXfsIYw7P5lTyXwWxaqVUEXH+/Hk+++wzq2MUOqfcR2CMWWCMqWuMqW2M+bd93ufGmM/t08YYM8ze3tgYE3mtZQvCPU0rE17/ADa5kK3tcPJhXj7wXyqt6UPXLa+z4Oxa0kxaQUVRSrmo3ApBWlrx/ntQYu4sblTFnw/u7c2L5e4l0D0oxz420lh4fiXdo1+hwp99eXn/FPYn5HjKQilVDI0cOZJ9+/bRrFkzWrRoQYcOHejfvz+NGzfm4MGDNGrUKKPvuHHjGDVqFAD79u2ja9euhIWF0a5dO3bt2mXRT3BjiuRYQzeqRkANxlR4kP80Hs7Cc+sZH7uA38+vwUb2an8u7QxjjsxizJFZtCjTjGeqdOO+4NvwcS9lQXKlSpbQkfML7LsPju6ea9vo0aOJjo4mKiqK5cuX0717d6Kjo6lZsyYHDx7MdbnBgwfz+eefU6dOHdatW8fQoUNZunRpAaQvGCWqEFzhIe50D2pD96A2nEw+x/QTi/j06AKOJB/Jsf+G+Cge2h3Fk3vH0y+4E89Vu5eGpUMLN7RSqtC1bNnyutfix8fHs3r1avr06ZMxLykpqaCjOVWJLASZhXgF8nL1vrxU7UHWXNzOJ0fn8+OZ5SSb7NfgXrbFM/XkXKaenEt7/1a8XuNB2gc000velCqmSpcunTHt4eGBzWbL+HzlOn2bzUZAQABRUVGFns9ZSnwhuEJEuNW/Ebf6NyIu9Wlmn1rK+NgFRCfszLH/8gvrWL51HfW86/Cv0Ad4ILg9nm66OZVyhmsdvilIZcuWJS4uLse2kJAQTp06xdmzZylTpgy//vorXbt2xc/Pj5o1a/Ldd9/Rp08fjDFs3bqVpk2bFnL6G1diThbnR1kPX56o3INtLT8jOvwLhlfqQ1k3/xz77k7cy4Bd/6bymv6MOfwtF1MvFXJapZSzBAUFERERQaNGjXjxxReztHl6evL666/TqlUrevToQf369TPaZs2axdSpU2natCk333wzc+fOLezoDpGieDNVeHi4KewH0yTbUvjx9EreOfQd2xNyvyLgHv+H+arRI5T10MNFqgAU4wfT7Ny5kwYNGlgdo9jIaXuKyEZjTPjVfXWPII+83DzpG9KRbS0+Y2Wz8XQJiODqoZLEeLHpxB3cuvwy/9mVzLEEW85fppRSLkQLQT6JCG39G7Oo6TvsbjGDR0LuxhMvAEqndcSdAOJSYdLBFG5bkcCILYlEX0zjWNIZbEYLg1LK9ejZTQfU9a3GtPrPMabWID45Ohd/WzsWHBP2XUo/3JZq4Ofjafx0PJXzvs9Tzkt4rXofHqrYhVJuXhanV0qpdLpH4ATBXgG8VXMgz9euxeK2PkxtXopW5f7etIluG7loDnMo6RBP7B1H1TUDOJccb2FipZT6mxYCJ3MToVMFD75t5cO8Nt7cXdGdix4/ZulzKekm7lvrxg9HU0i1Fb2T9Uqp4kULQQFq4u/O+03cuTO4PG6ZNnVASl8OXDY8vy2ZjisTmBObQooWBKWURbQQFDAvN09+bvwv9reaxZOVetPM5zbKu9fOaD+cYHgpOpkOKxMYs28P/zn0DfFpCRYmVko5w/Lly+nRowcA8+bNY/To0bn2vdHhr0eNGsW4ceNuOOMVWggKSQ3vinxe92k2tRjFqtt9eaGOJwGZnisdm2B48+BX/PPgJCqt7svbh2bpzWlKuaAbGZL6nnvuYeTIkbm2W/0cBL1qqJCJCH6eMLy2F4/U8GTm4RQmH0jhZOphLruvAiDedpHXD07hvcPf8kLV+/lHtd74e5SxOLlShUvedO5NmeaN6x9+PXjwIF27dqVVq1Zs3ryZunXrMnPmTBo2bMigQYNYtGgRw4cPJzAwkDfeeIOkpCRq167NtGnTKFOmDL/99hsjRoygfPnyNG/ePON7p0+fTmRkJBMmTODkyZMMGTKE/fv3AzBx4kTGjx+fMfx1ly5dGDt2LGPHjmXOnDkkJSXRq1cv3nzzTQD+/e9/M3PmTKpVq0ZwcDBhYWEObxvdI7BQGQ9haC0vVt3uS2jQTyBZ/0e9ZIvjzcPTqLS6L//cP42/UnIeA0Up5Ty7d+9m8ODBbN26FT8/v4x/qXt7e7Nq1So6d+7MO++8w++//86mTZsIDw/ngw8+IDExkSeeeIJffvmFlStXcuLEiRy//5lnnuH2229ny5YtbNq0iZtvvpnRo0dTu3ZtoqKiGDt2LIsWLWLv3r2sX7+eqKgoNm7cyIoVK9i4cSOzZ89m8+bN/Pjjj2zYsMEpP7MWAhdQ2kP4telwRlZ7CB/xzdaeYC7xnyMzqbymLyP3TeVcykULUipVMlSrVo2IiAgABgwYwKpV6XvqDz74IABr165lx44dRERE0KxZM2bMmMGhQ4fYtWsXNWvWpE6dOogIAwYMyPH7ly5dylNPPQWAu7s7/v7ZxzFbtGgRixYt4pZbbqF58+bs2rWLvXv3snLlSnr16oWvry9+fn7cc889TvmZtRC4iEBPP96tNYhjt37Lv6o/gq9b9kNBieYy78V+RdU1/Rh1YKaeVFaqAFw9rPyVz1eGpDbG0KVLF6KiooiKimLHjh1MnTo1x2VvlDGGV155JWMdMTExPPbYY05dR2ZaCFxMgEcZ3qo5kGNtvmFUjUGUdiubrU+Cucybh6dRZXU/PjjyPUm2ZAuSKlWwzBvGqa+8Onz4MGvWrAHgm2++oW3btlnaW7duzZ9//klMTAwAly9fZs+ePdSvX58DBw6wb9++jGVz0qlTJyZOnAikn3i+ePFituGv77zzTr744gvi49NvPD169CinTp3itttu46effiIhIYG4uDh++eWXPP9c16KFwEX5e5ThjdCHON5mNu+EPpHjMNgXbRd4fv+nVFvzEFOOLyDVFO8HbCtVGBo0aMCMGTNo0qQJ586dyziMc0VwcDDTp0+nX79+NGnShNatW7Nr1y68vb2ZNGkS3bt3p23bttSoUSPH7//4449ZtmwZjRs3JiwsjO3bt2cb/vqOO+6gf//+tGnThsaNG3P//fcTFxdH8+bNefDBB2nWrBn33Xcf7dq1c8rP7NAw1CISCHwLhAIHgQeMMX9d1acaMBOoCNiAScaYj+1to4AngNP27q8aYxZcb71WDENttfi0BD6O/Yl3D8/mki3nk8bVvaoxrvYg7gu+DTfRGl8s6TDUBergwYP06NGD6OhoS3M4Q2EOQz0SWGKMqQMssX++WirwvDGmAdAaGCYiDTO1f2iMaWZ/XbcIlFRl3H34Z43+xLb5mperDsBLvLP1OZx8hAd2vknD9UNYeG49RfFZE0qpwudoIegJzLBPzwDuvbqDMea4MWaTfToO2AlUcXC9JVaARxlG136Mw61nMaRSb9zxzNZnd+Jeum57mRabRvDnhW0WpFSqaAoNDS0WewP55WghCDHGHIf0P/hAhWt1FpFQ4BZgXabZw0Vkq4h8ISLlrrHsYBGJFJHI06dP59atxAjxCmRi3afZ12om/YK7Ijn8p9wYv5W2Uc/QMWokW+JjLEipVP7oXqxz5Hc7XrcQiMjvIhKdw6tnflYkImWAH4ARxpgrF8JPBGoDzYDjwPu5LW+MmWSMCTfGhAcHB+dn1cVaDe+KfN3wZXa0mEa3crfl2GfZhXUM27GCo/rENOXCvL29OXv2rBYDBxljOHv2LN7e2Q8f5+a6Q0wYYzrn1iYiJ0WkkjHmuIhUAk7l0s+T9CIwyxiTMSazMeZkpj6TgV/znFxlUd+3OvObvMnGuN2MiJnKqot/33HoZvw4fO5uOqxIYEB1D4bX9iLQS5+prFxL1apViY2NRff4Heft7U3VqlXz3N/RsYbmAQOB0fb3uVd3kPS7H6YCO40xH1zVVunKoSWgF1DyDs45WVjZeqy8ZQx/nI/imb2T2Xp5B/4pD+CGL8kGvjiUyndHUxlay5NHa3ji7a4FQbkGT09PatasaXWMEsnRcwSjgS4ishfoYv+MiFQWkStXAEUADwEdRSTK/upmbxsjIttEZCvQAfiHg3mU3e0BzYgKn8D8Ru/yQ7PeNPP/+z91XCq8tyeFDivjCdvwMlOPLyBN70FQqsRy6D4Cq5TE+wgcZYxh0ak03tuTzH77M5Xj3X/nrNdHANzkXYvP6jxFl8BslxgrV1KM7yNQBa+g7iNQRYSIcGeIBwsjfHi7oRflvBI57/llRntM4n6G7JzP3ng9oaxUSaOFoITxdBMequ7J200OIpJpFFPjQVL8Q9y5KoFXopM4laQFQamSQgtBCdWjfBgxrWbSO6gTAH6p9+BhQrAB38Sm0n5FAh/HJHM5tegdOlRK5Y8+oawEq+FdkR8avUZkXB+SU0L4bJ8bq86m7wlcToMPY1KYdSSV0v7j6RPSgMGVe+Ah7hanVko5m+4RKMLL1uPWwAC+DPdmelgp6pX5+5LSQylRLI9byLCYj7hp7SB+PbtGb/hRqpjRQqAyiAjtgz1YEOHDe428CC5lOO/5RUb7oeTD3B39Krdufp5t8fstTKqUciYtBCobdxEerOrJhOYnMO6x2drXxm2m6cYneGzXR5xJuWBBQqWUM2khULlq5X8TB1p9Sb/grkDWO5ANNr44OZcaawfw4ZEfSLGlWhNSKeUwLQTqmqqUCubrhi8TFTaJW8s2z9Z+2RbPc/snUGfdYyw8t96ChEopR2khUHnStMxNrLplHD/f/DaVPStnaz+UfJiu216mU9Qr7Ll8xIKESqkbpYVA5ZmI0LN8W/a3nsZ/ag7GW3yz9Vl6YS0NNwzi6b2fcSE13oKUSqn80kKg8q2UmxevVO/HgdZf0j/4Lq4+f5BGKhOOfUf1NQP477FfdEA7pVycFgJ1wyp6BTKr4UtENp/ILaVvztZ+0XaBIXs/oOH6J1lxfosFCZVSeaGFQDksrGw9NoZ9wjf1/0V5j+xPj9uTuI/bt4zg7m2jOJR4woKESqlr0UKgnEJE6BvSkUOtZ/LP6gPxlFLZ+vx67g9uWjeQV/dP43JaogUplVI50UKgnMrX3Zt3aj7CvpYzuTewY7b2VJJ598hMaq17lPln11iQUCl1NS0EqkBU867AT43/xcpm42ngUzdb+8mUE7y1+yj79PkHSllOC4EqUG39GxPdYiJT6r6Iv3u5jPlettqcuHgHXf9M4L3dOty1UlbSQqAKnJu48VilbhxqPZOnKt2HG+4EJT+F4E6KgYkHUui0KoH5J1J1ZFOlLKCFQBUaf48yfFZ3OEdaz2Zxm2aEBfz9v9/xRMOwqCT6bThP7+h32Xs5+2B3SqmC4VAhEJFAEVksInvt7+Vy6XdQRLaJSJSIROZ3eVW8VC5Vnpv93PmulTfjGnsR5PV32/8ufs1PZxfRYMMgXto3Ra8uUqoQOLpHMBJYYoypAyyxf85NB2NMM2NM+A0ur4oZNxHur+LJ0na+PFLdg1SJ5aLHzwCkkcLY2FkM3jlbDxcpVcAcLQQ9gRn26RnAvYW8vCoG/D2FUQ1L0bX6MpC/h7N2N0GsiO3BgMhEYvTqIqUKjKOFIMQYcxzA/l4hl34GWCQiG0Vk8A0sr0qAyfWf5Kv6/6SceyAA5VIeww0f/jxro+ufCby7O5l4vbpIKae77sPrReR3oGIOTf/Mx3oijDHHRKQCsFhEdhljVuRjeewFZDBA9erV87OoKiJEhP8L6czdQW2YfGwxcfEdmHk4DRuQauC/B1L4+Vgqr9X3oKz3DtqXa2Z1ZKWKhesWAmNM59zaROSkiFQyxhwXkUrAqVy+45j9/ZSI/AS0BFYAeVrevuwkYBJAeHi4/rOwGPPzKM3z1dOPEj5YzcbrO5LY8Ff6oaGTSYZHoudzzms8dwa0ZXK9p6nmrTuSSjnC0UND84CB9umBwNyrO4hIaREpe2UauAOIzuvyqmRrUNaNOS29+bBJKYJLCWlc4LznNAAWnl9FnfWPMPbIHFJ1qGulbpijhWA00EVE9gJd7J8RkcoissDeJwRYJSJbgPXAfGPMb9daXqnMRIRelT1Y0taHoIDp2CQuoy3JJPDS/oncvP5J1l/caWFKpYqu6x4auhZjzFmgUw7zjwHd7NP7gab5WV6pnPh5CjMaPcBDO4+w7fKuLG17EvfRavMwHg25mw9ueoIAjzIWpVSq6NE7i1WR0rTMTUSFf8qEm0bgI6WvajVMOzmPmmsH8s3JJXr/gVJ5pIVAFTlu4sawKj050HomvYKy71CeTztH/13vcFvUS8QkHLUgoVJFixYCVWSFeAXyY6PXWNxkHFU8K2drX3UxkgbrH+X1AzNJsiVbkFCpokELgSryOpcLI6b1NF6p9jDueGZpSyWFtw9Po866x1l+PsqihEq5Ni0EqljwdvPiP7UeZXuLKbQqk/1GsyPJR+iw5R88sP0/nE4+b0FCpVyXFgJVrNTzrc6a5h8wo94r+Ln5Z2v/7sxiaq57mOknFurJZKXstBCoYkdEeLjiHRxoPZOHKnTP1n7JFseju0fTbvOL7E84ZkFCpVyLFgJVbAV6+jGzwQv82ewTapUKzdb+Z9xG6m8YxHuHv9U7k1WJpoVAFXu3+jdiV8vJvB36BB54ZWlLMUmMPPA5TTYMZXPcXosSKmUtLQSqRPB08+C1Gv3Z2fKLHE8m70zYQ9/oL4jTYa5VCaSFQJUoN/lUYU3zD5hc90V83f4ehkKMD5cuDqbLygR+P5V6jW9QqvjRQqBKHBHh8Urd2NdqBncHtgcgIOVhPEwFTiQZHt+UxLCoRE4n6d6BKhm0EKgSq6JXIPMav8Hvjccx9eZeBGU6fTD/RBqdV11mTmwKay/s0EtNVbGmhUCVeJ0Cw+hVuRS/t/Xl/ip/D8h7IQWe3rGeNlHD6LLlnxxJzPW5SUoVaVoIlLIr5yWMa1yKr8K9qeYj2EjknOcnACy5sIY66x/h65NLLU6plPNpIVDqKm3Lu7OorQ+h5b8l1e1kxvwkWyKf7C1H9AW950AVL1oIlMqBj7vweYNuNPVtmDGvbFoPjsfXpefaRMbuSSbJpucNVPGghUCpXDQsHcqm8E8YX/tZKnvWICTtIQDSDHy6P4UeqxPYfF73DlTRp4VAqWtwEzeernovh9tMZXFEEC3L/f0rszfecN/aRF6IPsRbB78ixab3H6iiSQuBUnngLu6ElnZjdktv3m7oha97+vw0bEw49QFvHJpK4w1PsSU+xtqgSt0ALQRK5YObCA9V92RhhA9tg9yId/+NJPetAOxOjKH5xiG8un8aybYUi5MqlXcOFQIRCRSRxSKy1/5eLoc+9UQkKtProoiMsLeNEpGjmdq6OZJHqcJSzdeNaWGeePj+lGW+jTTePTKTBusHExm326J0SuWPo3sEI4Elxpg6wBL75yyMMbuNMc2MMc2AMOAykPm358Mr7caYBQ7mUarQeLp5EN3iM3oGdsjWtj/pIC03DeW5mP+SqM9LVi7O0ULQE5hhn54B3Hud/p2AfcaYQw6uVymXEOwVwM+NX+fHhm8R4B6Ypc1g48Ojs6m37nFWX4i2KKFS1+doIQgxxhwHsL9XuE7/vsA3V80bLiJbReSLnA4tXSEig0UkUkQiT58+7VhqpZysV3A79rWaxgPlu2RrO5x8hIioZxi251MupyVakE6pa7tuIRCR30UkOodXz/ysSES8gHuA7zLNngjUBpoBx4H3c1veGDPJGBNujAkPDg7Oz6qVKhSBnn58e/Or/NroPwR5lL+q1fDZ8e+ps+4xVpzfYkk+pXJz3UJgjOlsjGmUw2sucFJEKgHY3681KtddwCZjTMY9+8aYk8aYNGOMDZgMtHTsx1HKet2D2rCv1TQGVMh+7cOxlGPcvuUfDN0zgYS0JAvSKZWdo4eG5gED7dMDgbnX6NuPqw4LXSkidr0APZCqigV/jzJ82eBFFjUeSwWPkKtaDROP/0C99U+w7uIOS/IplZmjhWA00EVE9gJd7J8RkcoiknEFkIj42tt/vGr5MSKyTUS2Ah2AfziYRymX0iUwnJhWX/B4SPYjqUeSj9Bm89O8sG8ySXplkbKQFMUHboSHh5vIyMgbW3j2bKhY0bmBlMqDZX9tpu+O9ziVejJbW23vmnx/86s0K3PTtb/kxAno27eAEqriTkQ2GmPCr56vdxYrVUg6lLuFmFZf8FCF7tna9iUeIGzjEF4/MFPHLFKFTguBUoWorIcvMxu8wIJGowl0D8rSZiONtw9Po1nkMHZcOmhNQFUiaSFQygJ3BbUiptU07s/hvoMdCXtoGjmYzXH7LUimSiItBEpZpJxnWb67+VV+aPgmfm7+Wdo805rxclSIPg1NFQotBEpZrHfwbcS0mk63cu0AcDNlCUx+mj2X4N61iXwUk0yKPg1NFSAPqwMopdLHLPq18Zt8fWoJa8+6szg2kEQbpBr4KCaF30+l8UHjUtS1OqgqlnSPQCkXISL8X0hnPmnYgf9F+BAW8PevZ/RFGz1WJ9D54CymbJpKUbzsW7kuLQRKuaCapd2Y08qbV+t54SXp8y5IJEuSfuWJXx6n5zcPcCHxgrUhVbGhhUApF+UuwuCanvx6qw/1yl7irNf4jLZf9n5Pm8l36p6BcgotBEq5uLpl3fhH/cO4u136e6Zx49zxXjz55UbOxuvgdcoxWgiUKgK6BoWzNXwS9TxDAfBP7Yu3rRGLdpzkzo9WsHRX9mErlMorLQRKFRH1fKuzJfQtxnX+kOHhL2XMPxOfzKDpkbz60zYuJenwFCr/9PJRpYqQUm6ePB8xAoDODSrx4vdbOR2Xfmjo63WHWbpnOw3qrOPze0ZTyqOUlVFVEaJ7BEoVUe3rVWDhiNvoenP6aLqGNKIuvc30bR9x00dhbD+1y+KEqqjQQqBUERZY2ouJA5ozrk9TErznkOSe/qCb2EvbaTqxOeNWTtIri9R1aSFQqogTEbo1CcTHf3WW+Wkk8OLSJ2k7ubfec6CuSQuBUsWAr6cvW57axH0N7s/Wtvr4z1R7vxEL96yyIJkqCrQQKFVMBHgH8F2fOUzqMQlvd58sbXGpsXT9pj2P//g6NmOzKKFyVVoIlCpGRIQnwp5g45OR3Bzc6KrWNKZue5va77dj39lYS/Ip16SFQKliqGFwQzY8sZ6h4UOztR28tJoGExozYfUcC5IpV+RQIRCRPiKyXURsIpLtgciZ+nUVkd0iEiMiIzPNDxSRxSKy1/5ezpE8Sqm/+Xj68Gn3T/nxgR8J8M76q5XCeZ5e/CAdpgzkUnKCRQmVq3B0jyAa6A2syK2DiLgDnwJ3AQ2BfiLS0N48ElhijKkDLLF/Vko5Ua8Gvdg6ZAvtqrfL1rb86EyqjG3M8v0bLUimXIVDhcAYs9MYs/s63VoCMcaY/caYZGA20NPe1hOYYZ+eAdzrSB6lVM6q+Vdj2cBlvN3hbdzFPUvbhdR9dPzyVobPHa33HJRQhXGOoApwJNPnWPs8gBBjzHEA+3uFQsijVInk7ubOa7e9xspHVxLqH5qlzZDMp1GvUPfDjhw+f8KagMoy1y0EIvK7iETn8Op5vWWvfEUO8/L9zw4RGSwikSISefr06fwurpSya1OtDVFDoujfuH+2tpi45dQZ34gp6+dZkExZ5bqFwBjT2RjTKIfX3DyuIxaolulzVeCYffqkiFQCsL+fukaOScaYcGNMeHBwcB5XrZTKib+3P7N6z+LLXl9SxrNMlrZkc5YnFtxL12lPkpiizzooCQrj0NAGoI6I1BQRL6AvcOWfG/OAgfbpgUBei4tSygkGNBlA1JAoWlZumbVBDAsPT6LK2GasObTNmnCq0Dh6+WgvEYkF2gDzRWShfX5lEVkAYIxJBYYDC4GdwBxjzHb7V4wGuojIXqCL/bNSqhDVDqzNqkGreLXtq8hVR3LPpeyi+/Rn+X2HPvimOJOieJVAeHi4iYyMvLGFZ8+GihWdG0ipwnLiBPTtW2Bfv/zgcgb8OICjcUcBcDeBVEqcgDt+PNS6Bv/s3gBvT/frfItyVSKy0RiT7Z4vvbNYKZWhfWh7tj61ld4NeiMIdTxewR0/AL5ce4i7P1nFzuMXLU6pnE0LgVIqi0CfQL7v8z1rH1/L6n88w503h2S07T0VT89P/+SrtYew2XTwuuJCC4FSKm8vY8UAABK0SURBVBsRoWWVlpQr7cXnA8J4t3djfOyHhJJTbTw/bwY137+d/WePW5xUOYMWAqXUNYkI/VpW55en21K/YllSOcMZr/c5fHkVDSY05YsN/7M6onKQFgKlVJ7cVKEM3z/VCgn6GJuknydI5jSPz+/J6EWrsdmK3oUnKp0WAqVUniWlxVM9KOtDb/xT+/L50r94ZPoGzsTrDWhFkRYCpVSeBfkGsXzgcl689UUAKni2xC+1DwAr9pzmro9X8mfMGSsjqhughUAplS+e7p6M6TKG+f3ns3n4PJ66vU5G2+m4JAZMXce4hbtJTdOriooKLQRKqRvSrU43KvuFMPKu+swc1JLyZbwAMAY+Wbabxh88xo4Tx67zLcoVaCFQSjnstrrBLHi2HRE3BQFw3uMrdl2eTrP/Nmf8Sr2qyNVpIVBKOUWFst7MHNSKbi1OcNHzOwBSOMmzS+6h65SRJKakWpxQ5UYLgVLKadzdhENJP2adKaksPPoeoWM7suXoUWuCqWvSQqCUcqqfHvyJZ1s9m23+yZSVtJgczrhlCyxIpa5FC4FSyqm83L34qOtH/PDAD/iX8s/SliInePGPnnSe9DIJyXqoyFVoIVBKFYjeDXqz6clNhFUKy9ogqSw5PobQse2Jij2S88KqUGkhUEoVmFrlavHnoD8Z1mJYtrZTqX/SckoY45brgwmtpoVAKVWgSnmUYkK3CXzX5zv8SvllaUuR07y4vDedJo0gISXFooRKC4FSqlDc3/B+Nj+5mRaVW2RtEBtLj39MjbERbIo9YE24Ek4LgVKq0NQqV4tVg1bxfJvns7WdTtlAqylhjFn2vQXJSjYtBEqpQuXl7sW4O8bxa79fCfIJytKWKn/x8h8P0HHyUBJSki1KWPJoIVBKWaJ73e5EDYmiXfV2WRvEsOzYRKqNbcnG2L3WhCthHCoEItJHRLaLiE1EwnPpU01ElonITnvfZzO1jRKRoyISZX91cySPUqpoqepXlaUDl/Kv2/6FIFnazqZsodXUcN5dNsuidCWHo3sE0UBvYMU1+qQCzxtjGgCtgWEi0jBT+4fGmGb2l95yqFQJ4+HmwVsd3mLxQ4upWKZilrY0LvLqigHcPulRLicnWpSw+HOoEBhjdhpjdl+nz3FjzCb7dBywE6jiyHqVUsVPp1qdiHoyii61umRrW3F8OvU+vItTF7UYFIRCPUcgIqHALcC6TLOHi8hWEflCRMpdY9nBIhIpIpGnT58u4KRKKSuElAnhtwG/8W6nd3EX978bjBspF7rSbfwqVu/TJ6A523ULgYj8LiLRObx65mdFIlIG+AEYYYy5aJ89EagNNAOOA+/ntrwxZpIxJtwYEx4cHJyfVSulihA3cWNk25H88cgfVPOrBkBA6v/hbWvEmfgkBkxZx8Tl+zDGWJy0+PC4XgdjTGdHVyIinqQXgVnGmIwxao0xJzP1mQz86ui6lFLFQ0T1CKKGRDFh/QTaV3qS5+Zs5Ux8MjYD7/22i82H/2LcA03x8/a0OmqRV+CHhkREgKnATmPMB1e1Vcr0sRfpJ5+VUgqAQJ9AXr/9dW6rG8L8Z9oRXuPvo8eLdpyk6/ifmbj2WwsTFg+OXj7aS0RigTbAfBFZaJ9fWUSuXAEUATwEdMzhMtExIrJNRLYCHYB/OJJHKVV8hfh5883g1gyKqAmAIY1N8aMYurAv9341hJQ0HavoRklRPM4WHh5uIiMjb2zh2bOhYsXr91PKFZ04AX37Wp3Ccr9sOcajP47grNt3GfOq+jZlw5DfqVi2vIXJXJuIbDTGZLvnS+8sVkoVOU1qpHDR4+cs887EufHUzB0cv5BgUaqiSwuBUqrIqRFQgz8e+YMqZasC4G4CKZ/8PJuPxNFDLzHNNy0ESqkiqU21NkQN2czdde/mueaf4eWWfiL57KVkBkxZx+d/6CWmeXXdy0eVUspVlfctz7x+8wDo3egsw7/ezJn4JGwGRv8v/RLT4Xf40zikjsVJXZvuESilioXWtYKY/0zbLJeY/rxzAU0+r88z8/9Jmi3NwnSuTQuBUqrYuHKJ6aMRoaTKac54vQ/Y+CTyP4R93pFTl05ZHdElaSFQShUrnu5uvN6jIf5VPscmFzPmbzm9gps+bszS/X9YmM41aSFQShU7IsJHd71NoHfWewriUk7R+ctOvLL4TT1UlIkWAqVUsdSldhe2Dd3CrVXbZplvSGP06lHcOqUTx+OOW5TOtWghUEoVW5XLVuaPR5fxcsTL2drWH/+D+hOasDBmoQXJXIsWAqVUsebh5sHozqOZ338+/qWCsrRdTD5D11ldeWHRSyV6rCItBEqpEqFbnW5sH7qFNlVuy9b2/pqxtJnaloPnDxZ+MBeghUApVWJU8avCykFLea3dG8hVf/42Hl9P44lN+WHHDxals44WAqVUieLu5s7bHUex9OElBHiFZGmLT77I/d/dz9D5Q0lMLTnPR9ZCoJQqkdrXbM/eZ6O5tfId2domRk6k5eRW7Dqzy4JkhU8LgVKqxCrvW55Vj//G623fQ64aem3bqa00/28Y06OmF/vB67QQKKVKNBHhzU4vseKRlfh5Vs3SlpB6mUfnPsrDPz9MXFKcRQkLnhYCpZQC2tZozeHnomlR4e5sbV9t/YqwSWFsPr7ZgmQFTwuBUkrZ+Xv7s27IXF5q+RFCqSxte8/tpfXU1ny24TOL0hUcLQRKKZWJiPDeXc+yqP+flHGrmaUtOS2ZE3F/WZSs4DhUCESkj4hsFxGbiGR7IHKmfgdFZJuIRIlIZKb5gSKyWET22t/L5fYdSilVmDrXCePgc1to6HdfxjzvtOYsXNOSbbEXLEzmfI7uEUQDvYEVeejbwRjTzBiTuWCMBJYYY+oAS+yflVLKJQSVLkv0iO94vOF4PG1VKZ/8HCfjkunz39X8suWY1fGcxqFCYIzZaYzZ7cBX9ARm2KdnAPc6kkcppZxNRJjc52l+67eBAPuw1okpNp7+ZjPjFu7GZjOkpKUU6ZFMC+scgQEWichGERmcaX6IMeY4gP29QiHlUUqpfOlYvyJzh0VQq3zpjHkTlsUw5KuNvLT4FZp83oT/7f2fhQlv3HULgYj8LiLRObx65mM9EcaY5sBdwDARyT7q0/VzDBaRSBGJPH36dH4XV0oph9UKLsNPwyK4rW5wxry5uxbw0br3OXP5DN2+7sYLi14gOS3ZwpT5d91CYIzpbIxplMNrbl5XYow5Zn8/BfwEtLQ3nRSRSgD291wfKGqMmWSMCTfGhAcHB+fWTSmlCpS/jydfDAzn8bY1SeM8Z7w+yNL+7fZvi9zNZwV+aEhESotI2SvTwB2kn2QGmAcMtE8PBPJcXJRSyioe7m681qMhY++LoFxaTzD2P6XGjUENPiTIN+jaX+BiHL18tJeIxAJtgPkistA+v7KILLB3CwFWicgWYD0w3xjzm71tNNBFRPYCXeyflVKqSOjbIpSFj39IPfdxuNvK45/ajxnLfXhjbjSpaTar4+WZFMXBlMLDw01kZOT1O+Zk9myoWNG5gZQqLCdOQN++VqdQVzl6PoFHpi1jz8lUBHcAIm4K4tP+zQnw9QIgMTURbw9vK2MiIhuvuoQf0DuLlVLKYVUCfJg77A66N66SMe/PmLPc++mfxJyK48iFI9w0/iYmb5zskiOZaiFQSikn8PXyYEK/5ozoXCdj3sGzl7n305V0/6oPR+OOMvjXwfT9oS9/JbjWMBVaCJRSyknc3IQRnesy8f+a4+OZfogoNu1Ltp1Zl9FnzvY5NP28KcsPLrcoZXZaCJRSysnualyJ759qQ5UAHzxMZcRkPTdw5OIROs7oyEuLXyIpNcmilH/TQqCUUgXg5sr+zB0eQfuq91Ep6SM8bbWytBsMY1ePpfXU1uw8vdOilOm0ECilVAEpX6YUs55oxf+FtaZS0vv4pfQGI1n6RJ2Iovmk5ny6/lPLTiRrIVBKqQJUysOd9+5rwr+6NyEwbRAhyf/G3VY+S5/E1ESG/2843b/uzon4E4WeUQuBUkoVMBHh8Xa1mPJwOIEet1ApaQK+qe2y9ftfzP9oPLEx83bPK9R8WgiUUqqQdGoQwvdDbqWqf3nKp7xEUPJziPHJ0ufM5TP0nN2TJ395kkvJlwollxYCpZQqRA0r+zF3WARNqwZQJq0jlZMmUCqtYbZ+kzZNovmk5kQeu8FRFPJBC4FSShWyCn7efDu4DXc1qoiHCSEk+V0CUh7OGJ7iij1n99Bmahv+veLfpNnSCiyPFgKllLKAj5c7n/ZvzlPtayO445/6ACGJYylFlSz9Um2pvLbsNdrPaM/B8wcLJIsWAqWUsoibm/By1/qMub8Jnu5CKVOXCgkf42+7K1vfVYdX0WRikwI5kayFQCmlLPZAeDW+fKwVAb6euOFNQNIwgpP+RRnPcln6JaYmUs2vmtPXr4VAKaVcQOtaQfw0NIKa9mci+9paEXBxPKGlIzL6vNPxHW6pdIvT162FQCmlXETN8qX5aeittK4VCIA75bCdGcktZZ+nU+gdPN/m+QJZrxYCpZRyIQG+Xswc1Io+YVUBEIRzpzqQcvJlDp1NKJB1aiFQSikX4+Xhxpj7m/By1/oZ8w6dS6DXZ6tZve+M09enhUAppVyQiPBU+9p8PqA53p7pf6ovJKTw8NT1/Bbt3PGItBAopZQL69qoEnOebEOFsqUACCrjRdNq/k5dhxYCpZRycU2qBvDzsAjCapRjysMtqOTvc/2F8sGhQiAifURku4jYRCQ8lz71RCQq0+uiiIywt40SkaOZ2ro5kkcppYqrygE+fD+kDY2rOndvAMDDweWjgd7Af3PrYIzZDTQDEBF34CjwU6YuHxpjxjmYQymlij0RuX6nG+BQITDG7IR8hesE7DPGHHJkvUoppZynsM8R9AW+uWrecBHZKiJfiEi5nBYCEJHBIhIpIpGnT58u2JRKKVWCXLcQiMjvIhKdw6tnflYkIl7APcB3mWZPBGqTfujoOPB+bssbYyYZY8KNMeHBwcH5WbVSSqlruO6hIWNMZyet6y5gkzHmZKbvzpgWkcnAr05al1JKqTwqzEND/bjqsJCIVMr0sRfpJ5+VUkoVIkcvH+0lIrFAG2C+iCy0z68sIgsy9fMFugA/XvUVY0Rkm4hsBToA/3Akj1JKqfwTY4zVGfJNRE4DrnrlUXnA+YOBOI/mc4zmc4zmc5wjGWsYY7KdZC2ShcCViUikMSbHm+tcgeZzjOZzjOZzXEFk1CEmlFKqhNNCoJRSJZwWAuebZHWA69B8jtF8jtF8jnN6Rj1HoJRSJZzuESilVAmnhUAppUo4LQROJCIH7TfIRYlIpAvk+UJETolIdKZ5gSKyWET22t9zHejPonwu84wKEakmIstEZKf9uRvP2ue7xDa8Rj6X2IYi4i0i60Vkiz3fm/b5rrL9csvnEtsvU053EdksIr/aPzt9++k5AicSkYNAuDHGJW5IEZHbgHhgpjGmkX3eGOCcMWa0iIwEyhljXnahfKOAeFd4RoV9CJRKxphNIlIW2AjcCzyCC2zDa+R7ABfYhpI+Pn1pY0y8iHgCq4BnSX+GiStsv9zydcUFtt8VIvIcEA74GWN6FMTvsO4RFGPGmBXAuatm9wRm2KdnkP6HwxK55HMZxpjjxphN9uk4YCdQBRfZhtfI5xJMunj7R0/7y+A62y+3fC5DRKoC3YEpmWY7fftpIXAuAywSkY0iMtjqMLkIMcYch/Q/JEAFi/PkJE/PqChMIhIK3AKswwW34VX5wEW2of2wRhRwClhsjHGp7ZdLPnCR7Qd8BLwE2DLNc/r200LgXBHGmOakD7k9zH7oQ+VPnp9RUVhEpAzwAzDCGHPR6jxXyyGfy2xDY0yaMaYZUBVoKSKNrMqSk1zyucT2E5EewCljzMaCXpcWAicyxhyzv58i/bnMLa1NlKOT9mPLV44xn7I4TxbGmJP2X04bMBmLt6H92PEPwCxjzJXRc11mG+aUz9W2oT3TeWA56cffXWb7XZE5nwttvwjgHvu5x9lARxH5igLYfloInERESttP2CEipYE7cM3nK8wDBtqnBwJzLcySjbjQMyrsJxOnAjuNMR9kanKJbZhbPlfZhiISLCIB9mkfoDOwC9fZfjnmc5XtZ4x5xRhT1RgTSvpjfpcaYwZQANtPrxpyEhGpRfpeAKQ/+e1rY8y/LYyEiHwDtCd92NqTwBvAz8AcoDpwGOhjjLHkhG0u+dqTvktugIPAk1eOh1qQry2wEtjG38doXyX9OLzl2/Aa+frhAttQRJqQfjLTnfR/dM4xxrwlIkG4xvbLLd+XuMD2y0xE2gMv2K8acvr200KglFIlnB4aUkqpEk4LgVJKlXBaCJRSqoTTQqCUUiWcFgKllCrhtBAopVQJp4VAKaVKuP8HSM2BwQ0myYQAAAAASUVORK5CYII=\n",
+      "text/plain": [
+       "
    "
+      ]
+     },
+     "metadata": {
+      "filenames": {
+       "image/png": "/Users/hjensen/Teaching/FYS-STK4150/doc/src/LectureNotes/_build/jupyter_execute/chapter10_7_413.png"
+      },
+      "needs_background": "light"
+     },
+     "output_type": "display_data"
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Time:  4.521348709999998\n"
+     ]
+    }
+   ],
+   "source": [
+    "def test_rnn (x1, y_test, plot_min, plot_max):\n",
+    "    \"\"\"\n",
+    "        Inputs:\n",
+    "            x1 (a list or numpy array): The complete x component of the data set\n",
+    "            y_test (a list or numpy array): The complete y component of the data set\n",
+    "            plot_min (an int or float): the smallest x value used in the training data\n",
+    "            plot_max (an int or float): the largest x valye used in the training data\n",
+    "        Returns:\n",
+    "            None.\n",
+    "        Uses a trained recurrent neural network model to predict future points in the \n",
+    "        series.  Computes the MSE of the predicted data set from the true data set, saves\n",
+    "        the predicted data set to a csv file, and plots the predicted and true data sets w\n",
+    "        while also displaying the data range used for training.\n",
+    "    \"\"\"\n",
+    "    # Add the training data as the first dim points in the predicted data array as these\n",
+    "    # are known values.\n",
+    "    y_pred = y_test[:dim].tolist()\n",
+    "    # Generate the first input to the trained recurrent neural network using the last two \n",
+    "    # points of the training data.  Based on how the network was trained this means that it\n",
+    "    # will predict the first point in the data set after the training data.  All of the \n",
+    "    # brackets are necessary for Tensorflow.\n",
+    "    next_input = np.array([[[y_test[dim-2]], [y_test[dim-1]]]])\n",
+    "    # Save the very last point in the training data set.  This will be used later.\n",
+    "    last = [y_test[dim-1]]\n",
+    "\n",
+    "    # Iterate until the complete data set is created.\n",
+    "    for i in range (dim, len(y_test)):\n",
+    "        # Predict the next point in the data set using the previous two points.\n",
+    "        next = model.predict(next_input)\n",
+    "        # Append just the number of the predicted data set\n",
+    "        y_pred.append(next[0][0])\n",
+    "        # Create the input that will be used to predict the next data point in the data set.\n",
+    "        next_input = np.array([[last, next[0]]], dtype=np.float64)\n",
+    "        last = next\n",
+    "\n",
+    "    # Print the mean squared error between the known data set and the predicted data set.\n",
+    "    print('MSE: ', np.square(np.subtract(y_test, y_pred)).mean())\n",
+    "    # Save the predicted data set as a csv file for later use\n",
+    "    name = datatype + 'Predicted'+str(dim)+'.csv'\n",
+    "    np.savetxt(name, y_pred, delimiter=',')\n",
+    "    # Plot the known data set and the predicted data set.  The red box represents the region that was used\n",
+    "    # for the training data.\n",
+    "    fig, ax = plt.subplots()\n",
+    "    ax.plot(x1, y_test, label=\"true\", linewidth=3)\n",
+    "    ax.plot(x1, y_pred, 'g-.',label=\"predicted\", linewidth=4)\n",
+    "    ax.legend()\n",
+    "    # Created a red region to represent the points used in the training data.\n",
+    "    ax.axvspan(plot_min, plot_max, alpha=0.25, color='red')\n",
+    "    plt.show()\n",
+    "\n",
+    "# Check to make sure the data set is complete\n",
+    "assert len(X_tot) == len(y_tot)\n",
+    "\n",
+    "# This is the number of points that will be used in as the training data\n",
+    "dim=12\n",
+    "\n",
+    "# Separate the training data from the whole data set\n",
+    "X_train = X_tot[:dim]\n",
+    "y_train = y_tot[:dim]\n",
+    "\n",
+    "\n",
+    "# Generate the training data for the RNN, using a sequence of 2\n",
+    "rnn_input, rnn_training = format_data(y_train, 2)\n",
+    "\n",
+    "\n",
+    "# Create a recurrent neural network in Keras and produce a summary of the \n",
+    "# machine learning model\n",
+    "model = rnn(length_of_sequences = rnn_input.shape[1])\n",
+    "model.summary()\n",
+    "\n",
+    "# Start the timer.  Want to time training+testing\n",
+    "start = timer()\n",
+    "# Fit the model using the training data genenerated above using 150 training iterations and a 5%\n",
+    "# validation split.  Setting verbose to True prints information about each training iteration.\n",
+    "hist = model.fit(rnn_input, rnn_training, batch_size=None, epochs=150, \n",
+    "                 verbose=True,validation_split=0.05)\n",
+    "\n",
+    "for label in [\"loss\",\"val_loss\"]:\n",
+    "    plt.plot(hist.history[label],label=label)\n",
+    "\n",
+    "plt.ylabel(\"loss\")\n",
+    "plt.xlabel(\"epoch\")\n",
+    "plt.title(\"The final validation loss: {}\".format(hist.history[\"val_loss\"][-1]))\n",
+    "plt.legend()\n",
+    "plt.show()\n",
+    "\n",
+    "# Use the trained neural network to predict more points of the data set\n",
+    "test_rnn(X_tot, y_tot, X_tot[0], X_tot[dim-1])\n",
+    "# Stop the timer and calculate the total time needed.\n",
+    "end = timer()\n",
+    "print('Time: ', end-start)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Other Things to Try\n",
+    "\n",
+    "\n",
+    "Changing the size of the recurrent neural network and its parameters\n",
+    "can drastically change the results you get from the model.  The below\n",
+    "code takes the simple recurrent neural network from above and adds a\n",
+    "second hidden layer, changes the number of neurons in the hidden\n",
+    "layer, and explicitly declares the activation function of the hidden\n",
+    "layers to be a sigmoid function.  The loss function and optimizer can\n",
+    "also be changed but are kept the same as the above network.  These\n",
+    "parameters can be tuned to provide the optimal result from the\n",
+    "network.  For some ideas on how to improve the performance of a\n",
+    "[recurrent neural network](https://danijar.com/tips-for-training-recurrent-neural-networks)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Model: \"functional_3\"\n",
+      "_________________________________________________________________\n",
+      "Layer (type)                 Output Shape              Param #   \n",
+      "=================================================================\n",
+      "input_2 (InputLayer)         [(None, 2, 1)]            0         \n",
+      "_________________________________________________________________\n",
+      "RNN1 (SimpleRNN)             (None, 2, 500)            251000    \n",
+      "_________________________________________________________________\n",
+      "RNN2 (SimpleRNN)             (None, 500)               500500    \n",
+      "_________________________________________________________________\n",
+      "dense (Dense)                (None, 1)                 501       \n",
+      "=================================================================\n",
+      "Total params: 752,001\n",
+      "Trainable params: 752,001\n",
+      "Non-trainable params: 0\n",
+      "_________________________________________________________________\n",
+      "Epoch 1/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.4176"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 227ms/step - loss: 1.4176 - val_loss: 3.6488\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 2/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 5.4802"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 5.4802 - val_loss: 0.5738\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 3/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.4330"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 1.4330 - val_loss: 0.8091\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 4/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.2697"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.2697 - val_loss: 3.5191\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 5/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 2.1554"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 2.1554 - val_loss: 3.2451\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 6/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.9444"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 1.9444 - val_loss: 1.2933\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 7/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.5533"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.5533 - val_loss: 0.1028\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 8/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0521"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0521 - val_loss: 0.1031\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 9/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.5942"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.5942 - val_loss: 0.3703\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 10/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1041"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 1.1041 - val_loss: 0.2869\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 11/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.9592"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.9592 - val_loss: 0.0406\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 12/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.4307"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.4307 - val_loss: 0.0672\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 13/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0683"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0683 - val_loss: 0.5012\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 14/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.1235"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.1235 - val_loss: 1.0556\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 15/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.4083"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 0.4083 - val_loss: 1.3256\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 16/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.5737"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.5737 - val_loss: 1.1606\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 17/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.4712"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.4712 - val_loss: 0.7278\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 18/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.2276"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.2276 - val_loss: 0.3054\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 19/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0589"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0589 - val_loss: 0.0652\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 20/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0695"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0695 - val_loss: 8.7287e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 21/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.1958"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.1958 - val_loss: 0.0067\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 22/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.2957"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.2957 - val_loss: 0.0046\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 23/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.2815"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 20ms/step - loss: 0.2815 - val_loss: 0.0030\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 24/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.1768"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 20ms/step - loss: 0.1768 - val_loss: 0.0605\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 25/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0727"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0727 - val_loss: 0.2095\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 26/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0430"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0430 - val_loss: 0.4103\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 27/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0896"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 0.0896 - val_loss: 0.5719\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 28/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.1536"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.1536 - val_loss: 0.6165\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 29/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.1739"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.1739 - val_loss: 0.5322\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 30/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.1365"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 0.1365 - val_loss: 0.3724\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 31/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0773"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0773 - val_loss: 0.2118\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 32/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0432"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0432 - val_loss: 0.0999\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 33/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0528"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0528 - val_loss: 0.0440\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 34/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0865"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 0.0865 - val_loss: 0.0261\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 35/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.1092"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.1092 - val_loss: 0.0315\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 36/150"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.1012"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.1012 - val_loss: 0.0612\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 37/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0719"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 0.0719 - val_loss: 0.1231\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 38/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0466"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0466 - val_loss: 0.2126\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 39/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0432"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0432 - val_loss: 0.3043\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 40/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0587"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0587 - val_loss: 0.3636\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 41/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0747"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0747 - val_loss: 0.3676\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 42/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0759"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0759 - val_loss: 0.3193\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 43/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0623"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0623 - val_loss: 0.2431\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 44/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0468"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0468 - val_loss: 0.1684\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 45/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0417"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0417 - val_loss: 0.1143\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 46/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0485"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0485 - val_loss: 0.0857\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 47/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0580"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0580 - val_loss: 0.0803\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 48/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0605"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 0.0605 - val_loss: 0.0955\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 49/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0540"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 0.0540 - val_loss: 0.1299\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 50/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0452"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 0.0452 - val_loss: 0.1782\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 51/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0415"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0415 - val_loss: 0.2284\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 52/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0448"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0448 - val_loss: 0.2639\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 53/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0502"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0502 - val_loss: 0.2725\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 54/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0518"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0518 - val_loss: 0.2531\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 55/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0483"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0483 - val_loss: 0.2155\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 56/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0434"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 0.0434 - val_loss: 0.1742\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 57/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0415"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0415 - val_loss: 0.1411\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 58/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0435"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0435 - val_loss: 0.1224\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 59/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0464"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0464 - val_loss: 0.1194\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 60/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0470"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0470 - val_loss: 0.1310\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 61/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0448"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0448 - val_loss: 0.1544\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 62/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0422"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 0.0422 - val_loss: 0.1836\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 63/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0415"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 0.0415 - val_loss: 0.2101\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 64/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0429"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0429 - val_loss: 0.2253\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 65/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0444"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0444 - val_loss: 0.2246\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 66/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0443"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0443 - val_loss: 0.2096\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 67/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0428"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0428 - val_loss: 0.1869\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 68/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0415"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0415 - val_loss: 0.1645\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 69/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0415"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0415 - val_loss: 0.1485\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 70/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0425"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 20ms/step - loss: 0.0425 - val_loss: 0.1423\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 71/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0431"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 0.0431 - val_loss: 0.1461\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 72/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0427"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0427 - val_loss: 0.1584\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 73/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0418"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 0.0418 - val_loss: 0.1752\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 74/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0413"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 20ms/step - loss: 0.0413 - val_loss: 0.1915\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 75/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0416"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 0.0416 - val_loss: 0.2018\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 76/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0421"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0421 - val_loss: 0.2030\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 77/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0422"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 0.0422 - val_loss: 0.1955\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 78/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0418"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0418 - val_loss: 0.1827\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 79/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0413"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 21ms/step - loss: 0.0413 - val_loss: 0.1693\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 80/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0413"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0413 - val_loss: 0.1595\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 81/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0416"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 0.0416 - val_loss: 0.1557\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 82/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0418"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0418 - val_loss: 0.1584\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 83/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0416"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0416 - val_loss: 0.1662\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 84/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0413"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0413 - val_loss: 0.1764\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 85/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0411"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0411 - val_loss: 0.1856\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 86/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0413"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0413 - val_loss: 0.1906\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 87/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0414"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0414 - val_loss: 0.1899\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 88/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0414"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0414 - val_loss: 0.1844\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 89/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0412"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 20ms/step - loss: 0.0412 - val_loss: 0.1764\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 90/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0411"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0411 - val_loss: 0.1688\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 91/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0411"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0411 - val_loss: 0.1641\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 92/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0412"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0412 - val_loss: 0.1634\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 93/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0412"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0412 - val_loss: 0.1666\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 94/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0411"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0411 - val_loss: 0.1723\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 95/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0410"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0410 - val_loss: 0.1783\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 96/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0410"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0410 - val_loss: 0.1826\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 97/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0411"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0411 - val_loss: 0.1836\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 98/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0411"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0411 - val_loss: 0.1814\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 99/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0410"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0410 - val_loss: 0.1769\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 100/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0409"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 0.0409 - val_loss: 0.1720\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 101/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0409"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 20ms/step - loss: 0.0409 - val_loss: 0.1684\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 102/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0409"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0409 - val_loss: 0.1673\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 103/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0409"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0409 - val_loss: 0.1687\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 104/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0409"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0409 - val_loss: 0.1719\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 105/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0409"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 0.0409 - val_loss: 0.1756\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 106/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0408"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0408 - val_loss: 0.1784\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 107/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0408"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 20ms/step - loss: 0.0408 - val_loss: 0.1793\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 108/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0408"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 20ms/step - loss: 0.0408 - val_loss: 0.1781\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 109/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0408"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 0.0408 - val_loss: 0.1754\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 110/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0408"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0408 - val_loss: 0.1723\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 111/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0408"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 21ms/step - loss: 0.0408 - val_loss: 0.1701\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 112/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0408"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0408 - val_loss: 0.1693\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 113/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0407"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0407 - val_loss: 0.1702\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 114/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0407"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0407 - val_loss: 0.1722\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 115/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0407"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0407 - val_loss: 0.1745\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 116/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0407"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0407 - val_loss: 0.1760\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 117/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0407"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0407 - val_loss: 0.1764\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 118/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0407"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0407 - val_loss: 0.1754\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 119/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0406"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0406 - val_loss: 0.1736\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 120/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0406"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 0.0406 - val_loss: 0.1718\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 121/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0406"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0406 - val_loss: 0.1705\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 122/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0406"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0406 - val_loss: 0.1703\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 123/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0406"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0406 - val_loss: 0.1711\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 124/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0405"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0405 - val_loss: 0.1724\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 125/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0405"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0405 - val_loss: 0.1737\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 126/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0405"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0405 - val_loss: 0.1744\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 127/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0405"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0405 - val_loss: 0.1742\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 128/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0405"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 20ms/step - loss: 0.0405 - val_loss: 0.1734\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 129/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0405"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0405 - val_loss: 0.1721\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 130/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0404"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0404 - val_loss: 0.1711\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 131/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0404"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0404 - val_loss: 0.1706\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 132/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0404"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0404 - val_loss: 0.1707\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 133/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0404"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 0.0404 - val_loss: 0.1714\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 134/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0404"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0404 - val_loss: 0.1722\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 135/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0403"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 20ms/step - loss: 0.0403 - val_loss: 0.1728\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 136/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0403"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0403 - val_loss: 0.1729\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 137/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0403"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0403 - val_loss: 0.1725\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 138/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0403"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0403 - val_loss: 0.1717\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 139/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0403"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0403 - val_loss: 0.1710\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 140/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0402"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0402 - val_loss: 0.1705\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 141/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0402"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0402 - val_loss: 0.1704\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 142/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0402"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0402 - val_loss: 0.1707\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 143/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0402"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0402 - val_loss: 0.1712\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 144/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0402"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0402 - val_loss: 0.1716\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 145/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0402"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0402 - val_loss: 0.1717\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 146/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0401"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 0.0401 - val_loss: 0.1715\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 147/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0401"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0401 - val_loss: 0.1710\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 148/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0401"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0401 - val_loss: 0.1705\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 149/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0401"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0401 - val_loss: 0.1701\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 150/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0401"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 20ms/step - loss: 0.0401 - val_loss: 0.1699\n"
+     ]
+    },
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXgAAAEWCAYAAABsY4yMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO2deZgcVbn/P28vs2Rmsk/2kI0lIQlr2FTCvigIKsiOiCg/5YrCFUREBNGrKPfq9d6LLIIsEiAIqMiObGEni9mAEEhIwmSdbDOTZWa6u97fH+f0TE3PTDKTTGcmNe/nefrp7jpV57xVXf2tt97z1jmiqhiGYRjRI9bZBhiGYRj5wQTeMAwjopjAG4ZhRBQTeMMwjIhiAm8YhhFRTOANwzAiSrcXeBG5UUQe6KC6ikXkHyJSJSJ/EZHzReT5Dqp7iYgc3xF15dSrIrKn/3y7iFzflnV3oJ0OOxY59R4tIhUdXa9hRIHIC7yIbAq9AhHZGvp+fgc3dyYwEOinql9V1SmqemIHt5E3VPXbqvrzna1HREb6i0EiVPdudSzait/Xl0Vki4gs2NZFWESO8etWiciSVtb5voh8IiKbReQDEdnbLxcRuU5ElolItYg8LCI9Q9sNFZG/i8h6EakQkW+HyvqLyBsisk5ENorIWyLy2Zx2R4vIkyJSIyJrReQ3obIHRGSlb3ehiHwzZ/815392fai80DsOq71t/xCRoaHyz4jIu77duSLyuVDZKSLyurd5lYj8UUTKQuV9RWSqt3etiEzJOSZxEfmFiKzw9f9LRHqHjucvRGS5/z1eEZHxOXX/1f8OS0XkvFZ+rxv8/h8fWna1iMz3bX4iIle3tO0uQVW7zQtYAhyfs+xG4IEOqv8nHVVXW2zvoHoV2LMj1wVG+nUTu+A3PRqo2FXnUAvtvwX8FigGzgA2AuWtrHsocCFwKbCkhfJvAnOBfQEBxgB9fdlFwAJgOFAK/B24L7Tty8B/A0lgf2A9cIwvKwL2wTl0AnzJlyd8eQGwCPh3oMSvv1+o7vFAof88FlgFHNyW3xr4ITAH5/gUAX8GHvdlfYG1wFeBOHABsAHo48vPA04GegB9gGeA20N1/wF4HugJ9AL+Cfw2VP4L4CVghN/vCUCRLzsLWAGM9m3/CpgV2vYhYKo/1p8DqoDxOfs2Bpjn6zk+Z58PAhL+uC8FzumU87Oz/hid9GdcQssC/whwP1ADvAdMCpUPAR4DKoFPgO+1UvfPgHogBWwCLgG+DrweWkeBbwMf+RP5VkBCJ8tLwDp/0k8Bem/Ldr/8cP+Hi4eWfRmY6z8fihOhjcBK4P+Aghyb9vSf7wV+ESq72m+zAvhGzrqnAP8CqoFPgRtD2y3z627yryNaOBafAab7P8504DOhsleAnwNv+N/keaB/K8f9aEICD4zz22/0v+VpobIvAO/7OpcDV/nl/YEn/TbrgdeAWBvOp72BOqAstOw14Nvb2e54cgQeJ76fAse1ss2jwNU5x68WJ36l/niXh8rvBP7cQj0x4It+/QF+2aXAa238D+3jz4mz/PeRbFvgbwN+E/p+CvCh/3wq8F7O+guBS1qp6yvAvND3Z4DLQt//DXjOf+7jz70xrdR1DfBI6Pt4oNZ/LsH9l/cOlf8ZuDmnjmf8ObWEbThfwP8A/9uW49vRr8iHaNrIacDDQG/gCZwIIiIx4B84D2QocBxwhYiclFuBqt4A/BKYqqqlqnp3K22dChyC87LOArJ1Cc6LGIITqeG4i882UdW3gc3AsaHF5wEP+s8Z4EqciB3h9+Gy7dUrIicDVwEnAHvhRCnMZuBruGN2CvAdEfmSL5vs33v7Y/FWTt19gadwJ34/nAf8lIj0y9mHi4EBOA/zqjbYnMT9Xs/77S4HpojIPn6Vu4H/p6plOG/uJb/8B0AFUI7zNH+MEy1E5A8i8odWmhwPLFbVmtCyOX55exnmXxNE5FN/a/8zfw6COz8kvLtAIe63kdCycPmEcAMiMhd3UXgCuEtV1/iiw4ElIvKMD3W8IiITc7b9g4hswd1FrASezrF/qQ8N3SMi/UPL7wY+KyJDRKQHcD5OGFvapxbtDjEZd9HOcitwqoj0EZE+uDuobN0TgTRwpg/vLBSRfwtt+zCwp4js7c+bi4BnfdneQEZVF4bWb/K7ishXgXpVzT0OTXdGRIAjc+zeZZjAO15X1adVNYO7Uu/vlx+C84puUtV6VV0M/BE4ZyfaullVN6rqMtxt9QEAqvqxqr6gqnWqWokTvaPaWOdDwLkAPkb5Bb8MVZ2pqm+ralpVlwB3tLHes4B7VHW+qm4m52Kjqq+o6jxVDVR1rm+vrfaeAnykqn/2dj2EE44vhta5R1UXqupW3B3WAW2o93CcN3uz/71ewnnm5/ryFLCviPRU1Q2qOiu0fDAwQlVTqvqaetdLVS9T1dYuiKW4O5AwVUBZC+tuj2H+/UScOB3j7b7EL38G+KaPeffCeaAAPfwF5g3gehEpEpGDcGLXI9yAqu6HC2ecB7ye0/Y5uAvuENzF9+8iUhDa9jK/X0cCj+PuXMDdbR6CC4Mc7NeZEqp7Ie6Objnubm8ccJMvexMYIiLnikhSRC7C3ck2sRtARE7AifBPQ4tn4S7+6/wrgwvbZPepF06sR+H6x2709YC7SL0GfAhsxYWJrvRl2/xdRaQU58xdkWtnC9yI09l72rBuh2MC71gV+rwFKBLXQTgCdwJuzL5w3t3ADmyrFEBEBojrOFsuItXAAzivuy08CHxFRApxt7GzVHWpr3dv33m2ytf7yzbWOwQXMsiyNFwoIoeJ6zCsFJEqXOiprfYOya3Pfx8a+t7icWqLzaoatFLvGbiL31IReVVEjvDLbwE+Bp4XkcUi8qO27QabcIIZpicuBNRetvr333gHYAnuYvwFv/xPuIvoKzhv8GW/PJtBdD5OyD7FhUWmhMoaUNVaf0H9kYhkHZmtOCfnGVWtB/4Td2c1LmfbjKq+jhPP7/hlm1R1hr9Qrwa+C5wY6uy8DRd774cLfTyO97JVdR1wOi72vxoXb/9nrt0icjjuHD8zx6v+C+4CUoY77otw/5vw8bxJVbd6J+Th0PG8AXdhGu7t+xnwkr/L2N7v+jNc+OuT3OObY/d3cXe5p6hq3bbWzRcm8NvmU+ATVe0depWp6he2u2X7+RUuLLCfqvbEdTjl3r62iKq+jxOyz9M0PAPuD7YA2MvX++M21rsSd/Jn2SOn/EHcrf5wVe0F3B6qd3tDlK7AXTzD7IHz8naGFcDwUFijSb2qOl1VT8eFb/6GuzNAVWtU9QeqOhp3F/HvInJcG9p7DxgdzuzA3f3tyO34h7i4b4vHzt8p3aCqI1V1mG9jeWjflqrqqaparqqH4QT13W20l8R1MILr2G3PsLIJnKfdoqn+PXsu7A/cq6rrvcj9L3BoNoyjqq+q6iGq2hfXAb1P2G4RORB3nn1DVV/MaWt/4A5V3ayqm3DnYPa/OTfHnlz2x4VTK/zF6V5c3H5f3EUjISJ75ayf/V2PA77nnaZVuP/JIyKSvatCRL4B/AjXp9Jpabwm8NvmXaBaRK4Rl+MeF5EJInJIHtoqw3kOG8WlkbU3tepB4Hu4OOVfcuqtBjaJyFi859UGHgG+LiL7eq/mhhbsXa+qtSJyKO7CkqUSCGgUkFyeBvYWkfNEJCEiZ+P+WE+20bbWeAfXN/BDf8t/NE6wHxaRAnG5+L1UNYU7JhkAETlVRPb08dLs8sz2GvPe5GzgBh8a+TKwH65TvhkiEhORIpy4it+mwNe1BZe18UMRKRORYcC38MdEXNreGHHsiwvh3ZS9WxGRcX67AhG5ABfq+a0vO1xEPufLir0QDfTHC5zXe7iIHC8icVzoYS3wgb+zPEdESv35fxIudPSSr/swEdnH71s/XJjnFVXNhjimA18TkV4+1n0ZsEJV1/rtD/S/VU/cnUOFqj7nyybg4uKXq+o/Wjik03Fhq2IRKcZ1Fs/xx3MRLgRznbhUzXHA2TSeY9OBr4rIQG/7hf53+diHJB8HbhKREnEppafjwrfgBH4CLmx4AM6x+H+4PgHEpV//EjjBh3U7j231wEbtRRvSJMnJCsDd9j+ECxlsAN7OrWMbdX2d5lk0e4a+34vPWsF14MzEifxsfMfftmzPaXsPnKg+lbN8Ms6D34Q74W9qzSaaZ9H8yO93S1k0Z+LuGmpwf5r/y9n3m3BCvxEXG889Fp/z+1vl3z8XKnsF+GZrxzFn/47OOU7jgVd9ve8DX/bLC3BisQEn4tOzbeJir0twF4cK4PpQfbcTSs1rof2R3t6tOC88nC53JLApx1bNeb0SKu+JCyPU4O4ef0pjltXevv4t/rj/e44dV/jjvRkXXw9ngh2FE74aXJbQq8DknO2/ggtTVfv9Ge+Xl/v1N/qyecC3Qtudi8su24y767sfGBQq74cLF63xdbwOHBoqf8j/VlW4C9yAUNk9uHN6U+j1Xqh8FK5TfZ3fr2dxd6rZ8qF+2SZgMa6DPVtWhBPklX6/ZgEnh8r74u7yNuP6EM5rq67445HNpsu+Wj2H8vnKnjyGYRhGxLAQjWEYRkQxgTcMw4goJvCGYRgRxQTeMAwjoiS2v8quo3///jpy5MjONsMwDGO3YebMmWtVtbylsi4l8CNHjmTGjBmdbYZhGMZug4jkPhXegIVoDMMwIooJvGEYRkQxgTcMw4goXSoGbxhG9yOVSlFRUUFtbW1nm9KlKSoqYtiwYSSTyTZvYwJvGEanUlFRQVlZGSNHjsSN92bkoqqsW7eOiooKRo0a1ebtLERjGEanUltbS79+/Uzct4GI0K9fv3bf5ZjAG4bR6Zi4b58dOUaREvjXPqpk2botnW2GYRhGlyBSAn/l1Dnc/Xrnjq9vGMbuR2lpW2aE3P2IlMDXpzPUZ4Ltr2gYhtENiJTAZwIlnbEJTAzD2DFUlauvvpoJEyYwceJEpk6dCsDKlSuZPHkyBxxwABMmTOC1114jk8nw9a9/vWHd3/3ud51sfXMilSaZUSVjM1QZxm7Lz/7xHu+vqO7QOvcd0pMbvji+Tes+/vjjzJ49mzlz5rB27VoOOeQQJk+ezIMPPshJJ53EddddRyaTYcuWLcyePZvly5czf/58ADZu3NihdncEkfPgM4EJvGEYO8brr7/OueeeSzweZ+DAgRx11FFMnz6dQw45hHvuuYcbb7yRefPmUVZWxujRo1m8eDGXX345zz77LD179uxs85sRKQ8+HShpE3jD2G1pq6edL1qbo3ry5MlMmzaNp556igsvvJCrr76ar33ta8yZM4fnnnuOW2+9lUceeYQ//elPu9jibRMZDz4IFFX3bhiGsSNMnjyZqVOnkslkqKysZNq0aRx66KEsXbqUAQMG8K1vfYtLLrmEWbNmsXbtWoIg4IwzzuDnP/85s2bN6mzzmxEZDz4bezcP3jCMHeXLX/4yb731Fvvvvz8iwm9+8xsGDRrEfffdxy233EIymaS0tJT777+f5cuXc/HFFxMELnPvV7/6VSdb35zoCLwXdvPgDcNoL5s2bQLc06K33HILt9xyS5Pyiy66iIsuuqjZdl3Raw8TmRBNVuDNgzcMw3BER+B9iCawNEnDMAwgSgLvH3CyB50MwzAckRH4bGjG8uANwzAckRH4bGjGnmQ1DMNw5DWLRkSWADVABkir6qR8tZW2TlbDMIwm7Io0yWNUdW2+GwksTdIwDKMJkQnRmAdvGMauYFtjxy9ZsoQJEybsQmu2Tb4FXoHnRWSmiFyaz4YyDZ2sNh68YRgG5D9E81lVXSEiA4AXRGSBqk4Lr+CF/1KAPfbYY4cbylgWjWHs/jzzI1g1r2PrHDQRPn9zq8XXXHMNI0aM4LLLLgPgxhtvRESYNm0aGzZsIJVK8Ytf/ILTTz+9Xc3W1tbyne98hxkzZpBIJPjtb3/LMcccw3vvvcfFF19MfX09QRDw2GOPMWTIEM466ywqKirIZDJcf/31nH322Tu125BngVfVFf59jYj8FTgUmJazzp3AnQCTJk3aYXVOe8/dBN4wjPZwzjnncMUVVzQI/COPPMKzzz7LlVdeSc+ePVm7di2HH344p512Wrsmvr711lsBmDdvHgsWLODEE09k4cKF3H777Xz/+9/n/PPPp76+nkwmw9NPP82QIUN46qmnAKiqquqQfcubwItICRBT1Rr/+UTgpny1l43MWJqkYezGbMPTzhcHHngga9asYcWKFVRWVtKnTx8GDx7MlVdeybRp04jFYixfvpzVq1czaNCgNtf7+uuvc/nllwMwduxYRowYwcKFCzniiCP4j//4DyoqKvjKV77CXnvtxcSJE7nqqqu45pprOPXUUznyyCM7ZN/yGYMfCLwuInOAd4GnVPXZfDXW4MHbk6yGYbSTM888k0cffZSpU6dyzjnnMGXKFCorK5k5cyazZ89m4MCB1NbWtqvO1saWP++883jiiScoLi7mpJNO4qWXXmLvvfdm5syZTJw4kWuvvZabbuoYXzhvHryqLgb2z1f9udiDToZh7CjnnHMO3/rWt1i7di2vvvoqjzzyCAMGDCCZTPLyyy+zdOnSdtc5efJkpkyZwrHHHsvChQtZtmwZ++yzD4sXL2b06NF873vfY/HixcydO5exY8fSt29fLrjgAkpLS7n33ns7ZL8iM1xwdgwai8EbhtFexo8fT01NDUOHDmXw4MGcf/75fPGLX2TSpEkccMABjB07tt11XnbZZXz7299m4sSJJBIJ7r33XgoLC5k6dSoPPPAAyWSSQYMG8dOf/pTp06dz9dVXE4vFSCaT3HbbbR2yX9LabURnMGnSJJ0xY8YObfvmorWc98d36N0jyeyfntjBlhmGkS8++OADxo0b19lm7Ba0dKxEZGZrowRE5kEnS5M0DMNoSnRCNCbwhmHsIubNm8eFF17YZFlhYSHvvPNOJ1nUMpER+MAE3jB2W1S1XTnmnc3EiROZPXv2Lm1zR8LpkQnRmAdvGLsnRUVFrFu3bocErLugqqxbt46ioqJ2bRc5D94GGzOM3Ythw4ZRUVFBZWVlZ5vSpSkqKmLYsGHt2iYyAh8W9iBQYrHd53bPMLozyWSSUaNGdbYZkSQyIZrwZNvmxRuGYURI4MOTbQcWyzMMw4iOwGfMgzcMw2hCdAQ+JOqWSWMYhhEhgU+bwBuGYTQhMgIfnmw7bdP2GYZhREfgm6ZJdqIhhmEYXYTICLx58IZhGE2JjMCbB28YhtGUyAh80wedTOENwzAiI/DhB50si8YwDCNCAp8Jee02L6thGEaUBD4cosmYwBuGYURG4Jt0spoHbxiGER2Bb5omaQJvGIYRGYHPHQ/eMAyjuxMZgTcP3jAMoymREXgbbMwwDKMpkRF4Gy7YMAyjKXkXeBGJi8i/ROTJfLZjAm8YhtGUXeHBfx/4IN+NmMAbhmE0Ja8CLyLDgFOAu/LZDtiUfYZhGLnk24P/b+CHQN5H/7JOVsMwjKbkTeBF5FRgjarO3M56l4rIDBGZUVlZucPtBYESjwlgY9EYhmFAfj34zwKnicgS4GHgWBF5IHclVb1TVSep6qTy8vIdbiwdKAVxtzsZGy7YMAwjfwKvqteq6jBVHQmcA7ykqhfkq71MoBQksgKfr1YMwzB2HyKVB98o8KbwhmEYiV3RiKq+ArySzzYyTUI0+WzJMAxj9yBSHnyhefCGYRgNRErgsyEay4M3DMOIksBr2IM3gTcMw4iMwKebdLKawBuGYURG4DNB0Cjw9qCTYRhGlASexiwam3TbMAwjIgL/7h8ZVz/POlkNwzBCREPgX/gph9W/QyIeQwQCC9EYhmFEROBjSeKaJhETEjExD94wDIOoCHw8QYw0cRFiIk0m4DYMw+iuREPgY0liQYa4efCGYRgNRETgE8TUCXw8Jo158G/fBvMf61zbDMMwOoldMthY3oknSJBuLvDT74bCUphwRufaZxiG0QlExINPEtOswMcaQzRBClbNh3Rd59pnGIbRCURE4BPEyIZoaOxkzaSdyK+e37n2GYZhdALREPh4goRmfJpkyIPP1Lv3Ff/qPNsMwzA6iWgIvM+Dj/kYfMODTkHKvS83gTcMo/sRDYGPJ4njPPh4OE0yk3bvK2Z1nm2GYRidRDQEPpYgrhniks2i8TM6ZepBYlC5AOo3d66NhmEYu5hICLzGEiQkTTwWIy6hNMkgBQPHgwawck7nGmkYhrGLiYzAJ8kQj9GYBx9knLAPP8yttNzCNIZhdC+iIfCSIE7GefBZgc/4DtaeQ6DnUMukMQyj2xENgY8lSOR2smYzaOIFTuA3V3aukYZhGLuYSAh84EM0MT/YWKAhDz6WhERhY068YRhGNyESAq/SmCYZiwnpTEjg4wkn8DZcgWEY3YxICHwQi5OURg8+kxuiiZvAG4bR/YiEwKs0jcFnVBtDMg0hGhN4wzC6F5EQ+MALfOODTtr4FGs8aSEawzC6JXkTeBEpEpF3RWSOiLwnIj/LV1tZDz7eYojGBN4wjO5JPif8qAOOVdVNIpIEXheRZ1T17Y5uKCNxF6KJuzlZnQcfCtFYDN4wjG5I3gReVRXY5L8m/Ssvk6Vmn2SNiZCItxKisRi8YRjdjLzG4EUkLiKzgTXAC6r6TgvrXCoiM0RkRmXljj2MlBE3ZV8iFvLgWwrRqE3GbRhG9yGvAq+qGVU9ABgGHCoiE1pY505VnaSqk8rLy3eonQxx4qLERElkn2TNzaIhlBtvGIbRDdglWTSquhF4BTg5H/UHMRdpyj7N2jRE4/PgwcI0hmF0K9ok8CLyfRHpKY67RWSWiJy4nW3KRaS3/1wMHA8s2HmTm5MRJ/BxybSQReOfZAXraDUMo1vRVg/+G6paDZwIlAMXAzdvZ5vBwMsiMheYjovBP7nDlm4DJQ5AkoB4LNbyg05gAm8YRreirVk04t+/ANyjqnNERLa1garOBQ7cGePaStp78AnSxGM0HS44HKJJ1+4KcwzDMLoEbfXgZ4rI8ziBf05EyoAgf2a1j8B78AnNkIjFSGcCCLIx+FCIxkaUNAyjG9FWD/4S4ABgsapuEZG+uDBNlyAjXuAlTUziBIqFaAzD6Pa01YM/AvhQVTeKyAXAT4Cq/JnVPtL+OhXXgERcSAdB0xCNCbxhGN2Qtgr8bcAWEdkf+CGwFLg/b1a1k0xDDN6NRxMEhAQ+aWmShmF0S9oq8Gk/9MDpwO9V9fdAWf7Mah9pH4OPkyYu3oPPpknGLE3SMIzuSVtj8DUici1wIXCkiMRxY8t0CRo6WQmcB6+gmZRL/bEQjWEY3ZS2evBn40aH/IaqrgKGArfkzap2kvKdrHFNEY+57M0g7TtZwyEaS5M0DKMb0SaB96I+BeglIqcCtaradWLw2U5WH4MH0HQ9IBCLW5qkYRjdkrYOVXAW8C7wVeAs4B0ROTOfhrWHtN+NhLoRJQE0SLnwDFiIxjCMbklbY/DXAYeo6hpw48wA/wQezZdh7SHbyRrzMXjwIZq47yZIFPkVTeANw+g+tDUGH8uKu2ddO7bNO2nN5sGnGwSeTMpl0ECjJ29pkoZhdCPa6sE/KyLPAQ/572cDT+fHpPbT4MGHBD7IWIjGMIzuTZsEXlWvFpEzgM/iBh67U1X/mlfL2kE2Bh/Xxk5WwiGaWAIkZgJvGEa3os1zsqrqY8BjebRlh0lpyIP3g1xqJt0YohHxE29bmqRhGN2HbQq8iNTQ8kTZgptXu2derGon2eGCY7kx+GyIBvzE25YmaRhG92GbAq+qXWY4gm2R0myIJk0invXgQyEaaJx42zAMo5vQZTJhdoZsiEbIEJMWsmjABN4wjG5HJAS+IYsmSJOI+V0KckI08UJLkzQMo1sRCYFv6GQN3JR9gI/Bh0M0RebBG4bRrYiIwGe99jTxrAffLERTYAJvGEa3IhoC70M0BKmGsWhaDNGYwBuG0Y2IhMDX+xANmTQxL/DSLERjMXjDMLoXkRD4VNDYsdrowacti8YwjG5NJAQ+oz5ME6Qb0iQlqG8I0bzx8VrW14kJvGEY3YqICLySIQ6ZVMODTgTphhDNb55dwOwVWy1EYxhGtyIaAp9R0iRyPPjGGPzWVIY1WyGTsrFoDMPoPkRD4FXJEIOgcUanWCYFMSfwdemAepKk603gDcPoPuRN4EVkuIi8LCIfiMh7IvL9fLWVCbwHn2mcdFtCIZr6dEA9CUhZiMYwjO5Dm4cL3gHSwA9UdZaIlAEzReQFVX2/oxvKBEpG4hCEBF4b8+Dr0gF1JIlrPaqKZMerMQzDiDB58+BVdaWqzvKfa4APgKH5aCsTZDtZG0M0EjQ+yVqXyiCJIhJk+HhVVT5MMAzD6HLskhi8iIwEDgTeaaHsUhGZISIzKisrd6j+dBCQEd/Jmo3Bh0M0mYBh5b0BeHPhih1qwzAMY3cj7wIvIqW4maCuUNXq3HJVvVNVJ6nqpPLy8h1qIxNAhkTDg04xAgSFeAGZQElllF6lJQDMXLR6Z3bHMAxjtyGvAi8iSZy4T1HVx/PVTiYICMSFaOIxIUnaFcQS1KcD9zlRDED1pk35MsMwDKNLkc8sGgHuBj5Q1d/mqx1wT7K6TtYcgY8nGwQ+lnQdrpnU1nyaYhiG0WXIpwf/WeBC4FgRme1fX8hHQ5mGGHwqR+ALqEtnAJBEIQCBPexkGEY3IW9pkqr6Om5y7rzj0iR9HrwICZyoE0tQ1+DBFwEQpGzibcMwugfReJI1UNQPNpaIxUhmBT6ebPDgYwVO4NU8eMMwugmREPh0oGRi2TRJSEo4ROM9eN/JqulaVLWzTDUMw9hlRELgg0AJfIgmEYu1GKJJFLoYfJJ0wzLDMIwoEwmBT4dCNK6TNRSiSTkxj/sQTSH11KYynWWqYRjGLiMSAp8JlMCHaHKzaOoz3oP3nawFpNlSbwJvGEb0iY7A+xBNTAg96JSkznvryQIXgy8kxVbz4A3D6AZEQ+A168GnEBEKYj7GHm+MwScLncAXSIqt5sEbhtENiIbAB4pKAjLOcy+KZWPwBQ1PsiYKG0M05sEbhtEdiITApzONMXiAQslm0SQbPPiCwh6ujHrz4A3D6BZEQuAD9R58kAKgQMIhGifmBYXZLBrrZDUMo3sQCUuF12kAAB9SSURBVIFPB4rG4g0hmsJY8wedCot8J6ukLE3SMIxuQSQEPggUjSUbQjQFeA8+1jiaZEEijsYLKLAsGsMwugmREPh0kBOiaehkdSGaeExIxGMQL6SQlIVoDMPoFkRC4Jt48KoUhseiSQUUJvxuJgopwEI0hmF0DyIh8A0xeIAgTWEsFKLJBBSEBL5Q0mypT3eOoYZhGLuQSAj8RZ8ZydC+Pd2XIE2PeDaLJtnEg5dEIT1iabbW22BjhmFEn0gI/I8+P5a9BvdxXzKppgKfzlCY8N59vJDimD3oZBhG9yASAg9AzE9OFaQpivvx3lsI0RRLmq07G6LZtAZe/iWkbXYowzC6LtER+HijwPeINx0uONzJWiQd4MHPeQhe/TV89PzO1WMYhpFHoiPwsaR7z6QoigVkiEEsTl06LPBFFEn9zqdJLp/p3uc/tnP1GIZh5JHoCHzcC3yQoigekMbF3evSmcYQTUEpPajd+TTJ5f9y7wufhfrNO1eXYRhGnoiOwGdj8Jk0RZIhpQlUlfp00NjJWlBCMbU7F6LZtAaqlsHen4fUFifyhmEYXZDoCXyQpiAWkMKFZ5qEaApKKNLanQvRLJ/l3o/4NygbDPMf3zm7DcMw8kR0BD4UoimUDGnibKnPOIFPNnrwRcFWandG4FfMAonB0INg/JddR2tt1c7bbxiG0cFER+AbQjQpCiRDPQk216WpTwcUxBtj8AVaS239TqQ3Lp8J5eOgoAT2PA4y9bBq/s7bbxiG0cFESOCzHnyGAsmQ1jib69PuQaek383CUvee2tJiFXMrNvLnt5a03oaqE/ihB7nvfUe79w3b2MYwDKOTiI7AN+TBp0j6EM3mukzTPPiCEgAS6a0EgTar4v63lnL9399j6bpWMmM2fAJbN8DQg933XsNduMYE3jCMLkjeBF5E/iQia0Rk18QvQiGaJOmGEE1d+EnWAufBl0jLmTSrqmoBeGTGpy23scKnR2Y9+HgSeg0zgTcMo0uSTw/+XuDkPNbflFhjJ2uCrAefbpYmCdCjlVTJFVVbAfjLjArSmRYGJMsKeb+9Gpf1GWkCbxhGlyRvAq+q04D1+aq/GQ0hmgwJ0qRJsH6L60zNDdGUUNts4m1VZVVVLaP6l7Cmpo5XPqxs3kbNKijqBQU9GpeZwBuG0UXp9Bi8iFwqIjNEZEZlZQui2lZCQxXE1YVoNmzOFfgyAHq0EKKp3uom4z77kOGUlxXy8PQWwjQ1K13ue5jeI2DzGnui1TCMLkenC7yq3qmqk1R1Unl5+Y5XFGvsZE1omrTGWb/ZTeGX68GX0vxhp5XVLjwzvE8PvrjfEF5duIZMbkdszSooG9R0WZ+R7n3D0h233TAMIw90usB3GPHGNEkJUqQlzoaGEE1ODF6ah2hW+g7WQb2KGNW/B6mMsm5TXdM2qldC2ZCmy/qMcu8bTeANw+haREfgQ1k0EqQIYknW+xBNQQsx+NwBx1ZudAI/pHcRA3sWAbCqurZxhSCATdvy4Jd02K4YhmF0BPlMk3wIeAvYR0QqROSSfLUFNAnRkEmDJEMefNM0yR7UNQvRrKraSkygvLSQwb2KgUavHoAt69yk3rkx+B59XWzfBN4wjC5GIl8Vq+q5+aq7ReKNnaxk6iGeaPDgG55kTRSgsWSLefArqmoZ2LOIRDzGwF6FAKwOe/A1K917rgcvYpk0hmF0SSIUommMwZOuQ+MFoSyaeMNq6seEz522b1VVLYN6udBM/5JCEjFpePAJcB2s0NyDB+gzwgTeMIwuR3QEPpsHv2kVVFewpmAEm30YpiEGD1BQQmmLHvxWhvjQTCwmDOxZlCPwrXjw0OjBa/PhDwzDMDqL6Ah8Ngb/yWsALCndv6GoMCTwUpj14BufVM0+5JT14MFl0zTpZM168KUDm7fdZySka2HT6p3fD8MwjA4iQgLvQzTLZ0K8kDVl+zYUhUM0UlBCqdSxJdUYoqmudQ85DQ4LfEsefEk5JAqat51NlbQwjWEYXYgICbz34DUDww6hsKhxOIHcEE1ZrLbJpB8r/Rg02ewZaPTgNRt2aekhpyyWKmkYRhckQgIfc0P3Aoz4DCUFjV57YROBL6VEmqZJhh9yyjKoZxFb6jNU13pPv2ZFyx2sAL2HA2ICbxhGlyI6Ag+NYZoRR1BS2JgB2kzgc0aTDD/klGWgF/uGVMltefCJQug5ND8Cr2qdt4Zh7BDREvh4EiQOww6lpLDRg88N0fTIeZI1/JBTlmw8fmVVrXtwatOa1j14yE+q5JoF8Icj4JELTeQNw2g30RL4WAKGHACFpTkefKPYU1BCsW5tFqIZUOYecsoyyA9XsLqq1o0WiTbz4LfUp3no3WWkMoFPlezA8WgWPAV/PNZdND74B/zrgY6r2zCMbkG0BH7YJJhwJgAlBU7gRSAZl8Z1Ckopoo7ausaJt1dV1zaEZLIM6Om8+ZVVtaEc+KYe/JS3l3Ht4/OY8vZSJ/A1KyBVy06TroO/fQf67wmXz4ARn4PnfgwbW5lpyjAMowWiJfAXPAZHXAbQ4MEXxGOIhAXeDTi2ZcumhkWrq2sZ1LMxPAPO6+9XUuBy4RueYm3qwT85dwUAv3/xI7aUDHMLNy7b+f34+J9QWwXH/tRNCXj6/7kndJ/78c7XbRhGtyFaAh8im0XTpIMVoNANOFZTvbEhBXJVVW1DSCbMoF5FrKraCtVOyMMe/NJ1m5lTUcXpBwxhw5YUjy72HbwdEYef9xfo0Q9GH+W+9x0Fky6GD5+GLbtukizDMHZvoivw3oMvTMabFvgRJQsyW1i3uZ6tPhVyQEsC37OIVdV1UPUpxAugZEBD2ZNzXdjmhyeP5SsHDuW2uT6dcmcFvq4GPnwGxn+5cQA1gP3OcqNZvv+3navfMIxuQ4QFvhUPPjTx9sqNtQ3DEbTmwa+urnWdp733cLn2nn/MWcHBI/owtHcxlx41mpXpMtLx4p0X+AVPu2EPJn41x5j9oP8+MPcvO1e/YRjdhggLvI/BtyLwJdSyomprQ577oF7NBX5I72LWb64ns/4TN/eq5+M1NSxYVcOp+7mQzd4DyuhZlGRtYvDOC/y8v0Cv4TDs0KbLRZzoL3vTOlsNw2gTkRX4Hj6LpkmKJDRO+iG1rNzoBD5BmkFF6dwqGNXfXQx0w9LG4QiAaQvXAnDSeNfpGosJk0b2ZVGm/84JfF0NLH4F9j29yd1CAxNdhhDzH93xNlojCFz2jmEYkSGyAt9qJ6v34HvF61lZVcuqqlquS0xh1MNHNevAHFNeShlbSNRtdA8yeeavqKK8rJAhvRvHrjl4RB8W1PZDN3yy4w8lffKam5FqrxNbLu87CoYdAvM6UOA/egH+ayzc1Bd+OQTe+B97qMowIkJkBT4Rj1GYiLUQonEe/JDiDCuqXAz+c/H3iG1aBU/9oMmqI/r1YI/YGvclFKKZv7yKiUN7NVn3kJF9WaYDkNQW2Lx2x4z++J+QLIE9Dm99nYlnwer5sPr9HWsjzOJX4eHzobgvTL7aXVheuB6evNI9vWsYxm5NZAUeXBy+NQ9+UFGalRu3UrVxA2NkOfTaA957HOY/1rBqUTLOgWXV7osP0WypT/Pxmk1MyBH4/Yb1YoX4LJsNn7Roj6ryt38t56UFq6muTeUWwscvwKjJbmyb1hj/ZTccw7yd7GxdNQ8eOhf6jYGvPwnHXgdnT4HP/TvMvAdeumnn6jcMo9OJuMDHW43BlxemWVlVS+n6+cRQ+MItMPRgePbaJiGKiT02uA8+RPPBymoCpZkHX5SMIwPHuy8rZrdoz4sfrOGKqbP5xr0zOPCmF3hsZkVj4bpF7iGpvY7f9k6VlsOYY1yYZkdDKUEGnrjcXewu/JubOBxc3P/4G2DSN+CN38PC53as/lwqF8K7f4RnroGXf9kxdx+GYWyXSAv8Z8f05+ARfZouTBRALEnfZL0boqB6vls+/FA46CI3K9PajxpWH5NcS7X2IFPYG4B5FVVAc4EHGDVmLCu0H5klbzQrq08H/PLpDxhdXsKD3zyMiUN78R9Pf0BN1pP/+J++weO2v2MTvwpVy+DTd7a/bktMvwtW/As+fzOUtTBD1Um/goET4a//D6oqmpe3lfrN8Pz18IfD4emrYNafYdotcNsRcPdJbjA1wzDyRqQF/uYz9uM7R49pXlBQQu9EikygjK5fwIbCYc6Lzca+P327YdUhuoZlOoAVG92kIPOWV9O/tJCBPZuHUQ4e0YfpwT5klrzVzLt+4O2lLF67mZ+cMo7P7Nmfm04fz/rN9dw5bbFb4eN/Qr89XUfq9hh7CiSKdyxMU7UcXvw57Hk8jP9Ky+ski+Cs+1wc/tFvQCbV8nrbYv0ncOfR8Ob/wIHnwxXz4MfL4Qcfwsk3w9qFcMeR8Np/7Xi8f+sGeOdOePBsuGVP+PUo+K9xMPUCdxHb0b6QMCk/VMXGT117hrEbkdj+KhGkoJSeMZcSuH9sEev7HEofgH57QXEf5xkf9DUA+tSvYI6WU1m5ieF9e/gO1p5Nx7fxfGbP/vwnYzl965suXdKLdXVtit+/+BFH7tWfY/Zxcfr9hvXm1P0Gc9drn/C1A3pT/sk0FxpphSBQYjHfZmEZjDsV5kyFY65rDLG0hWevcU/EnvJfLre+NfqNgdN+7wT+pZ/DCe2IyVfMcKKrGfjaE41DLgCUDoDDvwMTznBe/Ys3wftPwOm3wqAJbat/y3p461Z45w6or3FTJu51krsw1dXA0jfdCJzPXgvjvggHfx1GHrnt/c1SWwWLXnLZRctnurs5DU3QXtTLPXA29GD/Ogj6jt5+3XU17qJWuRDWL3Z3ils3uO3iBW4YjN57uLGHeg2HnkPcubiteoMMbFnn6tq0BjZXurumIOPuVAt7OnuLekFRb3eeFPWCWHwbdQaQ2uLqqd/k3zdDpg7ihe4YJ4rde0GpC/Mlilq3U9U5CJk6SNf79zrI+MH+JO62jcXdhD3i3xu+x3K+e9s14/ZTA/cKMoA23abF7bMvaW6nqqtD1dUJfj1p3KYt51AXonsKfGEppVLHADYwRNbz/oCD3PJYDIYfBst86CMIKNq8nGU6lviaTRw+qh8franhxPEthDWA0sIEMuIIqPgTwdK3iHmBf/jdZVRtTXHNyWObXBiuPmkfnntvFW/+/Q5Oz9TB/mc3q3PBqmr+/NZSnpq3kr4lBfz6jP04ZGRf1xk671EndMdd37b9XvC0E77jb2yS198qE86AJW+4ePzAibDfV7e/zQf/gMe+5UI/5z/mRsRsidIBcNb98N7fXPbSnUfD5KvcfrU07y1A9Up453bnnddvds8LHPkDGLxf0/VUYc0HMOt+mPOg6zjvOwb2Pc2FwAbs23hRrN0Iq9+DT991wr7sLXcBLOrt7ujGngo9BzsRrq12Heir34dZ98E7t7k6ivs4sR8wzmUkJYogtdndQVR+6IS9enmjfRKDHv0bBTxd6/Ytk/McQqLYCX3ZYCeoEnM2bFnnXls3AO3thxEo7u2ytdzBcm+ZlDumqc3trM/vT7JH44xq4IU9JORdEYk1inp7t2si+jmBkCZ37y3U3dL2pQPg8pnts6MNdE+BLyihmC3sH1sEQHyPgxvLhh8GC5+FzesgU4+ka1mXHExN5Wbe9x2suRk0YQ44+AiqPu1B3fuvMuDA80hlAu55YwlHjO7XbLsR/Uo4/7ARjJzxE+rKx1E4+IAm5fOXV3H2HW8RKBw3bgBzKjZy1h1v8Z2jxnD1SeOQ8V9ygnf4ZVDSb9v7XLcJnr7aidsR3237sTrpl1C5AB7/lhOigy5seb0gA2/9H7xwgxu2+dyHoaT/9usf/yXnXT/zQ3jlVzB3qksFHXOsE+F0nesv+Oh5N0a+Bu7CM/kqJ6gtIQID93V9DMff4O4Q/vVnePN/4fXfuXXiBU7Is54awIDx8JnL3d3AsEMgvo2/RyYNlR84L3/5TKiY6Z5jCIt0QakLu438HPTfG8r3gfKx7uIaHmcInOe8udL1eVQtc4JfvdwNdFez0t21aABFPd2dTo9+7mJSOsBNBl860H0uKHHzIqTr3N1IXbV737oRtq539WxdD6mtgHc2BLdNQWmjV15Q0vRzvMDtW6oW0lvd9vVbGr381JbwD+CPcdJlhMUL/Huhu3jH/Qu8t5xp9MI1CH0Pcr77d9R58rF4U4+/ob4g5xXy9LPl2boaBFpyvPXsfmQ9em1sO7eubD1NT8Km52OWhruEgCZ3DQUl5INuK/CJdR/xzeRqUhqnbORBjWUNcfh3Grw87b0H76+s5n9f+oiYwAHDe7da9XH7Dmb6X/dhwrK3AHh63kpWVtXyiy+1HH64YmKK3rMW8QiXcVboRPh0/RYuvnc6vXsU8Phln2FgzyI216W56R/v84dX3IXph0f9yHnAb/4PnPCzhm2Xb9zK1Omf8t7yKob1KWb84BLOWPQT4tXL4av3NBeXbZEsgvMfhannwxPfhYrpTlx77+HKg8D1WTz3YyfE406Dr9wJyeJt1xumpB+cebcT7rf+D179Nbx6c9N1ivu6oaAnfcOFRNpsf7G7M9r/bOf9Ln3TeeHVK5zIFPduDLls7yIZJp6AQRPd6+CvNy5PbXUXwmRJ63ciLRGLubuesoEw7ODtr98Weg3tmHqM3ZbuKfBDDkSWvM4kWcE/MwdxbJ/eTcqIJZ3AD9gXgMLyMcyZuxER+OWXJzKwhYHJspQWJtjY/2DK19/F8opl3DltKaPLSxpi77n0XvgXMhLn5uX7MfTjtXx2z/58tLqGS+6bQV0qw4PfPKyhvZLCBDefMZF4XPjDK4tQxnDVxLOIv/k/MHA8a0aexs3PLOBvs5ejuCdx3/1kHftk7iCeeInpY69m4qCDybW+Ph0wc+kGZi3bwJZ61+E5cWgvDh/dj949CqCgh/PIX/gpzPgTzH7QeaY9+sGa951HWDoIzvAivaNxyrFfcK+a1e5iUVfj6hq8vwuxtDR8Q3so6gn7nLxzdWyPZHH7Lm6GkUfyKvAicjLweyAO3KWqN29nk13DCTfBCTdx0V3vsGBVDTNCU/WRLHaCMvcRFz9MljBoj72QeR9zy5n7c+bBw7ZbffmEY2HaXfzl9ht4L30mt5y5X2MHaZj1i2H2FHSvkyleNpAL7n6HE/cdyBsfr6MoGef+Sw5jr4FlTTYREX5x+gQyGeW2VxbxWvkZ3NFrEYMfv5S79E1eTB/FN48cy4WHj2B4/WL0xZuQj17isZKz+cHsA+n30UucefAwhvQuJhMoby5ay1uL1rHZT2GYiAkKZAJFBPYd3JPDR/djRL8e9B36fVLFX2HIwgfoUfMJBVXr+CQ4gJeDfXlry6GUvdyXwbNmMrhXEYN7FzG4VxGJWIzq2hQ1tWlqalNUb3XvgULP4gRlRUl6FiUpK0pQVpSgMBGnLp2hLj2eunRAXSpD3dqAdGYRBYkYRckYhYl4w3tBIkYmUDKBkg4C0pnsZyUeg3gsRiImxGNCMi7E/MUnUPfgmQKBKoEPlcYEYiKIP9YxoUm/iTT5LVr+3BaEdm+Qz9VbTBroyPpdG+1cv52ttL/+dpJH+wsSwsEj2pEs0VYbNE/jjohIHFgInABUANOBc1W11adcJk2apDNmzMiLPS3x2MwKlq3fwpUn7N204MWfw2v/CXueAEf/iNTgg1hdXcuwPj3aVG99KsOiO89nXOUzLDvqd+xxzDdcTFRijeGRDUvh3lNcDPPiZ9hYOobbX13MvW9+wp4DSvnj1yYxuNe2PcGXFqzmxifeZ836Ddxb9DsOZy5BvIjYwHGuE27jp85rPfIq9Ijv8vYnG7j79cW8uGBNQz/QHn17MHnv/kzeq5wjxvSjrChJKhMwt2Ijb368jjcXrWPmsg3Upxtj1fGYMLCskEG9ihjcu5iBZUVsTWVYWbWVVVW1rNi4lera5qmP8ZhQVpSgZ1ESEaipTVO9NUU6yM85aBi7C/1LC5nxk+085NgKIjJTVSe1WJZHgT8CuFFVT/LfrwVQ1V+1ts2uFvhWSde5tLPew3eijnqY4rNQCktdRxe4TqFEkevgSxbDRU+4OwZPTW2K4mS8yQTg2yKVCdhUm6ZPcRyWvgEfPAnrPnKTk/TfCw65xGVrhKhLZ6ipTRME2uJEJ812JROwfks9Gzan6FWcpLyskHhLdyQhNte5J4UD1QYPvUdBvJmnqKrUpgLn3demqE0FFCXjFCZi7j0ZoygRJx4T6tMBdekMtanG9/p0QDwmJOLeS4/FiMeFuAiBNnrz6UxA2nv64L10763HQtlvqs67D1T9Zw3ZGrKblpe3hfb+49r7H21//e3coN0t7Ipj1N71u9YxTcbdiLQ7QmcJ/JnAyar6Tf/9QuAwVf1uznqXApcC7LHHHgcvXbo0L/Z0CrVV7tH8IOOyHMB1wKVrncAfeGHbc78NwzBaYFsCn88YfEsuXrOriareCdwJzoPPoz27nqJe8Plfd7YVhmF0U/I5VEEFEI5xDANW5LE9wzAMI0Q+BX46sJeIjBKRAuAc4Ik8tmcYhmGEyFuIRlXTIvJd4DlcmuSfVPW9fLVnGIZhNCWvefCq+jTwdD7bMAzDMFom0sMFG4ZhdGdM4A3DMCKKCbxhGEZEMYE3DMOIKHl7knVHEJFKYEcfZe0PdMAcbXnFbNx5urp9YDZ2FGZj2xihquUtFXQpgd8ZRGRGa4/rdhXMxp2nq9sHZmNHYTbuPBaiMQzDiCgm8IZhGBElSgJ/Z2cb0AbMxp2nq9sHZmNHYTbuJJGJwRuGYRhNiZIHbxiGYYQwgTcMw4gou73Ai8jJIvKhiHwsIj/qbHsARGS4iLwsIh+IyHsi8n2/vK+IvCAiH/n3PturaxfYGheRf4nIk13RRhHpLSKPisgCfzyP6Eo2isiV/jeeLyIPiUhRV7BPRP4kImtEZH5oWat2ici1/j/0oYic1En23eJ/57ki8lcR6d1Z9rVmY6jsKhFREenfmTZuj91a4P3E3rcCnwf2Bc4VkX071yoA0sAPVHUccDjwb96uHwEvqupewIv+e2fzfeCD0PeuZuPvgWdVdSywP87WLmGjiAwFvgdMUtUJuGGxz+ki9t0LnJyzrEW7/Ll5DjDeb/MH/9/a1fa9AExQ1f2AhcC1nWhfazYiIsOBE4BloWWdZeM22a0FHjgU+FhVF6tqPfAwcHon24SqrlTVWf5zDU6UhuJsu8+vdh/wpc6x0CEiw4BTgLtCi7uMjSLSE5gM3A2gqvWqupEuZCNuyO1iEUkAPXCzlnW6fao6DVifs7g1u04HHlbVOlX9BPgY99/apfap6vOqmvZf38bNAtcp9rVmo+d3wA9pOgVpp9i4PXZ3gR8KfBr6XuGXdRlEZCRwIPAOMFBVV4K7CAADOs8yAP4bd6IGoWVdycbRQCVwjw8j3SUiJV3FRlVdDvwnzpNbCVSp6vNdxb4WaM2urvg/+gbwjP/cZewTkdOA5ao6J6eoy9gYZncX+DZN7N1ZiEgp8BhwhapWd7Y9YUTkVGCNqs7sbFu2QQI4CLhNVQ8ENtP5IaMGfAz7dGAUMAQoEZELOteqHaJL/Y9E5DpcmHNKdlELq+1y+0SkB3Ad8NOWiltY1ulatLsLfJed2FtEkjhxn6Kqj/vFq0VksC8fDKzpLPuAzwKnicgSXGjrWBF5gK5lYwVQoarv+O+P4gS/q9h4PPCJqlaqagp4HPhMF7Ivl9bs6jL/IxG5CDgVOF8bH9LpKvaNwV3M5/j/zTBglogMouvY2ITdXeC75MTeIiK4uPEHqvrbUNETwEX+80XA33e1bVlU9VpVHaaqI3HH7SVVvYCuZeMq4FMR2ccvOg54n65j4zLgcBHp4X/z43D9LV3Fvlxas+sJ4BwRKRSRUcBewLu72jgRORm4BjhNVbeEirqEfao6T1UHqOpI/7+pAA7y52mXsLEZqrpbv4Av4HrcFwHXdbY93qbP4W7P5gKz/esLQD9c9sJH/r1vZ9vq7T0aeNJ/7lI2AgcAM/yx/BvQpyvZCPwMWADMB/4MFHYF+4CHcP0CKZwQXbItu3Chh0XAh8DnO8m+j3Fx7Ox/5vbOsq81G3PKlwD9O9PG7b1sqALDMIyIsruHaAzDMIxWMIE3DMOIKCbwhmEYEcUE3jAMI6KYwBuGYUQUE3jD6ABE5OjsiJyG0VUwgTcMw4goJvBGt0JELhCRd0Vktojc4cfD3yQi/yUis0TkRREp9+seICJvh8Yn7+OX7yki/xSROX6bMb76Umkcu36Kf7rVMDoNE3ij2yAi44Czgc+q6gFABjgfKAFmqepBwKvADX6T+4Fr1I1PPi+0fApwq6rujxt7ZqVffiBwBW5ugtG48X4Mo9NIdLYBhrELOQ44GJjuneti3IBbATDVr/MA8LiI9AJ6q+qrfvl9wF9EpAwYqqp/BVDVWgBf37uqWuG/zwZGAq/nf7cMo2VM4I3uhAD3qeq1TRaKXJ+z3rbG79hW2KUu9DmD/b+MTsZCNEZ34kXgTBEZAA1zlI7A/Q/O9OucB7yuqlXABhE50i+/EHhV3bj+FSLyJV9HoR8n3DC6HOZhGN0GVX1fRH4CPC8iMdwogf+Gm0hkvIjMBKpwcXpwQ+re7gV8MXCxX34hcIeI3OTr+Oou3A3DaDM2mqTR7RGRTapa2tl2GEZHYyEawzCMiGIevGEYRkQxD94wDCOimMAbhmFEFBN4wzCMiGICbxiGEVFM4A3DMCLK/wfXVHNVOEpzuwAAAABJRU5ErkJggg==\n",
+      "text/plain": [
+       "
    "
+      ]
+     },
+     "metadata": {
+      "filenames": {
+       "image/png": "/Users/hjensen/Teaching/FYS-STK4150/doc/src/LectureNotes/_build/jupyter_execute/chapter10_9_421.png"
+      },
+      "needs_background": "light"
+     },
+     "output_type": "display_data"
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "MSE:  0.32920351498520983\n"
+     ]
+    },
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD4CAYAAADhNOGaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd3hUVfrA8e+bZEISEgIJoZfQe48UA1gABUEBFQQXxIooWHZ1VyyruOu6iK6uSHFRpCgLioKgsIJYfvSSQIDQA4QQCIROAulzfn9kCCEFEmaSmWTez/PMM3PPOXfum6vkzT3n3nPEGINSSin35eHsAJRSSjmXJgKllHJzmgiUUsrNaSJQSik3p4lAKaXcnJezA7gZVatWNaGhoTe388WLkJnp0HiUKjVeXlCpkrOjUGVUZGTkaWNMSN7yMpkIQkNDiYiIuLmdFyyAGjUcG5BSpeXECRg2zNlRqDJKRI4UVK5dQ0op5eYckghEpK+I7BORGBEZX0C9iMhkW/0OEelY1H2VUkqVLLsTgYh4AlOBfkBLYLiItMzTrB/QxPYaDUwvxr5KKaVKkCPGCDoDMcaYQwAisgAYCOzO1WYgMNdkz2exUUQqi0hNILQI+yql3EBGRgbx8fGkpqY6O5Qyz8fHhzp16mCxWIrU3hGJoDZwNNd2PNClCG1qF3FfpZQbiI+PJyAggNDQUETE2eGUWcYYzpw5Q3x8PA0aNCjSPo4YIyjov1jemewKa1OUfbO/QGS0iESISMSpU6eKGaJSytWlpqYSHBysScBOIkJwcHCxrqwckQjigbq5tusAx4vYpij7AmCMmWGMCTPGhIWE5LsNtkg++eUA355P5PPjy0jKvHxT36GUKjmaBByjuOfREYlgC9BERBqIiDcwDFiap81S4BHb3UNdgQvGmIQi7usQMYlJ/Ovn/TwR/ztPHfiAkPUP8PDu91h/IRqdilsp5c7sTgTGmExgHLAC2AN8Y4zZJSJjRGSMrdly4BAQA3wGPHu9fe2NqSDfRMRjyCDZ61cA0kwq80/9RHjUc4RufJRJcQtITD9XEodWSpUR58+fZ9q0ac4Oo9Q55DkCY8xyY0xTY0wjY8w/bGWfGmM+tX02xpixtvo2xpiI6+1bEu5rV4uw5oexyoV8dXHpcbxy+D/U3DCEvtvfZPmZjWSZrJIKRSnlogpLBFlZ5fv3gds8Wdy6diAfDrqfP1cZRJBncIFtrGSx4vwa+ke/SrV1w3jl0OccSilwyEIpVQ6NHz+egwcP0r59e2655RbuuOMOHn74Ydq0aUNsbCytW7fOafvBBx8wYcIEAA4ePEjfvn3p1KkTPXr0YO/evU76CW5OmZxr6GbVr1yfSdUe4t0241hxdjOT45ez6vwGrOTP9mezTjPp6DwmHZ3HLf7teb72PTwQ0hNfzwpOiFwp9xI6flmJfXfsxP6F1k2cOJHo6GiioqL4/fff6d+/P9HR0TRo0IDY2NhC9xs9ejSffvopTZo0YdOmTTz77LP8+uuvJRB9yXCrRHCFl3jSP7gb/YO7cTL9LLNPrGTqseUcTT9aYPstyVGM3BfF0wcmMzykF3+qO4iWFUNLN2ilVKnr3LnzDe/FT05OZv369QwZMiSnLC0traRDcyi3TAS5VfcO4pV6w/hL3YfYcHEXnxxbxqLTv5Nu8t+De9mazMyTS5h5cgm3B3bhzfoPcXvl9nrLm1LlVMWKFXM+e3l5YbVac7av3KdvtVqpXLkyUVFRpR6fo7h9IrhCRLg1sDW3BrYmKfM5FiT+yuT45USn7Cmw/e8XNvH7jk0082nCX0OHMjTkdiweejqVcoTrdd+UpICAAJKSkgqsq169OomJiZw5cwZ/f39+/PFH+vbtS6VKlWjQoAELFy5kyJAhGGPYsWMH7dq1K+Xob57bDBYXR4CXH0/VGsDOztOIDvuCcTWHEOARWGDbfakHGLH3H9Ta8DCT4r7mYualUo5WKeUowcHBhIeH07p1a/785z9fU2exWHjzzTfp0qULAwYMoHnz5jl18+bNY+bMmbRr145WrVqxZMmS0g7dLlIWH6YKCwszpb0wTbo1g0Wn1vDOkYXsSin8joD7Ah/hq9aPEuCl3UWqBJTjhWn27NlDixYtnB1GuVHQ+RSRSGNMWN62ekVQRN4eFoZVv5Odt0xjTfvJ9KkcTt6pksR4s/XEXdz6+2Xe3ZvO8RRrwV+mlFIuRBNBMYkI3QPbsLLdO+y7ZQ6PVr8XC94AVMy6E08qk5QJM2Iz6Lk6hRe3pxJ9MYvjaaexGk0MSinXo6ObdmjqV5dZzf/EpIaP88mxJQRae7D8uHDwUnZ3W6aB7xOyWJyQyXm/l6jiLbxRbwgja/Shgoe3k6NXSqlsekXgACHelflbg1G81KghP3f3ZWbHCnSpcvXUpnpEctHEcSTtCE8d+IA6G0ZwNj3ZiRErpdRVmggczEOEXtW8+LqLL0u7+XBvDU8uei26ps2ltMY8sNGD745lkGkte4P1SqnyRRNBCWob6Mm/2npyd0hVPHKd6soZwzh82fDSznTuXJPCN/EZZGhCUEo5iSaCEubtYeH7Nn/lUJd5PF3zftr79qSqZ6Oc+rgUw1+i07ljTQqTDu7n3SPzSc5KcWLESilH+P333xkwYAAAS5cuZeLEiYW2vdnprydMmMAHH3xw0zFeoYmglNT3qcGnTZ9j6y0TWHubHy83sVA517rS8SmGt2O/4vXYGdRcP4y/H5mnD6cp5YJuZkrq++67j/Hjxxda7+x1EPSuoVImIlSywLhG3jxa38LcuAw+O5zBycw4LnuuBSDZepE3Yz/nvbivebnOg/yx7v0Eevk7OXKlSpe87diHMs1bN+5+jY2NpW/fvnTp0oVt27bRtGlT5s6dS8uWLXn88cdZuXIl48aNIygoiLfeeou0tDQaNWrErFmz8Pf356effuLFF1+katWqdOzYMed7Z8+eTUREBFOmTOHkyZOMGTOGQ4cOATB9+nQmT56cM/11nz59eP/993n//ff55ptvSEtLY/Dgwbz99tsA/OMf/2Du3LnUrVuXkJAQOnXqZPe50SsCJ/L3Ep5t6M3a2/wIDV4Mcu3/qJesSbwdN4ua64fx+qFZnMsoeA4UpZTj7Nu3j9GjR7Njxw4qVaqU85e6j48Pa9eupXfv3rzzzjusWrWKrVu3EhYWxocffkhqaipPPfUUP/zwA2vWrOHEiRMFfv/zzz/Pbbfdxvbt29m6dSutWrVi4sSJNGrUiKioKN5//31WrlzJgQMH2Lx5M1FRUURGRrJ69WoiIyNZsGAB27ZtY9GiRWzZssUhP7MmAhdQ0Uv4sd04xtcdia/45atPMZd49+hcam0YxviDMzmbcdEJUSrlHurWrUt4eDgAI0aMYO3a7Cv1hx56CICNGzeye/duwsPDad++PXPmzOHIkSPs3buXBg0a0KRJE0SEESNGFPj9v/76K8888wwAnp6eBAbmn8ds5cqVrFy5kg4dOtCxY0f27t3LgQMHWLNmDYMHD8bPz49KlSpx3333OeRn1kTgIoIslfhnw8c5fuvX/LXeo/h55O8KSjWXeS/+K+psGM6Ew3N1UFmpEpB3Wvkr21empDbG0KdPH6KiooiKimL37t3MnDmzwH1vljGGV199NecYMTExPPHEEw49Rm6aCFxMZS9//tZgFMe7zWdC/cep6BGQr02KuczbcbOovX44Hx79ljRruhMiVapkmbeMQ19FFRcXx4YNGwCYP38+3bt3v6a+a9eurFu3jpiYGAAuX77M/v37ad68OYcPH+bgwYM5+xakV69eTJ8+HcgeeL548WK+6a/vvvtuvvjiC5KTsx88PXbsGImJifTs2ZPFixeTkpJCUlISP/zwQ5F/ruvRROCiAr38eSt0JAndFvBO6FMFToN90XqBlw5Npe6GkXyesJxMU74X2FaqNLRo0YI5c+bQtm1bzp49m9ONc0VISAizZ89m+PDhtG3blq5du7J37158fHyYMWMG/fv3p3v37tSvX7/A7//444/57bffaNOmDZ06dWLXrl35pr++6667ePjhh+nWrRtt2rThwQcfJCkpiY4dO/LQQw/Rvn17HnjgAXr06OGQn9muaahFJAj4GggFYoGhxphzedrUBeYCNQArMMMY87GtbgLwFHDK1vw1Y8zyGx3XGdNQO1tyVgofxy/mn3ELuGQteNC4nnddPmj0OA+E9MRDNMeXSzoNdYmKjY1lwIABREdHOzUORyjNaajHA78YY5oAv9i288oEXjLGtAC6AmNFpGWu+o+MMe1trxsmAXfl7+nL6/UfJr7bf3mlzgi8xSdfm7j0owzd8zYtN49hxdnNlMW1JpRSpc/eRDAQmGP7PAcYlLeBMSbBGLPV9jkJ2APUtvO4bquylz8TGz1BXNd5jKl5P55Y8rXZl3qAvjtf4ZatL7Luwk4nRKlU2RQaGlourgaKy95EUN0YkwDZv/CBatdrLCKhQAdgU67icSKyQ0S+EJEq19l3tIhEiEjEqVOnCmvmNqp7BzG96XMc7DKX4SF9kQL+U0Ym76B71PPcGTWe7ckxTohSqeLRq1jHKO55vGEiEJFVIhJdwGtgcQ4kIv7Ad8CLxpgrN8JPBxoB7YEE4F+F7W+MmWGMCTPGhIWEhBTn0OVafZ8a/LflK+y+ZRb3VOlZYJvfLmxi7O7VHNMV05QL8/Hx4cyZM5oM7GSM4cyZM/j45O8+LswNp5gwxvQurE5ETopITWNMgojUBBILaWchOwnMM8bkzMlsjDmZq81nwI9Fjlxdo7lfPZa1fZvIpH28GDOTtRevPnHoYSoRd/Ze7lidwoh6Xoxr5E2Qt66prFxLnTp1iI+PR6/47efj40OdOnWK3N7euYaWAqOAibb3JXkbSPbTDzOBPcaYD/PU1bzStQQMBtyvc87BOgU0Y02HSfzf+SieP/AZOy7vJjBjKB74kW7giyOZLDyWybMNLTxW34KPpyYE5RosFgsNGjRwdhhuyd4xgolAHxE5APSxbSMitUTkyh1A4cBI4E4RibK97rHVTRKRnSKyA7gD+KOd8Sib2yq3JypsCsta/5Pv2t9P+8Cr/6mTMuG9/RncsSaZTlteYWbCcrL0GQSl3JZdzxE4izs+R2AvYwwrE7N4b386h2xrKid7ruKM978BaOzTkGlNnqFPUL5bjJUrKcfPEaiSV1LPEagyQkS4u7oXK8J9+XtLb6p4p3Le8mVOfUzqIcbsWcaBZB1QVsrdaCJwMxYPYWQ9C39vG4tIrllMjRdpySO5e20Kr0ankZimCUEpd6GJwE0NqNqJmC5zuT+4FwCVMu/Dy1THCsyPz+T21Sl8HJPO5cyy13WolCoeXaHMjdX3qcF3rd8gImkI6RnVmXbQg7Vnsq8ELmfBRzEZzDuaScXAyQyp3oLRtQbgJZ5Ojlop5Wh6RaAIC2jGrUGV+TLMh9mdKtDM/+otpUcyovg9aQVjY/5N442P8+OZDfrAj1LljCYClUNEuD3Ei+XhvrzX2puQCobzli9y6o+kx3Fv9Gvcuu0ldiYfcmKkSilH0kSg8vEU4aE6FqZ0PIHxjM9XvzFpG+0in+KJvf/mdMYFJ0SolHIkTQSqUF0CG3O4y5cMD+kLXPsEssHKFyeXUH/jCD46+h0Z1kznBKmUspsmAnVdtSuE8N+WrxDVaQa3BnTMV3/ZmsyfDk2hyaYnWHF2sxMiVErZSxOBKpJ2/o1Z2+EDvm/1d2pZauWrP5IeR9+dr9Ar6lX2Xz7qhAiVUjdLE4EqMhFhYNXuHOo6i3cbjMZH/PK1+fXCRlpueZznDkzjQmayE6JUShWXJgJVbBU8vHm13nAOd/2Sh0P6kXf8IItMphxfSL0NI/jP8R90QjulXJwmAnXTangHMa/lX4joOJ0OFVvlq79ovcCYAx/ScvPTrD6/3QkRKqWKQhOBslungGZEdvqE+c3/SlWv/KvH7U89yG3bX+TenRM4knrCCREqpa5HE4FyCBFhWPU7OdJ1Lq/XG4VFKuRr8+PZ/6PxplG8dmgWl7NSnRClUqogOteQcig/Tx/eafAoT9e8h+cP/Ifvz/56TX0m6fzz6Fy+OLGSmc2ep39wNydFqhzFaqxkZGWQlpVGelY66VnpANQKyH93GUDE8QjOXD6Tsx1WK4xgv+B87RKSEog6EXVNmcHx05t0rt2Zqn5VCzz+thPbcrZr+NegY838t1ADrDy4kswSfpbmese3lyYCVSLq+lRjcZu/svbCIEbvm8KelP3X1J/MOMHf9h2jeVsrjfz1wrQ0Hb1wlDMpZ0hOTyYpLYmk9KSc92vKbOXJ6ck5ny9lXCItM/sX/pVf/AX9AmxRtQW7x+4u8PjjV43nl8O/5Gz/PPJnejfMvzT6mrg1PPTtQ477wQtR2PHXxq1l6LdDc7YfbPkgC4csLPA7Hvr2Ic6nni+xGG90fHtpIlAlqntgG6Jvmc6sEz/x0sHPuZB1DgBvayNOXLyLvutSeDLUwnONLPh56frJpWHIwiFsOrapRI9x5apAlQ36p5gqcR7iwRM17+FI17k8U/MBPPAkOP0ZBE8yDEw/nEGvtSksO5GpM5uWgoAKASV+DE0EZYteEahSE+jlz7Sm43ij/jDOpVXhzd3pRJ7PXv8gIdUwNiqNrkHppPt9ynsNR9LEr46TIy6fArwdnwgsHhYqeFXA29Mbb09vqlWsVmjbsFpheHpcXdciyDeowHY1/GvQt3HffOWCY68cg33zj09cOX6/xv1ytjvU6FDod/Rp2Ifk9JJ9gPJ6x7eXXYvXi0gQ8DUQCsQCQ40x5wpoFwskAVlA5pXFk4u6f166eH35YDWGRccz+ee+dM7Y/oA85zWbi5Zv8cTCn+oMZULoCPw8fZwbqCtxwOL1L698mRUHVxDgHUBAhQACvAPw9/a/ZjugQv4yf29//L398fHyyfmFX8GrAhYPCyLarVcWFLZ4vb2JYBJw1hgzUUTGA1WMMa8U0C4WCDPGnL6Z/fPSRFC+XMgwfHQgnc+PxnKswjiQq4OPfwgexZetRukvmisckAiU+yosEdg7RjAQmGP7PAcYVMr7q3Ig0CJMaFmBvvV+uyYJeJpgVscPYEREKjHJVidGqFT5Zm8iqG6MSQCwvRfWMWiAlSISKSKjb2J/5QY+a/40XzV/nSqe2X3GVTKewANf1p2x0nddCv/cl05ypg4mF8eBMwdYc2QNVqOJVBXuhoPFIrIKKKgv5fViHCfcGHNcRKoBP4vIXmPM6mLsjy2BjAaoV69ecXZVZYSI8Ifqvbk3uBufHf+ZpOQ7mBuXhRXINPCfwxl8fzyTN5p7EeCzm9urtHd2yC5v6papfLzpY2r61+SBFg8wJmwMrarlnxdKubcbXhEYY3obY1oX8FoCnBSRmgC298RCvuO47T0RWAx0tlUVaX/bvjOMMWHGmLCQkPzz2ajyo5JXRV6qN4gJLX1YFu7LLVWu/m96Ms3waPQy7tjxR/pu/ytHUwv9X8btWY2Vb3d/C0BCcgJTtkxh7+m9To5KuSJ7u4aWAqNsn0cBS/I2EJGKIhJw5TNwFxBd1P2Ve2sR4ME3nX34qG0FQioIWVzgvGUWACvOr6XJ5kd5/+g3ZOpU1/lsit/EsaRjOdt+Fj/6Nel3nT2Uu7I3EUwE+ojIAaCPbRsRqSUiy21tqgNrRWQ7sBlYZoz56Xr7K5WbiDC4lhe/dPcluPJsrJKUU5dmUvjLoem02vw0my/ucWKUrsfP4sfw1sPx9/YHYEDTAfhZ8i8mpJRdt486i94+6r62J8cwcs+/2Hm5oC4O4bHq9/Jh46eo7OVf6rGVipu4fTQlI4UVB1dQw78GXet0LaHAVFlQUrePKlWq2vk3JipsKlMav4ivVMxTa5h1cikNNo5i/slfdLoKG1+LL4OaD9IkoAqliUCVOR7iwdjaAzncdS6Dg3vlqz+fdZaH975Dz6i/EJNyrIBvUErlpolAlVnVvYNY1PoNfm77AbUt+ee+X3sxghabH+PNw3NJs+okaEoVRhOBKvN6V+lETNdZvFr3ETyxXFOXSQZ/j5tFk01P8vv5qEK+QSn3polAlQs+Ht682/Axdt3yOV388z9odjT9KHds/yNDd73LqfSSXUDE2b7b/R0rD64kIyvD2aGoMkITgSpXmvnVY0PHD5nT7FUqeQTmq194+mcabHqE2SdWlMvBZGMML//8Mnd/dTc1/1WTJ5c+SUJSgrPDUi5OE4Eqd0SER2rcxeGucxlZrX+++kvWJB7bN5Ee2/7MoZTjToiw5EQmRBJ7PhaAMylnmLdzXs5zBEoVRhOBKreCLJWY2+Jl1rX/hIYVQvPVr0uKpPmWx3kv7uty82Tywl3Xrmnbt3HfUlmRTJVtmghUuXdrYGv2dv6Mv4c+hRfe19RlmDTGH/6UtlueZVvSASdF6BjGGBbuvjYRDGk5xEnRqLJEE4FyCxYPL96o/zB7On9R4GDynpT9DIv+gqQyPM311oStHD5/OGe7gmcF7m16rxMjUmWFJgLlVhr71mZDxw/5rOmf8fO42ncuxpdLF0fTZ00KqxIzr/MNruvKTKNXaLeQKipNBMrtiAhP1ryHg13mcG/Q7QBUzngEL1ONE2mGJ7emMTYqlVNpZefqQLuFlD00ESi3VcM7iKVt3mJVmw+Y2WowwbmGD5adyKL32st8E5/Bxgu7Xf5W06gTURw8dzBn29vTm3ubabeQKhpNBMrt9QrqxOBaFVjV3Y8Ha19dtO9CBjy3ezPdosbSZ/vrLr0ITt6rgbsb3U2lCpWcFI0qazQRKGVTxVv4oE0Fvgrzoa6vYCWVs5ZPAPjlwgaabH6U/5781clR5qfdQspemgiUyqN7VU9WdvcltOrXZHqczClPs6byyYEqRF9wrWcOdpzcQczZmJxtb09v7mt2nxMjUmWNJgKlCuDrKXza4h7a+bXMKQvIGkBCclMGbkzl/f3ppFldY9wg79XAXY3uItAn//QaShVGE4FShWhZMZStYZ8wudEL1LLUp3rWSACyDEw9lMGA9SlsO+/cqwPtFlKOoIlAqevwEA+eqzOIuG4z+Tk8mM5Vrv6TOZBseGBjKi9HH+FvsV+RYS395w92Ju5k/5n9OdsWD4t2C6li00SgVBF4iiehFT1Y0NmHv7f0xs8zuzwLK1MSP+StIzNps+UZtifHXP+LHCzv3EJ9GvWhsk/lUo1BlX2aCJQqBg8RRtazsCLcl+7BHiR7/kSa5w4A9qXG0DFyDK8dmkW6teTXAtBuIeUodiUCEQkSkZ9F5IDtvUoBbZqJSFSu10URedFWN0FEjuWqu8eeeJQqLXX9PJjVyYKX3+Jryq1k8c+jc2mxeTQRSftKNIZdp3ax78zVY1g8LAxsNrBEj6nKJ3uvCMYDvxhjmgC/2LavYYzZZ4xpb4xpD3QCLgO5//V8dKXeGLPczniUKjUWDy+ib5nGwKA78tUdSoul89Zn+VPMf0gtofWS83YL9W7Ymyq++f4WU+qG7E0EA4E5ts9zgEE3aN8LOGiMOWLncZVyCSHelfm+zZssavk3KnsGXVNnsPLRsQU02/Qk6y9EO/zYebuFHmz5oMOPodyDvYmgujEmAcD2Xu0G7YcB8/OUjRORHSLyRUFdS1eIyGgRiRCRiFOnTtkXtVIONjikBwe7zGJo1T756uLSjxIe9Txj90/lclaqQ463K3EXe07vydn28vBiUPMb/R2mVMFumAhEZJWIRBfwKlZnpIh4A/cBuf+MmQ40AtoDCcC/CtvfGDPDGBNmjAkLCQkpzqGVKhVBlkp83eo1fmz9LsFeVfPUGqYlfEuTTU+w+vx2u4+V92qgV4NeBPkGFdJaqeu7YSIwxvQ2xrQu4LUEOCkiNQFs79eblasfsNUYk/PMvjHmpDEmyxhjBT4DOtv34yjlfP2Du3GwyyxGVMt/78PxjOPctv2PPLt/CilZaTd9jLxrD+jdQsoe9nYNLQVG2T6PApZcp+1w8nQLXUkiNoMBx3ekKuUEgV7+fNniz6xs8z7VvKrnqTVMT/iOZpufYtPF3cX+7j2n9rDr1K6cbU/x1G4hZRd7E8FEoI+IHAD62LYRkVoiknMHkIj42eoX5dl/kojsFJEdwB3AH+2MRymX0icojJguX/Bk9fw9qUfTj9Jt23O8fPAz0opxZ1FkQiReHleny+7VsBfBfsEOiVe5J3H1BTcKEhYWZiIiIm5u5wULoEYNxwakVBH8dm4bw3a/R2LmyXx1jXwa8G2r12jv3/j6X3LiBAwbxtmUsyzZu4SFuxcytNVQHm3/aMkErcoVEYk0xoTlLdcni5UqJXdU6UBMly8YWa1/vrqDqYfpFDmGNw/PLdKcRUG+QTzW4TGW/2G5JgFlN00ESpWiAC8/5rZ4meWtJxLkeW13jpUs/h43i/YRY9l9KdY5ASq3pIlAKSfoF9yFmC6zeLCA5w52p+ynXcRotiUdckJkyh1pIlDKSapYAljY6jW+a/k2lTyuXUjGktWeV6Kqu9xqaKp80kSglJPdH9KTmC6zuadKDwA8TABB6c+x/xIM2pjKv2PSybAaEtPPURZv7lCuz+vGTZRSJS3EuzI/tnmb/yb+wsYznvwcH0SqFTIN/Dsmg1WJWRz0+jOpmRd5cOVWhrQawi21bkFEnB26Kgf09lGlXNDhS1Ze3plG5HkrABlyjOM+T+fUe4gHCS8lUK3ijab3UuoqvX1UqTKkQUUPvuniw2vNvPEWsJJEQObV20671u6hSUA5jCYCpVyUpwijG1j48VZf2vrX5bLnhpy6Q3Gt+WbLUR0zUA6hiUApF9c0wIM/No/DyyMFAG9rI7zTevGX73bw9JeRnEm++cnrlAJNBEqVCX2DwzgbvogF1d/jVv8ZCN4ArNx9krv/vZpf9+aftkKpotJEoFQZ4efpw0OV67H8hdt4pFv9nPLTyek8PjuC1xbv5FLajaenUCovTQRKlTG+3p78bWBrZj92CyEBFXLK/7spjv6T17A17pwTo1NlkSYCpcqo25tVY8WLPenb6urt0LFnLjPk0w18+PN+MrKsToxOlSWaCJQqw4IqejN9REc+GNIO/wrZz4dmWQ2TfznAg9PXc/BUspMjVGWBJgKlyjgR4cFOdfjfCz3oHKyCtPMAABQRSURBVHp13eLt8RfoP3kNX248oreZquvSRKBUOVE3yI/5o7syvl9zLJ7ZU0+kZlj56/fRPDZ7C4kXU50coXJVmgiUKkc8PYQxtzViydjuNK3un1P++75T3P3v1fwUneDE6JSr0kSgVDnUslYllo7rzpPdG+SUnbucwZivtvLywu0k622mKhdNBEqVUz4WT94Y0JL/PtmFmoE+OeXfRsYzYPIath8978TolCuxKxGIyBAR2SUiVhHJN6NdrnZ9RWSfiMSIyPhc5UEi8rOIHLC9V7EnHqVUfrc2rspPL/bkvna1cspiz1zmgenrmfZ7DFlWHUh2d/ZeEUQD9wOrC2sgIp7AVKAf0BIYLiItbdXjgV+MMU2AX2zbSikHC/S1MHl4Bz566OptpplWw6Sf9jHi800kXEhxcoTKmexKBMaYPcaYfTdo1hmIMcYcMsakAwuAgba6gcAc2+c5wCB74lFKXd/gDnVY/nwPOtSrnFO24dAZ+v57jQ4ku7HSGCOoDRzNtR1vKwOoboxJALC96wTrSpWwesF+fPN0N56/szEetgXOLqRkDyS/umgnl9N1INnd3DARiMgqEYku4DXwRvte+YoCyordKSkio0UkQkQiTp06VdzdlVK5WDw9+NNdzVgwuhu1cg0kz98cx4BP1hJ97IITo1Ol7YaJwBjT2xjTuoDXkiIeIx6om2u7DnDc9vmkiNQEsL0nXieOGcaYMGNMWEhISBEPrZS6ns4NgvjfCz3p37ZmTtmhU5cYPG0dn60+hFUHkt1CaXQNbQGaiEgDEfEGhgFLbXVLgVG2z6OAoiYXpZSDBPpZmDK8A5MebIuftycAGVmGfyzfw6hZm/WJZDdg7+2jg0UkHugGLBORFbbyWiKyHMAYkwmMA1YAe4BvjDG7bF8xEegjIgeAPrZtpVQpExGGhtVl2fM9aFsnMKd8zYHT9P14Dat268I35ZmUxcmowsLCTERExM3tvGAB1Khx43ZKuaITJ2DYsBI9RHqmlY9W7efT/ztI7l8PI7vW5/X+LfCxeJbo8VXJEZFIY0y+Z770yWKl1DW8vTx4pW9z5j3ZhRqVrg4kf7nxCPd+spY9CRedGJ0qCZoIlFIFurVRVf73Qg/ublU9p+xAYjIDp67jK53aulzRRKCUKlSVit58OqIT/7y/Db62LqH0TCtvfB/Ns/O2ciElw8kRKkfQRKCUui4RYXjnevzwXHea1wjIKf9f9Anu+VjXSC4PNBEopYqkcTV/vh8bzoiu9XLKjp1PYeinG/j0/w7qMwdlmCYCpVSR+Vg8eWdQG6b/oSMBPlcnr5v4v708OnsLp5PTnByhuhmaCJRSxdavTc18k9et3n+Kfh+vYV3MaSdGpm6GJgKl1E2pG5Q9ed2Y2xrllJ1KSmPEzE18sGIfmVlWJ0anikMTgVLqplk8PRjfrzlzH+9MVX9vAIyBKb/FMGzGRo6f13UOygJNBEopu/VsGsLyF3oQ3jg4pyziyDn6fbyGlbtOODEyVRSaCJRSDlEtwIe5j3fhz3c3w9O20MGFlAxGfxnJhKW7SMvMcnKEqjCaCJRSDuPpIYy9ozFfj+56zToHs9fHcv+09Rw+fcmJ0anCaCJQSjlcWGgQy1/owV0tr05Psev4RQZMXsP32445MTJVEE0ESqkSUdnPm/+M7MTb97XC2zP7V82l9Cxe/DqKVxftJDVDu4pchSYCpVSJERFG3RrK4rG30rBqxZzy+ZvjuH/aemK1q8glaCJQSpW4VrUCWfpcd+5tVyunbHfCRQZ8spblOxOcGJkCTQRKqVLiX8GLycPa886g1jldRclpmTw7b6veVeRkmgiUUqVGRBjRtT6Lnr2VekF+OeWz18cy9NMNHD172YnRuS9NBEqpUte6diA/Pt+dvq2uLhu7Pf4C/Sev4WddH7nUaSJQSjlFJR8L00d05M0BLfGyPYB2MTWTp+ZG8O7yPWToXEWlRhOBUsppRITHuzfgmzHdqF3ZN6d8xupDDJuxkYQLOldRabArEYjIEBHZJSJWEQkrpE1dEflNRPbY2r6Qq26CiBwTkSjb6x574lFKlU0d61Xhx+e6c2fzajllkUfOcc/Ha/h9X6ITI3MP9l4RRAP3A6uv0yYTeMkY0wLoCowVkZa56j8yxrS3vZbbGY9SqoyqUtGbzx8JY3y/5jlzFZ27nMFjs7fotNYlzK5EYIzZY4zZd4M2CcaYrbbPScAeoLY9x1VKlU8eHsKY2xqxYHRXqleqAFyd1nrEzE0kXkx1coTlU6mOEYhIKNAB2JSreJyI7BCRL0SkynX2HS0iESIScerUqRKOVCnlTLeEBrHs+R70aFI1p2zjobPcM3kt6w/qCmiOdsNEICKrRCS6gNfA4hxIRPyB74AXjTEXbcXTgUZAeyAB+Fdh+xtjZhhjwowxYSEhIcU5tFKqDKrqX4HZj3XmT32aItk9RZxOTmPE55uY/vtBjDHODbAc8bpRA2NMb3sPIiIWspPAPGPMolzffTJXm8+AH+09llKq/PD0EJ7v1YRO9avwwoJtnE5Ox2rgvZ/2si3uHB8MbUclH4uzwyzzSrxrSEQEmAnsMcZ8mKeuZq7NwWQPPiul1DXCG1dl2fM9CKt/tfd45e6TDJyyjr0nLl5nT1UU9t4+OlhE4oFuwDIRWWErryUiV+4ACgdGAncWcJvoJBHZKSI7gDuAP9oTj1Kq/KpeyYf5o7vyeHiDnLLDpy8xeOp6XePATjfsGroeY8xiYHEB5ceBe2yf1wJSyP4j7Tm+Usq9WDw9ePPelnSoV5lXvtvB5fQsUjKy1zjYFneO1/u3xNtLn5MtLj1jSqky5952tVgyNpyGIVfXOJiz4QjDZmzQp5FvgiYCpVSZ1KR6AEvGhtOv9dWJ67bGnWeA3mJabJoIlFJlVoCPhWl/6Mjr97TIeRr5zKV0Rny+iU//T28xLSpNBEqpMk1EeKpnQ+Y92YWq/tlPI1sNTPzfXsZ8FcnF1AwnR+j6NBEopcqFrg2DWfZ892tuMV2xK/sW030nkpwYmevTRKCUKjeu3GL6WHhoTtnh05cYNHUdS6L0FtPCaCJQSpUrFk8P3rq3FZOHd8DP2xOAlIwsXlgQxYSlu0jP1FlM89JEoJQql+5rV4vv89xiOnt9LMM/26izmOahiUApVW41td1imntt5Mgj57h3ylq2xp1zYmSuRROBUqpcC7Ctjfxqv+bY7jDl5MU0hv1nIws2xzk3OBehiUApVe6JCE/f1og5j3emsl/2bKXpWVbGL9rJ64t3uv24gSYCpZTb6NEkhB/Gdad5jYCcsnmb4tx+3EATgVLKrdQN8mPRs7cyoO3VWfDdfdxAE4FSyu34eXvxyfAOBY4bfL3F/cYNNBEopdxS7nGDQN+r4wavfLeTN753r3EDTQRKKbdW0LjBVxvjePizjSQmuce4gSYCpZTbqxecf9wg4sg57v1kLdvcYNxAE4FSSlH4uMFDbjBuoIlAKaVsrowbzH7MvcYNNBEopVQePZu617iBXYlARIaIyC4RsYpI2HXaxYrIThGJEpGIXOVBIvKziBywvVcp7DuUUqo0XRk36J9n3OC+T9axM/6CEyNzPHuvCKKB+4HVRWh7hzGmvTEmd8IYD/xijGkC/GLbVkopl+Dn7cWU4R0Yn2vc4MTFVIb8Zz0/bD/u3OAcyK5EYIzZY4zZZ8dXDATm2D7PAQbZE49SSjmaiDDmtkbMeqwzAT5eAKRmWHlu/jY+WLEPq7Xsr4tcWmMEBlgpIpEiMjpXeXVjTAKA7b1aKcWjlFLFclvTEJaMDadh1avrG0z5LYYxX0VyKS3TiZHZ74aJQERWiUh0Aa+BxThOuDGmI9APGCsiPYsbqIiMFpEIEYk4depUcXdXSim7NQzxZ/HYcHo2DckpW7n7JA9MX8/Rs5edGJl9bpgIjDG9jTGtC3gtKepBjDHHbe+JwGKgs63qpIjUBLC9J17nO2YYY8KMMWEhISGFNVNKqRIV6Gvhi1FhPNm9QU7Z3hNJDJy6jk2HzjgxsptX4l1DIlJRRAKufAbuInuQGWApMMr2eRRQ5OSilFLO4uXpwRsDWjLpwbZ4e2b/Gj17KZ0/fL6J+WVwsRt7bx8dLCLxQDdgmYissJXXEpHltmbVgbUish3YDCwzxvxkq5sI9BGRA0Af27ZSSpUJQ8PqMn90F6r6ewOQaTW8umgnby2JJjOr7Dx8JsaUvRHvsLAwExERceOGBVmwAGrUuHE7pVzRiRMwbJizo1B5HDufwlNzItidcDGnLLxxMFMf7khlP28nRnYtEYnMcws/oE8WK6WU3WpX9uXbZ7pxT5urf2SuiznDoKnriElMcmJkRaOJQCmlHCD74bOOvNi7SU5Z7JnLDJ66nt/2FnofjEvQRKCUUg7i4SG82Lsp0//QEV+LJwBJaZk8PmcLM1YfxFW74jURKKWUg/VrU5Nvn+lG7cq+ABgD7y7fy0sLt5OakeXk6PLTRKCUUiWgVa1AlowLJ6z+1bk0F209xsOfbeR0cpoTI8tPE4FSSpWQqv4VmPdUF4aG1ckp2xp3nkFT17H/pOsMImsiUEqpElTBy5P3HmjLG/1bILYZTOPPpfDAtPX8337XmC5HE4FSSpUwEeHJHg35/JEw/LxzDSLP3sKXG484OTpNBEopVWp6tajOt2NupWagDwBZVsNfv4/m7R92keXE6aw1ESilVClqWasSS8aG07ZOYE7ZrHWxPDU3gmQnTWetiUAppUpZtUo+fD26G/1aX30S+de9iTw4fT3HzqeUejyaCJRSygl8vT2Z+nBHnrm9UU7Z3hNJDJyyju1Hz5dqLJoIlFLKSTw8hFf6NmfSg22xeGbfUnQ6OY2HZmxg+c6E0ouj1I6klFKqQEPD6vLlE12o7GcBstdEfnbeVqb+FlMq01JoIlBKKRfQtWEwi58Np0GuNZHfX7GPlxfuID2zZNc20ESglFIuokHViix+9la6NgzKKftuazwjZm7i3KX0EjuuJgKllHIhlf28mft4F4Z0ujotxebDZxk8bR2HTiWXyDE1ESillIvx9vJg0oNteaVv85yy2DOXGTxtPesPnnb48TQRKKWUCxIRnrm9EZ+O6IiPJftX9YWUDB6ZuZmfok849FiaCJRSyoX1bV2Tb57uRrWACgAE+3vTrm7gDfYqHk0ESinl4trWqcz3Y8PpVL8Knz9yCzUDfR36/XYlAhEZIiK7RMQqImGFtGkmIlG5XhdF5EVb3QQROZar7h574lFKqfKqVmVfvh3TjTZ1HHs1AOBl5/7RwP3AfwprYIzZB7QHEBFP4BiwOFeTj4wxH9gZh1JKlXtyZUEDB7MrERhj9kCxgusFHDTGOH8CbqWUUkDpjxEMA+bnKRsnIjtE5AsRqVLQTgAiMlpEIkQk4tQp11jVRymlyoMbJgIRWSUi0QW8BhbnQCLiDdwHLMxVPB1oRHbXUQLwr8L2N8bMMMaEGWPCQkJCinNopZRS13HDriFjTG8HHasfsNUYczLXd+d8FpHPgB8ddCyllFJFVJpdQ8PJ0y0kIjVzbQ4me/BZKaVUKbL39tHBIhIPdAOWicgKW3ktEVmeq50f0AdYlOcrJonIThHZAdwB/NGeeJRSShWflMZc144mIqcAV73zqCrg+MlAHEfjs4/GZx+Nz372xFjfGJNvkLVMJgJXJiIRxpgCH65zBRqffTQ++2h89iuJGHWKCaWUcnOaCJRSys1pInC8Gc4O4AY0PvtofPbR+Ozn8Bh1jEAppdycXhEopZSb00SglFJuThOBA4lIrO0BuSgRiXCBeL4QkUQRic5VFiQiP4vIAdt7oRP9OSk+l1mjQkTqishvIrLHtu7GC7ZylziH14nPJc6hiPiIyGYR2W6L721buaucv8Lic4nzlytOTxHZJiI/2rYdfv50jMCBRCQWCDPGuMQDKSLSE0gG5hpjWtvKJgFnjTETRWQ8UMUY84oLxTcBSHaFNSpsU6DUNMZsFZEAIBIYBDyKC5zD68Q3FBc4h5I9P31FY0yyiFiAtcALZK9h4grnr7D4+uIC5+8KEfkTEAZUMsYMKIl/w3pFUI4ZY1YDZ/MUDwTm2D7PIfsXh1MUEp/LMMYkGGO22j4nAXuA2rjIObxOfC7BZEu2bVpsL4PrnL/C4nMZIlIH6A98nqvY4edPE4FjGWCliESKyGhnB1OI6saYBMj+RQJUc3I8BSnSGhWlSURCgQ7AJlzwHOaJD1zkHNq6NaKAROBnY4xLnb9C4gMXOX/Av4G/ANZcZQ4/f5oIHCvcGNOR7Cm3x9q6PlTxFHmNitIiIv7Ad8CLxpiLzo4nrwLic5lzaIzJMsa0B+oAnUWktbNiKUgh8bnE+RORAUCiMSaypI+licCBjDHHbe+JZK/L3Nm5ERXopK1v+Uofc6KT47mGMeak7R+nFfgMJ59DW9/xd8A8Y8yV2XNd5hwWFJ+rnUNbTOeB38nuf3eZ83dF7vhc6PyFA/fZxh4XAHeKyFeUwPnTROAgIlLRNmCHiFQE7sI111dYCoyyfR4FLHFiLPmIC61RYRtMnAnsMcZ8mKvKJc5hYfG5yjkUkRARqWz77Av0BvbiOuevwPhc5fwZY141xtQxxoSSvczvr8aYEZTA+dO7hhxERBqSfRUA2Su//dcY8w8nhoSIzAduJ3va2pPAW8D3wDdAPSAOGGKMccqAbSHx3U72JbkBYoGnr/SHOiG+7sAaYCdX+2hfI7sf3unn8DrxDccFzqGItCV7MNOT7D86vzHG/E1EgnGN81dYfF/iAucvNxG5HXjZdteQw8+fJgKllHJz2jWklFJuThOBUkq5OU0ESinl5jQRKKWUm9NEoJRSbk4TgVJKuTlNBEop5eb+HzOZw/REVv+YAAAAAElFTkSuQmCC\n",
+      "text/plain": [
+       "
    "
+      ]
+     },
+     "metadata": {
+      "filenames": {
+       "image/png": "/Users/hjensen/Teaching/FYS-STK4150/doc/src/LectureNotes/_build/jupyter_execute/chapter10_9_423.png"
+      },
+      "needs_background": "light"
+     },
+     "output_type": "display_data"
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Time:  6.260979576\n"
+     ]
+    }
+   ],
+   "source": [
+    "def rnn_2layers(length_of_sequences, batch_size = None, stateful = False):\n",
+    "    \"\"\"\n",
+    "        Inputs:\n",
+    "            length_of_sequences (an int): the number of y values in \"x data\".  This is determined\n",
+    "                when the data is formatted\n",
+    "            batch_size (an int): Default value is None.  See Keras documentation of SimpleRNN.\n",
+    "            stateful (a boolean): Default value is False.  See Keras documentation of SimpleRNN.\n",
+    "        Returns:\n",
+    "            model (a Keras model): The recurrent neural network that is built and compiled by this\n",
+    "                method\n",
+    "        Builds and compiles a recurrent neural network with two hidden layers and returns the model.\n",
+    "    \"\"\"\n",
+    "    # Number of neurons in the input and output layers\n",
+    "    in_out_neurons = 1\n",
+    "    # Number of neurons in the hidden layer, increased from the first network\n",
+    "    hidden_neurons = 500\n",
+    "    # Define the input layer\n",
+    "    inp = Input(batch_shape=(batch_size, \n",
+    "                length_of_sequences, \n",
+    "                in_out_neurons))  \n",
+    "    # Create two hidden layers instead of one hidden layer.  Explicitly set the activation\n",
+    "    # function to be the sigmoid function (the default value is hyperbolic tangent)\n",
+    "    rnn1 = SimpleRNN(hidden_neurons, \n",
+    "                    return_sequences=True,  # This needs to be True if another hidden layer is to follow\n",
+    "                    stateful = stateful, activation = 'sigmoid',\n",
+    "                    name=\"RNN1\")(inp)\n",
+    "    rnn2 = SimpleRNN(hidden_neurons, \n",
+    "                    return_sequences=False, activation = 'sigmoid',\n",
+    "                    stateful = stateful,\n",
+    "                    name=\"RNN2\")(rnn1)\n",
+    "    # Define the output layer as a dense neural network layer (standard neural network layer)\n",
+    "    #and add it to the network immediately after the hidden layer.\n",
+    "    dens = Dense(in_out_neurons,name=\"dense\")(rnn2)\n",
+    "    # Create the machine learning model starting with the input layer and ending with the \n",
+    "    # output layer\n",
+    "    model = Model(inputs=[inp],outputs=[dens])\n",
+    "    # Compile the machine learning model using the mean squared error function as the loss \n",
+    "    # function and an Adams optimizer.\n",
+    "    model.compile(loss=\"mean_squared_error\", optimizer=\"adam\")  \n",
+    "    return model\n",
+    "\n",
+    "# Check to make sure the data set is complete\n",
+    "assert len(X_tot) == len(y_tot)\n",
+    "\n",
+    "# This is the number of points that will be used in as the training data\n",
+    "dim=12\n",
+    "\n",
+    "# Separate the training data from the whole data set\n",
+    "X_train = X_tot[:dim]\n",
+    "y_train = y_tot[:dim]\n",
+    "\n",
+    "\n",
+    "# Generate the training data for the RNN, using a sequence of 2\n",
+    "rnn_input, rnn_training = format_data(y_train, 2)\n",
+    "\n",
+    "\n",
+    "# Create a recurrent neural network in Keras and produce a summary of the \n",
+    "# machine learning model\n",
+    "model = rnn_2layers(length_of_sequences = 2)\n",
+    "model.summary()\n",
+    "\n",
+    "# Start the timer.  Want to time training+testing\n",
+    "start = timer()\n",
+    "# Fit the model using the training data genenerated above using 150 training iterations and a 5%\n",
+    "# validation split.  Setting verbose to True prints information about each training iteration.\n",
+    "hist = model.fit(rnn_input, rnn_training, batch_size=None, epochs=150, \n",
+    "                 verbose=True,validation_split=0.05)\n",
+    "\n",
+    "\n",
+    "# This section plots the training loss and the validation loss as a function of training iteration.\n",
+    "# This is not required for analyzing the couple cluster data but can help determine if the network is\n",
+    "# being overtrained.\n",
+    "for label in [\"loss\",\"val_loss\"]:\n",
+    "    plt.plot(hist.history[label],label=label)\n",
+    "\n",
+    "plt.ylabel(\"loss\")\n",
+    "plt.xlabel(\"epoch\")\n",
+    "plt.title(\"The final validation loss: {}\".format(hist.history[\"val_loss\"][-1]))\n",
+    "plt.legend()\n",
+    "plt.show()\n",
+    "\n",
+    "# Use the trained neural network to predict more points of the data set\n",
+    "test_rnn(X_tot, y_tot, X_tot[0], X_tot[dim-1])\n",
+    "# Stop the timer and calculate the total time needed.\n",
+    "end = timer()\n",
+    "print('Time: ', end-start)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Other Types of Recurrent Neural Networks\n",
+    "\n",
+    "Besides a simple recurrent neural network layer, there are two other\n",
+    "commonly used types of recurrent neural network layers: Long Short\n",
+    "Term Memory (LSTM) and Gated Recurrent Unit (GRU).  For a short\n",
+    "introduction to these layers see \n",
+    "and .\n",
+    "\n",
+    "The first network created below is similar to the previous network,\n",
+    "but it replaces the SimpleRNN layers with LSTM layers.  The second\n",
+    "network below has two hidden layers made up of GRUs, which are\n",
+    "preceeded by two dense (feeddorward) neural network layers.  These\n",
+    "dense layers \"preprocess\" the data before it reaches the recurrent\n",
+    "layers.  This architecture has been shown to improve the performance\n",
+    "of recurrent neural networks (see the link above and also\n",
+    "."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Model: \"functional_5\"\n",
+      "_________________________________________________________________\n",
+      "Layer (type)                 Output Shape              Param #   \n",
+      "=================================================================\n",
+      "input_3 (InputLayer)         [(None, 2, 1)]            0         \n",
+      "_________________________________________________________________\n",
+      "dnn (Dense)                  (None, 2, 125)            250       \n",
+      "_________________________________________________________________\n",
+      "dnn1 (Dense)                 (None, 2, 125)            15750     \n",
+      "_________________________________________________________________\n",
+      "RNN1 (GRU)                   (None, 2, 250)            282750    \n",
+      "_________________________________________________________________\n",
+      "RNN (GRU)                    (None, 250)               376500    \n",
+      "_________________________________________________________________\n",
+      "dense (Dense)                (None, 1)                 251       \n",
+      "=================================================================\n",
+      "Total params: 675,501\n",
+      "Trainable params: 675,501\n",
+      "Non-trainable params: 0\n",
+      "_________________________________________________________________\n",
+      "Epoch 1/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.2339"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 1s 653ms/step - loss: 0.2339 - val_loss: 0.5383\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 2/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.1652"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 0.1652 - val_loss: 0.3519\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 3/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.1008"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 0.1008 - val_loss: 0.1726\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 4/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0430"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0430 - val_loss: 0.0348\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 5/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0070"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 20ms/step - loss: 0.0070 - val_loss: 0.0070\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 6/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0194"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0194 - val_loss: 0.0534\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 7/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0503"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0503 - val_loss: 0.0415\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 8/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0430"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0430 - val_loss: 0.0088\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 9/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0204"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0204 - val_loss: 0.0014\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 10/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0061"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0061 - val_loss: 0.0239\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 11/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0052"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 0.0052 - val_loss: 0.0584\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 12/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0120"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 0.0120 - val_loss: 0.0860\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 13/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0191"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0191 - val_loss: 0.0977\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 14/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0225"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0225 - val_loss: 0.0928\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 15/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0213"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0213 - val_loss: 0.0754\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 16/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0166"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0166 - val_loss: 0.0516\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 17/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0106"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 0.0106 - val_loss: 0.0278\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 18/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0056"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0056 - val_loss: 0.0099\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 19/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0032"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0032 - val_loss: 0.0011\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 20/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0042"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0042 - val_loss: 4.2492e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 21/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0072"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0072 - val_loss: 0.0028\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 22/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0096"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0096 - val_loss: 0.0035\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 23/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0094"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0094 - val_loss: 0.0016\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 24/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0068"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0068 - val_loss: 2.1770e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 25/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0037"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0037 - val_loss: 0.0014\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 26/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0017"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 20ms/step - loss: 0.0017 - val_loss: 0.0061\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 27/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0015"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0015 - val_loss: 0.0121\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 28/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0025"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0025 - val_loss: 0.0166\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 29/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0037"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0037 - val_loss: 0.0177\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 30/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0042"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0042 - val_loss: 0.0151\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 31/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0037"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0037 - val_loss: 0.0101\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 32/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0024"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0024 - val_loss: 0.0046\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 33/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0011"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 0.0011 - val_loss: 8.8886e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 34/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 2.7391e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 2.7391e-04 - val_loss: 6.9638e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 35/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 3.0805e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 3.0805e-04 - val_loss: 0.0016\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 36/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 9.6065e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 9.6065e-04 - val_loss: 0.0037\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 37/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0016"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0016 - val_loss: 0.0045\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 38/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0016"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0016 - val_loss: 0.0035\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 39/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0011"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0011 - val_loss: 0.0016\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 40/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 3.6501e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 3.6501e-04 - val_loss: 2.5101e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 41/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 5.2387e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 5.2387e-05 - val_loss: 4.9809e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 42/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 2.7319e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 2.7319e-04 - val_loss: 5.0112e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 43/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 7.0581e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 7.0581e-04 - val_loss: 7.5034e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 44/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 9.3238e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 9.3238e-04 - val_loss: 5.0648e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 45/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 8.0866e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 8.0866e-04 - val_loss: 9.4408e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 46/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 4.7288e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 4.7288e-04 - val_loss: 5.9254e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 47/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.8597e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 20ms/step - loss: 1.8597e-04 - val_loss: 6.7079e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 48/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.2584e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 20ms/step - loss: 1.2584e-04 - val_loss: 0.0017\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 49/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 2.6867e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 2.6867e-04 - val_loss: 0.0024\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 50/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 4.3826e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 4.3826e-04 - val_loss: 0.0025\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 51/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 4.6733e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 4.6733e-04 - val_loss: 0.0018\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 52/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 3.3438e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 3.3438e-04 - val_loss: 9.4701e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 53/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.5339e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 1.5339e-04 - val_loss: 2.6460e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 54/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 5.4331e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 5.4331e-05 - val_loss: 6.5156e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 55/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 7.7712e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 7.7712e-05 - val_loss: 6.1830e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 56/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.6479e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 1.6479e-04 - val_loss: 1.8163e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 57/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 2.2423e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 2.2423e-04 - val_loss: 1.9299e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 58/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 2.0515e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 2.0515e-04 - val_loss: 9.4929e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 59/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.2535e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 1.2535e-04 - val_loss: 6.2223e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 60/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 4.6826e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 4.6826e-05 - val_loss: 3.4402e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 61/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 2.2165e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 2.2165e-05 - val_loss: 1.7489e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 62/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 5.5691e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 5.5691e-05 - val_loss: 3.1733e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 63/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.0595e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 1.0595e-04 - val_loss: 3.4700e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 64/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.2525e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 1.2525e-04 - val_loss: 2.4500e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 65/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 9.9924e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 9.9924e-05 - val_loss: 9.6135e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 66/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 5.5901e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 5.5901e-05 - val_loss: 6.8225e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 67/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 2.9911e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 2.9911e-05 - val_loss: 1.7879e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 68/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 3.7141e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 3.7141e-05 - val_loss: 8.5801e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 69/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 6.3071e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 6.3071e-05 - val_loss: 1.3386e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 70/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 8.0889e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 8.0889e-05 - val_loss: 1.1862e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 71/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 7.5159e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 20ms/step - loss: 7.5159e-05 - val_loss: 5.8102e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 72/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 5.2129e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 5.2129e-05 - val_loss: 7.2266e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 73/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 3.1149e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 20ms/step - loss: 3.1149e-05 - val_loss: 7.1946e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 74/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 2.6803e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 20ms/step - loss: 2.6803e-05 - val_loss: 5.2427e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 75/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 3.7385e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 24ms/step - loss: 3.7385e-05 - val_loss: 1.0039e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 76/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 4.8700e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 4.8700e-05 - val_loss: 1.1159e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 77/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 4.8312e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 20ms/step - loss: 4.8312e-05 - val_loss: 8.1357e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 78/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 3.6325e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 21ms/step - loss: 3.6325e-05 - val_loss: 3.6670e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 79/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 2.3281e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 2.3281e-05 - val_loss: 6.9527e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 80/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.9011e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 1.9011e-05 - val_loss: 6.2134e-08\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 81/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 2.4047e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 20ms/step - loss: 2.4047e-05 - val_loss: 3.3985e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 82/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 3.0829e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 25ms/step - loss: 3.0829e-05 - val_loss: 3.1615e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 83/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 3.1812e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 20ms/step - loss: 3.1812e-05 - val_loss: 1.7273e-08\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 84/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 2.6124e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 2.6124e-05 - val_loss: 6.8793e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 85/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.9199e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 1.9199e-05 - val_loss: 3.3150e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 86/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.6832e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 1.6832e-05 - val_loss: 7.2050e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 87/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.9787e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 1.9787e-05 - val_loss: 1.0364e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 88/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 2.3840e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 2.3840e-05 - val_loss: 1.1049e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 89/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 2.4491e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 24ms/step - loss: 2.4491e-05 - val_loss: 9.1017e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 90/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 2.1253e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 20ms/step - loss: 2.1253e-05 - val_loss: 5.8702e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 91/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.7418e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 20ms/step - loss: 1.7418e-05 - val_loss: 2.9923e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 92/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.6259e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 1.6259e-05 - val_loss: 1.2810e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 93/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.7905e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 1.7905e-05 - val_loss: 6.0232e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 94/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.9781e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 1.9781e-05 - val_loss: 5.3134e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 95/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.9548e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 1.9548e-05 - val_loss: 9.3216e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 96/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.7296e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 1.7296e-05 - val_loss: 1.9159e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 97/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.5052e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 1.5052e-05 - val_loss: 3.3857e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 98/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.4498e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 1.4498e-05 - val_loss: 4.7879e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 99/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.5386e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 1.5386e-05 - val_loss: 5.4022e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 100/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.6115e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 1.6115e-05 - val_loss: 4.8949e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 101/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.5549e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 1.5549e-05 - val_loss: 3.5722e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 102/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.4079e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 1.4079e-05 - val_loss: 2.1059e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 103/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.2963e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 20ms/step - loss: 1.2963e-05 - val_loss: 1.0307e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 104/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.2911e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 1.2911e-05 - val_loss: 4.7690e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 105/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.3470e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 1.3470e-05 - val_loss: 2.9656e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 106/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.3678e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 20ms/step - loss: 1.3678e-05 - val_loss: 3.4672e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 107/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.3126e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 1.3126e-05 - val_loss: 6.0939e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 108/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.2266e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 20ms/step - loss: 1.2266e-05 - val_loss: 1.0808e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 109/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1796e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 1.1796e-05 - val_loss: 1.6232e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 110/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1902e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 22ms/step - loss: 1.1902e-05 - val_loss: 1.9843e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 111/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.2143e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 1.2143e-05 - val_loss: 1.9755e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 112/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.2022e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 1.2022e-05 - val_loss: 1.6197e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 113/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.1506e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 20ms/step - loss: 1.1506e-05 - val_loss: 1.1183e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 114/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.0982e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 1.0982e-05 - val_loss: 6.8771e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 115/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.0778e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 1.0778e-05 - val_loss: 4.3037e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 116/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.0826e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 1.0826e-05 - val_loss: 3.4128e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 117/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.0803e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 1.0803e-05 - val_loss: 3.9190e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 118/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.0525e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 1.0525e-05 - val_loss: 5.7593e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 119/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.0114e-05"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 1.0114e-05 - val_loss: 8.7378e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 120/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 9.8203e-06"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 9.8203e-06 - val_loss: 1.2031e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 121/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 9.7284e-06"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 9.7284e-06 - val_loss: 1.4340e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 122/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 9.6966e-06"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 9.6966e-06 - val_loss: 1.4696e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 123/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 9.5506e-06"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 20ms/step - loss: 9.5506e-06 - val_loss: 1.3124e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 124/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 9.2787e-06"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 9.2787e-06 - val_loss: 1.0547e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 125/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 9.0185e-06"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 9.0185e-06 - val_loss: 8.0814e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 126/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 8.8755e-06"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 8.8755e-06 - val_loss: 6.4479e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 127/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 8.8137e-06"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 8.8137e-06 - val_loss: 5.8674e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 128/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 8.7187e-06"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 8.7187e-06 - val_loss: 6.2909e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 129/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 8.5374e-06"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 8.5374e-06 - val_loss: 7.5345e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 130/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 8.3250e-06"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 8.3250e-06 - val_loss: 9.2095e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 131/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 8.1651e-06"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 8.1651e-06 - val_loss: 1.0688e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 132/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 8.0688e-06"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 8.0688e-06 - val_loss: 1.1327e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 133/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 7.9771e-06"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 7.9771e-06 - val_loss: 1.0835e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 134/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 7.8467e-06"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 7.8467e-06 - val_loss: 9.4450e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 135/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 7.6825e-06"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 7.6825e-06 - val_loss: 7.7252e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 136/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 7.5362e-06"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 7.5362e-06 - val_loss: 6.2324e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 137/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 7.4299e-06"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 7.4299e-06 - val_loss: 5.2841e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 138/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 7.3370e-06"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 7.3370e-06 - val_loss: 4.9452e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 139/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 7.2211e-06"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 7.2211e-06 - val_loss: 5.1307e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 140/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 7.0810e-06"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 7.0810e-06 - val_loss: 5.6509e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 141/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 6.9456e-06"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 6.9456e-06 - val_loss: 6.2259e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 142/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 6.8358e-06"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 6.8358e-06 - val_loss: 6.5445e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 143/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 6.7413e-06"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 6.7413e-06 - val_loss: 6.4017e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 144/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 6.6380e-06"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 6.6380e-06 - val_loss: 5.8084e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 145/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 6.5196e-06"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 6.5196e-06 - val_loss: 4.9715e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 146/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 6.4007e-06"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 6.4007e-06 - val_loss: 4.1612e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 147/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 6.2981e-06"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 6.2981e-06 - val_loss: 3.5793e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 148/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 6.2091e-06"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 6.2091e-06 - val_loss: 3.3095e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 149/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 6.1165e-06"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 6.1165e-06 - val_loss: 3.3423e-06\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 150/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 6.0133e-06"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 6.0133e-06 - val_loss: 3.6012e-06\n"
+     ]
+    },
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZxVdf3H8ddn7mzAzLAvMoCAosiimIhYiaYlouaeYi5ppj8zzSxN/fmrLCvrV7+2XyaZueWGqfWjJLVyQXMDkUUUEFFkWId9WGb//P74ngt3LneGgZnLDJ338/GYx9yzfc/nnnvv+Zzv93zPOebuiIhIfOW0dQAiItK2lAhERGJOiUBEJOaUCEREYk6JQEQk5pQIRERiTokgjZndamYPtlJZHczsL2a20cz+aGYXmNmzrVT2h2b26dYoK61cN7MDo9eTzOxbzZl3D9bTatsirdzjzKystcsV+XcWu0RgZptT/urNbFvK8AWtvLpzgN5Ad3f/nLs/5O4ntvI6ssbdr3T321pajpkNjJJGbkrZ+9S2aC4ze97Mys1sk5nNNrPTdzH/x8xsWvT9W2Vm16ZMGxiVt9XM5qcmfjPbz8ymmNnyaNsOTCv3p2b2nplVRMtenDKth5n9y8zWmtkGM3vVzD6RMn1S2u+kyswqMsQ+xMwq0w+czOxLZrYoWvZpM+ubYdn8KK6ytPGjzOyl6OCpzMy+nTLtP9Pi2hb9hnukzPNpM5tpZlvMbKmZnZsy7fho2iYzW2xmVzTymTyX/n01s6vNbEa0Le7LtNzuMLPrzGxl9D7vMbOCtOkTzezd6H28b2bHtHSdTXL32P4BHwKfTht3K/BgK5X/X61VVnNib6VyHTiwNecFBkbz5u6Fz/Q4oGxvfYcyrP/Q5PsEjgIqgP0ambcHsBq4ACgAioFDUqa/CvwM6ACcDWwAekbTegNXAUdH23ZgWtnfBYYSDvaOAtYDH4+mFQIHR9MMOANY19jnA9wH3JNh/LPAS6nfceDY6D0NB/KBO4EXMyx7CzAt/bMC3gF+ACSAA4AVwGmNxHUr8FzK8LBo3ROAXKA7cEA0LQ/YCPxH9J6PBDYDh6WVeUEUV4PvK3BWtJ3uBO5r4XdkPLAq2kZdgReAH6VM/wywBBgbfUalQGlWv7dt9YNpD380nggeAx6IfsTzgNEp0/sCTwDlwAfAVxsp+7tANVATfeEuAy4BXk6Zx4ErgfeiH+odgEXTDgCeA9YCa4CHgC5NxR6NHwusBBIp484E5kSvxxB2MBuiH9mvgfy0mA6MXt8HfD9l2g3RMsuBL6bNewrwFrAJWArcmrLcR9G8m6O/ozNsi48D06Mf63SinVY07QXgNuBf0WfyLNCjke1+HCk7F+CQaPkN0Wd5Wsq0kwk7ngpgGXB9NL4H8NdomXWEnV3OHny/xgCVwJhGpv8Q+EMj0w4CqoDilHEvAVemzZdLhkSQobwpwDcyjM8BPhuV0SvD9E7R9jk2bfxEwu/kVhomgp8Cd6T9XpxohxyNGwS8S9hhpyeCrcCwlOE/AjdniMuA94EvpIx7GLitkfffO4qjY8q46cD5KcOdgYWE31DGAxfg+2RIBMCpwKzoO/MKcGgTn8XDwA9Thk8AVqYMvwJctrvft5b8xa5pqJlOAx4FuhB+QL8GMLMc4C/AbEKWPgH4mpmNTy/A3b9D+KFPdvcid/99I+s6lXB0chhwLuFoAcIX/XbCD+kQoD/hR9ckd38N2AIcnzL684QvH0AdcB1hZ3d09B6u2lW5ZnYScD3haGUIkH5+YgtwMWGbnQJ82czOiKaNi/53ibbFq2lldwOeAn5FOIr7GfCUmXVPew+XAr0IR5rXNyPmPMLn9Wy03DXAQ2Z2cDTL74H/cPdiYAQh8QJ8AygDehJ2IP9J2DFgZr8xs9/sYr1/NbNK4HVCEprRyKxjgXVm9oqZrbZwPmlANG04sNjdU5tkZkfjd4uZdSB8x+aljZ9DSFRTgLvdfXWGxc8mHPRMS1muBPgeYTvttLroL3UYwvZN+l/CNt2WYflfABebWV70OR0N/CPDfMcQPpsnUsaNjeKba2YrzOzB6LuFu68CHgEuNbOEmR0N7A+8nLL8DwlH/CszrK9RZvYx4B5CbaM78FtgSnpzT4rhhM8yaTbQ28y6m1kCGA30jJrXyszs19FnmDVKBJm97O5T3b0O+ANhJw3hx9TT3b/n7tXuvhj4HeHoaE/9yN03uPtHwPPAKAB3X+Tuf3f3KncvJ+wcj21mmY8A5wOYWTHhyPeRqNw33f01d6919w8JX9rmlHsucK+7v+3uW0hLSu7+grvPdfd6d58Tra+58Z4CvOfuf4jiegSYTzhSTbrX3Re6+zbCkeioZpQ7FigibONqd3+OcKR/fjS9BhhmZiXuvt7dZ6aM3w/Y391r3P0ljw7V3P0qd28ycbr7qYRmnpOBZ9y9vpFZ+wFfAK4FBhBqmI9E04oItaNUG6Nyd9ckws7mmbQ4DwVKCEn25QzLEcX3QPL9R24Dfu/uSzPMPxU418wOjXZe3yY6EgcwszMJR9p/amR9fyWcW9tG+A783t2nNxLX4+6+OWVcP+AiQvIaQmhS+9+U6Y9E8VQRale3JN+DmY0GPpE2f3NdDvzW3V939zp3vz9ax9hG5k//bJOviwnJLY+wDY4hfM8PJzQzZ40SQWapRwRbgcLoxNH+QN/oBNsGM9tAOLLp3YrrKgIws15m9qiZLTOzTcCDhKP45ngYOCs6IjkLmOnuS6JyD4qOWFdG5f6wmeX2JTT5JC1JnWhmR9mOE6UbCU1ezY23b3p50XBpynDG7dScmNN2xKnlnk3YWS8xsxejo0SAnwCLgGejk4o3Ne9t7BAlkL8B483stEZm2wb8yd2nu3sloTnx42bWmdCEVpI2fwmhmabZzOwnhKPxc9N25sk4K6PEe5OZHZa2bH9CMn8gZdwoQm3w55nW5+7/BL5DOFJfQmjCrADKzKwT8N+EmlmmWLsBTxNqG4WEWvB4M7sqbb4OwOeA+9OK2MaOA4bNhO/2ydEyQ4HJhFprPuGo/JtmdkpU0/8NcK2712aKbRf2B76Rtl/oT9hXXJBycvtv0fzpn23ydQU7akn/6+4r3H0N4SDw5D2Iq9mUCHbPUuADd++S8lfs7tn4kG4nHEkd6u4lwIU0rHI3yt3fIfwIJ9CwWQhC1Xc+MCQq9z+bWe4Kwpc7aUDa9IcJTQz93b0z4Sg0We6ubnG7nPBjSjWA0G7fEsuB/tEPfadyox3w6YRmoz8Tahq4e4W7f8PdBxNqJV83sxP2MIZcwvmeTObQcNskXxuhGWdwVKNLOoy05p2mmNl3Cd+BE9190y5mzwMGp427GHglqvkmHUc4+f+Rma0kNNGdbWbJ2hTufoe7D3H3XoSEkAu8TThKHwi8FC37JLBfdFAyMFp/nbs/ENUMywhNtOm/r7MI525eSBufvj1TjQAWuPszUa11AaE5cgJhRzwamBzFlayBlDWzt85S4Adp+4WO7v6Ih95xRdHfhGj+eexoZSB6vcrd17r7ekKz5F69LbQSwe55A9hkZjdauEYgYWYjzOzILKyrmHDksMHMSgknanfHw8BXCe3zf0wrdxOwOTpK+nIzy3sMuMTMhplZR8JRX3q869y90szGEBJQUjlQz847mqSpwEFm9nkzyzWz8wg9QP7azNga8zrh3MU3ozbn4wg79kctdF+8wMw6u3sNYZvUAZjZqWZ2oJlZyvi6Xa3MzIaa2YTou5FnZhcStv+LjSxyL3CmhS6TecC3CM2SG9x9IeHk43fMrDBqUjmUlDZxMysk9DYCKIiGk9NuJnwGn3H3tWlxjjWzT0bboIOZ3Uio1b6eFt/FhA4Dqe4iJLZR0d8kwg51fDKm6Ddh0fmOu4BfRju4twkHE8llv0ToPTOKsDNdGIqwz5tZjpn1Ac6jYXs6ZG6uSm7PS81scPQdvZEd36G3gCEWupCamR1AOD83m9A00zclrmTiOSK5TaLvZSGhN1Miep/J7qW/A66MasVmZp2imkZjzXgPAJdFv6WuhGaf1O18L3BN1CrQFfgaLf8tNC397HGc/mhG91HSuj4SvjCPEJoq1gOvpZfRRFmXsHOvoQNThu8j6qVDqLq+SUgGs4hOYDYVe9q6BxB2vk+ljR9HqBFsJrSTfq+xmNi519BN0fvO1GvoHEItpILwpf112nv/HiEhbCC0naZvi09G73dj9P+TKdNeAL7U2HZMe3/HpW2n4YQd8UZCD6Ezo/H5hGaI9YSd/fTkOgkn0z8kJJEy4Fsp5U0CJjWy7kMIO46K6H1OT64vmn4MsDltmS8TaijrCSe2+6d9914gNBcsSP+8o+3f4C9tWhU7emptBv4zmnYsYQdYQTiyfhEYl1b20dH7L870Xpv4jnchHJlvib4rt5PSg62pzyoadzw7eo+tJOxkU3v6lAK1NNJtmdC8Vh79/QHomjLtXEIyqog+1x+ToTcYGbo7R+8zfXvfmjL9pCjuZG+8Pza17YCvE5LgJsKOvyBlWh6hqWpDtA1+BRQ2d7+2J3/JrooiIhJTahoSEYk5JQIRkZhTIhARiTklAhGRmMvd9SztS48ePXzgwIFtHYaIyD7lzTffXOPuPTNN2+cSwcCBA5kxo7Fbt4iISCZmln71/nZqGhIRiTklAhGRmFMiEBGJuX3uHIGIxFNNTQ1lZWVUVla2dSjtWmFhIf369SMvL6/ZyygRiMg+oaysjOLiYgYOHEi4H6Ckc3fWrl1LWVkZgwYNavZyahoSkX1CZWUl3bt3VxJogpnRvXv33a41KRGIyD5DSWDX9mQbxScRrHoHnvs+bFnT1pGIiLQr8UkEaxbCtJ/A5lVtHYmI7KOKiprzhNR9T3wSQW708KbaqraNQ0SknYlRIoie6KdEICIt5O7ccMMNjBgxgpEjRzJ58mQAVqxYwbhx4xg1ahQjRozgpZdeoq6ujksuuWT7vD//+c/bOPqdxaf76PZEoD7IIvu67/5lHu8s39SqZQ7rW8J3Pju8WfM++eSTzJo1i9mzZ7NmzRqOPPJIxo0bx8MPP8z48eO55ZZbqKurY+vWrcyaNYtly5bx9ttvA7Bhw4ZWjbs1qEYgIrKbXn75Zc4//3wSiQS9e/fm2GOPZfr06Rx55JHce++93HrrrcydO5fi4mIGDx7M4sWLueaaa3j66acpKSlp6/B3EqMaQfIcgWoEIvu65h65Z0tjz3ofN24c06ZN46mnnuKiiy7ihhtu4OKLL2b27Nk888wz3HHHHTz22GPcc889eznipsWoRhAlgrrqto1DRPZ548aNY/LkydTV1VFeXs60adMYM2YMS5YsoVevXlx++eVcdtllzJw5kzVr1lBfX8/ZZ5/NbbfdxsyZM9s6/J3EqEagcwQi0jrOPPNMXn31VQ477DDMjP/+7/+mT58+3H///fzkJz8hLy+PoqIiHnjgAZYtW8all15KfX09ALfffnsbR78za6yK016NHj3a9+jBNJvL4acHwsk/hTGXt35gIpJV7777Loccckhbh7FPyLStzOxNdx+daf4YNQ2pRiAikkmMEoFOFouIZJLVRGBmJ5nZAjNbZGY3ZZh+nJltNLNZ0d+3sxZMIg8wqNXJYhGRVFk7WWxmCeAO4DNAGTDdzKa4+ztps77k7qdmK46UgEKtQDUCEZEGslkjGAMscvfF7l4NPAqcnsX17Vpuvi4oExFJk81EUAosTRkui8alO9rMZpvZ38ws41UiZnaFmc0wsxnl5eV7HpFqBCIiO8lmIsj0dIT0vqozgf3d/TDgf4E/ZyrI3e9y99HuPrpnz557HlFugS4oExFJk81EUAb0TxnuByxPncHdN7n75uj1VCDPzHpkLSLVCERkL2nq2QUffvghI0aM2IvRNC2biWA6MMTMBplZPjARmJI6g5n1sei5amY2JopnbdYiShToHIGISJqs9Rpy91ozuxp4BkgA97j7PDO7Mpo+CTgH+LKZ1QLbgImezUudcwtUIxD5d/C3m2Dl3NYts89ImPCjRiffeOON7L///lx11VUA3HrrrZgZ06ZNY/369dTU1PD973+f00/fvT4xlZWVfPnLX2bGjBnk5ubys5/9jE996lPMmzePSy+9lOrqaurr63niiSfo27cv5557LmVlZdTV1fGtb32L8847r0VvG7J8r6GouWdq2rhJKa9/Dfw6mzE0kFuoGoGI7JGJEyfyta99bXsieOyxx3j66ae57rrrKCkpYc2aNYwdO5bTTjtttx4gf8cddwAwd+5c5s+fz4knnsjChQuZNGkS1157LRdccAHV1dXU1dUxdepU+vbty1NPPQXAxo0bW+W9xeemcxBqBJWts+FEpA01ceSeLYcffjirV69m+fLllJeX07VrV/bbbz+uu+46pk2bRk5ODsuWLWPVqlX06dOn2eW+/PLLXHPNNQAMHTqU/fffn4ULF3L00Ufzgx/8gLKyMs466yyGDBnCyJEjuf7667nxxhs59dRTOeaYY1rlvcXnFhOgGoGItMg555zD448/zuTJk5k4cSIPPfQQ5eXlvPnmm8yaNYvevXtTWbl7zc+NtYZ//vOfZ8qUKXTo0IHx48fz3HPPcdBBB/Hmm28ycuRIbr75Zr73ve+1xtuKW40gX+cIRGSPTZw4kcsvv5w1a9bw4osv8thjj9GrVy/y8vJ4/vnnWbJkyW6XOW7cOB566CGOP/54Fi5cyEcffcTBBx/M4sWLGTx4MF/96ldZvHgxc+bMYejQoXTr1o0LL7yQoqIi7rvvvlZ5XzFLBKoRiMieGz58OBUVFZSWlrLffvtxwQUX8NnPfpbRo0czatQohg4duttlXnXVVVx55ZWMHDmS3Nxc7rvvPgoKCpg8eTIPPvggeXl59OnTh29/+9tMnz6dG264gZycHPLy8rjzzjtb5X3F53kEAH+5FuZPhRvea92gRCTr9DyC5tPzCJqSWwh1qhGIiKSKWdOQLigTkb1n7ty5XHTRRQ3GFRQU8Prrr7dRRJnFKxEkogvK3MNtqUVkn+Luu9VHv62NHDmSWbNm7dV17klzf8yahqLHVerGcyL7nMLCQtauXbtHO7q4cHfWrl1LYWHhbi0XrxpB6uMqk0lBRPYJ/fr1o6ysjBbdij4GCgsL6dev324tE7NEkHyAvWoEIvuavLw8Bg0a1NZh/FuKWdOQHmAvIpIuZokgWSNQzyERkaSYJgLVCEREkmKWCKKmIV1UJiKyXcwSgZqGRETSxSsRJNQ0JCKSLl6JQDUCEZGdxCwRqPuoiEi6mCUCXVAmIpIuZolANQIRkXQxSwQ6RyAiki6miUA1AhGRpJglgmTTkGoEIiJJ8UoEOblgObqyWEQkRbwSgVmoFahpSERku3glAoBEvpqGRERSxC8RqEYgItJAVhOBmZ1kZgvMbJGZ3dTEfEeaWZ2ZnZPNeIDQc0g1AhGR7bKWCMwsAdwBTACGAeeb2bBG5vsx8Ey2Ymkgt1CJQEQkRTZrBGOARe6+2N2rgUeB0zPMdw3wBLA6i7HsoBqBiEgD2UwEpcDSlOGyaNx2ZlYKnAlMaqogM7vCzGaY2Yzy8vKWRZVboHMEIiIpspkILMM4Txv+BXCju9c1VZC73+Xuo919dM+ePVsWlZqGREQayM1i2WVA/5ThfsDytHlGA4+aGUAP4GQzq3X3P2ctqtwC2LY+a8WLiOxrspkIpgNDzGwQsAyYCHw+dQZ3H5R8bWb3AX/NahIA1QhERNJkLRG4e62ZXU3oDZQA7nH3eWZ2ZTS9yfMCWZPI1zkCEZEU2awR4O5Tgalp4zImAHe/JJuxbKcagYhIAzG8sli9hkREUsUwERTqUZUiIilimAhUIxARSRXPRFBXBZ5+SYOISDzFMxGAThiLiERimAiSj6tU85CICMQyEUQ1gjqdMBYRgVgmAtUIRERSxS8RJHSOQEQkVfwSwfaTxaoRiIhALBNBsmlINQIREYhlIlDTkIhIqhgmAp0sFhFJFcNEkB/+q0YgIgLEMhGoRiAikiqGiUDnCEREUsUwEUQ1gjolAhERiGMi0AVlIiINxC8R6IIyEZEGYpgIdEGZiEiq+CWCRC5YQolARCQSv0QA0XOL1TQkIgKxTQT5qhGIiERimghUIxARSYppIihQjUBEJBLTRFCoC8pERCIxTQSqEYiIJMUzESQKdI5ARCSS1URgZieZ2QIzW2RmN2WYfrqZzTGzWWY2w8w+mc14tlONQERku6wlAjNLAHcAE4BhwPlmNixttn8Ch7n7KOCLwN3ZiqcB9RoSEdkumzWCMcAid1/s7tXAo8DpqTO4+2Z392iwE+DsDbkFUFu9V1YlItLeZTMRlAJLU4bLonENmNmZZjYfeIpQK9iJmV0RNR3NKC8vb3lkqhGIiGyXzURgGcbtdMTv7n9y96HAGcBtmQpy97vcfbS7j+7Zs2fLI9M5AhGR7bKZCMqA/inD/YDljc3s7tOAA8ysRxZjCnLVa0hEJCmbiWA6MMTMBplZPjARmJI6g5kdaGYWvf4YkA+szWJMQW6hagQiIpHcbBXs7rVmdjXwDJAA7nH3eWZ2ZTR9EnA2cLGZ1QDbgPNSTh5nT26BriwWEYk0KxGY2bXAvUAFoYvn4cBN7v5sU8u5+1Rgatq4SSmvfwz8eDdjbrncQqirhvp6yInnNXUiIknN3Qt+0d03AScCPYFLgR9lLapsS+SH/6oViIg0OxEkewCdDNzr7rPJ3Cto37D9cZU6YSwi0txE8KaZPUtIBM+YWTFQn72wsmz7A+x1UZmISHNPFl8GjAIWu/tWM+tGaB7aN6lGICKyXXNrBEcDC9x9g5ldCPwXsDF7YWXZ9hqBzhGIiDQ3EdwJbDWzw4BvAkuAB7IWVbZtTwSqEYiINDcR1Eb9+08HfunuvwSKsxdWlm1vGlKNQESkuecIKszsZuAi4JjoFtN52Qsry5I1AnUfFRFpdo3gPKCKcD3BSsJdRH+StaiyTSeLRUS2a1YiiHb+DwGdzexUoNLd991zBMkLytQ0JCLSvERgZucCbwCfA84FXjezc7IZWFapRiAisl1zzxHcAhzp7qsBzKwn8A/g8WwFllXqPioisl1zzxHkJJNAZO1uLNv+qNeQiMh2za0RPG1mzwCPRMPnkXZX0X2KagQiIts1KxG4+w1mdjbwCcLN5u5y9z9lNbJs0gVlIiLbNfvBNO7+BPBEFmPZexKqEYiIJDWZCMysggwPnCfUCtzdS7ISVbYlciEnVzUCERF2ccLX3YvdvSTDX/G+lgReeX8N5/32VVZs3BZGJJ9SJiISc/tuz5/dtK26jtc/WMeqTVFzUG6BagQiIsQoEXTtFK4mXr8lqgUklAhERCBGiaB7lAjWJhNBboFOFouIEKNE0C1KBOu2JJuGCpUIRESIUSIoKsglL2Gs21ITRqhGICICxCgRmBndOuWn1Ah0jkBEBGKUCAC6dSpgnc4RiIg0EKtE0L1TfsrJ4kLVCEREiFki6Nopf0f30dwCXVAmIkKWE4GZnWRmC8xskZndlGH6BWY2J/p7xcwOy2Y8qhGIiOwsa4kgesD9HcAEYBhwvpkNS5vtA+BYdz8UuA24K1vxQOhCWlFZS3VtfXRBmc4RiIhks0YwBljk7ovdvRp4FDg9dQZ3f8Xd10eDrwH9shjP9msJNmytVq8hEZFINhNBKbA0ZbgsGteYy4C/ZZpgZleY2Qwzm1FeXr7HAXVLvbpYF5SJiADZTQSWYVymW1pjZp8iJIIbM01397vcfbS7j+7Zs+ceB7Tj6uJqdR8VEYk0+8E0e6AM6J8y3A9Ynj6TmR0K3A1McPe1WYxn+/2G1iVrBPU1UF8HOYlsrlZEpF3LZo1gOjDEzAaZWT4wEZiSOoOZDQCeBC5y94VZjAXYcQfSkAjCa9UKRCTuslYjcPdaM7saeAZIAPe4+zwzuzKaPgn4NtAd+I2ZAdS6++hsxdS1Yz5m0TmC4sIwsrYS8jtma5UiIu1eNpuGcPepwNS0cZNSXn8J+FI2Y0iVyDG6dMgLF5V11XOLRUQgZlcWA9GN56JzBAB1SgQiEm+xTARrt1SFXkOgGoGIxF4sE8G6LdXhymLQRWUiEnsxTAQF4eE0yaYh1QhEJOZilwi6d8pn/dZq6hNR99GabW0bkIhIG4tdIujaKZ+6emeLJ68jUNOQiMRb7BJB8uriDTVRz9marW0YjYhI24tdIth+v6GavDCiWolAROItdomge1F0B9Kq6P5CqhGISMzFLhH0LArdRsuroreuk8UiEnOxSwTJG8+t2ppMBKoRiEi8xS4R5CVy6NIxjzVba8NFZUoEIhJzsUsEED3EfnM15HVQ05CIxF48E0FRAWs2V0F+J/UaEpHYi2Ui6FGUH55JkNdBTUMiEnuxTATdOxWwdnOVmoZERIhrIijKZ/3WGjyvo2oEIhJ7sUwEPaJrCWpyCpUIRCT2YpoIwrUElRSoaUhEYi+WiaB7VCOoNNUIRETimQiiq4u3er66j4pI7MUzEUQ1gs11eWoaEpHYi2UiKCnMJS9hVNTnq2lIRGIvlonAzOjeqYCNtblQXwN1NW0dkohIm4llIgDoUZzPhuTDaVQrEJEYi20i6N6pgHXVycdV6jyBiMRXfBNBUT5rqqOnlFVvadtgRETaUGwTQY+iAtZU6illIiJZTQRmdpKZLTCzRWZ2U4bpQ83sVTOrMrPrsxlLuu6d8tlYF64nUCIQkTjLzVbBZpYA7gA+A5QB081siru/kzLbOuCrwBnZiqMx3YsK2ObJRKCmIRGJr2zWCMYAi9x9sbtXA48Cp6fO4O6r3X06sNf7b3Yvymcb4cIy1QhEJM6ymQhKgaUpw2XRuN1mZleY2Qwzm1FeXt4qwfUsKmDr9kTQSt1HX78LZtwD7q1TnojIXpC1piHAMozboz2ku98F3AUwevToVtnL9iouoDLZNNQa9xtatxievhG8HlbPh5Nuh5xEy8sVEcmybCaCMqB/ynA/YHkW17dbuhcVUGWt2DT0r19BTi4cdj688VuwHJjwo5aXKyKSZdlsGpoODDGzQWaWD0wEpmRxfbslkWMUdCwJAy1tGtq0AmY9BIdfCKf9CkaeG4Zrq1oeqIhIlmUtEbh7LRe7a88AABKkSURBVHA18AzwLvCYu88zsyvN7EoAM+tjZmXA14H/MrMyMyvJVkzpuhQXhRctTQSv/hrq6+DjXw3DIz8HVZvgg2ktK1dEZC/IZtMQ7j4VmJo2blLK65WEJqM20aOkA1UbCihoSSJwhzmT4ZBTodugMG7wsZBfDO9OgSGfaZ1gRUSyJLZXFkM4Yby1pY+r3LQMtpTDwGN2jMstgIPGw/ynQk1BRKQdi3ciKClgq+fjLbnX0PK3wv++hzccf8hnYeta+OjVPS9bRGQviHUi6BldXVy9rYWJwBLQe3jD8Qd+GnIL4d2/tCxIEZEsi3Ui6FVSyFYKqK5sSSKYBb2GQV6HhuMLimDQsbDoHy0LUkQky2KdCHoWF7CNAuoqN+9ZAe6hRtD3sMzTB34C1i6CLWv2PEgRkSyLdSJIXl1cv6dXFm9cCtvW7Xx+IKn/2PD/o9f2rHwRkb0g1omgZ9RryPe0+2hjJ4qT+o6CRIFOGItIuxbrRNAxP5fanEJy9rT76PJZ4bYSvYZnnp5bAKVHqEYgIu1arBMBAHkdSdTtaSJ4KzpRXNj4PAPGwopZrXNjOxGRLIh9IrCCjuTW7+E9gVbOgf0aOVGcNOBoqK+FZW/u2TpERLIs9okgt6ATBV7Z8BkC70yBql30JNqyJlww1mtY0/P1PxIwNQ+JSLsV+0SQV9iJBPVQVx1GrH0fHrsIXridt5dtpKq2kVtElM8P/3se1PQKOnQNyWKpEoGItE+xTwQFHYsB2LK5IozYsASA+hn3ctGvn+brj83GMz1xrHxB+N/j4F2vZMBRsPQN3XdIRNql2CeCDh3DrajXbVgfRmwIT9fMqdnCBTl/56k5K3jg1SU7L7hmIeR1gs7NuHnqgKPDbalXv9NaYYuItJrYJ4JOnUKNYP3GjWHExqVgOSztOpZLcp/h2MFFfP+pd3h72caGC5bPD81ClumJnGkG6MIyEWm/Yp8IOnfuDMCa9VGNYGMZFPflz8UT6WGbuHPUR+Qncpg8fWnDBcsXNq9ZCKBzfygp1YVlItIuxT4R9OjaFYBVa1Kahrr056mNg6m0DnRcM4cjBnbj9Q/W7liochNULN/1ieIks1ArWPJqw95JIiLtQOwTQV5hJwDK1yVrBB9RV1zKovItrOs0GFa/y1GDurFw1WbWbYl6Fq1ZGP73HNr8FQ04OiSPjUt3Pa+IyF4U+0RAUS8AateXhV49m5azPq83tfVOTfehsPpdxg7uBsAbyVrB7vQYStp+nuD11opcRKRVKBF0HURVTkd6b5lP3aYVUF/LUu8BQMfSEbB1DSO71FCYl8Nri9eFZdYsgEQ+dB3Y/PX0GgYFJTpPICLtjhJBTg4bugxnGIspX7oIgIXbupCfyKHboHD7iPx1Czhi/668/kGUCMoXQLcDIJG7G+tJQL8jYcm/WvsdiIi0iBIB4PsdxjBbwrplocnnrYpihvQuItEnuqvo6vkcNag781duYuPWmpAIejbeLFRX71TX1u884cBPh26n6z7IxtsQEdkjSgRA0aAjKbAaCj74JwCvlHdgaJ8SKOoNhV1g9TuMGdQNd5i5YDGs/xB6j9ipHHfnL7OX84kfPceI7zzDmb/5Fw+8+uGOK5OHnhz+L5i6d96YiEgzKBEARQNHA1Ba/hL1hd34aLMxtE9x6PbZaxisfpdR/btQmJdD2ax/AB4eQ5mirt656qGZXPPIW3QvyucLH9+funrn2/83j+v/OCfcs6jrwJBA5j/V+m/CPdQ0Fj4L2za0fvki8m9rNxq5/411G8wW60in+q2sSvTHDE44JPQmotdQePsJCnNzOHnkfjDvATyvECs9okER//PsAv729kpuGH8wVx57AIkcw9351T8X8fN/LGTZhq3cd+kYCg8+GV76KWxZC526t078C5+FKVfD5lVhOL8IjrgExt0AHbq0zjpE5N+WagQAOTms7Bja/OdtKeHTh/RmcM9wDyJ6DYPKjVCxgolHDuBwf4fyziPD08ciU+eu4DcvvM/5YwbwlU8dSCIn3HbCzLj200P4+XmH8foH67jqoZnUHDQBvB4WPt3yuN3hpZ/Bw+eGbrCn/gIufAIOngCv3Qn3nQKbV7d8PSLyb02JILK5W2jz/7C2O5cfM3jHhORFY6vf5cg+xrCcJbxQteNE8YKVFVz/x9l8bEAXbj0t87MJzjy8H98/YwTPzV/NdS86XtK3yeah8ooq/jZ3BX9/ZxVvLllHfX0jVyM/dxv887sw4iz44rMw+tJwQvrsu+HCx2HdYrh3QrhthohII9Q0FMkpPRyW/gHv3I8jB3bdMSH54JlF/8DqajCcJ9cN5IjyzfToVMAVf5hBp4Jc7rzwCApyE42Wf8FR+7OlqpYfTp3PKT2P5qSFf8ZWzoU+I7fPs3BVBZNeeJ+/zllBdd2OXkcH9S7iP8YdwBmHl26vbfCvX8FL/xOagE79xc43vzvgeLjwyVBbuGcCXPxn6H5ASzeTiPwbsoz32m+tws1OAn4JJIC73f1HadMtmn4ysBW4xN1nNlXm6NGjfcaMGa0e6+qy9yn+3VjePv5ejjz21IYT/+9qeOtB6D8GX/4WI7b9jpqcAkoKc9m4rYZHrxjLEft3a9Z6Hn3jI378p1d5rvAG6kv6U/WFZ/hgXSV/emsZT84sozAvwcTDe3HGiG7kdOzK/JUV3P3SYuavrODg3sXcPOFgjl11P/b8D2D4mXD278M1CkB1bT3vl29mdUUV67dU06kglwFVCzno2YuxRB58fjL0PTxzYBUrYeXc0CMqtwB6Dw8ntlOawERk32Vmb7r76IzTspUIzCwBLAQ+A5QB04Hz3f2dlHlOBq4hJIKjgF+6+1FNlZutRABQW1NDbl7ezhOqt8LdJ4TnCQz4OK8d9yDPzV/Nh2u2cObhpUwYud9uref5+at55f9+yy3bfsr/1JzDI3Un0Ce3gtv2+xeHbX2NnC3RSd/S0TD0FPzgk/nbyhIenPo8n9vyEGcm/sXb3cfzxqG34Yl83ltVwdvLN7JgZQU1dTt/nkNyyni44Md08w3MHXQZHP0Vhg7qT2HVWlj0D+rmPEbO4hcwGi5bk+jI2tLjqTpgPIVDjqVb7/7kJVJaE+vroHIj9TVVVFkhleRT5Qnq3CnKz6VTQYJcc6hYEe7PtOa98H/9h1BXE2oxJaV410FYt0HQbRB0GQgduzXv9t4i0mxtlQiOBm519/HR8M0A7n57yjy/BV5w90ei4QXAce6+orFys5kImlS+AH53PBzzdTjmGy0uzuvr2fD7s+i67PkdI3M7wNBToMdBgIcTysvfCtMKSsLDbYDHS77AN8tPpN7DzrJrxzxGlHZmeN/ODOtbQmmXQrp0zGdzZS2rNlUyf2UF7y/5iPFLf8HJPg2ADd6JLrYFgDLvwZN1n+SlukNZ4r3pYFUcYh9xbM5sxiem083C85vXejHVVoCZUeRbKGLrTu+r2hNso4BtFFCP0ZON5NmOJ7NtoiNL6UOl55Hj9fS1cnpbw+6u28ingk5so5B6C7UdC1sko/QElpy/efN5g6HG5gOPpnqD8jPP23CJxqJKX3Ln+TOPazDd0stseh27ovTbvq044FzGXvCdPVq2qUSQzXMEpUDqrTbLCEf9u5qnFGiQCMzsCuAKgAEDBrR6oM3S82C4bh4UFLdKcZaTQ9dLHoXFL8CGj8LIkeeEo+Gk426CjcvCBWgr54RmnUHHck73Azittp5t1XXU1tfTrVM+1sQR9InD+wBDgBNYO+8F1iz4F5Wr3mNmogfvl3ycyh7DKe3aieu6FNK3cwdKOuRhwNaaOpZs2MzCpW+RX/YquRuXUFu1lbr6eqrzSqjJLaamoDM5iQIKrZoOVFHgleTVb8Ort1FXW8sHiW6U5/RkbeEA1nYYyNa87mBGXsJI5OSQlzDy6ysp2VZG8bYyirYtp7hqJXm1m8mr2wZe3/BRodbwRXKwqV13w3EZP41m7ZTTp6X/31WJDaY34wBsl2WkT/ddTJd9Xm5x7+yUm5VSg0y/pPRvZnPmwd3vAu6CUCNoeWh7qLX75OcVwsEnNT1P51IYc/lOo/Nzc8jP3f1OX92HH0f34cdtHz6+iXm7AqVdOsDAE4ETd3tdu+ewLJcvIo3JZvfRMqB/ynA/YPkezCMiIlmUzUQwHRhiZoPMLB+YCExJm2cKcLEFY4GNTZ0fEBGR1pe1piF3rzWzq4FnCN1H73H3eWZ2ZTR9EjCV0GNoEaH76KXZikdERDLL6gVl7j6VsLNPHTcp5bUDX8lmDCIi0jTdYkJEJOaUCEREYk6JQEQk5pQIRERiLqs3ncsGMysHluzh4j2ANa0YTjYoxtahGFuHYmy59hLf/u7eM9OEfS4RtISZzWjsXhvthWJsHYqxdSjGlmvv8YGahkREYk+JQEQk5uKWCO5q6wCaQTG2DsXYOhRjy7X3+OJ1jkBERHYWtxqBiIikUSIQEYm52CQCMzvJzBaY2SIzu6mt4wEws/5m9ryZvWtm88zs2mh8NzP7u5m9F/3v2sZxJszsLTP7azuNr4uZPW5m86NteXQ7jPG66DN+28weMbPCto7RzO4xs9Vm9nbKuEZjMrObo9/PAjMb34Yx/iT6rOeY2Z/MrEvKtHYRY8q0683MzaxHW8a4K7FIBGaWAO4AJgDDgPPNbFjbRgVALfANdz8EGAt8JYrrJuCf7j4E+Gc03JauBd5NGW5v8f0SeNrdhxIedfYu7ShGMysFvgqMdvcRhNuyT2wHMd4HpD8iL2NM0fdyIjA8WuY30e+qLWL8OzDC3Q8FFgI3t8MYMbP+wGeAj1LGtVWMTYpFIgDGAIvcfbG7VwOPAqe3cUy4+wp3nxm9riDswEoJsd0fzXY/cEbbRAhm1g84Bbg7ZXR7iq8EGAf8HsDdq919A+0oxkgu0MHMcoGOhCfxtWmM7j4NWJc2urGYTgcedfcqd/+A8AyRMW0Ro7s/6+610eBrhCcbtqsYIz8HvknDx++2SYy7EpdEUAosTRkui8a1G2Y2EDgceB3onXxSW/S/V9tFxi8IX+b6lHHtKb7BQDlwb9R8dbeZdWpPMbr7MuCnhCPDFYQn8T3bnmJM0VhM7fU39EXgb9HrdhOjmZ0GLHP32WmT2k2MqeKSCCzDuHbTb9bMioAngK+5+6a2jifJzE4FVrv7m20dSxNygY8Bd7r74cAW2r6pqoGonf10YBDQF+hkZhe2bVS7rd39hszsFkLz6kPJURlm2+sxmllH4Bbg25kmZxjX5vuiuCSCMqB/ynA/QtW8zZlZHiEJPOTuT0ajV5nZftH0/YDVbRTeJ4DTzOxDQnPa8Wb2YDuKD8JnW+bur0fDjxMSQ3uK8dPAB+5e7u41wJPAx9tZjEmNxdSufkNm9gXgVOAC33ExVHuJ8QBC0p8d/Xb6ATPNrA/tJ8YG4pIIpgNDzGyQmeUTTtZMaeOYMDMjtG2/6+4/S5k0BfhC9PoLwP/t7dgA3P1md+/n7gMJ2+w5d7+wvcQH4O4rgaVmdnA06gTgHdpRjIQmobFm1jH6zE8gnA9qTzEmNRbTFGCimRWY2SBgCPBGG8SHmZ0E3Aic5u5bUya1ixjdfa6793L3gdFvpwz4WPRdbRcx7sTdY/EHnEzoYfA+cEtbxxPF9ElCtXAOMCv6OxnoTuix8V70v1s7iPU44K/R63YVHzAKmBFtxz8DXdthjN8F5gNvA38ACto6RuARwjmLGsLO6rKmYiI0d7wPLAAmtGGMiwjt7MnfzKT2FmPa9A+BHm0Z467+dIsJEZGYi0vTkIiINEKJQEQk5pQIRERiTolARCTmlAhERGJOiUBkLzKz45J3cRVpL5QIRERiTolAJAMzu9DM3jCzWWb22+iZDJvN7H/MbKaZ/dPMekbzjjKz11Luj981Gn+gmf3DzGZHyxwQFV9kO56f8FB0tbFIm1EiEEljZocA5wGfcPdRQB1wAdAJmOnuHwNeBL4TLfIAcKOH++PPTRn/EHCHux9GuLfQimj84cDXCM/GGEy4p5NIm8lt6wBE2qETgCOA6dHBegfCzdfqgcnRPA8CT5pZZ6CLu78Yjb8f+KOZFQOl7v4nAHevBIjKe8Pdy6LhWcBA4OXsvy2RzJQIRHZmwP3ufnODkWbfSpuvqfuzNNXcU5Xyug79DqWNqWlIZGf/BM4xs16w/Tm++xN+L+dE83weeNndNwLrzeyYaPxFwIsenitRZmZnRGUURPepF2l3dCQiksbd3zGz/wKeNbMcwl0lv0J46M1wM3sT2Eg4jwDhds2Toh39YuDSaPxFwG/N7HtRGZ/bi29DpNl091GRZjKzze5e1NZxiLQ2NQ2JiMScagQiIjGnGoGISMwpEYiIxJwSgYhIzCkRiIjEnBKBiEjM/T98HAq+RmR2LgAAAABJRU5ErkJggg==\n",
+      "text/plain": [
+       "
    "
+      ]
+     },
+     "metadata": {
+      "filenames": {
+       "image/png": "/Users/hjensen/Teaching/FYS-STK4150/doc/src/LectureNotes/_build/jupyter_execute/chapter10_11_326.png"
+      },
+      "needs_background": "light"
+     },
+     "output_type": "display_data"
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "MSE:  1.9124180119966666e-05\n"
+     ]
+    },
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD4CAYAAADhNOGaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd3gUVffA8e9JsiGhBRJC770TSWgGURQUASkqAoqiIoiC/fUVyyv4s6Fiw4KCVEURC4KAgKBIBwMEiNQAASIdhCSSvvf3R5aQCgm7yWyS83mefXb3njszJwPkMDN37ogxBqWUUiWXh9UJKKWUspYWAqWUKuG0ECilVAmnhUAppUo4LQRKKVXCeVmdwNWoVKmSqVu37tUtHBMDKSkuzUepQuPlBeXLW52FKqI2b9582hgTmLW9SBaCunXrEhYWdnULz5kDVau6NiGlCsvx4zBokNVZqCJKRA7l1K6nhpRSqoRzSSEQkR4iskdEIkVkTA5xEZGJjvh2EWmb12WVUkoVLKcLgYh4Ap8AtwLNgcEi0jxLt1uBRo7XCGBSPpZVSilVgFxxjaA9EGmMOQAgInOAvsDODH36ArNM2nwWG0SkgohUA+rmYVmlVAmQnJxMdHQ0CQkJVqdS5Pn4+FCzZk1sNlue+ruiENQAjmT4Hg10yEOfGnlcVilVAkRHR1OuXDnq1q2LiFidTpFljOHMmTNER0dTr169PC3jimsEOf2JZZ3JLrc+eVk2bQUiI0QkTETCTp06lc8UlVLuLiEhgYCAAC0CThIRAgIC8nVk5YpCEA3UyvC9JnA0j33ysiwAxpjJxpgQY0xIYGC2YbB58tGKfXx/7iRfHF1EbMqFq1qHUqrgaBFwjfzuR1cUgj+BRiJST0S8gUHAgix9FgD3OUYPdQTOG2OO5XFZl4g8Gcu7v+5lWPRKhu+bQOC6O7h751usOx+BTsWtlCrJnC4ExpgUYDSwFNgFzDXG/CUiI0VkpKPbYuAAEAlMAR693LLO5pSTuWHRGJKJ8/oNgESTwDenlhAa/hh1N9zP24fncDLpn4LYtFKqiDh37hyffvqp1WkUOpfcR2CMWWyMaWyMaWCMed3R9pkx5jPHZ2OMGeWItzLGhF1u2YLQp011QpoexC7ns8UOJx3muYOfU239AHpse5nFZzaQalILKhWllJvKrRCkphbv3wcl5s7iljX8eK/f7TxbsR/+ngE59rGTytJzq+kV8TyV1w7iuQNfcCA+x0sWSqliaMyYMezfv5+goCDatWtH165dufvuu2nVqhVRUVG0bNkyve+ECRMYN24cAPv376dHjx4EBwdz3XXXsXv3bot+gqtTJOcaulp1KtTh7coDeaPVaJae3cTE6MUsP7ceO9mr/dnU07x9ZDZvH5lNu7JBPF6jJ3cEdsHXs5QFmStVstQds6jA1h01vleusfHjxxMREUF4eDgrV66kV69eREREUK9ePaKionJdbsSIEXz22Wc0atSIjRs38uijj/Lbb78VQPYFo0QVgou8xJNeAZ3oFdCJE0lnmXF8GZ/8vZgjSUdy7P9nXDj37gnn4X0TGRx4E0/X6kfzMnULN2mlVKFr3779Fcfix8XFsW7dOgYMGJDelpiYWNCpuVSJLAQZVfH257nag/hvrYGsj/mLj/5exI+nV5Jkso/BvWCPY+qJ+Uw9MZ8b/Drwcp2B3FAhSIe8KVVMlSlTJv2zl5cXdrs9/fvFcfp2u50KFSoQHh5e6Pm5SokvBBeJCNf6teRav5bEpjzGnJO/MTF6MRHxu3Lsv/L8RlZu30gTn0b8r+5d3BV4AzYP3Z1KucLlTt8UpHLlyhEbG5tjrEqVKpw8eZIzZ85QtmxZFi5cSI8ePShfvjz16tXju+++Y8CAARhj2L59O23atCnk7K9eiblYnB/lvEozvHpvdrT/lIiQaYyuNoByHn459t2TsI8hu1+n+vq7efvwt8Sk/FvI2SqlXCUgIIDQ0FBatmzJs88+mylms9l4+eWX6dChA71796Zp06bpsdmzZzN16lTatGlDixYtmD9/fmGn7hQpijdThYSEmMJ+ME2SPZkfT63mtUPf8Vd87iMC+vjdx1ct76ecl54uUgWgGD+YZteuXTRr1szqNIqNnPaniGw2xoRk7atHBHnk7WFjUJUb2dHuU1YHTaR7hVCyTpUkxpstx2/m2pUXeGN3Ekfj7TmvTCml3IgWgnwSETr7tWJZm9fY024m91e5DRveAJRJvRFPKhCbApOjkumyKp4ntyUQEZPK0cTT2I0WBqWU+9Grm05oXLoW05s+zdv1H+Sjv+fjZ7+OxUeF/f+mnW5LMfDTsVTmHUvhXOlnqOgtvFR7APdW7U4pD2+Ls1dKqTR6ROACgd4V+L96Q3mmQX1+7ezL1Lal6FDx0q5N8NhMjDnMocRDDN83gZrrh3A2Kc7CjJVS6hItBC7mIcJNlb34toMvCzr5cFtVT2K8fszU59/EhtyxwYMf/k4mxV70LtYrpYoXLQQFqLWfJ++29uSWwEp4ZNjVFZIHcfCC4ZkdSdy4Op650ckka0FQSllEC0EB8/aw8VOr/3Ggw2wernY7Qb5dqOTZID1+ON7w34gkuq6O5+39e3nj0DfEpcZbmLFSyhVWrlxJ7969AViwYAHjx4/Pte/VTn89btw4JkyYcNU5XqSFoJDU8anKZ40fY0u7cay5vjT/aWSjQobnSkfHG16J+ooXoyZTbd0gXj00W29OU8oNXc2U1H369GHMmDG5xq1+DoKOGipkIkJ5G4xu4M39dWzMOpzMlIPJnEg5zAXPNQDE2WN4OeoL3jr8Lf+peSdP1bodP6+yFmeuVOGSV1x7U6YZe+XTr1FRUfTo0YMOHTqwdetWGjduzKxZs2jevDkPPvggy5YtY/To0fj7+zN27FgSExNp0KAB06dPp2zZsixZsoQnn3ySSpUq0bZt2/T1zpgxg7CwMD7++GNOnDjByJEjOXDgAACTJk1i4sSJ6dNfd+/enXfeeYd33nmHuXPnkpiYSP/+/XnllVcAeP3115k1axa1atUiMDCQ4OBgp/eNHhFYqKyX8Gh9b9ZcX5q6AfNAMv9F/dceyyuHp1Nt3SBePDCdf5JzngNFKeU6e/bsYcSIEWzfvp3y5cun/0/dx8eHNWvW0K1bN1577TWWL1/Oli1bCAkJ4b333iMhIYHhw4fz888/s3r1ao4fP57j+h9//HGuv/56tm3bxpYtW2jRogXjx4+nQYMGhIeH884777Bs2TL27dvHpk2bCA8PZ/PmzaxatYrNmzczZ84ctm7dyo8//siff/7pkp9ZC4EbKOMlLGwzmjG17sVXSmeLx5t/eePILKqvH8SY/VM5mxxjQZZKlQy1atUiNDQUgCFDhrBmTdqR+sCBAwHYsGEDO3fuJDQ0lKCgIGbOnMmhQ4fYvXs39erVo1GjRogIQ4YMyXH9v/32G4888ggAnp6e+Plln8ds2bJlLFu2jGuuuYa2bduye/du9u3bx+rVq+nfvz+lS5emfPny9OnTxyU/sxYCN+FvK8+b9R/k6LXf8r/a91PaI/upoARzgbeiv6Lm+sGMOzhLLyorVQCyTit/8fvFKamNMXTv3p3w8HDCw8PZuXMnU6dOzXHZq2WM4fnnn0/fRmRkJMOGDXPpNjLSQuBmKniV5f/qDeVop28YV+dByniUy9Yn3lzglcPTqbFuMO8d+Z5Ee5IFmSpVsMxY49JXXh0+fJj169cD8M0339C5c+dM8Y4dO7J27VoiIyMBuHDhAnv37qVp06YcPHiQ/fv3py+bk5tuuolJkyYBaReeY2Jisk1/fcsttzBt2jTi4tJuPP377785efIkXbp0Yd68ecTHxxMbG8vPP/+c55/rcrQQuCk/r7KMrXsvxzrN4bW6w3OcBjvGfp5nDnxCrfX38sWxxaSY4v2AbaUKQ7NmzZg5cyatW7fm7Nmz6adxLgoMDGTGjBkMHjyY1q1b07FjR3bv3o2Pjw+TJ0+mV69edO7cmTp16uS4/g8//JDff/+dVq1aERwczF9//ZVt+uubb76Zu+++m06dOtGqVSvuvPNOYmNjadu2LQMHDiQoKIg77riD6667ziU/s1PTUIuIP/AtUBeIAu4yxvyTpU8tYBZQFbADk40xHzpi44DhwClH9xeMMYuvtF0rpqG2WlxqPB9Gz+PNw3P4157zRePa3rWY0OBB7gjsgodojS+WdBrqAhUVFUXv3r2JiIiwNA9XKMxpqMcAK4wxjYAVju9ZpQDPGGOaAR2BUSLSPEP8fWNMkON1xSJQUpX19OXFOncT3elrnqs5BG/xydbncNIR7tr1Cs03jWTp2U0UxWdNKKUKn7OFoC8w0/F5JtAvawdjzDFjzBbH51hgF1DDye2WWBW8yjK+wTAOd5zNyGq344ktW589CfvoseM52m15krXnd1iQpVJFU926dYvF0UB+OVsIqhhjjkHaL3yg8uU6i0hd4BpgY4bm0SKyXUSmiUjFyyw7QkTCRCTs1KlTuXUrMap4+zOp8WPs7zCLwYE9kBz+KDfHbadz+OPcGD6GbXGRFmSpVP7oUaxr5Hc/XrEQiMhyEYnI4dU3PxsSkbLAD8CTxpiLA+EnAQ2AIOAY8G5uyxtjJhtjQowxIYGBgfnZdLFWx6cqXzd/jp3tptOzYpcc+/x+fiOjdq7ib31imnJjPj4+nDlzRouBk4wxnDlzBh+f7KePc3PFKSaMMd1yi4nICRGpZow5JiLVgJO59LORVgRmG2PS52Q2xpzI0GcKsDDPmatMmpauzaLWr7A5dg9PRk5lTcylOw49THkOn72NrqviGVLbi9ENvPH31mcqK/dSs2ZNoqOj0SN+5/n4+FCzZs0893d2rqEFwFBgvON9ftYOknb3w1RglzHmvSyxahdPLQH9gZJ3cs7Fgss1YfU1b/PHuXAe3zeF7Rd24pd8Fx6UJsnAtEMpfPd3Co/Wt/FAHRs+nloQlHuw2WzUq1fP6jRKJGevEYwHuovIPqC74zsiUl1ELo4ACgXuBW4UkXDHq6cj9raI7BCR7UBX4Ckn81EO11cIIjzkYxa1fJMfgm4nyO/SH3VsCry1N5muq+MI/vM5ph5bTKreg6BUieXUfQRWKYn3ETjLGMOyk6m8tTeJA45nKsd5LueM9wcANPSpz6eNHqG7f7YhxsqdFOP7CFTBK6j7CFQRISLcUsWLpaG+vNrcm4reCZyzfZkej0w4wMhdi9gXpxeUlSpptBCUMDYP4d7aNl5tHYVIhllMjReJcfdyy5p4no9I5GSiFgSlSgotBCVU70rBRHaYxe0BNwFQPqUPXqYKduCb6BRuWBXPh5FJXEgpeqcOlVL5o08oK8Hq+FTlh5YvERY7gKTkKny634M1Z9KOBC6kwvuRycw+kkIZv4kMqNKMEdV74yWeFmetlHI1PSJQhJRrwrX+FfgyxIcZwaVoUvbSkNJDyeGsjF3KqMgPaLjhQRaeWa83/ChVzGghUOlEhBsCvVgc6stbLb0JLGU4Z5uWHj+UdJjbIl7g2q3PsCPugIWZKqVcSQuBysZThIE1bXzc9jjGMzpbfEPsVtpsHs6w3R9wOvm8BRkqpVxJC4HKVQe/hhzs8CWDA3sAme9ANtiZdmI+dTYM4f0jP5BsT7EmSaWU07QQqMuqUSqQr5s/R3jwZK4t1zZb/II9jqcPfEyjjcNYenaTBRkqpZylhUDlSZuyDVlzzQR+avEq1W3Vs8UPJR2mx47nuCn8efZeOGJBhkqpq6WFQOWZiNC3UmcOdJzOG/VG4COls/X57fwGmv/5II/t+5TzKXEWZKmUyi8tBCrfSnl483ztwRzs+CV3B95K1usHqaTw8dHvqL1+CJ8f/VkntFPKzWkhUFetqrc/s5v/l7C2k7imTIts8Rj7eUbue4/mmx5m1bltFmSolMoLLQTKacHlmrA5+CO+afo/Knllf3rc3oT9XL/tSW7bMY5DCcctyFApdTlaCJRLiAiDqtzIoY6zeLH2UGxSKlufhWf/oOHGobxwYDoXUhMsyFIplRMtBMqlSnv68Fq9+9nffhb9/G/MFk8hiTePzKL+xgdYdGa9BRkqpbLSQqAKRC2fysxr9T9WB02kmW/jbPETycf5vz1/s1+ff6CU5bQQqALV2a8VEe0m8UXjZ/HzrJje7m1vwPGYm+mxNp639uh010pZSQuBKnAe4sGwaj051HEWj1S7Aw88CUh6BMGTZAOTDiZz05p4Fh1P0ZlNlbKAFgJVaPy8yvJp49Ec6TiHXzsFEVzh0l+/YwmGUeGJDP7zHLdHvMm+C9knu1NKFQynCoGI+IvIryKyz/FeMZd+USKyQ0TCRSQsv8ur4qV6qUq0KO/Jdx18mNDKmwDvS7FfYr5m3pllNPvzQf67/wsdXaRUIXD2iGAMsMIY0whY4fiem67GmCBjTMhVLq+KGQ8R7qxh47frSnN/bS9SJJoYr58ASCWZd6JnM2LXHD1dpFQBc7YQ9AVmOj7PBPoV8vKqGPCzCeOal6JH7d9BLk1n7WkCWBXdmyFhCUTq6CKlCoyzhaCKMeYYgOO9ci79DLBMRDaLyIirWF6VAFOaPsxXTV+koqc/ABWTh+GBL2vP2OmxNp439yQRp6OLlHK5Kz68XkSWA1VzCL2Yj+2EGmOOikhl4FcR2W2MWZWP5XEUkBEAtWvXzs+iqogQEe6p0o3bAjox5eivxMZ1ZdbhVOxAioHPDybz09EUXmrqRTmfndxQMcjqlJUqFq5YCIwx3XKLicgJEalmjDkmItWAk7ms46jj/aSIzAPaA6uAPC3vWHYyMBkgJCRE/1tYjJX3KsMztdPOEg6sZeflnYn8+U/aqaETiYb7IxZx1nsit1TozJQmj1HLRw8klXKGs6eGFgBDHZ+HAvOzdhCRMiJS7uJn4GYgIq/Lq5KtWTkP5rb34f3WpQgsJaRynnO26QAsPbeGRpvu550jc0nRqa6VumrOFoLxQHcR2Qd0d3xHRKqLyGJHnyrAGhHZBmwCFhljllxueaUyEhH6V/diRWdfAirMwC6x6bFEE89/D0yixaaH2RSzy8IslSq6rnhq6HKMMWeAm3JoPwr0dHw+ALTJz/JK5aS8TZjZ8i7u3XWEHRd2Z4rtTdhPh62jeKDKbbzXcDgVvMpalKVSRY/eWayKlDZlGxIe8gkfN3wSXymTJWqYfmIB9TYM5ZsTK/T+A6XySAuBKnI8xINRNfpysOMs+gdkP6A8l3qWu3e/Rpfw/xIZ/7cFGSpVtGghUEVWFW9/fmz5Er+2nkANW/Vs8TUxYTTb9AAvH5xFoj3JggyVKhq0EKgir1vFYCI7Tuf5WvfhiS1TLIVkXj08nUYbH2LluXCLMlTKvWkhUMWCj4c3b9R/gL/afUGHstlvNDuSdISu257irr/e4FTSOQsyVMp9aSFQxUqT0rVZ3/Y9ZjZ5nvIeftni353+lXob72PG8aV6MVkpBy0EqtgREe6rejMHO87i3sq9ssX/tcfywJ7xXLf1WQ7EH7UgQ6XcixYCVWz528ozq9l/WBv0EfVL1c0WXxu7maZ/Pshbh7/VO5NViaaFQBV71/q1ZHf7KbxadzheeGeKJZtExhz8jNZ/PsrW2H0WZaiUtbQQqBLB5uHFS3XuZlf7aTleTN4Vv5dBEdOI1WmuVQmkhUCVKA19a7C+7XtMafwspT0uTUMhxpd/Y0bQfXU8y0+mXGYNShU/WghUiSMiPFStJ/s7zOQ2/xsAqJB8H16mMscTDQ9tSWRUeAKnEvXoQJUMWghUiVXV258FrcayvNUEprboT0CGyweLjqfSbc0F5kYns+H8Th1qqoo1LQSqxLvJP5j+1UuxvHNp7qxxaULe88nw2M5NdAofRfdtL3IkIdfnJilVpGkhUMqhorcwoVUpvgrxoZavYCeBs7aPAFhxfj2NNt3P1yd+szhLpVxPC4FSWXSu5Mmyzr7UrfQtKR4n0tsT7Ql8tK8iEef1ngNVvGghUCoHvp7CZ8160qZ08/S2cqm9ORbXmL4bEnhnbxKJdr1uoIoHLQRK5aJ5mbpsCfmIiQ2eoLqtDlVS7wUg1cAnB5LpvS6eref06EAVfVoIlLoMD/HgsZr9ONxpKr+GBtC+4qV/MvviDHdsSOA/EYf4v6ivSLbr/QeqaNJCoFQeeIondct4MKe9D68296a0Z1p7KnY+PvkeYw9NpdWfj7AtLtLaRJW6CloIlMoHDxHurW1jaagvnQM8iPNcQqLndgD2JETSdvNIXjgwnSR7ssWZKpV3ThUCEfEXkV9FZJ/jvWIOfZqISHiGV4yIPOmIjRORvzPEejqTj1KFpVZpD6YH2/AqPS9Tu51U3jwyi2abRhAWu8ei7JTKH2ePCMYAK4wxjYAVju+ZGGP2GGOCjDFBQDBwAcj4r+f9i3FjzGIn81Gq0Ng8vIho9yl9/btmix1IjKL9lkd5OvJzEvR5ycrNOVsI+gIzHZ9nAv2u0P8mYL8x5pCT21XKLQR6V+CnVi/zY/P/o4Knf6aYwc77f8+hycaHWHc+wqIMlboyZwtBFWPMMQDHe+Ur9B8EfJOlbbSIbBeRaTmdWrpIREaISJiIhJ06dcq5rJVysf6B17G/w3TuqtQ9W+xw0hFCwx9n1N5PuJCaYEF2Sl3eFQuBiCwXkYgcXn3zsyER8Qb6AN9laJ4ENACCgGPAu7ktb4yZbIwJMcaEBAYG5mfTShUKf1t5vm3xAgtbvkGAV6UsUcOnx76n0cZhrDq3zZL8lMrNFQuBMaabMaZlDq/5wAkRqQbgeL/crFy3AluMMen37BtjThhjUo0xdmAK0N65H0cp6/UK6MT+DtMZUjn72IejyUe5fttTPLr3Y+JTEy3ITqnsnD01tAAY6vg8FJh/mb6DyXJa6GIRcegP6IlUVSz4eZXly2bPsqzVO1T2qpIlaph07AeabBrOxpidluSnVEbOFoLxQHcR2Qd0d3xHRKqLSPoIIBEp7Yj/mGX5t0Vkh4hsB7oCTzmZj1Jupbt/CJEdpvFQlexnUo8kHaHT1sf4z/4pJOrIImUhKYoP3AgJCTFhYWFXt/CcOVC1qmsTUioPfv9nK4N2vsXJlBPZYg186vF9ixcIKtvw8is5fhwGDSqgDFVxJyKbjTEhWdv1zmKlCknXitcQ2WEa91bulS22P+EgwZtH8vLBWTpnkSp0WgiUKkTlvEozq9l/WNxyPP6eAZlidlJ59fB0gsJGsfPfKGsSVCWSFgKlLHBrQAciO0znzhzuO9gZv5c2YSPYGnvAgsxUSaSFQCmLVLSV47sWL/BD81co7+GXKWZLDeK58Cr6NDRVKLQQKGWx2wO7ENlhBj0rXgeAhymHf9Jj7P0X+m1I4IPIJJL1aWiqAHlZnYBSKm3OooWtXuHrkyvYcMaTX6P9SbBDioEPIpNZfjKV91qVorHViapiSY8IlHITIsI9VbrxUfOu/BLqS3CFS/88I2Ls9F4XT7eo2XyxZSpFcdi3cl9aCJRyQ/XKeDC3gw8vNPHGW9LazksYKxIXMvznh+g1+3bOxp+1NklVbGghUMpNeYowop6Nhdf60qTcv5zxnpge+2X/T4RO6aFHBsoltBAo5eYal/PgqaaH8fT491Kj8eCf4wN4+MvNnInTyeuUc7QQKFUE9AgIYUfIFJrb6gPglzIQH3tzlu08wS0frOK33dmnrVAqr7QQKFVENC5di631XuHd7h8yOuS/6e2n45J4cEYYL8zbwb+JOj2Fyj8dPqpUEeItXjx97eMAdGtWnWe/386p2LRTQ19vPMyKvTuoVGM+s27/lIDSAZdblVLp9IhAqSLqhiaVWfpkF3q0SJtN15DKtn9fY/H+udT/sDlL9i2zOENVVGghUKoI8y/jzaQhbZkwoA3xpb4j0XMXADFJJ7n161sYPv8JklL1WQfq8rQQKFXEiQj9rqlKrep7s8W+CJ9I4w/bsuvULgsyU0WFFgKligEvDy82PLSG50LHIEim2KHYv2g16RomrPlY7ztQOdJCoFQxYfO0Mb7bm/w+9HeqlqmRKZZqEnl2xWOETrmV0xdOW5ShcldaCJQqZq6vez07R+3g9qZ3ZoutP7aUuu83Z+EevZCsLtFCoFQxVNG3It/fNZfpfafj61U6U+zflFPcNqcHw+Y9RXJqskUZKnfiVCEQkQEi8peI2EUk2wORM/TrISJ7RCRSRMZkaPcXkV9FZJ/jvaIz+SilLhER7g+6n+2PbCO4WrssUcO07R/Q6MMQ9p6OtCQ/5T6cPSKIAG4HVuXWQUQ8gU+AW4HmwGARae4IjwFWGGMaASsc35VSLtTQvyHrh63lxetezOFC8nZafNqGTzZOtyg75Q6cKgTGmF3GmD1X6NYeiDTGHDDGJAFzgL6OWF9gpuPzTKCfM/kopXJm87Tx2o2vseK+FVQpUy1TLMVcYPSSB+k2fQCxibEWZaisVBjXCGoARzJ8j3a0AVQxxhwDcLxXLoR8lCqxutbryl+P7uC2xn2yxVYc/p7a77VgddR6CzJTVrpiIRCR5SISkcOr75WWvbiKHNryPZhZREaISJiIhJ06dSq/iyulHAJKBzB/0E980vMTSnn6ZIqdSzrC9TOv46nFr2A3dosyVIXtioXAGNPNGNMyh9f8PG4jGqiV4XtN4Kjj8wkRqQbgeD95mTwmG2NCjDEhgYGBedy0UionIsKj7R4lbMSfNKvUPFPMkMoHf46j5Udd+Pv80VzWoIqTwjg19CfQSETqiYg3MAhY4IgtAIY6Pg8F8lpclFIu0LJySzaPCGNkyMhssV3/rKXBxBZ8Ff6jBZmpwuTs8NH+IhINdAIWichSR3t1EVkMYIxJAUYDS4FdwFxjzF+OVYwHuovIPqC747tSqhD52nyZ1GsS8wbOo0Ip/0yxRPs5Rs+fwK9/HbcoO1UYpCjOPRISEmLCwsKubuE5c6BqVdcmpFRhOX4cBg0qsNVHx0Rzzw9DWHX4DwA8jT/VEj7CEz/u7ViHF3s1w8fmWWDbVwVLRDYbY7Ld86V3Fiul0tUsX5Pfhq7gta6v4eVho5HX83jiB8CXGw5x20dr2HUsxuIslatpIVBKZeLp4cmLXV4k6omDrHvqcW5pUSU9tu9kHH0/WctXGw6RmJJoYXnvzuMAABIQSURBVJbKlbQQKKVyVKN8DSqW8eazIcG8eXsrfB2nhJJS7Dy9YDpV3m7MqqhNFmepXEELgVLqskSEwe1r8/NjnWlatRwpnOaM9/ucTz7MDTM7M2bpO/qcgyJOC4FSKk8aVi7L9490wKPSROySdp3AkMxb68cw9pcl2O1aDIoqLQRKqTyzeULPpplnMq2QMoRZq+zcP+NPTsfpdYOiSAuBUirPSnmV4rPenzHnjjmUtZWjsq095VPSHoCzau8pbv1wNWsj9QloRY0WAqVUvg1sOZCtI7ewZfR8Hrm+UXr7qdhEhkzdyISle0hJ1bmKigotBEqpq9LQvyE1yldlzK1NmfVgeyqV9QbAGPjo993Ue+dm5u/83eIsVV5oIVBKOa1L40AWP3EdoQ0DADjn9RXRiSvo9103hv3wks5k6ua0ECilXKJyOR9mPdiBnu2OE2P73tFqZ1rE6zR5/3qiz+t8Re5KC4FSymU8PYTqgceytUfGrqHhh634dvtSC7JSV6KFQCnlUi91eYmfB/9MRZ8sM5ma0wz6sSf3fPtfPVXkZrQQKKVcrnfj3mwbGU5ordDMAbHz9e53aPjedRw5p6eK3IUWAqVUgajlV4vfh/7O852fzxY7GLeORhNbMXf7MgsyU1lpIVBKFRibp403bnqDJfcsIcC3UqZYojnNwB9v5YHvX9RTRRbTQqCUKnC3NLyFbSPD6Vyrc+aA2Jnx1xs0/eAGjsbk+shyVcC0ECilCkWN8jX4/f7feS70uWyxfTGrafhhK+bvXFn4iSktBEqpwuPl4cX4buNZOHhhtlFF8faT9JvbjZE//Z9Oa13ItBAopQpdr8a9CB+5lY41OmYOSCqfbxtLi4ndORl3xprkSiAtBEopS9T2q80fD/zB0x2fzhbbdW4F9d5vxZK9ayzIrORxqhCIyAAR+UtE7CISkkufWiLyu4jscvR9IkNsnIj8LSLhjldPZ/JRShUt3p7evHvLu8wbOI/ypfwyxS7Yj9Hzmxt54uc39VRRAXP2iCACuB1YdZk+KcAzxphmQEdglIg0zxB/3xgT5HgtdjIfpVQR1K9pP7Y+vIXgasGZ2g3JTNzyAkEf9+TshXMWZVf8OVUIjDG7jDF7rtDnmDFmi+NzLLALqOHMdpVSxU/9ivVZ++BaRrcbnS22/ewSmn54IyfOx1uQWfFXqNcIRKQucA2wMUPzaBHZLiLTRKTiZZYdISJhIhJ26tSpAs5UKWWFUl6l+KjnR8y9cy5lvctdChgPPGIH0uujtazbr09Ac7UrFgIRWS4iETm8+uZnQyJSFvgBeNIYE+NongQ0AIKAY8C7uS1vjJlsjAkxxoQEBgbmZ9NKqSJmQIsBbBmxmaAqQQBUSLkXH3tLTsclMuSLjUxauV+vG7iQ15U6GGO6ObsREbGRVgRmG2N+zLDuExn6TAEWOrstpVTx0CigEesfWs8XW76gTcW7eOrbbZyOS8Ju4K0lu9l6+B8m3NWG8j42q1Mt8gr81JCICDAV2GWMeS9LrFqGr/1Ju/islFIA+Hj5MLr9aK5rVJlFj19HSJ1LZ4+X7TxBj4kLmLjuKwszLB6cHT7aX0SigU7AIhFZ6mivLiIXRwCFAvcCN+YwTPRtEdkhItuBrsBTzuSjlCq+qpT34ZsRHXkwtB4AhhS2xI3liV/vpc+Xw0lOTbY4w6JLiuJ5tpCQEBMWFnZ1C8+ZA1WrujYhpQrL8eMwaJDVWVju521Huf/HJzjr8X16W80yQfz58K9ULVfpMkuWbCKy2RiT7Z4vvbNYKVXktKmbQqzXgkxtp2PgkVk7OaZDTPNNC4FSqsip7VebNQ+uplb52gB4mgAqJT3L1iOx9J64RoeY5pMWAqVUkdS+Rnu2PryFPo378J/gSXh7VADgzL9JDPliI5/9oUNM8+qKw0eVUspdBZQOYP7g+QD0a3GG0V9v5XRcInYD439JG2L6zK2VaVyptsWZujc9IlBKFQsd6wew6PHOmYaY/rRrMU0/acj/lk/Qo4PL0EKglCo2Lg4xfSC0LilyktPe72JI5rW1z3L91P7EJcVZnaJb0kKglCpWbJ4e/K93M8pW+xi7xKS3r/57PvXeb8O2Y3rfalZaCJRSxY6HePBat+cpYyuXqf10wgGCp7Tns00zLcrMPWkhUEoVS3c2v5MtD4fRIrBlpvZUE88jv9zPoLkPk5SaZFF27kULgVKq2Goc0JhNwzdyX+v7ssW+3TWZlh93Ijom2oLM3IsWAqVUsVbaVpoZ/WYwufdkbB7emWL7zm2hycTWLI1cblF27kELgVKq2BMRhgcPZ92wtdQsl/meggup/9Bj9i28tPxV7MZuUYbW0kKglCoxQqqHsO2Rrdxcv0eWiJ3X177MjdN7cy6h5D0bWQuBUqpE8ff155chixh3/TgEyRT748gvNPmoDeHHwy3KzhpaCJRSJY6HeDD2hrH8cs8vlPfO/Kj0kxcO025yR77YPM2i7AqfFgKlVIl1S8Nb2P7IVtpUDs7UnmISGb5wGPf+OIyElASLsis8WgiUUiVanQp12Dh8LQ9dMyJb7Ksd02j7WSeizkUVfmKFSAuBUqrEK+VViil9PmdG3xnYPHwyxXadCaflp0H8su8Xi7IreFoIlFLKYWjQUMJGbKRG2XqZ2v9NPk/Pr3vx0oqXSbWnWpRdwdFCoJRSGbSu0pq/Rm3llvq3ZYkYXl/zKp9umm5JXgXJqUIgIgNE5C8RsYtItgciZ+gXJSI7RCRcRMIytPuLyK8iss/xXjG3dSilVGHx8/HjlyHzebXrm0iGX5M+qdfw1Yra7Ig+b2F2rufsEUEEcDuwKg99uxpjgowxGQvGGGCFMaYRsMLxXSmlLCcivNRlDCvuW045WwCeJpBKSf/hRGwyAz5fx8/bjlqdoss4VQiMMbuMMXucWEVf4OJ8sDOBfs7ko5RSrta1Xld2jd7G57d+SwWfAAASku089s1WJizdg91e9J98VljXCAywTEQ2i0jGMVpVjDHHABzvlQspH6WUyrMa5WswrENX5o8KpX6lMuntH/8eycivNvPppskc/OeghRk654qFQESWi0hEDq+++dhOqDGmLXArMEpEuuQ3UREZISJhIhJ26tSp/C6ulFJOqx9YlnmjQunSODC9bf7uxYz6ZSTXfB7M4n2LLczu6l2xEBhjuhljWubwmp/XjRhjjjreTwLzgPaO0AkRqQbgeD95mXVMNsaEGGNCAgMDc+umlFIFys/XxrShITzUuR4pcoLT3u8AhvOJ/9D76968u+5dq1PMtwI/NSQiZUSk3MXPwM2kXWQGWAAMdXweCuS5uCillFW8PD14qXdz2rfYil3i0tuNEf45X93CzK6Os8NH+4tINNAJWCQiSx3t1UXk4jFSFWCNiGwDNgGLjDFLHLHxQHcR2Qd0d3xXSqki4Yd7JvJ4yDgu/iqtkHIfX/1RnrHzI0hJLTrPNvByZmFjzDzSTvVkbT8K9HR8PgC0yWX5M8BNzuSglFJWERE+7DWWznU68szPnyDxdwAwc/0hIk/F8cndbalQ2vsKa7Ge3lmslFJOGtDyFnY/8yO9WlVLb1sbeYZ+n6wl8mQsZy6cIT453sIML08LgVJKuUBpby8+HtyWJ7s1Sm+LOnOBfp+s5qYZfeg8vbPbDjHVQqCUUi7i4SE82a0xk+5pi6/NE4AjqVPZdmodW45tIXhysFvOYqqFQCmlXOzWVtX4/pFO+JYLJ8b2Y3r7Pwn/0OvrXoxbOQ67cZ+LyVoIlFKqALSo7seikcOpXirz/bMGwyt/vEKvr3txNv6sRdllpoVAKaUKSP2Ayux/ZgVdqjwBJvOv2yWRSwieHMyWY1ssyu4SLQRKKVWAfGxerHz4fZ5pOwsP45cpFnUuimunXsvULVMtyi6NFgKllCpgIsKEPvcw+7bf8TVNM8USUxN56OeHeGjBQySkJFiSnxYCpZQqJIOCr2HdsNVU8cg+Z+fUrVMJnRZK1LmoQs9LC4FSShWioFqV2P7EHK4p+z/ElMoUuzjEdEnkklyWLhhaCJRSqpBVLu/D2sfGcmetmXjZq2WKnY0/S8/ZPXll5SuFNsRUC4FSSlnA19uTOQ/cxUvt5uOb2jFTzGAY98c4en/du1CGmGohUEopi3h4CGN7t2P6bd8SkHJ/tiGmv0T+UihDTLUQKKWUxQa2q83CByZQX97MdYjptK3TCmz7WgiUUsoNdKwfwPJRo2hfegre9iaZYompiQxbMIzhC4YXyBBTLQRKKeUm6lUqwy+j+9Kn+mTKpfTKFv9i6xe8+serLt+uFgKllHIjFUp7M3tYZx5u/ToBSc9kGmLaoEJjxnQe4/JtaiFQSik34+3lwdt3tubV7o9QNfFdvOzVEOOL57ln2RGd6PLtaSFQSik3JCI8ckMDpt1zO3XtE6mcNJbE+GrcN3UTSyKOu3RbWgiUUsqN9WhZjR8e7kbtMsEABJT1pk0tvysslT9aCJRSys21rlmBn0aFElynIl/c145qfr4uXb9ThUBEBojIXyJiF5GQXPo0EZHwDK8YEXnSERsnIn9niPV0Jh+llCquqlfw5fuRnWhV07VHAwBeTi4fAdwOfJ5bB2PMHiAIQEQ8gb+BeRm6vG+MmeBkHkopVeyJSIGs16lCYIzZBflK7iZgvzHmkDPbVUop5TqFfY1gEPBNlrbRIrJdRKaJSMXcFhSRESISJiJhp06dKtgslVKqBLliIRCR5SISkcMr+5MVLr8eb6AP8F2G5klAA9JOHR0D3s1teWPMZGNMiDEmJDAwMD+bVkopdRlXPDVkjOnmom3dCmwxxpzIsO70zyIyBVjoom0ppZTKo8I8NTSYLKeFRCTjExn6k3bxWSmlVCFydvhofxGJBjoBi0RkqaO9uogsztCvNNAd+DHLKt4WkR0ish3oCjzlTD5KKaXyT4wxVueQbyJyCnDXkUeVgNNWJ3EZmp9zND/naH7OcybHOsaYbBdZi2QhcGciEmaMyfHmOneg+TlH83OO5ue8gshRp5hQSqkSTguBUkqVcFoIXG+y1QlcgebnHM3POZqf81yeo14jUEqpEk6PCJRSqoTTQqCUUiWcFgIXEpEoxw1y4SIS5gb5TBORkyISkaHNX0R+FZF9jvdcJ/qzKD+3eUaFiNQSkd9FZJfjuRtPONrdYh9eJj+32Ici4iMim0RkmyO/Vxzt7rL/csvPLfZfhjw9RWSriCx0fHf5/tNrBC4kIlFAiDHGLW5IEZEuQBwwyxjT0tH2NnDWGDNeRMYAFY0xz7lRfuOAOHd4RoVjCpRqxpgtIlIO2Az0A+7HDfbhZfK7CzfYh5I2P30ZY0yciNiANcATpD3DxB32X2759cAN9t9FIvI0EAKUN8b0Loh/w3pEUIwZY1YBZ7M09wVmOj7PJO0XhyVyyc9tGGOOGWO2OD7HAruAGrjJPrxMfm7BpIlzfLU5Xgb32X+55ec2RKQm0Av4IkOzy/efFgLXMsAyEdksIiOsTiYXVYwxxyDtFwlQ2eJ8cpKnZ1QUJhGpC1wDbMQN92GW/MBN9qHjtEY4cBL41RjjVvsvl/zATfYf8AHwX8Ceoc3l+08LgWuFGmPakjbl9ijHqQ+VP3l+RkVhEZGywA/Ak8aYGKvzySqH/NxmHxpjUo0xQUBNoL2ItLQql5zkkp9b7D8R6Q2cNMZsLuhtaSFwIWPMUcf7SdKey9ze2oxydMJxbvniOeaTFueTiTHmhOMfpx2YgsX70HHu+AdgtjHm4uy5brMPc8rP3fahI6dzwErSzr+7zf67KGN+brT/QoE+jmuPc4AbReQrCmD/aSFwEREp47hgh4iUAW7GPZ+vsAAY6vg8FJhvYS7ZiBs9o8JxMXEqsMsY816GkFvsw9zyc5d9KCKBIlLB8dkX6Absxn32X475ucv+M8Y8b4ypaYypS9pjfn8zxgyhAPafjhpyERGpT9pRAKQ9+e1rY8zrFqaEiHwD3EDatLUngLHAT8BcoDZwGBhgjLHkgm0u+d1A2iG5AaKAhy+eD7Ugv87AamAHl87RvkDaeXjL9+Fl8huMG+xDEWlN2sVMT9L+0znXGPN/IhKAe+y/3PL7EjfYfxmJyA3Afxyjhly+/7QQKKVUCaenhpRSqoTTQqCUUiWcFgKllCrhtBAopVQJp4VAKaVKOC0ESilVwmkhUEqpEu7/Ae9vahOyZL/GAAAAAElFTkSuQmCC\n",
+      "text/plain": [
+       "
    "
+      ]
+     },
+     "metadata": {
+      "filenames": {
+       "image/png": "/Users/hjensen/Teaching/FYS-STK4150/doc/src/LectureNotes/_build/jupyter_execute/chapter10_11_328.png"
+      },
+      "needs_background": "light"
+     },
+     "output_type": "display_data"
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Time:  9.464447478\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Model: \"functional_7\"\n",
+      "_________________________________________________________________\n",
+      "Layer (type)                 Output Shape              Param #   \n",
+      "=================================================================\n",
+      "input_4 (InputLayer)         [(None, 1, 1)]            0         \n",
+      "_________________________________________________________________\n",
+      "RNN (SimpleRNN)              (None, 200)               40400     \n",
+      "_________________________________________________________________\n",
+      "dense (Dense)                (None, 1)                 201       \n",
+      "=================================================================\n",
+      "Total params: 40,601\n",
+      "Trainable params: 40,601\n",
+      "Non-trainable params: 0\n",
+      "_________________________________________________________________\n",
+      "Epoch 1/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 1.4102"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 154ms/step - loss: 1.4102 - val_loss: 2.1967\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 2/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.9243"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.9243 - val_loss: 1.4521\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 3/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.5465"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.5465 - val_loss: 0.8738\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 4/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.2756"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.2756 - val_loss: 0.4570\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 5/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.1066"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 20ms/step - loss: 0.1066 - val_loss: 0.1890\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 6/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0284"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0284 - val_loss: 0.0477\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 7/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0229"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 0.0229 - val_loss: 0.0012\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 8/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0654"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0654 - val_loss: 0.0122\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 9/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.1283"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.1283 - val_loss: 0.0453\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 10/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.1870"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.1870 - val_loss: 0.0747\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 11/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.2252"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.2252 - val_loss: 0.0870\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 12/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.2363"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.2363 - val_loss: 0.0801\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 13/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.2218"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 0.2218 - val_loss: 0.0593\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 14/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.1884"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 0.1884 - val_loss: 0.0332\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 15/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.1445"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.1445 - val_loss: 0.0111\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 16/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0988"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0988 - val_loss: 3.2715e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 17/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0585"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0585 - val_loss: 0.0055\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 18/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0286"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 0.0286 - val_loss: 0.0275\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 19/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0116"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0116 - val_loss: 0.0639\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 20/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0072"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0072 - val_loss: 0.1091\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 21/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0129"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 0.0129 - val_loss: 0.1561\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 22/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0249"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 0.0249 - val_loss: 0.1975\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 23/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0388"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0388 - val_loss: 0.2272\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 24/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0505"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0505 - val_loss: 0.2413\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 25/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0572"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0572 - val_loss: 0.2389\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 26/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0578"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0578 - val_loss: 0.2215\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 27/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0526"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0526 - val_loss: 0.1927\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 28/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0430"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0430 - val_loss: 0.1571\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 29/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0313"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0313 - val_loss: 0.1197\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 30/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0198"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0198 - val_loss: 0.0845\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 31/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0105"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0105 - val_loss: 0.0548\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 32/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0045"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0045 - val_loss: 0.0320\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 33/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0022"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0022 - val_loss: 0.0165\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 34/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0032"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 22ms/step - loss: 0.0032 - val_loss: 0.0071\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 35/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0065"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0065 - val_loss: 0.0024\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 36/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0105"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 0.0105 - val_loss: 5.8779e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 37/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0140"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0140 - val_loss: 9.0393e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 38/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0161"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0161 - val_loss: 2.5875e-05\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 39/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0163"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0163 - val_loss: 1.0453e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 40/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0146"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0146 - val_loss: 5.5861e-04\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 41/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0117"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 0.0117 - val_loss: 0.0019\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 42/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0083"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 0.0083 - val_loss: 0.0047\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 43/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0050"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0050 - val_loss: 0.0091\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 44/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0027"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0027 - val_loss: 0.0151\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 45/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0015"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0015 - val_loss: 0.0221\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 46/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0015"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0015 - val_loss: 0.0294\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 47/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0023"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0023 - val_loss: 0.0358\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 48/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0036"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0036 - val_loss: 0.0407\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 49/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0048"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0048 - val_loss: 0.0432\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 50/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0056"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0056 - val_loss: 0.0432\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 51/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0057"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0057 - val_loss: 0.0409\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 52/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0053"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0053 - val_loss: 0.0368\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 53/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0043"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0043 - val_loss: 0.0315\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 54/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0032"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0032 - val_loss: 0.0258\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 55/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0023"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0023 - val_loss: 0.0203\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 56/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0016"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 0.0016 - val_loss: 0.0156\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 57/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0013"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0013 - val_loss: 0.0118\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 58/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0014"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0014 - val_loss: 0.0089\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 59/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0017"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0017 - val_loss: 0.0071\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 60/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0022"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0022 - val_loss: 0.0060\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 61/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0025"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0025 - val_loss: 0.0055\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 62/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0026"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0026 - val_loss: 0.0057\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 63/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0026"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0026 - val_loss: 0.0064\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 64/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0023"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 0.0023 - val_loss: 0.0076\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 65/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0019"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0019 - val_loss: 0.0093\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 66/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0016"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 0.0016 - val_loss: 0.0113\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 67/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0013"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0013 - val_loss: 0.0136\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 68/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0012"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0012 - val_loss: 0.0160\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 69/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0012"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0012 - val_loss: 0.0181\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 70/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0013"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 0.0013 - val_loss: 0.0198\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 71/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0014"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0014 - val_loss: 0.0209\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 72/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0015"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0015 - val_loss: 0.0213\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 73/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0016"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 0.0016 - val_loss: 0.0210\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 74/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0015"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0015 - val_loss: 0.0202\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 75/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0014"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 0.0014 - val_loss: 0.0189\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 76/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0013"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0013 - val_loss: 0.0174\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 77/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0011"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0011 - val_loss: 0.0158\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 78/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0011"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0011 - val_loss: 0.0143\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 79/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0010"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0010 - val_loss: 0.0130\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 80/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0010"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0010 - val_loss: 0.0120\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 81/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0011"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0011 - val_loss: 0.0114\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 82/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0011"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0011 - val_loss: 0.0110\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 83/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0011"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0011 - val_loss: 0.0110\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 84/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0011"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0011 - val_loss: 0.0113\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 85/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0011"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0011 - val_loss: 0.0118\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 86/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 0.0011"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 0.0011 - val_loss: 0.0125\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 87/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 9.9906e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 9.9906e-04 - val_loss: 0.0133\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 88/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 9.5899e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 9.5899e-04 - val_loss: 0.0142\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 89/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 9.3780e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 9.3780e-04 - val_loss: 0.0150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 90/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 9.3488e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 9.3488e-04 - val_loss: 0.0156\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 91/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 9.4357e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 9.4357e-04 - val_loss: 0.0161\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 92/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 9.5451e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 9.5451e-04 - val_loss: 0.0163\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 93/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 9.5936e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 9.5936e-04 - val_loss: 0.0163\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 94/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 9.5352e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 9.5352e-04 - val_loss: 0.0161\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 95/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 9.3717e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 9.3717e-04 - val_loss: 0.0156\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 96/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 9.1432e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 9.1432e-04 - val_loss: 0.0150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 97/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 8.9074e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 8.9074e-04 - val_loss: 0.0144\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 98/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 8.7159e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 8.7159e-04 - val_loss: 0.0138\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 99/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 8.5960e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 8.5960e-04 - val_loss: 0.0132\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 100/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 8.5449e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 8.5449e-04 - val_loss: 0.0127\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 101/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 8.5362e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 8.5362e-04 - val_loss: 0.0124\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 102/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 8.5331e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 8.5331e-04 - val_loss: 0.0122\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 103/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 8.5041e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 8.5041e-04 - val_loss: 0.0121\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 104/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 8.4335e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 8.4335e-04 - val_loss: 0.0121\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 105/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 8.3246e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 8.3246e-04 - val_loss: 0.0122\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 106/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 8.1954e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 8.1954e-04 - val_loss: 0.0124\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 107/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 8.0688e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 8.0688e-04 - val_loss: 0.0127\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 108/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 7.9636e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 7.9636e-04 - val_loss: 0.0129\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 109/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 7.8874e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 7.8874e-04 - val_loss: 0.0131\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 110/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 7.8361e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 7.8361e-04 - val_loss: 0.0132\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 111/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 7.7972e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 7.7972e-04 - val_loss: 0.0133\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 112/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 7.7562e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 7.7562e-04 - val_loss: 0.0132\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 113/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 7.7023e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 7.7023e-04 - val_loss: 0.0131\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 114/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 7.6324e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 7.6324e-04 - val_loss: 0.0129\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 115/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 7.5506e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 7.5506e-04 - val_loss: 0.0126\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 116/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 7.4655e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 7.4655e-04 - val_loss: 0.0123\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 117/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 7.3860e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 7.3860e-04 - val_loss: 0.0121\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 118/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 7.3174e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 7.3174e-04 - val_loss: 0.0118\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 119/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 7.2605e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 7.2605e-04 - val_loss: 0.0115\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 120/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 7.2114e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 7.2114e-04 - val_loss: 0.0114\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 121/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 7.1643e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 7.1643e-04 - val_loss: 0.0112\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 122/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 7.1139e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 7.1139e-04 - val_loss: 0.0111\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 123/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 7.0577e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 7.0577e-04 - val_loss: 0.0111\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 124/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 6.9964e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 6.9964e-04 - val_loss: 0.0111\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 125/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 6.9330e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 6.9330e-04 - val_loss: 0.0112\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 126/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 6.8710e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 6.8710e-04 - val_loss: 0.0112\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 127/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 6.8134e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 6.8134e-04 - val_loss: 0.0112\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 128/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 6.7611e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 6.7611e-04 - val_loss: 0.0113\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 129/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 6.7125e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 6.7125e-04 - val_loss: 0.0113\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 130/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 6.6659e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 6.6659e-04 - val_loss: 0.0112\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 131/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 6.6186e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 6.6186e-04 - val_loss: 0.0112\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 132/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 6.5694e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 6.5694e-04 - val_loss: 0.0111\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 133/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 6.5184e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 6.5184e-04 - val_loss: 0.0110\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 134/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 6.4666e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 6.4666e-04 - val_loss: 0.0108\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 135/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 6.4156e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 6.4156e-04 - val_loss: 0.0107\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 136/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 6.3666e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 6.3666e-04 - val_loss: 0.0105\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 137/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 6.3201e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 6.3201e-04 - val_loss: 0.0104\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 138/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 6.2756e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 6.2756e-04 - val_loss: 0.0103\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 139/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 6.2321e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 6.2321e-04 - val_loss: 0.0102\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 140/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 6.1888e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 6.1888e-04 - val_loss: 0.0102\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 141/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 6.1449e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 6.1449e-04 - val_loss: 0.0101\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 142/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 6.1007e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 6.1007e-04 - val_loss: 0.0101\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 143/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 6.0565e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 6.0565e-04 - val_loss: 0.0101\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 144/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 6.0129e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 18ms/step - loss: 6.0129e-04 - val_loss: 0.0101\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 145/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 5.9706e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 5.9706e-04 - val_loss: 0.0100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 146/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 5.9295e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 5.9295e-04 - val_loss: 0.0100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 147/150\n",
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 5.8894e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 16ms/step - loss: 5.8894e-04 - val_loss: 0.0100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 148/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 5.8501e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 5.8501e-04 - val_loss: 0.0100\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 149/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 5.8111e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 17ms/step - loss: 5.8111e-04 - val_loss: 0.0099\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 150/150\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\r",
+      "1/1 [==============================] - ETA: 0s - loss: 5.7722e-04"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
+      "1/1 [==============================] - 0s 19ms/step - loss: 5.7722e-04 - val_loss: 0.0098\n"
+     ]
+    },
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd5xU1fn48c8zZfsuvXcRRQFBg4UYkdg1lliiWKOJ+rXEqIlEjTEhxvw0mq9+jZqg39hFxZZEY//akMRCEayICghLXTps35nn98c5A3eH3WWAHWbJfd6v1+zMLXPuM3d27nPPuefeK6qKMcaY8IrkOgBjjDG5ZYnAGGNCzhKBMcaEnCUCY4wJOUsExhgTcpYIjDEm5CwRNENExovIo61UVqGIPC8ia0XkKRE5U0RebaWy54vIYa1RVlq5KiK7+tcTROT6TObdhuW02rpIK3eMiJS3drnG/CcKbSIQkQ2BR1JEqgPDZ7by4k4BugGdVPUHqjpRVY9o5WVkjapepKq/295yRKS/TxqxQNk71brIlP+sb4pIlYjMbilZi/MHEVnpH7eIiGRSln/vdSKyQETWicgTIlIWmN5RRCaJyAr/mJiaLiKdReRffplrRORdETkw8N6hIvKKf99mJxyJyKMissQvd46InJ82/VQR+VxE1ovIZyLy/SbKyPOfqTwwrm/a73OD/7/5uZ8+xv9mg9N/GHj/gyJSlzY9GpgeFZEbRWSxj+1DEWnvp01Ie1+tiKwPvPcnIjLNj3+wic9zqP88Vf476xeYNk5EPvHLnCci4zb7Z3DzHew/741NTc+G0CYCVS1JPYAFwHGBcRNbeXH9gDmq2tDK5Zq263HgQ6ATcB3wtIh0aWbeC4HvA8OBvYBjgf/KsKxzgLOBA4GeQCFwZ+C9NwIdgF2AgbgdkvF+2gbgR0AXP88fgOcDiboeeBL4cTNx3wT0V9Uy4HjgRhH5FoCI9AIeBX4GlAHjgMdEpGtaGeOA5cERqrog7fc5DEgCzwRmWxycR1UfSiv3lrTpicC03wLfBkb52M4GavyyL0pb9uPAU8Hl4tbp/ekrQ0Q6A88C1wMdgWnApOAsuO+rA3AU8BMRGZtWRhy4A3g/vfysUtXQP4D5wGFp48bjfgQPA+uBT4GRgek9cf+YFcA84KfNlP1boA73o9qA+1GdC0wJzKPARcCXwGrgbkD8tIHAG8BKYAUwEWjfUux+/AHAUiAaGHci8JF/vR/wLrAGWALcBeSlxbSrf/0gcGNg2jj/nsW4DUlw3u/hNlrrgIXA+MD7Fvh5N/jHqCbWxbeBqcBa//ztwLS3gN8B//LfyatA52bW+xigPDC8h3//Gv9dHh+YdgzwmS9zEXCVH98Z+Kd/zyrgHSCSwf/TbkAtUBoY9w5wUTPz/xu4MDD8Y+C9TMoCngbGpa2/GqDID78EXBKYfinwShMxRIDj/PfTNW3aroBu4TPv7v8nTvXD+wPL0+apAEYFhgcAnwNHB7+rJsr+DfBmc99tE/M3+n9Nm9bB/+8NzOB7LPb/Ewc3Me1G4MG0cRcC/057fzUwuJny/wTcmTbuGuCWlj5DNh6hrRFk6HjgCaA98BxuY4mIRIDngVlAL+BQ4AoROTK9AFX9DfD/gEnq9jLua2ZZxwL74vYKTwVSZQluz6snbmPWh017dM1S1feASuCQwOgzgMf86wRwJW5jN8p/hku2VK6IHAVcBRwODALSmzwqcXs97XFJ4eJAk8Bo/9zer4t308ruCLyA+4F0Am4DXhCRTmmf4TygK5DnY9lSzHHc9/Wqf99lwEQR2d3Pch/wX6paCgzFJV6AnwPluD3mbsAvcRtKROTPIvLnZhY5BJirqusD42b58c3NP6uZebdUlvgHgeF83HcDbqfiWBHpICIdgJNxyWHTG0Q+wiWP54C/qmqjPfSW+PVQBczGJYIX/aRpwOcicrxvivk+LqF9FHj7nbh1Wr2FxZwDpO/xdxWRZb6J5XYRKU6bfomIrBKR6SJycmD8MKABOEVElvomrUubWe7JuOQ1eQvxpTT6HlW1EviaJr53ERHgINxOSWpcP9yO1Q0ZLq/VWCJo2RRVfVFdtfIR3EYa3Aa7i6reoKp1qjoX+F9gbHMFZeBmVV2jqguAN4ERAKr6laq+pqq1qlqB2zgenGGZjwOnA4hIKW7P93Ff7nRVfU9VG1R1PnBPhuWeCjygqp/4f/TxwYmq+paqfqyqSVX9yC8v03i/B3ypqo/4uB7HbWCOC8zzgKrOUdVqXI1tRAblHgCU4NZxnaq+gdvTP91Prwf2FJEyVV2tqjMC43sA/VS1XlXfUb/bpqqXqGpzibMEV6MJWguUZjj/WqDEbyy2VNZLwPnijiO0A67244v88wxcwlzpHwmgUQJT1b1wTSRnAFOaibFJfh2U4jZqz+I29vjfzMO4HY9a//xf/n8GETkRiKnq31oqX0QOwiXhpwOjZ+O+9x64HZ1v4X4XKX/CJcKuuGaaBwPHPnoD7XA1rQG443fjReTwJhb/Q+Dh1Heega353sfjtr8PpMV9vapuyHB5rcYSQcuWBl5XAQW+/bQf0NMfYFsjImtwezbdWnFZJQAi0lXcAcBFIrIO1+7aOcMyHwNOEpF84CRghqp+48vdTUT+6feK1uFqLZmU2xPX5JPyTXCiiOzvD5JViMhaXJNXpvH2TC/PD/cKDDe5njKJWVWTzZR7Mi5JfiMib4vIKD/+VuAr4FURmSsi12T2MdiA27AGleGaGTKZvwzY4DdAWyrrflyyfQu3d/mmH586+PoUMAe3MSrD7aFu1htOVWt84r1GRIanT2+JqiZUdQpuI3sxgLgD2rfgmnHycDsDfxWREX7v/RZczWxLfgg8E9w4qupSVf3M72zMA36B26Cnps9Q1ZV+Z+JFXHPqSX5yqvZxg6pW+52VJ3Df/0Yi0sfH/PBWrIqMvncR+QmulvM9Va31447DNf9NIgcsEWybhcA8VW0feJSq6jFbfOfWuwnXHLGXuoNyZ9G4KaBZqvoZboN3NI2bhQD+gtuzGuTL/WWG5S7BNU+l9E2b/hiuiaGPqrYDJgTK3dKe1WJckg3qi2u33x6LgT6+SW+zclV1qqqegNuD/DuupoGqrlfVn6vqLrhayc9E5NAMlvcpsIuvhaUMJ9AM0MT8w5uZt8Wy/MbwN6raX1V7+/GL2LTOhgP3qGql35hOIG2jlyaOO7C8LWK4Y1rg9tgnq+o0H+NU3AHQw3B76/2Bd0RkKa4m0cPvlPRPFSYihcAP2LxZKJ3S8v9ucPpHgXEtOQfX3j93C/MFNfoefcIbSOPmnx/hjgMcqqrB7s2HAiP9OlgKnIZrbv7HVix/m1ki2DYfAOtE5Gpx5whExXW12zcLyyrF7Wms8T0xmuxy1oLHgJ/i2ueDvR9KcQd0N4jIYPyeXAaeBM4VkT1FpAh3IC893lWqWiMi++ESUEoFrvdHcxuaF4HdROQMEYmJyGnAnrhmnO3xPu7YxS9EJC4iY3Ab9ifEdV88U0TaqWo9bp0kAETkWBHZ1TfRpMYnml7EJqo6B5gJ/EZECnwzyF407vUS9DAuyfQSkZ64YxMPZlKWuO6hA8XZE9dEckOg9jMV13RU6DesF+LbsUXkABH5jl8HhSJyNa5W+76fLiJSgNujxy8/37/uKiJjRaTE//8fiWtqeyOw3INEZISff29c89FHwCe4nYkR/nE+sMy/DtY2T8QdqH8zMC7VfbSvj68PcDPwj8D0U3xcERE5Arfz9Jxfn1/jDrZfJyL5IrIHbqOb/j92Tuo7SFt2zK+TKBD16yTVy+pvwFAROdnP82tc54zZ/r1n4mrehzeRYK7HNVel1slzuObm89JjyIptPcr8n/Sg+V5DjwaG++P2ImJ+uCeuSr4U19PnvfQyWijrXDbvNbRrYPhBfI8B3IGm6bhkMBN/ALOl2NOW3Re38X0hbfxoXI1gA+6HcUNzMbF5r6Fr/OduqtfQKbhayHrcj+uutM9+Ay4hrMG13aevi+/4z7vWP38nMO0t4Pzm1mPa5xuTtp6GAG/7cj8DTvTj84CX/Xe4DrcB+46fdqVfv5W4ppbrA+VNACa0sN77+3irgS+C3xFug7ghMCy4ppJV/nELvtdYBmXt5sdV+fX+s7Q4BuAOlK/0Zb+MqwWCa/qY5b+rVX79jG7ifz74mO+ndfHzr/Hr7WPggrRl/wTXtLYemAv8PJPvKjD+FeB3TYz/Ga7GU4VLHHeyea+qtT6uWcDYtPf38uthg4/rv9Kmj/LfeWkTyx7fxDoZH5h+GO53Ve2/s/6BafPY1Hsw9Wjyf4gd3Gso1UXRGGNMSFnTkDHGhJwlAmOMCTlLBMYYE3KWCIwxJuRiW56lbencubP2798/12EYY8xOZfr06StUtckLH+50iaB///5MmzYt12EYY8xORUTSz9rfyJqGjDEm5CwRGGNMyFkiMMaYkNvpjhEYY8Kpvr6e8vJyampqch1Km1ZQUEDv3r2Jx+MZv8cSgTFmp1BeXk5paSn9+/dHJKML8IaOqrJy5UrKy8sZMGBAxu+zpiFjzE6hpqaGTp06WRJogYjQqVOnra41WSIwxuw0LAls2baso/AkgmWfwRs3QuWKXEdijDFtSngSwYo5MPlW2LAs15EYY3ZSJSWZ3Bl15xOeRBArcM8NtbmNwxhj2pgQJYJ892yJwBiznVSVcePGMXToUIYNG8akSe6e80uWLGH06NGMGDGCoUOH8s4775BIJDj33HM3znv77bfnOPrNhaf76MYagfVBNmZn99vnP+Wzxetatcw9e5bxm+OGZDTvs88+y8yZM5k1axYrVqxg3333ZfTo0Tz22GMceeSRXHfddSQSCaqqqpg5cyaLFi3ik08+AWDNmjWtGndrCFGNIM89W43AGLOdpkyZwumnn040GqVbt24cfPDBTJ06lX333ZcHHniA8ePH8/HHH1NaWsouu+zC3Llzueyyy3j55ZcpKyvLdfibsRqBMWank+mee7Y0d6/30aNHM3nyZF544QXOPvtsxo0bxznnnMOsWbN45ZVXuPvuu3nyySe5//77d3DELQtRjcAngkRdbuMwxuz0Ro8ezaRJk0gkElRUVDB58mT2228/vvnmG7p27coFF1zAj3/8Y2bMmMGKFStIJpOcfPLJ/O53v2PGjBm5Dn8zIaoRpA4WW43AGLN9TjzxRN59912GDx+OiHDLLbfQvXt3HnroIW699Vbi8TglJSU8/PDDLFq0iPPOO49kMgnATTfdlOPoNyfNVXHaqpEjR+o23ZimcgXcOhCO+SPsd0HrB2aMyarPP/+cPfbYI9dh7BSaWlciMl1VRzY1f3iahqKpg8VWIzDGmKDwJAI7WGyMMU0KTyKIxgGx7qPGGJMmPIlAxNUKLBEYY0wj4UkE4HoOWSIwxphGQpgI7BiBMcYEZS0RiEgfEXlTRD4XkU9F5PIm5hER+ZOIfCUiH4nIPtmKB7AagTHGNCGbNYIG4OequgdwAHCpiOyZNs/RwCD/uBD4Sxbj8ccIrEZgjMm+lu5dMH/+fIYOHboDo2lZ1hKBqi5R1Rn+9Xrgc6BX2mwnAA+r8x7QXkR6ZCsmYvl2iQljjEmzQy4xISL9gb2B99Mm9QIWBobL/bglae+/EFdjoG/fvtseiNUIjPnP8NI1sPTj1i2z+zA4+uZmJ1999dX069ePSy65BIDx48cjIkyePJnVq1dTX1/PjTfeyAknnLBVi62pqeHiiy9m2rRpxGIxbrvtNr773e/y6aefct5551FXV0cymeSZZ56hZ8+enHrqqZSXl5NIJLj++us57bTTtutjww5IBCJSAjwDXKGq6RcQb+ouy5td80JV7wXuBXeJiW0OJmrHCIwx22bs2LFcccUVGxPBk08+ycsvv8yVV15JWVkZK1as4IADDuD444/fqhvI33333QB8/PHHzJ49myOOOII5c+YwYcIELr/8cs4880zq6upIJBK8+OKL9OzZkxdeeAGAtWvXtspny2oiEJE4LglMVNVnm5ilHOgTGO4NLM5aQLF8qGl7N4UwxmylFvbcs2Xvvfdm+fLlLF68mIqKCjp06ECPHj248sormTx5MpFIhEWLFrFs2TK6d++ecblTpkzhsssuA2Dw4MH069ePOXPmMGrUKH7/+99TXl7OSSedxKBBgxg2bBhXXXUVV199NcceeywHHXRQq3y2bPYaEuA+4HNVva2Z2Z4DzvG9hw4A1qrqkmbm3X52QpkxZjuccsopPP3000yaNImxY8cyceJEKioqmD59OjNnzqRbt27U1Gxd83NzF/4844wzeO655ygsLOTII4/kjTfeYLfddmP69OkMGzaMa6+9lhtuuKE1PlZWawQHAmcDH4vITD/ul0BfAFWdALwIHAN8BVQB52UxHus+aozZLmPHjuWCCy5gxYoVvP322zz55JN07dqVeDzOm2++yTfffLPVZY4ePZqJEydyyCGHMGfOHBYsWMDuu+/O3Llz2WWXXfjpT3/K3Llz+eijjxg8eDAdO3bkrLPOoqSkhAcffLBVPlfWEoGqTqHpYwDBeRS4NFsxbMZqBMaY7TBkyBDWr19Pr1696NGjB2eeeSbHHXccI0eOZMSIEQwePHiry7zkkku46KKLGDZsGLFYjAcffJD8/HwmTZrEo48+Sjwep3v37vz6179m6tSpjBs3jkgkQjwe5y9/aZ0e9+G5HwHA85fD7Bdh3JetG5QxJuvsfgSZs/sRtMRqBMYYs5nw3KoS7FpDxpgd6uOPP+bss89uNC4/P5/3308/pSq3QpYICiBRC6rustTGmJ2Kqm5VH/1cGzZsGDNnztzyjK1oW5r7Q9Y05G9gb5eZMGanU1BQwMqVK7dpQxcWqsrKlSspKCjYqveFq0YQ9YmgoWZTUjDG7BR69+5NeXk5FRUVuQ6lTSsoKKB3795b9Z5wJYLUxt8OGBuz04nH4wwYMCDXYfxHClnTkN3A3hhj0oU0EdgxAmOMSQlZIggcIzDGGAOELhGkagR2jMAYY1JClgjy3LPVCIwxZqOQJQI7WGyMMelClgis+6gxxqQLWSLwNYKEJQJjjEkJWSKwGoExxqQLVyKIWvdRY4xJF65EYN1HjTFmMyFLBFYjMMaYdCFLBHaJCWOMSReuRBCNgUStRmCMMQHhSgRgt6s0xpg0IU0EdrDYGGNSQpgICqxGYIwxASFMBPl2z2JjjAkIYSKwGoExxgSFLxFE8+wYgTHGBIQvEViNwBhjGglhIrBeQ8YYExTCRFBgicAYYwJCmAisRmCMMUEhTQR2jMAYY1JCmAisacgYY4JCmAisRmCMMUEhTAQFdmaxMcYEhDARWI3AGGOCspYIROR+EVkuIp80M32MiKwVkZn+8etsxdJIqkaQTO6QxRljTFsXy2LZDwJ3AQ+3MM87qnpsFmPYXDTPPSdqIVK4QxdtjDFtUdZqBKo6GViVrfK32cbbVVrzkDHGQO6PEYwSkVki8pKIDGluJhG5UESmici0ioqK7VvixhvYWxdSY4yB3CaCGUA/VR0O3An8vbkZVfVeVR2pqiO7dOmyfUvdWCOwRGCMMZDDRKCq61R1g3/9IhAXkc5ZX7DVCIwxppGcJQIR6S4i4l/v52NZmfUFb0wEdozAGGMgi72GRORxYAzQWUTKgd8AcQBVnQCcAlwsIg1ANTBWVTVb8WwU8z2FLBEYYwyQxUSgqqdvYfpduO6lO1bcJ4L66h2+aGOMaYty3Wtox7NEYIwxjYQmEbz79UrO+N/3WFbjP3J9VW4DMsaYNiI0iWBdTT3//nola+p9a5glAmOMAUKUCIrzXAKoTPpLTFjTkDHGACFKBEX5UQA2JONuhNUIjDEGCFEiSNUI1idSicBqBMYYAyFKBEV5rkZQWa8QzbcagTHGeKFJBMX5rkZQXZdwXUitRmCMMUCIEsHGGkFdA8SLrEZgjDFeaBJBfixCNCJU1VqNwBhjgkKTCESEong0UCOwRGCMMRCiRACuC+mmGoE1DRljDIQsERTnxXyNwJqGjDEmJVSJoCg/SlVdwg4WG2NMQLgSQV6MytoGyLNjBMYYkxKqRFCcl6oRWNOQMcakhCoRFOXH7DwCY4xJE6pEUJwXtfMIjDEmTagSQVFeoEbQUAPJZK5DMsaYnAtVIij2vYZ04w3srVZgjDGhSgRFeTESSaUhWuBGWPOQMcaEKxEU+wvP1Uq+G1FXmcNojDGmbQhVIijyl6KuwScCqxEYY0y4EkHqLmWbEoF1ITXGmIwSgYhcLiJl4twnIjNE5IhsB9faUvctrsZuYG+MMSmZ1gh+pKrrgCOALsB5wM1ZiypLUjWCqqQlAmOMSck0EYh/PgZ4QFVnBcbtNDbepUxTicCahowxJtNEMF1EXsUlgldEpBTY6c7GSt23eEMy7kZYjcAYY4hlON+PgRHAXFWtEpGOuOahnUqq++j6RCoRWI3AGGMyrRGMAr5Q1TUichbwK2Bt9sLKjlT30U2JwGoExhiTaSL4C1AlIsOBXwDfAA9nLaosKYy7GsG6BqsRGGNMSqaJoEFVFTgBuENV7wBKsxdWdkQjQmE8yvp6AYlYjcAYY8j8GMF6EbkWOBs4SESiQDx7YWVPcX6UyvqkvyeBJQJjjMm0RnAaUIs7n2Ap0Au4NWtRZVFRXoyq2tQN7K1pyBhjMkoEfuM/EWgnIscCNaq60x0jAHcuQeXGG9hbjcAYYzK9xMSpwAfAD4BTgfdF5JRsBpYtxfkxqjcmAqsRGGNMpscIrgP2VdXlACLSBfg/4Onm3iAi9wPHAstVdWgT0wW4A3eSWhVwrqrO2Lrwt15RXpQNG5uGrEZgjDGZHiOIpJKAtzKD9z4IHNXC9KOBQf5xIa6LatYVbbxvsTUNGWMMZF4jeFlEXgEe98OnAS+29AZVnSwi/VuY5QTgYd8t9T0RaS8iPVR1SYYxbZPijfctLoSqldlclDHG7BQySgSqOk5ETgYOxF1s7l5V/dt2LrsXsDAwXO7HbZYIRORCXK2Bvn37btdCi/x9i61pyBhjnExrBKjqM8Azrbjspq5eqs0s+17gXoCRI0c2OU+mivNiVNY2+KYhu1WlMca0mAhEZD1Nb5wFUFUt245llwN9AsO9gcXbUV5GivJi1DYkScYKiFiNwBhjWk4EqprNy0g8B/xERJ4A9gfWZvv4ALgziwHqowXkWyIwxpjMm4a2log8DowBOotIOfAb/GUpVHUC7mDzMcBXuO6jO+Sy1kX+LmX1UkB+fRWogux099gxxphWk7VEoKqnb2G6Apdma/nNSdUIaiWfEk1Cog5i+Ts6DGOMaTMyPY/gP0aqRlArfuNvZxcbY0IudImgxN+cplrtBvbGGAMhTARlhan7FlsiMMYYCGMiKHC3Udh0A3trGjLGhFvoEkFpQeq+xVYjMMYYCGEiSB0j2Hjf4roNOYzGGGNyL3SJIBaNUJwXZXWiwI2otURgjAm3rJ1H0JaVFcZZ2eCvnFG7PrfBGGNMjoUyEZQWxFhR7wcsERhjQi6kiSBORZ1vFbNEYIwJudAdIwAoK4ixplYgVgC163IdjjHG5FQoE0FpQZx1NfWQX2o1AmNM6IU0EcRYX9PgEoF1HzXGhFwoE0FZYZz1NfVoXonVCIwxoRfKRFBaEKM+oSTzrGnIGGNCmQhS1xtqiBXbwWJjTOiFMhGkrjdUF7WmIWOMCWUiSNUIaqJFlgiMMaEXzkTg70lQHbFEYIwxoUwEpb5GUEmRu2dxQ22OIzLGmNwJaSJwNYJKCt0IqxUYY0IslIkgdYxgnaYSgfUcMsaEVygTQVFelGhEWJtM3ZPAagTGmPAKZSIQEUoLYqxJWCIwxphQJgJwxwlWN+S7AUsExpgQC28iyI+zoj6VCOzCc8aY8AptIigrjFFRl+cG7GCxMSbEQpsISgviLK9zvYesacgYE2YhTgQxVtREQaKWCIwxoRbaRFBWEGddbQPk24XnjDHhFuJEEGNDbQNqt6s0xoRceBNBYRxV/M1p7GCxMSa8QpsIUtcbaohZ05AxJtxCnAhcj6H6WLElAmNMqIU2EaQuPFcbsURgjAm30CaC9kUuEVTZzWmMMSGX1UQgIkeJyBci8pWIXNPE9DEislZEZvrHr7MZT1DnEnd5iQ1aaInAGBNqsWwVLCJR4G7gcKAcmCoiz6nqZ2mzvqOqx2YrjuZ0KPb3JEgWQH0lJBMQie7oMIwxJueyWSPYD/hKVeeqah3wBHBCFpe3VfJjUXcF0tSlqOvswnPGmHDKZiLoBSwMDJf7celGicgsEXlJRIY0VZCIXCgi00RkWkVFRasF2Kk4j5V2KWpjTMhlMxFIE+M0bXgG0E9VhwN3An9vqiBVvVdVR6rqyC5durRagJ1K8llRn7oCqSUCY0w4ZTMRlAN9AsO9gcXBGVR1napu8K9fBOIi0jmLMTXSsTiP5bWWCIwx4ZbNRDAVGCQiA0QkDxgLPBecQUS6i4j41/v5eFZmMaZGOpfksbjGX4q6xi4zYYwJp6z1GlLVBhH5CfAKEAXuV9VPReQiP30CcApwsYg0ANXAWFVNbz7Kmo7FeXxQXQB5QPWqHbVYY4xpU7KWCGBjc8+LaeMmBF7fBdyVzRha0qk4n4pkiRuobL2D0MYYszMJ7ZnFAJ1K8lhHMRqJQeWKXIdjjDE5Ee5EUJwPCPX5naxGYIwJrVAngo7FrsdQTV4HqxEYY0Ir1Imgc4lLBBtiHaxGYIwJrVAngg6+RrAu0h6qrEZgjAmnUCeCeDRCWUGM1ZRa05AxJrRCnQjAXY56uZa5i87VV+c6HGOM2eFCnwg6FuextL7UDVitwBgTQqFPBJ1K8lhUX+QG7ICxMSaEQp8IOhbns6C22A1YjcAYE0KhTwSdS/KYV1XoBqznkDEmhEKfCDoW51Gh7dyANQ0ZY0Io9ImgU0k+VeSTjOZbIjDGhJIlguI8QKgv6ASVO+xWCMYY02aEPhF0K3P3LK6Kd7QagTEmlEKfCHp3cF1H10qZJQJjTChl9cY0O4OCeJRuZflUaBn9qxZuUxmqir/jZnas+BI+fgoK2kHHgTDoCIiEPocbY1pJ6BMBQL+OxSxeXwK1FaAKGW7U11bVc/+/5vHAv+bRs30hN1DD9ewAABQaSURBVJ00jL37dmi9wBrq4J0/wpTbIVG3afyuh8NJ90JRx9ZbljEmtGy3EujTsYgFNUXQUOOuOZSBytoGjvnTO9zx+pfs278ja6vrOekv/+aet79uvcBe+Bm8/QfY8wS46kv4xTw45o8w7224ZzSsnt96yzLGhJYlAqBvxyLmV6cuM5HZSWV3vvEVi9ZUM/H8/bnv3H159crRHDWkO394eTYzFqze/qA+fBQ+fAQO+jmc/Fco6epqAPtdAD96GWrXwaSzoK5q+5dljAk1SwRAv05FrKTMDWSQCL6u2MB9U+Zyyrd6c+CunQEoLYhzyyl70aNdIVc9OYvqusS2B7TsM3jh5zBgNHz3us2n9/oWnPRXWPoJ/PNK15xljDHbyBIBrmlohaYSwfItzv/b5z+jIB7l6qMGNxqfSgZzV1Ty369+se0BvfZriBfCyfdBJNr0PLsdAWOuhY+egFlPbPuyjDGhZ4kA1zS0SN2e/Zba3b9ctp7Jcyq4eMxAupTmbzb9wF07c9rIPjz07nwWrtqGZpuFH8BXr8GBl7vmoJaMHge994NXfmknwxljtpklAtyF52rzOlAZaw8VLe/JT5q6kFhEOHVkn2bnueLwQYgId7z+5dYH8+bvoagz7HfhlueNROC4O9zxgld/tfXLMsYYLBEAICKuVhDtAyvmNDtfXUOSZz9cxGF7dKNzyea1gZQe7Qo554B+PDujnK+Wr888kPn/grlvwXeuhLzizN7TbU/49k9h1mMw9+3Ml2WMMZ4lAq9PxyLmJHtCxexmD76+/vkyVlXWcdp+zdcGUi757q4UxqP896vNJ5ZGVOHN/wcl3WDfH29N6HDwL6DDAHfguL5m695rjAk9SwRev45FfFTTDapXN9tzaNK0hfRoV8DoQV22WF7H4jzOP2gXXvpkKR+Vr9lyAPMmwzdTXHfReOHWBR8vhOP+B1Z97U5AM8aYrWCJwOvbqYjZiR5uYMXmxwmWr6th8pwKTt6nN9FIZmcen3/QANoXxfnjlmoFqdpAWS/Y54dbG7qzyxjYa6w7C3n559tWhjEmlCwReH06FvFVspcbaOKA8XOzFpNUOGmfXhmXWVoQ55IxA5k8p4L35rbQq+fr12Hhe742ULC1oW9y5O8hvwyevxySyW0vxxgTKpYIvF27lLCYTtRHC5s8YPzsjEUM79OeXbqUbFW554zqT7eyfG595Qu0qWMPqdpAu76w99nbGr5T3Nklg4Xvw4wHt68sY0xoWCLw+nQson+nYsqjvTerEXyxdD2fLVnHiSN6bhq5YTn832/hzZtg8YfNHmAuiEf56aGDmP7Nat78oomT1ea8Aoumw8HjIJa3/R9k+OnujOTXxsP6pdtfnjHmP54lgoAxu3dlVk13tGJ2o/F/n7mIaEQ4drhPBP/6E/xpb/jXHe6icPeOgUdPhtqmL1h36sg+9O1YxK2vzCGZDCQMVXfeQIf+bgPeGkTg2P+BRC08eyEkt+NSF8aYULBEEPDdwV2Zk+iJrF8CNesAaEgk+ceHizh4ty7u3IEZj8Br10P/g+DSD2DcV3DEja7//8PHN3mGbzwa4WeH78bnS9bxz4+XbJow+5+w9CM4+BqIxjOOM5FU6hMtHAPoNBCOudVdpfStmzMu1xgTTnY/goD9B3TkqYg/R2DFl9D7Wzwzo5zFa2sYf/wQWDILXrwKBhwMYyduug7Qty9zN4x5+jx45AQ47yXIL21U9vHDezLh7a/5w0uzOXi3LrSL1sMr10Hn3WHYD1qMa21VPc/MKOf12cuYOn81dQ1JRGC3rqXsO6ADJ+3Tm737tG98c5y9z4YF78HkW6HnCBj8vdZcVcaY/yCWCAIK4lHK+g2FctCF71PbbQS3v/YlI/q05/CBRXDPOVDYsemLwQ0+Bk6bCI+dCk+dC6dPguim1RuJCL8/cRin3fMu456axT09/oms+QbOfbHRfEGVtQ3cN2Ue//vOXNbXNDCoawln7t+X9oV5NCSTzCpfy7MzFvHoewsY1qsdl4wZyJFDuhOJiGsiOuaPrivpU+fCqY/A7kdlb+W1ZP1SKJ8GSz+GteWwYRloAiQCxV2gtAd0H+quqtq+X8Y3BjLGtA5LBGn2HLI3Hy7YlcFT7uKh6u+ydF0Nt582Ann1V7D6G7e3X9LMCWWDDoNjb3PdN5+/HI6/s9EtJb/VrwPXHD2Yp158heS8O4mOOAv6H7hZMarKK58u5bfPf8aStTUcsWc3rjhsN/bsWbbZvBtqG/jbh4t4YMo8Lp44gz17lPGzw3fj0D26InlFcPaz8MiJ8OTZLp69Tsv+hjaZcAfAv3gJ5rwMyz/zEwRKu7uzp6NxSDa4A/Prl7jX4BJtr32gz/7Q9wDoNRLyirIbrzEhJ012aWytwkWOAu4AosBfVfXmtOnipx8DVAHnquqMlsocOXKkTps2LUsRw9K1Ndxw2+38WW5mXP2FLB/4Ax76zmq3p3/gFXD4b7dcyJs3wds3w/Az4IS7GtUedNU8Vv75KKS+kv/d6wmuOH4UBfFN02csWM0tL8/mvbmrGNy9lBu/P5SR/bd8S8qGRJJ/zFzMHa9/yYJVVQzv3Y4LRw/kiCHdiNethcdPhwXvwh7Hw9F/gLKeTZajqqyuqmf+ykrWVtWjKPFohA5FeXQpzadLSb6rcaSrXg1fvwFfvuYeVStAotDv2+4ey30PgO57NX2eREMdLP8UFs2AxTPc8/LPAYVIDHoMh76jXBndhriuts3UorZJMgFVK11PsMrlsKHCPy+H6lVQXw0Nte45UeeSWDRv0yOv2DUF5pVAfol7Dr7OL4G80k3D8aJtu+d0MumWH3w01EKi3nUOSNS516lxyQa3/qLxTTFvHA68jvjpweFINDs7DKq+h11wu+NrsFYTzCoRma6qI5uclq1EICJRYA5wOFAOTAVOV9XPAvMcA1yGSwT7A3eo6v4tlZvtRACwfF01efcfQrJ6Hex/ER2n3uaaLy58E2LNX2yukbdvcT2CBh4Ch/zKbQTLp8Ez56N1G3hgl9u4YUYBvTsUsl//jpQUxHh/7iq+WLaeTsV5XHbIrpx1QD9i0a3bYNQnkjw7o5y73vyKhauq6Vqaz+F7duPAgR3Yd9EjdJ7636BJagYexZIehzEnthuf1XRg3qo65q+oZP7KStbXNDRbfn4swu7tkwwvXc/Q/OXsrnPpt2EW7VfNRDSJFnZAdzmEyOBjYNdDSeS3p6Y+QXV9guq6BLUNCarrktQ0uOHq+gR1DUni0QiFeVEKYhEK4lGKdT1lFTMpXvYBeYunElsyA0nUuiAicdfTqtNAV8MoaA8F7aCwPcQKXZOTCCBuY1hf6TbidVVQs9Zt5CsrNm3wq1aCNnHwPZoPRZ3cJTzihe67j+a5MjdudGugrtL1GKuvzPBbEpc8UklCom75mnBJSZPueeNy/CPZ/PfS+iSQJHzSVf9H1a+vwEa9xeeNb85sucHvL/VaImxKGMHXmcwXAaHpaRAoQwIxSBPT2Yb3ZDqdJqazeRlDToR9tu18o1wlglHAeFU90g9fC6CqNwXmuQd4S1Uf98NfAGNUdUkTRQI7JhEA8Pnz7laQ4PZGj78TOg/aujLevxfevNFtfKL5bq+tqBOc/XfosRdvzF7GxPcW8MnitayrbmBk/w4cvFsXTt+vL8X527fHm0gqb89ZzuMfLOTfX62g0t8xrY8s48zo65wafYuO4rq7JlVYJyUkogVEolGi0SixaJRIaq8wmUDqq5CGaqKJaqK6qUtqvUb5TPvxdnIv3kqMYKbuSpIIEYFoRKhPtM7/Vx71DJH5DIosYtfoMvrLUvqxhE6spYxK8iSzjWSV5rOS9qyijJW0YyXtWUk7VtGOFf71SnHjN1DIxh9mE9KnCEkKqaWYaoqpoYjqja/do3rjuCJqKPLjIiRJEiFJhETgOUGUemIbH3US96/jgfGbXteljUsQIS5JYiSI0UCMBuI0uGFN+NdueON4P2/6MIAiaQ83jsDrJJs2WsH5CM4rm+aXQAmCEvFTIk0sbdM4Ni5p03PT722uvEijqBTRxlHT6FM0Hi+B96WGU58oOJzp9PTlpi8nOG7lwJM4+KxfNvs/2ZJcJYJTgKNU9Xw/fDawv6r+JDDPP4GbVXWKH34duFpVp6WVdSFwIUDfvn2/9c0332Ql5kaSSfjgXncQs/93tr2c2vUw42FYtxh67+tO9iravKlHVRv3+mlF9Ykknyxay8LV1SxfV4OIUBJTBrCQAXVz6Fi/nGj1Srd3m9rbCz4iUdecES9ye8aF7aFdH7TDANaW7sqCdQkWrKpi+bpa6hJJ6huS1CWSNCSVgliUwrwIhfEo+fEohfEoBf65MC9CfixKfixCXSJJTX2SWl97qKlPbqwt1DUkqG1IuteJJLUNyc3Ox4glaylMrCOWrEMDP/gkUeqihdRHCqiP5JPcwmExzXDPtbV/NlvzO8x0zkyLbO3PvDWrJvOPveO/l9YqK9P1m4kxu3flmGE9tum9LSWCbB4sbmqrlr5GMpkHVb0XuBdcjWD7Q8tAJAIHXLT95eSXwqhLtzhbtpIAuPMY9u7bgb37dkibsgtw8DaXK0B7oH0Z7NW7/XZEaIzJpWyeUFYOBC/c3xtYvA3zGGOMyaJsJoKpwCARGSAiecBY4Lm0eZ4DzhHnAGBtS8cHjDHGtL6sNQ2paoOI/AR4Bdd99H5V/VRELvLTJwAv4noMfYXrPnpetuIxxhjTtKyeUKaqL+I29sFxEwKvFdhyA7oxxpissYvOGWNMyFkiMMaYkLNEYIwxIWeJwBhjQi6rF53LBhGpALb11OLOwIpWDCcbLMbWYTG2Dotx+7WV+PqpapOXTt7pEsH2EJFpzZ1i3VZYjK3DYmwdFuP2a+vxgTUNGWNM6FkiMMaYkAtbIrg31wFkwGJsHRZj67AYt19bjy9cxwiMMcZsLmw1AmOMMWksERhjTMiFJhGIyFEi8oWIfCUi1+Q6HgAR6SMib4rI5yLyqYhc7sd3FJHXRORL/5x+R5kdHWdURD70d5Rri/G1F5GnRWS2X5ej2mCMV/rv+BMReVxECnIdo4jcLyLLReSTwLhmYxKRa/3v5wsROTKHMd7qv+uPRORvItI+MK1NxBiYdpWIqIh0zmWMWxKKRCAiUeBu4GhgT+B0Edkzt1EB0AD8XFX3AA4ALvVxXQO8rqqDgNf9cC5dDnweGG5r8d0BvKyqg4HhuFjbTIwi0gv4KTBSVYfiLss+tg3E+CBwVNq4JmPy/5djgSH+PX/2v6tcxPgaMFRV9wLmANe2wRgRkT7A4cCCwLhcxdiiUCQCYD/gK1Wdq6p1wBPACTmOCVVdoqoz/Ov1uA1YL1xsD/nZHgK+n5sIQUR6A98D/hoY3ZbiKwNGA/cBqGqdqq6hDcXoxYBCEYkBRbg78eU0RlWdDKxKG91cTCcAT6hqrarOw91DZL9cxKiqr6pqgx98D3dnwzYVo3c78Asa3343JzFuSVgSQS9gYWC43I9rM0SkP7A38D7QLXWnNv/cNXeR8T+4f+ZkYFxbim8XoAJ4wDdf/VVEittSjKq6CPgjbs9wCe5OfK+2pRgDmouprf6GfgS85F+3mRhF5HhgkarOSpvUZmIMCksiaOrO8G2m36yIlADPAFeo6rpcx5MiIscCy1V1eq5jaUEM2Af4i6ruDVSS+6aqRnw7+wnAAKAnUCwiZ+U2qq3W5n5DInIdrnl1YmpUE7Pt8BhFpAi4Dvh1U5ObGJfzbVFYEkE50Ccw3BtXNc85EYnjksBEVX3Wj14mIj389B7A8hyFdyBwvIjMxzWnHSIij7ah+MB9t+Wq+r4ffhqXGNpSjIcB81S1QlXrgWeBb7exGFOai6lN/YZE5IfAscCZuulkqLYS40Bc0p/lfzu9gRki0p22E2MjYUkEU4FBIjJARPJwB2uey3FMiIjg2rY/V9XbApOeA37oX/8Q+MeOjg1AVa9V1d6q2h+3zt5Q1bPaSnwAqroUWCgiu/tRhwKf0YZixDUJHSAiRf47PxR3PKgtxZjSXEzPAWNFJF9EBgCDgA9yEB8ichRwNXC8qlYFJrWJGFX1Y1Xtqqr9/W+nHNjH/6+2iRg3o6qheADH4HoYfA1cl+t4fEzfwVULPwJm+scxQCdcj40v/XPHNhDrGOCf/nWbig8YAUzz6/HvQIc2GONvgdnAJ8AjQH6uYwQexx2zqMdtrH7cUky45o6vgS+Ao3MY41e4dvbUb2ZCW4sxbfp8oHMuY9zSwy4xYYwxIReWpiFjjDHNsERgjDEhZ4nAGGNCzhKBMcaEnCUCY4wJOUsExuxAIjImdRVXY9oKSwTGGBNylgiMaYKInCUiH4jITBG5x9+TYYOI/LeIzBCR10Wki593hIi8F7g+fgc/flcR+T8RmeXfM9AXXyKb7p8w0Z9tbEzOWCIwJo2I7AGcBhyoqiOABHAmUAzMUNV9gLeB3/i3PAxcre76+B8Hxk8E7lbV4bhrCy3x4/cGrsDdG2MX3DWdjMmZWK4DMKYNOhT4FjDV76wX4i6+lgQm+XkeBZ4VkXZAe1V9249/CHhKREqBXqr6NwBVrQHw5X2gquV+eCbQH5iS/Y9lTNMsERizOQEeUtVrG40UuT5tvpauz9JSc09t4HUC+x2aHLOmIWM29zpwioh0hY338e2H+72c4uc5A5iiqmuB1SJykB9/NvC2uvtKlIvI930Z+f469ca0ObYnYkwaVf1MRH4FvCoiEdxVJS/F3fRmiIhMB9bijiOAu1zzBL+hnwuc58efDdwjIjf4Mn6wAz+GMRmzq48akyER2aCqJbmOw5jWZk1DxhgTclYjMMaYkLMagTHGhJwlAmOMCTlLBMYYE3KWCIwxJuQsERhjTMj9fz5pEvxp0VaKAAAAAElFTkSuQmCC\n",
+      "text/plain": [
+       "
    "
+      ]
+     },
+     "metadata": {
+      "filenames": {
+       "image/png": "/Users/hjensen/Teaching/FYS-STK4150/doc/src/LectureNotes/_build/jupyter_execute/chapter10_11_687.png"
+      },
+      "needs_background": "light"
+     },
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD4CAYAAADhNOGaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deVhV1frA8e/LYZZJEBFH1JwnVHLIqUHL1DIb1WtZVmbZeLv9stnu7ZaZt3uzwdLMobzZpGnpTZudB1RUHFFERBGcAZlh/f7giCgHBQ9wDvB+nuc855y11t77ZZe87LX2WluMMSillKq5XBwdgFJKKcfSRKCUUjWcJgKllKrhNBEopVQNp4lAKaVqOFdHB3Al6tSpY8LCwq5s45QUyM0t13iUqjJcXcHPz9FRKAfZtGnTcWNM8MXlVTIRhIWFERkZeWUbz58P9eqVb0BKVRVHj8Lw4Y6OQjmIiBy0Va5dQ0opVcOVSyIQkYEiskdE9onIBBv1IiJTrfXbRKRLabdVSilVsexOBCJiAT4EbgbaAiNEpO1FzW4GWlhfY4FpZdhWKaVUBSqPMYJuwD5jTCyAiMwHhgI7i7QZCsw1BetZrBORABEJBcJKsa1SqgbIyckhISGBzMxMR4dS5Xl6etKwYUPc3NxK1b48EkED4FCR7wlA91K0aVDKbZVSNUBCQgK+vr6EhYUhIo4Op8oyxnDixAkSEhJo2rRpqbYpjzECW//FLl7JrqQ2pdm2YAciY0UkUkQijx07VsYQlVLOLjMzk6CgIE0CdhIRgoKCynRlVR6JIAFoVOR7Q+BIKduUZlsAjDHTjTERxpiI4OBit8GWyvu/xvDt6WQ+PbKE1Nz0K9qHUqriaBIoH2U9j+WRCDYCLUSkqYi4A8OBxRe1WQzcZ717qAdwxhiTWMpty8W+5FT+9fNeHkz4g4djphC85g5G7nybNWei0aW4lVI1md2JwBiTCzwOLAN2AV8bY3aIyDgRGWdtthSIBfYBM4DHLrWtvTHZ8nVkAoYc0lx/AyDLZPLlsZ/oFfUEYevuZ3L8fJKzT1XEoZVSVcTp06f56KOPHB1GpSuXeQTGmKXGmJbGmObGmH9ayz42xnxs/WyMMeOt9R2MMZGX2rYi3NqpPhGtD5AvZ4rVxWfH8/yBTwhdexcDt77K0hPryDN5FRWKUspJlZQI8vKq9++DGjOzuH0Df9697Xaeq30bgZYgm23yyWPZ6ZUMjn6BuquH83zsp8Rm2ByyUEpVQxMmTGD//v2Eh4dz9dVXc9111zFy5Eg6dOhAXFwc7du3L2w7ZcoUJk6cCMD+/fsZOHAgXbt2pU+fPuzevdtBP8GVqZJrDV2pJgFNmFz3Ht7s8DjLTm5gasJSfjm9lnyKZ/uTeceZfGgekw/N42qfcJ5sMIg7gvviZfFwQORK1SxhE5ZU2L7jJg0usW7SpElER0cTFRXFH3/8weDBg4mOjqZp06bExcWVuN3YsWP5+OOPadGiBevXr+exxx7jt99+q4DoK0aNSgTnuIqFwUE9GRzUk6Tsk8w+upwPDy/lUPYhm+03pkVx754oHomZyojgG/hro9toWyuscoNWSlW6bt26XfZe/LS0NNasWcNdd91VWJaVlVXRoZWrGpkIigpxD+T5xsP5v0b3sDZlB+8fXsKC43+QbYrfg5uen8bMpEXMTFrEtf7debXJPVwbEK63vClVTdWqVavws6urK/n5+YXfz92nn5+fT0BAAFFRUZUeX3mp8YngHBHhGv/2XOPfntTcJ5if/BtTE5YSnbHLZvs/zqznj23raeXZglfC7ubu4Gtxc9HTqVR5uFT3TUXy9fUlNTXVZl1ISAjJycmcOHECHx8ffvzxRwYOHIifnx9Nmzblm2++4a677sIYw7Zt2+jUqVMlR3/lasxgcVn4unrzcP0hbO/2EdERn/F46F34uvjbbLsnM4ZRu/9J/bUjmRz/FSm5Zys5WqVUeQkKCqJXr160b9+e55577oI6Nzc3Xn31Vbp3786QIUNo3bp1Yd28efOYOXMmnTp1ol27dixatKiyQ7eLVMXJVBEREaayH0yTnZ/DgmMreePgN+zIKPmOgFv97+OL9vfj66rdRcoJOfGDaXbt2kWbNm0cHUa1Yet8isgmY0zExW31iqCU3F3cGB5yPduv/oiV4VMZENCLi5dKEuPO5qM3cs0f6by5O5sjGfm2d6aUUk5EE0EZiQi9/TuwvNMb7Ll6DveH3IIb7gDUyrseCwGk5sL0uBz6rsjg6a2ZRKfkcSTrOPlGE4NSyvno6KYdWno3YlbrvzK52RjeP7wI//w+LD0i7D9b0N2Wa+D7xDwWJuZy2vtZarsLLze+i3vrDcDDxd3B0SulVAG9IigHwe4B/L3paJ5t3oyfe3sxs4sH3WufP7WZLptIMfEczDrIwzFTaLh2FCez0xwYsVJKnaeJoJy5iHBDXVe+6u7F4p6e3FLPQorrggvanM26ijvWufDd4Rxy86veYL1SqnrRRFCBOvpb+FdHCzcF18GlyKkOyBnOgXTDs9uzuX5lBl8n5JCjCUEp5SCaCCqYu4sb33d4hdju83gk9HbCvfpSx9K8sD4+w/B/0dlctzKDyfv38ubBL0nLy3BgxEqp8vDHH38wZMgQABYvXsykSZNKbHuly19PnDiRKVOmXHGM52giqCRNPOvxccsn2Hz1RFb18+ZvLdwIKPJc6YQMw+txX/BS3HRC1wznHwfn6eQ0pZzQlSxJfeuttzJhwoQS6x39HAS9a6iSiQh+bvB4c3fub+LG3PgcZhzIISk3nnTLKgDS8lN4Ne5T3o7/ir81vJNnGt2Ov6uPgyNXqnLJ6+U7KdO8dvnu17i4OAYOHEj37t3ZsmULLVu2ZO7cubRt25YxY8awfPlyHn/8cQIDA3nttdfIysqiefPmzJo1Cx8fH3766Seefvpp6tSpQ5cuXQr3O3v2bCIjI/nggw9ISkpi3LhxxMbGAjBt2jSmTp1auPz1gAEDeOedd3jnnXf4+uuvycrKYtiwYbz++usA/POf/2Tu3Lk0atSI4OBgunbtave50SsCB/JxFR5r5s6qft6EBS0EufB/1LP5qbweP4vQNcN5KXYWp3Jsr4GilCo/e/bsYezYsWzbtg0/P7/Cv9Q9PT1ZtWoV/fv354033uCXX35h8+bNRERE8O6775KZmcnDDz/MDz/8wMqVKzl69KjN/T/55JP069ePrVu3snnzZtq1a8ekSZNo3rw5UVFRvPPOOyxfvpyYmBg2bNhAVFQUmzZtYsWKFWzatIn58+ezZcsWFixYwMaNG8vlZ9ZE4ARquQo/dnqcCY3uxUu8i9VnmLO8eWgu9dcOZ8L+mZzMSXFAlErVDI0aNaJXr14AjBo1ilWrCq7U77nnHgDWrVvHzp076dWrF+Hh4cyZM4eDBw+ye/dumjZtSosWLRARRo0aZXP/v/32G48++igAFosFf//i65gtX76c5cuX07lzZ7p06cLu3buJiYlh5cqVDBs2DG9vb/z8/Lj11lvL5WfWROAkAt38eKvZGI5c8xWvNL4fb5fiXUGZJp23E76g4doRTDwwVweVlaoAFy8rf+77uSWpjTEMGDCAqKgooqKi2LlzJzNnzrS57ZUyxvDCCy8UHmPfvn08+OCD5XqMojQROJkAVx/+3nQ0R3p+ycQmY6jl4lusTYZJ5/X4WTRYM4J3D31LVn62AyJVqmKZ10y5vkorPj6etWvXAvDll1/Su3fvC+p79OjB6tWr2bdvHwDp6ens3buX1q1bc+DAAfbv31+4rS033HAD06ZNAwoGnlNSUootf33TTTfx2WefkZZWMPH08OHDJCcn07dvXxYuXEhGRgapqan88MMPpf65LkUTgZPyd/XhtbB7Sew5nzfCHra5DHZK/hmejf2QRmvv5dPEpeSa6v2AbaUqQ5s2bZgzZw4dO3bk5MmThd045wQHBzN79mxGjBhBx44d6dGjB7t378bT05Pp06czePBgevfuTZMmTWzu/7333uP333+nQ4cOdO3alR07dhRb/vrGG29k5MiR9OzZkw4dOnDnnXeSmppKly5duOeeewgPD+eOO+6gT58+5fIz27UMtYgEAl8BYUAccLcx5tRFbRoBc4F6QD4w3RjznrVuIvAwcMza/EVjzNLLHdcRy1A7WlpeBu8lLOSt+Pmczbc9aNzYvRFTmo/hjuC+uIjmeGWDLkN9SXFxcQwZMoTo6GiHxlEeKnMZ6gnAr8aYFsCv1u8XywWeNca0AXoA40WkbZH6fxtjwq2vyyaBmsrH4sVLTUaS0PO/PN9wFO7iWaxNfPYh7t71Om03jGPZyQ1UxWdNKKUqn72JYCgwx/p5DnDbxQ2MMYnGmM3Wz6nALqCBncetsQJcfZjU/EHie8xjXOjtWHAr1mZPZgwDtz/P1ZufZvWZ7Q6IUqmqKSwsrFpcDZSVvYkgxBiTCAW/8IG6l2osImFAZ2B9keLHRWSbiHwmIrUvse1YEYkUkchjx46V1KzGCHEPZFrLJ9jffS4jggciNv5TbkrbRu+oJ7k+agJb0/Y5IEqlykavYstHWc/jZROBiPwiItE2XkPLciAR8QG+A542xpy7EX4a0BwIBxKBf5W0vTFmujEmwhgTERwcXJZDV2tNPOvx37bPs/PqWQyq3ddmm9/PrGf8zhUc1iemKSfm6enJiRMnNBnYyRjDiRMn8PQs3n1ckssuMWGM6V9SnYgkiUioMSZRREKB5BLauVGQBOYZYwrXZDbGJBVpMwP4sdSRqwu09m7Mko6vsyl1D0/vm8mqlPMzDl2MH/Enb+G6FRmMauzK483dCXTXZyor59KwYUMSEhLQK377eXp60rBhw1K3t3etocXAaGCS9X3RxQ2kYPbDTGCXMebdi+pCz3UtAcOAmtc5V866+rZiZefJ/Hk6iidjZrAtfSf+OXfjgjfZBj47mMs3h3N5rJkbDzRxw9OiCUE5Bzc3N5o2beroMGoke8cIJgEDRCQGGGD9jojUF5FzdwD1Au4FrheRKOtrkLVusohsF5FtwHXAM3bGo6z6BYQTFfEBS9q/xXfhtxPuf/4/dWouvL03h+tWptF14/PMTFxKns5BUKrGsmsegaPUxHkE9jLGsDw5j7f3ZhNrfaZymuUXTrj/B4CrPJvxUYtHGRBY7BZjVZ048TwCVfEqah6BqiJEhJtCXFnWy4t/tHWntnsmp90+L6zflxnLuF1LiEnTAWWlahpNBDWMm4twb2M3/tExDpEiq5gaV7LS7uWmVRm8EJ1FcpYmBKVqCk0ENdSQOl3Z130utwfdAIBf7q24mhDygS8Tcrl2RQbv7csmPbfqdR0qpcpGn1BWgzXxrMd37V8mMvUusnNC+Gi/C6tOFFwJpOfBv/flMO9QLrX8p3JXSBvG1h+Cq1gcHLVSqrzpFYEiwrcV1wQG8HmEJ7O7etDK5/wtpQdzovgjdRnj9/2Hq9aN4ccTa3XCj1LVjCYCVUhEuDbYlaW9vHi7vTvBHobTbp8V1h/MjueW6Be5ZsuzbE+LdWCkSqnypIlAFWMR4Z6GbnzQ5SjGklCsfl3qFjptepgHd/+H4zlnHBChUqo8aSJQJerufxUHun/OiOCBwIUzkA35fJa0iCbrRvHvQ9+Rk5/rmCCVUnbTRKAuqYFHMP9t+zxRXadzjW+XYvXp+Wn8NfYDWqx/kGUnNzggQqWUvTQRqFLp5HMVqzpP4ft2/6C+W/1i9Qez4xm4/XluiHqBvemHHBChUupKaSJQpSYiDK3Tm9ges3iz6Vg8xbtYm9/OrKPtxjE8EfMRZ3LTHBClUqqsNBGoMvNwceeFxiM40ONzRgbfzMXjB3nk8sGRb2i8dhSfHPlBF7RTyslpIlBXrJ57IPPa/h+RXabRuVa7YvUp+WcYF/MubTc8worTWx0QoVKqNDQRKLt19W3Fpq7v82XrV6jjWvzpcXsz99Nv69Pcsn0iBzOPOiBCpdSlaCJQ5UJEGB5yPQd7zOWlxqNxE49ibX48+SdXrR/Ni7GzSM/LdECUSilbNBGocuVt8eSNpvezv9tcbgu8vlh9Ltm8dWguzdY/wJITax0QoVLqYpoIVIVo5FmXhR1eYWX4VNp4tSxWn5RzlL/vOcx+ff6BUg6niUBVqN7+HYi+ehqftnwOf0vtwnL3/OYcTbmRgaszeHuPLnetlCNpIlAVzkVceDB0EAd7zOXR0DtwwUJQ9qMIFnIMTDuQww2rMlhyNFdXNlXKATQRqErj7+rDRy0f51CP+fzcM5yuAef/90vMNIyPymLExtPcHv0WMenFF7tTSlUMuxKBiASKyM8iEmN9r11CuzgR2S4iUSISWdbtVfVS36MO7fwsfNPdkykd3AlyP1/3v5T/svDEctpsHMP/7f9U7y5SqhLYe0UwAfjVGNMC+NX6vSTXGWPCjTERV7i9qmZcRLizgRu/9fHm/sau5EoCKa7fA5BHDu8kzGPsrvnaXaRUBbM3EQwF5lg/zwFuq+TtVTXg7yZMbOvBwMa/g5xfztpigliRMIRRkZns07uLlKow9iaCEGNMIoD1vW4J7QywXEQ2icjYK9he1QAzWj/CF61forYlEIDaOQ/igherT+QzcHUGb+3JJk3vLlKq3F324fUi8gtQz0bVS2U4Ti9jzBERqQv8LCK7jTEryrA91gQyFqBx48Zl2VRVESLCX0L6c0tQT2Yc+ZnUtOuYG59HPpBr4JMDOXx/JJeXW7vi67mTa2uHOzpkpaqFyyYCY0z/kupEJElEQo0xiSISCiSXsI8j1vdkEVkIdANWAKXa3rrtdGA6QEREhP5ZWI35udbi2cYFvYT3NMrn1Z1ZbDxV0DWUlGW4P3oJJ92nclNAb2a0eoJGnnohqZQ97O0aWgyMtn4eDSy6uIGI1BIR33OfgRuB6NJur2q2Nr4ufN3Nk3939CDYQ8jjDKfdZgGw7PQqWmy4n3cOfU2uLnWt1BWzNxFMAgaISAwwwPodEakvIkutbUKAVSKyFdgALDHG/HSp7ZUqSkQYVt+VX3t7ERQwm3xJLazLMhn8X+w02m14hA0puxwYpVJV12W7hi7FGHMCuMFG+RFgkPVzLNCpLNsrZYufmzCn/d3cu+sQ29N3X1C3N3M/3beM54GQW3j3qocJcPVxUJRKVT06s1hVKZ18riIq4kM+uOppvKTWRbWGWUmLabpuNF8m/arzD5QqJU0EqspxERfGNxjKgR5zGRZU/ILydN5JRu5+g75R/8e+jMMOiFCpqkUTgaqyQtwDWdD+ZX7uOIUGbvWL1a9KiaTNhgd49cBcsvKzHRChUlWDJgJV5fWv3ZV9PWbxQqP7sOB2QV0uOfwjfhYt1j/EH6ejHBShUs5NE4GqFjxd3Hmz2QPsuPpTuvsUn2h2KPsQ1219hrt3vMmx7NMOiFAp56WJQFUrrbwbs7bLu8xp9QJ+Lv7F6r85/jNN19/H7KPLdDBZKStNBKraERHuq3cjB3rM5d66g4vVn81P5YE9k+iz5TliM444IEKlnIsmAlVtBbr5MbfN31gd/j7NPMKK1a9O3UTrjWN4O/4rnZmsajRNBKrau8a/Pbu7zeAfYQ/jivsFdTkmiwkHPqbjxsfYkhrjoAiVcixNBKpGcHNx5eUmI9nV7TObg8m7MvYyPPozUnWZa1UDaSJQNcpVXg1Y2+VdZrR8Dm+X88tQiPHibMpYBqzM4Jfk3EvsQanqRxOBqnFEhIdCB7G/+xxuCbwWgICc+3A1dTmaZXhocxbjozI5lqVXB6pm0ESgaqx67oEs7vAav3SYwsx2wwgqMnyw5Gge/Vel83VCDuvO7NRbTVW1polA1Xg3BHZlWH0PfuntzZ0Nzi/IeyYHnti5gZ5R4xmw9SUOZZb43CSlqjRNBEpZ1XYXpnTw4IsITxp5CflkctLtfQB+PbOWFhvu579Jvzk4SqXKnyYCpS7Su46F5b29CKvzFbkuSYXlWfmZvB9Tm+gzOudAVS+aCJSywcsifNxmEJ282xaW+eYNITGtJUPXZfLO3myy8nXcQFUPmgiUKkHbWmFsjnifqc2for5bE0Ly7gUgz8CHsTkMWZPBltN6daCqPk0ESl2Ci7jwRMPbiO85k597BdGt9vl/MjFphjvWZfK36IP8Pe4LcvJ1/oGqmjQRKFUKFrEQVsuF+d08+Udbd7wtBeV55PNB8ru8dnAmHTY+yta0fY4NVKkroIlAqTJwEeHexm4s6+VF7yAX0iw/kWXZBsCezH102TSOF2NnkZ2f4+BIlSo9uxKBiASKyM8iEmN9r22jTSsRiSryShGRp611E0XkcJG6QfbEo1RlaeTtwqyubrh6L7ygPJ883jo0lzYbxhKZusdB0SlVNvZeEUwAfjXGtAB+tX6/gDFmjzEm3BgTDnQF0oGi/3r+fa7eGLPUzniUqjRuLq5EX/0RQwOvK1YXmxVHt82P8dd9n5Cpz0tWTs7eRDAUmGP9PAe47TLtbwD2G2MO2nlcpZxCsHsA33d4lQVt/06AJfCCOkM+/z48n1brH2LNmWgHRajU5dmbCEKMMYkA1ve6l2k/HPjyorLHRWSbiHxmq2vpHBEZKyKRIhJ57Ngx+6JWqpwNC+7D/u6zuLvOgGJ18dmH6BX1JOP3fkh6XqYDolPq0i6bCETkFxGJtvEaWpYDiYg7cCvwTZHiaUBzIBxIBP5V0vbGmOnGmAhjTERwcHBZDq1UpQh08+Ordi/yY/s3CXKtc1Gt4aPEb2mx/kFWnN7qkPiUKsllE4Expr8xpr2N1yIgSURCAazvl1qV62ZgszGmcM6+MSbJGJNnjMkHZgDd7PtxlHK8wUE92d99FqPqFr/34UjOEfptfYbH9n5ARl6WA6JTqjh7u4YWA6Otn0cDiy7RdgQXdQudSyJWwwDtSFXVgr+rD5+3eY7lHd6hrmvIRbWGaYnf0WrDw6xP2emQ+JQqyt5EMAkYICIxwADrd0SkvogU3gEkIt7W+gUXbT9ZRLaLyDbgOuAZO+NRyqkMCIxgX/fPeCikeE/qoexD9NzyBH/bP4MsvbNIOZBUxQduREREmMjIyCvbeP58qFevfANSqhR+P7WF4TvfJjk3qVhdc8+mfNvuRcJ9rqqQY0em7mHakUXsPRNL46u60q9JP/o16UfLoJaISIUcUzkfEdlkjIkoVq6JQKnKk5qbzviYj/g8eUmxOhcsvNT4Pl5pMhI3F1cbW5dNnsnjxxNreTfhG1ac2WazTUitEPo26Uu/Jv3o26Qv7eq2w0V0wYHqShPBOZoIlBP434n1jNr1DifzThSra+vVkm/avUDbWmFXtO+zeRnMPvoT/zn8HfsyDpdp20CvQPo07lNwxRDWj04hnbC4WK4oDuV8NBGco4lAOYlTOamM3fs+3x7/uVidK25s6PIxnX2blWmf/0n4lr8fnMup3NRyidHPw4/ejXsXXjF0De2Km8WtXPatKl9JicD+60+l1BWp7ebLN+1eZMGx3jyw+11S8s8U1rnlhfN8VAjvdsyjvX/p/yIXKDEJ3BDQhTFe1xDXJpQ/D/7J6vjVnM05e8n9pWSlsDRmKUtjCu798Hbz5sHODzL15qmljkk5P+0MVMrBbg/uy77usxlUuw8ALsaXwOwn2HsWbluXyX/2ZZNTyqehjak3CD9LrcLvbuLK6JCbiOo6g186/YuRfr14sc+LLBu1jFPPn2L9Q+uZ3H8yQ1oOwd/D/7L7T89Jx7WE8QtjDMv3L+d4+vFSxaqch14RKOUEgt0D+LHD6/w3+VfWnbDwc0IgmfmQa+A/+3L4JTmPdzt40MA7i9lHf8LH4sXoegOL7cfX1ZuxoUOYdfR/PFp/KI/VH0qoR5DNY7pZ3OjWoBvdGnTjuV7PkZefx7akbaw4uII/D/7JioMrOJFRfAzj6vpX29xfQkoCN31xEwDNajfj2ibXMnPoTDvOiqosmgiUchIiwl9C+vOXEDjQJJ+/bc9i0+l8AKJT8hm05gT7Pe8GoJFHXUbW7W/z7qKXmozi9bD78bZ4lun4FhcLnUM70zm0M0/1eIp8k8+uY7v48+CfBa+4P0k6m0S3BrYXANhweEPh59hTsdT3rV/isbYe3UqTgCYEeAaUKUZVMTQRKOWEmtZy4evunsyMy2XK3myyDaTK+VnIh7KS+e74CobXvb7YtgGuPuUSg4u40K5uO9rVbcdjVz+GMYa403GEBYTZbF80EUDJVw7GGPrM6kNqdiqN/RvTMaQjHet2LHgP6UiLoBYldj+piqFnWyknZRFhbFM3rq1j4Yltx/k158IB2okHFnBP8HWVNiFMRGhau2mJ9fV86hFeL5ztSdvJM3klXjkcPHOQ1OyCAe34M/HEn4nnx70/FtZ7WDxoV7ddYYLoVK8THep2ILiWLjZZUTQRKOXkWvq68EzreP7YkU6eARfjh0/uzaRlDOaRLVm81d6DIHfHzw5+puczPNPzGdJz0ok6GkWroFY2221Lsj257ZysvCw2J25mc+LmC8rr+dSjY0hH2tRpQ+s6rWkV1IrWdVpTz6eezo62kyYCpaqAgUERHL/mO344Fscne0M5mOsOwPLkPDavSmdyew+ur+sc/5y93by5ptE1Jdbn5efRLrgdu4/vJs/klXq/R9OOcjTtKMv3L7+g3NfdtyAx1GlVmBxaBbWiZVBLPFw9rvjnqEl0QplSVUxGnuGtPdnMjc+9oHxkI1deauVOLddL/HV89CgMH17BEZZOVm4Wu47vYlvStgteSWeLr8V0Jb6/53uGti6+2F9WbhauLq41csa0TihTqprwsgh/b+vB9cEWnovO5lhWwR9z/z2Uy5oTebzb0YMuAc7/S87D1YPweuGE1wu/oDwpLYntydsvSA47ju0gO69sK7S2rtPaZvmsqFk8+b8nCQsIo1ntZoxoP4LR4aNttq0pNBEoVUVdG+zKsl4WXtyRxU9JBV0scemGu9ZnMr6ZG080d8PNper1nYf4hBDiE0L/Zv0Ly3Lycog5GUN0cjR7ju9hz4k97D6+mz0n9pCWnVZsH64urjSrbXt5jgOnDpCTX7C/mJMx9GjYw2a7tOw0Hlz8IE0DmtKsdjOa1W5G04CmNPRrWO26nDQRKFWFBboL08I9+O5ILhN3ZpOWB4QuDzcAABYZSURBVHkGpu7P4c/jBZPQmvtU/QUE3CxutA1uS9vgtheUG2M4knrkfGKwJomc/JwS10SKPR17wfdLJYyvd3xtsy6kVgiN/BvRyM/68r/wPdQ3tErdAlt1IlVK2SQi3NnAje61LTy7PYsNpwomoW09k8/gNRm81NqdUY1cq+WdNSJCA78GNPBrwPVNi8+psOVE+oWzpUtKBLGnYm2WAySdTSLpbBKRR2yPVVrEQqhvKI39GxdLFqG+oYT6hFLPp57TXFloIlCqmmjk7cKX3TyZcSCHf8XkkGMgMx9e2ZnNr8l5TG7vTl1HB+kEfhv9GylZKRw4dYDYU7F0qNvBZrtLJYLLyTN5JKQkkJCScMl2gV6BzL9jPgOaDyhWdzrzNMlnkwn1CcXXw/eKYykNTQRKVSMWEcY1c6dvHQtPb8tib1rBQPIfx/O4aXUGbzVwpfgKRTWPn4cfnep1olO9TiW2GdJyCLW9ahckjNOxxJ6K5cCpAxxNO4qhfO62PJlxEh932zPBl8Ys5S8L/gJALbda3B9+Px8M+qBcjnsxTQRKVUNt/Sws7unFlJhsPo0ruM30VA6Mi/Pizm+2MvHWdvh46D//S2kR1IIWQS2KlWfnZXMk9QiHzhziUMqh8+9FPpdlBdZQ31Cb5YmpiYWfz+acRai4rj39P0GpasrTIrzc2oPrg115dnsWiZkFf8V+uymByLiTvDe8M50a6aJvZeVucScsIKzENZcAMnIySEhJuDBRnDlEQmoCiamJJKYlknw2mXyTTz0f2/OaEtMSL/heUsIoD3YlAhG5C5gItAG6GWNsjpyIyEDgPcACfGqMmWQtDwS+AsKAOOBuY8wpe2JSSl3omiALP/Xy4pWdWSxOtN5meiKdO6at4a83tuSRvs2xVMHbTJ2Zl5tXiVcU5+Tm53I8/TierrZXifVx96GJfxMS0xLJzssm1KfiEoFdM4tFpA2QD3wC/M1WIhARC7AXGAAkABuBEcaYnSIyGThpjJkkIhOA2saY5y93XJ1ZrNSVWbjzOK8k+ZKWdX5Wcs9mQbx7TydC/b0cGJkqiTGGU5mncLe4lzieUFolzSy26wZjY8wuY8yeyzTrBuwzxsQaY7KB+cC5ed9DgTnWz3OA2+yJRyl1acMCc1n6ZB86Nz7fJbQ29gQD/7OSn6ITL7GlchQRIdAr0O4kcCmVMdOkAXCoyPcEaxlAiDEmEcD6rne3KVXBGgd58/UjPXny+qs41yN0JiOHcV9s5oUF20nPzr30DlS1c9lEICK/iEi0jVfx1ZxK2IWNsjL3R4nIWBGJFJHIY8eOlXVzpVQRbhYX/npjK+aP7Ul9//N91F9uiGfI+6uIPnzGgdGpynbZRGCM6W+MaW/jtaiUx0gAGhX53hA4Yv2cJCKhANb35EvEMd0YE2GMiQgO1gdUKFUeujUN5H9P9WVwx/MDkbHHzjLso9XMWBFLfn7VW51YlV1ldA1tBFqISFMRcQeGA4utdYuBc8v+jQZKm1yUUuXE39uND0Z0ZvKdHfF2L1i1NCfP8M+luxg9awPJKZkOjlBVNLsSgYgME5EEoCewRESWWcvri8hSAGNMLvA4sAzYBXxtjNlh3cUkYICIxFBwV9Eke+JRSl0ZEeHuiEYsebIPHRv6F5avjDnOwPdW8svO8nlGgHJO+mAapWqSUjyYJjs3n3//speP/9xP0V8P9/ZowkuD2+Dp5vzPOlC2Vcjto0qp6sfd1YXnB7Zm3kPdqed3fiD583UHueX9VexKTHFgdKoiaCJQStl0TfM6/O+pPtzULqSwLCY5jaEfruaLdQepir0JyjZNBEqpEtWu5c7Ho7ry1u0d8LJ2CWXn5vPy99E8Nm8zZzJyHByhKg+aCJRSlyQijOjWmB+e6E3reufXxf9f9FEGvbeSzfG6PFhVp4lAKVUqV9X14fvxvRjVo3Fh2eHTGdz98Vo+/nO/zjmowjQRKKVKzdPNwhu3dWDaX7rg61mweHFuvmHS/3Zz/+yNHE/LcnCE6kpoIlBKldnNHUKLLV63Yu8xbn5vJav3lf6hLMo5aCJQSl2RRoEFi9eN69e8sOxYahajZq5nyrI95OblOzA6VRaaCJRSV8zN4sKEm1szd0w36vi4A2AMfPD7PoZPX8eR0xkOjlCVhiYCpZTd+rYMZulTfeh1VVBhWeTBU9z83kqW7zjqwMhUaWgiUEqVi7q+nswd053nbmpV+OjLMxk5jP18ExMX7yArN8/BEaqSaCJQSpUbi4sw/rqr+GpsjwueczB7TRy3f7SGA8fPOjA6VRJNBEqpchcRFsjSp/pwY9vzy1PsOJLCkKkr+X7LYQdGpmzRRKCUqhAB3u58cm9XXr+1He6Wgl81Z7PzePqrKF5YsJ3MHO0qchaaCJRSFUZEGH1NGAvHX0OzOrUKy7/cEM/tH60hTruKnIImAqVUhWtX35/FT/Tmlk71C8t2JqYw5P1VLN2e6MDIFGgiUEpVEh8PV6YOD+eN29oXdhWlZeXy2LzNeleRg2kiUEpVGhFhVI8mLHjsGhoHeheWz14Tx90fr+XQyXQHRldzaSJQSlW69g38+fHJ3gxsd/6xsVsTzjB46kp+1ucjVzpNBEoph/DzdGPaqC68OqQtrtYJaCmZuTw8N5I3l+4iR9cqqjSaCJRSDiMijOndlK/H9aRBgFdh+fQVsQyfvo7EM7pWUWWwKxGIyF0iskNE8kUkooQ2jUTkdxHZZW37VJG6iSJyWESirK9B9sSjlKqaujSuzY9P9Ob61nULyzYdPMWg91byx55kB0ZWM9h7RRAN3A6suESbXOBZY0wboAcwXkTaFqn/tzEm3Ppaamc8SqkqqnYtdz69L4IJN7cuXKvoVHoOD8zeqMtaVzC7EoExZpcxZs9l2iQaYzZbP6cCu4AG9hxXKVU9ubgI4/o1Z/7YHoT4eQDnl7UeNXM9ySmZDo6weqrUMQIRCQM6A+uLFD8uIttE5DMRqX2JbceKSKSIRB47dqyCI1VKOdLVYYEsebIPfVrUKSxbF3uSQVNXsWa/PgGtvF02EYjILyISbeM1tCwHEhEf4DvgaWNMirV4GtAcCAcSgX+VtL0xZroxJsIYExEcHFyWQyulqqA6Ph7MfqAbfx3QEinoKeJ4WhajPl3PtD/2Y4xxbIDViOvlGhhj+tt7EBFxoyAJzDPGLCiy76QibWYAP9p7LKVU9WFxEZ68oQVdm9TmqflbOJ6WTb6Bt3/azZb4U0y5uxN+nm6ODrPKq/CuIRERYCawyxjz7kV1oUW+DqNg8FkppS7Q66o6LHmyDxFNzvceL9+ZxNAPVrP7aMoltlSlYe/to8NEJAHoCSwRkWXW8voicu4OoF7AvcD1Nm4TnSwi20VkG3Ad8Iw98Silqq8QP0++HNuDMb2aFpYdOH6WYR+u0Wcc2OmyXUOXYoxZCCy0UX4EGGT9vAqQEra/157jK6VqFjeLC6/e0pbOjQN4/rttpGfnkZFT8IyDLfGneGlwW9xddZ5sWekZU0pVObd0qs+i8b1oFnz+GQdz1h5k+PS1Ohv5CmgiUEpVSS1CfFk0vhc3tz+/cN3m+NMM0VtMy0wTgVKqyvL1dOOjv3ThpUFtCmcjnzibzahP1/Pxn3qLaWlpIlBKVWkiwsN9mzHvoe7U8SmYjZxvYNL/djPui02kZOY4OELnp4lAKVUt9GgWxJIne19wi+myHQW3mO45murAyJyfJgKlVLVx7hbTB3qFFZYdOH6W2z5czaIovcW0JJoIlFLVipvFhdduacfUEZ3xdrcAkJGTx1Pzo5i4eAfZubqK6cU0ESilqqVbO9Xn+4tuMZ29Jo4RM9bpKqYX0USglKq2WlpvMS36bORNB09xywer2Bx/yoGRORdNBEqpas3X+mzkF25ujfUOU5JSshj+yTrmb4h3bHBOQhOBUqraExEe6decOWO6EeBdsFppdl4+ExZs56WF22v8uIEmAqVUjdGnRTA/PN6b1vV8C8vmrY+v8eMGmgiUUjVKo0BvFjx2DUM6nl8Fv6aPG2giUErVON7urrw/orPNcYOvNta8cQNNBEqpGqnouIG/1/lxg+e/287L39escQNNBEqpGs3WuMEX6+IZOWMdyak1Y9xAE4FSqsZrHFR83CDy4ClueX8VW2rAuIEmAqWUouRxg3tqwLiBJgKllLI6N24w+4GaNW6giUAppS7St2XNGjewKxGIyF0iskNE8kUk4hLt4kRku4hEiUhkkfJAEflZRGKs77VL2odSSlWmc+MGgy8aN7j1/dVsTzjjwMjKn71XBNHA7cCKUrS9zhgTbowpmjAmAL8aY1oAv1q/K6WUU/B2d+WDEZ2ZUGTc4GhKJnd9soYfth5xbHDlyK5EYIzZZYzZY8cuhgJzrJ/nALfZE49SSpU3EWFcv+bMeqAbvp6uAGTm5PPEl1uYsmwP+flV/7nIlTVGYIDlIrJJRMYWKQ8xxiQCWN/rVlI8SilVJv1aBrNofC+a1Tn/fIMPft/HuC82cTYr14GR2e+yiUBEfhGRaBuvoWU4Ti9jTBfgZmC8iPQta6AiMlZEIkUk8tixY2XdXCml7NYs2IeF43vRt2VwYdnynUncMW0Nh06mOzAy+1w2ERhj+htj2tt4LSrtQYwxR6zvycBCoJu1KklEQgGs78mX2Md0Y0yEMSYiODi4pGZKKVWh/L3c+Gx0BA/1blpYtvtoKkM/XM362BMOjOzKVXjXkIjUEhHfc5+BGykYZAZYDIy2fh4NlDq5KKWUo7haXHh5SFsm39kRd0vBr9GTZ7P5y6fr+bIKPuzG3ttHh4lIAtATWCIiy6zl9UVkqbVZCLBKRLYCG4AlxpifrHWTgAEiEgMMsH5XSqkq4e6IRnw5tjt1fNwByM03vLBgO68tiiY3r+pMPhNjqt6Id0REhImMjLx8Q1vmz4d69S7fTqnq6OhRGD7c0VFUO4dPZ/DwnEh2JqYUlvW6KogPR3YhwNvdgZFdSEQ2XXQLP6Azi5VSym4NArz49tGeDOpw/o/M1ftOcNuHq9mXnOrAyEpHE4FSSpWDgslnXXi6f4vCsrgT6Qz7cA2/7y7xPhinoIlAKaXKiYuL8HT/lkz7Sxe83CwApGblMmbORqav2I+zdsVrIlBKqXJ2c4dQvn20Jw0CvAAwBt5cuptnv9lKZk6eg6MrThOBUkpVgHb1/Vn0eC8impxfS3PB5sOMnLGO42lZDoysOE0ESilVQer4eDDv4e7cHdGwsGxz/Glu+3A1e5OcZxBZE4FSSlUgD1cLb9/RkZcHt0GsK5gmnMrgjo/W8Ode51guRxOBUkpVMBHhoT7N+PS+CLzdiwwiz97I5+sOOjg6TQRKKVVpbmgTwrfjriHU3xOAvHzDK99H8/oPO8hz4HLWmgiUUqoSta3vx6LxvejY0L+wbNbqOB6eG0mag5az1kSglFKVrK6fJ1+N7cnN7c/PRP5tdzJ3TlvD4dMZlR6PJgKllHIAL3cLH47swqPXNi8s2300laEfrGbrodOVGosmAqWUchAXF+H5ga2ZfGdH3CwFtxQdT8vinulrWbo9sfLiqLQjKaWUsunuiEZ8/mB3ArzdgIJnIj82bzMf/r6vUpal0ESglFJOoEezIBY+1oumRZ6J/M6yPfztm21k51bssw00ESillJNoWqcWCx+7hh7NAgvLvtucwKiZ6zl1NrvCjquJQCmlnEiAtztzx3Tnrq7nl6XYcOAkwz5aTeyxtAo5piYCpZRyMu6uLky+syPPD2xdWBZ3Ip1hH61hzf7j5X48TQRKKeWERIRHr23Ox6O64OlW8Kv6TEYO983cwE/RR8v1WJoIlFLKiQ1sH8rXj/Skrq8HAEE+7nRq5H+ZrcpGE4FSSjm5jg0D+H58L7o2qc2n911NqL9Xue7frkQgIneJyA4RyReRiBLatBKRqCKvFBF52lo3UUQOF6kbZE88SilVXdUP8OLbcT3p0LB8rwYAXO3cPhq4HfikpAbGmD1AOICIWIDDwMIiTf5tjJliZxxKKVXtybkHGpQzuxKBMWYXlCm4G4D9xhjHL8CtlFIKqPwxguHAlxeVPS4i20TkMxGpbWsjABEZKyKRIhJ57JhzPNVHKaWqg8smAhH5RUSibbyGluVAIuIO3Ap8U6R4GtCcgq6jROBfJW1vjJlujIkwxkQEBweX5dBKKaUu4bJdQ8aY/uV0rJuBzcaYpCL7LvwsIjOAH8vpWEoppUqpMruGRnBRt5CIhBb5OoyCwWellFKVyN7bR4eJSALQE1giIsus5fVFZGmRdt7AAGDBRbuYLCLbRWQbcB3wjD3xKKWUKjupjLWuy5uIHAOc9c6jOkD5LwZSfjQ++2h89tH47GdPjE2MMcUGWatkInBmIhJpjLE5uc4ZaHz20fjso/HZryJi1CUmlFKqhtNEoJRSNZwmgvI33dEBXIbGZx+Nzz4an/3KPUYdI1BKqRpOrwiUUqqG00SglFI1nCaCciQicdYJclEiEukE8XwmIskiEl2kLFBEfhaRGOt7iQv9OSg+p3lGhYg0EpHfRWSX9bkbT1nLneIcXiI+pziHIuIpIhtEZKs1vtet5c5y/kqKzynOX5E4LSKyRUR+tH4v9/OnYwTlSETigAhjjFNMSBGRvkAaMNcY095aNhk4aYyZJCITgNrGmOedKL6JQJozPKPCugRKqDFms4j4ApuA24D7cYJzeIn47sYJzqEUrE9fyxiTJiJuwCrgKQqeYeIM56+k+AbiBOfvHBH5KxAB+BljhlTEv2G9IqjGjDErgJMXFQ8F5lg/z6HgF4dDlBCf0zDGJBpjNls/pwK7gAY4yTm8RHxOwRRIs351s74MznP+SorPaYhIQ2Aw8GmR4nI/f5oIypcBlovIJhEZ6+hgShBijEmEgl8kQF0Hx2NLqZ5RUZlEJAzoDKzHCc/hRfGBk5xDa7dGFJAM/GyMcarzV0J84CTnD/gP8H9AfpGycj9/mgjKVy9jTBcKltweb+36UGVT6mdUVBYR8QG+A542xqQ4Op6L2YjPac6hMSbPGBMONAS6iUh7R8ViSwnxOcX5E5EhQLIxZlNFH0sTQTkyxhyxvidT8Fzmbo6NyKYka9/yuT7mZAfHcwFjTJL1H2c+MAMHn0Nr3/F3wDxjzLnVc53mHNqKz9nOoTWm08AfFPS/O835O6dofE50/noBt1rHHucD14vIF1TA+dNEUE5EpJZ1wA4RqQXciHM+X2ExMNr6eTSwyIGxFCNO9IwK62DiTGCXMebdIlVOcQ5Lis9ZzqGIBItIgPWzF9Af2I3znD+b8TnL+TPGvGCMaWiMCaPgMb+/GWNGUQHnT+8aKici0oyCqwAoePLbf40x/3RgSIjIl8C1FCxbmwS8BnwPfA00BuKBu4wxDhmwLSG+aym4JDdAHPDIuf5QB8TXG1gJbOd8H+2LFPTDO/wcXiK+ETjBORSRjhQMZloo+KPza2PM30UkCOc4fyXF9zlOcP6KEpFrgb9Z7xoq9/OniUAppWo47RpSSqkaThOBUkrVcJoIlFKqhtNEoJRSNZwmAqWUquE0ESilVA2niUAppWq4/wdYVI2SwyNtgAAAAABJRU5ErkJggg==\n",
+      "text/plain": [
+       "
    "
+      ]
+     },
+     "metadata": {
+      "filenames": {
+       "image/png": "/Users/hjensen/Teaching/FYS-STK4150/doc/src/LectureNotes/_build/jupyter_execute/chapter10_11_688.png"
+      },
+      "needs_background": "light"
+     },
+     "output_type": "display_data"
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Time:  4.337630193999999\n"
+     ]
+    }
+   ],
+   "source": [
+    "def lstm_2layers(length_of_sequences, batch_size = None, stateful = False):\n",
+    "    \"\"\"\n",
+    "        Inputs:\n",
+    "            length_of_sequences (an int): the number of y values in \"x data\".  This is determined\n",
+    "                when the data is formatted\n",
+    "            batch_size (an int): Default value is None.  See Keras documentation of SimpleRNN.\n",
+    "            stateful (a boolean): Default value is False.  See Keras documentation of SimpleRNN.\n",
+    "        Returns:\n",
+    "            model (a Keras model): The recurrent neural network that is built and compiled by this\n",
+    "                method\n",
+    "        Builds and compiles a recurrent neural network with two LSTM hidden layers and returns the model.\n",
+    "    \"\"\"\n",
+    "    # Number of neurons on the input/output layer and the number of neurons in the hidden layer\n",
+    "    in_out_neurons = 1\n",
+    "    hidden_neurons = 250\n",
+    "    # Input Layer\n",
+    "    inp = Input(batch_shape=(batch_size, \n",
+    "                length_of_sequences, \n",
+    "                in_out_neurons)) \n",
+    "    # Hidden layers (in this case they are LSTM layers instead if SimpleRNN layers)\n",
+    "    rnn= LSTM(hidden_neurons, \n",
+    "                    return_sequences=True,\n",
+    "                    stateful = stateful,\n",
+    "                    name=\"RNN\", use_bias=True, activation='tanh')(inp)\n",
+    "    rnn1 = LSTM(hidden_neurons, \n",
+    "                    return_sequences=False,\n",
+    "                    stateful = stateful,\n",
+    "                    name=\"RNN1\", use_bias=True, activation='tanh')(rnn)\n",
+    "    # Output layer\n",
+    "    dens = Dense(in_out_neurons,name=\"dense\")(rnn1)\n",
+    "    # Define the midel\n",
+    "    model = Model(inputs=[inp],outputs=[dens])\n",
+    "    # Compile the model\n",
+    "    model.compile(loss='mean_squared_error', optimizer='adam')  \n",
+    "    # Return the model\n",
+    "    return model\n",
+    "\n",
+    "def dnn2_gru2(length_of_sequences, batch_size = None, stateful = False):\n",
+    "    \"\"\"\n",
+    "        Inputs:\n",
+    "            length_of_sequences (an int): the number of y values in \"x data\".  This is determined\n",
+    "                when the data is formatted\n",
+    "            batch_size (an int): Default value is None.  See Keras documentation of SimpleRNN.\n",
+    "            stateful (a boolean): Default value is False.  See Keras documentation of SimpleRNN.\n",
+    "        Returns:\n",
+    "            model (a Keras model): The recurrent neural network that is built and compiled by this\n",
+    "                method\n",
+    "        Builds and compiles a recurrent neural network with four hidden layers (two dense followed by\n",
+    "        two GRU layers) and returns the model.\n",
+    "    \"\"\"    \n",
+    "    # Number of neurons on the input/output layers and hidden layers\n",
+    "    in_out_neurons = 1\n",
+    "    hidden_neurons = 250\n",
+    "    # Input layer\n",
+    "    inp = Input(batch_shape=(batch_size, \n",
+    "                length_of_sequences, \n",
+    "                in_out_neurons)) \n",
+    "    # Hidden Dense (feedforward) layers\n",
+    "    dnn = Dense(hidden_neurons/2, activation='relu', name='dnn')(inp)\n",
+    "    dnn1 = Dense(hidden_neurons/2, activation='relu', name='dnn1')(dnn)\n",
+    "    # Hidden GRU layers\n",
+    "    rnn1 = GRU(hidden_neurons, \n",
+    "                    return_sequences=True,\n",
+    "                    stateful = stateful,\n",
+    "                    name=\"RNN1\", use_bias=True)(dnn1)\n",
+    "    rnn = GRU(hidden_neurons, \n",
+    "                    return_sequences=False,\n",
+    "                    stateful = stateful,\n",
+    "                    name=\"RNN\", use_bias=True)(rnn1)\n",
+    "    # Output layer\n",
+    "    dens = Dense(in_out_neurons,name=\"dense\")(rnn)\n",
+    "    # Define the model\n",
+    "    model = Model(inputs=[inp],outputs=[dens])\n",
+    "    # Compile the mdoel\n",
+    "    model.compile(loss='mean_squared_error', optimizer='adam')  \n",
+    "    # Return the model\n",
+    "    return model\n",
+    "\n",
+    "# Check to make sure the data set is complete\n",
+    "assert len(X_tot) == len(y_tot)\n",
+    "\n",
+    "# This is the number of points that will be used in as the training data\n",
+    "dim=12\n",
+    "\n",
+    "# Separate the training data from the whole data set\n",
+    "X_train = X_tot[:dim]\n",
+    "y_train = y_tot[:dim]\n",
+    "\n",
+    "\n",
+    "# Generate the training data for the RNN, using a sequence of 2\n",
+    "rnn_input, rnn_training = format_data(y_train, 2)\n",
+    "\n",
+    "\n",
+    "# Create a recurrent neural network in Keras and produce a summary of the \n",
+    "# machine learning model\n",
+    "# Change the method name to reflect which network you want to use\n",
+    "model = dnn2_gru2(length_of_sequences = 2)\n",
+    "model.summary()\n",
+    "\n",
+    "# Start the timer.  Want to time training+testing\n",
+    "start = timer()\n",
+    "# Fit the model using the training data genenerated above using 150 training iterations and a 5%\n",
+    "# validation split.  Setting verbose to True prints information about each training iteration.\n",
+    "hist = model.fit(rnn_input, rnn_training, batch_size=None, epochs=150, \n",
+    "                 verbose=True,validation_split=0.05)\n",
+    "\n",
+    "\n",
+    "# This section plots the training loss and the validation loss as a function of training iteration.\n",
+    "# This is not required for analyzing the couple cluster data but can help determine if the network is\n",
+    "# being overtrained.\n",
+    "for label in [\"loss\",\"val_loss\"]:\n",
+    "    plt.plot(hist.history[label],label=label)\n",
+    "\n",
+    "plt.ylabel(\"loss\")\n",
+    "plt.xlabel(\"epoch\")\n",
+    "plt.title(\"The final validation loss: {}\".format(hist.history[\"val_loss\"][-1]))\n",
+    "plt.legend()\n",
+    "plt.show()\n",
+    "\n",
+    "# Use the trained neural network to predict more points of the data set\n",
+    "test_rnn(X_tot, y_tot, X_tot[0], X_tot[dim-1])\n",
+    "# Stop the timer and calculate the total time needed.\n",
+    "end = timer()\n",
+    "print('Time: ', end-start)\n",
+    "\n",
+    "\n",
+    "# ### Training Recurrent Neural Networks in the Standard Way (i.e. learning the relationship between the X and Y data)\n",
+    "# \n",
+    "# Finally, comparing the performace of a recurrent neural network using the standard data formatting to the performance of the network with time sequence data formatting shows the benefit of this type of data formatting with extrapolation.\n",
+    "\n",
+    "# Check to make sure the data set is complete\n",
+    "assert len(X_tot) == len(y_tot)\n",
+    "\n",
+    "# This is the number of points that will be used in as the training data\n",
+    "dim=12\n",
+    "\n",
+    "# Separate the training data from the whole data set\n",
+    "X_train = X_tot[:dim]\n",
+    "y_train = y_tot[:dim]\n",
+    "\n",
+    "# Reshape the data for Keras specifications\n",
+    "X_train = X_train.reshape((dim, 1))\n",
+    "y_train = y_train.reshape((dim, 1))\n",
+    "\n",
+    "\n",
+    "# Create a recurrent neural network in Keras and produce a summary of the \n",
+    "# machine learning model\n",
+    "# Set the sequence length to 1 for regular data formatting \n",
+    "model = rnn(length_of_sequences = 1)\n",
+    "model.summary()\n",
+    "\n",
+    "# Start the timer.  Want to time training+testing\n",
+    "start = timer()\n",
+    "# Fit the model using the training data genenerated above using 150 training iterations and a 5%\n",
+    "# validation split.  Setting verbose to True prints information about each training iteration.\n",
+    "hist = model.fit(X_train, y_train, batch_size=None, epochs=150, \n",
+    "                 verbose=True,validation_split=0.05)\n",
+    "\n",
+    "\n",
+    "# This section plots the training loss and the validation loss as a function of training iteration.\n",
+    "# This is not required for analyzing the couple cluster data but can help determine if the network is\n",
+    "# being overtrained.\n",
+    "for label in [\"loss\",\"val_loss\"]:\n",
+    "    plt.plot(hist.history[label],label=label)\n",
+    "\n",
+    "plt.ylabel(\"loss\")\n",
+    "plt.xlabel(\"epoch\")\n",
+    "plt.title(\"The final validation loss: {}\".format(hist.history[\"val_loss\"][-1]))\n",
+    "plt.legend()\n",
+    "plt.show()\n",
+    "\n",
+    "# Use the trained neural network to predict the remaining data points\n",
+    "X_pred = X_tot[dim:]\n",
+    "X_pred = X_pred.reshape((len(X_pred), 1))\n",
+    "y_model = model.predict(X_pred)\n",
+    "y_pred = np.concatenate((y_tot[:dim], y_model.flatten()))\n",
+    "\n",
+    "# Plot the known data set and the predicted data set.  The red box represents the region that was used\n",
+    "# for the training data.\n",
+    "fig, ax = plt.subplots()\n",
+    "ax.plot(X_tot, y_tot, label=\"true\", linewidth=3)\n",
+    "ax.plot(X_tot, y_pred, 'g-.',label=\"predicted\", linewidth=4)\n",
+    "ax.legend()\n",
+    "# Created a red region to represent the points used in the training data.\n",
+    "ax.axvspan(X_tot[0], X_tot[dim], alpha=0.25, color='red')\n",
+    "plt.show()\n",
+    "\n",
+    "# Stop the timer and calculate the total time needed.\n",
+    "end = timer()\n",
+    "print('Time: ', end-start)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Solving ODEs with Deep Learning\n",
+    "\n",
+    "The Universal Approximation Theorem states that a neural network can\n",
+    "approximate any function at a single hidden layer along with one input\n",
+    "and output layer to any given precision.  \n",
+    "\n",
+    "\n",
+    "\n",
+    "## Ordinary Differential Equations\n",
+    "\n",
+    "An ordinary differential equation (ODE) is an equation involving functions having one variable.\n",
+    "\n",
+    "In general, an ordinary differential equation looks like"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation} \\label{ode} \\tag{1}\n",
+    "f\\left(x, \\, g(x), \\, g'(x), \\, g''(x), \\, \\dots \\, , \\, g^{(n)}(x)\\right) = 0\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "where $g(x)$ is the function to find, and $g^{(n)}(x)$ is the $n$-th derivative of $g(x)$.\n",
+    "\n",
+    "The $f\\left(x, g(x), g'(x), g''(x), \\, \\dots \\, , g^{(n)}(x)\\right)$ is just a way to write that there is an expression involving $x$ and $g(x), \\ g'(x), \\ g''(x), \\, \\dots \\, , \\text{ and } g^{(n)}(x)$ on the left side of the equality sign in ([1](#ode)).\n",
+    "The highest order of derivative, that is the value of $n$, determines to the order of the equation.\n",
+    "The equation is referred to as a $n$-th order ODE.\n",
+    "Along with ([1](#ode)), some additional conditions of the function $g(x)$ are typically given\n",
+    "for the solution to be unique.\n",
+    "\n",
+    "\n",
+    "## The trial solution\n",
+    "\n",
+    "Let the trial solution $g_t(x)$ be"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation}\n",
+    "\tg_t(x) = h_1(x) + h_2(x,N(x,P))\n",
+    "\\label{_auto1} \\tag{2}\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "where $h_1(x)$ is a function that makes $g_t(x)$ satisfy a given set\n",
+    "of conditions, $N(x,P)$ a neural network with weights and biases\n",
+    "described by $P$ and $h_2(x, N(x,P))$ some expression involving the\n",
+    "neural network.  The role of the function $h_2(x, N(x,P))$, is to\n",
+    "ensure that the output from $N(x,P)$ is zero when $g_t(x)$ is\n",
+    "evaluated at the values of $x$ where the given conditions must be\n",
+    "satisfied.  The function $h_1(x)$ should alone make $g_t(x)$ satisfy\n",
+    "the conditions.\n",
+    "\n",
+    "But what about the network $N(x,P)$?\n",
+    "\n",
+    "\n",
+    "As described previously, an optimization method could be used to minimize the parameters of a neural network, that being its weights and biases, through backward propagation.\n",
+    "\n",
+    "\n",
+    "\n",
+    "## Minimization process\n",
+    "\n",
+    "For the minimization to be defined, we need to have a cost function at hand to minimize.\n",
+    "\n",
+    "It is given that $f\\left(x, \\, g(x), \\, g'(x), \\, g''(x), \\, \\dots \\, , \\, g^{(n)}(x)\\right)$ should be equal to zero in ([1](#ode)).\n",
+    "We can choose to consider the mean squared error as the cost function for an input $x$.\n",
+    "Since we are looking at one input, the cost function is just $f$ squared.\n",
+    "The cost function $c\\left(x, P \\right)$ can therefore be expressed as"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "C\\left(x, P\\right) = \\big(f\\left(x, \\, g(x), \\, g'(x), \\, g''(x), \\, \\dots \\, , \\, g^{(n)}(x)\\right)\\big)^2\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "If $N$ inputs are given as a vector $\\boldsymbol{x}$ with elements $x_i$ for $i = 1,\\dots,N$,\n",
+    "the cost function becomes"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation} \\label{cost} \\tag{3}\n",
+    "\tC\\left(\\boldsymbol{x}, P\\right) = \\frac{1}{N} \\sum_{i=1}^N \\big(f\\left(x_i, \\, g(x_i), \\, g'(x_i), \\, g''(x_i), \\, \\dots \\, , \\, g^{(n)}(x_i)\\right)\\big)^2\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The neural net should then find the parameters $P$ that minimizes the cost function in\n",
+    "([3](#cost)) for a set of $N$ training samples $x_i$.\n",
+    "\n",
+    "\n",
+    "## Minimizing the cost function using gradient descent and automatic differentiation\n",
+    "\n",
+    "To perform the minimization using gradient descent, the gradient of $C\\left(\\boldsymbol{x}, P\\right)$ is needed.\n",
+    "It might happen so that finding an analytical expression of the gradient of $C(\\boldsymbol{x}, P)$ from ([3](#cost)) gets too messy, depending on which cost function one desires to use.\n",
+    "\n",
+    "Luckily, there exists libraries that makes the job for us through automatic differentiation.\n",
+    "Automatic differentiation is a method of finding the derivatives numerically with very high precision.\n",
+    "\n",
+    "\n",
+    "\n",
+    "## Example: Exponential decay\n",
+    "\n",
+    "An exponential decay of a quantity $g(x)$ is described by the equation"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation} \\label{solve_expdec} \\tag{4}\n",
+    "  g'(x) = -\\gamma g(x)\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "with $g(0) = g_0$ for some chosen initial value $g_0$.\n",
+    "\n",
+    "The analytical solution of ([4](#solve_expdec)) is"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation}\n",
+    "  g(x) = g_0 \\exp\\left(-\\gamma x\\right)\n",
+    "\\label{_auto2} \\tag{5}\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Having an analytical solution at hand, it is possible to use it to compare how well a neural network finds a solution of ([4](#solve_expdec)).\n",
+    "\n",
+    "\n",
+    "\n",
+    "## The function to solve for\n",
+    "\n",
+    "The program will use a neural network to solve"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation} \\label{solveode} \\tag{6}\n",
+    "g'(x) = -\\gamma g(x)\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "where $g(0) = g_0$ with $\\gamma$ and $g_0$ being some chosen values.\n",
+    "\n",
+    "In this example, $\\gamma = 2$ and $g_0 = 10$.\n",
+    "\n",
+    "\n",
+    "## The trial solution\n",
+    "To begin with, a trial solution $g_t(t)$ must be chosen. A general trial solution for ordinary differential equations could be"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "g_t(x, P) = h_1(x) + h_2(x, N(x, P))\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "with $h_1(x)$ ensuring that $g_t(x)$ satisfies some conditions and $h_2(x,N(x, P))$ an expression involving $x$ and the output from the neural network $N(x,P)$ with $P $ being the collection of the weights and biases for each layer. For now, it is assumed that the network consists of one input layer, one hidden layer, and one output layer.\n",
+    "\n",
+    "\n",
+    "## Setup of Network\n",
+    "\n",
+    "In this network, there are no weights and bias at the input layer, so $P = \\{ P_{\\text{hidden}},  P_{\\text{output}} \\}$.\n",
+    "If there are $N_{\\text{hidden} }$ neurons in the hidden layer, then $P_{\\text{hidden}}$ is a $N_{\\text{hidden} } \\times (1 + N_{\\text{input}})$ matrix, given that there are $N_{\\text{input}}$ neurons in the input layer.\n",
+    "\n",
+    "The first column in $P_{\\text{hidden} }$ represents the bias for each neuron in the hidden layer and the second column represents the weights for each neuron in the hidden layer from the input layer.\n",
+    "If there are $N_{\\text{output} }$ neurons in the output layer, then $P_{\\text{output}} $ is a $N_{\\text{output} } \\times (1 + N_{\\text{hidden} })$ matrix.\n",
+    "\n",
+    "Its first column represents the bias of each neuron and the remaining columns represents the weights to each neuron.\n",
+    "\n",
+    "It is given that $g(0) = g_0$. The trial solution must fulfill this condition to be a proper solution of ([6](#solveode)). A possible way to ensure that $g_t(0, P) = g_0$, is to let $F(N(x,P)) = x \\cdot N(x,P)$ and $A(x) = g_0$. This gives the following trial solution:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation} \\label{trial} \\tag{7}\n",
+    "g_t(x, P) = g_0 + x \\cdot N(x, P)\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Reformulating the problem\n",
+    "\n",
+    "We wish that our neural network manages to minimize a given cost function.\n",
+    "\n",
+    "A reformulation of out equation, ([6](#solveode)), must therefore be done,\n",
+    "such that it describes the problem a neural network can solve for.\n",
+    "\n",
+    "The neural network must find the set of weights and biases $P$ such that the trial solution in ([7](#trial)) satisfies ([6](#solveode)).\n",
+    "\n",
+    "The trial solution"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "g_t(x, P) = g_0 + x \\cdot N(x, P)\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "has been chosen such that it already solves the condition $g(0) = g_0$. What remains, is to find $P$ such that"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation} \\label{nnmin} \\tag{8}\n",
+    "g_t'(x, P) = - \\gamma g_t(x, P)\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "is fulfilled as *best as possible*.\n",
+    "\n",
+    "\n",
+    "## More technicalities\n",
+    "\n",
+    "The left hand side and right hand side of ([8](#nnmin)) must be computed separately, and then the neural network must choose weights and biases, contained in $P$, such that the sides are equal as best as possible.\n",
+    "This means that the absolute or squared difference between the sides must be as close to zero, ideally equal to zero.\n",
+    "In this case, the difference squared shows to be an appropriate measurement of how erroneous the trial solution is with respect to $P$ of the neural network.\n",
+    "\n",
+    "This gives the following cost function our neural network must solve for:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\min_{P}\\Big\\{ \\big(g_t'(x, P) - ( -\\gamma g_t(x, P) \\big)^2 \\Big\\}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "(the notation $\\min_{P}\\{ f(x, P) \\}$ means that we desire to find $P$ that yields the minimum of $f(x, P)$)\n",
+    "\n",
+    "or, in terms of weights and biases for the hidden and output layer in our network:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\min_{P_{\\text{hidden} }, \\ P_{\\text{output} }}\\Big\\{ \\big(g_t'(x, \\{ P_{\\text{hidden} }, P_{\\text{output} }\\}) - ( -\\gamma g_t(x, \\{ P_{\\text{hidden} }, P_{\\text{output} }\\}) \\big)^2 \\Big\\}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "for an input value $x$.\n",
+    "\n",
+    "\n",
+    "## More details\n",
+    "\n",
+    "If the neural network evaluates $g_t(x, P)$ at more values for $x$, say $N$ values $x_i$ for $i = 1, \\dots, N$, then the *total* error to minimize becomes"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation} \\label{min} \\tag{9}\n",
+    "\\min_{P}\\Big\\{\\frac{1}{N} \\sum_{i=1}^N  \\big(g_t'(x_i, P) - ( -\\gamma g_t(x_i, P) \\big)^2 \\Big\\}\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Letting $\\boldsymbol{x}$ be a vector with elements $x_i$ and $C(\\boldsymbol{x}, P) = \\frac{1}{N} \\sum_i  \\big(g_t'(x_i, P) - ( -\\gamma g_t(x_i, P) \\big)^2$ denote the cost function, the minimization problem that our network must solve, becomes"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\min_{P} C(\\boldsymbol{x}, P)\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "In terms of $P_{\\text{hidden} }$ and $P_{\\text{output} }$, this could also be expressed as\n",
+    "\n",
+    "$$\n",
+    "\\min_{P_{\\text{hidden} }, \\ P_{\\text{output} }} C(\\boldsymbol{x}, \\{P_{\\text{hidden} }, P_{\\text{output} }\\})\n",
+    "$$\n",
+    "\n",
+    "\n",
+    "## A possible implementation of a neural network\n",
+    "\n",
+    "For simplicity, it is assumed that the input is an array $\\boldsymbol{x} = (x_1, \\dots, x_N)$ with $N$ elements. It is at these points the neural network should find $P$ such that it fulfills ([9](#min)).\n",
+    "\n",
+    "First, the neural network must feed forward the inputs.\n",
+    "This means that $\\boldsymbol{x}s$ must be passed through an input layer, a hidden layer and a output layer. The input layer in this case, does not need to process the data any further.\n",
+    "The input layer will consist of $N_{\\text{input} }$ neurons, passing its element to each neuron in the hidden layer.  The number of neurons in the hidden layer will be $N_{\\text{hidden} }$.\n",
+    "\n",
+    "\n",
+    "## Technicalities\n",
+    "\n",
+    "For the $i$-th in the hidden layer with weight $w_i^{\\text{hidden} }$ and bias $b_i^{\\text{hidden} }$, the weighting from the $j$-th neuron at the input layer is:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\begin{aligned}\n",
+    "z_{i,j}^{\\text{hidden}} &= b_i^{\\text{hidden}} + w_i^{\\text{hidden}}x_j \\\\\n",
+    "&=\n",
+    "\\begin{pmatrix}\n",
+    "b_i^{\\text{hidden}} & w_i^{\\text{hidden}}\n",
+    "\\end{pmatrix}\n",
+    "\\begin{pmatrix}\n",
+    "1 \\\\\n",
+    "x_j\n",
+    "\\end{pmatrix}\n",
+    "\\end{aligned}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Final technicalities I\n",
+    "\n",
+    "The result after weighting the inputs at the $i$-th hidden neuron can be written as a vector:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\begin{aligned}\n",
+    "\\boldsymbol{z}_{i}^{\\text{hidden}} &= \\Big( b_i^{\\text{hidden}} + w_i^{\\text{hidden}}x_1 , \\ b_i^{\\text{hidden}} + w_i^{\\text{hidden}} x_2, \\ \\dots \\, , \\ b_i^{\\text{hidden}} + w_i^{\\text{hidden}} x_N\\Big)  \\\\\n",
+    "&=\n",
+    "\\begin{pmatrix}\n",
+    " b_i^{\\text{hidden}}  & w_i^{\\text{hidden}}\n",
+    "\\end{pmatrix}\n",
+    "\\begin{pmatrix}\n",
+    "1  & 1 & \\dots & 1 \\\\\n",
+    "x_1 & x_2 & \\dots & x_N\n",
+    "\\end{pmatrix} \\\\\n",
+    "&= \\boldsymbol{p}_{i, \\text{hidden}}^T X\n",
+    "\\end{aligned}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Final technicalities II\n",
+    "\n",
+    "The vector $\\boldsymbol{p}_{i, \\text{hidden}}^T$ constitutes each row in $P_{\\text{hidden} }$, which contains the weights for the neural network to minimize according to ([9](#min)).\n",
+    "\n",
+    "After having found $\\boldsymbol{z}_{i}^{\\text{hidden}} $ for every $i$-th neuron within the hidden layer, the vector will be sent to an activation function $a_i(\\boldsymbol{z})$.\n",
+    "\n",
+    "In this example, the sigmoid function has been chosen to be the activation function for each hidden neuron:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "f(z) = \\frac{1}{1 + \\exp{(-z)}}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "It is possible to use other activations functions for the hidden layer also.\n",
+    "\n",
+    "The output $\\boldsymbol{x}_i^{\\text{hidden}}$ from each $i$-th hidden neuron is:\n",
+    "\n",
+    "$$\n",
+    "\\boldsymbol{x}_i^{\\text{hidden} } = f\\big(  \\boldsymbol{z}_{i}^{\\text{hidden}} \\big)\n",
+    "$$\n",
+    "\n",
+    "The outputs $\\boldsymbol{x}_i^{\\text{hidden} } $ are then sent to the output layer.\n",
+    "\n",
+    "The output layer consists of one neuron in this case, and combines the\n",
+    "output from each of the neurons in the hidden layers. The output layer\n",
+    "combines the results from the hidden layer using some weights $w_i^{\\text{output}}$\n",
+    "and biases $b_i^{\\text{output}}$. In this case,\n",
+    "it is assumes that the number of neurons in the output layer is one.\n",
+    "\n",
+    "\n",
+    "## Final technicalities III\n",
+    "\n",
+    "\n",
+    "The procedure of weighting the output neuron $j$ in the hidden layer to the $i$-th neuron in the output layer is similar as for the hidden layer described previously."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\begin{aligned}\n",
+    "z_{1,j}^{\\text{output}} & =\n",
+    "\\begin{pmatrix}\n",
+    "b_1^{\\text{output}} & \\boldsymbol{w}_1^{\\text{output}}\n",
+    "\\end{pmatrix}\n",
+    "\\begin{pmatrix}\n",
+    "1 \\\\\n",
+    "\\boldsymbol{x}_j^{\\text{hidden}}\n",
+    "\\end{pmatrix}\n",
+    "\\end{aligned}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Final technicalities IV\n",
+    "\n",
+    "Expressing $z_{1,j}^{\\text{output}}$ as a vector gives the following way of weighting the inputs from the hidden layer:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\boldsymbol{z}_{1}^{\\text{output}} =\n",
+    "\\begin{pmatrix}\n",
+    "b_1^{\\text{output}} & \\boldsymbol{w}_1^{\\text{output}}\n",
+    "\\end{pmatrix}\n",
+    "\\begin{pmatrix}\n",
+    "1  & 1 & \\dots & 1 \\\\\n",
+    "\\boldsymbol{x}_1^{\\text{hidden}} & \\boldsymbol{x}_2^{\\text{hidden}} & \\dots & \\boldsymbol{x}_N^{\\text{hidden}}\n",
+    "\\end{pmatrix}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "In this case we seek a continuous range of values since we are approximating a function. This means that after computing $\\boldsymbol{z}_{1}^{\\text{output}}$ the neural network has finished its feed forward step, and $\\boldsymbol{z}_{1}^{\\text{output}}$ is the final output of the network.\n",
+    "\n",
+    "\n",
+    "## Back propagation\n",
+    "\n",
+    "The next step is to decide how the parameters should be changed such that they minimize the cost function.\n",
+    "\n",
+    "The chosen cost function for this problem is"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "C(\\boldsymbol{x}, P) = \\frac{1}{N} \\sum_i  \\big(g_t'(x_i, P) - ( -\\gamma g_t(x_i, P) \\big)^2\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "In order to minimize the cost function, an optimization method must be chosen.\n",
+    "\n",
+    "Here, gradient descent with a constant step size has been chosen.\n",
+    "\n",
+    "\n",
+    "## Gradient descent\n",
+    "\n",
+    "The idea of the gradient descent algorithm is to update parameters in\n",
+    "a direction where the cost function decreases goes to a minimum.\n",
+    "\n",
+    "In general, the update of some parameters $\\boldsymbol{\\omega}$ given a cost\n",
+    "function defined by some weights $\\boldsymbol{\\omega}$, $C(\\boldsymbol{x},\n",
+    "\\boldsymbol{\\omega})$, goes as follows:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\boldsymbol{\\omega}_{\\text{new} } = \\boldsymbol{\\omega} - \\lambda \\nabla_{\\boldsymbol{\\omega}} C(\\boldsymbol{x}, \\boldsymbol{\\omega})\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "for a number of iterations or until $ \\big|\\big| \\boldsymbol{\\omega}_{\\text{new} } - \\boldsymbol{\\omega} \\big|\\big|$ becomes smaller than some given tolerance.\n",
+    "\n",
+    "The value of $\\lambda$ decides how large steps the algorithm must take\n",
+    "in the direction of $ \\nabla_{\\boldsymbol{\\omega}} C(\\boldsymbol{x}, \\boldsymbol{\\omega})$.\n",
+    "The notation $\\nabla_{\\boldsymbol{\\omega}}$ express the gradient with respect\n",
+    "to the elements in $\\boldsymbol{\\omega}$.\n",
+    "\n",
+    "In our case, we have to minimize the cost function $C(\\boldsymbol{x}, P)$ with\n",
+    "respect to the two sets of weights and biases, that is for the hidden\n",
+    "layer $P_{\\text{hidden} }$ and for the output layer $P_{\\text{output}\n",
+    "}$ .\n",
+    "\n",
+    "This means that $P_{\\text{hidden} }$ and $P_{\\text{output} }$ is updated by"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\begin{aligned}\n",
+    "P_{\\text{hidden},\\text{new}} &= P_{\\text{hidden}} - \\lambda \\nabla_{P_{\\text{hidden}}} C(\\boldsymbol{x}, P)  \\\\\n",
+    "P_{\\text{output},\\text{new}} &= P_{\\text{output}} - \\lambda \\nabla_{P_{\\text{output}}} C(\\boldsymbol{x}, P)\n",
+    "\\end{aligned}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## The code for solving the ODE"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Initial cost: 367.01\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Final cost: 0.0666807\n",
+      "Max absolute difference: 0.0437499\n"
+     ]
+    },
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAl4AAAJcCAYAAAAo6aqNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3gU5d7G8e9vk5BQQg89NAsgCAiICBZsx4aCIiqCqEfsvRzsx97LsRdAREURRFFRLIiCohTpvShFOqGEGiDlef+YwTfGNCDJ7G7uz3VxkexsuWf22dl7n53dmHMOERERESl+oaADiIiIiJQWKl4iIiIiJUTFS0RERKSEqHiJiIiIlBAVLxEREZESouIlIiIiUkJUvEqImdU0s5/MbLuZPR90nqCZWVkzG2VmW83s46Dz5MfMLjezCUHnOBhm1tnMVhXzbYwzs76FON/xZraoOLNIeCvsWCnC21tuZqeW1O0diIPZz5hZfTPbYWYxB5mhyO+Xg9n2RbVe4UbFKx/+gEnz7/j1ZvaOmVU4wKu7GtgIVHTO3VGEMSPVBUBNoJpzrkfQYYJgZg3NzJlZbNBZSpJz7mfnXJOgcxSGmXUxsylmttPMNpnZB2ZWL9vyy80s099H7DCzZf5+4vBs59l3P+/I8e+iYNYqvJnZQ2Y25CAuP9jMHivKTOEmZ5lxzv3pnKvgnMsMMtfBitb1yknFq2DnOOcqAG2Ao4H79+fC5gkBDYD57gC+sTZKn5gbAIudcxlBhojSbfs3pWEdi4OZXQB8CLwEVAeaA3uACWZWJdtZJ/r7iErAqUAaMM3MWuS4ysr+k8i+f8OKfy2Kn8aXyH5yzulfHv+A5cCp2X5/FvjS/7kD8CuQCswCOmc73zjgceAXvJ3wECAd2AvswNs5xwMvAmv8fy8C8f7lOwOrgLuAdcD7wEPAx/51bQfmAIcD9wAbgJXAv7JluAJY4J93KXBNtmX7rv8O/7JrgSuyLS8LPA+sALYCE4CyBa13Ltuvmb8tUoF5wLn+6Q/72yLd3x5X5nLZh4DhwHv+OswD2mVbXgf4BEgBlgE3Z1s2GHgs5/rmuF/vAmbjPZHGAncDf/i3NR84L9v5Lwcm5LGODQEHXAb8iTereV+25aFs173JX6eq/rI//cvu8P8d62/ztv7y3v7yI/zf+wKf+T/v7/jJuQ1u9tezXi7rdCgw3r/vNwLDsi3rCPzmL/sN6Jhj3Pf1s6UCLbItS8J7LNTI4/64078/tgLDgIRsy/vhjdE1/vU74NA87o8DHvc5rsf8+6JfjtNDwFzgkfzGBvAlMCLHGIkt5H6nKvCOv75b9t3n/rKrgN+BzcAXQJ1syxxwPbDEX/9HgUOAicA2vLFXJse2uNe/j5cDvbJd19nADP9yK4GHchnzV+KN4Z/80//tb/stwLdAg2yXOQ1Y6N+/r+KNr765rPsZ/H3fMCvb4/0Lf71/B67KY9tdzd/3taMKOca6ADPxxu2vQMt87p+X/G2yDZgGHL8f+61C7WeA14Dnc9zuKOBWvMdzFt7jaQfe42PffRKb3xgCquCNzRT/9C/Jtg/Afwznsd7tgan+eq8HXsi27Fx/XVP962iW4/F9akH75kKuV57joKBtH07/Ag8Qzv9yDJhk/458FKiL9yR6Ft6O+DT/96Rsg/dPvFfIsUBcLgPuEWAS3hNREt6D/dFsgzEDeBrvSaysP6h2A6f71/keXuG4z7/+q4Bl2a7/bLydrgEnAruANjmu/xH/smf5y6v4y1/z16EuEIP3ZBtf0Hrn2HZx/gPjXqAMcLL/YGiS7UEyJJ9tv299z/IzPAlM8peF8HZ4//WvuzHek+zp/vKc27oz/3yin+nfp/sKZQ+8B3UIuAjYCdT2l11OwcVrgH8/tcIrc8385bf693M9fxu+BQzNcdnYbNf3HnCH/3N/vJ30ddmW3XaA4+evbQA8AEzP7X7zlw/FG1chIAE4zj+9Kt7O+lK8MdjT/71atnHf1/95EPB4tuu8Afgmn/tjir/9q+I9eV/rLzsDrzw2B8rh7ZzzK14HPO5zXE9T/3Ya5bLsYbxZrjzHBl4JWZ/X/VzAfucrvGJQxc95on/6yXglqY1/v76CX3r85Q7vSaki/z87Nxbv8VEJ74n+shzb4gX/uk7EG/NNsi0/0h8DLfGeaLvlWJ/3gPJ446sb3uO9mT827gd+9c9fHe/J+gJ/fW7zbzuvJ/iHyLFvwCtqr+ONx9Z4xeGUPC4/mGyP/0KMsTZ4RfwYvH3NZf754/O4/t5ANX8978AbnwkF7bf2Zz+DV3LWAKFs23AXUDPnc1NuY4y8x1A1oDveYykR78V89mI/Lp/7ZSJwqf9zBaCD//Ph/nqc5t9WP38slMmZNed9Q+77gvzWK89xUNC2D6d/gQcI53/+INiB1+JX+Hd4WbyZhPdznPdb/n+nNg7/FXG25TkH3B/AWdl+Px1Ynm0w7uXvr8geAsZk+/0cP1uM/3uiP0Ar57EunwG3ZLv+NP7+hL8BbzYr5C9rlct15LveOU4/Hm+HFMp22lD8V84Urnh9n+33I4A0/+djgD9znP8e4J08tnVuD+5/F3DfzwS6+j9fTsHFK/urxinAxf7PC8j2BAHUxntFHkvuxetK4Itsl+0LfOT/voL/LxH7O346A6vxnmgnAJXyWff38EpfvRynXwpMyXHaRODybON+X/E6FVia7Xy/AH3yuT96Z/v9GeBN/+dBwJPZlh1KPsXrQMd9Lpc7zr+dhFyWXQssyW9s4BXG9BxjJDXHv2a5XK423qv+3Mrg28Az2X6v4I+lhv7vDuiUbfk04K5svz8PvJhtW2QA5bMtHw48kMd2fBH4X471aZxt+ddkm7nG24/swjukoA9/Lx+GN9tWqOKF9wIpE0jMdtqTwOA8Lj+Y3ItXXmPsDfwXLdmWL8IvK4UYY1vw95fks9/K47J57mfwHv+n+T/fCIzOsT65FpT8xlAut98a2JLt93H53C8/4b3oqJ7j9AeA4Tnu+9X474ZQRMWroHGwv9s+yH86xqtg3ZxzlZ1zDZxz1zvn0vB2Jj3MLHXfP7wdde1sl1tZwPXWwXsi3WeFf9o+Kc653Tkusz7bz2nARvf/Bx2m+f9XADCzM81skplt9vOdhfeqaZ9N7u/HV+3yL1sd79XEH7lkLsx6Z1+/lc65rBzrWDeX8+ZlXY58Cf7xJA2AOjly3It3sH5h/e3+MbM+ZjYz2/W14O/ba3+z7vsQRgNgZLbrXYC388gr63jgeDOrhfeqbRjQycwa4s1azPTPdyDjpzLeWzFPOue25rMu/fCeHKeY2Twz+3cet7nvdnO7T38AyprZMWbWAG8HPzKf28xr+9Xh7/dVvo+rgxj3OW30/89tbNfOtjwvdfHeDsmuur8v2fdvQS6XSwY2O+e25LLsb9vfObcDb8Y5+/bPuY/I+Xv2dd3inNuZ7fe/xpB/v/1oZilmthWvbOZ8PGS/LxoAL2Ub55vxxlBdctyHzntWLGj/mF0dvG2yPUfW/dmXQP6P0Tty7E+S+fvj6S9mdoeZLfA/kZ2K97jMvm3y2m/t737mXbzZNfz/3y/keuY5hsysnJm9ZWYrzGwbXpmqXMhPDV6JN7u10Mx+M7Mu/uk5x2UW3v27v/dPQQozDvLc9uFExevArMSb+cm+Ey3vnHsq23lcAdexBu8Bv099/7TCXj5PZhaPd/zTc3hT05WB0Xg7woJsxJuuPSSXZYVZ733WAMn+Bwv2qY/3SuhgrcR7WzV7jkTn3Fn+8p14U+n71MrlOv7avn4xGID3qrKav73mUrjtVZisZ+bImuCcW00u97Fz7ne8HcbNeG8jbcfbmVyN92p4X5E9kPGzBe9YlnfMrFNegZ1z65xzVznn6gDXAK+b2aG53Oa+2/3HfernHI73duQleMdGbs95vkJYi/c27T7JeZ3xIMd9TovwZmX+9olbfzx3x3sLLz/nAT8fwO2uBKqaWeVclv1t+5tZeby3jg70MVXFv459so+hD/Hetkx2zlUC3uSf2zH7GFuJdzxd9nFe1jn3K959+Nf9ZmZGPvcj/xy7a/C2SWKOrHmt9/7uO1fivS2ePXs559zQnGc0s+PxZv4vxJtRqox3zFiBY+wA9jNDgK5m1grvLdzPsi3Lbx3zG0N3AE2AY5xzFYET9sUrKL9zbolzrife4Q1PAyP88ZNzXO67f3O7fwraN+e3Xvs7DsKWiteBGQKcY2anm1mMmSWY9z1J9Qq85P8bCtxvZklmVh3veKUD/gh1DmXwjttIATLM7EzgX4W5oP+EOQh4wczq+Ot3rP+ktj/rPRnvQdbPzOLMrDPe26MfHfzqMQXYZmZ3mfd9YDFm1sLMjvaXzwTOMrOq/szRrQVcX3m8B3wKgJldgfdKtCi8CTzu73Tx7++u/rIUvLcEGue4zHi8nfN4//dxOX6HAxw/zrlxQC+8WbhjcjuPmfXIdp9uwds2mXgl5nAzu8TMYs37OoQj8A7Qzc2HeMex9PJ/PhDDgSvMrJmZlcNbz7wc8LjPyZ+VuRNvG1/ij7NawEC8Y6j+l/My/jhsZGav4L2F8vAB3O5avLftXjezKv5jZ9+T44d426K1/3h8ApjsnFt+AKu4z8NmVsYvFF3wjvkB79CFzc653WbWHq885+dN4B4zaw5gZpXMbF9p/Qpobmbn+7MPN5P7i6F91gMN971oc86txDuG8Ul/n9MSb/blg3wun/MxlZ8BwLX+LJ+ZWXkzOzvHE/w+iXhv0aYAsWb2X7zxUBj7tZ9xzq3C+wDL+8An/rst++S5jgWMoUS8mc9UM6sKPFjI7JhZbzNL8p8jUv2TM/Eeo2eb2SlmFodX7vbg3Wc5FbRvzm+99ncchC0VrwPgD4CueG9vpeC9wvgP+7c9H8P7hMhsvE8oTvdPK4p82/F2bsPxnjgvwXv1Wlh3+pl+w3vL4Gm8Y7UKvd7Oub14n3Q5E28W7XW8Y3wWHtha/e26M/FKXGu8DxhsxHtCrOSf5X28T1wuB77De7suv+ubj3f8y0S8B/6ReMckFYWX8Lb9d2a2He+A+GP8292F/+lX/62HDv5lxuPtIH/K43c4iPHjnBuD9+m/L8ysbS5nORqYbGY7/Oy3OOeWOec24T0534H3Flc/oItzLte33Zxz+8p3Hbwngv3mnPsaeBn4Ee+A3Yn+oj25nPdgx33O6xuGd1zbbXhjbD7eMZ6d/G2xz7H+ttqGV5IrAkc75+bkuMpU+/v3eN2ex01finfs1kK8Y9Bu9fOMxTue5hO8WaRDgIsPdP3wZlK34M0kfIB3sPm+x+f1wCP+mP0v3jbNk3NuJN5+4iP/Lay5eI99/PHRA3gKb9wcRv6Pr33lb5OZTfd/7ol3vM8avLesH/THcW7eBo7wH1Of5XGe7Nmn4n046VW87fE73vFWufkWbywvxnubazeFfNv0APcz7/rny/k245N4LwpSzezOXC6X6xjCO1avLN54ngR8U5jsvjOAef5YfwnvONbdzrlFeG+FvuJf7zl4X8O0N5frKGjfXNB67c84CFvmvbATEQl/ZtYM70k93gX8HXCRzJ+BHuKc259Zeilh/kzVELwPUGQVdH6JDJrxEpGwZmbn+W+HVcGbVRml0iXRzn/b7hZgoEpXdFHxEpFwdw3eW9t/4B1Tcl2wcUSKlz+zm4r3CdoXA44jRUxvNYqIiIiUEM14iYiIiJSQsPtisdxUr17dNWzYMOgYIiIiIgWaNm3aRudcUm7LIqJ4NWzYkKlTpwYdQ0RERKRAZpbzr3z8RW81ioiIiJQQFS8RERGREqLiJSIiIlJCIuIYLxERETl46enprFq1it27dwcdJSokJCRQr1494uLiCn0ZFS8REZFSYtWqVSQmJtKwYUPMLOg4Ec05x6ZNm1i1ahWNGjUq9OX0VqOIiEgpsXv3bqpVq6bSVQTMjGrVqu337KGKl4iISCmi0lV0DmRbqniJiIiIlBAVLxEREYk4gwcP5sYbbyzwPGvWrPnr9759+zJ//vz9vq1x48bRpUuX/b5cblS8REREJCrlLF4DBw7kiCOOCDCRipeIiIiUsG7dutG2bVuaN29O//79AahQoQL33XcfrVq1okOHDqxfvx6AUaNGccwxx3DUUUdx6qmn/nX6Ptu3b6dRo0akp6cDsG3bNho2bMjHH3/M1KlT6dWrF61btyYtLY3OnTv/9ScIv/nmG9q0aUOrVq045ZRTAJgyZQodO3bkqKOOomPHjixatKjI111fJyEiIlIKPTxqHvPXbCvS6zyiTkUePKd5gecbNGgQVatWJS0tjaOPPpru3buzc+dOOnTowOOPP06/fv0YMGAA999/P8cddxyTJk3CzBg4cCDPPPMMzz///F/XlZiYSOfOnfnqq6/o1q0bH330Ed27d6dHjx689tprPPfcc7Rr1+5vt5+SksJVV13FTz/9RKNGjdi8eTMATZs25aeffiI2Npbvv/+ee++9l08++aRIt5GKl4iIiJSol19+mZEjRwKwcuVKlixZQpkyZf46jqpt27aMGTMG8L577KKLLmLt2rXs3bs31+/M6tu3L8888wzdunXjnXfeYcCAAfne/qRJkzjhhBP+uq6qVasCsHXrVi677DKWLFmCmf01i1aUVLxERERKocLMTBWHcePG8f333zNx4kTKlStH586d2b17N3FxcX99PUNMTAwZGRkA3HTTTdx+++2ce+65jBs3joceeugf19mpUyeWL1/O+PHjyczMpEWLFvlmcM7l+lUQDzzwACeddBIjR45k+fLldO7c+aDXNycd4yUiIiIlZuvWrVSpUoVy5cqxcOFCJk2aVOD569atC8C7776b5/n69OlDz549ueKKK/46LTExke3bt//jvMceeyzjx49n2bJlAH+91Zj9tgYPHrxf61VYKl4iIiJSYs444wwyMjJo2bIlDzzwAB06dMj3/A899BA9evTg+OOPp3r16nmer1evXmzZsoWePXv+ddrll1/Otdde+9fB9fskJSXRv39/zj//fFq1asVFF10EQL9+/bjnnnvo1KkTmZmZB7mmuTPnXLFccVFq166d2/cpBBERETkwCxYsoFmzZkHHKBYjRozg888/5/333y/R281tm5rZNOdcu9zOr2O8REREJKLddNNNfP3114wePTroKAVS8RIREZGI9sorrwQdodB0jJeIiIhICSm24mVmg8xsg5nNzXZaVTMbY2ZL/P+rFNfti4iIiISb4pzxGgyckeO0u4GxzrnDgLH+72EhI31v0BFEREQkyhVb8XLO/QRsznFyV2Dfl3C8C3QrrtvfH1NGvsK6J44kbec/v+tDREREpKiU9DFeNZ1zawH8/2vkdUYzu9rMpprZ1JSUlGINlVjncOq5dcz67IVivR0REREp3cL24HrnXH/nXDvnXLukpKRiva1mx5zO3PjWHLbkbXbv0qyXiIiIFI+SLl7rzaw2gP//hhK+/TyFOt9NNbYya+T/go4iIiIStZYvX06zZs246qqraN68Of/6179IS0ujc+fO3HXXXbRv357DDz+cn3/+OeioxaKkv8frC+Ay4Cn//89L+PbzdMSxZzLnx9YcuuRtdu+6jYRyiUFHEhERKT5f3w3r5hTtddY6Es58qsCzLVmyhKFDhzJgwAAuvPBCPvnkEwAyMjKYMmUKo0eP5uGHH+b7778v2nxhoDi/TmIoMBFoYmarzOxKvMJ1mpktAU7zfw8b1vkuqpHK7M9eDDqKiIhI1GrUqBGtW7cGoG3btixfvhyA888//x+nRZtim/FyzvXMY9EpxXWbB6v5sWcyd1wrGi8e6M96VQg6koiISPEoxMxUcYmPj//r55iYmL/+gPW+02NiYsjIyAgkW3EL24Prg2BmuBPvojqpzPnipaDjiIiISJRR8cqhRcezmBvXkkYL+7MnbUfQcURERCSK6I9k52BmZJ7Qj+pje/Pb5y9z9MX3Bh1JREQkajRs2JC5c//6a4Lceeed/zhP9erVo/YYL8145aLlcV2YG3ckjRb2Z+/uXUHHERERkSih4pULMyPj+H5UZwuzdayXiIiIFBEVrzy0Oq4Lc+Na0GB+f9L3aNZLRESig3Mu6AhR40C2pYpXHiwUIv24/5DEZmZ/8XLQcURERA5aQkICmzZtUvkqAs45Nm3aREJCwn5dTgfX56P18ecy7+fmJM/vT/qem4mLLxd0JBERkQNWr149Vq1aRUpKStBRokJCQgL16tXbr8uoeOXDQiF2d/oPNcZfzvRRr9Lmgn5BRxIRETlgcXFxNGrUKOgYpZreaixAmxO7Mi/2COrNe5OMPWlBxxEREZEIpuJVAAuFSOv4H2q4Tcz58tWg44iIiEgEU/EqhLaduzEvphl1576hWS8RERE5YCpehWChELs63kkNt4m5X70WdBwRERGJUCpehdS28/nMj2lK7TlvkLl3d9BxREREJAKpeBVSKCbEjmPvpKbbyJyvXg86joiIiEQgFa/90O6k7syPaULt2a+Tmb4n6DgiIiISYVS89kMoJsS2Y+6kpkth7ldvBB1HREREIoyK1346+pQLWBBzOLVmvUqWZr1ERERkP6h47aeYmBCp7W+npkth3tdvBh1HREREIoiK1wFof+pFLAgdRo0ZmvUSERGRwlPxOgAxMSG2tL+dmm4D8795K+g4IiIiEiFUvA7Qvlmv6jNeJSt9b9BxREREJAKoeB2g2NgYNre7jVpZ61nwrWa9REREpGAqXgfhmH9dzMLQoVSb/gouQ7NeIiIikj8Vr4MQGxvDxra3erNe3wwIOo6IiIiEORWvg3TM6ZewMHQIVaa/rFkvERERyZeK10GKi40hpc2t1M5ax8LvBgYdR0RERMKYilcR6HBGLxZZYypPfRmXmR50HBEREQlTKl5FIC42hvVtbqV21loWadZLRERE8qDiVUQ6nNGbRdaYSlNf0qyXiIiI5ErFq4iUiYthbetbqJ25lsVjBgUdR0RERMKQilcROvas3iy2RlT87UXNeomIiMg/qHgVofi4WNa0upnamWtYMvadoOOIiIhImFHxKmLHnt2HxdaQCpM16yUiIiJ/p+JVxOLjYlnV8mbqZK5myQ/vBh1HREREwoiKVzHoeHYfltCACpNegKzMoOOIiIhImFDxKgYJZeL488ib/FmvwUHHERERkTCh4lVMOp1zOUuoT3nNeomIiIhPxauYJJSJY0WLG6mTsYo/xr0XdBwREREJAypexajTOf/md+pT9tfnNOslIiIiKl7FqWx8HMta3ECdjFUsHf9+0HFEREQkYCpexcyb9Uom4RfNeomIiJR2Kl7FrFx8GZY1v4E6GStZ9tOQoOOIiIhIgFS8SkDHc67kD+pR5pfnNeslIiJSiql4lYDyCWX4vdkN1E1fwfKfPww6joiIiARExauEdDrXm/WKm/AsZGUFHUdEREQCoOJVQiqUjef3ptdRN30Ff07QrJeIiEhppOJVgjqe25el1CXmZ816iYiIlEYqXiUosVwCi5pcR9305fz5y9Cg44iIiEgJU/EqYR3Pvcqb9fpJs14iIiKljYpXCatUPoGFh19L3fRlrPx1WNBxREREpASpeAWg07lXs8zVwX56RrNeIiIipYiKVwAqVUhgweHXUm/vUlZPGh50HBERESkhKl4B6dj1apa52rhxT2vWS0REpJRQ8QpI5QplmX/YNdTbu5Q1k0cEHUdERERKgIpXgDp2vYblrjZZ457SrJeIiEgpoOIVoCqJ5Zh76NXU2/MHa6d8EnQcERERKWYqXgE7tus1rHC1yPzxKXAu6DgiIiJSjFS8AlatYnnmNL6Kent+Z+2UT4OOIyIiIsVIxSsMdDjvOla4mmT+8KRmvURERKKYilcYqF6xPLMbX0W9PUtYP3Vk0HFERESkmKh4hYkO3a5nhatJ+tgnNOslIiISpVS8wkRSpfLMbNiXeruXsGHaZ0HHERERkWKg4hVGju12HX+6Guz9XrNeIiIi0UjFK4zUqJLIjAZ9qbd7MSnTvwg6joiIiBQxFa8w0+G861nparBbs14iIiJRR8UrzNSsksj0Bv8mOW0hKdNHBR1HREREipCKVxhq3+0GVrokzXqJiIhEGRWvMFS7akWm1v83yWkL2Djzy6DjiIiISBFR8QpT7bvdwCpXnbQxj2vWS0REJEqoeIWputUqMSX53yTvWsDmWV8FHUdERESKgIpXGDu66w2sdtXZ+Z1mvURERKKBilcYS06qzOR6V5C8az6bZ38ddBwRERE5SCpeYe7objf6s16PadZLREQkwql4hbnkpMpMrnsZyTvnkTrn26DjiIiIyEFQ8YoAbbvdyBpXje3fPqpZLxERkQim4hUBGtSoyq+1LyN551y2zP0u6DgiIiJygFS8IkTb825ijavKjm806yUiIhKpVLwiRKOaVfm1Vh+Sd85h6/wxQccRERGRA6DiFUHanHcza11Vtn2tWS8REZFIpOIVQRrXqsYvtS4lecdsts7/Pug4IiIisp9UvCLMUV1v8ma9dKyXiIhIxFHxijCH1EliQs3eJG+fxbYFPwQdR0RERPaDilcEOqrrzaxzVdj69SOa9RIREYkgKl4R6NC6SfxUozfJ22eyY+GPQccRERGRQlLxilCtut7MeleZLV8/GnQUERERKSQVrwjVpF4Nxif1JnnbdM16iYiIRAgVrwjWsuvNbHCV2TJas14iIiKRQMUrgjVNrsmP1S8heds0diwaF3QcERERKYCKV4Q7suutmvUSERGJECpeEe6I+jX5odolJG+dys7F44OOIyIiIvlQ8YoCLc69lRRXic1fadZLREQknKl4RYEWDWsytmpPkrf+xq7ffw46joiIiORBxStKNO/qzXpt+vKRoKOIiIhIHlS8osSRDWsztupFJKdOIe33CUHHERERkVyoeEWRZl1uJcVVZONXmvUSEREJRypeUaTVIXX5vsrFJG+ZTNofvwQdR0RERHJQ8YoyTbvcykZXUcd6iYiIhCEVryhz1KF1GVP5QuptmcSepb8GHUdERESyUfGKQod3uY1NLpEUzXqJiIiEFRWvKNT2sHp8W+lC6m2eyJ6lE4OOIyIiIj4Vryh12Nm3erNe+oSjiIhI2FDxilJHN4FXUkcAACAASURBVKnPtxV7UG/Tr+xZPinoOCIiIoKKV1Q75Ozb2OwqkDJKs14iIiLhQMUrirVvkszXiT2ot+kX9iyfEnQcERGRUi+Q4mVmt5nZPDOba2ZDzSwhiBzRzsw45Oxb2eIq6BOOIiIiYaDEi5eZ1QVuBto551oAMcDFJZ2jtDimaQNGJ15AvY0/s3fF5KDjiIiIlGpBvdUYC5Q1s1igHLAmoBxRz8xodJb3NxxTP7kDsrKCjiQiIlJqlXjxcs6tBp4D/gTWAludc9/lPJ+ZXW1mU81sakpKSknHjCrHNmvIyGrXUGPbHLZNHBR0HBERkVIriLcaqwBdgUZAHaC8mfXOeT7nXH/nXDvnXLukpKSSjhlVzIzTL7mVqa4JobEPw67NQUcSEREplYJ4q/FUYJlzLsU5lw58CnQMIEep0qB6BRa3fZiEzB2s/uTuoOOIiIiUSkEUrz+BDmZWzswMOAVYEECOUqf7Wf/iszJdqP3HcH29hIiISACCOMZrMjACmA7M8TP0L+kcpVF8bAz1z3+UFFeJLR/fCFmZQUcSEREpVQL5VKNz7kHnXFPnXAvn3KXOuT1B5CiN2jdryHf1bqLWzkVs+PGNoOOIiIiUKvrm+lLorItvZArNKTfhSdyODUHHERERKTVUvEqhaokJbDzhCcpkpbH8o/8EHUdERKTUUPEqpc7ofCJflT+fRqs+Y9uin4OOIyIiUiqoeJVSoZBxRM/HWOOqsWPkLZCZEXQkERGRqKfiVYo1Sa7FpMPvoM7uP1jxzYtBxxEREYl6Kl6l3BkXXM2k0FFU/+059m5ZHXQcERGRqKbiVcqVi48j64yniXXpLB96R9BxREREopqKl9Cx/TGMqXoxh2/4mg2zxgQdR0REJGqpeAkAbXs/ykpXg/Qvb8dl6PtsRUREioOKlwBQu1pV5re+j7rpf7L482eCjiMiIhKVVLzkL6ec24dJce2pP+cVdqasCDqOiIhI1FHxkr/ExoRI7PY85rJY8cEtQccRERGJOipe8jfNm7dkQu3LOCL1R5ZPHhV0HBERkaii4iX/cHSvh1lBbeK+7Ufm3t1BxxEREYkaKl7yD5USK7D62Iepm7WGOcMfCTqOiIhI1FDxklwd+68LmVL2OJou6c/GlYuCjiMiIhIVVLwkV2ZGrQv/RyYh1gy7Leg4IiIiUUHFS/JUv9HhzGh0NS13/MLcH4cFHUdERCTiqXhJvo7ueR/LrR5Vf3qA3bt2BB1HREQkoql4Sb7i48uy45SnqePWM/PD/wYdR0REJKKpeEmBWhzXhWkVT6XNyndZsXh20HFEREQiloqXFErDnv9jr8Wx5ZNbcVlZQccRERGJSCpeUijVatdnYdMbab1nGpNHvxt0HBERkYik4iWF1uaCu1gW24iGUx8jNXVL0HFEREQijoqXFFooNg7Oep5abGTmB/cFHUdERCTiqHjJfmnU5hRmVT+bThs+Yt6sKUHHERERiSgqXrLfDr3kedIsgb2j7iA9IzPoOCIiIhFDxUv2W/mqtVl91B0clTGb8Z++FXQcERGRiKHiJQek2Tm3srzM4bSc9wyr160POo6IiEhEUPGSAxOKodz5L1KdVOZ9eA/OuaATiYiIhD0VLzlgNZp2YlHd8zh560h+nfhT0HFERETCnoqXHJRDez7LzlB5yo+5i52704OOIyIiEtZUvOSgxCVWZ0vH+2jtFvD9Ry8FHUdERCSsqXjJQWt4yjX8Wa45nZa9xIJlfwYdR0REJGypeMnBC4Wo0uNlqtgOlg67h6wsHWgvIiKSGxUvKRKJjdqxvNHFnJH2Fd+M+SboOCIiImFJxUuKTOMLn2BHTCXqTnyAlG1pQccREREJOypeUmSsbBV2n/QQrVjC2A+fCzqOiIhI2FHxkiJV87jLWVXxKP619k0mzV0SdBwREZGwouIlRcuMpIteoaLtYuNn97I7XX9EW0REZB8VLyly8XWPZF3TyzkrfQyfffl50HFERETChoqXFIt65z3CtrhqtJj5CEvXbw06joiISFhQ8ZLiEZ+Inf44LWwZ44c+oz+iLSIigoqXFKNK7S5ibdX2nL9lEN9MmRN0HBERkcCpeEnxMaPGxa9SzvaS8c0DbN2lP6ItIiKlm4qXFKuYGk1IbXU157hxDPtkWNBxREREAqXiJcUu6ez7SS1TixOWPMX05SlBxxEREQmMipcUvzLlie/yNE1DK/lt2NNkZGYFnUhERCQQKl5SIsoe2ZWUmsdzya4hDP9xStBxREREAqHiJSXDjOoXvkSCZVDx50dYnao/oi0iIqWPipeUGKt2CLva30QX+4WPhg0JOo6IiEiJU/GSElXptH5sS6hL19Uv8P2clUHHERERKVEqXlKy4spSrtvzHBpaw+LPn2bX3oygE4mIiJQYFS8pcbFNz2RL8qlcnj6cQV9NCDqOiIhIiVHxkkBUOf8FYkNwyIzHWbhuW9BxRERESoSKlwSjSgMyO93OmaEpDB86mKws/RFtERGJfipeEpiynW9je/mGXLrlVT6e/HvQcURERIqdipcEJzaeCuf9j0ah9Wz89lk27tgTdCIREZFipeIlgbJDT2b7IV240n3KmyN/CDqOiIhIsVLxksAlnvsMoZhYjln8DL/+sTHoOCIiIsVGxUuCV6ku1vkuTouZzlcjBrEnIzPoRCIiIsVCxUvCQlynG9lZ8VCu3dmft3+YH3QcERGRYqHiJeEhJo7y571IcigFN+F5lm/cGXQiERGRIqfiJeGj0fGkNe3OVTaKV0d8i3P6bi8REYkuKl4SVsqe/QTEJXDO6v8xataaoOOIiIgUKRUvCS+JtYg95T5OjJnNhFGD2JqWHnQiERGRIqPiJWEn1P5q0qoewa0Zg3h59Iyg44iIiBQZFS8JPzGxlO32InVsM0kzXmbmytSgE4mIiBQJFS8JT/WPIf3IS+gbO5o3R4wmIzMr6EQiIiIHTcVLwlbcGY/i4srTZ/MrvPvr8qDjiIiIHDQVLwlf5asTe9qDdIyZz8Ix77B2a1rQiURERA6KipeENWt3BXtqtOI/9j7PfP5b0HFEREQOioqXhLdQDPFd/0eSpdJi8ev8sHB90IlEREQOmIqXhL+6bclqcxmXx37L4E+/Im2v/oi2iIhEJhUviQgxpz5IVnwlbtr9Bi+PXRx0HBERkQOi4iWRoVxV4k5/hKNDi9n4y2AWrdsedCIREZH9puIlkaN1b9LrHM09sR/y5Ke/kpWlP6ItIiKRRcVLIkcoRNw5L1CZHZy8ZgAjpq0KOpGIiMh+UfGSyFK7Jda+L71jv2fk6C/ZtGNP0IlEREQKTcVLIo6dfD9ZZatzV+YAnho9P+g4IiIihabiJZEnoRKxZzxO69AfxMwawqSlm4JOJCIiUigqXhKZWl5IZnJH7on7iGc+/ZW9Gfoj2iIiEv5UvCQymRHT5XkSLY0eqW8z4OelQScSEREpkIqXRK6aRxDqcB09Y39k/NjRrNi0M+hEIiIi+VLxksjW+W4yy9fioZi3ue/TmWTqu71ERCSMqXhJZItPJObMJznCltN4+TBeHrsk6EQiIiJ5UvGSyNf8PFzjk7ivzDBG//gjPy7aEHQiERGRXKl4SeQzw857kzJlExmY8Ar3fjSRVVt2BZ1KRETkH1S8JDok1sIueJv6bjX3ZfXn+iHT2JORGXQqERGRv1HxkujR+ESs8710sQm0WDeSR0bpW+1FRCS8qHhJdDn+Djj0VB4p8x4zp4znE/0hbRERCSMqXhJdQiE4rz8xiUkMKvcKT302iYXrtgWdSkREBFDxkmhUvhrW411quI08G/sW170/jW2704NOJSIiouIlUSq5PXbao3R2Uzht68f85+NZOKcvVxURkWCpeEn06nAdNDuXu+M+YtP88fp7jiIiEjgVL4leZtD1VaxKAwaWe42B30xh8tJNQacSEZFSTMVLoltCJazHu1RiB6+XfYObP5zKhm27g04lIiKllIqXRL/aLbGznqVd5iwu3TucGz+cQXpmVtCpRESkFFLxktKhTR9o1ZMbQp8Q/+c4nv12UdCJRESkFFLxktLBDM5+Hktqyptl3+SLn37jm7lrg04lIiKljIqXlB5lysOF71EuJoN3KrzO3R9PZ2nKjqBTiYhIKaLiJaVL0uHYOS/RLGMBt9tQrhsynV17M4JOJSIipYSKl5Q+R14AR/elD6NosPEH7h85V1+uKiIiJULFS0qn05+AOkfxSsIAfps5nQ+n/Bl0IhERKQVUvKR0io2HHoMpExfD+4mv8eQXM5m1MjXoVCIiEuUCKV5mVtnMRpjZQjNbYGbHBpFDSrkqDbHz3qLh3t95LGEI138wnS079wadSkREolhQM14vAd8455oCrYAFAeWQ0q7JmdDpFrplfkeHHWO5ddhMsrJ0vJeIiBSPEi9eZlYROAF4G8A5t9c5p/d4JDgn/xcadOLp+IGsXjKDV374PehEIiISpYKY8WoMpADvmNkMMxtoZuVznsnMrjazqWY2NSUlpeRTSukREwsXDCImIZEhia/Rf+xsxi/WmBMRkaIXRPGKBdoAbzjnjgJ2AnfnPJNzrr9zrp1zrl1SUlJJZ5TSJrEW1v1taqav5JUK73HL0OmsTk0LOpWIiESZIIrXKmCVc26y//sIvCImEqzGJ2Kd7+Xk9HGcnzWG6z+Yzp6MzKBTiYhIFCnx4uWcWwesNLMm/kmnAPNLOodIro6/Aw49lftjBpOxagaPfanPfYiISNEJ6lONNwEfmNlsoDXwREA5RP4uFILz+hOqUIMhFV/j80nzGTljVdCpREQkSgRSvJxzM/3jt1o657o557YEkUMkV+WrQY/BVE7fwMBK73DPp7NZtG570KlERCQK6JvrRXKT3B477VHa7/mVa+O+4boh09i+Oz3oVCIiEuFUvETy0uE6aHYut7ghVN8yg34jZuuPaYuIyEFR8RLJixl0fRWr0oBBFV5n8tzFvD1hWdCpREQkgql4ieQnoRL0eJfyGVsZUmUAT389nynLNgedSkREIpSKl0hBarfEznqWI9KmcW+FL7nxw+ls2L476FQiIhKBVLxECqNNH2jVk8v3fkSL3dO46cMZZGRmBZ1KREQiTKGLl5lVMbPmZtbYzFTYpHQxg7Ofx5Ka8kbZN1m+7Hee/W5R0KlERCTC5FugzKySmd1rZnOAScBbwHBghZl9bGYnlURIkbBQpjxc+B7x7GVY1bd4e/xivp23LuhUIiISQQqauRoBrASOd841cc4d53/xaTLwFNDVzK4s9pQi4SLpcDjnJRrumsMzVT7jzuGzWL5xZ9CpREQkQsTmt9A5d1o+y6YB04o8kUi4O/IC+HMi5/82kJ9Ch3LtkLKMvL4TZcvEBJ1MRETCXKGO1co5q2VmMWb2YPFEEokApz8BdY7iudg32LX+d+7/bK6+XFVERApU2IPkTzGz0WZW28xa4B3vlViMuUTCW2w89BhMbEyIj6u+xZfTl/LRbyuDTiUiImGuUMXLOXcJ8C4wBxgN3Oqcu7M4g4mEvSoN4by3qLlzIa9XH8GDn89j9qrUoFOJiEgYK+xbjYcBtwCfAMuBS82sXDHmEokMTc6ETrdwyo4vuaTcZK4bMp3UXXuDTiUiImGqsG81jgIecM5dA5wILAF+K7ZUIpHk5P9C/Y484N4icfsf3DpsJllZOt5LRET+qbDFq71zbiyA8zwPdCu+WCIRJCYWLhhETHx5PqryBpMXreS1H38POpWIiIShgr5A9TgA59y2nMucc0vMrKJ/sL1I6VaxNnQfSKUdS3m/xlBe+H4RPy9JCTqViIiEmYJmvLqb2a9m9l8zO9vM2pvZCWb2bzN7H/gSKFsCOUXCX+PO2En30m7bGG6t9As3D53B6tS0oFOJiEgYKegLVG8zsyrABUAPoBaQBiwA3nTO/VL8EUUiyPF3wp+TuHnZQCZk1ueGD8oz/JpjKROrP28qIiKFOMbLObcFqAjMBsYAE4CNQFMza1288UQiTCgE5w/AKiTxbvlXWbpyNY9/NT/oVCIiEiYK+zK8LXAtUBuoA1wNdAYGmFm/4okmEqHKV4Megym3ex0f13qfdycu5/OZq4NOJSIiYaCwxasa0MY5d6dz7g6gHZAEnABcXkzZRCJXcns47RGapP7EI0njuPuTOSxevz3oVCIiErDCFq/6QPZvhUwHGjjn0oA9RZ5KJBp0uB6ancOlOwbRsczvXDtkGjv2ZASdSkREAlTY4vUhMMnMHvT/OPYvwFAzKw/oABaR3JhB19ewyvV5Pf4Vtm9ax10jZuuPaYuIlGKF/VuNjwJXAanAVuBa59wjzrmdzrlexRlQJKIlVIIL3yN+zxY+rfkOX89ZzaBflgedSkREApLv10lk55ybBkwrxiwi0al2SzjrGZJH3cKLtZtw++gYWtWrRLuGVYNOJiIiJUxfLiRSEtpcBi0v5pwt79G14iJu+HA6Kdt1eKSISGmj4iVSEsygywtYUlOe5hXi09Zz89AZZGRmBZ1MRERKkIqXSEkpUx4ufJfYzN18Wn0Avy1dz/NjFgedSkRESpCKl0hJSmoC575M9S0zGJT8NW+M+4Mx89cHnUpEREqIipdISTvyAji6LyekDKVv0nxuHz6TFZt2Bp1KRERKgIqXSBBOfwLqHMW9e14mmQ1cO2Q6u9Mzg04lIiLFTMVLJAix8dBjMKGQMazKGyxdt5H7P5urL1cVEYlyKl4iQanSEM57i8Qt8xjeYBQjpq1i2G8rg04lIiLFSMVLJEhNzoROt9Bq3Sf0qzOb/34xj7mrtwadSkREiomKl0jQTv4v1O/Iddtfpm3ZDVw7ZBqpu/YWfDkREYk4Kl4iQYuJhQsGYWXKM6j8K2zblsrtw2eRlaXjvUREoo2Kl0g4qFgbug+kbOrvfFZ/BD8sXM/r434POpWIiBQxFS+RcNG4M5x0L43XfsVTDafzwpjFTFiyMehUIiJShFS8RMLJ8XfCIadwUcqrnFFtPTd/NIO1W9OCTiUiIkVExUsknIRCcP4ArHx1Xgq9SJn0bVz/wXT2ZuiPaYuIRAMVL5FwU74a9BhM3I7VfFbvQ2b8uYUnRi8IOpWIiBQBFS+RcJTcHk57hFprvqf/oZMZ/Otyvpi1JuhUIiJykFS8RMJVh+uh2TmctuZ1etVey92fzGbWytSgU4mIyEFQ8RIJV2bQ9TWsUjKPpD/HIeXT6DNoCvPXbAs6mYiIHCAVL5FwllAJLnyPmLTNDE96hwpx0PvtySxZvz3oZCIicgBUvETCXe2WcNazlP1zPN80GkacZXHJwMks27gz6GQiIrKfVLxEIkHby+Dk+0lcNILvDv2UrMxMeg2YxMrNu4JOJiIi+0HFSyRSnPAfOKEflRZ+xJimo9ixJ51LBk7SF6yKiEQQFS+RSHLSvdDpFqrOf5+xzb9ly8699BowmQ3bdwedTERECkHFSySSmMGpD0OH60maN4gxR/7A2q1p9B44mc079wadTkRECqDiJRJpzOD0J+DovtSe+xbftZ7Aik27uPTtyWzdlR50OhERyYeKl0gkMoMzn4U2fUie8yrfHDWJxeu3c9k7U9ixJyPodCIikgcVL5FIFQpBl5egVU8azXmRL9tOZ87qrfz7nd/YtVflS0QkHKl4iUSyUAi6vgYtutNk9rN83nY2U1ds5ur3prE7PTPodCIikoOKl0ikC8XAeW9Bs3NpMedJRrRbwITfN3L9B9PZm5EVdDoREclGxUskGsTEQfe34fAzaTPnUT5qt5gfFm7g5qEzyMhU+RIRCRcqXiLRIrYMXPguHHoqHeY+zLttl/LNvHXc8fEsMrNc0OlERAQVL5HoEhsPFw2BRidw4vz/0r/NCj6fuYZ7Pp1NlsqXiEjgVLxEok1cWej5EdQ/ln8tuJ+XW69k+NRVPPjFPJxT+RIRCZKKl0g0KlMOLhkGddtyzuL7ePbI1bw/aQVPjF6g8iUiEiAVL5FoFZ8IvUdgtVpywdL7eKz5Ogb8vIwXxiwOOpmISKml4iUSzRIqwaWfYklN6bXiXu5rtp5Xfvid1378PehkIiKlkoqXSLQrWwUu/Qyregh9V97L7Yen8Oy3ixj489Kgk4mIlDoqXiKlQflq0OdzrHJ9blp3HzccuonHvlrA+5NWBJ1MRKRUUfESKS0qJMFlX2AVanJnyr30bbyZBz6by/CpK4NOJiJSaqh4iZQmibXgslFYuWrct/k+ejdI5a5PZvP5zNVBJxMRKRVUvERKm0p1vfIVX5FHt91P97pbuX34LL6ZuzboZCIiUU/FS6Q0qlzfe9sxNoFndv2Xs2pt5aahM/hx4Yagk4mIRDUVL5HSqmpjuOwLQqEQL+19kJOStnHNkGlMWLIx6GQiIlFLxUukNKt+GPT5glBWJm9mPkzHKtvp+95vTFm2OehkIiJRScVLpLSr0RT6fE4oI4237RGOqridK96Zwow/twSdTEQk6qh4iQjUagGXfkbM3m28H/sYTcrt4LJBU5i7emvQyUREooqKl4h46rSG3iOJTdvMsIQnaBC/g0vfnszi9duDTiYiEjVUvETk/9VrC71HELdzHZ+Uf4qk0HYuGTCZpSk7gk4mIhIVVLxE5O/qd4BewymzbSWjKj1Lxaxt9Bo4mZWbdwWdTEQk4ql4icg/NTwOeg4lPnUpX1V9gdi92+g5YBJrUtOCTiYiEtFUvEQkd4ecBBd/QNkti/i2+otk7NpKr4GT2bBtd9DJREQiloqXiOTtsNOgx7uU2zSX72u+yvZtqfQaOJlNO/YEnUxEJCKpeIlI/pqeBd3fpkLKDMbWfoMNm7dw6dtT2LorPehkIiIRR8VLRArWvBuc9xaVNkzhx3r9+XPDZvq8M4Xtu1W+RET2h4qXiBROyx7Q9TWqrvuVH+u/zZLVG/n34N/YtTcj6GQiIhFDxUtECq/1JXDOiyStHc8PDd5l1ooU+r47ld3pmUEnExGJCCpeIrJ/2l4OZz1HrbVj+aHhEKYs3cC1Q6axJ0PlS0SkICpeIrL/2l8Fpz9BvbXf8X2jofy0aD03fTiD9MysoJOJiIQ1FS8ROTDH3gCnPkTDNaP5tvFwxsxfy+3DZ5GZ5YJOJiIStmKDDiAiEey42yBjL4eNe4KvGsVw9qzulIkJ8ewFLQmFLOh0IiJhR8VLRA7Oif0gcw9H/Pw8IxvF0m16NxLiQjzWrQVmKl8iItmpeInIwTGDkx+AzL20/vUVhjeM48LJZxMfG8MDXZqpfImIZKPiJSIHzwxOexQy02k/+U3ebxDDpb+cQdkyIf5zetOg04mIhA0VLxEpGmZwxlOQsYfjp73DwPqx9P3RSIiN4aZTDgs6nYhIWFDxEpGiYwZnvwCZ6Zw68x1eqx/LDWMgIS6Gq05oHHQ6EZHAqXiJSNEKheDclyErnbNnD2BPvVhuHw3xcSH6HNsw6HQiIoFS8RKRoheKga6vQ8Yezp//BnvqxnDP5xAfG+Kio+sHnU5EJDAqXiJSPGJioftAyMqg58JX2V07hrs/hfjYGLodVTfodCIigdA314tI8YmJgwvegcNO54otL3FXzanc8fEsvp6zNuhkIiKBUPESkeIVWwYufA8OOZlrUv/HzdWnc9PQGYxdsD7oZCIiJU7FS0SKX1wCXPwh1vA4bt7+PFdVm8V1Q6bz85KUoJOJiJQoFS8RKRlxZeGSYVjyMfTb8Sy9K8/hqvemMmnppqCTiYiUGBUvESk5ZcpDr4+xum14IO0ZuleYx5WDf2Paii1BJxMRKREqXiJSsuITodcIrGZzHtv7DGeUnc/l70xh7uqtQScTESl2Kl4iUvLKVoZLR2LVD+e5jKfpHLeQ3m9PZt4alS8RiW4qXiISjHJVoc9nWNVGvOSepEPMInq8OZHv5q0LOpmISLFR8RKR4JSvDpd9QahSPV7nSbpWWc41Q6bx+rjfcc4FnU5EpMipeIlIsCrUgMtGEapYmye2388T9afzzDeLuH34LHanZwadTkSkSKl4iUjwKtaGvmOxxifSc/1zjGr0KV/OWMHF/SexYfvuoNOJiBSZwIqXmcWY2Qwz+zKoDCISRspWhkuGQ6dbOXLtCCbXe5mUdavo+uov+sSjiESNIGe8bgEWBHj7IhJuQjFw2sPQ/W2qps7jx4oP0TTrD3q8OVF/31FEokIgxcvM6gFnAwODuH0RCXNHXgBXfkuZ2BgGuQe4pspUrvtgOi+PXaKD7kUkogU14/Ui0A/IyusMZna1mU01s6kpKfp7biKlTu1WcPU4rG5bbt32LO/U/YIXxyzkpqEzdNC9iESsEi9eZtYF2OCcm5bf+Zxz/Z1z7Zxz7ZKSkkoonYiElfLVoc/ncPRVnLTpI8bXfpUJc5Zw4VsTWbdVB92LSOQJYsarE3CumS0HPgJONrMhAeQQkUgQEwdnPwfnvkLy1mn8Wu0x2LCAc1+dwKyVqUGnExHZLyVevJxz9zjn6jnnGgIXAz8453qXdA4RiTBt+sAVoynHHkbGP8gp9hsXvjWRL2atCTqZiEih6Xu8RCRyJLeHq8cRU6MpT+59iscqjeKWodN4/rtFZGXpoHsRCX+BFi/n3DjnXJcgM4hIhKlYBy4fDa170WPnB4xKeotBP8zhhg+ns2tvRtDpRETypRkvEYk8cQnQ9TU442ma7/iVCdUeZ+G8mVzwxkTWpKYFnU5EJE8qXiISmcygw7XYpSOpkpXKdxUeInnzr5z76i9M/3NL0OlERHKl4iUika3xiXD1j8RVrc+boae40r7g4v4T+XT6qqCTiYj8g4qXiES+Kg3hyu+wZudyXfq7DErsz73Dp/DU1wt10L2IhBUVLxGJDmXKQ4/BcMp/6ZQ2jrFVnuSL8ZO5+v1p7Nijg+5FJDyoeIlI9DCD4+/ALhlGnax1jE38v/buO8rK8tD3+PeZytB7FwYVK4iAoiIoiA17Cxp7CRqNWBJjy0miN4nKicbYezSi2BsK3WoHmwAAGthJREFUWADp0kREQBFQEZAyDB2EYWae+wecexOPZYzMfvfM/n7WmrXE2az5LZ4F85139rz7j2z4dAynPTCRRas2Jb1OkgwvSdXQbkcR+o+ioG5jBufdQo81r3HSveOZ+sWqpJdJynCGl6TqqXF76D+SrF378F88ys1Zj3D+I+N4ftqipJdJymCGl6Tqq0Y9+Pkz0PM3HFf6Nq/Uuo2/vjiWP78xhzKfdC8pAYaXpOotKxv6/AF+9gTt4xeMrPNHpkwYwS/+OZV1m7cmvU5ShjG8JGWGvU8mXPQ2dWsV8ErBn2i04GVOuX8iC4s3Jr1MUgYxvCRljuYdof9ostseyO05D3D+uoc45d6xvLegOOllkjKE4SUps9RqBGe/AgdcytkM5ZGsWxnw2AgGT/4y6WWSMoDhJSnzZOdA39vgxPvpzCcMq/kHnnx1KDcNmU1pWXnS6yRVY4aXpMzV+SzCBcNpUhAYUnATyyc9xwVPTGXtJp90L6lyGF6SMlvrroSLR5PXch8eyLuLg764n1PuG8dnRRuSXiapGjK8JKlOczj/Deh8Dpdlv8ofN/6Zs+97h/HzVia9TFI1Y3hJEkBOPpxwDxxzOz2zPuS5rN9x0+Ov8uR7XyS9TFI1YnhJ0v8IAbr1J5z7Gq3zNzMk//eMev0p/uvVj9jqk+4l7QCGlyR9U2EPwiWjKWi2K4/n3U7dqfdw3mOTWbOpJOllkqo4w0uSvk39NoQL3yJ0OIVrc5/j7MU3cfq9I5m/Yn3SyyRVYYaXJH2XvJpw6mNw+M30zZrM3Zuu51f3vcrouSuSXiapijK8JOn7hAA9riKc9SLt81bxfNaNPPzPJ3hs/OfEGJNeJ6mKMbwkqSLaH07WJaOp07A5g/Ju5cvhd3LDSzMpKfVJ95IqzvCSpIpqtAtZ/UeStdtR3Jz7T7rM+D0XPDKO4g1bkl4mqYowvCTpx6hRl3DGYDj0OvrljOG3y37Nhfe+ztxlPule0g8zvCTpx8rKgt43Qr9BdMz9ikc3X8NN9z/OyI+XJ71MUpozvCTpP7XXCWT3H0GDenV5Mutm3nrqdh4as8An3Uv6ToaXJP0UzfYm55LRZBUezH/nPkz+O9dz7fPT2VJalvQySWnI8JKkn6pmQ7LPeZl44K84P+dtTpl1OZc8+BZF633SvaR/Z3hJ0o6QnUM4+hY4+SG65c7nL0UD+O09TzHnq3VJL5OURgwvSdqROp1B9kVv0aR2Lg+WXM+jD97Om7OWJb1KUpowvCRpR2vVhbxLx5LVcl/+lnUXnz97DfePmuuT7iUZXpJUKWo3Je/CoZR2Pp9Lc15nz3f789unxrFmU0nSyyQlyPCSpMqSk0fOiXcRj72TQ3JmccX8i7jhjvt4Z473+5IyleElSZUs7H8h2RcMp1m9mjxQdhNFgy/h+qe9+iVlIsNLklKhzQHkD5hE2UFXcEbOGK769Bz+z+138NZsn3gvZRLDS5JSJbeA7KP+RFb/kdRv1Iy/ld9GybPnceOgUaze6NUvKRMYXpKUaq26UOOycZT1+h19c97nmvnncMcdf+LNj5YmvUxSJTO8JCkJOXlk97qWnEvHU6PZbvy5/G5yn/85v3/yLVZ59UuqtgwvSUpS0z2o+csRlB15K4fkfsx1C87joTtu5M2PliS9TFIlMLwkKWlZ2WR3v4zcAZOh9X7cUP4IDV44hZufGELxBl/vUapODC9JShcNCqn9i9cpPf4e9s1dzHWfX8hTd/ya4R8uSnqZpB3E8JKkdBICOV3PJf/KaWxtdxhXxqdo9dLx3PKP51np1S+pyjO8JCkd1W1BnfOeo+zUx9k1fy2/XfhLXrnjUoZ98EXSyyT9BIaXJKWrEMjueAo1r36fTbufTP/4Eu1fOYa/PvqkV7+kKsrwkqR0V7Mh9c58jLKfv0DzgjJ+s+gK3rr9fIa9P48YY9LrJP0IhpckVRHZux9JnV9PY13HczmLYXR8rS93P/wwReu9+iVVFYaXJFUl+XWof9rdlJ43lDq1anLl0muZcMfpDJsyx6tfUhVgeElSFZTTrgf1fz2F1Z1/xfGMYb+hfXngwbtYsX5z0tMkfQ/DS5KqqtwaNDjxFug/iqw6zbhs+R+ZccdJvDnpQ69+SWnK8JKkKi67VWcaXz2B4gOupxfTOHB4Xx6//1ZWrP066WmSvsHwkqTqIDuXRn1vIPuyCWyuvysXFg1k3p1H8eaEKV79ktKI4SVJ1Uh2091pfuVoinr+mS5hLj3ePp7B9/6eFWs3JT1NEoaXJFU/WVk06TOAvAGTWdOoM2cV38PiO3vz9thxXv2SEmZ4SVI1ld2wkNYDhrOiz520D4s5dOTJvHTXb1i+en3S06SMZXhJUnUWAk17XkjNq9/nq2aHctqax1h1V09GjHrHq19SAgwvScoA2XWb0+6yl1h+9CO0yFpDrzH9GHrnL1lWvCbpaVJGMbwkKYM0O7AfdX/zAZ+1Op7j1j3L1/ccxKi3X/Pql5QihpckZZisWg3Y7eInWXbCYGpll9JrwnmM/Nt5LCsqSnqaVO0ZXpKUoZp3OZbGv53OJ23O4LB1Qyi/90DGDHvWq19SJTK8JCmDZdWow14XPciK014h5hRw6JRLGH/7z1i67Kukp0nVkuElSaJ5x960uG4qM3fuz4EbRpH3wIGMf+1Rr35JO5jhJUkCICuvgH3OvZ2VP3+TtXlN6PHBb5j238exdMkXSU+Tqg3DS5L0b1rs0Y3C6yYxfber2GfTZGo+3J1JL91FLC9PeppU5RlekqT/JSsnly5n3syqc9/lq/x2HPjRH5g1sA9LF85NeppUpRlekqTv1GKXjux+3Vim7HUjO2+eQ71/9GTac7cQy0qTniZVSYaXJOl7ZWVn063fday9cBzzanRkv48HMm9gD5bOn5H0NKnKMbwkSRXSsu1u7HPdO0zc5xaabllEo0F9mPH07yjfWpL0NKnKMLwkSRUWsrLofsqv2HTxe0yveTD7zruXRQO7seyT95KeJlUJhpck6Udr2aoNB1z7GmO7/J2CrWto/MwxzHriSso2r096mpTWDC9J0n8khMAhJ1xA6aWTGFf7KDp88QTrB+7N3Nf+Sty6Oel5UloyvCRJP0nL5s3pdc0zjO/1HAtCG3b/4M8U3dqRz0Y8AuVlSc+T0orhJUn6yUII9Oh1NPv8biwj93+Y4liHncdfw5JbO7No4vPgSw9JgOElSdqBcrOz6HPs6RReP4W39v4rJVu3stPb/fliYHdWfPhO0vOkxBlekqQdriA/h6N+djENr5nOsHa/I//rZTR95TQW3HE4q+dNTnqelBjDS5JUaerVLuCY864lXPEBQ1teToN1H9Pg6SP59J6T2bBkTtLzpJQzvCRJla55o/oce/FfWHvxNIY2PI+WKydS45GDmfvweWwuXpj0PCllDC9JUsq0a9WCY6+4my/Pfo8RtU+kcMkbhHu6MvfJKyhdX5T0PKnSGV6SpJTbq/3OHH3NE8w6eRTj8nux64In2XJHR+a/8F/EzeuSnidVGsNLkpSYrvt2os/1LzDxqNeZntOJXWffw9qBHfj8jduhdEvS86QdzvCSJCUqhEDP7j056IbhvNP9aebThnbT/sTKWzuw+N1HvQmrqhXDS5KUFnKyszjiyOPocOMYhu77AMvL6tB6zG9YelsXVkx+0ZuwqlowvCRJaaVGbjbHnnQmO103idd2u42vt5TQdPhFLPrrQaye5U1YVbUZXpKktFS3II8Tz7yU2r+eyittbiBn43IavHgan995BBs+m5L0POk/YnhJktJa03q1OfnC6ym5bBqvNP0V9dbMofaTR7DgvlPZsvTjpOdJP4rhJUmqEto2a8TJl93C8gsm82q9c2i2Yjw5D3VnwaMXULrqy6TnSRVieEmSqpQ9C1tz0tX3Mvf0cQwrOIHWi4ZQfncXPnvqSuLGlUnPk76X4SVJqpK67rUbx137BFOOe4dRuYfQdt4/+fr2Dix8+Y+wZX3S86RvZXhJkqqsEAI99+/CETe8xMjerzIldKLtzL+zduDeLBn+N2/CqrRjeEmSqrzsrMCRvXpx0I3DGLL/IOaW70SryTdTfFsHVox9zJuwKm0YXpKkaiM/J5sTjj2BvW4Yw0sd7mXp1to0HfVrlg/szKr3X/ImrEqc4SVJqnZq5+dw6mnn0Pya93h+57+wYfNWGr5+IV/d3p0NH49Mep4ymOElSaq2GtepQb9zLydvwGSea3EdccNyaj93Cov+fiSbF05Nep4ykOElSar2dmpcl9MvuZGN/SfzXMNLqbl6DjUeP5yFD5zK1uWfJD1PGcTwkiRljN1aN+H0K27jy7Mn8EKts2i0bDxZDxzEwn9cQPlqb8Kqymd4SZIyTuf2bTntmvuYcfJoXss/juYLh1B6VxcWPXOVN2FVpQqxCvyEx3777RenTZuW9AxJUjVUXh4ZMWkqW0fextGlo9iSVcCaThfTsu81kF8n6XmqgkII78cY9/u293nFS5KU0bKyAkd278YRN7zIGz1eYhIdaTnj76wfuDcr3rnTm7BqhzK8JEkC8nKyOPGIPhxw/TCe3/cJZpe1oemEm1g9sCOrxv8DykqTnqhqwPCSJOlf1MrPod9JJ7P7taN4ere7WFxSi4Yjrqbor11ZN2UwlJYkPVFVmM/xkiTpe3y1ehMjX36UAxc+SPusJazNacSmfc6jxWGXQe0mSc9TGvq+53gZXpIkVcDnReuZ+OazFM4fxMHhQ0rIZelOx9LyqKvJbb1v0vOURgwvSZJ2kPWbtzJizFiypj7CEVtHUTNsYXHdztQ55HLqdT4JsnOSnqiEGV6SJO1g5eWRiXM+Y8nIh+i+6mV2CkWsymnGpn0voNVhlxBqNkx6ohKSVuEVQtgJeBJoDpQDD8cY7/q+32N4SZLS2Rcr1jHlracpXDCIbsxmC/ksaXMCrY6+ivyWHZKepxRLt/BqAbSIMU4PIdQB3gdOijHO+a7fY3hJkqqCjVtKeXfsu2RPeYjeJaOpEbaysN7+1Dl0AA33PQ6yspOeqBRIq/D6XwNCeA24N8b4znc9xvCSJFUlMUamzJ7HkpEPctCqV2gRVlGU24JNnS6iTZ/+hIL6SU9UJUrb8AohFAJjgQ4xxnXfeN/FwMUAbdq06bpw4cKU75Mk6adaVLSWaW8+SeGCQXRmLpsoYEnbk2jT92rym++e9DxVgrQMrxBCbWAM8JcY48vf91iveEmSqrqvS8oYO/ptsqc+xCElY8kLZSyodxB1ew+gyT59Ict7mlcXaRdeIYRc4A3grRjj337o8YaXJKm6iDEyfc5clo68nwOKX6VJWMvS3DZ8ve9FtDv8IoIvzF3lpVV4hRAC8E9gVYzxqor8HsNLklQdfbVyDdOHP067BYPYmwVsoBZfFp5Ku2OuoqDpLknP038o3cKrBzAO+Ihtt5MAuDHGOOy7fo/hJUmqzjaXlDJhzHBypz5E9y0TyAqRefV7Uq/3AJrvcwSEkPRE/QhpFV7/CcNLkpQJYox89PEclo64n/2LX6Vh2MCi3HZ83fkX7NrnQrLyayY9URVgeEmSVMUsL17NjGGP0m7BIHZjIWtDHRYW9mOXY66kVpO2Sc/T9zC8JEmqorZsLWXy6NfJnfow3ba8RyTwSYPeNDhsAK069vLbkGnI8JIkqRqYM+cjlo24l67Fr1MvbOTzvPZ83fkX7NHnfLLyaiQ9T9sZXpIkVSNFq1Yxa+hDFC4YRDuWUBzqs7DwdHY95grqNmmd9LyMZ3hJklQNlWwt4/13XyZ32sPsVzKFrTGb2Q0Pp+FhA2jTsWfS8zKW4SVJUjX36ZwZLB9xN52Lh1I7bObTvD35usvFdOhzNtm5eUnPyyiGlyRJGWJV8UpmD7ufdguepjXLWEEjvmh3Brsfezn1GrdMel5GMLwkScowpaWlzBj1PLnTHqFTyXS2xFxmNjySRn2uZOcOByQ9r1ozvCRJymAL5kyjaMTddCoeTkEoYXZeR7Z0uZh9+vycnNzcpOdVO4aXJElibfFyPh52H4ULBtOcIpbShM92PpO9jr2cBo2aJj2v2jC8JEnS/1NWupWPRj5D/vsPs2fJR2yK+cxoeDQNe1/O7h33J3hT1p/E8JIkSd9q4exJrBxxFx1WvUN+2Mr8rEK+an0srXqewy7t90x6XpVkeEmSpO+1rngp80Y8Tr0Fr7FryScAzM7eg6LC42l3yFm0bdsu4YVVh+ElSZIqrHjRXBaOfYpGnw+hbekXlMXAzNxOrNnlBNr3OpPWLVokPTGtGV6SJOk/UrTgAxaPe4oWX75B8/JlbIk5zMjfj43tT2Sv3v1o3rhx0hPTjuElSZJ+mhhZ/sl7LJvwFK2WDKdxXMXGmM8HBQdRsufJdDz0VJrUr5P0yrRgeEmSpB2nvIyvZo6keNJg2i4fQd24nrWxJtNrHULscCqdex5PgzoFSa9MjOElSZIqR9lWFk8byrppz1JY9C412cyKWJ8ZdXuR06kf+x18BHULMuu1Ig0vSZJU6WLJRhZNfpVN05+n3eoJ5LOVRbEJHzU4nILO/eh2QE9q1aj+d8o3vCRJUkrFr9ewcOILlH74IoXrppBDOfNiaz5pdAT1up1Bt677UyM3O+mZlcLwkiRJiSlfX8TC8YMJs16icOOHAMyKOzO/2dE0PuAMunXqSF5OVsIrdxzDS5IkpYXSVV+yaPxgcj9+mdZfz6U8BqaHPfiy5TE0P/B0uu3dnpzsqh1hhpckSUo7Jcs/Zcm4QdT89FWalXzJ1pjN5LAPS3c6hrbd+9F197ZkZ1W91400vCRJUvqKkS1LZrJk3CDqLRhCo9LlbI65TMjqysrC42nf4xT2bdeCrCoSYYaXJEmqGmJk8+fvsXT8UzRaOJS6ZWtYHwsYn92NtbueyF49TqDjTo0JIX0jzPCSJElVT1kpm+aNZsXEp2m66C1qxo0UxzqMzz2YTbufxL4H92WPFvXSLsIML0mSVLWVbmHj7DcpnvwMzZaOIj9uYWlsyPj8nmzd8xS6de/Drs3S4yWLDC9JklR9bNnA+pmvs3bKMzQvmkAOpXxe3oz3avaGDqfR46CDadOoZmLzDC9JklQ9bVrF2g9eYeP7z9Fs1VSyKefj8jZMrd2b3E79OPSArrSsn9rXjTS8JElS9bd+OWumPcfmD16g+bqZALxf3p4Z9fpQu/Np9N6/I03r1Kj0GYaXJEnKLKsXsnrKM5TNfJHGG+dRFgOT4l6UH3oDPfscX6kf+vvCK6dSP7IkSVISGrSlwVHXw1HXw4pPWDtlMHvNepmtzWolOssrXpIkKTP8T/NU8u0nvOIlSZKUBvf7qtqvQilJklSFGF6SJEkpYnhJkiSliOElSZKUIoaXJElSihhekiRJKWJ4SZIkpYjhJUmSlCKGlyRJUooYXpIkSSlieEmSJKWI4SVJkpQihpckSVKKGF6SJEkpYnhJkiSliOElSZKUIoaXJElSihhekiRJKWJ4SZIkpYjhJUmSlCKGlyRJUooYXpIkSSkSYoxJb/hBIYQiYGElf5jGwMpK/hj68TyX9OOZpCfPJf14JukpFefSNsbY5NveUSXCKxVCCNNijPslvUP/znNJP55JevJc0o9nkp6SPhe/1ShJkpQihpckSVKKGF7/38NJD9C38lzSj2eSnjyX9OOZpKdEz8XneEmSJKWIV7wkSZJSxPCSJElKkYwLrxDC0SGEuSGE+SGE67/l/SGEcPf2988MIXRJYmcmqcCZnLX9LGaGECaGEDolsTPT/NC5/Mvj9g8hlIUQTkvlvkxUkTMJIfQKIcwIIcwOIYxJ9cZMVIF/w+qFEF4PIXy4/VwuSGJnJgkh/COEsCKEMOs73p/c5/oYY8a8AdnAAmBnIA/4ENjrG485BhgOBOBAYHLSu6vzWwXPpDvQYPt/9/VM0uNc/uVxo4BhwGlJ767ObxX8u1IfmAO02f7rpknvru5vFTyXG4GB2/+7CbAKyEt6e3V+Aw4BugCzvuP9iX2uz7QrXt2A+THGz2KMJcCzwInfeMyJwJNxm0lA/RBCi1QPzSA/eCYxxokxxtXbfzkJaJ3ijZmoIn9XAAYALwErUjkuQ1XkTM4EXo4xfgkQY/RcKl9FziUCdUIIAajNtvAqTe3MzBJjHMu2P+fvktjn+kwLr1bAon/59eLt/+/HPkY7zo/9876IbV+lqHL94LmEEFoBJwMPpnBXJqvI35XdgAYhhNEhhPdDCOembF3mqsi53AvsCXwFfARcGWMsT808fYfEPtfnpOKDpJHwLf/vm/fTqMhjtONU+M87hNCbbeHVo1IXCSp2Ln8Hrosxlm37Ql6VrCJnkgN0BfoABcB7IYRJMcZPK3tcBqvIuRwFzAAOA3YB3gkhjIsxrqvscfpOiX2uz7TwWgzs9C+/bs22r0B+7GO041TozzuEsA/wKNA3xlicom2ZrCLnsh/w7PboagwcE0IojTG+mpqJGaei/36tjDFuBDaGEMYCnQDDq/JU5FwuAG6L255cND+E8DmwBzAlNRP1LRL7XJ9p32qcCrQPIbQLIeQBZwBDvvGYIcC523/i4UBgbYxxaaqHZpAfPJMQQhvgZeAcv3JPmR88lxhjuxhjYYyxEHgRuMzoqlQV+ffrNaBnCCEnhFATOAD4OMU7M01FzuVLtl2FJITQDNgd+CylK/VNiX2uz6grXjHG0hDC5cBbbPtJlH/EGGeHEH65/f0Psu2ns44B5gOb2PaViipJBc/kD0Aj4P7tV1dKY4KvLJ8JKnguSqGKnEmM8eMQwpvATKAceDTG+K0/Tq8do4J/V/4EPBFC+Iht3+K6Lsa4MrHRGSCE8AzQC2gcQlgM/BHIheQ/1/uSQZIkSSmSad9qlCRJSozhJUmSlCKGlyRJUooYXpIkSSlieEmSJKWI4SVJkpQihpckSVKKGF6SMkoIYf8QwswQQo0QQq0QwuwQQoekd0nKDN5AVVLGCSH8GajBtheSXhxjvDXhSZIyhOElKeNsf029qcBmoHuMsSzhSZIyhN9qlJSJGgK1gTpsu/IlSSnhFS9JGSeEMAR4FmgHtIgxXp7wJEkZIifpAZKUSiGEc4HSGOPgEEI2MDGEcFiMcVTS2yRVf17xkiRJShGf4yVJkpQihpckSVKKGF6SJEkpYnhJkiSliOElSZKUIoaXJElSihhekiRJKfJ/AST8L0Oa7aCKAAAAAElFTkSuQmCC\n",
+      "text/plain": [
+       "
    "
+      ]
+     },
+     "metadata": {
+      "filenames": {
+       "image/png": "/Users/hjensen/Teaching/FYS-STK4150/doc/src/LectureNotes/_build/jupyter_execute/chapter10_59_2.png"
+      },
+      "needs_background": "light"
+     },
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "import autograd.numpy as np\n",
+    "from autograd import grad, elementwise_grad\n",
+    "import autograd.numpy.random as npr\n",
+    "from matplotlib import pyplot as plt\n",
+    "\n",
+    "def sigmoid(z):\n",
+    "    return 1/(1 + np.exp(-z))\n",
+    "\n",
+    "# Assuming one input, hidden, and output layer\n",
+    "def neural_network(params, x):\n",
+    "\n",
+    "    # Find the weights (including and biases) for the hidden and output layer.\n",
+    "    # Assume that params is a list of parameters for each layer.\n",
+    "    # The biases are the first element for each array in params,\n",
+    "    # and the weights are the remaning elements in each array in params.\n",
+    "\n",
+    "    w_hidden = params[0]\n",
+    "    w_output = params[1]\n",
+    "\n",
+    "    # Assumes input x being an one-dimensional array\n",
+    "    num_values = np.size(x)\n",
+    "    x = x.reshape(-1, num_values)\n",
+    "\n",
+    "    # Assume that the input layer does nothing to the input x\n",
+    "    x_input = x\n",
+    "\n",
+    "    ## Hidden layer:\n",
+    "\n",
+    "    # Add a row of ones to include bias\n",
+    "    x_input = np.concatenate((np.ones((1,num_values)), x_input ), axis = 0)\n",
+    "\n",
+    "    z_hidden = np.matmul(w_hidden, x_input)\n",
+    "    x_hidden = sigmoid(z_hidden)\n",
+    "\n",
+    "    ## Output layer:\n",
+    "\n",
+    "    # Include bias:\n",
+    "    x_hidden = np.concatenate((np.ones((1,num_values)), x_hidden ), axis = 0)\n",
+    "\n",
+    "    z_output = np.matmul(w_output, x_hidden)\n",
+    "    x_output = z_output\n",
+    "\n",
+    "    return x_output\n",
+    "\n",
+    "# The trial solution using the deep neural network:\n",
+    "def g_trial(x,params, g0 = 10):\n",
+    "    return g0 + x*neural_network(params,x)\n",
+    "\n",
+    "# The right side of the ODE:\n",
+    "def g(x, g_trial, gamma = 2):\n",
+    "    return -gamma*g_trial\n",
+    "\n",
+    "# The cost function:\n",
+    "def cost_function(P, x):\n",
+    "\n",
+    "    # Evaluate the trial function with the current parameters P\n",
+    "    g_t = g_trial(x,P)\n",
+    "\n",
+    "    # Find the derivative w.r.t x of the neural network\n",
+    "    d_net_out = elementwise_grad(neural_network,1)(P,x)\n",
+    "\n",
+    "    # Find the derivative w.r.t x of the trial function\n",
+    "    d_g_t = elementwise_grad(g_trial,0)(x,P)\n",
+    "\n",
+    "    # The right side of the ODE\n",
+    "    func = g(x, g_t)\n",
+    "\n",
+    "    err_sqr = (d_g_t - func)**2\n",
+    "    cost_sum = np.sum(err_sqr)\n",
+    "\n",
+    "    return cost_sum / np.size(err_sqr)\n",
+    "\n",
+    "# Solve the exponential decay ODE using neural network with one input, hidden, and output layer\n",
+    "def solve_ode_neural_network(x, num_neurons_hidden, num_iter, lmb):\n",
+    "    ## Set up initial weights and biases\n",
+    "\n",
+    "    # For the hidden layer\n",
+    "    p0 = npr.randn(num_neurons_hidden, 2 )\n",
+    "\n",
+    "    # For the output layer\n",
+    "    p1 = npr.randn(1, num_neurons_hidden + 1 ) # +1 since bias is included\n",
+    "\n",
+    "    P = [p0, p1]\n",
+    "\n",
+    "    print('Initial cost: %g'%cost_function(P, x))\n",
+    "\n",
+    "    ## Start finding the optimal weights using gradient descent\n",
+    "\n",
+    "    # Find the Python function that represents the gradient of the cost function\n",
+    "    # w.r.t the 0-th input argument -- that is the weights and biases in the hidden and output layer\n",
+    "    cost_function_grad = grad(cost_function,0)\n",
+    "\n",
+    "    # Let the update be done num_iter times\n",
+    "    for i in range(num_iter):\n",
+    "        # Evaluate the gradient at the current weights and biases in P.\n",
+    "        # The cost_grad consist now of two arrays;\n",
+    "        # one for the gradient w.r.t P_hidden and\n",
+    "        # one for the gradient w.r.t P_output\n",
+    "        cost_grad =  cost_function_grad(P, x)\n",
+    "\n",
+    "        P[0] = P[0] - lmb * cost_grad[0]\n",
+    "        P[1] = P[1] - lmb * cost_grad[1]\n",
+    "\n",
+    "    print('Final cost: %g'%cost_function(P, x))\n",
+    "\n",
+    "    return P\n",
+    "\n",
+    "def g_analytic(x, gamma = 2, g0 = 10):\n",
+    "    return g0*np.exp(-gamma*x)\n",
+    "\n",
+    "# Solve the given problem\n",
+    "if __name__ == '__main__':\n",
+    "    # Set seed such that the weight are initialized\n",
+    "    # with same weights and biases for every run.\n",
+    "    npr.seed(15)\n",
+    "\n",
+    "    ## Decide the vales of arguments to the function to solve\n",
+    "    N = 10\n",
+    "    x = np.linspace(0, 1, N)\n",
+    "\n",
+    "    ## Set up the initial parameters\n",
+    "    num_hidden_neurons = 10\n",
+    "    num_iter = 10000\n",
+    "    lmb = 0.001\n",
+    "\n",
+    "    # Use the network\n",
+    "    P = solve_ode_neural_network(x, num_hidden_neurons, num_iter, lmb)\n",
+    "\n",
+    "    # Print the deviation from the trial solution and true solution\n",
+    "    res = g_trial(x,P)\n",
+    "    res_analytical = g_analytic(x)\n",
+    "\n",
+    "    print('Max absolute difference: %g'%np.max(np.abs(res - res_analytical)))\n",
+    "\n",
+    "    # Plot the results\n",
+    "    plt.figure(figsize=(10,10))\n",
+    "\n",
+    "    plt.title('Performance of neural network solving an ODE compared to the analytical solution')\n",
+    "    plt.plot(x, res_analytical)\n",
+    "    plt.plot(x, res[0,:])\n",
+    "    plt.legend(['analytical','nn'])\n",
+    "    plt.xlabel('x')\n",
+    "    plt.ylabel('g(x)')\n",
+    "    plt.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## The network with one input layer, specified number of hidden layers, and one output layer\n",
+    "\n",
+    "It is also possible to extend the construction of our network into a more general one, allowing the network to contain more than one hidden layers.\n",
+    "\n",
+    "The number of neurons within each hidden layer are given as a list of integers in the program below."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "import autograd.numpy as np\n",
+    "from autograd import grad, elementwise_grad\n",
+    "import autograd.numpy.random as npr\n",
+    "from matplotlib import pyplot as plt\n",
+    "\n",
+    "def sigmoid(z):\n",
+    "    return 1/(1 + np.exp(-z))\n",
+    "\n",
+    "# The neural network with one input layer and one output layer,\n",
+    "# but with number of hidden layers specified by the user.\n",
+    "def deep_neural_network(deep_params, x):\n",
+    "    # N_hidden is the number of hidden layers\n",
+    "\n",
+    "    N_hidden = np.size(deep_params) - 1 # -1 since params consists of\n",
+    "                                        # parameters to all the hidden\n",
+    "                                        # layers AND the output layer.\n",
+    "\n",
+    "    # Assumes input x being an one-dimensional array\n",
+    "    num_values = np.size(x)\n",
+    "    x = x.reshape(-1, num_values)\n",
+    "\n",
+    "    # Assume that the input layer does nothing to the input x\n",
+    "    x_input = x\n",
+    "\n",
+    "    # Due to multiple hidden layers, define a variable referencing to the\n",
+    "    # output of the previous layer:\n",
+    "    x_prev = x_input\n",
+    "\n",
+    "    ## Hidden layers:\n",
+    "\n",
+    "    for l in range(N_hidden):\n",
+    "        # From the list of parameters P; find the correct weigths and bias for this layer\n",
+    "        w_hidden = deep_params[l]\n",
+    "\n",
+    "        # Add a row of ones to include bias\n",
+    "        x_prev = np.concatenate((np.ones((1,num_values)), x_prev ), axis = 0)\n",
+    "\n",
+    "        z_hidden = np.matmul(w_hidden, x_prev)\n",
+    "        x_hidden = sigmoid(z_hidden)\n",
+    "\n",
+    "        # Update x_prev such that next layer can use the output from this layer\n",
+    "        x_prev = x_hidden\n",
+    "\n",
+    "    ## Output layer:\n",
+    "\n",
+    "    # Get the weights and bias for this layer\n",
+    "    w_output = deep_params[-1]\n",
+    "\n",
+    "    # Include bias:\n",
+    "    x_prev = np.concatenate((np.ones((1,num_values)), x_prev), axis = 0)\n",
+    "\n",
+    "    z_output = np.matmul(w_output, x_prev)\n",
+    "    x_output = z_output\n",
+    "\n",
+    "    return x_output\n",
+    "\n",
+    "# The trial solution using the deep neural network:\n",
+    "def g_trial_deep(x,params, g0 = 10):\n",
+    "    return g0 + x*deep_neural_network(params, x)\n",
+    "\n",
+    "# The right side of the ODE:\n",
+    "def g(x, g_trial, gamma = 2):\n",
+    "    return -gamma*g_trial\n",
+    "\n",
+    "# The same cost function as before, but calls deep_neural_network instead.\n",
+    "def cost_function_deep(P, x):\n",
+    "\n",
+    "    # Evaluate the trial function with the current parameters P\n",
+    "    g_t = g_trial_deep(x,P)\n",
+    "\n",
+    "    # Find the derivative w.r.t x of the neural network\n",
+    "    d_net_out = elementwise_grad(deep_neural_network,1)(P,x)\n",
+    "\n",
+    "    # Find the derivative w.r.t x of the trial function\n",
+    "    d_g_t = elementwise_grad(g_trial_deep,0)(x,P)\n",
+    "\n",
+    "    # The right side of the ODE\n",
+    "    func = g(x, g_t)\n",
+    "\n",
+    "    err_sqr = (d_g_t - func)**2\n",
+    "    cost_sum = np.sum(err_sqr)\n",
+    "\n",
+    "    return cost_sum / np.size(err_sqr)\n",
+    "\n",
+    "# Solve the exponential decay ODE using neural network with one input and one output layer,\n",
+    "# but with specified number of hidden layers from the user.\n",
+    "def solve_ode_deep_neural_network(x, num_neurons, num_iter, lmb):\n",
+    "    # num_hidden_neurons is now a list of number of neurons within each hidden layer\n",
+    "\n",
+    "    # The number of elements in the list num_hidden_neurons thus represents\n",
+    "    # the number of hidden layers.\n",
+    "\n",
+    "    # Find the number of hidden layers:\n",
+    "    N_hidden = np.size(num_neurons)\n",
+    "\n",
+    "    ## Set up initial weights and biases\n",
+    "\n",
+    "    # Initialize the list of parameters:\n",
+    "    P = [None]*(N_hidden + 1) # + 1 to include the output layer\n",
+    "\n",
+    "    P[0] = npr.randn(num_neurons[0], 2 )\n",
+    "    for l in range(1,N_hidden):\n",
+    "        P[l] = npr.randn(num_neurons[l], num_neurons[l-1] + 1) # +1 to include bias\n",
+    "\n",
+    "    # For the output layer\n",
+    "    P[-1] = npr.randn(1, num_neurons[-1] + 1 ) # +1 since bias is included\n",
+    "\n",
+    "    print('Initial cost: %g'%cost_function_deep(P, x))\n",
+    "\n",
+    "    ## Start finding the optimal weights using gradient descent\n",
+    "\n",
+    "    # Find the Python function that represents the gradient of the cost function\n",
+    "    # w.r.t the 0-th input argument -- that is the weights and biases in the hidden and output layer\n",
+    "    cost_function_deep_grad = grad(cost_function_deep,0)\n",
+    "\n",
+    "    # Let the update be done num_iter times\n",
+    "    for i in range(num_iter):\n",
+    "        # Evaluate the gradient at the current weights and biases in P.\n",
+    "        # The cost_grad consist now of N_hidden + 1 arrays; the gradient w.r.t the weights and biases\n",
+    "        # in the hidden layers and output layers evaluated at x.\n",
+    "        cost_deep_grad =  cost_function_deep_grad(P, x)\n",
+    "\n",
+    "        for l in range(N_hidden+1):\n",
+    "            P[l] = P[l] - lmb * cost_deep_grad[l]\n",
+    "\n",
+    "    print('Final cost: %g'%cost_function_deep(P, x))\n",
+    "\n",
+    "    return P\n",
+    "\n",
+    "def g_analytic(x, gamma = 2, g0 = 10):\n",
+    "    return g0*np.exp(-gamma*x)\n",
+    "\n",
+    "# Solve the given problem\n",
+    "if __name__ == '__main__':\n",
+    "    npr.seed(15)\n",
+    "\n",
+    "    ## Decide the vales of arguments to the function to solve\n",
+    "    N = 10\n",
+    "    x = np.linspace(0, 1, N)\n",
+    "\n",
+    "    ## Set up the initial parameters\n",
+    "    num_hidden_neurons = np.array([10,10])\n",
+    "    num_iter = 10000\n",
+    "    lmb = 0.001\n",
+    "\n",
+    "    P = solve_ode_deep_neural_network(x, num_hidden_neurons, num_iter, lmb)\n",
+    "\n",
+    "    res = g_trial_deep(x,P)\n",
+    "    res_analytical = g_analytic(x)\n",
+    "\n",
+    "    plt.figure(figsize=(10,10))\n",
+    "\n",
+    "    plt.title('Performance of a deep neural network solving an ODE compared to the analytical solution')\n",
+    "    plt.plot(x, res_analytical)\n",
+    "    plt.plot(x, res[0,:])\n",
+    "    plt.legend(['analytical','dnn'])\n",
+    "    plt.ylabel('g(x)')\n",
+    "    plt.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Example: Population growth\n",
+    "\n",
+    "A logistic model of population growth assumes that a population converges toward an equilibrium.\n",
+    "The population growth can be modeled by"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation} \\label{log} \\tag{10}\n",
+    "\tg'(t) = \\alpha g(t)(A - g(t))\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "where $g(t)$ is the population density at time $t$, $\\alpha > 0$ the growth rate and $A > 0$ is the maximum population number in the environment.\n",
+    "Also, at $t = 0$ the population has the size $g(0) = g_0$, where $g_0$ is some chosen constant.\n",
+    "\n",
+    "In this example, similar network as for the exponential decay using Autograd has been used to solve the equation. However, as the implementation might suffer from e.g numerical instability\n",
+    "and high execution time (this might be more apparent in the examples solving PDEs),\n",
+    "using a library like  TensorFlow is recommended.\n",
+    "Here, we stay with a more simple approach and implement for comparison, the simple forward Euler method.\n",
+    "\n",
+    "\n",
+    "## Setting up the problem\n",
+    "\n",
+    "Here, we will model a population $g(t)$ in an environment having carrying capacity $A$.\n",
+    "The population follows the model"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation} \\label{solveode_population} \\tag{11}\n",
+    "g'(t) = \\alpha g(t)(A - g(t))\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "where $g(0) = g_0$.\n",
+    "\n",
+    "In this example, we let $\\alpha = 2$, $A = 1$, and $g_0 = 1.2$.\n",
+    "\n",
+    "\n",
+    "## The trial solution\n",
+    "\n",
+    "We will get a slightly different trial solution, as the boundary conditions are different\n",
+    "compared to the case for exponential decay.\n",
+    "\n",
+    "A possible trial solution satisfying the condition $g(0) = g_0$ could be\n",
+    "\n",
+    "$$\n",
+    "h_1(t) = g_0 + t \\cdot N(t,P)\n",
+    "$$\n",
+    "\n",
+    "with $N(t,P)$ being the output from the neural network with weights and biases for each layer collected in the set $P$.\n",
+    "\n",
+    "The analytical solution is\n",
+    "\n",
+    "$$\n",
+    "g(t) = \\frac{Ag_0}{g_0 + (A - g_0)\\exp(-\\alpha A t)}\n",
+    "$$\n",
+    "\n",
+    "\n",
+    "## The program using Autograd\n",
+    "\n",
+    "The network will be the similar as for the exponential decay example, but with some small modifications for our problem."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "import autograd.numpy as np\n",
+    "from autograd import grad, elementwise_grad\n",
+    "import autograd.numpy.random as npr\n",
+    "from matplotlib import pyplot as plt\n",
+    "\n",
+    "def sigmoid(z):\n",
+    "    return 1/(1 + np.exp(-z))\n",
+    "\n",
+    "# Function to get the parameters.\n",
+    "# Done such that one can easily change the paramaters after one's liking.\n",
+    "def get_parameters():\n",
+    "    alpha = 2\n",
+    "    A = 1\n",
+    "    g0 = 1.2\n",
+    "    return alpha, A, g0\n",
+    "\n",
+    "def deep_neural_network(P, x):\n",
+    "    # N_hidden is the number of hidden layers\n",
+    "    N_hidden = np.size(P) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n",
+    "\n",
+    "    # Assumes input x being an one-dimensional array\n",
+    "    num_values = np.size(x)\n",
+    "    x = x.reshape(-1, num_values)\n",
+    "\n",
+    "    # Assume that the input layer does nothing to the input x\n",
+    "    x_input = x\n",
+    "\n",
+    "    # Due to multiple hidden layers, define a variable referencing to the\n",
+    "    # output of the previous layer:\n",
+    "    x_prev = x_input\n",
+    "\n",
+    "    ## Hidden layers:\n",
+    "\n",
+    "    for l in range(N_hidden):\n",
+    "        # From the list of parameters P; find the correct weigths and bias for this layer\n",
+    "        w_hidden = P[l]\n",
+    "\n",
+    "        # Add a row of ones to include bias\n",
+    "        x_prev = np.concatenate((np.ones((1,num_values)), x_prev ), axis = 0)\n",
+    "\n",
+    "        z_hidden = np.matmul(w_hidden, x_prev)\n",
+    "        x_hidden = sigmoid(z_hidden)\n",
+    "\n",
+    "        # Update x_prev such that next layer can use the output from this layer\n",
+    "        x_prev = x_hidden\n",
+    "\n",
+    "    ## Output layer:\n",
+    "\n",
+    "    # Get the weights and bias for this layer\n",
+    "    w_output = P[-1]\n",
+    "\n",
+    "    # Include bias:\n",
+    "    x_prev = np.concatenate((np.ones((1,num_values)), x_prev), axis = 0)\n",
+    "\n",
+    "    z_output = np.matmul(w_output, x_prev)\n",
+    "    x_output = z_output\n",
+    "\n",
+    "    return x_output\n",
+    "\n",
+    "\n",
+    "def cost_function_deep(P, x):\n",
+    "\n",
+    "    # Evaluate the trial function with the current parameters P\n",
+    "    g_t = g_trial_deep(x,P)\n",
+    "\n",
+    "    # Find the derivative w.r.t x of the trial function\n",
+    "    d_g_t = elementwise_grad(g_trial_deep,0)(x,P)\n",
+    "\n",
+    "    # The right side of the ODE\n",
+    "    func = f(x, g_t)\n",
+    "\n",
+    "    err_sqr = (d_g_t - func)**2\n",
+    "    cost_sum = np.sum(err_sqr)\n",
+    "\n",
+    "    return cost_sum / np.size(err_sqr)\n",
+    "\n",
+    "# The right side of the ODE:\n",
+    "def f(x, g_trial):\n",
+    "    alpha,A, g0 = get_parameters()\n",
+    "    return alpha*g_trial*(A - g_trial)\n",
+    "\n",
+    "# The trial solution using the deep neural network:\n",
+    "def g_trial_deep(x, params):\n",
+    "    alpha,A, g0 = get_parameters()\n",
+    "    return g0 + x*deep_neural_network(params,x)\n",
+    "\n",
+    "# The analytical solution:\n",
+    "def g_analytic(t):\n",
+    "    alpha,A, g0 = get_parameters()\n",
+    "    return A*g0/(g0 + (A - g0)*np.exp(-alpha*A*t))\n",
+    "\n",
+    "def solve_ode_deep_neural_network(x, num_neurons, num_iter, lmb):\n",
+    "    # num_hidden_neurons is now a list of number of neurons within each hidden layer\n",
+    "\n",
+    "    # Find the number of hidden layers:\n",
+    "    N_hidden = np.size(num_neurons)\n",
+    "\n",
+    "    ## Set up initial weigths and biases\n",
+    "\n",
+    "    # Initialize the list of parameters:\n",
+    "    P = [None]*(N_hidden + 1) # + 1 to include the output layer\n",
+    "\n",
+    "    P[0] = npr.randn(num_neurons[0], 2 )\n",
+    "    for l in range(1,N_hidden):\n",
+    "        P[l] = npr.randn(num_neurons[l], num_neurons[l-1] + 1) # +1 to include bias\n",
+    "\n",
+    "    # For the output layer\n",
+    "    P[-1] = npr.randn(1, num_neurons[-1] + 1 ) # +1 since bias is included\n",
+    "\n",
+    "    print('Initial cost: %g'%cost_function_deep(P, x))\n",
+    "\n",
+    "    ## Start finding the optimal weigths using gradient descent\n",
+    "\n",
+    "    # Find the Python function that represents the gradient of the cost function\n",
+    "    # w.r.t the 0-th input argument -- that is the weights and biases in the hidden and output layer\n",
+    "    cost_function_deep_grad = grad(cost_function_deep,0)\n",
+    "\n",
+    "    # Let the update be done num_iter times\n",
+    "    for i in range(num_iter):\n",
+    "        # Evaluate the gradient at the current weights and biases in P.\n",
+    "        # The cost_grad consist now of N_hidden + 1 arrays; the gradient w.r.t the weights and biases\n",
+    "        # in the hidden layers and output layers evaluated at x.\n",
+    "        cost_deep_grad =  cost_function_deep_grad(P, x)\n",
+    "\n",
+    "        for l in range(N_hidden+1):\n",
+    "            P[l] = P[l] - lmb * cost_deep_grad[l]\n",
+    "\n",
+    "    print('Final cost: %g'%cost_function_deep(P, x))\n",
+    "\n",
+    "    return P\n",
+    "\n",
+    "if __name__ == '__main__':\n",
+    "    npr.seed(4155)\n",
+    "\n",
+    "    ## Decide the vales of arguments to the function to solve\n",
+    "    Nt = 10\n",
+    "    T = 1\n",
+    "    t = np.linspace(0,T, Nt)\n",
+    "\n",
+    "    ## Set up the initial parameters\n",
+    "    num_hidden_neurons = [100, 50, 25]\n",
+    "    num_iter = 1000\n",
+    "    lmb = 1e-3\n",
+    "\n",
+    "    P = solve_ode_deep_neural_network(t, num_hidden_neurons, num_iter, lmb)\n",
+    "\n",
+    "    g_dnn_ag = g_trial_deep(t,P)\n",
+    "    g_analytical = g_analytic(t)\n",
+    "\n",
+    "    # Find the maximum absolute difference between the solutons:\n",
+    "    diff_ag = np.max(np.abs(g_dnn_ag - g_analytical))\n",
+    "    print(\"The max absolute difference between the solutions is: %g\"%diff_ag)\n",
+    "\n",
+    "    plt.figure(figsize=(10,10))\n",
+    "\n",
+    "    plt.title('Performance of neural network solving an ODE compared to the analytical solution')\n",
+    "    plt.plot(t, g_analytical)\n",
+    "    plt.plot(t, g_dnn_ag[0,:])\n",
+    "    plt.legend(['analytical','nn'])\n",
+    "    plt.xlabel('t')\n",
+    "    plt.ylabel('g(t)')\n",
+    "\n",
+    "    plt.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Using forward Euler to solve the ODE\n",
+    "\n",
+    "A straightforward way of solving an ODE numerically, is to use Euler's method.\n",
+    "\n",
+    "Euler's method uses Taylor series to approximate the value at a function $f$ at a step $\\Delta x$ from $x$:\n",
+    "\n",
+    "$$\n",
+    "f(x + \\Delta x) \\approx f(x) + \\Delta x f'(x)\n",
+    "$$\n",
+    "\n",
+    "In our case, using Euler's method to approximate the value of $g$ at a step $\\Delta t$ from $t$ yields"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\begin{aligned}\n",
+    "  g(t + \\Delta t) &\\approx g(t) + \\Delta t g'(t) \\\\\n",
+    "  &= g(t) + \\Delta t \\big(\\alpha g(t)(A - g(t))\\big)\n",
+    "\\end{aligned}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "along with the condition that $g(0) = g_0$.\n",
+    "\n",
+    "Let $t_i = i \\cdot \\Delta t$ where $\\Delta t = \\frac{T}{N_t-1}$ where $T$ is the final time our solver must solve for and $N_t$ the number of values for $t \\in [0, T]$ for $i = 0, \\dots, N_t-1$.\n",
+    "\n",
+    "For $i \\geq 1$, we have that"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\begin{aligned}\n",
+    "t_i &= i\\Delta t \\\\\n",
+    "&= (i - 1)\\Delta t + \\Delta t \\\\\n",
+    "&= t_{i-1} + \\Delta t\n",
+    "\\end{aligned}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Now, if $g_i = g(t_i)$ then"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation}\n",
+    "  \\begin{aligned}\n",
+    "  g_i &= g(t_i) \\\\\n",
+    "  &= g(t_{i-1} + \\Delta t) \\\\\n",
+    "  &\\approx g(t_{i-1}) + \\Delta t \\big(\\alpha g(t_{i-1})(A - g(t_{i-1}))\\big) \\\\\n",
+    "  &= g_{i-1} + \\Delta t \\big(\\alpha g_{i-1}(A - g_{i-1})\\big)\n",
+    "  \\end{aligned}\n",
+    "\\end{equation} \\label{odenum} \\tag{12}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "for $i \\geq 1$ and $g_0 = g(t_0) = g(0) = g_0$.\n",
+    "\n",
+    "Equation ([12](#odenum)) could be implemented in the following way,\n",
+    "extending the program that uses the network using Autograd:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "# Assume that all function definitions from the example program using Autograd\n",
+    "# are located here.\n",
+    "\n",
+    "if __name__ == '__main__':\n",
+    "    npr.seed(4155)\n",
+    "\n",
+    "    ## Decide the vales of arguments to the function to solve\n",
+    "    Nt = 10\n",
+    "    T = 1\n",
+    "    t = np.linspace(0,T, Nt)\n",
+    "\n",
+    "    ## Set up the initial parameters\n",
+    "    num_hidden_neurons = [100,50,25]\n",
+    "    num_iter = 1000\n",
+    "    lmb = 1e-3\n",
+    "\n",
+    "    P = solve_ode_deep_neural_network(t, num_hidden_neurons, num_iter, lmb)\n",
+    "\n",
+    "    g_dnn_ag = g_trial_deep(t,P)\n",
+    "    g_analytical = g_analytic(t)\n",
+    "\n",
+    "    # Find the maximum absolute difference between the solutons:\n",
+    "    diff_ag = np.max(np.abs(g_dnn_ag - g_analytical))\n",
+    "    print(\"The max absolute difference between the solutions is: %g\"%diff_ag)\n",
+    "\n",
+    "    plt.figure(figsize=(10,10))\n",
+    "\n",
+    "    plt.title('Performance of neural network solving an ODE compared to the analytical solution')\n",
+    "    plt.plot(t, g_analytical)\n",
+    "    plt.plot(t, g_dnn_ag[0,:])\n",
+    "    plt.legend(['analytical','nn'])\n",
+    "    plt.xlabel('t')\n",
+    "    plt.ylabel('g(t)')\n",
+    "\n",
+    "    ## Find an approximation to the funtion using forward Euler\n",
+    "\n",
+    "    alpha, A, g0 = get_parameters()\n",
+    "    dt = T/(Nt - 1)\n",
+    "\n",
+    "    # Perform forward Euler to solve the ODE\n",
+    "    g_euler = np.zeros(Nt)\n",
+    "    g_euler[0] = g0\n",
+    "\n",
+    "    for i in range(1,Nt):\n",
+    "        g_euler[i] = g_euler[i-1] + dt*(alpha*g_euler[i-1]*(A - g_euler[i-1]))\n",
+    "\n",
+    "    # Print the errors done by each method\n",
+    "    diff1 = np.max(np.abs(g_euler - g_analytical))\n",
+    "    diff2 = np.max(np.abs(g_dnn_ag[0,:] - g_analytical))\n",
+    "\n",
+    "    print('Max absolute difference between Euler method and analytical: %g'%diff1)\n",
+    "    print('Max absolute difference between deep neural network and analytical: %g'%diff2)\n",
+    "\n",
+    "    # Plot results\n",
+    "    plt.figure(figsize=(10,10))\n",
+    "\n",
+    "    plt.plot(t,g_euler)\n",
+    "    plt.plot(t,g_analytical)\n",
+    "    plt.plot(t,g_dnn_ag[0,:])\n",
+    "\n",
+    "    plt.legend(['euler','analytical','dnn'])\n",
+    "    plt.xlabel('Time t')\n",
+    "    plt.ylabel('g(t)')\n",
+    "\n",
+    "    plt.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Example: Solving the one dimensional Poisson equation\n",
+    "\n",
+    "The Poisson equation for $g(x)$ in one dimension is"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation} \\label{poisson} \\tag{13}\n",
+    "  -g''(x) = f(x)\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "where $f(x)$ is a given function for $x \\in (0,1)$.\n",
+    "\n",
+    "The conditions that $g(x)$ is chosen to fulfill, are"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\begin{align*}\n",
+    "  g(0) &= 0 \\\\\n",
+    "  g(1) &= 0\n",
+    "\\end{align*}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "This equation can be solved numerically using programs where e.g Autograd and TensorFlow are used.\n",
+    "The results from the networks can then be compared to the analytical solution.\n",
+    "In addition, it could be interesting to see how a typical method for numerically solving second order ODEs compares to the neural networks.\n",
+    "\n",
+    "\n",
+    "## The specific equation to solve for\n",
+    "\n",
+    "Here, the function $g(x)$ to solve for follows the equation"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "-g''(x) = f(x),\\qquad x \\in (0,1)\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "where $f(x)$ is a given function, along with the chosen conditions"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{aligned}\n",
+    "g(0) = g(1) = 0\n",
+    "\\end{aligned}\\label{cond} \\tag{14}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "In this example, we consider the case when $f(x) = (3x + x^2)\\exp(x)$.\n",
+    "\n",
+    "For this case, a possible trial solution satisfying the conditions could be"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "g_t(x) = x \\cdot (1-x) \\cdot N(P,x)\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The analytical solution for this problem is"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "g(x) = x(1 - x)\\exp(x)\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Solving the equation using Autograd"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "import autograd.numpy as np\n",
+    "from autograd import grad, elementwise_grad\n",
+    "import autograd.numpy.random as npr\n",
+    "from matplotlib import pyplot as plt\n",
+    "\n",
+    "def sigmoid(z):\n",
+    "    return 1/(1 + np.exp(-z))\n",
+    "\n",
+    "def deep_neural_network(deep_params, x):\n",
+    "    # N_hidden is the number of hidden layers\n",
+    "    N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n",
+    "\n",
+    "    # Assumes input x being an one-dimensional array\n",
+    "    num_values = np.size(x)\n",
+    "    x = x.reshape(-1, num_values)\n",
+    "\n",
+    "    # Assume that the input layer does nothing to the input x\n",
+    "    x_input = x\n",
+    "\n",
+    "    # Due to multiple hidden layers, define a variable referencing to the\n",
+    "    # output of the previous layer:\n",
+    "    x_prev = x_input\n",
+    "\n",
+    "    ## Hidden layers:\n",
+    "\n",
+    "    for l in range(N_hidden):\n",
+    "        # From the list of parameters P; find the correct weigths and bias for this layer\n",
+    "        w_hidden = deep_params[l]\n",
+    "\n",
+    "        # Add a row of ones to include bias\n",
+    "        x_prev = np.concatenate((np.ones((1,num_values)), x_prev ), axis = 0)\n",
+    "\n",
+    "        z_hidden = np.matmul(w_hidden, x_prev)\n",
+    "        x_hidden = sigmoid(z_hidden)\n",
+    "\n",
+    "        # Update x_prev such that next layer can use the output from this layer\n",
+    "        x_prev = x_hidden\n",
+    "\n",
+    "    ## Output layer:\n",
+    "\n",
+    "    # Get the weights and bias for this layer\n",
+    "    w_output = deep_params[-1]\n",
+    "\n",
+    "    # Include bias:\n",
+    "    x_prev = np.concatenate((np.ones((1,num_values)), x_prev), axis = 0)\n",
+    "\n",
+    "    z_output = np.matmul(w_output, x_prev)\n",
+    "    x_output = z_output\n",
+    "\n",
+    "    return x_output\n",
+    "\n",
+    "def solve_ode_deep_neural_network(x, num_neurons, num_iter, lmb):\n",
+    "    # num_hidden_neurons is now a list of number of neurons within each hidden layer\n",
+    "\n",
+    "    # Find the number of hidden layers:\n",
+    "    N_hidden = np.size(num_neurons)\n",
+    "\n",
+    "    ## Set up initial weigths and biases\n",
+    "\n",
+    "    # Initialize the list of parameters:\n",
+    "    P = [None]*(N_hidden + 1) # + 1 to include the output layer\n",
+    "\n",
+    "    P[0] = npr.randn(num_neurons[0], 2 )\n",
+    "    for l in range(1,N_hidden):\n",
+    "        P[l] = npr.randn(num_neurons[l], num_neurons[l-1] + 1) # +1 to include bias\n",
+    "\n",
+    "    # For the output layer\n",
+    "    P[-1] = npr.randn(1, num_neurons[-1] + 1 ) # +1 since bias is included\n",
+    "\n",
+    "    print('Initial cost: %g'%cost_function_deep(P, x))\n",
+    "\n",
+    "    ## Start finding the optimal weigths using gradient descent\n",
+    "\n",
+    "    # Find the Python function that represents the gradient of the cost function\n",
+    "    # w.r.t the 0-th input argument -- that is the weights and biases in the hidden and output layer\n",
+    "    cost_function_deep_grad = grad(cost_function_deep,0)\n",
+    "\n",
+    "    # Let the update be done num_iter times\n",
+    "    for i in range(num_iter):\n",
+    "        # Evaluate the gradient at the current weights and biases in P.\n",
+    "        # The cost_grad consist now of N_hidden + 1 arrays; the gradient w.r.t the weights and biases\n",
+    "        # in the hidden layers and output layers evaluated at x.\n",
+    "        cost_deep_grad =  cost_function_deep_grad(P, x)\n",
+    "\n",
+    "        for l in range(N_hidden+1):\n",
+    "            P[l] = P[l] - lmb * cost_deep_grad[l]\n",
+    "\n",
+    "    print('Final cost: %g'%cost_function_deep(P, x))\n",
+    "\n",
+    "    return P\n",
+    "\n",
+    "## Set up the cost function specified for this Poisson equation:\n",
+    "\n",
+    "# The right side of the ODE\n",
+    "def f(x):\n",
+    "    return (3*x + x**2)*np.exp(x)\n",
+    "\n",
+    "def cost_function_deep(P, x):\n",
+    "\n",
+    "    # Evaluate the trial function with the current parameters P\n",
+    "    g_t = g_trial_deep(x,P)\n",
+    "\n",
+    "    # Find the derivative w.r.t x of the trial function\n",
+    "    d2_g_t = elementwise_grad(elementwise_grad(g_trial_deep,0))(x,P)\n",
+    "\n",
+    "    right_side = f(x)\n",
+    "\n",
+    "    err_sqr = (-d2_g_t - right_side)**2\n",
+    "    cost_sum = np.sum(err_sqr)\n",
+    "\n",
+    "    return cost_sum/np.size(err_sqr)\n",
+    "\n",
+    "# The trial solution:\n",
+    "def g_trial_deep(x,P):\n",
+    "    return x*(1-x)*deep_neural_network(P,x)\n",
+    "\n",
+    "# The analytic solution;\n",
+    "def g_analytic(x):\n",
+    "    return x*(1-x)*np.exp(x)\n",
+    "\n",
+    "if __name__ == '__main__':\n",
+    "    npr.seed(4155)\n",
+    "\n",
+    "    ## Decide the vales of arguments to the function to solve\n",
+    "    Nx = 10\n",
+    "    x = np.linspace(0,1, Nx)\n",
+    "\n",
+    "    ## Set up the initial parameters\n",
+    "    num_hidden_neurons = [200,100]\n",
+    "    num_iter = 1000\n",
+    "    lmb = 1e-3\n",
+    "\n",
+    "    P = solve_ode_deep_neural_network(x, num_hidden_neurons, num_iter, lmb)\n",
+    "\n",
+    "    g_dnn_ag = g_trial_deep(x,P)\n",
+    "    g_analytical = g_analytic(x)\n",
+    "\n",
+    "    # Find the maximum absolute difference between the solutons:\n",
+    "    max_diff = np.max(np.abs(g_dnn_ag - g_analytical))\n",
+    "    print(\"The max absolute difference between the solutions is: %g\"%max_diff)\n",
+    "\n",
+    "    plt.figure(figsize=(10,10))\n",
+    "\n",
+    "    plt.title('Performance of neural network solving an ODE compared to the analytical solution')\n",
+    "    plt.plot(x, g_analytical)\n",
+    "    plt.plot(x, g_dnn_ag[0,:])\n",
+    "    plt.legend(['analytical','nn'])\n",
+    "    plt.xlabel('x')\n",
+    "    plt.ylabel('g(x)')\n",
+    "    plt.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Comparing with a numerical scheme\n",
+    "\n",
+    "The Poisson equation is possible to solve using Taylor series to approximate the second derivative.\n",
+    "\n",
+    "Using Taylor series, the second derivative can be expressed as\n",
+    "\n",
+    "$$\n",
+    "g''(x) = \\frac{g(x + \\Delta x) - 2g(x) + g(x-\\Delta x)}{\\Delta x^2} + E_{\\Delta x}(x)\n",
+    "$$\n",
+    "\n",
+    "where $\\Delta x$ is a small step size and $E_{\\Delta x}(x)$ being the error term.\n",
+    "\n",
+    "Looking away from the error terms gives an approximation to the second derivative:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation} \\label{approx} \\tag{15}\n",
+    "g''(x) \\approx \\frac{g(x + \\Delta x) - 2g(x) + g(x-\\Delta x)}{\\Delta x^2}\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "If $x_i = i \\Delta x = x_{i-1} + \\Delta x$ and $g_i = g(x_i)$ for $i = 1,\\dots N_x - 2$ with $N_x$ being the number of values for $x$, ([15](#approx)) becomes"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\begin{aligned}\n",
+    "g''(x_i) &\\approx \\frac{g(x_i + \\Delta x) - 2g(x_i) + g(x_i -\\Delta x)}{\\Delta x^2} \\\\\n",
+    "&= \\frac{g_{i+1} - 2g_i + g_{i-1}}{\\Delta x^2}\n",
+    "\\end{aligned}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Since we know from our problem that"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\begin{aligned}\n",
+    "-g''(x) &= f(x) \\\\\n",
+    "&= (3x + x^2)\\exp(x)\n",
+    "\\end{aligned}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "along with the conditions $g(0) = g(1) = 0$,\n",
+    "the following scheme can be used to find an approximate solution for $g(x)$ numerically:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation}\n",
+    "  \\begin{aligned}\n",
+    "  -\\Big( \\frac{g_{i+1} - 2g_i + g_{i-1}}{\\Delta x^2} \\Big) &= f(x_i) \\\\\n",
+    "  -g_{i+1} + 2g_i - g_{i-1} &= \\Delta x^2 f(x_i)\n",
+    "  \\end{aligned}\n",
+    "\\end{equation} \\label{odesys} \\tag{16}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "for $i = 1, \\dots, N_x - 2$ where $g_0 = g_{N_x - 1} = 0$ and $f(x_i) = (3x_i + x_i^2)\\exp(x_i)$, which is given for our specific problem.\n",
+    "\n",
+    "The equation can be rewritten into a matrix equation:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\begin{aligned}\n",
+    "\\begin{pmatrix}\n",
+    "2 & -1 & 0 & \\dots & 0 \\\\\n",
+    "-1 & 2 & -1 & \\dots & 0 \\\\\n",
+    "\\vdots & & \\ddots & & \\vdots \\\\\n",
+    "0 & \\dots & -1 & 2 & -1  \\\\\n",
+    "0 & \\dots & 0 & -1 & 2\\\\\n",
+    "\\end{pmatrix}\n",
+    "\\begin{pmatrix}\n",
+    "g_1 \\\\\n",
+    "g_2 \\\\\n",
+    "\\vdots \\\\\n",
+    "g_{N_x - 3} \\\\\n",
+    "g_{N_x - 2}\n",
+    "\\end{pmatrix}\n",
+    "&=\n",
+    "\\Delta x^2\n",
+    "\\begin{pmatrix}\n",
+    "f(x_1) \\\\\n",
+    "f(x_2) \\\\\n",
+    "\\vdots \\\\\n",
+    "f(x_{N_x - 3}) \\\\\n",
+    "f(x_{N_x - 2})\n",
+    "\\end{pmatrix} \\\\\n",
+    "\\boldsymbol{A}\\boldsymbol{g} &= \\boldsymbol{f},\n",
+    "\\end{aligned}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "which makes it possible to solve for the vector $\\boldsymbol{g}$.\n",
+    "\n",
+    "\n",
+    "## Setting up the code\n",
+    "\n",
+    "We can then compare the result from this numerical scheme with the output from our network using Autograd:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "import autograd.numpy as np\n",
+    "from autograd import grad, elementwise_grad\n",
+    "import autograd.numpy.random as npr\n",
+    "from matplotlib import pyplot as plt\n",
+    "\n",
+    "def sigmoid(z):\n",
+    "    return 1/(1 + np.exp(-z))\n",
+    "\n",
+    "def deep_neural_network(deep_params, x):\n",
+    "    # N_hidden is the number of hidden layers\n",
+    "    N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n",
+    "\n",
+    "    # Assumes input x being an one-dimensional array\n",
+    "    num_values = np.size(x)\n",
+    "    x = x.reshape(-1, num_values)\n",
+    "\n",
+    "    # Assume that the input layer does nothing to the input x\n",
+    "    x_input = x\n",
+    "\n",
+    "    # Due to multiple hidden layers, define a variable referencing to the\n",
+    "    # output of the previous layer:\n",
+    "    x_prev = x_input\n",
+    "\n",
+    "    ## Hidden layers:\n",
+    "\n",
+    "    for l in range(N_hidden):\n",
+    "        # From the list of parameters P; find the correct weigths and bias for this layer\n",
+    "        w_hidden = deep_params[l]\n",
+    "\n",
+    "        # Add a row of ones to include bias\n",
+    "        x_prev = np.concatenate((np.ones((1,num_values)), x_prev ), axis = 0)\n",
+    "\n",
+    "        z_hidden = np.matmul(w_hidden, x_prev)\n",
+    "        x_hidden = sigmoid(z_hidden)\n",
+    "\n",
+    "        # Update x_prev such that next layer can use the output from this layer\n",
+    "        x_prev = x_hidden\n",
+    "\n",
+    "    ## Output layer:\n",
+    "\n",
+    "    # Get the weights and bias for this layer\n",
+    "    w_output = deep_params[-1]\n",
+    "\n",
+    "    # Include bias:\n",
+    "    x_prev = np.concatenate((np.ones((1,num_values)), x_prev), axis = 0)\n",
+    "\n",
+    "    z_output = np.matmul(w_output, x_prev)\n",
+    "    x_output = z_output\n",
+    "\n",
+    "    return x_output\n",
+    "\n",
+    "def solve_ode_deep_neural_network(x, num_neurons, num_iter, lmb):\n",
+    "    # num_hidden_neurons is now a list of number of neurons within each hidden layer\n",
+    "\n",
+    "    # Find the number of hidden layers:\n",
+    "    N_hidden = np.size(num_neurons)\n",
+    "\n",
+    "    ## Set up initial weigths and biases\n",
+    "\n",
+    "    # Initialize the list of parameters:\n",
+    "    P = [None]*(N_hidden + 1) # + 1 to include the output layer\n",
+    "\n",
+    "    P[0] = npr.randn(num_neurons[0], 2 )\n",
+    "    for l in range(1,N_hidden):\n",
+    "        P[l] = npr.randn(num_neurons[l], num_neurons[l-1] + 1) # +1 to include bias\n",
+    "\n",
+    "    # For the output layer\n",
+    "    P[-1] = npr.randn(1, num_neurons[-1] + 1 ) # +1 since bias is included\n",
+    "\n",
+    "    print('Initial cost: %g'%cost_function_deep(P, x))\n",
+    "\n",
+    "    ## Start finding the optimal weigths using gradient descent\n",
+    "\n",
+    "    # Find the Python function that represents the gradient of the cost function\n",
+    "    # w.r.t the 0-th input argument -- that is the weights and biases in the hidden and output layer\n",
+    "    cost_function_deep_grad = grad(cost_function_deep,0)\n",
+    "\n",
+    "    # Let the update be done num_iter times\n",
+    "    for i in range(num_iter):\n",
+    "        # Evaluate the gradient at the current weights and biases in P.\n",
+    "        # The cost_grad consist now of N_hidden + 1 arrays; the gradient w.r.t the weights and biases\n",
+    "        # in the hidden layers and output layers evaluated at x.\n",
+    "        cost_deep_grad =  cost_function_deep_grad(P, x)\n",
+    "\n",
+    "        for l in range(N_hidden+1):\n",
+    "            P[l] = P[l] - lmb * cost_deep_grad[l]\n",
+    "\n",
+    "    print('Final cost: %g'%cost_function_deep(P, x))\n",
+    "\n",
+    "    return P\n",
+    "\n",
+    "## Set up the cost function specified for this Poisson equation:\n",
+    "\n",
+    "# The right side of the ODE\n",
+    "def f(x):\n",
+    "    return (3*x + x**2)*np.exp(x)\n",
+    "\n",
+    "def cost_function_deep(P, x):\n",
+    "\n",
+    "    # Evaluate the trial function with the current parameters P\n",
+    "    g_t = g_trial_deep(x,P)\n",
+    "\n",
+    "    # Find the derivative w.r.t x of the trial function\n",
+    "    d2_g_t = elementwise_grad(elementwise_grad(g_trial_deep,0))(x,P)\n",
+    "\n",
+    "    right_side = f(x)\n",
+    "\n",
+    "    err_sqr = (-d2_g_t - right_side)**2\n",
+    "    cost_sum = np.sum(err_sqr)\n",
+    "\n",
+    "    return cost_sum/np.size(err_sqr)\n",
+    "\n",
+    "# The trial solution:\n",
+    "def g_trial_deep(x,P):\n",
+    "    return x*(1-x)*deep_neural_network(P,x)\n",
+    "\n",
+    "# The analytic solution;\n",
+    "def g_analytic(x):\n",
+    "    return x*(1-x)*np.exp(x)\n",
+    "\n",
+    "if __name__ == '__main__':\n",
+    "    npr.seed(4155)\n",
+    "\n",
+    "    ## Decide the vales of arguments to the function to solve\n",
+    "    Nx = 10\n",
+    "    x = np.linspace(0,1, Nx)\n",
+    "\n",
+    "    ## Set up the initial parameters\n",
+    "    num_hidden_neurons = [200,100]\n",
+    "    num_iter = 1000\n",
+    "    lmb = 1e-3\n",
+    "\n",
+    "    P = solve_ode_deep_neural_network(x, num_hidden_neurons, num_iter, lmb)\n",
+    "\n",
+    "    g_dnn_ag = g_trial_deep(x,P)\n",
+    "    g_analytical = g_analytic(x)\n",
+    "\n",
+    "    # Find the maximum absolute difference between the solutons:\n",
+    "\n",
+    "    plt.figure(figsize=(10,10))\n",
+    "\n",
+    "    plt.title('Performance of neural network solving an ODE compared to the analytical solution')\n",
+    "    plt.plot(x, g_analytical)\n",
+    "    plt.plot(x, g_dnn_ag[0,:])\n",
+    "    plt.legend(['analytical','nn'])\n",
+    "    plt.xlabel('x')\n",
+    "    plt.ylabel('g(x)')\n",
+    "\n",
+    "    ## Perform the computation using the numerical scheme\n",
+    "\n",
+    "    dx = 1/(Nx - 1)\n",
+    "\n",
+    "    # Set up the matrix A\n",
+    "    A = np.zeros((Nx-2,Nx-2))\n",
+    "\n",
+    "    A[0,0] = 2\n",
+    "    A[0,1] = -1\n",
+    "\n",
+    "    for i in range(1,Nx-3):\n",
+    "        A[i,i-1] = -1\n",
+    "        A[i,i] = 2\n",
+    "        A[i,i+1] = -1\n",
+    "\n",
+    "    A[Nx - 3, Nx - 4] = -1\n",
+    "    A[Nx - 3, Nx - 3] = 2\n",
+    "\n",
+    "    # Set up the vector f\n",
+    "    f_vec = dx**2 * f(x[1:-1])\n",
+    "\n",
+    "    # Solve the equation\n",
+    "    g_res = np.linalg.solve(A,f_vec)\n",
+    "\n",
+    "    g_vec = np.zeros(Nx)\n",
+    "    g_vec[1:-1] = g_res\n",
+    "\n",
+    "    # Print the differences between each method\n",
+    "    max_diff1 = np.max(np.abs(g_dnn_ag - g_analytical))\n",
+    "    max_diff2 = np.max(np.abs(g_vec - g_analytical))\n",
+    "    print(\"The max absolute difference between the analytical solution and DNN Autograd: %g\"%max_diff1)\n",
+    "    print(\"The max absolute difference between the analytical solution and numerical scheme: %g\"%max_diff2)\n",
+    "\n",
+    "    # Plot the results\n",
+    "    plt.figure(figsize=(10,10))\n",
+    "\n",
+    "    plt.plot(x,g_vec)\n",
+    "    plt.plot(x,g_analytical)\n",
+    "    plt.plot(x,g_dnn_ag[0,:])\n",
+    "\n",
+    "    plt.legend(['numerical scheme','analytical','dnn'])\n",
+    "    plt.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Partial Differential Equations\n",
+    "\n",
+    "A partial differential equation (PDE) has a solution here the function\n",
+    "is defined by multiple variables.  The equation may involve all kinds\n",
+    "of combinations of which variables the function is differentiated with\n",
+    "respect to.\n",
+    "\n",
+    "In general, a partial differential equation for a function $g(x_1,\\dots,x_N)$ with $N$ variables may be expressed as"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation} \\label{PDE} \\tag{17}\n",
+    "  f\\left(x_1, \\, \\dots \\, , x_N, \\frac{\\partial g(x_1,\\dots,x_N) }{\\partial x_1}, \\dots , \\frac{\\partial g(x_1,\\dots,x_N) }{\\partial x_N}, \\frac{\\partial g(x_1,\\dots,x_N) }{\\partial x_1\\partial x_2}, \\, \\dots \\, , \\frac{\\partial^n g(x_1,\\dots,x_N) }{\\partial x_N^n} \\right) = 0\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "where $f$ is an expression involving all kinds of possible mixed derivatives of $g(x_1,\\dots,x_N)$ up to an order $n$. In order for the solution to be unique, some additional conditions must also be given.\n",
+    "\n",
+    "\n",
+    "## Type of problem\n",
+    "\n",
+    "The problem our network must solve for, is similar to the ODE case.\n",
+    "We must have a trial solution $g_t$ at hand.\n",
+    "\n",
+    "For instance, the trial solution could be expressed as"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\begin{align*}\n",
+    "  g_t(x_1,\\dots,x_N) = h_1(x_1,\\dots,x_N) + h_2(x_1,\\dots,x_N,N(x_1,\\dots,x_N,P))\n",
+    "\\end{align*}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "where $h_1(x_1,\\dots,x_N)$ is a function that ensures $g_t(x_1,\\dots,x_N)$ satisfies some given conditions.\n",
+    "The neural network $N(x_1,\\dots,x_N,P)$ has weights and biases described by $P$ and $h_2(x_1,\\dots,x_N,N(x_1,\\dots,x_N,P))$ is an expression using the output from the neural network in some way.\n",
+    "\n",
+    "The role of the function $h_2(x_1,\\dots,x_N,N(x_1,\\dots,x_N,P))$, is to ensure that the output of $N(x_1,\\dots,x_N,P)$ is zero when $g_t(x_1,\\dots,x_N)$ is evaluated at the values of $x_1,\\dots,x_N$ where the given conditions must be satisfied. The function $h_1(x_1,\\dots,x_N)$ should alone make $g_t(x_1,\\dots,x_N)$ satisfy the conditions.\n",
+    "\n",
+    "\n",
+    "\n",
+    "## Network requirements\n",
+    "\n",
+    "The network tries then the minimize the cost function following the\n",
+    "same ideas as described for the ODE case, but now with more than one\n",
+    "variables to consider.  The concept still remains the same; find a set\n",
+    "of parameters $P$ such that the expression $f$ in ([17](#PDE)) is as\n",
+    "close to zero as possible.\n",
+    "\n",
+    "As for the ODE case, the cost function is the mean squared error that\n",
+    "the network must try to minimize. The cost function for the network to\n",
+    "minimize is"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "C\\left(x_1, \\dots, x_N, P\\right) = \\left(  f\\left(x_1, \\, \\dots \\, , x_N, \\frac{\\partial g(x_1,\\dots,x_N) }{\\partial x_1}, \\dots , \\frac{\\partial g(x_1,\\dots,x_N) }{\\partial x_N}, \\frac{\\partial g(x_1,\\dots,x_N) }{\\partial x_1\\partial x_2}, \\, \\dots \\, , \\frac{\\partial^n g(x_1,\\dots,x_N) }{\\partial x_N^n} \\right) \\right)^2\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## More details\n",
+    "\n",
+    "If we let $\\boldsymbol{x} = \\big( x_1, \\dots, x_N \\big)$ be an array containing the values for $x_1, \\dots, x_N$ respectively, the cost function can be reformulated into the following:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "C\\left(\\boldsymbol{x}, P\\right) = f\\left( \\left( \\boldsymbol{x}, \\frac{\\partial g(\\boldsymbol{x}) }{\\partial x_1}, \\dots , \\frac{\\partial g(\\boldsymbol{x}) }{\\partial x_N}, \\frac{\\partial g(\\boldsymbol{x}) }{\\partial x_1\\partial x_2}, \\, \\dots \\, , \\frac{\\partial^n g(\\boldsymbol{x}) }{\\partial x_N^n} \\right) \\right)^2\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "If we also have $M$ different sets of values for $x_1, \\dots, x_N$, that is $\\boldsymbol{x}_i = \\big(x_1^{(i)}, \\dots, x_N^{(i)}\\big)$ for $i = 1,\\dots,M$ being the rows in matrix $X$, the cost function can be generalized into"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "C\\left(X, P \\right) = \\sum_{i=1}^M f\\left( \\left( \\boldsymbol{x}_i, \\frac{\\partial g(\\boldsymbol{x}_i) }{\\partial x_1}, \\dots , \\frac{\\partial g(\\boldsymbol{x}_i) }{\\partial x_N}, \\frac{\\partial g(\\boldsymbol{x}_i) }{\\partial x_1\\partial x_2}, \\, \\dots \\, , \\frac{\\partial^n g(\\boldsymbol{x}_i) }{\\partial x_N^n} \\right) \\right)^2.\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Example: The diffusion equation\n",
+    "\n",
+    "In one spatial dimension, the equation reads"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\frac{\\partial g(x,t)}{\\partial t} = \\frac{\\partial^2 g(x,t)}{\\partial x^2}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "where a possible choice of conditions are"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\begin{align*}\n",
+    "g(0,t) &= 0 ,\\qquad t \\geq 0 \\\\\n",
+    "g(1,t) &= 0, \\qquad t \\geq 0 \\\\\n",
+    "g(x,0) &= u(x),\\qquad x\\in [0,1]\n",
+    "\\end{align*}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "with $u(x)$ being some given function.\n",
+    "\n",
+    "\n",
+    "## Defining the problem\n",
+    "\n",
+    "For this case, we want to find $g(x,t)$ such that"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation}\n",
+    "  \\frac{\\partial g(x,t)}{\\partial t} = \\frac{\\partial^2 g(x,t)}{\\partial x^2}\n",
+    "\\end{equation} \\label{diffonedim} \\tag{18}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "and"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\begin{align*}\n",
+    "g(0,t) &= 0 ,\\qquad t \\geq 0 \\\\\n",
+    "g(1,t) &= 0, \\qquad t \\geq 0 \\\\\n",
+    "g(x,0) &= u(x),\\qquad x\\in [0,1]\n",
+    "\\end{align*}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "with $u(x) = \\sin(\\pi x)$.\n",
+    "\n",
+    "First, let us set up the deep neural network.\n",
+    "The deep neural network will follow the same structure as discussed in the examples solving the ODEs.\n",
+    "First, we will look into how Autograd could be used in a network tailored to solve for bivariate functions.\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "## Setting up the network using Autograd\n",
+    "\n",
+    "The only change to do here, is to extend our network such that\n",
+    "functions of multiple parameters are correctly handled.  In this case\n",
+    "we have two variables in our function to solve for, that is time $t$\n",
+    "and position $x$.  The variables will be represented by a\n",
+    "one-dimensional array in the program.  The program will evaluate the\n",
+    "network at each possible pair $(x,t)$, given an array for the desired\n",
+    "$x$-values and $t$-values to approximate the solution at."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "def sigmoid(z):\n",
+    "    return 1/(1 + np.exp(-z))\n",
+    "\n",
+    "def deep_neural_network(deep_params, x):\n",
+    "    # x is now a point and a 1D numpy array; make it a column vector\n",
+    "    num_coordinates = np.size(x,0)\n",
+    "    x = x.reshape(num_coordinates,-1)\n",
+    "\n",
+    "    num_points = np.size(x,1)\n",
+    "\n",
+    "    # N_hidden is the number of hidden layers\n",
+    "    N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n",
+    "\n",
+    "    # Assume that the input layer does nothing to the input x\n",
+    "    x_input = x\n",
+    "    x_prev = x_input\n",
+    "\n",
+    "    ## Hidden layers:\n",
+    "\n",
+    "    for l in range(N_hidden):\n",
+    "        # From the list of parameters P; find the correct weigths and bias for this layer\n",
+    "        w_hidden = deep_params[l]\n",
+    "\n",
+    "        # Add a row of ones to include bias\n",
+    "        x_prev = np.concatenate((np.ones((1,num_points)), x_prev ), axis = 0)\n",
+    "\n",
+    "        z_hidden = np.matmul(w_hidden, x_prev)\n",
+    "        x_hidden = sigmoid(z_hidden)\n",
+    "\n",
+    "        # Update x_prev such that next layer can use the output from this layer\n",
+    "        x_prev = x_hidden\n",
+    "\n",
+    "    ## Output layer:\n",
+    "\n",
+    "    # Get the weights and bias for this layer\n",
+    "    w_output = deep_params[-1]\n",
+    "\n",
+    "    # Include bias:\n",
+    "    x_prev = np.concatenate((np.ones((1,num_points)), x_prev), axis = 0)\n",
+    "\n",
+    "    z_output = np.matmul(w_output, x_prev)\n",
+    "    x_output = z_output\n",
+    "\n",
+    "    return x_output[0][0]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Setting up the network using Autograd; The trial solution\n",
+    "\n",
+    "The cost function must then iterate through the given arrays\n",
+    "containing values for $x$ and $t$, defines a point $(x,t)$ the deep\n",
+    "neural network and the trial solution is evaluated at, and then finds\n",
+    "the Jacobian of the trial solution.\n",
+    "\n",
+    "A possible trial solution for this PDE is\n",
+    "\n",
+    "$$\n",
+    "g_t(x,t) = h_1(x,t) + x(1-x)tN(x,t,P)\n",
+    "$$\n",
+    "\n",
+    "with $A(x,t)$ being a function ensuring that $g_t(x,t)$ satisfies our given conditions, and $N(x,t,P)$ being the output from the deep neural network using weights and biases for each layer from $P$.\n",
+    "\n",
+    "To fulfill the conditions, $A(x,t)$ could be:\n",
+    "\n",
+    "$$\n",
+    "h_1(x,t) = (1-t)\\Big(u(x) - \\big((1-x)u(0) + x u(1)\\big)\\Big) = (1-t)u(x) = (1-t)\\sin(\\pi x)\n",
+    "$$\n",
+    "since $(0) = u(1) = 0$ and $u(x) = \\sin(\\pi x)$.\n",
+    "\n",
+    "\n",
+    "## Why the jacobian?\n",
+    "\n",
+    "The Jacobian is used because the program must find the derivative of\n",
+    "the trial solution with respect to $x$ and $t$.\n",
+    "\n",
+    "This gives the necessity of computing the Jacobian matrix, as we want\n",
+    "to evaluate the gradient with respect to $x$ and $t$ (note that the\n",
+    "Jacobian of a scalar-valued multivariate function is simply its\n",
+    "gradient).\n",
+    "\n",
+    "In Autograd, the differentiation is by default done with respect to\n",
+    "the first input argument of your Python function. Since the points is\n",
+    "an array representing $x$ and $t$, the Jacobian is calculated using\n",
+    "the values of $x$ and $t$.\n",
+    "\n",
+    "To find the second derivative with respect to $x$ and $t$, the\n",
+    "Jacobian can be found for the second time. The result is a Hessian\n",
+    "matrix, which is the matrix containing all the possible second order\n",
+    "mixed derivatives of $g(x,t)$."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "# Set up the trial function:\n",
+    "def u(x):\n",
+    "    return np.sin(np.pi*x)\n",
+    "\n",
+    "def g_trial(point,P):\n",
+    "    x,t = point\n",
+    "    return (1-t)*u(x) + x*(1-x)*t*deep_neural_network(P,point)\n",
+    "\n",
+    "# The right side of the ODE:\n",
+    "def f(point):\n",
+    "    return 0.\n",
+    "\n",
+    "# The cost function:\n",
+    "def cost_function(P, x, t):\n",
+    "    cost_sum = 0\n",
+    "\n",
+    "    g_t_jacobian_func = jacobian(g_trial)\n",
+    "    g_t_hessian_func = hessian(g_trial)\n",
+    "\n",
+    "    for x_ in x:\n",
+    "        for t_ in t:\n",
+    "            point = np.array([x_,t_])\n",
+    "\n",
+    "            g_t = g_trial(point,P)\n",
+    "            g_t_jacobian = g_t_jacobian_func(point,P)\n",
+    "            g_t_hessian = g_t_hessian_func(point,P)\n",
+    "\n",
+    "            g_t_dt = g_t_jacobian[1]\n",
+    "            g_t_d2x = g_t_hessian[0][0]\n",
+    "\n",
+    "            func = f(point)\n",
+    "\n",
+    "            err_sqr = ( (g_t_dt - g_t_d2x) - func)**2\n",
+    "            cost_sum += err_sqr\n",
+    "\n",
+    "    return cost_sum"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Setting up the network using Autograd; The full program\n",
+    "\n",
+    "Having set up the network, along with the trial solution and cost function, we can now see how the deep neural network performs by comparing the results to the analytical solution.\n",
+    "\n",
+    "The analytical solution of our problem is\n",
+    "\n",
+    "$$\n",
+    "g(x,t) = \\exp(-\\pi^2 t)\\sin(\\pi x)\n",
+    "$$\n",
+    "\n",
+    "A possible way to implement a neural network solving the PDE, is given below.\n",
+    "Be aware, though, that it is fairly slow for the parameters used.\n",
+    "A better result is possible, but requires more iterations, and thus longer time to complete.\n",
+    "\n",
+    "\n",
+    "Indeed, the program below is not optimal in its implementation, but rather serves as an example on how to implement and use a neural network to solve a PDE.\n",
+    "Using TensorFlow results in a much better execution time. Try it!"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "import autograd.numpy as np\n",
+    "from autograd import jacobian,hessian,grad\n",
+    "import autograd.numpy.random as npr\n",
+    "from matplotlib import cm\n",
+    "from matplotlib import pyplot as plt\n",
+    "from mpl_toolkits.mplot3d import axes3d\n",
+    "\n",
+    "## Set up the network\n",
+    "\n",
+    "def sigmoid(z):\n",
+    "    return 1/(1 + np.exp(-z))\n",
+    "\n",
+    "def deep_neural_network(deep_params, x):\n",
+    "    # x is now a point and a 1D numpy array; make it a column vector\n",
+    "    num_coordinates = np.size(x,0)\n",
+    "    x = x.reshape(num_coordinates,-1)\n",
+    "\n",
+    "    num_points = np.size(x,1)\n",
+    "\n",
+    "    # N_hidden is the number of hidden layers\n",
+    "    N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n",
+    "\n",
+    "    # Assume that the input layer does nothing to the input x\n",
+    "    x_input = x\n",
+    "    x_prev = x_input\n",
+    "\n",
+    "    ## Hidden layers:\n",
+    "\n",
+    "    for l in range(N_hidden):\n",
+    "        # From the list of parameters P; find the correct weigths and bias for this layer\n",
+    "        w_hidden = deep_params[l]\n",
+    "\n",
+    "        # Add a row of ones to include bias\n",
+    "        x_prev = np.concatenate((np.ones((1,num_points)), x_prev ), axis = 0)\n",
+    "\n",
+    "        z_hidden = np.matmul(w_hidden, x_prev)\n",
+    "        x_hidden = sigmoid(z_hidden)\n",
+    "\n",
+    "        # Update x_prev such that next layer can use the output from this layer\n",
+    "        x_prev = x_hidden\n",
+    "\n",
+    "    ## Output layer:\n",
+    "\n",
+    "    # Get the weights and bias for this layer\n",
+    "    w_output = deep_params[-1]\n",
+    "\n",
+    "    # Include bias:\n",
+    "    x_prev = np.concatenate((np.ones((1,num_points)), x_prev), axis = 0)\n",
+    "\n",
+    "    z_output = np.matmul(w_output, x_prev)\n",
+    "    x_output = z_output\n",
+    "\n",
+    "    return x_output[0][0]\n",
+    "\n",
+    "## Define the trial solution and cost function\n",
+    "def u(x):\n",
+    "    return np.sin(np.pi*x)\n",
+    "\n",
+    "def g_trial(point,P):\n",
+    "    x,t = point\n",
+    "    return (1-t)*u(x) + x*(1-x)*t*deep_neural_network(P,point)\n",
+    "\n",
+    "# The right side of the ODE:\n",
+    "def f(point):\n",
+    "    return 0.\n",
+    "\n",
+    "# The cost function:\n",
+    "def cost_function(P, x, t):\n",
+    "    cost_sum = 0\n",
+    "\n",
+    "    g_t_jacobian_func = jacobian(g_trial)\n",
+    "    g_t_hessian_func = hessian(g_trial)\n",
+    "\n",
+    "    for x_ in x:\n",
+    "        for t_ in t:\n",
+    "            point = np.array([x_,t_])\n",
+    "\n",
+    "            g_t = g_trial(point,P)\n",
+    "            g_t_jacobian = g_t_jacobian_func(point,P)\n",
+    "            g_t_hessian = g_t_hessian_func(point,P)\n",
+    "\n",
+    "            g_t_dt = g_t_jacobian[1]\n",
+    "            g_t_d2x = g_t_hessian[0][0]\n",
+    "\n",
+    "            func = f(point)\n",
+    "\n",
+    "            err_sqr = ( (g_t_dt - g_t_d2x) - func)**2\n",
+    "            cost_sum += err_sqr\n",
+    "\n",
+    "    return cost_sum /( np.size(x)*np.size(t) )\n",
+    "\n",
+    "## For comparison, define the analytical solution\n",
+    "def g_analytic(point):\n",
+    "    x,t = point\n",
+    "    return np.exp(-np.pi**2*t)*np.sin(np.pi*x)\n",
+    "\n",
+    "## Set up a function for training the network to solve for the equation\n",
+    "def solve_pde_deep_neural_network(x,t, num_neurons, num_iter, lmb):\n",
+    "    ## Set up initial weigths and biases\n",
+    "    N_hidden = np.size(num_neurons)\n",
+    "\n",
+    "    ## Set up initial weigths and biases\n",
+    "\n",
+    "    # Initialize the list of parameters:\n",
+    "    P = [None]*(N_hidden + 1) # + 1 to include the output layer\n",
+    "\n",
+    "    P[0] = npr.randn(num_neurons[0], 2 + 1 ) # 2 since we have two points, +1 to include bias\n",
+    "    for l in range(1,N_hidden):\n",
+    "        P[l] = npr.randn(num_neurons[l], num_neurons[l-1] + 1) # +1 to include bias\n",
+    "\n",
+    "    # For the output layer\n",
+    "    P[-1] = npr.randn(1, num_neurons[-1] + 1 ) # +1 since bias is included\n",
+    "\n",
+    "    print('Initial cost: ',cost_function(P, x, t))\n",
+    "\n",
+    "    cost_function_grad = grad(cost_function,0)\n",
+    "\n",
+    "    # Let the update be done num_iter times\n",
+    "    for i in range(num_iter):\n",
+    "        cost_grad =  cost_function_grad(P, x , t)\n",
+    "\n",
+    "        for l in range(N_hidden+1):\n",
+    "            P[l] = P[l] - lmb * cost_grad[l]\n",
+    "\n",
+    "    print('Final cost: ',cost_function(P, x, t))\n",
+    "\n",
+    "    return P\n",
+    "\n",
+    "if __name__ == '__main__':\n",
+    "    ### Use the neural network:\n",
+    "    npr.seed(15)\n",
+    "\n",
+    "    ## Decide the vales of arguments to the function to solve\n",
+    "    Nx = 10; Nt = 10\n",
+    "    x = np.linspace(0, 1, Nx)\n",
+    "    t = np.linspace(0,1,Nt)\n",
+    "\n",
+    "    ## Set up the parameters for the network\n",
+    "    num_hidden_neurons = [100, 25]\n",
+    "    num_iter = 250\n",
+    "    lmb = 0.01\n",
+    "\n",
+    "    P = solve_pde_deep_neural_network(x,t, num_hidden_neurons, num_iter, lmb)\n",
+    "\n",
+    "    ## Store the results\n",
+    "    g_dnn_ag = np.zeros((Nx, Nt))\n",
+    "    G_analytical = np.zeros((Nx, Nt))\n",
+    "    for i,x_ in enumerate(x):\n",
+    "        for j, t_ in enumerate(t):\n",
+    "            point = np.array([x_, t_])\n",
+    "            g_dnn_ag[i,j] = g_trial(point,P)\n",
+    "\n",
+    "            G_analytical[i,j] = g_analytic(point)\n",
+    "\n",
+    "    # Find the map difference between the analytical and the computed solution\n",
+    "    diff_ag = np.abs(g_dnn_ag - G_analytical)\n",
+    "    print('Max absolute difference between the analytical solution and the network: %g'%np.max(diff_ag))\n",
+    "\n",
+    "    ## Plot the solutions in two dimensions, that being in position and time\n",
+    "\n",
+    "    T,X = np.meshgrid(t,x)\n",
+    "\n",
+    "    fig = plt.figure(figsize=(10,10))\n",
+    "    ax = fig.gca(projection='3d')\n",
+    "    ax.set_title('Solution from the deep neural network w/ %d layer'%len(num_hidden_neurons))\n",
+    "    s = ax.plot_surface(T,X,g_dnn_ag,linewidth=0,antialiased=False,cmap=cm.viridis)\n",
+    "    ax.set_xlabel('Time $t$')\n",
+    "    ax.set_ylabel('Position $x$');\n",
+    "\n",
+    "\n",
+    "    fig = plt.figure(figsize=(10,10))\n",
+    "    ax = fig.gca(projection='3d')\n",
+    "    ax.set_title('Analytical solution')\n",
+    "    s = ax.plot_surface(T,X,G_analytical,linewidth=0,antialiased=False,cmap=cm.viridis)\n",
+    "    ax.set_xlabel('Time $t$')\n",
+    "    ax.set_ylabel('Position $x$');\n",
+    "\n",
+    "    fig = plt.figure(figsize=(10,10))\n",
+    "    ax = fig.gca(projection='3d')\n",
+    "    ax.set_title('Difference')\n",
+    "    s = ax.plot_surface(T,X,diff_ag,linewidth=0,antialiased=False,cmap=cm.viridis)\n",
+    "    ax.set_xlabel('Time $t$')\n",
+    "    ax.set_ylabel('Position $x$');\n",
+    "\n",
+    "    ## Take some slices of the 3D plots just to see the solutions at particular times\n",
+    "    indx1 = 0\n",
+    "    indx2 = int(Nt/2)\n",
+    "    indx3 = Nt-1\n",
+    "\n",
+    "    t1 = t[indx1]\n",
+    "    t2 = t[indx2]\n",
+    "    t3 = t[indx3]\n",
+    "\n",
+    "    # Slice the results from the DNN\n",
+    "    res1 = g_dnn_ag[:,indx1]\n",
+    "    res2 = g_dnn_ag[:,indx2]\n",
+    "    res3 = g_dnn_ag[:,indx3]\n",
+    "\n",
+    "    # Slice the analytical results\n",
+    "    res_analytical1 = G_analytical[:,indx1]\n",
+    "    res_analytical2 = G_analytical[:,indx2]\n",
+    "    res_analytical3 = G_analytical[:,indx3]\n",
+    "\n",
+    "    # Plot the slices\n",
+    "    plt.figure(figsize=(10,10))\n",
+    "    plt.title(\"Computed solutions at time = %g\"%t1)\n",
+    "    plt.plot(x, res1)\n",
+    "    plt.plot(x,res_analytical1)\n",
+    "    plt.legend(['dnn','analytical'])\n",
+    "\n",
+    "    plt.figure(figsize=(10,10))\n",
+    "    plt.title(\"Computed solutions at time = %g\"%t2)\n",
+    "    plt.plot(x, res2)\n",
+    "    plt.plot(x,res_analytical2)\n",
+    "    plt.legend(['dnn','analytical'])\n",
+    "\n",
+    "    plt.figure(figsize=(10,10))\n",
+    "    plt.title(\"Computed solutions at time = %g\"%t3)\n",
+    "    plt.plot(x, res3)\n",
+    "    plt.plot(x,res_analytical3)\n",
+    "    plt.legend(['dnn','analytical'])\n",
+    "\n",
+    "    plt.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Example: Solving the wave equation with Neural Networks\n",
+    "\n",
+    "The wave equation is"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\frac{\\partial^2 g(x,t)}{\\partial t^2} = c^2\\frac{\\partial^2 g(x,t)}{\\partial x^2}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "with $c$ being the specified wave speed.\n",
+    "\n",
+    "Here, the chosen conditions are"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\begin{align*}\n",
+    "\tg(0,t) &= 0 \\\\\n",
+    "\tg(1,t) &= 0 \\\\\n",
+    "\tg(x,0) &= u(x) \\\\\n",
+    "\t\\frac{\\partial g(x,t)}{\\partial t} \\Big |_{t = 0} &= v(x)\n",
+    "\\end{align*}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "where $\\frac{\\partial g(x,t)}{\\partial t} \\Big |_{t = 0}$ means the derivative of $g(x,t)$ with respect to $t$ is evaluated at $t = 0$, and $u(x)$ and $v(x)$ being given functions.\n",
+    "\n",
+    "\n",
+    "## The problem to solve for\n",
+    "\n",
+    "The wave equation to solve for, is"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation} \\label{wave} \\tag{19}\n",
+    "\\frac{\\partial^2 g(x,t)}{\\partial t^2} = c^2 \\frac{\\partial^2 g(x,t)}{\\partial x^2}\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "where $c$ is the given wave speed.\n",
+    "The chosen conditions for this equation are"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{aligned}\n",
+    "g(0,t) &= 0, &t \\geq 0 \\\\\n",
+    "g(1,t) &= 0, &t \\geq 0 \\\\\n",
+    "g(x,0) &= u(x), &x\\in[0,1] \\\\\n",
+    "\\frac{\\partial g(x,t)}{\\partial t}\\Big |_{t = 0} &= v(x), &x \\in [0,1]\n",
+    "\\end{aligned} \\label{condwave} \\tag{20}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "In this example, let $c = 1$ and $u(x) = \\sin(\\pi x)$ and $v(x) = -\\pi\\sin(\\pi x)$.\n",
+    "\n",
+    "\n",
+    "\n",
+    "## The trial solution\n",
+    "Setting up the network is done in similar matter as for the example of solving the diffusion equation.\n",
+    "The only things we have to change, is the trial solution such that it satisfies the conditions from ([20](#condwave)) and the cost function.\n",
+    "\n",
+    "The trial solution becomes slightly different since we have other conditions than in the example of solving the diffusion equation. Here, a possible trial solution $g_t(x,t)$ is\n",
+    "\n",
+    "$$\n",
+    "g_t(x,t) = h_1(x,t) + x(1-x)t^2N(x,t,P)\n",
+    "$$\n",
+    "\n",
+    "where\n",
+    "\n",
+    "$$\n",
+    "h_1(x,t) = (1-t^2)u(x) + tv(x)\n",
+    "$$\n",
+    "\n",
+    "Note that this trial solution satisfies the conditions only if $u(0) = v(0) = u(1) = v(1) = 0$, which is the case in this example.\n",
+    "\n",
+    "\n",
+    "## The analytical solution\n",
+    "\n",
+    "The analytical solution for our specific problem, is\n",
+    "\n",
+    "$$\n",
+    "g(x,t) = \\sin(\\pi x)\\cos(\\pi t) - \\sin(\\pi x)\\sin(\\pi t)\n",
+    "$$\n",
+    "\n",
+    "\n",
+    "## Solving the wave equation - the full program using Autograd"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 16,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "import autograd.numpy as np\n",
+    "from autograd import hessian,grad\n",
+    "import autograd.numpy.random as npr\n",
+    "from matplotlib import cm\n",
+    "from matplotlib import pyplot as plt\n",
+    "from mpl_toolkits.mplot3d import axes3d\n",
+    "\n",
+    "## Set up the trial function:\n",
+    "def u(x):\n",
+    "    return np.sin(np.pi*x)\n",
+    "\n",
+    "def v(x):\n",
+    "    return -np.pi*np.sin(np.pi*x)\n",
+    "\n",
+    "def h1(point):\n",
+    "    x,t = point\n",
+    "    return (1 - t**2)*u(x) + t*v(x)\n",
+    "\n",
+    "def g_trial(point,P):\n",
+    "    x,t = point\n",
+    "    return h1(point) + x*(1-x)*t**2*deep_neural_network(P,point)\n",
+    "\n",
+    "## Define the cost function\n",
+    "def cost_function(P, x, t):\n",
+    "    cost_sum = 0\n",
+    "\n",
+    "    g_t_hessian_func = hessian(g_trial)\n",
+    "\n",
+    "    for x_ in x:\n",
+    "        for t_ in t:\n",
+    "            point = np.array([x_,t_])\n",
+    "\n",
+    "            g_t_hessian = g_t_hessian_func(point,P)\n",
+    "\n",
+    "            g_t_d2x = g_t_hessian[0][0]\n",
+    "            g_t_d2t = g_t_hessian[1][1]\n",
+    "\n",
+    "            err_sqr = ( (g_t_d2t - g_t_d2x) )**2\n",
+    "            cost_sum += err_sqr\n",
+    "\n",
+    "    return cost_sum / (np.size(t) * np.size(x))\n",
+    "\n",
+    "## The neural network\n",
+    "def sigmoid(z):\n",
+    "    return 1/(1 + np.exp(-z))\n",
+    "\n",
+    "def deep_neural_network(deep_params, x):\n",
+    "    # x is now a point and a 1D numpy array; make it a column vector\n",
+    "    num_coordinates = np.size(x,0)\n",
+    "    x = x.reshape(num_coordinates,-1)\n",
+    "\n",
+    "    num_points = np.size(x,1)\n",
+    "\n",
+    "    # N_hidden is the number of hidden layers\n",
+    "    N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n",
+    "\n",
+    "    # Assume that the input layer does nothing to the input x\n",
+    "    x_input = x\n",
+    "    x_prev = x_input\n",
+    "\n",
+    "    ## Hidden layers:\n",
+    "\n",
+    "    for l in range(N_hidden):\n",
+    "        # From the list of parameters P; find the correct weigths and bias for this layer\n",
+    "        w_hidden = deep_params[l]\n",
+    "\n",
+    "        # Add a row of ones to include bias\n",
+    "        x_prev = np.concatenate((np.ones((1,num_points)), x_prev ), axis = 0)\n",
+    "\n",
+    "        z_hidden = np.matmul(w_hidden, x_prev)\n",
+    "        x_hidden = sigmoid(z_hidden)\n",
+    "\n",
+    "        # Update x_prev such that next layer can use the output from this layer\n",
+    "        x_prev = x_hidden\n",
+    "\n",
+    "    ## Output layer:\n",
+    "\n",
+    "    # Get the weights and bias for this layer\n",
+    "    w_output = deep_params[-1]\n",
+    "\n",
+    "    # Include bias:\n",
+    "    x_prev = np.concatenate((np.ones((1,num_points)), x_prev), axis = 0)\n",
+    "\n",
+    "    z_output = np.matmul(w_output, x_prev)\n",
+    "    x_output = z_output\n",
+    "\n",
+    "    return x_output[0][0]\n",
+    "\n",
+    "## The analytical solution\n",
+    "def g_analytic(point):\n",
+    "    x,t = point\n",
+    "    return np.sin(np.pi*x)*np.cos(np.pi*t) - np.sin(np.pi*x)*np.sin(np.pi*t)\n",
+    "\n",
+    "def solve_pde_deep_neural_network(x,t, num_neurons, num_iter, lmb):\n",
+    "    ## Set up initial weigths and biases\n",
+    "    N_hidden = np.size(num_neurons)\n",
+    "\n",
+    "    ## Set up initial weigths and biases\n",
+    "\n",
+    "    # Initialize the list of parameters:\n",
+    "    P = [None]*(N_hidden + 1) # + 1 to include the output layer\n",
+    "\n",
+    "    P[0] = npr.randn(num_neurons[0], 2 + 1 ) # 2 since we have two points, +1 to include bias\n",
+    "    for l in range(1,N_hidden):\n",
+    "        P[l] = npr.randn(num_neurons[l], num_neurons[l-1] + 1) # +1 to include bias\n",
+    "\n",
+    "    # For the output layer\n",
+    "    P[-1] = npr.randn(1, num_neurons[-1] + 1 ) # +1 since bias is included\n",
+    "\n",
+    "    print('Initial cost: ',cost_function(P, x, t))\n",
+    "\n",
+    "    cost_function_grad = grad(cost_function,0)\n",
+    "\n",
+    "    # Let the update be done num_iter times\n",
+    "    for i in range(num_iter):\n",
+    "        cost_grad =  cost_function_grad(P, x , t)\n",
+    "\n",
+    "        for l in range(N_hidden+1):\n",
+    "            P[l] = P[l] - lmb * cost_grad[l]\n",
+    "\n",
+    "\n",
+    "    print('Final cost: ',cost_function(P, x, t))\n",
+    "\n",
+    "    return P\n",
+    "\n",
+    "if __name__ == '__main__':\n",
+    "    ### Use the neural network:\n",
+    "    npr.seed(15)\n",
+    "\n",
+    "    ## Decide the vales of arguments to the function to solve\n",
+    "    Nx = 10; Nt = 10\n",
+    "    x = np.linspace(0, 1, Nx)\n",
+    "    t = np.linspace(0,1,Nt)\n",
+    "\n",
+    "    ## Set up the parameters for the network\n",
+    "    num_hidden_neurons = [50,20]\n",
+    "    num_iter = 1000\n",
+    "    lmb = 0.01\n",
+    "\n",
+    "    P = solve_pde_deep_neural_network(x,t, num_hidden_neurons, num_iter, lmb)\n",
+    "\n",
+    "    ## Store the results\n",
+    "    res = np.zeros((Nx, Nt))\n",
+    "    res_analytical = np.zeros((Nx, Nt))\n",
+    "    for i,x_ in enumerate(x):\n",
+    "        for j, t_ in enumerate(t):\n",
+    "            point = np.array([x_, t_])\n",
+    "            res[i,j] = g_trial(point,P)\n",
+    "\n",
+    "            res_analytical[i,j] = g_analytic(point)\n",
+    "\n",
+    "    diff = np.abs(res - res_analytical)\n",
+    "    print(\"Max difference between analytical and solution from nn: %g\"%np.max(diff))\n",
+    "\n",
+    "    ## Plot the solutions in two dimensions, that being in position and time\n",
+    "\n",
+    "    T,X = np.meshgrid(t,x)\n",
+    "\n",
+    "    fig = plt.figure(figsize=(10,10))\n",
+    "    ax = fig.gca(projection='3d')\n",
+    "    ax.set_title('Solution from the deep neural network w/ %d layer'%len(num_hidden_neurons))\n",
+    "    s = ax.plot_surface(T,X,res,linewidth=0,antialiased=False,cmap=cm.viridis)\n",
+    "    ax.set_xlabel('Time $t$')\n",
+    "    ax.set_ylabel('Position $x$');\n",
+    "\n",
+    "\n",
+    "    fig = plt.figure(figsize=(10,10))\n",
+    "    ax = fig.gca(projection='3d')\n",
+    "    ax.set_title('Analytical solution')\n",
+    "    s = ax.plot_surface(T,X,res_analytical,linewidth=0,antialiased=False,cmap=cm.viridis)\n",
+    "    ax.set_xlabel('Time $t$')\n",
+    "    ax.set_ylabel('Position $x$');\n",
+    "\n",
+    "\n",
+    "    fig = plt.figure(figsize=(10,10))\n",
+    "    ax = fig.gca(projection='3d')\n",
+    "    ax.set_title('Difference')\n",
+    "    s = ax.plot_surface(T,X,diff,linewidth=0,antialiased=False,cmap=cm.viridis)\n",
+    "    ax.set_xlabel('Time $t$')\n",
+    "    ax.set_ylabel('Position $x$');\n",
+    "\n",
+    "    ## Take some slices of the 3D plots just to see the solutions at particular times\n",
+    "    indx1 = 0\n",
+    "    indx2 = int(Nt/2)\n",
+    "    indx3 = Nt-1\n",
+    "\n",
+    "    t1 = t[indx1]\n",
+    "    t2 = t[indx2]\n",
+    "    t3 = t[indx3]\n",
+    "\n",
+    "    # Slice the results from the DNN\n",
+    "    res1 = res[:,indx1]\n",
+    "    res2 = res[:,indx2]\n",
+    "    res3 = res[:,indx3]\n",
+    "\n",
+    "    # Slice the analytical results\n",
+    "    res_analytical1 = res_analytical[:,indx1]\n",
+    "    res_analytical2 = res_analytical[:,indx2]\n",
+    "    res_analytical3 = res_analytical[:,indx3]\n",
+    "\n",
+    "    # Plot the slices\n",
+    "    plt.figure(figsize=(10,10))\n",
+    "    plt.title(\"Computed solutions at time = %g\"%t1)\n",
+    "    plt.plot(x, res1)\n",
+    "    plt.plot(x,res_analytical1)\n",
+    "    plt.legend(['dnn','analytical'])\n",
+    "\n",
+    "    plt.figure(figsize=(10,10))\n",
+    "    plt.title(\"Computed solutions at time = %g\"%t2)\n",
+    "    plt.plot(x, res2)\n",
+    "    plt.plot(x,res_analytical2)\n",
+    "    plt.legend(['dnn','analytical'])\n",
+    "\n",
+    "    plt.figure(figsize=(10,10))\n",
+    "    plt.title(\"Computed solutions at time = %g\"%t3)\n",
+    "    plt.plot(x, res3)\n",
+    "    plt.plot(x,res_analytical3)\n",
+    "    plt.legend(['dnn','analytical'])\n",
+    "\n",
+    "    plt.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Resources on differential equations and deep learning\n",
+    "\n",
+    "1. [Artificial neural networks for solving ordinary and partial differential equations by I.E. Lagaris et al](https://pdfs.semanticscholar.org/d061/df393e0e8fbfd0ea24976458b7d42419040d.pdf)\n",
+    "\n",
+    "2. [Neural networks for solving differential equations by A. Honchar](https://becominghuman.ai/neural-networks-for-solving-differential-equations-fa230ac5e04c)\n",
+    "\n",
+    "3. [Solving differential equations using neural networks by M.M Chiaramonte and M. Kiener](http://cs229.stanford.edu/proj2013/ChiaramonteKiener-SolvingDifferentialEquationsUsingNeuralNetworks.pdf)\n",
+    "\n",
+    "4. [Introduction to Partial Differential Equations by A. Tveito, R. Winther](https://www.springer.com/us/book/9783540225515)"
+   ]
+  }
+ ],
+ "metadata": {
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.8.3"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}
\ No newline at end of file
diff --git a/doc/src/LectureNotes/_build/jupyter_execute/chapter10.py b/doc/src/LectureNotes/_build/jupyter_execute/chapter10.py
new file mode 100644
index 000000000..dd645ae7a
--- /dev/null
+++ b/doc/src/LectureNotes/_build/jupyter_execute/chapter10.py
@@ -0,0 +1,3058 @@
+# Recurrent Neural Networks
+
+[Overview video](https://www.youtube.com/watch?v=SEnXr6v2ifU&ab_channel=AlexanderAmini).
+See also lecture on Thursday October 22 and examples from [week 42](https://compphysics.github.io/MachineLearning/doc/pub/week42/html/week42.html).
+
+[IN5400 at UiO Lecture](https://www.uio.no/studier/emner/matnat/ifi/IN5400/v20/material/week10/in5400_2020_week10_recurrent_neural_network.pdf)
+
+[CS231 at Stanford Lecture](https://www.youtube.com/watch?v=6niqTuYFZLQ&list=PLzUTmXVwsnXod6WNdg57Yc3zFx_f-RYsq&index=10&ab_channel=StanfordUniversitySchoolofEngineering)
+
+## Recurrent neural networks: Overarching view
+
+Till now our focus has been, including convolutional neural networks
+as well, on feedforward neural networks. The output or the activations
+flow only in one direction, from the input layer to the output layer.
+
+A recurrent neural network (RNN) looks very much like a feedforward
+neural network, except that it also has connections pointing
+backward. 
+
+RNNs are used to analyze time series data such as stock prices, and
+tell you when to buy or sell. In autonomous driving systems, they can
+anticipate car trajectories and help avoid accidents. More generally,
+they can work on sequences of arbitrary lengths, rather than on
+fixed-sized inputs like all the nets we have discussed so far. For
+example, they can take sentences, documents, or audio samples as
+input, making them extremely useful for natural language processing
+systems such as automatic translation and speech-to-text.
+
+
+
+
+## Set up of an RNN
+
+
+Text to come.
+
+
+## A simple example
+
+%matplotlib inline
+
+# Start importing packages
+import pandas as pd
+import numpy as np
+import matplotlib.pyplot as plt
+import tensorflow as tf
+from tensorflow.keras import datasets, layers, models
+from tensorflow.keras.layers import Input
+from tensorflow.keras.models import Model, Sequential 
+from tensorflow.keras.layers import Dense, SimpleRNN, LSTM, GRU
+from tensorflow.keras import optimizers     
+from tensorflow.keras import regularizers           
+from tensorflow.keras.utils import to_categorical 
+
+
+
+# convert into dataset matrix
+def convertToMatrix(data, step):
+ X, Y =[], []
+ for i in range(len(data)-step):
+  d=i+step  
+  X.append(data[i:d,])
+  Y.append(data[d,])
+ return np.array(X), np.array(Y)
+
+step = 4
+N = 1000    
+Tp = 800    
+
+t=np.arange(0,N)
+x=np.sin(0.02*t)+2*np.random.rand(N)
+df = pd.DataFrame(x)
+df.head()
+
+plt.plot(df)
+plt.show()
+
+values=df.values
+train,test = values[0:Tp,:], values[Tp:N,:]
+
+# add step elements into train and test
+test = np.append(test,np.repeat(test[-1,],step))
+train = np.append(train,np.repeat(train[-1,],step))
+ 
+trainX,trainY =convertToMatrix(train,step)
+testX,testY =convertToMatrix(test,step)
+trainX = np.reshape(trainX, (trainX.shape[0], 1, trainX.shape[1]))
+testX = np.reshape(testX, (testX.shape[0], 1, testX.shape[1]))
+
+model = Sequential()
+model.add(SimpleRNN(units=32, input_shape=(1,step), activation="relu"))
+model.add(Dense(8, activation="relu")) 
+model.add(Dense(1))
+model.compile(loss='mean_squared_error', optimizer='rmsprop')
+model.summary()
+
+model.fit(trainX,trainY, epochs=100, batch_size=16, verbose=2)
+trainPredict = model.predict(trainX)
+testPredict= model.predict(testX)
+predicted=np.concatenate((trainPredict,testPredict),axis=0)
+
+trainScore = model.evaluate(trainX, trainY, verbose=0)
+print(trainScore)
+
+index = df.index.values
+plt.plot(index,df)
+plt.plot(index,predicted)
+plt.axvline(df.index[Tp], c="r")
+plt.show()
+
+## An extrapolation example
+
+The following code provides an example of how recurrent neural
+networks can be used to extrapolate to unknown values of physics data
+sets.  Specifically, the data sets used in this program come from
+a quantum mechanical many-body calculation of energies as functions of the number of particles.
+
+
+# For matrices and calculations
+import numpy as np
+# For machine learning (backend for keras)
+import tensorflow as tf
+# User-friendly machine learning library
+# Front end for TensorFlow
+import tensorflow.keras
+# Different methods from Keras needed to create an RNN
+# This is not necessary but it shortened function calls 
+# that need to be used in the code.
+from tensorflow.keras import datasets, layers, models
+from tensorflow.keras.layers import Input
+from tensorflow.keras import regularizers
+from tensorflow.keras.models import Model, Sequential
+from tensorflow.keras.layers import Dense, SimpleRNN, LSTM, GRU
+# For timing the code
+from timeit import default_timer as timer
+# For plotting
+import matplotlib.pyplot as plt
+
+
+# The data set
+datatype='VaryDimension'
+X_tot = np.arange(2, 42, 2)
+y_tot = np.array([-0.03077640549, -0.08336233266, -0.1446729567, -0.2116753732, -0.2830637392, -0.3581341341, -0.436462435, -0.5177783846,
+	-0.6019067271, -0.6887363571, -0.7782028952, -0.8702784034, -0.9649652536, -1.062292565, -1.16231451, 
+	-1.265109911, -1.370782966, -1.479465113, -1.591317992, -1.70653767])
+
+## Formatting the Data
+
+The way the recurrent neural networks are trained in this program
+differs from how machine learning algorithms are usually trained.
+Typically a machine learning algorithm is trained by learning the
+relationship between the x data and the y data.  In this program, the
+recurrent neural network will be trained to recognize the relationship
+in a sequence of y values.  This is type of data formatting is
+typically used time series forcasting, but it can also be used in any
+extrapolation (time series forecasting is just a specific type of
+extrapolation along the time axis).  This method of data formatting
+does not use the x data and assumes that the y data are evenly spaced.
+
+For a standard machine learning algorithm, the training data has the
+form of (x,y) so the machine learning algorithm learns to assiciate a
+y value with a given x value.  This is useful when the test data has x
+values within the same range as the training data.  However, for this
+application, the x values of the test data are outside of the x values
+of the training data and the traditional method of training a machine
+learning algorithm does not work as well.  For this reason, the
+recurrent neural network is trained on sequences of y values of the
+form ((y1, y2), y3), so that the network is concerned with learning
+the pattern of the y data and not the relation between the x and y
+data.  As long as the pattern of y data outside of the training region
+stays relatively stable compared to what was inside the training
+region, this method of training can produce accurate extrapolations to
+y values far removed from the training data set.
+
+
+
+
+
+
+
+
+
+# FORMAT_DATA
+def format_data(data, length_of_sequence = 2):  
+    """
+        Inputs:
+            data(a numpy array): the data that will be the inputs to the recurrent neural
+                network
+            length_of_sequence (an int): the number of elements in one iteration of the
+                sequence patter.  For a function approximator use length_of_sequence = 2.
+        Returns:
+            rnn_input (a 3D numpy array): the input data for the recurrent neural network.  Its
+                dimensions are length of data - length of sequence, length of sequence, 
+                dimnsion of data
+            rnn_output (a numpy array): the training data for the neural network
+        Formats data to be used in a recurrent neural network.
+    """
+
+    X, Y = [], []
+    for i in range(len(data)-length_of_sequence):
+        # Get the next length_of_sequence elements
+        a = data[i:i+length_of_sequence]
+        # Get the element that immediately follows that
+        b = data[i+length_of_sequence]
+        # Reshape so that each data point is contained in its own array
+        a = np.reshape (a, (len(a), 1))
+        X.append(a)
+        Y.append(b)
+    rnn_input = np.array(X)
+    rnn_output = np.array(Y)
+
+    return rnn_input, rnn_output
+
+
+# ## Defining the Recurrent Neural Network Using Keras
+# 
+# The following method defines a simple recurrent neural network in keras consisting of one input layer, one hidden layer, and one output layer.
+
+def rnn(length_of_sequences, batch_size = None, stateful = False):
+    """
+        Inputs:
+            length_of_sequences (an int): the number of y values in "x data".  This is determined
+                when the data is formatted
+            batch_size (an int): Default value is None.  See Keras documentation of SimpleRNN.
+            stateful (a boolean): Default value is False.  See Keras documentation of SimpleRNN.
+        Returns:
+            model (a Keras model): The recurrent neural network that is built and compiled by this
+                method
+        Builds and compiles a recurrent neural network with one hidden layer and returns the model.
+    """
+    # Number of neurons in the input and output layers
+    in_out_neurons = 1
+    # Number of neurons in the hidden layer
+    hidden_neurons = 200
+    # Define the input layer
+    inp = Input(batch_shape=(batch_size, 
+                length_of_sequences, 
+                in_out_neurons))  
+    # Define the hidden layer as a simple RNN layer with a set number of neurons and add it to 
+    # the network immediately after the input layer
+    rnn = SimpleRNN(hidden_neurons, 
+                    return_sequences=False,
+                    stateful = stateful,
+                    name="RNN")(inp)
+    # Define the output layer as a dense neural network layer (standard neural network layer)
+    #and add it to the network immediately after the hidden layer.
+    dens = Dense(in_out_neurons,name="dense")(rnn)
+    # Create the machine learning model starting with the input layer and ending with the 
+    # output layer
+    model = Model(inputs=[inp],outputs=[dens])
+    # Compile the machine learning model using the mean squared error function as the loss 
+    # function and an Adams optimizer.
+    model.compile(loss="mean_squared_error", optimizer="adam")  
+    return model
+
+## Predicting New Points With A Trained Recurrent Neural Network
+
+def test_rnn (x1, y_test, plot_min, plot_max):
+    """
+        Inputs:
+            x1 (a list or numpy array): The complete x component of the data set
+            y_test (a list or numpy array): The complete y component of the data set
+            plot_min (an int or float): the smallest x value used in the training data
+            plot_max (an int or float): the largest x valye used in the training data
+        Returns:
+            None.
+        Uses a trained recurrent neural network model to predict future points in the 
+        series.  Computes the MSE of the predicted data set from the true data set, saves
+        the predicted data set to a csv file, and plots the predicted and true data sets w
+        while also displaying the data range used for training.
+    """
+    # Add the training data as the first dim points in the predicted data array as these
+    # are known values.
+    y_pred = y_test[:dim].tolist()
+    # Generate the first input to the trained recurrent neural network using the last two 
+    # points of the training data.  Based on how the network was trained this means that it
+    # will predict the first point in the data set after the training data.  All of the 
+    # brackets are necessary for Tensorflow.
+    next_input = np.array([[[y_test[dim-2]], [y_test[dim-1]]]])
+    # Save the very last point in the training data set.  This will be used later.
+    last = [y_test[dim-1]]
+
+    # Iterate until the complete data set is created.
+    for i in range (dim, len(y_test)):
+        # Predict the next point in the data set using the previous two points.
+        next = model.predict(next_input)
+        # Append just the number of the predicted data set
+        y_pred.append(next[0][0])
+        # Create the input that will be used to predict the next data point in the data set.
+        next_input = np.array([[last, next[0]]], dtype=np.float64)
+        last = next
+
+    # Print the mean squared error between the known data set and the predicted data set.
+    print('MSE: ', np.square(np.subtract(y_test, y_pred)).mean())
+    # Save the predicted data set as a csv file for later use
+    name = datatype + 'Predicted'+str(dim)+'.csv'
+    np.savetxt(name, y_pred, delimiter=',')
+    # Plot the known data set and the predicted data set.  The red box represents the region that was used
+    # for the training data.
+    fig, ax = plt.subplots()
+    ax.plot(x1, y_test, label="true", linewidth=3)
+    ax.plot(x1, y_pred, 'g-.',label="predicted", linewidth=4)
+    ax.legend()
+    # Created a red region to represent the points used in the training data.
+    ax.axvspan(plot_min, plot_max, alpha=0.25, color='red')
+    plt.show()
+
+# Check to make sure the data set is complete
+assert len(X_tot) == len(y_tot)
+
+# This is the number of points that will be used in as the training data
+dim=12
+
+# Separate the training data from the whole data set
+X_train = X_tot[:dim]
+y_train = y_tot[:dim]
+
+
+# Generate the training data for the RNN, using a sequence of 2
+rnn_input, rnn_training = format_data(y_train, 2)
+
+
+# Create a recurrent neural network in Keras and produce a summary of the 
+# machine learning model
+model = rnn(length_of_sequences = rnn_input.shape[1])
+model.summary()
+
+# Start the timer.  Want to time training+testing
+start = timer()
+# Fit the model using the training data genenerated above using 150 training iterations and a 5%
+# validation split.  Setting verbose to True prints information about each training iteration.
+hist = model.fit(rnn_input, rnn_training, batch_size=None, epochs=150, 
+                 verbose=True,validation_split=0.05)
+
+for label in ["loss","val_loss"]:
+    plt.plot(hist.history[label],label=label)
+
+plt.ylabel("loss")
+plt.xlabel("epoch")
+plt.title("The final validation loss: {}".format(hist.history["val_loss"][-1]))
+plt.legend()
+plt.show()
+
+# Use the trained neural network to predict more points of the data set
+test_rnn(X_tot, y_tot, X_tot[0], X_tot[dim-1])
+# Stop the timer and calculate the total time needed.
+end = timer()
+print('Time: ', end-start)
+
+## Other Things to Try
+
+
+Changing the size of the recurrent neural network and its parameters
+can drastically change the results you get from the model.  The below
+code takes the simple recurrent neural network from above and adds a
+second hidden layer, changes the number of neurons in the hidden
+layer, and explicitly declares the activation function of the hidden
+layers to be a sigmoid function.  The loss function and optimizer can
+also be changed but are kept the same as the above network.  These
+parameters can be tuned to provide the optimal result from the
+network.  For some ideas on how to improve the performance of a
+[recurrent neural network](https://danijar.com/tips-for-training-recurrent-neural-networks).
+
+def rnn_2layers(length_of_sequences, batch_size = None, stateful = False):
+    """
+        Inputs:
+            length_of_sequences (an int): the number of y values in "x data".  This is determined
+                when the data is formatted
+            batch_size (an int): Default value is None.  See Keras documentation of SimpleRNN.
+            stateful (a boolean): Default value is False.  See Keras documentation of SimpleRNN.
+        Returns:
+            model (a Keras model): The recurrent neural network that is built and compiled by this
+                method
+        Builds and compiles a recurrent neural network with two hidden layers and returns the model.
+    """
+    # Number of neurons in the input and output layers
+    in_out_neurons = 1
+    # Number of neurons in the hidden layer, increased from the first network
+    hidden_neurons = 500
+    # Define the input layer
+    inp = Input(batch_shape=(batch_size, 
+                length_of_sequences, 
+                in_out_neurons))  
+    # Create two hidden layers instead of one hidden layer.  Explicitly set the activation
+    # function to be the sigmoid function (the default value is hyperbolic tangent)
+    rnn1 = SimpleRNN(hidden_neurons, 
+                    return_sequences=True,  # This needs to be True if another hidden layer is to follow
+                    stateful = stateful, activation = 'sigmoid',
+                    name="RNN1")(inp)
+    rnn2 = SimpleRNN(hidden_neurons, 
+                    return_sequences=False, activation = 'sigmoid',
+                    stateful = stateful,
+                    name="RNN2")(rnn1)
+    # Define the output layer as a dense neural network layer (standard neural network layer)
+    #and add it to the network immediately after the hidden layer.
+    dens = Dense(in_out_neurons,name="dense")(rnn2)
+    # Create the machine learning model starting with the input layer and ending with the 
+    # output layer
+    model = Model(inputs=[inp],outputs=[dens])
+    # Compile the machine learning model using the mean squared error function as the loss 
+    # function and an Adams optimizer.
+    model.compile(loss="mean_squared_error", optimizer="adam")  
+    return model
+
+# Check to make sure the data set is complete
+assert len(X_tot) == len(y_tot)
+
+# This is the number of points that will be used in as the training data
+dim=12
+
+# Separate the training data from the whole data set
+X_train = X_tot[:dim]
+y_train = y_tot[:dim]
+
+
+# Generate the training data for the RNN, using a sequence of 2
+rnn_input, rnn_training = format_data(y_train, 2)
+
+
+# Create a recurrent neural network in Keras and produce a summary of the 
+# machine learning model
+model = rnn_2layers(length_of_sequences = 2)
+model.summary()
+
+# Start the timer.  Want to time training+testing
+start = timer()
+# Fit the model using the training data genenerated above using 150 training iterations and a 5%
+# validation split.  Setting verbose to True prints information about each training iteration.
+hist = model.fit(rnn_input, rnn_training, batch_size=None, epochs=150, 
+                 verbose=True,validation_split=0.05)
+
+
+# This section plots the training loss and the validation loss as a function of training iteration.
+# This is not required for analyzing the couple cluster data but can help determine if the network is
+# being overtrained.
+for label in ["loss","val_loss"]:
+    plt.plot(hist.history[label],label=label)
+
+plt.ylabel("loss")
+plt.xlabel("epoch")
+plt.title("The final validation loss: {}".format(hist.history["val_loss"][-1]))
+plt.legend()
+plt.show()
+
+# Use the trained neural network to predict more points of the data set
+test_rnn(X_tot, y_tot, X_tot[0], X_tot[dim-1])
+# Stop the timer and calculate the total time needed.
+end = timer()
+print('Time: ', end-start)
+
+## Other Types of Recurrent Neural Networks
+
+Besides a simple recurrent neural network layer, there are two other
+commonly used types of recurrent neural network layers: Long Short
+Term Memory (LSTM) and Gated Recurrent Unit (GRU).  For a short
+introduction to these layers see 
+and .
+
+The first network created below is similar to the previous network,
+but it replaces the SimpleRNN layers with LSTM layers.  The second
+network below has two hidden layers made up of GRUs, which are
+preceeded by two dense (feeddorward) neural network layers.  These
+dense layers "preprocess" the data before it reaches the recurrent
+layers.  This architecture has been shown to improve the performance
+of recurrent neural networks (see the link above and also
+.
+
+def lstm_2layers(length_of_sequences, batch_size = None, stateful = False):
+    """
+        Inputs:
+            length_of_sequences (an int): the number of y values in "x data".  This is determined
+                when the data is formatted
+            batch_size (an int): Default value is None.  See Keras documentation of SimpleRNN.
+            stateful (a boolean): Default value is False.  See Keras documentation of SimpleRNN.
+        Returns:
+            model (a Keras model): The recurrent neural network that is built and compiled by this
+                method
+        Builds and compiles a recurrent neural network with two LSTM hidden layers and returns the model.
+    """
+    # Number of neurons on the input/output layer and the number of neurons in the hidden layer
+    in_out_neurons = 1
+    hidden_neurons = 250
+    # Input Layer
+    inp = Input(batch_shape=(batch_size, 
+                length_of_sequences, 
+                in_out_neurons)) 
+    # Hidden layers (in this case they are LSTM layers instead if SimpleRNN layers)
+    rnn= LSTM(hidden_neurons, 
+                    return_sequences=True,
+                    stateful = stateful,
+                    name="RNN", use_bias=True, activation='tanh')(inp)
+    rnn1 = LSTM(hidden_neurons, 
+                    return_sequences=False,
+                    stateful = stateful,
+                    name="RNN1", use_bias=True, activation='tanh')(rnn)
+    # Output layer
+    dens = Dense(in_out_neurons,name="dense")(rnn1)
+    # Define the midel
+    model = Model(inputs=[inp],outputs=[dens])
+    # Compile the model
+    model.compile(loss='mean_squared_error', optimizer='adam')  
+    # Return the model
+    return model
+
+def dnn2_gru2(length_of_sequences, batch_size = None, stateful = False):
+    """
+        Inputs:
+            length_of_sequences (an int): the number of y values in "x data".  This is determined
+                when the data is formatted
+            batch_size (an int): Default value is None.  See Keras documentation of SimpleRNN.
+            stateful (a boolean): Default value is False.  See Keras documentation of SimpleRNN.
+        Returns:
+            model (a Keras model): The recurrent neural network that is built and compiled by this
+                method
+        Builds and compiles a recurrent neural network with four hidden layers (two dense followed by
+        two GRU layers) and returns the model.
+    """    
+    # Number of neurons on the input/output layers and hidden layers
+    in_out_neurons = 1
+    hidden_neurons = 250
+    # Input layer
+    inp = Input(batch_shape=(batch_size, 
+                length_of_sequences, 
+                in_out_neurons)) 
+    # Hidden Dense (feedforward) layers
+    dnn = Dense(hidden_neurons/2, activation='relu', name='dnn')(inp)
+    dnn1 = Dense(hidden_neurons/2, activation='relu', name='dnn1')(dnn)
+    # Hidden GRU layers
+    rnn1 = GRU(hidden_neurons, 
+                    return_sequences=True,
+                    stateful = stateful,
+                    name="RNN1", use_bias=True)(dnn1)
+    rnn = GRU(hidden_neurons, 
+                    return_sequences=False,
+                    stateful = stateful,
+                    name="RNN", use_bias=True)(rnn1)
+    # Output layer
+    dens = Dense(in_out_neurons,name="dense")(rnn)
+    # Define the model
+    model = Model(inputs=[inp],outputs=[dens])
+    # Compile the mdoel
+    model.compile(loss='mean_squared_error', optimizer='adam')  
+    # Return the model
+    return model
+
+# Check to make sure the data set is complete
+assert len(X_tot) == len(y_tot)
+
+# This is the number of points that will be used in as the training data
+dim=12
+
+# Separate the training data from the whole data set
+X_train = X_tot[:dim]
+y_train = y_tot[:dim]
+
+
+# Generate the training data for the RNN, using a sequence of 2
+rnn_input, rnn_training = format_data(y_train, 2)
+
+
+# Create a recurrent neural network in Keras and produce a summary of the 
+# machine learning model
+# Change the method name to reflect which network you want to use
+model = dnn2_gru2(length_of_sequences = 2)
+model.summary()
+
+# Start the timer.  Want to time training+testing
+start = timer()
+# Fit the model using the training data genenerated above using 150 training iterations and a 5%
+# validation split.  Setting verbose to True prints information about each training iteration.
+hist = model.fit(rnn_input, rnn_training, batch_size=None, epochs=150, 
+                 verbose=True,validation_split=0.05)
+
+
+# This section plots the training loss and the validation loss as a function of training iteration.
+# This is not required for analyzing the couple cluster data but can help determine if the network is
+# being overtrained.
+for label in ["loss","val_loss"]:
+    plt.plot(hist.history[label],label=label)
+
+plt.ylabel("loss")
+plt.xlabel("epoch")
+plt.title("The final validation loss: {}".format(hist.history["val_loss"][-1]))
+plt.legend()
+plt.show()
+
+# Use the trained neural network to predict more points of the data set
+test_rnn(X_tot, y_tot, X_tot[0], X_tot[dim-1])
+# Stop the timer and calculate the total time needed.
+end = timer()
+print('Time: ', end-start)
+
+
+# ### Training Recurrent Neural Networks in the Standard Way (i.e. learning the relationship between the X and Y data)
+# 
+# Finally, comparing the performace of a recurrent neural network using the standard data formatting to the performance of the network with time sequence data formatting shows the benefit of this type of data formatting with extrapolation.
+
+# Check to make sure the data set is complete
+assert len(X_tot) == len(y_tot)
+
+# This is the number of points that will be used in as the training data
+dim=12
+
+# Separate the training data from the whole data set
+X_train = X_tot[:dim]
+y_train = y_tot[:dim]
+
+# Reshape the data for Keras specifications
+X_train = X_train.reshape((dim, 1))
+y_train = y_train.reshape((dim, 1))
+
+
+# Create a recurrent neural network in Keras and produce a summary of the 
+# machine learning model
+# Set the sequence length to 1 for regular data formatting 
+model = rnn(length_of_sequences = 1)
+model.summary()
+
+# Start the timer.  Want to time training+testing
+start = timer()
+# Fit the model using the training data genenerated above using 150 training iterations and a 5%
+# validation split.  Setting verbose to True prints information about each training iteration.
+hist = model.fit(X_train, y_train, batch_size=None, epochs=150, 
+                 verbose=True,validation_split=0.05)
+
+
+# This section plots the training loss and the validation loss as a function of training iteration.
+# This is not required for analyzing the couple cluster data but can help determine if the network is
+# being overtrained.
+for label in ["loss","val_loss"]:
+    plt.plot(hist.history[label],label=label)
+
+plt.ylabel("loss")
+plt.xlabel("epoch")
+plt.title("The final validation loss: {}".format(hist.history["val_loss"][-1]))
+plt.legend()
+plt.show()
+
+# Use the trained neural network to predict the remaining data points
+X_pred = X_tot[dim:]
+X_pred = X_pred.reshape((len(X_pred), 1))
+y_model = model.predict(X_pred)
+y_pred = np.concatenate((y_tot[:dim], y_model.flatten()))
+
+# Plot the known data set and the predicted data set.  The red box represents the region that was used
+# for the training data.
+fig, ax = plt.subplots()
+ax.plot(X_tot, y_tot, label="true", linewidth=3)
+ax.plot(X_tot, y_pred, 'g-.',label="predicted", linewidth=4)
+ax.legend()
+# Created a red region to represent the points used in the training data.
+ax.axvspan(X_tot[0], X_tot[dim], alpha=0.25, color='red')
+plt.show()
+
+# Stop the timer and calculate the total time needed.
+end = timer()
+print('Time: ', end-start)
+
+# Solving ODEs with Deep Learning
+
+The Universal Approximation Theorem states that a neural network can
+approximate any function at a single hidden layer along with one input
+and output layer to any given precision.  
+
+
+
+## Ordinary Differential Equations
+
+An ordinary differential equation (ODE) is an equation involving functions having one variable.
+
+In general, an ordinary differential equation looks like
+
+
+
    
+
+$$
+\begin{equation} \label{ode} \tag{1}
+f\left(x, \, g(x), \, g'(x), \, g''(x), \, \dots \, , \, g^{(n)}(x)\right) = 0
+\end{equation}
+$$
+
+where $g(x)$ is the function to find, and $g^{(n)}(x)$ is the $n$-th derivative of $g(x)$.
+
+The $f\left(x, g(x), g'(x), g''(x), \, \dots \, , g^{(n)}(x)\right)$ is just a way to write that there is an expression involving $x$ and $g(x), \ g'(x), \ g''(x), \, \dots \, , \text{ and } g^{(n)}(x)$ on the left side of the equality sign in ([1](#ode)).
+The highest order of derivative, that is the value of $n$, determines to the order of the equation.
+The equation is referred to as a $n$-th order ODE.
+Along with ([1](#ode)), some additional conditions of the function $g(x)$ are typically given
+for the solution to be unique.
+
+
+## The trial solution
+
+Let the trial solution $g_t(x)$ be
+
+
+
    
+
+$$
+\begin{equation}
+	g_t(x) = h_1(x) + h_2(x,N(x,P))
+\label{_auto1} \tag{2}
+\end{equation}
+$$
+
+where $h_1(x)$ is a function that makes $g_t(x)$ satisfy a given set
+of conditions, $N(x,P)$ a neural network with weights and biases
+described by $P$ and $h_2(x, N(x,P))$ some expression involving the
+neural network.  The role of the function $h_2(x, N(x,P))$, is to
+ensure that the output from $N(x,P)$ is zero when $g_t(x)$ is
+evaluated at the values of $x$ where the given conditions must be
+satisfied.  The function $h_1(x)$ should alone make $g_t(x)$ satisfy
+the conditions.
+
+But what about the network $N(x,P)$?
+
+
+As described previously, an optimization method could be used to minimize the parameters of a neural network, that being its weights and biases, through backward propagation.
+
+
+
+## Minimization process
+
+For the minimization to be defined, we need to have a cost function at hand to minimize.
+
+It is given that $f\left(x, \, g(x), \, g'(x), \, g''(x), \, \dots \, , \, g^{(n)}(x)\right)$ should be equal to zero in ([1](#ode)).
+We can choose to consider the mean squared error as the cost function for an input $x$.
+Since we are looking at one input, the cost function is just $f$ squared.
+The cost function $c\left(x, P \right)$ can therefore be expressed as
+
+$$
+C\left(x, P\right) = \big(f\left(x, \, g(x), \, g'(x), \, g''(x), \, \dots \, , \, g^{(n)}(x)\right)\big)^2
+$$
+
+If $N$ inputs are given as a vector $\boldsymbol{x}$ with elements $x_i$ for $i = 1,\dots,N$,
+the cost function becomes
+
+
+
    
+
+$$
+\begin{equation} \label{cost} \tag{3}
+	C\left(\boldsymbol{x}, P\right) = \frac{1}{N} \sum_{i=1}^N \big(f\left(x_i, \, g(x_i), \, g'(x_i), \, g''(x_i), \, \dots \, , \, g^{(n)}(x_i)\right)\big)^2
+\end{equation}
+$$
+
+The neural net should then find the parameters $P$ that minimizes the cost function in
+([3](#cost)) for a set of $N$ training samples $x_i$.
+
+
+## Minimizing the cost function using gradient descent and automatic differentiation
+
+To perform the minimization using gradient descent, the gradient of $C\left(\boldsymbol{x}, P\right)$ is needed.
+It might happen so that finding an analytical expression of the gradient of $C(\boldsymbol{x}, P)$ from ([3](#cost)) gets too messy, depending on which cost function one desires to use.
+
+Luckily, there exists libraries that makes the job for us through automatic differentiation.
+Automatic differentiation is a method of finding the derivatives numerically with very high precision.
+
+
+
+## Example: Exponential decay
+
+An exponential decay of a quantity $g(x)$ is described by the equation
+
+
+
    
+
+$$
+\begin{equation} \label{solve_expdec} \tag{4}
+  g'(x) = -\gamma g(x)
+\end{equation}
+$$
+
+with $g(0) = g_0$ for some chosen initial value $g_0$.
+
+The analytical solution of ([4](#solve_expdec)) is
+
+
+
    
+
+$$
+\begin{equation}
+  g(x) = g_0 \exp\left(-\gamma x\right)
+\label{_auto2} \tag{5}
+\end{equation}
+$$
+
+Having an analytical solution at hand, it is possible to use it to compare how well a neural network finds a solution of ([4](#solve_expdec)).
+
+
+
+## The function to solve for
+
+The program will use a neural network to solve
+
+
+
    
+
+$$
+\begin{equation} \label{solveode} \tag{6}
+g'(x) = -\gamma g(x)
+\end{equation}
+$$
+
+where $g(0) = g_0$ with $\gamma$ and $g_0$ being some chosen values.
+
+In this example, $\gamma = 2$ and $g_0 = 10$.
+
+
+## The trial solution
+To begin with, a trial solution $g_t(t)$ must be chosen. A general trial solution for ordinary differential equations could be
+
+$$
+g_t(x, P) = h_1(x) + h_2(x, N(x, P))
+$$
+
+with $h_1(x)$ ensuring that $g_t(x)$ satisfies some conditions and $h_2(x,N(x, P))$ an expression involving $x$ and the output from the neural network $N(x,P)$ with $P $ being the collection of the weights and biases for each layer. For now, it is assumed that the network consists of one input layer, one hidden layer, and one output layer.
+
+
+## Setup of Network
+
+In this network, there are no weights and bias at the input layer, so $P = \{ P_{\text{hidden}},  P_{\text{output}} \}$.
+If there are $N_{\text{hidden} }$ neurons in the hidden layer, then $P_{\text{hidden}}$ is a $N_{\text{hidden} } \times (1 + N_{\text{input}})$ matrix, given that there are $N_{\text{input}}$ neurons in the input layer.
+
+The first column in $P_{\text{hidden} }$ represents the bias for each neuron in the hidden layer and the second column represents the weights for each neuron in the hidden layer from the input layer.
+If there are $N_{\text{output} }$ neurons in the output layer, then $P_{\text{output}} $ is a $N_{\text{output} } \times (1 + N_{\text{hidden} })$ matrix.
+
+Its first column represents the bias of each neuron and the remaining columns represents the weights to each neuron.
+
+It is given that $g(0) = g_0$. The trial solution must fulfill this condition to be a proper solution of ([6](#solveode)). A possible way to ensure that $g_t(0, P) = g_0$, is to let $F(N(x,P)) = x \cdot N(x,P)$ and $A(x) = g_0$. This gives the following trial solution:
+
+
+
    
+
+$$
+\begin{equation} \label{trial} \tag{7}
+g_t(x, P) = g_0 + x \cdot N(x, P)
+\end{equation}
+$$
+
+## Reformulating the problem
+
+We wish that our neural network manages to minimize a given cost function.
+
+A reformulation of out equation, ([6](#solveode)), must therefore be done,
+such that it describes the problem a neural network can solve for.
+
+The neural network must find the set of weights and biases $P$ such that the trial solution in ([7](#trial)) satisfies ([6](#solveode)).
+
+The trial solution
+
+$$
+g_t(x, P) = g_0 + x \cdot N(x, P)
+$$
+
+has been chosen such that it already solves the condition $g(0) = g_0$. What remains, is to find $P$ such that
+
+
+
    
+
+$$
+\begin{equation} \label{nnmin} \tag{8}
+g_t'(x, P) = - \gamma g_t(x, P)
+\end{equation}
+$$
+
+is fulfilled as *best as possible*.
+
+
+## More technicalities
+
+The left hand side and right hand side of ([8](#nnmin)) must be computed separately, and then the neural network must choose weights and biases, contained in $P$, such that the sides are equal as best as possible.
+This means that the absolute or squared difference between the sides must be as close to zero, ideally equal to zero.
+In this case, the difference squared shows to be an appropriate measurement of how erroneous the trial solution is with respect to $P$ of the neural network.
+
+This gives the following cost function our neural network must solve for:
+
+$$
+\min_{P}\Big\{ \big(g_t'(x, P) - ( -\gamma g_t(x, P) \big)^2 \Big\}
+$$
+
+(the notation $\min_{P}\{ f(x, P) \}$ means that we desire to find $P$ that yields the minimum of $f(x, P)$)
+
+or, in terms of weights and biases for the hidden and output layer in our network:
+
+$$
+\min_{P_{\text{hidden} }, \ P_{\text{output} }}\Big\{ \big(g_t'(x, \{ P_{\text{hidden} }, P_{\text{output} }\}) - ( -\gamma g_t(x, \{ P_{\text{hidden} }, P_{\text{output} }\}) \big)^2 \Big\}
+$$
+
+for an input value $x$.
+
+
+## More details
+
+If the neural network evaluates $g_t(x, P)$ at more values for $x$, say $N$ values $x_i$ for $i = 1, \dots, N$, then the *total* error to minimize becomes
+
+
+
    
+
+$$
+\begin{equation} \label{min} \tag{9}
+\min_{P}\Big\{\frac{1}{N} \sum_{i=1}^N  \big(g_t'(x_i, P) - ( -\gamma g_t(x_i, P) \big)^2 \Big\}
+\end{equation}
+$$
+
+Letting $\boldsymbol{x}$ be a vector with elements $x_i$ and $C(\boldsymbol{x}, P) = \frac{1}{N} \sum_i  \big(g_t'(x_i, P) - ( -\gamma g_t(x_i, P) \big)^2$ denote the cost function, the minimization problem that our network must solve, becomes
+
+$$
+\min_{P} C(\boldsymbol{x}, P)
+$$
+
+In terms of $P_{\text{hidden} }$ and $P_{\text{output} }$, this could also be expressed as
+
+$$
+\min_{P_{\text{hidden} }, \ P_{\text{output} }} C(\boldsymbol{x}, \{P_{\text{hidden} }, P_{\text{output} }\})
+$$
+
+
+## A possible implementation of a neural network
+
+For simplicity, it is assumed that the input is an array $\boldsymbol{x} = (x_1, \dots, x_N)$ with $N$ elements. It is at these points the neural network should find $P$ such that it fulfills ([9](#min)).
+
+First, the neural network must feed forward the inputs.
+This means that $\boldsymbol{x}s$ must be passed through an input layer, a hidden layer and a output layer. The input layer in this case, does not need to process the data any further.
+The input layer will consist of $N_{\text{input} }$ neurons, passing its element to each neuron in the hidden layer.  The number of neurons in the hidden layer will be $N_{\text{hidden} }$.
+
+
+## Technicalities
+
+For the $i$-th in the hidden layer with weight $w_i^{\text{hidden} }$ and bias $b_i^{\text{hidden} }$, the weighting from the $j$-th neuron at the input layer is:
+
+$$
+\begin{aligned}
+z_{i,j}^{\text{hidden}} &= b_i^{\text{hidden}} + w_i^{\text{hidden}}x_j \\
+&=
+\begin{pmatrix}
+b_i^{\text{hidden}} & w_i^{\text{hidden}}
+\end{pmatrix}
+\begin{pmatrix}
+1 \\
+x_j
+\end{pmatrix}
+\end{aligned}
+$$
+
+## Final technicalities I
+
+The result after weighting the inputs at the $i$-th hidden neuron can be written as a vector:
+
+$$
+\begin{aligned}
+\boldsymbol{z}_{i}^{\text{hidden}} &= \Big( b_i^{\text{hidden}} + w_i^{\text{hidden}}x_1 , \ b_i^{\text{hidden}} + w_i^{\text{hidden}} x_2, \ \dots \, , \ b_i^{\text{hidden}} + w_i^{\text{hidden}} x_N\Big)  \\
+&=
+\begin{pmatrix}
+ b_i^{\text{hidden}}  & w_i^{\text{hidden}}
+\end{pmatrix}
+\begin{pmatrix}
+1  & 1 & \dots & 1 \\
+x_1 & x_2 & \dots & x_N
+\end{pmatrix} \\
+&= \boldsymbol{p}_{i, \text{hidden}}^T X
+\end{aligned}
+$$
+
+## Final technicalities II
+
+The vector $\boldsymbol{p}_{i, \text{hidden}}^T$ constitutes each row in $P_{\text{hidden} }$, which contains the weights for the neural network to minimize according to ([9](#min)).
+
+After having found $\boldsymbol{z}_{i}^{\text{hidden}} $ for every $i$-th neuron within the hidden layer, the vector will be sent to an activation function $a_i(\boldsymbol{z})$.
+
+In this example, the sigmoid function has been chosen to be the activation function for each hidden neuron:
+
+$$
+f(z) = \frac{1}{1 + \exp{(-z)}}
+$$
+
+It is possible to use other activations functions for the hidden layer also.
+
+The output $\boldsymbol{x}_i^{\text{hidden}}$ from each $i$-th hidden neuron is:
+
+$$
+\boldsymbol{x}_i^{\text{hidden} } = f\big(  \boldsymbol{z}_{i}^{\text{hidden}} \big)
+$$
+
+The outputs $\boldsymbol{x}_i^{\text{hidden} } $ are then sent to the output layer.
+
+The output layer consists of one neuron in this case, and combines the
+output from each of the neurons in the hidden layers. The output layer
+combines the results from the hidden layer using some weights $w_i^{\text{output}}$
+and biases $b_i^{\text{output}}$. In this case,
+it is assumes that the number of neurons in the output layer is one.
+
+
+## Final technicalities III
+
+
+The procedure of weighting the output neuron $j$ in the hidden layer to the $i$-th neuron in the output layer is similar as for the hidden layer described previously.
+
+$$
+\begin{aligned}
+z_{1,j}^{\text{output}} & =
+\begin{pmatrix}
+b_1^{\text{output}} & \boldsymbol{w}_1^{\text{output}}
+\end{pmatrix}
+\begin{pmatrix}
+1 \\
+\boldsymbol{x}_j^{\text{hidden}}
+\end{pmatrix}
+\end{aligned}
+$$
+
+## Final technicalities IV
+
+Expressing $z_{1,j}^{\text{output}}$ as a vector gives the following way of weighting the inputs from the hidden layer:
+
+$$
+\boldsymbol{z}_{1}^{\text{output}} =
+\begin{pmatrix}
+b_1^{\text{output}} & \boldsymbol{w}_1^{\text{output}}
+\end{pmatrix}
+\begin{pmatrix}
+1  & 1 & \dots & 1 \\
+\boldsymbol{x}_1^{\text{hidden}} & \boldsymbol{x}_2^{\text{hidden}} & \dots & \boldsymbol{x}_N^{\text{hidden}}
+\end{pmatrix}
+$$
+
+In this case we seek a continuous range of values since we are approximating a function. This means that after computing $\boldsymbol{z}_{1}^{\text{output}}$ the neural network has finished its feed forward step, and $\boldsymbol{z}_{1}^{\text{output}}$ is the final output of the network.
+
+
+## Back propagation
+
+The next step is to decide how the parameters should be changed such that they minimize the cost function.
+
+The chosen cost function for this problem is
+
+$$
+C(\boldsymbol{x}, P) = \frac{1}{N} \sum_i  \big(g_t'(x_i, P) - ( -\gamma g_t(x_i, P) \big)^2
+$$
+
+In order to minimize the cost function, an optimization method must be chosen.
+
+Here, gradient descent with a constant step size has been chosen.
+
+
+## Gradient descent
+
+The idea of the gradient descent algorithm is to update parameters in
+a direction where the cost function decreases goes to a minimum.
+
+In general, the update of some parameters $\boldsymbol{\omega}$ given a cost
+function defined by some weights $\boldsymbol{\omega}$, $C(\boldsymbol{x},
+\boldsymbol{\omega})$, goes as follows:
+
+$$
+\boldsymbol{\omega}_{\text{new} } = \boldsymbol{\omega} - \lambda \nabla_{\boldsymbol{\omega}} C(\boldsymbol{x}, \boldsymbol{\omega})
+$$
+
+for a number of iterations or until $ \big|\big| \boldsymbol{\omega}_{\text{new} } - \boldsymbol{\omega} \big|\big|$ becomes smaller than some given tolerance.
+
+The value of $\lambda$ decides how large steps the algorithm must take
+in the direction of $ \nabla_{\boldsymbol{\omega}} C(\boldsymbol{x}, \boldsymbol{\omega})$.
+The notation $\nabla_{\boldsymbol{\omega}}$ express the gradient with respect
+to the elements in $\boldsymbol{\omega}$.
+
+In our case, we have to minimize the cost function $C(\boldsymbol{x}, P)$ with
+respect to the two sets of weights and biases, that is for the hidden
+layer $P_{\text{hidden} }$ and for the output layer $P_{\text{output}
+}$ .
+
+This means that $P_{\text{hidden} }$ and $P_{\text{output} }$ is updated by
+
+$$
+\begin{aligned}
+P_{\text{hidden},\text{new}} &= P_{\text{hidden}} - \lambda \nabla_{P_{\text{hidden}}} C(\boldsymbol{x}, P)  \\
+P_{\text{output},\text{new}} &= P_{\text{output}} - \lambda \nabla_{P_{\text{output}}} C(\boldsymbol{x}, P)
+\end{aligned}
+$$
+
+## The code for solving the ODE
+
+import autograd.numpy as np
+from autograd import grad, elementwise_grad
+import autograd.numpy.random as npr
+from matplotlib import pyplot as plt
+
+def sigmoid(z):
+    return 1/(1 + np.exp(-z))
+
+# Assuming one input, hidden, and output layer
+def neural_network(params, x):
+
+    # Find the weights (including and biases) for the hidden and output layer.
+    # Assume that params is a list of parameters for each layer.
+    # The biases are the first element for each array in params,
+    # and the weights are the remaning elements in each array in params.
+
+    w_hidden = params[0]
+    w_output = params[1]
+
+    # Assumes input x being an one-dimensional array
+    num_values = np.size(x)
+    x = x.reshape(-1, num_values)
+
+    # Assume that the input layer does nothing to the input x
+    x_input = x
+
+    ## Hidden layer:
+
+    # Add a row of ones to include bias
+    x_input = np.concatenate((np.ones((1,num_values)), x_input ), axis = 0)
+
+    z_hidden = np.matmul(w_hidden, x_input)
+    x_hidden = sigmoid(z_hidden)
+
+    ## Output layer:
+
+    # Include bias:
+    x_hidden = np.concatenate((np.ones((1,num_values)), x_hidden ), axis = 0)
+
+    z_output = np.matmul(w_output, x_hidden)
+    x_output = z_output
+
+    return x_output
+
+# The trial solution using the deep neural network:
+def g_trial(x,params, g0 = 10):
+    return g0 + x*neural_network(params,x)
+
+# The right side of the ODE:
+def g(x, g_trial, gamma = 2):
+    return -gamma*g_trial
+
+# The cost function:
+def cost_function(P, x):
+
+    # Evaluate the trial function with the current parameters P
+    g_t = g_trial(x,P)
+
+    # Find the derivative w.r.t x of the neural network
+    d_net_out = elementwise_grad(neural_network,1)(P,x)
+
+    # Find the derivative w.r.t x of the trial function
+    d_g_t = elementwise_grad(g_trial,0)(x,P)
+
+    # The right side of the ODE
+    func = g(x, g_t)
+
+    err_sqr = (d_g_t - func)**2
+    cost_sum = np.sum(err_sqr)
+
+    return cost_sum / np.size(err_sqr)
+
+# Solve the exponential decay ODE using neural network with one input, hidden, and output layer
+def solve_ode_neural_network(x, num_neurons_hidden, num_iter, lmb):
+    ## Set up initial weights and biases
+
+    # For the hidden layer
+    p0 = npr.randn(num_neurons_hidden, 2 )
+
+    # For the output layer
+    p1 = npr.randn(1, num_neurons_hidden + 1 ) # +1 since bias is included
+
+    P = [p0, p1]
+
+    print('Initial cost: %g'%cost_function(P, x))
+
+    ## Start finding the optimal weights using gradient descent
+
+    # Find the Python function that represents the gradient of the cost function
+    # w.r.t the 0-th input argument -- that is the weights and biases in the hidden and output layer
+    cost_function_grad = grad(cost_function,0)
+
+    # Let the update be done num_iter times
+    for i in range(num_iter):
+        # Evaluate the gradient at the current weights and biases in P.
+        # The cost_grad consist now of two arrays;
+        # one for the gradient w.r.t P_hidden and
+        # one for the gradient w.r.t P_output
+        cost_grad =  cost_function_grad(P, x)
+
+        P[0] = P[0] - lmb * cost_grad[0]
+        P[1] = P[1] - lmb * cost_grad[1]
+
+    print('Final cost: %g'%cost_function(P, x))
+
+    return P
+
+def g_analytic(x, gamma = 2, g0 = 10):
+    return g0*np.exp(-gamma*x)
+
+# Solve the given problem
+if __name__ == '__main__':
+    # Set seed such that the weight are initialized
+    # with same weights and biases for every run.
+    npr.seed(15)
+
+    ## Decide the vales of arguments to the function to solve
+    N = 10
+    x = np.linspace(0, 1, N)
+
+    ## Set up the initial parameters
+    num_hidden_neurons = 10
+    num_iter = 10000
+    lmb = 0.001
+
+    # Use the network
+    P = solve_ode_neural_network(x, num_hidden_neurons, num_iter, lmb)
+
+    # Print the deviation from the trial solution and true solution
+    res = g_trial(x,P)
+    res_analytical = g_analytic(x)
+
+    print('Max absolute difference: %g'%np.max(np.abs(res - res_analytical)))
+
+    # Plot the results
+    plt.figure(figsize=(10,10))
+
+    plt.title('Performance of neural network solving an ODE compared to the analytical solution')
+    plt.plot(x, res_analytical)
+    plt.plot(x, res[0,:])
+    plt.legend(['analytical','nn'])
+    plt.xlabel('x')
+    plt.ylabel('g(x)')
+    plt.show()
+
+## The network with one input layer, specified number of hidden layers, and one output layer
+
+It is also possible to extend the construction of our network into a more general one, allowing the network to contain more than one hidden layers.
+
+The number of neurons within each hidden layer are given as a list of integers in the program below.
+
+import autograd.numpy as np
+from autograd import grad, elementwise_grad
+import autograd.numpy.random as npr
+from matplotlib import pyplot as plt
+
+def sigmoid(z):
+    return 1/(1 + np.exp(-z))
+
+# The neural network with one input layer and one output layer,
+# but with number of hidden layers specified by the user.
+def deep_neural_network(deep_params, x):
+    # N_hidden is the number of hidden layers
+
+    N_hidden = np.size(deep_params) - 1 # -1 since params consists of
+                                        # parameters to all the hidden
+                                        # layers AND the output layer.
+
+    # Assumes input x being an one-dimensional array
+    num_values = np.size(x)
+    x = x.reshape(-1, num_values)
+
+    # Assume that the input layer does nothing to the input x
+    x_input = x
+
+    # Due to multiple hidden layers, define a variable referencing to the
+    # output of the previous layer:
+    x_prev = x_input
+
+    ## Hidden layers:
+
+    for l in range(N_hidden):
+        # From the list of parameters P; find the correct weigths and bias for this layer
+        w_hidden = deep_params[l]
+
+        # Add a row of ones to include bias
+        x_prev = np.concatenate((np.ones((1,num_values)), x_prev ), axis = 0)
+
+        z_hidden = np.matmul(w_hidden, x_prev)
+        x_hidden = sigmoid(z_hidden)
+
+        # Update x_prev such that next layer can use the output from this layer
+        x_prev = x_hidden
+
+    ## Output layer:
+
+    # Get the weights and bias for this layer
+    w_output = deep_params[-1]
+
+    # Include bias:
+    x_prev = np.concatenate((np.ones((1,num_values)), x_prev), axis = 0)
+
+    z_output = np.matmul(w_output, x_prev)
+    x_output = z_output
+
+    return x_output
+
+# The trial solution using the deep neural network:
+def g_trial_deep(x,params, g0 = 10):
+    return g0 + x*deep_neural_network(params, x)
+
+# The right side of the ODE:
+def g(x, g_trial, gamma = 2):
+    return -gamma*g_trial
+
+# The same cost function as before, but calls deep_neural_network instead.
+def cost_function_deep(P, x):
+
+    # Evaluate the trial function with the current parameters P
+    g_t = g_trial_deep(x,P)
+
+    # Find the derivative w.r.t x of the neural network
+    d_net_out = elementwise_grad(deep_neural_network,1)(P,x)
+
+    # Find the derivative w.r.t x of the trial function
+    d_g_t = elementwise_grad(g_trial_deep,0)(x,P)
+
+    # The right side of the ODE
+    func = g(x, g_t)
+
+    err_sqr = (d_g_t - func)**2
+    cost_sum = np.sum(err_sqr)
+
+    return cost_sum / np.size(err_sqr)
+
+# Solve the exponential decay ODE using neural network with one input and one output layer,
+# but with specified number of hidden layers from the user.
+def solve_ode_deep_neural_network(x, num_neurons, num_iter, lmb):
+    # num_hidden_neurons is now a list of number of neurons within each hidden layer
+
+    # The number of elements in the list num_hidden_neurons thus represents
+    # the number of hidden layers.
+
+    # Find the number of hidden layers:
+    N_hidden = np.size(num_neurons)
+
+    ## Set up initial weights and biases
+
+    # Initialize the list of parameters:
+    P = [None]*(N_hidden + 1) # + 1 to include the output layer
+
+    P[0] = npr.randn(num_neurons[0], 2 )
+    for l in range(1,N_hidden):
+        P[l] = npr.randn(num_neurons[l], num_neurons[l-1] + 1) # +1 to include bias
+
+    # For the output layer
+    P[-1] = npr.randn(1, num_neurons[-1] + 1 ) # +1 since bias is included
+
+    print('Initial cost: %g'%cost_function_deep(P, x))
+
+    ## Start finding the optimal weights using gradient descent
+
+    # Find the Python function that represents the gradient of the cost function
+    # w.r.t the 0-th input argument -- that is the weights and biases in the hidden and output layer
+    cost_function_deep_grad = grad(cost_function_deep,0)
+
+    # Let the update be done num_iter times
+    for i in range(num_iter):
+        # Evaluate the gradient at the current weights and biases in P.
+        # The cost_grad consist now of N_hidden + 1 arrays; the gradient w.r.t the weights and biases
+        # in the hidden layers and output layers evaluated at x.
+        cost_deep_grad =  cost_function_deep_grad(P, x)
+
+        for l in range(N_hidden+1):
+            P[l] = P[l] - lmb * cost_deep_grad[l]
+
+    print('Final cost: %g'%cost_function_deep(P, x))
+
+    return P
+
+def g_analytic(x, gamma = 2, g0 = 10):
+    return g0*np.exp(-gamma*x)
+
+# Solve the given problem
+if __name__ == '__main__':
+    npr.seed(15)
+
+    ## Decide the vales of arguments to the function to solve
+    N = 10
+    x = np.linspace(0, 1, N)
+
+    ## Set up the initial parameters
+    num_hidden_neurons = np.array([10,10])
+    num_iter = 10000
+    lmb = 0.001
+
+    P = solve_ode_deep_neural_network(x, num_hidden_neurons, num_iter, lmb)
+
+    res = g_trial_deep(x,P)
+    res_analytical = g_analytic(x)
+
+    plt.figure(figsize=(10,10))
+
+    plt.title('Performance of a deep neural network solving an ODE compared to the analytical solution')
+    plt.plot(x, res_analytical)
+    plt.plot(x, res[0,:])
+    plt.legend(['analytical','dnn'])
+    plt.ylabel('g(x)')
+    plt.show()
+
+## Example: Population growth
+
+A logistic model of population growth assumes that a population converges toward an equilibrium.
+The population growth can be modeled by
+
+
+
    
+
+$$
+\begin{equation} \label{log} \tag{10}
+	g'(t) = \alpha g(t)(A - g(t))
+\end{equation}
+$$
+
+where $g(t)$ is the population density at time $t$, $\alpha > 0$ the growth rate and $A > 0$ is the maximum population number in the environment.
+Also, at $t = 0$ the population has the size $g(0) = g_0$, where $g_0$ is some chosen constant.
+
+In this example, similar network as for the exponential decay using Autograd has been used to solve the equation. However, as the implementation might suffer from e.g numerical instability
+and high execution time (this might be more apparent in the examples solving PDEs),
+using a library like  TensorFlow is recommended.
+Here, we stay with a more simple approach and implement for comparison, the simple forward Euler method.
+
+
+## Setting up the problem
+
+Here, we will model a population $g(t)$ in an environment having carrying capacity $A$.
+The population follows the model
+
+
+
    
+
+$$
+\begin{equation} \label{solveode_population} \tag{11}
+g'(t) = \alpha g(t)(A - g(t))
+\end{equation}
+$$
+
+where $g(0) = g_0$.
+
+In this example, we let $\alpha = 2$, $A = 1$, and $g_0 = 1.2$.
+
+
+## The trial solution
+
+We will get a slightly different trial solution, as the boundary conditions are different
+compared to the case for exponential decay.
+
+A possible trial solution satisfying the condition $g(0) = g_0$ could be
+
+$$
+h_1(t) = g_0 + t \cdot N(t,P)
+$$
+
+with $N(t,P)$ being the output from the neural network with weights and biases for each layer collected in the set $P$.
+
+The analytical solution is
+
+$$
+g(t) = \frac{Ag_0}{g_0 + (A - g_0)\exp(-\alpha A t)}
+$$
+
+
+## The program using Autograd
+
+The network will be the similar as for the exponential decay example, but with some small modifications for our problem.
+
+import autograd.numpy as np
+from autograd import grad, elementwise_grad
+import autograd.numpy.random as npr
+from matplotlib import pyplot as plt
+
+def sigmoid(z):
+    return 1/(1 + np.exp(-z))
+
+# Function to get the parameters.
+# Done such that one can easily change the paramaters after one's liking.
+def get_parameters():
+    alpha = 2
+    A = 1
+    g0 = 1.2
+    return alpha, A, g0
+
+def deep_neural_network(P, x):
+    # N_hidden is the number of hidden layers
+    N_hidden = np.size(P) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer
+
+    # Assumes input x being an one-dimensional array
+    num_values = np.size(x)
+    x = x.reshape(-1, num_values)
+
+    # Assume that the input layer does nothing to the input x
+    x_input = x
+
+    # Due to multiple hidden layers, define a variable referencing to the
+    # output of the previous layer:
+    x_prev = x_input
+
+    ## Hidden layers:
+
+    for l in range(N_hidden):
+        # From the list of parameters P; find the correct weigths and bias for this layer
+        w_hidden = P[l]
+
+        # Add a row of ones to include bias
+        x_prev = np.concatenate((np.ones((1,num_values)), x_prev ), axis = 0)
+
+        z_hidden = np.matmul(w_hidden, x_prev)
+        x_hidden = sigmoid(z_hidden)
+
+        # Update x_prev such that next layer can use the output from this layer
+        x_prev = x_hidden
+
+    ## Output layer:
+
+    # Get the weights and bias for this layer
+    w_output = P[-1]
+
+    # Include bias:
+    x_prev = np.concatenate((np.ones((1,num_values)), x_prev), axis = 0)
+
+    z_output = np.matmul(w_output, x_prev)
+    x_output = z_output
+
+    return x_output
+
+
+def cost_function_deep(P, x):
+
+    # Evaluate the trial function with the current parameters P
+    g_t = g_trial_deep(x,P)
+
+    # Find the derivative w.r.t x of the trial function
+    d_g_t = elementwise_grad(g_trial_deep,0)(x,P)
+
+    # The right side of the ODE
+    func = f(x, g_t)
+
+    err_sqr = (d_g_t - func)**2
+    cost_sum = np.sum(err_sqr)
+
+    return cost_sum / np.size(err_sqr)
+
+# The right side of the ODE:
+def f(x, g_trial):
+    alpha,A, g0 = get_parameters()
+    return alpha*g_trial*(A - g_trial)
+
+# The trial solution using the deep neural network:
+def g_trial_deep(x, params):
+    alpha,A, g0 = get_parameters()
+    return g0 + x*deep_neural_network(params,x)
+
+# The analytical solution:
+def g_analytic(t):
+    alpha,A, g0 = get_parameters()
+    return A*g0/(g0 + (A - g0)*np.exp(-alpha*A*t))
+
+def solve_ode_deep_neural_network(x, num_neurons, num_iter, lmb):
+    # num_hidden_neurons is now a list of number of neurons within each hidden layer
+
+    # Find the number of hidden layers:
+    N_hidden = np.size(num_neurons)
+
+    ## Set up initial weigths and biases
+
+    # Initialize the list of parameters:
+    P = [None]*(N_hidden + 1) # + 1 to include the output layer
+
+    P[0] = npr.randn(num_neurons[0], 2 )
+    for l in range(1,N_hidden):
+        P[l] = npr.randn(num_neurons[l], num_neurons[l-1] + 1) # +1 to include bias
+
+    # For the output layer
+    P[-1] = npr.randn(1, num_neurons[-1] + 1 ) # +1 since bias is included
+
+    print('Initial cost: %g'%cost_function_deep(P, x))
+
+    ## Start finding the optimal weigths using gradient descent
+
+    # Find the Python function that represents the gradient of the cost function
+    # w.r.t the 0-th input argument -- that is the weights and biases in the hidden and output layer
+    cost_function_deep_grad = grad(cost_function_deep,0)
+
+    # Let the update be done num_iter times
+    for i in range(num_iter):
+        # Evaluate the gradient at the current weights and biases in P.
+        # The cost_grad consist now of N_hidden + 1 arrays; the gradient w.r.t the weights and biases
+        # in the hidden layers and output layers evaluated at x.
+        cost_deep_grad =  cost_function_deep_grad(P, x)
+
+        for l in range(N_hidden+1):
+            P[l] = P[l] - lmb * cost_deep_grad[l]
+
+    print('Final cost: %g'%cost_function_deep(P, x))
+
+    return P
+
+if __name__ == '__main__':
+    npr.seed(4155)
+
+    ## Decide the vales of arguments to the function to solve
+    Nt = 10
+    T = 1
+    t = np.linspace(0,T, Nt)
+
+    ## Set up the initial parameters
+    num_hidden_neurons = [100, 50, 25]
+    num_iter = 1000
+    lmb = 1e-3
+
+    P = solve_ode_deep_neural_network(t, num_hidden_neurons, num_iter, lmb)
+
+    g_dnn_ag = g_trial_deep(t,P)
+    g_analytical = g_analytic(t)
+
+    # Find the maximum absolute difference between the solutons:
+    diff_ag = np.max(np.abs(g_dnn_ag - g_analytical))
+    print("The max absolute difference between the solutions is: %g"%diff_ag)
+
+    plt.figure(figsize=(10,10))
+
+    plt.title('Performance of neural network solving an ODE compared to the analytical solution')
+    plt.plot(t, g_analytical)
+    plt.plot(t, g_dnn_ag[0,:])
+    plt.legend(['analytical','nn'])
+    plt.xlabel('t')
+    plt.ylabel('g(t)')
+
+    plt.show()
+
+## Using forward Euler to solve the ODE
+
+A straightforward way of solving an ODE numerically, is to use Euler's method.
+
+Euler's method uses Taylor series to approximate the value at a function $f$ at a step $\Delta x$ from $x$:
+
+$$
+f(x + \Delta x) \approx f(x) + \Delta x f'(x)
+$$
+
+In our case, using Euler's method to approximate the value of $g$ at a step $\Delta t$ from $t$ yields
+
+$$
+\begin{aligned}
+  g(t + \Delta t) &\approx g(t) + \Delta t g'(t) \\
+  &= g(t) + \Delta t \big(\alpha g(t)(A - g(t))\big)
+\end{aligned}
+$$
+
+along with the condition that $g(0) = g_0$.
+
+Let $t_i = i \cdot \Delta t$ where $\Delta t = \frac{T}{N_t-1}$ where $T$ is the final time our solver must solve for and $N_t$ the number of values for $t \in [0, T]$ for $i = 0, \dots, N_t-1$.
+
+For $i \geq 1$, we have that
+
+$$
+\begin{aligned}
+t_i &= i\Delta t \\
+&= (i - 1)\Delta t + \Delta t \\
+&= t_{i-1} + \Delta t
+\end{aligned}
+$$
+
+Now, if $g_i = g(t_i)$ then
+
+
+
    
+
+$$
+\begin{equation}
+  \begin{aligned}
+  g_i &= g(t_i) \\
+  &= g(t_{i-1} + \Delta t) \\
+  &\approx g(t_{i-1}) + \Delta t \big(\alpha g(t_{i-1})(A - g(t_{i-1}))\big) \\
+  &= g_{i-1} + \Delta t \big(\alpha g_{i-1}(A - g_{i-1})\big)
+  \end{aligned}
+\end{equation} \label{odenum} \tag{12}
+$$
+
+for $i \geq 1$ and $g_0 = g(t_0) = g(0) = g_0$.
+
+Equation ([12](#odenum)) could be implemented in the following way,
+extending the program that uses the network using Autograd:
+
+# Assume that all function definitions from the example program using Autograd
+# are located here.
+
+if __name__ == '__main__':
+    npr.seed(4155)
+
+    ## Decide the vales of arguments to the function to solve
+    Nt = 10
+    T = 1
+    t = np.linspace(0,T, Nt)
+
+    ## Set up the initial parameters
+    num_hidden_neurons = [100,50,25]
+    num_iter = 1000
+    lmb = 1e-3
+
+    P = solve_ode_deep_neural_network(t, num_hidden_neurons, num_iter, lmb)
+
+    g_dnn_ag = g_trial_deep(t,P)
+    g_analytical = g_analytic(t)
+
+    # Find the maximum absolute difference between the solutons:
+    diff_ag = np.max(np.abs(g_dnn_ag - g_analytical))
+    print("The max absolute difference between the solutions is: %g"%diff_ag)
+
+    plt.figure(figsize=(10,10))
+
+    plt.title('Performance of neural network solving an ODE compared to the analytical solution')
+    plt.plot(t, g_analytical)
+    plt.plot(t, g_dnn_ag[0,:])
+    plt.legend(['analytical','nn'])
+    plt.xlabel('t')
+    plt.ylabel('g(t)')
+
+    ## Find an approximation to the funtion using forward Euler
+
+    alpha, A, g0 = get_parameters()
+    dt = T/(Nt - 1)
+
+    # Perform forward Euler to solve the ODE
+    g_euler = np.zeros(Nt)
+    g_euler[0] = g0
+
+    for i in range(1,Nt):
+        g_euler[i] = g_euler[i-1] + dt*(alpha*g_euler[i-1]*(A - g_euler[i-1]))
+
+    # Print the errors done by each method
+    diff1 = np.max(np.abs(g_euler - g_analytical))
+    diff2 = np.max(np.abs(g_dnn_ag[0,:] - g_analytical))
+
+    print('Max absolute difference between Euler method and analytical: %g'%diff1)
+    print('Max absolute difference between deep neural network and analytical: %g'%diff2)
+
+    # Plot results
+    plt.figure(figsize=(10,10))
+
+    plt.plot(t,g_euler)
+    plt.plot(t,g_analytical)
+    plt.plot(t,g_dnn_ag[0,:])
+
+    plt.legend(['euler','analytical','dnn'])
+    plt.xlabel('Time t')
+    plt.ylabel('g(t)')
+
+    plt.show()
+
+## Example: Solving the one dimensional Poisson equation
+
+The Poisson equation for $g(x)$ in one dimension is
+
+
+
    
+
+$$
+\begin{equation} \label{poisson} \tag{13}
+  -g''(x) = f(x)
+\end{equation}
+$$
+
+where $f(x)$ is a given function for $x \in (0,1)$.
+
+The conditions that $g(x)$ is chosen to fulfill, are
+
+$$
+\begin{align*}
+  g(0) &= 0 \\
+  g(1) &= 0
+\end{align*}
+$$
+
+This equation can be solved numerically using programs where e.g Autograd and TensorFlow are used.
+The results from the networks can then be compared to the analytical solution.
+In addition, it could be interesting to see how a typical method for numerically solving second order ODEs compares to the neural networks.
+
+
+## The specific equation to solve for
+
+Here, the function $g(x)$ to solve for follows the equation
+
+$$
+-g''(x) = f(x),\qquad x \in (0,1)
+$$
+
+where $f(x)$ is a given function, along with the chosen conditions
+
+
+
    
+
+$$
+\begin{aligned}
+g(0) = g(1) = 0
+\end{aligned}\label{cond} \tag{14}
+$$
+
+In this example, we consider the case when $f(x) = (3x + x^2)\exp(x)$.
+
+For this case, a possible trial solution satisfying the conditions could be
+
+$$
+g_t(x) = x \cdot (1-x) \cdot N(P,x)
+$$
+
+The analytical solution for this problem is
+
+$$
+g(x) = x(1 - x)\exp(x)
+$$
+
+## Solving the equation using Autograd
+
+import autograd.numpy as np
+from autograd import grad, elementwise_grad
+import autograd.numpy.random as npr
+from matplotlib import pyplot as plt
+
+def sigmoid(z):
+    return 1/(1 + np.exp(-z))
+
+def deep_neural_network(deep_params, x):
+    # N_hidden is the number of hidden layers
+    N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer
+
+    # Assumes input x being an one-dimensional array
+    num_values = np.size(x)
+    x = x.reshape(-1, num_values)
+
+    # Assume that the input layer does nothing to the input x
+    x_input = x
+
+    # Due to multiple hidden layers, define a variable referencing to the
+    # output of the previous layer:
+    x_prev = x_input
+
+    ## Hidden layers:
+
+    for l in range(N_hidden):
+        # From the list of parameters P; find the correct weigths and bias for this layer
+        w_hidden = deep_params[l]
+
+        # Add a row of ones to include bias
+        x_prev = np.concatenate((np.ones((1,num_values)), x_prev ), axis = 0)
+
+        z_hidden = np.matmul(w_hidden, x_prev)
+        x_hidden = sigmoid(z_hidden)
+
+        # Update x_prev such that next layer can use the output from this layer
+        x_prev = x_hidden
+
+    ## Output layer:
+
+    # Get the weights and bias for this layer
+    w_output = deep_params[-1]
+
+    # Include bias:
+    x_prev = np.concatenate((np.ones((1,num_values)), x_prev), axis = 0)
+
+    z_output = np.matmul(w_output, x_prev)
+    x_output = z_output
+
+    return x_output
+
+def solve_ode_deep_neural_network(x, num_neurons, num_iter, lmb):
+    # num_hidden_neurons is now a list of number of neurons within each hidden layer
+
+    # Find the number of hidden layers:
+    N_hidden = np.size(num_neurons)
+
+    ## Set up initial weigths and biases
+
+    # Initialize the list of parameters:
+    P = [None]*(N_hidden + 1) # + 1 to include the output layer
+
+    P[0] = npr.randn(num_neurons[0], 2 )
+    for l in range(1,N_hidden):
+        P[l] = npr.randn(num_neurons[l], num_neurons[l-1] + 1) # +1 to include bias
+
+    # For the output layer
+    P[-1] = npr.randn(1, num_neurons[-1] + 1 ) # +1 since bias is included
+
+    print('Initial cost: %g'%cost_function_deep(P, x))
+
+    ## Start finding the optimal weigths using gradient descent
+
+    # Find the Python function that represents the gradient of the cost function
+    # w.r.t the 0-th input argument -- that is the weights and biases in the hidden and output layer
+    cost_function_deep_grad = grad(cost_function_deep,0)
+
+    # Let the update be done num_iter times
+    for i in range(num_iter):
+        # Evaluate the gradient at the current weights and biases in P.
+        # The cost_grad consist now of N_hidden + 1 arrays; the gradient w.r.t the weights and biases
+        # in the hidden layers and output layers evaluated at x.
+        cost_deep_grad =  cost_function_deep_grad(P, x)
+
+        for l in range(N_hidden+1):
+            P[l] = P[l] - lmb * cost_deep_grad[l]
+
+    print('Final cost: %g'%cost_function_deep(P, x))
+
+    return P
+
+## Set up the cost function specified for this Poisson equation:
+
+# The right side of the ODE
+def f(x):
+    return (3*x + x**2)*np.exp(x)
+
+def cost_function_deep(P, x):
+
+    # Evaluate the trial function with the current parameters P
+    g_t = g_trial_deep(x,P)
+
+    # Find the derivative w.r.t x of the trial function
+    d2_g_t = elementwise_grad(elementwise_grad(g_trial_deep,0))(x,P)
+
+    right_side = f(x)
+
+    err_sqr = (-d2_g_t - right_side)**2
+    cost_sum = np.sum(err_sqr)
+
+    return cost_sum/np.size(err_sqr)
+
+# The trial solution:
+def g_trial_deep(x,P):
+    return x*(1-x)*deep_neural_network(P,x)
+
+# The analytic solution;
+def g_analytic(x):
+    return x*(1-x)*np.exp(x)
+
+if __name__ == '__main__':
+    npr.seed(4155)
+
+    ## Decide the vales of arguments to the function to solve
+    Nx = 10
+    x = np.linspace(0,1, Nx)
+
+    ## Set up the initial parameters
+    num_hidden_neurons = [200,100]
+    num_iter = 1000
+    lmb = 1e-3
+
+    P = solve_ode_deep_neural_network(x, num_hidden_neurons, num_iter, lmb)
+
+    g_dnn_ag = g_trial_deep(x,P)
+    g_analytical = g_analytic(x)
+
+    # Find the maximum absolute difference between the solutons:
+    max_diff = np.max(np.abs(g_dnn_ag - g_analytical))
+    print("The max absolute difference between the solutions is: %g"%max_diff)
+
+    plt.figure(figsize=(10,10))
+
+    plt.title('Performance of neural network solving an ODE compared to the analytical solution')
+    plt.plot(x, g_analytical)
+    plt.plot(x, g_dnn_ag[0,:])
+    plt.legend(['analytical','nn'])
+    plt.xlabel('x')
+    plt.ylabel('g(x)')
+    plt.show()
+
+## Comparing with a numerical scheme
+
+The Poisson equation is possible to solve using Taylor series to approximate the second derivative.
+
+Using Taylor series, the second derivative can be expressed as
+
+$$
+g''(x) = \frac{g(x + \Delta x) - 2g(x) + g(x-\Delta x)}{\Delta x^2} + E_{\Delta x}(x)
+$$
+
+where $\Delta x$ is a small step size and $E_{\Delta x}(x)$ being the error term.
+
+Looking away from the error terms gives an approximation to the second derivative:
+
+
+
    
+
+$$
+\begin{equation} \label{approx} \tag{15}
+g''(x) \approx \frac{g(x + \Delta x) - 2g(x) + g(x-\Delta x)}{\Delta x^2}
+\end{equation}
+$$
+
+If $x_i = i \Delta x = x_{i-1} + \Delta x$ and $g_i = g(x_i)$ for $i = 1,\dots N_x - 2$ with $N_x$ being the number of values for $x$, ([15](#approx)) becomes
+
+$$
+\begin{aligned}
+g''(x_i) &\approx \frac{g(x_i + \Delta x) - 2g(x_i) + g(x_i -\Delta x)}{\Delta x^2} \\
+&= \frac{g_{i+1} - 2g_i + g_{i-1}}{\Delta x^2}
+\end{aligned}
+$$
+
+Since we know from our problem that
+
+$$
+\begin{aligned}
+-g''(x) &= f(x) \\
+&= (3x + x^2)\exp(x)
+\end{aligned}
+$$
+
+along with the conditions $g(0) = g(1) = 0$,
+the following scheme can be used to find an approximate solution for $g(x)$ numerically:
+
+
+
    
+
+$$
+\begin{equation}
+  \begin{aligned}
+  -\Big( \frac{g_{i+1} - 2g_i + g_{i-1}}{\Delta x^2} \Big) &= f(x_i) \\
+  -g_{i+1} + 2g_i - g_{i-1} &= \Delta x^2 f(x_i)
+  \end{aligned}
+\end{equation} \label{odesys} \tag{16}
+$$
+
+for $i = 1, \dots, N_x - 2$ where $g_0 = g_{N_x - 1} = 0$ and $f(x_i) = (3x_i + x_i^2)\exp(x_i)$, which is given for our specific problem.
+
+The equation can be rewritten into a matrix equation:
+
+$$
+\begin{aligned}
+\begin{pmatrix}
+2 & -1 & 0 & \dots & 0 \\
+-1 & 2 & -1 & \dots & 0 \\
+\vdots & & \ddots & & \vdots \\
+0 & \dots & -1 & 2 & -1  \\
+0 & \dots & 0 & -1 & 2\\
+\end{pmatrix}
+\begin{pmatrix}
+g_1 \\
+g_2 \\
+\vdots \\
+g_{N_x - 3} \\
+g_{N_x - 2}
+\end{pmatrix}
+&=
+\Delta x^2
+\begin{pmatrix}
+f(x_1) \\
+f(x_2) \\
+\vdots \\
+f(x_{N_x - 3}) \\
+f(x_{N_x - 2})
+\end{pmatrix} \\
+\boldsymbol{A}\boldsymbol{g} &= \boldsymbol{f},
+\end{aligned}
+$$
+
+which makes it possible to solve for the vector $\boldsymbol{g}$.
+
+
+## Setting up the code
+
+We can then compare the result from this numerical scheme with the output from our network using Autograd:
+
+import autograd.numpy as np
+from autograd import grad, elementwise_grad
+import autograd.numpy.random as npr
+from matplotlib import pyplot as plt
+
+def sigmoid(z):
+    return 1/(1 + np.exp(-z))
+
+def deep_neural_network(deep_params, x):
+    # N_hidden is the number of hidden layers
+    N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer
+
+    # Assumes input x being an one-dimensional array
+    num_values = np.size(x)
+    x = x.reshape(-1, num_values)
+
+    # Assume that the input layer does nothing to the input x
+    x_input = x
+
+    # Due to multiple hidden layers, define a variable referencing to the
+    # output of the previous layer:
+    x_prev = x_input
+
+    ## Hidden layers:
+
+    for l in range(N_hidden):
+        # From the list of parameters P; find the correct weigths and bias for this layer
+        w_hidden = deep_params[l]
+
+        # Add a row of ones to include bias
+        x_prev = np.concatenate((np.ones((1,num_values)), x_prev ), axis = 0)
+
+        z_hidden = np.matmul(w_hidden, x_prev)
+        x_hidden = sigmoid(z_hidden)
+
+        # Update x_prev such that next layer can use the output from this layer
+        x_prev = x_hidden
+
+    ## Output layer:
+
+    # Get the weights and bias for this layer
+    w_output = deep_params[-1]
+
+    # Include bias:
+    x_prev = np.concatenate((np.ones((1,num_values)), x_prev), axis = 0)
+
+    z_output = np.matmul(w_output, x_prev)
+    x_output = z_output
+
+    return x_output
+
+def solve_ode_deep_neural_network(x, num_neurons, num_iter, lmb):
+    # num_hidden_neurons is now a list of number of neurons within each hidden layer
+
+    # Find the number of hidden layers:
+    N_hidden = np.size(num_neurons)
+
+    ## Set up initial weigths and biases
+
+    # Initialize the list of parameters:
+    P = [None]*(N_hidden + 1) # + 1 to include the output layer
+
+    P[0] = npr.randn(num_neurons[0], 2 )
+    for l in range(1,N_hidden):
+        P[l] = npr.randn(num_neurons[l], num_neurons[l-1] + 1) # +1 to include bias
+
+    # For the output layer
+    P[-1] = npr.randn(1, num_neurons[-1] + 1 ) # +1 since bias is included
+
+    print('Initial cost: %g'%cost_function_deep(P, x))
+
+    ## Start finding the optimal weigths using gradient descent
+
+    # Find the Python function that represents the gradient of the cost function
+    # w.r.t the 0-th input argument -- that is the weights and biases in the hidden and output layer
+    cost_function_deep_grad = grad(cost_function_deep,0)
+
+    # Let the update be done num_iter times
+    for i in range(num_iter):
+        # Evaluate the gradient at the current weights and biases in P.
+        # The cost_grad consist now of N_hidden + 1 arrays; the gradient w.r.t the weights and biases
+        # in the hidden layers and output layers evaluated at x.
+        cost_deep_grad =  cost_function_deep_grad(P, x)
+
+        for l in range(N_hidden+1):
+            P[l] = P[l] - lmb * cost_deep_grad[l]
+
+    print('Final cost: %g'%cost_function_deep(P, x))
+
+    return P
+
+## Set up the cost function specified for this Poisson equation:
+
+# The right side of the ODE
+def f(x):
+    return (3*x + x**2)*np.exp(x)
+
+def cost_function_deep(P, x):
+
+    # Evaluate the trial function with the current parameters P
+    g_t = g_trial_deep(x,P)
+
+    # Find the derivative w.r.t x of the trial function
+    d2_g_t = elementwise_grad(elementwise_grad(g_trial_deep,0))(x,P)
+
+    right_side = f(x)
+
+    err_sqr = (-d2_g_t - right_side)**2
+    cost_sum = np.sum(err_sqr)
+
+    return cost_sum/np.size(err_sqr)
+
+# The trial solution:
+def g_trial_deep(x,P):
+    return x*(1-x)*deep_neural_network(P,x)
+
+# The analytic solution;
+def g_analytic(x):
+    return x*(1-x)*np.exp(x)
+
+if __name__ == '__main__':
+    npr.seed(4155)
+
+    ## Decide the vales of arguments to the function to solve
+    Nx = 10
+    x = np.linspace(0,1, Nx)
+
+    ## Set up the initial parameters
+    num_hidden_neurons = [200,100]
+    num_iter = 1000
+    lmb = 1e-3
+
+    P = solve_ode_deep_neural_network(x, num_hidden_neurons, num_iter, lmb)
+
+    g_dnn_ag = g_trial_deep(x,P)
+    g_analytical = g_analytic(x)
+
+    # Find the maximum absolute difference between the solutons:
+
+    plt.figure(figsize=(10,10))
+
+    plt.title('Performance of neural network solving an ODE compared to the analytical solution')
+    plt.plot(x, g_analytical)
+    plt.plot(x, g_dnn_ag[0,:])
+    plt.legend(['analytical','nn'])
+    plt.xlabel('x')
+    plt.ylabel('g(x)')
+
+    ## Perform the computation using the numerical scheme
+
+    dx = 1/(Nx - 1)
+
+    # Set up the matrix A
+    A = np.zeros((Nx-2,Nx-2))
+
+    A[0,0] = 2
+    A[0,1] = -1
+
+    for i in range(1,Nx-3):
+        A[i,i-1] = -1
+        A[i,i] = 2
+        A[i,i+1] = -1
+
+    A[Nx - 3, Nx - 4] = -1
+    A[Nx - 3, Nx - 3] = 2
+
+    # Set up the vector f
+    f_vec = dx**2 * f(x[1:-1])
+
+    # Solve the equation
+    g_res = np.linalg.solve(A,f_vec)
+
+    g_vec = np.zeros(Nx)
+    g_vec[1:-1] = g_res
+
+    # Print the differences between each method
+    max_diff1 = np.max(np.abs(g_dnn_ag - g_analytical))
+    max_diff2 = np.max(np.abs(g_vec - g_analytical))
+    print("The max absolute difference between the analytical solution and DNN Autograd: %g"%max_diff1)
+    print("The max absolute difference between the analytical solution and numerical scheme: %g"%max_diff2)
+
+    # Plot the results
+    plt.figure(figsize=(10,10))
+
+    plt.plot(x,g_vec)
+    plt.plot(x,g_analytical)
+    plt.plot(x,g_dnn_ag[0,:])
+
+    plt.legend(['numerical scheme','analytical','dnn'])
+    plt.show()
+
+## Partial Differential Equations
+
+A partial differential equation (PDE) has a solution here the function
+is defined by multiple variables.  The equation may involve all kinds
+of combinations of which variables the function is differentiated with
+respect to.
+
+In general, a partial differential equation for a function $g(x_1,\dots,x_N)$ with $N$ variables may be expressed as
+
+
+
    
+
+$$
+\begin{equation} \label{PDE} \tag{17}
+  f\left(x_1, \, \dots \, , x_N, \frac{\partial g(x_1,\dots,x_N) }{\partial x_1}, \dots , \frac{\partial g(x_1,\dots,x_N) }{\partial x_N}, \frac{\partial g(x_1,\dots,x_N) }{\partial x_1\partial x_2}, \, \dots \, , \frac{\partial^n g(x_1,\dots,x_N) }{\partial x_N^n} \right) = 0
+\end{equation}
+$$
+
+where $f$ is an expression involving all kinds of possible mixed derivatives of $g(x_1,\dots,x_N)$ up to an order $n$. In order for the solution to be unique, some additional conditions must also be given.
+
+
+## Type of problem
+
+The problem our network must solve for, is similar to the ODE case.
+We must have a trial solution $g_t$ at hand.
+
+For instance, the trial solution could be expressed as
+
+$$
+\begin{align*}
+  g_t(x_1,\dots,x_N) = h_1(x_1,\dots,x_N) + h_2(x_1,\dots,x_N,N(x_1,\dots,x_N,P))
+\end{align*}
+$$
+
+where $h_1(x_1,\dots,x_N)$ is a function that ensures $g_t(x_1,\dots,x_N)$ satisfies some given conditions.
+The neural network $N(x_1,\dots,x_N,P)$ has weights and biases described by $P$ and $h_2(x_1,\dots,x_N,N(x_1,\dots,x_N,P))$ is an expression using the output from the neural network in some way.
+
+The role of the function $h_2(x_1,\dots,x_N,N(x_1,\dots,x_N,P))$, is to ensure that the output of $N(x_1,\dots,x_N,P)$ is zero when $g_t(x_1,\dots,x_N)$ is evaluated at the values of $x_1,\dots,x_N$ where the given conditions must be satisfied. The function $h_1(x_1,\dots,x_N)$ should alone make $g_t(x_1,\dots,x_N)$ satisfy the conditions.
+
+
+
+## Network requirements
+
+The network tries then the minimize the cost function following the
+same ideas as described for the ODE case, but now with more than one
+variables to consider.  The concept still remains the same; find a set
+of parameters $P$ such that the expression $f$ in ([17](#PDE)) is as
+close to zero as possible.
+
+As for the ODE case, the cost function is the mean squared error that
+the network must try to minimize. The cost function for the network to
+minimize is
+
+$$
+C\left(x_1, \dots, x_N, P\right) = \left(  f\left(x_1, \, \dots \, , x_N, \frac{\partial g(x_1,\dots,x_N) }{\partial x_1}, \dots , \frac{\partial g(x_1,\dots,x_N) }{\partial x_N}, \frac{\partial g(x_1,\dots,x_N) }{\partial x_1\partial x_2}, \, \dots \, , \frac{\partial^n g(x_1,\dots,x_N) }{\partial x_N^n} \right) \right)^2
+$$
+
+## More details
+
+If we let $\boldsymbol{x} = \big( x_1, \dots, x_N \big)$ be an array containing the values for $x_1, \dots, x_N$ respectively, the cost function can be reformulated into the following:
+
+$$
+C\left(\boldsymbol{x}, P\right) = f\left( \left( \boldsymbol{x}, \frac{\partial g(\boldsymbol{x}) }{\partial x_1}, \dots , \frac{\partial g(\boldsymbol{x}) }{\partial x_N}, \frac{\partial g(\boldsymbol{x}) }{\partial x_1\partial x_2}, \, \dots \, , \frac{\partial^n g(\boldsymbol{x}) }{\partial x_N^n} \right) \right)^2
+$$
+
+If we also have $M$ different sets of values for $x_1, \dots, x_N$, that is $\boldsymbol{x}_i = \big(x_1^{(i)}, \dots, x_N^{(i)}\big)$ for $i = 1,\dots,M$ being the rows in matrix $X$, the cost function can be generalized into
+
+$$
+C\left(X, P \right) = \sum_{i=1}^M f\left( \left( \boldsymbol{x}_i, \frac{\partial g(\boldsymbol{x}_i) }{\partial x_1}, \dots , \frac{\partial g(\boldsymbol{x}_i) }{\partial x_N}, \frac{\partial g(\boldsymbol{x}_i) }{\partial x_1\partial x_2}, \, \dots \, , \frac{\partial^n g(\boldsymbol{x}_i) }{\partial x_N^n} \right) \right)^2.
+$$
+
+## Example: The diffusion equation
+
+In one spatial dimension, the equation reads
+
+$$
+\frac{\partial g(x,t)}{\partial t} = \frac{\partial^2 g(x,t)}{\partial x^2}
+$$
+
+where a possible choice of conditions are
+
+$$
+\begin{align*}
+g(0,t) &= 0 ,\qquad t \geq 0 \\
+g(1,t) &= 0, \qquad t \geq 0 \\
+g(x,0) &= u(x),\qquad x\in [0,1]
+\end{align*}
+$$
+
+with $u(x)$ being some given function.
+
+
+## Defining the problem
+
+For this case, we want to find $g(x,t)$ such that
+
+
+
    
+
+$$
+\begin{equation}
+  \frac{\partial g(x,t)}{\partial t} = \frac{\partial^2 g(x,t)}{\partial x^2}
+\end{equation} \label{diffonedim} \tag{18}
+$$
+
+and
+
+$$
+\begin{align*}
+g(0,t) &= 0 ,\qquad t \geq 0 \\
+g(1,t) &= 0, \qquad t \geq 0 \\
+g(x,0) &= u(x),\qquad x\in [0,1]
+\end{align*}
+$$
+
+with $u(x) = \sin(\pi x)$.
+
+First, let us set up the deep neural network.
+The deep neural network will follow the same structure as discussed in the examples solving the ODEs.
+First, we will look into how Autograd could be used in a network tailored to solve for bivariate functions.
+
+
+
+
+## Setting up the network using Autograd
+
+The only change to do here, is to extend our network such that
+functions of multiple parameters are correctly handled.  In this case
+we have two variables in our function to solve for, that is time $t$
+and position $x$.  The variables will be represented by a
+one-dimensional array in the program.  The program will evaluate the
+network at each possible pair $(x,t)$, given an array for the desired
+$x$-values and $t$-values to approximate the solution at.
+
+def sigmoid(z):
+    return 1/(1 + np.exp(-z))
+
+def deep_neural_network(deep_params, x):
+    # x is now a point and a 1D numpy array; make it a column vector
+    num_coordinates = np.size(x,0)
+    x = x.reshape(num_coordinates,-1)
+
+    num_points = np.size(x,1)
+
+    # N_hidden is the number of hidden layers
+    N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer
+
+    # Assume that the input layer does nothing to the input x
+    x_input = x
+    x_prev = x_input
+
+    ## Hidden layers:
+
+    for l in range(N_hidden):
+        # From the list of parameters P; find the correct weigths and bias for this layer
+        w_hidden = deep_params[l]
+
+        # Add a row of ones to include bias
+        x_prev = np.concatenate((np.ones((1,num_points)), x_prev ), axis = 0)
+
+        z_hidden = np.matmul(w_hidden, x_prev)
+        x_hidden = sigmoid(z_hidden)
+
+        # Update x_prev such that next layer can use the output from this layer
+        x_prev = x_hidden
+
+    ## Output layer:
+
+    # Get the weights and bias for this layer
+    w_output = deep_params[-1]
+
+    # Include bias:
+    x_prev = np.concatenate((np.ones((1,num_points)), x_prev), axis = 0)
+
+    z_output = np.matmul(w_output, x_prev)
+    x_output = z_output
+
+    return x_output[0][0]
+
+## Setting up the network using Autograd; The trial solution
+
+The cost function must then iterate through the given arrays
+containing values for $x$ and $t$, defines a point $(x,t)$ the deep
+neural network and the trial solution is evaluated at, and then finds
+the Jacobian of the trial solution.
+
+A possible trial solution for this PDE is
+
+$$
+g_t(x,t) = h_1(x,t) + x(1-x)tN(x,t,P)
+$$
+
+with $A(x,t)$ being a function ensuring that $g_t(x,t)$ satisfies our given conditions, and $N(x,t,P)$ being the output from the deep neural network using weights and biases for each layer from $P$.
+
+To fulfill the conditions, $A(x,t)$ could be:
+
+$$
+h_1(x,t) = (1-t)\Big(u(x) - \big((1-x)u(0) + x u(1)\big)\Big) = (1-t)u(x) = (1-t)\sin(\pi x)
+$$
+since $(0) = u(1) = 0$ and $u(x) = \sin(\pi x)$.
+
+
+## Why the jacobian?
+
+The Jacobian is used because the program must find the derivative of
+the trial solution with respect to $x$ and $t$.
+
+This gives the necessity of computing the Jacobian matrix, as we want
+to evaluate the gradient with respect to $x$ and $t$ (note that the
+Jacobian of a scalar-valued multivariate function is simply its
+gradient).
+
+In Autograd, the differentiation is by default done with respect to
+the first input argument of your Python function. Since the points is
+an array representing $x$ and $t$, the Jacobian is calculated using
+the values of $x$ and $t$.
+
+To find the second derivative with respect to $x$ and $t$, the
+Jacobian can be found for the second time. The result is a Hessian
+matrix, which is the matrix containing all the possible second order
+mixed derivatives of $g(x,t)$.
+
+# Set up the trial function:
+def u(x):
+    return np.sin(np.pi*x)
+
+def g_trial(point,P):
+    x,t = point
+    return (1-t)*u(x) + x*(1-x)*t*deep_neural_network(P,point)
+
+# The right side of the ODE:
+def f(point):
+    return 0.
+
+# The cost function:
+def cost_function(P, x, t):
+    cost_sum = 0
+
+    g_t_jacobian_func = jacobian(g_trial)
+    g_t_hessian_func = hessian(g_trial)
+
+    for x_ in x:
+        for t_ in t:
+            point = np.array([x_,t_])
+
+            g_t = g_trial(point,P)
+            g_t_jacobian = g_t_jacobian_func(point,P)
+            g_t_hessian = g_t_hessian_func(point,P)
+
+            g_t_dt = g_t_jacobian[1]
+            g_t_d2x = g_t_hessian[0][0]
+
+            func = f(point)
+
+            err_sqr = ( (g_t_dt - g_t_d2x) - func)**2
+            cost_sum += err_sqr
+
+    return cost_sum
+
+## Setting up the network using Autograd; The full program
+
+Having set up the network, along with the trial solution and cost function, we can now see how the deep neural network performs by comparing the results to the analytical solution.
+
+The analytical solution of our problem is
+
+$$
+g(x,t) = \exp(-\pi^2 t)\sin(\pi x)
+$$
+
+A possible way to implement a neural network solving the PDE, is given below.
+Be aware, though, that it is fairly slow for the parameters used.
+A better result is possible, but requires more iterations, and thus longer time to complete.
+
+
+Indeed, the program below is not optimal in its implementation, but rather serves as an example on how to implement and use a neural network to solve a PDE.
+Using TensorFlow results in a much better execution time. Try it!
+
+import autograd.numpy as np
+from autograd import jacobian,hessian,grad
+import autograd.numpy.random as npr
+from matplotlib import cm
+from matplotlib import pyplot as plt
+from mpl_toolkits.mplot3d import axes3d
+
+## Set up the network
+
+def sigmoid(z):
+    return 1/(1 + np.exp(-z))
+
+def deep_neural_network(deep_params, x):
+    # x is now a point and a 1D numpy array; make it a column vector
+    num_coordinates = np.size(x,0)
+    x = x.reshape(num_coordinates,-1)
+
+    num_points = np.size(x,1)
+
+    # N_hidden is the number of hidden layers
+    N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer
+
+    # Assume that the input layer does nothing to the input x
+    x_input = x
+    x_prev = x_input
+
+    ## Hidden layers:
+
+    for l in range(N_hidden):
+        # From the list of parameters P; find the correct weigths and bias for this layer
+        w_hidden = deep_params[l]
+
+        # Add a row of ones to include bias
+        x_prev = np.concatenate((np.ones((1,num_points)), x_prev ), axis = 0)
+
+        z_hidden = np.matmul(w_hidden, x_prev)
+        x_hidden = sigmoid(z_hidden)
+
+        # Update x_prev such that next layer can use the output from this layer
+        x_prev = x_hidden
+
+    ## Output layer:
+
+    # Get the weights and bias for this layer
+    w_output = deep_params[-1]
+
+    # Include bias:
+    x_prev = np.concatenate((np.ones((1,num_points)), x_prev), axis = 0)
+
+    z_output = np.matmul(w_output, x_prev)
+    x_output = z_output
+
+    return x_output[0][0]
+
+## Define the trial solution and cost function
+def u(x):
+    return np.sin(np.pi*x)
+
+def g_trial(point,P):
+    x,t = point
+    return (1-t)*u(x) + x*(1-x)*t*deep_neural_network(P,point)
+
+# The right side of the ODE:
+def f(point):
+    return 0.
+
+# The cost function:
+def cost_function(P, x, t):
+    cost_sum = 0
+
+    g_t_jacobian_func = jacobian(g_trial)
+    g_t_hessian_func = hessian(g_trial)
+
+    for x_ in x:
+        for t_ in t:
+            point = np.array([x_,t_])
+
+            g_t = g_trial(point,P)
+            g_t_jacobian = g_t_jacobian_func(point,P)
+            g_t_hessian = g_t_hessian_func(point,P)
+
+            g_t_dt = g_t_jacobian[1]
+            g_t_d2x = g_t_hessian[0][0]
+
+            func = f(point)
+
+            err_sqr = ( (g_t_dt - g_t_d2x) - func)**2
+            cost_sum += err_sqr
+
+    return cost_sum /( np.size(x)*np.size(t) )
+
+## For comparison, define the analytical solution
+def g_analytic(point):
+    x,t = point
+    return np.exp(-np.pi**2*t)*np.sin(np.pi*x)
+
+## Set up a function for training the network to solve for the equation
+def solve_pde_deep_neural_network(x,t, num_neurons, num_iter, lmb):
+    ## Set up initial weigths and biases
+    N_hidden = np.size(num_neurons)
+
+    ## Set up initial weigths and biases
+
+    # Initialize the list of parameters:
+    P = [None]*(N_hidden + 1) # + 1 to include the output layer
+
+    P[0] = npr.randn(num_neurons[0], 2 + 1 ) # 2 since we have two points, +1 to include bias
+    for l in range(1,N_hidden):
+        P[l] = npr.randn(num_neurons[l], num_neurons[l-1] + 1) # +1 to include bias
+
+    # For the output layer
+    P[-1] = npr.randn(1, num_neurons[-1] + 1 ) # +1 since bias is included
+
+    print('Initial cost: ',cost_function(P, x, t))
+
+    cost_function_grad = grad(cost_function,0)
+
+    # Let the update be done num_iter times
+    for i in range(num_iter):
+        cost_grad =  cost_function_grad(P, x , t)
+
+        for l in range(N_hidden+1):
+            P[l] = P[l] - lmb * cost_grad[l]
+
+    print('Final cost: ',cost_function(P, x, t))
+
+    return P
+
+if __name__ == '__main__':
+    ### Use the neural network:
+    npr.seed(15)
+
+    ## Decide the vales of arguments to the function to solve
+    Nx = 10; Nt = 10
+    x = np.linspace(0, 1, Nx)
+    t = np.linspace(0,1,Nt)
+
+    ## Set up the parameters for the network
+    num_hidden_neurons = [100, 25]
+    num_iter = 250
+    lmb = 0.01
+
+    P = solve_pde_deep_neural_network(x,t, num_hidden_neurons, num_iter, lmb)
+
+    ## Store the results
+    g_dnn_ag = np.zeros((Nx, Nt))
+    G_analytical = np.zeros((Nx, Nt))
+    for i,x_ in enumerate(x):
+        for j, t_ in enumerate(t):
+            point = np.array([x_, t_])
+            g_dnn_ag[i,j] = g_trial(point,P)
+
+            G_analytical[i,j] = g_analytic(point)
+
+    # Find the map difference between the analytical and the computed solution
+    diff_ag = np.abs(g_dnn_ag - G_analytical)
+    print('Max absolute difference between the analytical solution and the network: %g'%np.max(diff_ag))
+
+    ## Plot the solutions in two dimensions, that being in position and time
+
+    T,X = np.meshgrid(t,x)
+
+    fig = plt.figure(figsize=(10,10))
+    ax = fig.gca(projection='3d')
+    ax.set_title('Solution from the deep neural network w/ %d layer'%len(num_hidden_neurons))
+    s = ax.plot_surface(T,X,g_dnn_ag,linewidth=0,antialiased=False,cmap=cm.viridis)
+    ax.set_xlabel('Time $t$')
+    ax.set_ylabel('Position $x$');
+
+
+    fig = plt.figure(figsize=(10,10))
+    ax = fig.gca(projection='3d')
+    ax.set_title('Analytical solution')
+    s = ax.plot_surface(T,X,G_analytical,linewidth=0,antialiased=False,cmap=cm.viridis)
+    ax.set_xlabel('Time $t$')
+    ax.set_ylabel('Position $x$');
+
+    fig = plt.figure(figsize=(10,10))
+    ax = fig.gca(projection='3d')
+    ax.set_title('Difference')
+    s = ax.plot_surface(T,X,diff_ag,linewidth=0,antialiased=False,cmap=cm.viridis)
+    ax.set_xlabel('Time $t$')
+    ax.set_ylabel('Position $x$');
+
+    ## Take some slices of the 3D plots just to see the solutions at particular times
+    indx1 = 0
+    indx2 = int(Nt/2)
+    indx3 = Nt-1
+
+    t1 = t[indx1]
+    t2 = t[indx2]
+    t3 = t[indx3]
+
+    # Slice the results from the DNN
+    res1 = g_dnn_ag[:,indx1]
+    res2 = g_dnn_ag[:,indx2]
+    res3 = g_dnn_ag[:,indx3]
+
+    # Slice the analytical results
+    res_analytical1 = G_analytical[:,indx1]
+    res_analytical2 = G_analytical[:,indx2]
+    res_analytical3 = G_analytical[:,indx3]
+
+    # Plot the slices
+    plt.figure(figsize=(10,10))
+    plt.title("Computed solutions at time = %g"%t1)
+    plt.plot(x, res1)
+    plt.plot(x,res_analytical1)
+    plt.legend(['dnn','analytical'])
+
+    plt.figure(figsize=(10,10))
+    plt.title("Computed solutions at time = %g"%t2)
+    plt.plot(x, res2)
+    plt.plot(x,res_analytical2)
+    plt.legend(['dnn','analytical'])
+
+    plt.figure(figsize=(10,10))
+    plt.title("Computed solutions at time = %g"%t3)
+    plt.plot(x, res3)
+    plt.plot(x,res_analytical3)
+    plt.legend(['dnn','analytical'])
+
+    plt.show()
+
+## Example: Solving the wave equation with Neural Networks
+
+The wave equation is
+
+$$
+\frac{\partial^2 g(x,t)}{\partial t^2} = c^2\frac{\partial^2 g(x,t)}{\partial x^2}
+$$
+
+with $c$ being the specified wave speed.
+
+Here, the chosen conditions are
+
+$$
+\begin{align*}
+	g(0,t) &= 0 \\
+	g(1,t) &= 0 \\
+	g(x,0) &= u(x) \\
+	\frac{\partial g(x,t)}{\partial t} \Big |_{t = 0} &= v(x)
+\end{align*}
+$$
+
+where $\frac{\partial g(x,t)}{\partial t} \Big |_{t = 0}$ means the derivative of $g(x,t)$ with respect to $t$ is evaluated at $t = 0$, and $u(x)$ and $v(x)$ being given functions.
+
+
+## The problem to solve for
+
+The wave equation to solve for, is
+
+
+
    
+
+$$
+\begin{equation} \label{wave} \tag{19}
+\frac{\partial^2 g(x,t)}{\partial t^2} = c^2 \frac{\partial^2 g(x,t)}{\partial x^2}
+\end{equation}
+$$
+
+where $c$ is the given wave speed.
+The chosen conditions for this equation are
+
+
+
    
+
+$$
+\begin{aligned}
+g(0,t) &= 0, &t \geq 0 \\
+g(1,t) &= 0, &t \geq 0 \\
+g(x,0) &= u(x), &x\in[0,1] \\
+\frac{\partial g(x,t)}{\partial t}\Big |_{t = 0} &= v(x), &x \in [0,1]
+\end{aligned} \label{condwave} \tag{20}
+$$
+
+In this example, let $c = 1$ and $u(x) = \sin(\pi x)$ and $v(x) = -\pi\sin(\pi x)$.
+
+
+
+## The trial solution
+Setting up the network is done in similar matter as for the example of solving the diffusion equation.
+The only things we have to change, is the trial solution such that it satisfies the conditions from ([20](#condwave)) and the cost function.
+
+The trial solution becomes slightly different since we have other conditions than in the example of solving the diffusion equation. Here, a possible trial solution $g_t(x,t)$ is
+
+$$
+g_t(x,t) = h_1(x,t) + x(1-x)t^2N(x,t,P)
+$$
+
+where
+
+$$
+h_1(x,t) = (1-t^2)u(x) + tv(x)
+$$
+
+Note that this trial solution satisfies the conditions only if $u(0) = v(0) = u(1) = v(1) = 0$, which is the case in this example.
+
+
+## The analytical solution
+
+The analytical solution for our specific problem, is
+
+$$
+g(x,t) = \sin(\pi x)\cos(\pi t) - \sin(\pi x)\sin(\pi t)
+$$
+
+
+## Solving the wave equation - the full program using Autograd
+
+import autograd.numpy as np
+from autograd import hessian,grad
+import autograd.numpy.random as npr
+from matplotlib import cm
+from matplotlib import pyplot as plt
+from mpl_toolkits.mplot3d import axes3d
+
+## Set up the trial function:
+def u(x):
+    return np.sin(np.pi*x)
+
+def v(x):
+    return -np.pi*np.sin(np.pi*x)
+
+def h1(point):
+    x,t = point
+    return (1 - t**2)*u(x) + t*v(x)
+
+def g_trial(point,P):
+    x,t = point
+    return h1(point) + x*(1-x)*t**2*deep_neural_network(P,point)
+
+## Define the cost function
+def cost_function(P, x, t):
+    cost_sum = 0
+
+    g_t_hessian_func = hessian(g_trial)
+
+    for x_ in x:
+        for t_ in t:
+            point = np.array([x_,t_])
+
+            g_t_hessian = g_t_hessian_func(point,P)
+
+            g_t_d2x = g_t_hessian[0][0]
+            g_t_d2t = g_t_hessian[1][1]
+
+            err_sqr = ( (g_t_d2t - g_t_d2x) )**2
+            cost_sum += err_sqr
+
+    return cost_sum / (np.size(t) * np.size(x))
+
+## The neural network
+def sigmoid(z):
+    return 1/(1 + np.exp(-z))
+
+def deep_neural_network(deep_params, x):
+    # x is now a point and a 1D numpy array; make it a column vector
+    num_coordinates = np.size(x,0)
+    x = x.reshape(num_coordinates,-1)
+
+    num_points = np.size(x,1)
+
+    # N_hidden is the number of hidden layers
+    N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer
+
+    # Assume that the input layer does nothing to the input x
+    x_input = x
+    x_prev = x_input
+
+    ## Hidden layers:
+
+    for l in range(N_hidden):
+        # From the list of parameters P; find the correct weigths and bias for this layer
+        w_hidden = deep_params[l]
+
+        # Add a row of ones to include bias
+        x_prev = np.concatenate((np.ones((1,num_points)), x_prev ), axis = 0)
+
+        z_hidden = np.matmul(w_hidden, x_prev)
+        x_hidden = sigmoid(z_hidden)
+
+        # Update x_prev such that next layer can use the output from this layer
+        x_prev = x_hidden
+
+    ## Output layer:
+
+    # Get the weights and bias for this layer
+    w_output = deep_params[-1]
+
+    # Include bias:
+    x_prev = np.concatenate((np.ones((1,num_points)), x_prev), axis = 0)
+
+    z_output = np.matmul(w_output, x_prev)
+    x_output = z_output
+
+    return x_output[0][0]
+
+## The analytical solution
+def g_analytic(point):
+    x,t = point
+    return np.sin(np.pi*x)*np.cos(np.pi*t) - np.sin(np.pi*x)*np.sin(np.pi*t)
+
+def solve_pde_deep_neural_network(x,t, num_neurons, num_iter, lmb):
+    ## Set up initial weigths and biases
+    N_hidden = np.size(num_neurons)
+
+    ## Set up initial weigths and biases
+
+    # Initialize the list of parameters:
+    P = [None]*(N_hidden + 1) # + 1 to include the output layer
+
+    P[0] = npr.randn(num_neurons[0], 2 + 1 ) # 2 since we have two points, +1 to include bias
+    for l in range(1,N_hidden):
+        P[l] = npr.randn(num_neurons[l], num_neurons[l-1] + 1) # +1 to include bias
+
+    # For the output layer
+    P[-1] = npr.randn(1, num_neurons[-1] + 1 ) # +1 since bias is included
+
+    print('Initial cost: ',cost_function(P, x, t))
+
+    cost_function_grad = grad(cost_function,0)
+
+    # Let the update be done num_iter times
+    for i in range(num_iter):
+        cost_grad =  cost_function_grad(P, x , t)
+
+        for l in range(N_hidden+1):
+            P[l] = P[l] - lmb * cost_grad[l]
+
+
+    print('Final cost: ',cost_function(P, x, t))
+
+    return P
+
+if __name__ == '__main__':
+    ### Use the neural network:
+    npr.seed(15)
+
+    ## Decide the vales of arguments to the function to solve
+    Nx = 10; Nt = 10
+    x = np.linspace(0, 1, Nx)
+    t = np.linspace(0,1,Nt)
+
+    ## Set up the parameters for the network
+    num_hidden_neurons = [50,20]
+    num_iter = 1000
+    lmb = 0.01
+
+    P = solve_pde_deep_neural_network(x,t, num_hidden_neurons, num_iter, lmb)
+
+    ## Store the results
+    res = np.zeros((Nx, Nt))
+    res_analytical = np.zeros((Nx, Nt))
+    for i,x_ in enumerate(x):
+        for j, t_ in enumerate(t):
+            point = np.array([x_, t_])
+            res[i,j] = g_trial(point,P)
+
+            res_analytical[i,j] = g_analytic(point)
+
+    diff = np.abs(res - res_analytical)
+    print("Max difference between analytical and solution from nn: %g"%np.max(diff))
+
+    ## Plot the solutions in two dimensions, that being in position and time
+
+    T,X = np.meshgrid(t,x)
+
+    fig = plt.figure(figsize=(10,10))
+    ax = fig.gca(projection='3d')
+    ax.set_title('Solution from the deep neural network w/ %d layer'%len(num_hidden_neurons))
+    s = ax.plot_surface(T,X,res,linewidth=0,antialiased=False,cmap=cm.viridis)
+    ax.set_xlabel('Time $t$')
+    ax.set_ylabel('Position $x$');
+
+
+    fig = plt.figure(figsize=(10,10))
+    ax = fig.gca(projection='3d')
+    ax.set_title('Analytical solution')
+    s = ax.plot_surface(T,X,res_analytical,linewidth=0,antialiased=False,cmap=cm.viridis)
+    ax.set_xlabel('Time $t$')
+    ax.set_ylabel('Position $x$');
+
+
+    fig = plt.figure(figsize=(10,10))
+    ax = fig.gca(projection='3d')
+    ax.set_title('Difference')
+    s = ax.plot_surface(T,X,diff,linewidth=0,antialiased=False,cmap=cm.viridis)
+    ax.set_xlabel('Time $t$')
+    ax.set_ylabel('Position $x$');
+
+    ## Take some slices of the 3D plots just to see the solutions at particular times
+    indx1 = 0
+    indx2 = int(Nt/2)
+    indx3 = Nt-1
+
+    t1 = t[indx1]
+    t2 = t[indx2]
+    t3 = t[indx3]
+
+    # Slice the results from the DNN
+    res1 = res[:,indx1]
+    res2 = res[:,indx2]
+    res3 = res[:,indx3]
+
+    # Slice the analytical results
+    res_analytical1 = res_analytical[:,indx1]
+    res_analytical2 = res_analytical[:,indx2]
+    res_analytical3 = res_analytical[:,indx3]
+
+    # Plot the slices
+    plt.figure(figsize=(10,10))
+    plt.title("Computed solutions at time = %g"%t1)
+    plt.plot(x, res1)
+    plt.plot(x,res_analytical1)
+    plt.legend(['dnn','analytical'])
+
+    plt.figure(figsize=(10,10))
+    plt.title("Computed solutions at time = %g"%t2)
+    plt.plot(x, res2)
+    plt.plot(x,res_analytical2)
+    plt.legend(['dnn','analytical'])
+
+    plt.figure(figsize=(10,10))
+    plt.title("Computed solutions at time = %g"%t3)
+    plt.plot(x, res3)
+    plt.plot(x,res_analytical3)
+    plt.legend(['dnn','analytical'])
+
+    plt.show()
+
+## Resources on differential equations and deep learning
+
+1. [Artificial neural networks for solving ordinary and partial differential equations by I.E. Lagaris et al](https://pdfs.semanticscholar.org/d061/df393e0e8fbfd0ea24976458b7d42419040d.pdf)
+
+2. [Neural networks for solving differential equations by A. Honchar](https://becominghuman.ai/neural-networks-for-solving-differential-equations-fa230ac5e04c)
+
+3. [Solving differential equations using neural networks by M.M Chiaramonte and M. Kiener](http://cs229.stanford.edu/proj2013/ChiaramonteKiener-SolvingDifferentialEquationsUsingNeuralNetworks.pdf)
+
+4. [Introduction to Partial Differential Equations by A. Tveito, R. Winther](https://www.springer.com/us/book/9783540225515)
\ No newline at end of file
diff --git a/doc/src/LectureNotes/_build/jupyter_execute/chapter10_11_326.png b/doc/src/LectureNotes/_build/jupyter_execute/chapter10_11_326.png
new file mode 100644
index 000000000..805fe41b2
Binary files /dev/null and b/doc/src/LectureNotes/_build/jupyter_execute/chapter10_11_326.png differ
diff --git a/doc/src/LectureNotes/_build/jupyter_execute/chapter10_11_328.png b/doc/src/LectureNotes/_build/jupyter_execute/chapter10_11_328.png
new file mode 100644
index 000000000..15cd8bd4a
Binary files /dev/null and b/doc/src/LectureNotes/_build/jupyter_execute/chapter10_11_328.png differ
diff --git a/doc/src/LectureNotes/_build/jupyter_execute/chapter10_11_687.png b/doc/src/LectureNotes/_build/jupyter_execute/chapter10_11_687.png
new file mode 100644
index 000000000..69b7bde49
Binary files /dev/null and b/doc/src/LectureNotes/_build/jupyter_execute/chapter10_11_687.png differ
diff --git a/doc/src/LectureNotes/_build/jupyter_execute/chapter10_11_688.png b/doc/src/LectureNotes/_build/jupyter_execute/chapter10_11_688.png
new file mode 100644
index 000000000..a6433f182
Binary files /dev/null and b/doc/src/LectureNotes/_build/jupyter_execute/chapter10_11_688.png differ
diff --git a/doc/src/LectureNotes/_build/jupyter_execute/chapter10_1_0.png b/doc/src/LectureNotes/_build/jupyter_execute/chapter10_1_0.png
new file mode 100644
index 000000000..1d4c16704
Binary files /dev/null and b/doc/src/LectureNotes/_build/jupyter_execute/chapter10_1_0.png differ
diff --git a/doc/src/LectureNotes/_build/jupyter_execute/chapter10_1_197.png b/doc/src/LectureNotes/_build/jupyter_execute/chapter10_1_197.png
new file mode 100644
index 000000000..35d7d55c4
Binary files /dev/null and b/doc/src/LectureNotes/_build/jupyter_execute/chapter10_1_197.png differ
diff --git a/doc/src/LectureNotes/_build/jupyter_execute/chapter10_59_2.png b/doc/src/LectureNotes/_build/jupyter_execute/chapter10_59_2.png
new file mode 100644
index 000000000..bb70c1d74
Binary files /dev/null and b/doc/src/LectureNotes/_build/jupyter_execute/chapter10_59_2.png differ
diff --git a/doc/src/LectureNotes/_build/jupyter_execute/chapter10_7_411.png b/doc/src/LectureNotes/_build/jupyter_execute/chapter10_7_411.png
new file mode 100644
index 000000000..33b4a158d
Binary files /dev/null and b/doc/src/LectureNotes/_build/jupyter_execute/chapter10_7_411.png differ
diff --git a/doc/src/LectureNotes/_build/jupyter_execute/chapter10_7_413.png b/doc/src/LectureNotes/_build/jupyter_execute/chapter10_7_413.png
new file mode 100644
index 000000000..93e7d1fc9
Binary files /dev/null and b/doc/src/LectureNotes/_build/jupyter_execute/chapter10_7_413.png differ
diff --git a/doc/src/LectureNotes/_build/jupyter_execute/chapter10_9_421.png b/doc/src/LectureNotes/_build/jupyter_execute/chapter10_9_421.png
new file mode 100644
index 000000000..ec63b7726
Binary files /dev/null and b/doc/src/LectureNotes/_build/jupyter_execute/chapter10_9_421.png differ
diff --git a/doc/src/LectureNotes/_build/jupyter_execute/chapter10_9_423.png b/doc/src/LectureNotes/_build/jupyter_execute/chapter10_9_423.png
new file mode 100644
index 000000000..b7afeb9ff
Binary files /dev/null and b/doc/src/LectureNotes/_build/jupyter_execute/chapter10_9_423.png differ
diff --git a/doc/src/LectureNotes/_build/jupyter_execute/chapter11.ipynb b/doc/src/LectureNotes/_build/jupyter_execute/chapter11.ipynb
new file mode 100644
index 000000000..bbc30b5cd
--- /dev/null
+++ b/doc/src/LectureNotes/_build/jupyter_execute/chapter11.ipynb
@@ -0,0 +1,1055 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "# Data Analysis and Machine Learning: \n",
+    "\n",
+    "  \n",
+    "**Christian Forssén**, Department of Physics, Chalmers University of Technology, Sweden  \n",
+    "\n",
+    " **Morten Hjorth-Jensen**, Department of Physics, University of Oslo and Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University\n",
+    "\n",
+    "Date: **Dec 23, 2020**\n",
+    "\n",
+    "Copyright 1999-2020, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "# Elements of Bayesian theory and Bayesian Neural Networks\n",
+    "\n",
+    "\n",
+    "## Why Bayesian Statistics?\n",
+    "\n",
+    "We have already made ourselves familiar with elements of a statistical\n",
+    "data analysis via quantities like the bias-variance tradeoff as well\n",
+    "as some central distribution functions such as the Normal\n",
+    "distribution, the binomial distribution and other probability\n",
+    "distribution functions.  \n",
+    "\n",
+    "In essentially all the Machine Learning algorithms we have studied,\n",
+    "our focus has been on a so-called **frequentist approach**, where\n",
+    "knowledge of an underlying likelihood function has not been\n",
+    "emphasized. Our data, whether we had a classification or a regression\n",
+    "problem, have been our central points of departure.\n",
+    "\n",
+    "Here we wish to merge this approach with the derivation of a\n",
+    "likelihood function which can be used to make prediction on how our\n",
+    "system under study evolves.  We will venture into the realm of what is\n",
+    "called Bayesian Neural Networks. To get an overarching view on what\n",
+    "this entails, the following figure conveys the essential differences\n",
+    "between a standard Neural network that we have met earlier and a\n",
+    "Bayesian Neural Network. In order to get there, we need to present\n",
+    "some of the basic elements of Bayesian statistics, starting with the\n",
+    "product rule and Bayes' theorem.\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "## Inference\n",
+    "Inference:\n",
+    "  :    \n",
+    "  \"the act of passing from one proposition, statement or judgment considered as true to another whose truth is believed to follow from that of the former\" (Webster) \n",
+    "  Do premises $A, B, \\ldots \\to$ hypothesis, $H$? \n",
+    "\n",
+    "Deductive inference:\n",
+    "  :    \n",
+    "  Premises allow definite determination of truth/falsity of H (syllogisms, symbolic logic, Boolean algebra) \n",
+    "  $B(H|A,B,...) = 0$ or $1$\n",
+    "\n",
+    "Inductive inference:\n",
+    "  :    \n",
+    "  Premises bear on truth/falsity of H, but don’t allow its definite determination (weak syllogisms, analogies)\n",
+    "  $A, B, C, D$ share properties $x, y, z$; $E$ has properties $x, y$\n",
+    "  $\\to$ $E$ probably has property $z$.\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "## Statistical Inference\n",
+    "* Quantify the strength of inductive inferences from facts, in the form of data ($D$), and other premises, e.g. models, to hypotheses about the phenomena producing the data.\n",
+    "\n",
+    "* Quantify via probabilities, or averages calculated using probabilities. Frequentists ($\\mathcal{F}$) and Bayesians ($\\mathcal{B}$) use probabilities very differently for this.\n",
+    "\n",
+    "* To the pioneers such as Bernoulli, Bayes and Laplace, a probability represented a *degree-of-belief* or plausability: how much they thought that something as true based on the evidence at hand. This is the Bayesian approach.\n",
+    "\n",
+    "* To the 19th century scholars, this seemed too vague and subjective. They redefined probability as the *long run relative frequency* with which an event occurred, given (infinitely) many repeated (experimental) trials.\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "## Some history\n",
+    "Adapted from D.S. Sivia[^Sivia]:\n",
+    "\n",
+    "[^Sivia]: Sivia, Devinderjit, and John Skilling. Data Analysis : A Bayesian Tutorial, OUP Oxford, 2006\n",
+    "\n",
+    "> Although the frequency definition appears to be more objective, its range of validity is also far more limited. For example, Laplace used (his) probability theory to estimate the mass of Saturn, given orbital data that were available to him from various astronomical observatories. In essence, he computed the posterior pdf for the mass M , given the data and all the relevant background information I (such as a knowledge of the laws of classical mechanics): prob(M|{data},I); this is shown schematically in the figure [Fig. 1.2].\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "
    \n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "> To Laplace, the (shaded) area under the posterior pdf curve between $m_1$ and $m_2$ was a measure of how much he believed that the mass of Saturn lay in the range $m_1 \\le M \\le m_2$. As such, the position of the maximum of the posterior pdf represents a best estimate of the mass; its width, or spread, about this optimal value gives an indication of the uncertainty in the estimate. Laplace stated that: ‘ . . . it is a bet of 11,000 to 1 that the error of this result is not 1/100th of its value.’ He would have won the bet, as another 150 years’ accumulation of data has changed the estimate by only 0.63%!\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "> According to the frequency definition, however, we are not permitted to use probability theory to tackle this problem. This is because the mass of Saturn is a constant and not a random variable; therefore, it has no frequency distribution and so probability theory cannot be used.\n",
+    "> \n",
+    "> If the pdf [of Fig. 1.2] had to be interpreted in terms of the frequency definition, we would have to imagine a large ensemble of universes in which everything remains constant apart from the mass of Saturn.\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "> As this scenario appears quite far-fetched, we might be inclined to think of [Fig. 1.2] in terms of the distribution of the measurements of the mass in many repetitions of the experiment. Although we are at liberty to think about a problem in any way that facilitates its solution, or our understanding of it, having to seek a frequency interpretation for every data analysis problem seems rather perverse.\n",
+    "> For example, what do we mean by the ‘measurement of the mass’ when the data consist of orbital periods? Besides, why should we have to think about many repetitions of an experiment that never happened? What we really want to do is to make the best inference of the mass given the (few) data that we actually have; this is precisely the Bayes and Laplace view of probability.\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "> Faced with the realization that the frequency definition of probability theory did not permit most real-life scientific problems to be addressed, a new subject was invented — statistics! To estimate the mass of Saturn, for example, one has to relate the mass to the data through some function called the statistic; since the data are subject to ‘random’ noise, the statistic becomes the random variable to which the rules of probability the- ory can be applied. But now the question arises: How should we choose the statistic? The frequentist approach does not yield a natural way of doing this and has, therefore, led to the development of several alternative schools of orthodox or conventional statis- tics. The masters, such as Fisher, Neyman and Pearson, provided a variety of different principles, which has merely resulted in a plethora of tests and procedures without any clear underlying rationale. This lack of unifying principles is, perhaps, at the heart of the shortcomings of the cook-book approach to statistics that students are often taught even today.\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "## The Bayesian recipe\n",
+    "Assess hypotheses by calculating their probabilities $p(H_i | \\ldots)$ conditional on known and/or presumed information using the rules of probability theory.\n",
+    "\n",
+    "\n",
+    "Probability Theory Axioms:\n",
+    "Product (AND) rule :\n",
+    "  :    \n",
+    "  $p(A, B | I) = p(A|I) p(B|A, I) = p(B|I)p(A|B,I)$\n",
+    "  Should read $p(A,B|I)$ as the probability for propositions $A$ AND $B$ being true given that $I$ is true.\n",
+    "\n",
+    "Sum (OR) rule:\n",
+    "  :    \n",
+    "  $p(A + B | I) = p(A | I) + p(B | I) - p(A, B | I)$\n",
+    "  $p(A+B|I)$ is the probability that proposition $A$ OR $B$ is true given that $I$ is true.\n",
+    "\n",
+    "Normalization:\n",
+    "  :    \n",
+    "  $p(A|I) + p(\\bar{A}|I) = 1$\n",
+    "  $\\bar{A}$ denotes the proposition that $A$ is false.\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "## Bayes' theorem\n",
+    "Bayes' theorem follows directly from the product rule"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "$$\n",
+    "p(A|B,I) = \\frac{p(B|A,I) p(A|I)}{p(B|I)}.\n",
+    "$$\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The importance of this property to data analysis becomes apparent if we replace $A$ and $B$ by hypothesis($H$) and data($D$):"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation}\n",
+    "p(H|D,I) = \\frac{p(D|H,I) p(H|I)}{p(D|I)}.\n",
+    "\\label{eq:bayes} \\tag{1}\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The power of Bayes’ theorem lies in the fact that it relates the quantity of interest, the probability that the hypothesis is true given the data, to the term we have a better chance of being able to assign, the probability that we would have observed the measured data if the hypothesis was true.\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "The various terms in Bayes’ theorem have formal names. \n",
+    "* The quantity on the far right, $p(H|I)$, is called the *prior* probability; it represents our state of knowledge (or ignorance) about the truth of the hypothesis before we have analysed the current data. \n",
+    "\n",
+    "* This is modified by the experimental measurements through $p(D|H,I)$, the *likelihood* function, \n",
+    "\n",
+    "* The denominator $p(D|I)$ is called the *evidence*. It does not depend on the hypothesis and can be regarded as a normalization constant.\n",
+    "\n",
+    "* Together, these yield the *posterior* probability, $p(H|D, I )$, representing our state of knowledge about the truth of the hypothesis in the light of the data. \n",
+    "\n",
+    "In a sense, Bayes’ theorem encapsulates the process of learning.\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "## The friends of Bayes' theorem\n",
+    "Normalization:\n",
+    "  :    \n",
+    "  $\\sum_i p(H_i|\\ldots) = 1$.\n",
+    "\n",
+    "Marginalization:\n",
+    "  :    \n",
+    "  $\\sum_i p(A,H_i|I) = \\sum_i p(H_i|A,I) p(A|I) = p(A|I)$.\n",
+    "\n",
+    "Marginalization (continuum limit):\n",
+    "  :    \n",
+    "  $\\int dx p(A,H(x)|I) = p(A|I)$.\n",
+    "\n",
+    "In the above, $H_i$ is an exclusive and exhaustive list of hypotheses. For example,let’s imagine that there are five candidates in a presidential election; then $H_1$ could be the proposition that the first candidate will win, and so on. The probability that $A$ is true, for example that unemployment will be lower in a year’s time (given all relevant information $I$, but irrespective of whoever becomes president) is then given by $\\sum_i p(A,H_i|I)$.\n",
+    "\n",
+    "In the continuum limit of propositions we must understand $p(\\ldots)$ as a pdf (probability density function).\n",
+    "\n",
+    "Marginalization is a very powerful device in data analysis because it enables us to deal with nuisance parameters; that is, quantities which necessarily enter the analysis but are of no intrinsic interest. The unwanted background signal present in many experimental measurements are examples of nuisance parameters.\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "## Inference With Parametric Models\n",
+    "Inductive inference with parametric models is a very important tool in the natural sciences.\n",
+    "* Consider $N$ different models $M_i$ ($i = 1, \\ldots, N$), each with parameters $\\boldsymbol{\\alpha}_i$. Each of them implies a sampling distribution (conditional predictive distribution for possible data)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "$$\n",
+    "p(D|\\boldsymbol{\\alpha}_i, M_i)\n",
+    "$$\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "* The $\\boldsymbol{\\alpha}_i$ dependence when we fix attention on the actual, observed data ($D_\\mathrm{obs}$) is the likelihood function:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "$$\n",
+    "\\mathcal{L}_i (\\boldsymbol{\\alpha}_i) \\equiv p(D_\\mathrm{obs}|\\boldsymbol{\\alpha}_i, M_i)\n",
+    "$$\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "* We may be uncertain about $i$ (model uncertainty),\n",
+    "\n",
+    "* or uncertain about $\\boldsymbol{\\alpha}_i$ (parameter uncertainty).\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "Parameter Estimation:\n",
+    "  :    \n",
+    "  Premise = choice of model (pick specific $i$)\n",
+    "  $\\Rightarrow$ What can we say about $\\boldsymbol{\\alpha}_i$?\n",
+    "\n",
+    "Model comparison:\n",
+    "  :    \n",
+    "  Premise = $\\{M_i\\}$\n",
+    "  $\\Rightarrow$ What can we say about $i$?\n",
+    "\n",
+    "Model adequacy:\n",
+    "  :    \n",
+    "  Premise = $M_1$\n",
+    "  $\\Rightarrow$ Is $M_1$ adequate?\n",
+    "\n",
+    "Hybrid Uncertainty:\n",
+    "  :    \n",
+    "  Models share some common params: $\\boldsymbol{\\alpha}_1 = \\{ \\boldsymbol{\\varphi}, \\boldsymbol{\\eta}_i\\}$\n",
+    "  $\\Rightarrow$ What can we say about $\\boldsymbol{\\varphi}$? (Systematic error is an example)\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "## Illustrative examples with python code\n",
+    "* Is this a fair coin? (analytical)\n",
+    "\n",
+    "* Flux from a star (single parameter, MCMC)\n",
+    "\n",
+    "* The lighthouse problem (two parameters, MCMC)\n",
+    "\n",
+    "* Linear fit with outliers (nuisance parameters)\n",
+    "\n",
+    "* ...\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "## Example: Is this a fair coin?\n",
+    "Let us begin with the analysis of data from a simple coin-tossing experiment. \n",
+    "Given that we had observed 6 heads in 8 flips, would you think it was a fair coin? By fair, we mean that we would be prepared to lay an even 1 : 1 bet on the outcome of a flip being a head or a tail. If we decide that the coin was fair, the question which follows naturally is how sure are we that this was so; if it was not fair, how unfair do we think it was? Furthermore, if we were to continue collecting data for this particular coin, observing the outcomes of additional flips, how would we update our belief on the fairness of the coin?\n",
+    "\n",
+    "A sensible way of formulating this problem is to consider a large number of hypotheses about the range in which the bias-weighting of the coin might lie. If we denote the bias-weighting by $H$, then $H = 0$ and $H = 1$ can represent a coin which produces a tail or a head on every flip, respectively. There is a continuum of possibilities for the value of H between these limits, with $H = 0.5$ indicating a fair coin. Our state of knowledge about the fairness, or the degree of unfairness, of the coin is then completely summarized by specifying how much we believe these various propositions to be true. \n",
+    "\n",
+    "Let us perform a computer simulation of a coin-tossing experiment. This provides the data that we will be analysing."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "0\n",
+    " \n",
+    "<\n",
+    "<\n",
+    "<\n",
+    "!\n",
+    "!\n",
+    "C\n",
+    "O\n",
+    "D\n",
+    "E\n",
+    "_\n",
+    "B\n",
+    "L\n",
+    "O\n",
+    "C\n",
+    "K\n",
+    " \n",
+    " \n",
+    "p\n",
+    "y\n",
+    "c\n",
+    "o\n",
+    "d"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "ename": "NameError",
+     "evalue": "name 'np' is not defined",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
+      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
+      "\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrandom\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mseed\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m999\u001b[0m\u001b[0;34m)\u001b[0m         \u001b[0;31m# for reproducibility\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      2\u001b[0m \u001b[0ma\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0.6\u001b[0m                       \u001b[0;31m# biased coin\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      3\u001b[0m \u001b[0mflips\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrandom\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrand\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m2\u001b[0m\u001b[0;34m**\u001b[0m\u001b[0;36m12\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;31m# simulates 4096 coin flips\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m \u001b[0mheads\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mflips\u001b[0m\u001b[0;34m<\u001b[0m\u001b[0ma\u001b[0m               \u001b[0;31m# boolean array, heads[i]=True if flip i is heads\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
+      "\u001b[0;31mNameError\u001b[0m: name 'np' is not defined"
+     ]
+    }
+   ],
+   "source": [
+    "np.random.seed(999)         # for reproducibility\n",
+    "a=0.6                       # biased coin\n",
+    "flips=np.random.rand(2**12) # simulates 4096 coin flips\n",
+    "heads=flips\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation}\n",
+    "\\int_0^1 p(H|D,I) dH = 1.\n",
+    "\\label{eq:coin_posterior_norm} \\tag{2}\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The prior pdf, $p(H|I)$, represents what we know about the coin given only the information $I$ that we are dealing with a ‘strange coin’. We could keep a very open mind about the nature of the coin; a simple probability assignment which reflects this is a uniform, or flat, prior"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation}\n",
+    "p(H|I) = \\left\\{ \\begin{array}{ll}\n",
+    "1 & 0 \\le H \\le 1, \\\\\n",
+    "0 & \\mathrm{otherwise}.\n",
+    "\\end{array} \\right.\n",
+    "\\label{eq:coin_prior_uniform} \\tag{3}\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "We will get back later to the choice of prior and its effect on the analysis.\n",
+    "\n",
+    "This prior state of knowledge, or ignorance, is modified by the data through the likelihood function $p(D|H,I)$. It is a measure of the chance that we would have obtained the data that we actually observed, if the value of the bias-weighting was given (as known). If, in the conditioning information $I$, we assume that the flips of the coin were independent events, so that the outcome of one did not influence that of another, then the probability of obtaining the data `R heads in N tosses' is given by the binomial distribution (we leave a formal definition of this to a statistics textbook)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation}\n",
+    "p(D|H,I) \\propto H^R (1-H)^{N-R}.\n",
+    "\\label{_auto1} \\tag{4}\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "It seems reasonable because $H$ is the chance of obtaining a head on any flip, and there were $R$ of them, and $1-H$ is the corresponding probability for a tail, of which there were $N-R$. We note that this binomial distribution also contains a normalization factor, but we will ignore it since it does not depend explicitly on $H$, the quantity of interest. It will be absorbed by the normalization condition ([2](#eq:coin_posterior_norm)).\n",
+    "\n",
+    "We perform the setup of this Bayesian framework on the computer."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "def prior(H):\n",
+    "    p=np.zeros_like(H)\n",
+    "    p[(0<=x)&(x<=1)]=1      # allowed range: 0<=H<=1\n",
+    "    return p                # uniform prior\n",
+    "def likelihood(H,data):\n",
+    "    N = len(data)\n",
+    "    no_of_heads = sum(data)\n",
+    "    no_of_tails = N - no_of_heads\n",
+    "    return H**no_of_heads * (1-H)**no_of_tails\n",
+    "def posterior(H,data):\n",
+    "    p=prior(H)*likelihood(H,data)\n",
+    "    norm=np.trapz(p,H)\n",
+    "    return p/norm"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The next step is to confront this setup with the simulated data. To get a feel for the result, it is instructive to see how the posterior pdf evolves as we obtain more and more data pertaining to the coin. The results of such an analyses is shown in Fig. [fig:coinflipping](#fig:coinflipping)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "x=np.linspace(0,1,100)\n",
+    "fig, axs = plt.subplots(nrows=4,ncols=3,sharex=True,sharey='row')\n",
+    "axs_vec=np.reshape(axs,-1)\n",
+    "axs_vec[0].plot(x,prior(x))\n",
+    "for ndouble in range(11):\n",
+    "    ax=axs_vec[1+ndouble]\n",
+    "    ax.plot(x,posterior(x,heads[:2**ndouble]))\n",
+    "    ax.text(0.1, 0.8, '$N={0}$'.format(2**ndouble), transform=ax.transAxes)\n",
+    "for row in range(4): axs[row,0].set_ylabel('$p(H|D_\\mathrm{obs},I)$')\n",
+    "for col in range(3): axs[-1,col].set_xlabel('$H$')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "\n",
+    "
    \n",
+    "\n",
+    "
    The evolution of the posterior pdf for the bias-weighting of a coin, as the number of data available increases. The figure on the top left-hand corner of each panel shows the number of data included in the analysis.
    \n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "The panel in the top left-hand corner shows the posterior pdf for $H$ given no data, i.e., it is the same as the prior pdf of Eq. ([3](#eq:coin_prior_uniform)). It indicates that we have no more reason to believe that the coin is fair than we have to think that it is double-headed, double-tailed, or of any other intermediate bias-weighting.\n",
+    "\n",
+    "The first flip is obviously tails. At this point we have no evidence that the coin has a side with heads, as indicated by the pdf going to zero as $H \\to 1$. The second flip is obviously heads and we have now excluded both extreme options $H=0$ (double-tailed) and $H=1$ (double-headed). We can note that the posterior at this point has the simple form $p(H|D,I) = H(1-H)$ for $0 \\le H \\le 1$.\n",
+    "\n",
+    "The remainder of Fig. [fig:coinflipping](#fig:coinflipping) shows how the posterior pdf evolves as the number of data analysed becomes larger and larger. We see that the position of the maximum moves around, but that the amount by which it does so decreases with the increasing number of observations. The width of the posterior pdf also becomes narrower with more data, indicating that we are becoming increasingly confident in our estimate of the bias-weighting. For the coin in this example, the best estimate of $H$ eventually converges to 0.6, which, of course, was the value chosen to simulate the flips.\n",
+    "\n",
+    "\n",
+    "## A few words on different priors\n",
+    "* uniform\n",
+    "\n",
+    "* Gaussian\n",
+    "\n",
+    "* Jeffrey's prior\n",
+    "\n",
+    "Repeat the coin flipping experiment with other priors.\n",
+    "\n",
+    "\n",
+    "## Bayesian parameter estimation (single parameter)\n",
+    "We will now consider the very important task of model parameter estimation using statistical inference. \n",
+    "[CF 1: maybe stress that model parameters are not random variables, and the meaning of parameter estimation is therefore very different between frequentist and bayesian approaches.]\n",
+    "\n",
+    "Throughout this section we will consider a specific example that involves a model with a single parameter: \"Measured flux from a star\".\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "### Example: Measured flux from a star\n",
+    "\n",
+    "Adapted from the blog [Pythonic Perambulations](http://jakevdp.github.io) by Jake VanderPlas.\n",
+    "\n",
+    "Imagine that we point our telescope to the sky, and observe the light coming from a single star. For the time being, we'll assume that the star's true flux is constant with time, i.e. that is it has a fixed value $F_\\mathrm{true}$ (we'll also ignore effects like sky noise and other sources of systematic error). We'll assume that we perform a series of $N$ measurements with our telescope, where the ith measurement reports the observed photon flux $F_i$ and error $e_i$[^errors].\n",
+    "The question is, given this set of measurements $D = \\{F_i, e_i\\}$, what is our best estimate of the true flux $F_\\mathrm{true}$?\n",
+    "\n",
+    "[^errors]: We'll make the reasonable assumption that errors are Gaussian. In a Frequentist perspective, $e_i$ is the standard deviation of the results of a single measurement event in the limit of repetitions of *that event*. In the Bayesian perspective, $e_i$ is the standard deviation of the (Gaussian) probability distribution describing our knowledge of that particular measurement given its observed value.\n",
+    "\n",
+    "Because the measurements are number counts, a Poisson distribution is a good approximation to the measurement process:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "np.random.seed(1)      # for repeatability\n",
+    "F_true = 1000          # true flux, say number of photons measured in 1 second\n",
+    "N = 50                 # number of measurements\n",
+    "F = stats.poisson(F_true).rvs(N)\n",
+    "                       # N measurements of the flux\n",
+    "e = np.sqrt(F)         # errors on Poisson counts estimated via square root"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Now let's make a simple visualization of the \"observed\" data, see Fig. [fig:flux](#fig:flux)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "fig, ax = plt.subplots()\n",
+    "ax.errorbar(F, np.arange(N), xerr=e, fmt='ok', ecolor='gray', alpha=0.5)\n",
+    "ax.vlines([F_true], 0, N, linewidth=5, alpha=0.2)\n",
+    "ax.set_xlabel(\"Flux\");ax.set_ylabel(\"measurement number\");"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "\n",
+    "
    \n",
+    "\n",
+    "
    Single photon counts (flux measurements).
    \n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "These measurements each have a different error $e_i$ which is estimated from Poisson statistics using the standard square-root rule. In this toy example we already know the true flux $F_\\mathrm{true}$, but the question is this: given our measurements and errors, what is our best estimate of the true flux?\n",
+    "\n",
+    "Let's take a look at the frequentist and Bayesian approaches to solving this.\n",
+    "\n",
+    "### Simple Photon Counts: Frequentist Approach\n",
+    "\n",
+    "We'll start with the classical frequentist maximum likelihood approach. Given a single observation $D_i = (F_i, e_i)$, we can compute the probability distribution of the measurement given the true flux Ftrue given our assumption of Gaussian errors"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation}\n",
+    "p(D_i | F_\\mathrm{true}, I) = \\frac{1}{\\sqrt{2\\pi e_i^2}} \\exp \\left( \\frac{-(F_i-F_\\mathrm{true})^2}{2e_i^2} \\right).\n",
+    "\\label{_auto2} \\tag{5}\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "This should be read \"the probability of $D_i$ given $F_\\mathrm{true}$\n",
+    "equals ...\". You should recognize this as a normal distribution with mean $F_\\mathrm{true}$ and standard deviation $e_i$.\n",
+    "\n",
+    "We construct the *likelihood function* by computing the product of the probabilities for each data point"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation}\n",
+    "\\mathcal{L}(D | F_\\mathrm{true}, I) = \\prod_{i=1}^N p(D_i | F_\\mathrm{true}, I),\n",
+    "\\label{_auto3} \\tag{6}\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "here $D = \\{D_i\\}$ represents the entire set of measurements. Because the value of the likelihood can become very small, it is often more convenient to instead compute the log-likelihood. Combining the previous two equations and computing the log, we have"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation}\n",
+    "\\log\\mathcal{L} = -\\frac{1}{2} \\sum_{i=1}^N \\left[ \\log(2\\pi e_i^2) +  \\frac{(F_i-F_\\mathrm{true})^2}{e_i^2} \\right].\n",
+    "\\label{_auto4} \\tag{7}\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "What we'd like to do is determine $F_\\mathrm{true}$ such that the likelihood is maximized. For this simple problem, the maximization can be computed analytically (i.e. by setting $d\\log\\mathcal{L}/d F_\\mathrm{true} = 0$). This results in the following observed estimate of $F_\\mathrm{true}$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation}\n",
+    "F_\\mathrm{est} = \\sum_{i=1}^N w_i F_i; \\quad w_i = 1/e_i^2.\n",
+    "\\label{_auto5} \\tag{8}\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Notice that in the special case of all errors $e_i$ being equal, this reduces to"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation}\n",
+    "F_\\mathrm{est} = \\frac{1}{N} \\sum_{i=1} F_i.\n",
+    "\\label{_auto6} \\tag{9}\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "That is, in agreement with intuition, $F_\\mathrm{est}$ is simply the mean of the observed data when errors are equal.\n",
+    "\n",
+    "We can go further and ask what the error of our estimate is. In the frequentist approach, this can be accomplished by fitting a Gaussian approximation to the likelihood curve at maximum; in this simple case this can also be solved analytically (the sum of Gaussians is also a Gaussian). It can be shown that the standard deviation of this Gaussian approximation is"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation}\n",
+    "\\sigma_\\mathrm{est} = \\sum_{i=1}^N w_i.\n",
+    "\\label{_auto7} \\tag{10}\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "These results are fairly simple calculations; let's evaluate them for our toy dataset:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "w=1./e**2\n",
+    "print(\"\"\"\n",
+    "F_true = {0}\n",
+    "F_est = {1:.0f} +/- {2:.0f} (based on {3} measurements) \"\"\"\\\n",
+    "          .format(F_true, (w * F).sum() / w.sum(), w.sum() ** -0.5, N))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "`F_true = 1000` \n",
+    "`F_est = 998 +/- 4 (based on 50 measurements)` \n",
+    "\n",
+    "We find that for 50 measurements of the flux, our estimate has an error of about 0.4% and is consistent with the input value.\n",
+    "\n",
+    "\n",
+    "### Simple Photon Counts: Bayesian Approach\n",
+    "\n",
+    "The Bayesian approach, as you might expect, begins and ends with probabilities. Our hypothesis is that the star has a constant flux $F_\\mathrm{true}$. It recognizes that what we fundamentally want to compute is our knowledge of the parameters in question given the data and other information (such as our knowledge of uncertainties for the observed values), i.e. in this case, $p(F_\\mathrm{true} | D,I)$.\n",
+    "Note that this formulation of the problem is fundamentally contrary to the frequentist philosophy, which says that probabilities have no meaning for model parameters like $F_\\mathrm{true}$. Nevertheless, within the Bayesian philosophy this is perfectly acceptable.\n",
+    "\n",
+    "To compute this result, Bayesians next apply Bayes' Theorem ([1](#eq:bayes)).\n",
+    "If we set the prior $p(F_\\mathrm{true}|I) \\propto 1$ (a flat prior), we find\n",
+    "$p(F_\\mathrm{true}|D,I) \\propto p(D | F_\\mathrm{true},I) \\equiv \\mathcal{L}(D | F_\\mathrm{true},I)$\n",
+    "and the Bayesian probability is maximized at precisely the same value as the frequentist result! So despite the philosophical differences, we see that (for this simple problem at least) the Bayesian and frequentist point estimates are equivalent.\n",
+    "\n",
+    "### A note about priors\n",
+    "\n",
+    "The prior allows inclusion of other information into the computation, which becomes very useful in cases where multiple measurement strategies are being combined to constrain a single model. The necessity to specify a prior, however, is one of the more controversial pieces of Bayesian analysis.\n",
+    "A frequentist will point out that the prior is problematic when no true prior information is available. Though it might seem straightforward to use a noninformative prior like the flat prior mentioned above, there are some [surprisingly subtleties](http://normaldeviate.wordpress.com/2013/07/13/lost-causes-in-statistics-ii-noninformative- priors/comment-page-1/) involved. It turns out that in many situations, a truly noninformative prior does not exist! Frequentists point out that the subjective choice of a prior which necessarily biases your result has no place in statistical data analysis.\n",
+    "A Bayesian would counter that frequentism doesn't solve this problem, but simply skirts the question. Frequentism can often be viewed as simply a special case of the Bayesian approach for some (implicit) choice of the prior: a Bayesian would say that it's better to make this implicit choice explicit, even if the choice might include some subjectivity.\n",
+    "\n",
+    "### Simple Photon Counts: Bayesian approach in practice\n",
+    "\n",
+    "Leaving these philosophical debates aside for the time being, let's address how Bayesian results are generally computed in practice. For a one parameter problem like the one considered here, it's as simple as computing the posterior probability $p(F_\\mathrm{true} | D,I)$ as a function of $F_\\mathrm{true}$: this is the distribution reflecting our knowledge of the parameter $F_\\mathrm{true}$.\n",
+    "But as the dimension of the model grows, this direct approach becomes increasingly intractable. For this reason, Bayesian calculations often depend on sampling methods such as Markov Chain Monte Carlo (MCMC). For this practical example, let us apply an MCMC approach using Dan Foreman-Mackey's [emcee](http://dan.iel.fm/emcee/current/) package. Keep in mind here that the goal is to generate a set of points drawn from the posterior probability distribution, and to use those points to determine the answer we seek.\n",
+    "To perform this MCMC, we start by defining Python functions for the prior $p(F_\\mathrm{true} | I)$, the likelihood $p(D | F_\\mathrm{true},I)$, and the posterior $p(F_\\mathrm{true} | D,I)$, noting that none of these need be properly normalized. Our model here is one-dimensional, but to handle multi-dimensional models we'll define the model in terms of an array of parameters $\\boldsymbol{\\alpha}$, which in this case is $\\boldsymbol{\\alpha} = [F_\\mathrm{true}]$"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "def log_prior(alpha):\n",
+    "    return 0 # flat prior\n",
+    "\n",
+    "def log_likelihood(alpha, F, e):\n",
+    "    return -0.5 * np.sum(np.log(2 * np.pi * e ** 2) \\\n",
+    "                             + (F - alpha[0]) ** 2 / e ** 2)\n",
+    "                             \n",
+    "def log_posterior(alpha, F, e):\n",
+    "    return log_prior(alpha) + log_likelihood(alpha, F, e)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Now we set up the problem, including generating some random starting guesses for the multiple chains of points."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "ndim = 1      # number of parameters in the model\n",
+    "nwalkers = 50 # number of MCMC walkers\n",
+    "nburn = 1000  # \"burn-in\" period to let chains stabilize\n",
+    "nsteps = 2000 # number of MCMC steps to take\n",
+    "# we'll start at random locations between 0 and 2000\n",
+    "starting_guesses = 2000 * np.random.rand(nwalkers, ndim)\n",
+    "sampler = emcee.EnsembleSampler(nwalkers, ndim, log_posterior, args=[F,e])\n",
+    "sampler.run_mcmc(starting_guesses, nsteps)\n",
+    "# Shape of sampler.chain  = (nwalkers, nsteps, ndim)\n",
+    "# Flatten the sampler chain and discard burn-in points:\n",
+    "samples = sampler.chain[:, nburn:, :].reshape((-1, ndim))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "If this all worked correctly, the array sample should contain a series of 50,000 points drawn from the posterior. Let's plot them and check. See results in Fig. [fig:flux-bayesian](#fig:flux-bayesian)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "fig, ax = plt.subplots()\n",
+    "ax.hist(samples, bins=50, histtype=\"stepfilled\", alpha=0.3, normed=True)\n",
+    "ax.set_xlabel(r'$F_\\mathrm{est}$')\n",
+    "ax.set_ylabel(r'$p(F_\\mathrm{est}|D,I)$')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "\n",
+    "
    \n",
+    "\n",
+    "
    Bayesian posterior pdf (represented by a histogram of MCMC samples) from flux measurements.
    \n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "### Best estimates and confidence intervals\n",
+    "\n",
+    "The posterior distribution from our Bayesian data analysis is the key quantity that encodes our inference about the values of the model parameters, given the data and the relevant background information. Often, however, we wish to summarize this result with just a few numbers: the best estimate and a measure of its reliability. \n",
+    "\n",
+    "There are a few different options for this. The choice of the most appropriate one depends mainly on the shape of the posterior distribution:\n",
+    "\n",
+    "*Symmetric posterior pdfs*: Since the probability (density) associated with any particular value of the parameter is a measure of how much we believe that it lies in the neighbourhood of that point, our best estimate is given by the maximum of the posterior pdf. If we denote the quantity of interest by $X$, with a posterior pdf $P =p(X|D,I)$, then the best estimate of its value $X_0$ is given by the condition $dP/dX|_{X=X_0}=0$. Strictly speaking, we should also check the sign of the second derivative to ensure that $X_0$ represents a maximum.\n",
+    "\n",
+    "To obtain a measure of the reliability of this best estimate, we need to look at the width or spread of the posterior pdf about $X_0$. When considering the behaviour of any function in the neighbourhood of a particular point, it is often helpful to carry out a Taylor series expansion; this is simply a standard tool for (locally) approximating a complicated function by a low-order polynomial. The linear term is zero at the maximum and the quadratic term is often the dominating one determining the width of the posterior pdf. Ignoring all the higher-order terms we arrive at the Gaussian approximation"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation}\n",
+    "p(X|D,I) \\approx \\frac{1}{\\sigma\\sqrt{2\\pi}} \\exp \\left[ -\\frac{(x-\\mu)^2}{2\\sigma^2} \\right],\n",
+    "\\label{_auto8} \\tag{11}\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "where the mean $\\mu = X_0$ and the variance $\\sigma = \\left( - \\left. \\frac{d^2L}{dX^2} \\right|_{X_0} \\right)^{-1/2}$, where $L$ is the logarithm of the posterior $P$. Our inference about the quantity of interest is conveyed very concisely, therefore, by the statement $X = X_0 \\pm \\sigma$, and"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "$$\n",
+    "p(X_0-\\sigma < X < X_0+\\sigma | D,I) = \\int_{X_0-\\sigma}^{X_0+\\sigma} p(X|D,I) dX \\approx 0.67.\n",
+    "$$\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "*Asymmetric posterior pdfs*: While the maximum of the posterior ($X_0$) can still be regarded as giving the best estimate, the true value is now more likely to be on one side of this rather than the other. Alternatively one can compute the mean value, $\\langle X \\rangle = \\int X p(X|D,I) dX$, although this tends to overemphasise very long tails. The best option is probably a compromise that can be employed when having access to a large sample from the posterior (as provided by an MCMC), namely to give the median of this ensamble.\n",
+    "\n",
+    "Furthermore, the concept of an error-bar does not seem appropriate in this case, as it implicitly entails the idea of symmetry. A good way of expressing the reliability with which a parameter can be inferred, for an asymmetric posterior pdf, is rather through a *confidence interval*. Since the area under the posterior pdf between $X_1$ and $X_2$ is proportional to how much we believe that $X$ lies in that range, the shortest interval that encloses 67% of the area represents a sensible measure of the uncertainty of the estimate. Obviously we can choose to provide some other degree-of-belief that we think is relevant for the case at hand. Assuming that the posterior pdf has been normalized, to have unit area, we need to find $X_1$ and $X_2$ such that:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "$$\n",
+    "p(X_1 < X < X_2 | D,I) = \\int_{X_1}^{X_2} p(X|D,I) dX \\approx 0.67, \n",
+    "$$\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "where the difference $X_2 - X_1$ is as small as possible. The region $X_1 < X < X_2$ is then called the shortest 67% confidence interval. \n",
+    "\n",
+    "*Multimodal posterior pdfs*: We can sometimes obtain posteriors which are multimodal; i.e. contains several disconnected regions with large probabilities. There is no difficulty when one of the maxima is very much larger than the others: we can simply ignore the subsidiary solutions, to a good approximation, and concentrate on the global maximum. The problem arises when there are several maxima of comparable magnitude. What do we now mean by a best estimate, and how should we quantify its reliability? The idea of a best estimate and an error-bar, or even a confidence interval, is merely an attempt to summarize the posterior with just two or three numbers; sometimes this just can’t be done, and so these concepts are not valid. For the bimodal case we might be able to characterize the posterior in terms of a few numbers: two best estimates and their associated error-bars, or disjoint confidence intervals. For a general multimodal pdf, the most honest thing we can do is just display the posterior itself.\n",
+    "\n",
+    "### Simple Photon Counts: Best estimates and confidence intervals\n",
+    "\n",
+    "To compute these numbers for our example, you would run:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "sampper=np.percentile(samples, [2.5, 16.5, 50, 83.5, 97.5],axis=0).flatten()\n",
+    "print(\"\"\"\n",
+    "F_true = {0}\n",
+    "Based on {1} measurements the posterior point estimates are:\n",
+    "...F_est = {2:.0f} +/- {3:.0f}\n",
+    "or using credible intervals:\n",
+    "...F_est = {4:.0f}          (posterior median) \n",
+    "...F_est in [{5:.0f}, {6:.0f}] (67% credible interval) \n",
+    "...F_est in [{7:.0f}, {8:.0f}] (95% credible interval) \"\"\"\\\n",
+    "          .format(F_true, N, np.mean(samples), np.std(samples), \\\n",
+    "                      sampper[2], sampper[1], sampper[3], sampper[0], sampper[4]))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "`F_true = 1000` \n",
+    "`Based on 50 measurements the posterior point estimates are:` \n",
+    "`...F_est = 998 +/- 4` \n",
+    "`or using credible intervals:` \n",
+    "`...F_est = 998          (posterior median)`  \n",
+    "`...F_est in [993, 1002] (67% credible interval)`  \n",
+    "`...F_est in [989, 1006] (95% credible interval)`  \n",
+    "\n",
+    "In this particular example, the posterior pdf is actually a Gaussian (since it is constructed as a product of Gaussians), and the mean and variance from the quadratic approximation will agree exactly with the frequentist approach.\n",
+    "\n",
+    "From this final result you might come away with the impression that the Bayesian method is unnecessarily complicated, and in this case it certainly is. Using an MCMC sampler to characterize a one-dimensional normal distribution is a bit like using the Death Star to destroy a beach ball, but we did this here because it demonstrates an approach that can scale to complicated posteriors in many, many dimensions, and can provide nice results in more complicated situations where an analytic likelihood approach is not possible.\n",
+    "\n",
+    "Furthermore, as data and models grow in complexity, the two approaches can diverge greatly. \n",
+    "\n",
+    "\n",
+    "## Bayesian parameter estimation (multiple parameters, covariance)\n",
+    "* multidimensional posterior pdf:s\n",
+    "\n",
+    "* nuisance parameters (e.g. background subtraction?)\n",
+    "\n",
+    "* corner plots, covariance, correlations\n",
+    "\n",
+    "* best example?\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "## Bayesian model selection\n",
+    "* Bayesian evidence\n",
+    "\n",
+    "* Occam's razor\n",
+    "\n",
+    "* Best example? How many spectral lines are there?"
+   ]
+  }
+ ],
+ "metadata": {
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.8.3"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}
\ No newline at end of file
diff --git a/doc/src/LectureNotes/_build/jupyter_execute/chapter11.py b/doc/src/LectureNotes/_build/jupyter_execute/chapter11.py
new file mode 100644
index 000000000..80b97fed9
--- /dev/null
+++ b/doc/src/LectureNotes/_build/jupyter_execute/chapter11.py
@@ -0,0 +1,712 @@
+
+# Data Analysis and Machine Learning: 
+
+  
+**Christian Forssén**, Department of Physics, Chalmers University of Technology, Sweden  
+
+ **Morten Hjorth-Jensen**, Department of Physics, University of Oslo and Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University
+
+Date: **Dec 23, 2020**
+
+Copyright 1999-2020, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
+
+
+
+
+# Elements of Bayesian theory and Bayesian Neural Networks
+
+
+## Why Bayesian Statistics?
+
+We have already made ourselves familiar with elements of a statistical
+data analysis via quantities like the bias-variance tradeoff as well
+as some central distribution functions such as the Normal
+distribution, the binomial distribution and other probability
+distribution functions.  
+
+In essentially all the Machine Learning algorithms we have studied,
+our focus has been on a so-called **frequentist approach**, where
+knowledge of an underlying likelihood function has not been
+emphasized. Our data, whether we had a classification or a regression
+problem, have been our central points of departure.
+
+Here we wish to merge this approach with the derivation of a
+likelihood function which can be used to make prediction on how our
+system under study evolves.  We will venture into the realm of what is
+called Bayesian Neural Networks. To get an overarching view on what
+this entails, the following figure conveys the essential differences
+between a standard Neural network that we have met earlier and a
+Bayesian Neural Network. In order to get there, we need to present
+some of the basic elements of Bayesian statistics, starting with the
+product rule and Bayes' theorem.
+
+
+
+
+## Inference
+Inference:
+  :    
+  "the act of passing from one proposition, statement or judgment considered as true to another whose truth is believed to follow from that of the former" (Webster) 
+  Do premises $A, B, \ldots \to$ hypothesis, $H$? 
+
+Deductive inference:
+  :    
+  Premises allow definite determination of truth/falsity of H (syllogisms, symbolic logic, Boolean algebra) 
+  $B(H|A,B,...) = 0$ or $1$
+
+Inductive inference:
+  :    
+  Premises bear on truth/falsity of H, but don’t allow its definite determination (weak syllogisms, analogies)
+  $A, B, C, D$ share properties $x, y, z$; $E$ has properties $x, y$
+  $\to$ $E$ probably has property $z$.
+
+
+
+
+## Statistical Inference
+* Quantify the strength of inductive inferences from facts, in the form of data ($D$), and other premises, e.g. models, to hypotheses about the phenomena producing the data.
+
+* Quantify via probabilities, or averages calculated using probabilities. Frequentists ($\mathcal{F}$) and Bayesians ($\mathcal{B}$) use probabilities very differently for this.
+
+* To the pioneers such as Bernoulli, Bayes and Laplace, a probability represented a *degree-of-belief* or plausability: how much they thought that something as true based on the evidence at hand. This is the Bayesian approach.
+
+* To the 19th century scholars, this seemed too vague and subjective. They redefined probability as the *long run relative frequency* with which an event occurred, given (infinitely) many repeated (experimental) trials.
+
+
+
+
+## Some history
+Adapted from D.S. Sivia[^Sivia]:
+
+[^Sivia]: Sivia, Devinderjit, and John Skilling. Data Analysis : A Bayesian Tutorial, OUP Oxford, 2006
+
+> Although the frequency definition appears to be more objective, its range of validity is also far more limited. For example, Laplace used (his) probability theory to estimate the mass of Saturn, given orbital data that were available to him from various astronomical observatories. In essence, he computed the posterior pdf for the mass M , given the data and all the relevant background information I (such as a knowledge of the laws of classical mechanics): prob(M|{data},I); this is shown schematically in the figure [Fig. 1.2].
+
+
+
+
+
+
+
+
+
+
    
+
+
+
+
+
+
+> To Laplace, the (shaded) area under the posterior pdf curve between $m_1$ and $m_2$ was a measure of how much he believed that the mass of Saturn lay in the range $m_1 \le M \le m_2$. As such, the position of the maximum of the posterior pdf represents a best estimate of the mass; its width, or spread, about this optimal value gives an indication of the uncertainty in the estimate. Laplace stated that: ‘ . . . it is a bet of 11,000 to 1 that the error of this result is not 1/100th of its value.’ He would have won the bet, as another 150 years’ accumulation of data has changed the estimate by only 0.63%!
+
+
+
+
+
+
+> According to the frequency definition, however, we are not permitted to use probability theory to tackle this problem. This is because the mass of Saturn is a constant and not a random variable; therefore, it has no frequency distribution and so probability theory cannot be used.
+> 
+> If the pdf [of Fig. 1.2] had to be interpreted in terms of the frequency definition, we would have to imagine a large ensemble of universes in which everything remains constant apart from the mass of Saturn.
+
+
+
+
+
+
+> As this scenario appears quite far-fetched, we might be inclined to think of [Fig. 1.2] in terms of the distribution of the measurements of the mass in many repetitions of the experiment. Although we are at liberty to think about a problem in any way that facilitates its solution, or our understanding of it, having to seek a frequency interpretation for every data analysis problem seems rather perverse.
+> For example, what do we mean by the ‘measurement of the mass’ when the data consist of orbital periods? Besides, why should we have to think about many repetitions of an experiment that never happened? What we really want to do is to make the best inference of the mass given the (few) data that we actually have; this is precisely the Bayes and Laplace view of probability.
+
+
+
+
+
+
+> Faced with the realization that the frequency definition of probability theory did not permit most real-life scientific problems to be addressed, a new subject was invented — statistics! To estimate the mass of Saturn, for example, one has to relate the mass to the data through some function called the statistic; since the data are subject to ‘random’ noise, the statistic becomes the random variable to which the rules of probability the- ory can be applied. But now the question arises: How should we choose the statistic? The frequentist approach does not yield a natural way of doing this and has, therefore, led to the development of several alternative schools of orthodox or conventional statis- tics. The masters, such as Fisher, Neyman and Pearson, provided a variety of different principles, which has merely resulted in a plethora of tests and procedures without any clear underlying rationale. This lack of unifying principles is, perhaps, at the heart of the shortcomings of the cook-book approach to statistics that students are often taught even today.
+
+
+
+
+
+
+## The Bayesian recipe
+Assess hypotheses by calculating their probabilities $p(H_i | \ldots)$ conditional on known and/or presumed information using the rules of probability theory.
+
+
+Probability Theory Axioms:
+Product (AND) rule :
+  :    
+  $p(A, B | I) = p(A|I) p(B|A, I) = p(B|I)p(A|B,I)$
+  Should read $p(A,B|I)$ as the probability for propositions $A$ AND $B$ being true given that $I$ is true.
+
+Sum (OR) rule:
+  :    
+  $p(A + B | I) = p(A | I) + p(B | I) - p(A, B | I)$
+  $p(A+B|I)$ is the probability that proposition $A$ OR $B$ is true given that $I$ is true.
+
+Normalization:
+  :    
+  $p(A|I) + p(\bar{A}|I) = 1$
+  $\bar{A}$ denotes the proposition that $A$ is false.
+
+
+
+
+## Bayes' theorem
+Bayes' theorem follows directly from the product rule
+
+$$
+$$
+p(A|B,I) = \frac{p(B|A,I) p(A|I)}{p(B|I)}.
+$$
+$$
+
+The importance of this property to data analysis becomes apparent if we replace $A$ and $B$ by hypothesis($H$) and data($D$):
+
+
+
    
+
+$$
+\begin{equation}
+p(H|D,I) = \frac{p(D|H,I) p(H|I)}{p(D|I)}.
+\label{eq:bayes} \tag{1}
+\end{equation}
+$$
+
+The power of Bayes’ theorem lies in the fact that it relates the quantity of interest, the probability that the hypothesis is true given the data, to the term we have a better chance of being able to assign, the probability that we would have observed the measured data if the hypothesis was true.
+
+
+
+
+The various terms in Bayes’ theorem have formal names. 
+* The quantity on the far right, $p(H|I)$, is called the *prior* probability; it represents our state of knowledge (or ignorance) about the truth of the hypothesis before we have analysed the current data. 
+
+* This is modified by the experimental measurements through $p(D|H,I)$, the *likelihood* function, 
+
+* The denominator $p(D|I)$ is called the *evidence*. It does not depend on the hypothesis and can be regarded as a normalization constant.
+
+* Together, these yield the *posterior* probability, $p(H|D, I )$, representing our state of knowledge about the truth of the hypothesis in the light of the data. 
+
+In a sense, Bayes’ theorem encapsulates the process of learning.
+
+
+
+
+## The friends of Bayes' theorem
+Normalization:
+  :    
+  $\sum_i p(H_i|\ldots) = 1$.
+
+Marginalization:
+  :    
+  $\sum_i p(A,H_i|I) = \sum_i p(H_i|A,I) p(A|I) = p(A|I)$.
+
+Marginalization (continuum limit):
+  :    
+  $\int dx p(A,H(x)|I) = p(A|I)$.
+
+In the above, $H_i$ is an exclusive and exhaustive list of hypotheses. For example,let’s imagine that there are five candidates in a presidential election; then $H_1$ could be the proposition that the first candidate will win, and so on. The probability that $A$ is true, for example that unemployment will be lower in a year’s time (given all relevant information $I$, but irrespective of whoever becomes president) is then given by $\sum_i p(A,H_i|I)$.
+
+In the continuum limit of propositions we must understand $p(\ldots)$ as a pdf (probability density function).
+
+Marginalization is a very powerful device in data analysis because it enables us to deal with nuisance parameters; that is, quantities which necessarily enter the analysis but are of no intrinsic interest. The unwanted background signal present in many experimental measurements are examples of nuisance parameters.
+
+
+
+
+## Inference With Parametric Models
+Inductive inference with parametric models is a very important tool in the natural sciences.
+* Consider $N$ different models $M_i$ ($i = 1, \ldots, N$), each with parameters $\boldsymbol{\alpha}_i$. Each of them implies a sampling distribution (conditional predictive distribution for possible data)
+
+$$
+$$
+p(D|\boldsymbol{\alpha}_i, M_i)
+$$
+$$
+
+* The $\boldsymbol{\alpha}_i$ dependence when we fix attention on the actual, observed data ($D_\mathrm{obs}$) is the likelihood function:
+
+$$
+$$
+\mathcal{L}_i (\boldsymbol{\alpha}_i) \equiv p(D_\mathrm{obs}|\boldsymbol{\alpha}_i, M_i)
+$$
+$$
+
+* We may be uncertain about $i$ (model uncertainty),
+
+* or uncertain about $\boldsymbol{\alpha}_i$ (parameter uncertainty).
+
+
+
+
+Parameter Estimation:
+  :    
+  Premise = choice of model (pick specific $i$)
+  $\Rightarrow$ What can we say about $\boldsymbol{\alpha}_i$?
+
+Model comparison:
+  :    
+  Premise = $\{M_i\}$
+  $\Rightarrow$ What can we say about $i$?
+
+Model adequacy:
+  :    
+  Premise = $M_1$
+  $\Rightarrow$ Is $M_1$ adequate?
+
+Hybrid Uncertainty:
+  :    
+  Models share some common params: $\boldsymbol{\alpha}_1 = \{ \boldsymbol{\varphi}, \boldsymbol{\eta}_i\}$
+  $\Rightarrow$ What can we say about $\boldsymbol{\varphi}$? (Systematic error is an example)
+
+
+
+
+## Illustrative examples with python code
+* Is this a fair coin? (analytical)
+
+* Flux from a star (single parameter, MCMC)
+
+* The lighthouse problem (two parameters, MCMC)
+
+* Linear fit with outliers (nuisance parameters)
+
+* ...
+
+
+
+
+## Example: Is this a fair coin?
+Let us begin with the analysis of data from a simple coin-tossing experiment. 
+Given that we had observed 6 heads in 8 flips, would you think it was a fair coin? By fair, we mean that we would be prepared to lay an even 1 : 1 bet on the outcome of a flip being a head or a tail. If we decide that the coin was fair, the question which follows naturally is how sure are we that this was so; if it was not fair, how unfair do we think it was? Furthermore, if we were to continue collecting data for this particular coin, observing the outcomes of additional flips, how would we update our belief on the fairness of the coin?
+
+A sensible way of formulating this problem is to consider a large number of hypotheses about the range in which the bias-weighting of the coin might lie. If we denote the bias-weighting by $H$, then $H = 0$ and $H = 1$ can represent a coin which produces a tail or a head on every flip, respectively. There is a continuum of possibilities for the value of H between these limits, with $H = 0.5$ indicating a fair coin. Our state of knowledge about the fairness, or the degree of unfairness, of the coin is then completely summarized by specifying how much we believe these various propositions to be true. 
+
+Let us perform a computer simulation of a coin-tossing experiment. This provides the data that we will be analysing.
+
+0
+ 
+<
+<
+<
+!
+!
+C
+O
+D
+E
+_
+B
+L
+O
+C
+K
+ 
+ 
+p
+y
+c
+o
+d
+
+np.random.seed(999)         # for reproducibility
+a=0.6                       # biased coin
+flips=np.random.rand(2**12) # simulates 4096 coin flips
+heads=flips    
+
    
+
+$$
+\begin{equation}
+\int_0^1 p(H|D,I) dH = 1.
+\label{eq:coin_posterior_norm} \tag{2}
+\end{equation}
+$$
+
+The prior pdf, $p(H|I)$, represents what we know about the coin given only the information $I$ that we are dealing with a ‘strange coin’. We could keep a very open mind about the nature of the coin; a simple probability assignment which reflects this is a uniform, or flat, prior
+
+
+
    
+
+$$
+\begin{equation}
+p(H|I) = \left\{ \begin{array}{ll}
+1 & 0 \le H \le 1, \\
+0 & \mathrm{otherwise}.
+\end{array} \right.
+\label{eq:coin_prior_uniform} \tag{3}
+\end{equation}
+$$
+
+We will get back later to the choice of prior and its effect on the analysis.
+
+This prior state of knowledge, or ignorance, is modified by the data through the likelihood function $p(D|H,I)$. It is a measure of the chance that we would have obtained the data that we actually observed, if the value of the bias-weighting was given (as known). If, in the conditioning information $I$, we assume that the flips of the coin were independent events, so that the outcome of one did not influence that of another, then the probability of obtaining the data `R heads in N tosses' is given by the binomial distribution (we leave a formal definition of this to a statistics textbook)
+
+
+
    
+
+$$
+\begin{equation}
+p(D|H,I) \propto H^R (1-H)^{N-R}.
+\label{_auto1} \tag{4}
+\end{equation}
+$$
+
+It seems reasonable because $H$ is the chance of obtaining a head on any flip, and there were $R$ of them, and $1-H$ is the corresponding probability for a tail, of which there were $N-R$. We note that this binomial distribution also contains a normalization factor, but we will ignore it since it does not depend explicitly on $H$, the quantity of interest. It will be absorbed by the normalization condition ([2](#eq:coin_posterior_norm)).
+
+We perform the setup of this Bayesian framework on the computer.
+
+def prior(H):
+    p=np.zeros_like(H)
+    p[(0<=x)&(x<=1)]=1      # allowed range: 0<=H<=1
+    return p                # uniform prior
+def likelihood(H,data):
+    N = len(data)
+    no_of_heads = sum(data)
+    no_of_tails = N - no_of_heads
+    return H**no_of_heads * (1-H)**no_of_tails
+def posterior(H,data):
+    p=prior(H)*likelihood(H,data)
+    norm=np.trapz(p,H)
+    return p/norm
+
+The next step is to confront this setup with the simulated data. To get a feel for the result, it is instructive to see how the posterior pdf evolves as we obtain more and more data pertaining to the coin. The results of such an analyses is shown in Fig. [fig:coinflipping](#fig:coinflipping).
+
+x=np.linspace(0,1,100)
+fig, axs = plt.subplots(nrows=4,ncols=3,sharex=True,sharey='row')
+axs_vec=np.reshape(axs,-1)
+axs_vec[0].plot(x,prior(x))
+for ndouble in range(11):
+    ax=axs_vec[1+ndouble]
+    ax.plot(x,posterior(x,heads[:2**ndouble]))
+    ax.text(0.1, 0.8, '$N={0}$'.format(2**ndouble), transform=ax.transAxes)
+for row in range(4): axs[row,0].set_ylabel('$p(H|D_\mathrm{obs},I)$')
+for col in range(3): axs[-1,col].set_xlabel('$H$')
+
+
+
+
    
+
+
    The evolution of the posterior pdf for the bias-weighting of a coin, as the number of data available increases. The figure on the top left-hand corner of each panel shows the number of data included in the analysis.
    
+
+
+
+
+
+The panel in the top left-hand corner shows the posterior pdf for $H$ given no data, i.e., it is the same as the prior pdf of Eq. ([3](#eq:coin_prior_uniform)). It indicates that we have no more reason to believe that the coin is fair than we have to think that it is double-headed, double-tailed, or of any other intermediate bias-weighting.
+
+The first flip is obviously tails. At this point we have no evidence that the coin has a side with heads, as indicated by the pdf going to zero as $H \to 1$. The second flip is obviously heads and we have now excluded both extreme options $H=0$ (double-tailed) and $H=1$ (double-headed). We can note that the posterior at this point has the simple form $p(H|D,I) = H(1-H)$ for $0 \le H \le 1$.
+
+The remainder of Fig. [fig:coinflipping](#fig:coinflipping) shows how the posterior pdf evolves as the number of data analysed becomes larger and larger. We see that the position of the maximum moves around, but that the amount by which it does so decreases with the increasing number of observations. The width of the posterior pdf also becomes narrower with more data, indicating that we are becoming increasingly confident in our estimate of the bias-weighting. For the coin in this example, the best estimate of $H$ eventually converges to 0.6, which, of course, was the value chosen to simulate the flips.
+
+
+## A few words on different priors
+* uniform
+
+* Gaussian
+
+* Jeffrey's prior
+
+Repeat the coin flipping experiment with other priors.
+
+
+## Bayesian parameter estimation (single parameter)
+We will now consider the very important task of model parameter estimation using statistical inference. 
+[CF 1: maybe stress that model parameters are not random variables, and the meaning of parameter estimation is therefore very different between frequentist and bayesian approaches.]
+
+Throughout this section we will consider a specific example that involves a model with a single parameter: "Measured flux from a star".
+
+
+
+
+### Example: Measured flux from a star
+
+Adapted from the blog [Pythonic Perambulations](http://jakevdp.github.io) by Jake VanderPlas.
+
+Imagine that we point our telescope to the sky, and observe the light coming from a single star. For the time being, we'll assume that the star's true flux is constant with time, i.e. that is it has a fixed value $F_\mathrm{true}$ (we'll also ignore effects like sky noise and other sources of systematic error). We'll assume that we perform a series of $N$ measurements with our telescope, where the ith measurement reports the observed photon flux $F_i$ and error $e_i$[^errors].
+The question is, given this set of measurements $D = \{F_i, e_i\}$, what is our best estimate of the true flux $F_\mathrm{true}$?
+
+[^errors]: We'll make the reasonable assumption that errors are Gaussian. In a Frequentist perspective, $e_i$ is the standard deviation of the results of a single measurement event in the limit of repetitions of *that event*. In the Bayesian perspective, $e_i$ is the standard deviation of the (Gaussian) probability distribution describing our knowledge of that particular measurement given its observed value.
+
+Because the measurements are number counts, a Poisson distribution is a good approximation to the measurement process:
+
+np.random.seed(1)      # for repeatability
+F_true = 1000          # true flux, say number of photons measured in 1 second
+N = 50                 # number of measurements
+F = stats.poisson(F_true).rvs(N)
+                       # N measurements of the flux
+e = np.sqrt(F)         # errors on Poisson counts estimated via square root
+
+Now let's make a simple visualization of the "observed" data, see Fig. [fig:flux](#fig:flux).
+
+fig, ax = plt.subplots()
+ax.errorbar(F, np.arange(N), xerr=e, fmt='ok', ecolor='gray', alpha=0.5)
+ax.vlines([F_true], 0, N, linewidth=5, alpha=0.2)
+ax.set_xlabel("Flux");ax.set_ylabel("measurement number");
+
+
+
+
    
+
+
    Single photon counts (flux measurements).
    
+
+
+
+
+
+These measurements each have a different error $e_i$ which is estimated from Poisson statistics using the standard square-root rule. In this toy example we already know the true flux $F_\mathrm{true}$, but the question is this: given our measurements and errors, what is our best estimate of the true flux?
+
+Let's take a look at the frequentist and Bayesian approaches to solving this.
+
+### Simple Photon Counts: Frequentist Approach
+
+We'll start with the classical frequentist maximum likelihood approach. Given a single observation $D_i = (F_i, e_i)$, we can compute the probability distribution of the measurement given the true flux Ftrue given our assumption of Gaussian errors
+
+
+
    
+
+$$
+\begin{equation}
+p(D_i | F_\mathrm{true}, I) = \frac{1}{\sqrt{2\pi e_i^2}} \exp \left( \frac{-(F_i-F_\mathrm{true})^2}{2e_i^2} \right).
+\label{_auto2} \tag{5}
+\end{equation}
+$$
+
+This should be read "the probability of $D_i$ given $F_\mathrm{true}$
+equals ...". You should recognize this as a normal distribution with mean $F_\mathrm{true}$ and standard deviation $e_i$.
+
+We construct the *likelihood function* by computing the product of the probabilities for each data point
+
+
+
    
+
+$$
+\begin{equation}
+\mathcal{L}(D | F_\mathrm{true}, I) = \prod_{i=1}^N p(D_i | F_\mathrm{true}, I),
+\label{_auto3} \tag{6}
+\end{equation}
+$$
+
+here $D = \{D_i\}$ represents the entire set of measurements. Because the value of the likelihood can become very small, it is often more convenient to instead compute the log-likelihood. Combining the previous two equations and computing the log, we have
+
+
+
    
+
+$$
+\begin{equation}
+\log\mathcal{L} = -\frac{1}{2} \sum_{i=1}^N \left[ \log(2\pi e_i^2) +  \frac{(F_i-F_\mathrm{true})^2}{e_i^2} \right].
+\label{_auto4} \tag{7}
+\end{equation}
+$$
+
+What we'd like to do is determine $F_\mathrm{true}$ such that the likelihood is maximized. For this simple problem, the maximization can be computed analytically (i.e. by setting $d\log\mathcal{L}/d F_\mathrm{true} = 0$). This results in the following observed estimate of $F_\mathrm{true}$
+
+
+
    
+
+$$
+\begin{equation}
+F_\mathrm{est} = \sum_{i=1}^N w_i F_i; \quad w_i = 1/e_i^2.
+\label{_auto5} \tag{8}
+\end{equation}
+$$
+
+Notice that in the special case of all errors $e_i$ being equal, this reduces to
+
+
+
    
+
+$$
+\begin{equation}
+F_\mathrm{est} = \frac{1}{N} \sum_{i=1} F_i.
+\label{_auto6} \tag{9}
+\end{equation}
+$$
+
+That is, in agreement with intuition, $F_\mathrm{est}$ is simply the mean of the observed data when errors are equal.
+
+We can go further and ask what the error of our estimate is. In the frequentist approach, this can be accomplished by fitting a Gaussian approximation to the likelihood curve at maximum; in this simple case this can also be solved analytically (the sum of Gaussians is also a Gaussian). It can be shown that the standard deviation of this Gaussian approximation is
+
+
+
    
+
+$$
+\begin{equation}
+\sigma_\mathrm{est} = \sum_{i=1}^N w_i.
+\label{_auto7} \tag{10}
+\end{equation}
+$$
+
+These results are fairly simple calculations; let's evaluate them for our toy dataset:
+
+w=1./e**2
+print("""
+F_true = {0}
+F_est = {1:.0f} +/- {2:.0f} (based on {3} measurements) """\
+          .format(F_true, (w * F).sum() / w.sum(), w.sum() ** -0.5, N))
+
+`F_true = 1000` 
+`F_est = 998 +/- 4 (based on 50 measurements)` 
+
+We find that for 50 measurements of the flux, our estimate has an error of about 0.4% and is consistent with the input value.
+
+
+### Simple Photon Counts: Bayesian Approach
+
+The Bayesian approach, as you might expect, begins and ends with probabilities. Our hypothesis is that the star has a constant flux $F_\mathrm{true}$. It recognizes that what we fundamentally want to compute is our knowledge of the parameters in question given the data and other information (such as our knowledge of uncertainties for the observed values), i.e. in this case, $p(F_\mathrm{true} | D,I)$.
+Note that this formulation of the problem is fundamentally contrary to the frequentist philosophy, which says that probabilities have no meaning for model parameters like $F_\mathrm{true}$. Nevertheless, within the Bayesian philosophy this is perfectly acceptable.
+
+To compute this result, Bayesians next apply Bayes' Theorem ([1](#eq:bayes)).
+If we set the prior $p(F_\mathrm{true}|I) \propto 1$ (a flat prior), we find
+$p(F_\mathrm{true}|D,I) \propto p(D | F_\mathrm{true},I) \equiv \mathcal{L}(D | F_\mathrm{true},I)$
+and the Bayesian probability is maximized at precisely the same value as the frequentist result! So despite the philosophical differences, we see that (for this simple problem at least) the Bayesian and frequentist point estimates are equivalent.
+
+### A note about priors
+
+The prior allows inclusion of other information into the computation, which becomes very useful in cases where multiple measurement strategies are being combined to constrain a single model. The necessity to specify a prior, however, is one of the more controversial pieces of Bayesian analysis.
+A frequentist will point out that the prior is problematic when no true prior information is available. Though it might seem straightforward to use a noninformative prior like the flat prior mentioned above, there are some [surprisingly subtleties](http://normaldeviate.wordpress.com/2013/07/13/lost-causes-in-statistics-ii-noninformative- priors/comment-page-1/) involved. It turns out that in many situations, a truly noninformative prior does not exist! Frequentists point out that the subjective choice of a prior which necessarily biases your result has no place in statistical data analysis.
+A Bayesian would counter that frequentism doesn't solve this problem, but simply skirts the question. Frequentism can often be viewed as simply a special case of the Bayesian approach for some (implicit) choice of the prior: a Bayesian would say that it's better to make this implicit choice explicit, even if the choice might include some subjectivity.
+
+### Simple Photon Counts: Bayesian approach in practice
+
+Leaving these philosophical debates aside for the time being, let's address how Bayesian results are generally computed in practice. For a one parameter problem like the one considered here, it's as simple as computing the posterior probability $p(F_\mathrm{true} | D,I)$ as a function of $F_\mathrm{true}$: this is the distribution reflecting our knowledge of the parameter $F_\mathrm{true}$.
+But as the dimension of the model grows, this direct approach becomes increasingly intractable. For this reason, Bayesian calculations often depend on sampling methods such as Markov Chain Monte Carlo (MCMC). For this practical example, let us apply an MCMC approach using Dan Foreman-Mackey's [emcee](http://dan.iel.fm/emcee/current/) package. Keep in mind here that the goal is to generate a set of points drawn from the posterior probability distribution, and to use those points to determine the answer we seek.
+To perform this MCMC, we start by defining Python functions for the prior $p(F_\mathrm{true} | I)$, the likelihood $p(D | F_\mathrm{true},I)$, and the posterior $p(F_\mathrm{true} | D,I)$, noting that none of these need be properly normalized. Our model here is one-dimensional, but to handle multi-dimensional models we'll define the model in terms of an array of parameters $\boldsymbol{\alpha}$, which in this case is $\boldsymbol{\alpha} = [F_\mathrm{true}]$
+
+def log_prior(alpha):
+    return 0 # flat prior
+
+def log_likelihood(alpha, F, e):
+    return -0.5 * np.sum(np.log(2 * np.pi * e ** 2) \
+                             + (F - alpha[0]) ** 2 / e ** 2)
+                             
+def log_posterior(alpha, F, e):
+    return log_prior(alpha) + log_likelihood(alpha, F, e)
+
+Now we set up the problem, including generating some random starting guesses for the multiple chains of points.
+
+ndim = 1      # number of parameters in the model
+nwalkers = 50 # number of MCMC walkers
+nburn = 1000  # "burn-in" period to let chains stabilize
+nsteps = 2000 # number of MCMC steps to take
+# we'll start at random locations between 0 and 2000
+starting_guesses = 2000 * np.random.rand(nwalkers, ndim)
+sampler = emcee.EnsembleSampler(nwalkers, ndim, log_posterior, args=[F,e])
+sampler.run_mcmc(starting_guesses, nsteps)
+# Shape of sampler.chain  = (nwalkers, nsteps, ndim)
+# Flatten the sampler chain and discard burn-in points:
+samples = sampler.chain[:, nburn:, :].reshape((-1, ndim))
+
+If this all worked correctly, the array sample should contain a series of 50,000 points drawn from the posterior. Let's plot them and check. See results in Fig. [fig:flux-bayesian](#fig:flux-bayesian).
+
+fig, ax = plt.subplots()
+ax.hist(samples, bins=50, histtype="stepfilled", alpha=0.3, normed=True)
+ax.set_xlabel(r'$F_\mathrm{est}$')
+ax.set_ylabel(r'$p(F_\mathrm{est}|D,I)$')
+
+
+
+
    
+
+
    Bayesian posterior pdf (represented by a histogram of MCMC samples) from flux measurements.
    
+
+
+
+
+
+### Best estimates and confidence intervals
+
+The posterior distribution from our Bayesian data analysis is the key quantity that encodes our inference about the values of the model parameters, given the data and the relevant background information. Often, however, we wish to summarize this result with just a few numbers: the best estimate and a measure of its reliability. 
+
+There are a few different options for this. The choice of the most appropriate one depends mainly on the shape of the posterior distribution:
+
+*Symmetric posterior pdfs*: Since the probability (density) associated with any particular value of the parameter is a measure of how much we believe that it lies in the neighbourhood of that point, our best estimate is given by the maximum of the posterior pdf. If we denote the quantity of interest by $X$, with a posterior pdf $P =p(X|D,I)$, then the best estimate of its value $X_0$ is given by the condition $dP/dX|_{X=X_0}=0$. Strictly speaking, we should also check the sign of the second derivative to ensure that $X_0$ represents a maximum.
+
+To obtain a measure of the reliability of this best estimate, we need to look at the width or spread of the posterior pdf about $X_0$. When considering the behaviour of any function in the neighbourhood of a particular point, it is often helpful to carry out a Taylor series expansion; this is simply a standard tool for (locally) approximating a complicated function by a low-order polynomial. The linear term is zero at the maximum and the quadratic term is often the dominating one determining the width of the posterior pdf. Ignoring all the higher-order terms we arrive at the Gaussian approximation
+
+
+
    
+
+$$
+\begin{equation}
+p(X|D,I) \approx \frac{1}{\sigma\sqrt{2\pi}} \exp \left[ -\frac{(x-\mu)^2}{2\sigma^2} \right],
+\label{_auto8} \tag{11}
+\end{equation}
+$$
+
+where the mean $\mu = X_0$ and the variance $\sigma = \left( - \left. \frac{d^2L}{dX^2} \right|_{X_0} \right)^{-1/2}$, where $L$ is the logarithm of the posterior $P$. Our inference about the quantity of interest is conveyed very concisely, therefore, by the statement $X = X_0 \pm \sigma$, and
+
+$$
+$$
+p(X_0-\sigma < X < X_0+\sigma | D,I) = \int_{X_0-\sigma}^{X_0+\sigma} p(X|D,I) dX \approx 0.67.
+$$
+$$
+
+*Asymmetric posterior pdfs*: While the maximum of the posterior ($X_0$) can still be regarded as giving the best estimate, the true value is now more likely to be on one side of this rather than the other. Alternatively one can compute the mean value, $\langle X \rangle = \int X p(X|D,I) dX$, although this tends to overemphasise very long tails. The best option is probably a compromise that can be employed when having access to a large sample from the posterior (as provided by an MCMC), namely to give the median of this ensamble.
+
+Furthermore, the concept of an error-bar does not seem appropriate in this case, as it implicitly entails the idea of symmetry. A good way of expressing the reliability with which a parameter can be inferred, for an asymmetric posterior pdf, is rather through a *confidence interval*. Since the area under the posterior pdf between $X_1$ and $X_2$ is proportional to how much we believe that $X$ lies in that range, the shortest interval that encloses 67% of the area represents a sensible measure of the uncertainty of the estimate. Obviously we can choose to provide some other degree-of-belief that we think is relevant for the case at hand. Assuming that the posterior pdf has been normalized, to have unit area, we need to find $X_1$ and $X_2$ such that:
+
+$$
+$$
+p(X_1 < X < X_2 | D,I) = \int_{X_1}^{X_2} p(X|D,I) dX \approx 0.67, 
+$$
+$$
+
+where the difference $X_2 - X_1$ is as small as possible. The region $X_1 < X < X_2$ is then called the shortest 67% confidence interval. 
+
+*Multimodal posterior pdfs*: We can sometimes obtain posteriors which are multimodal; i.e. contains several disconnected regions with large probabilities. There is no difficulty when one of the maxima is very much larger than the others: we can simply ignore the subsidiary solutions, to a good approximation, and concentrate on the global maximum. The problem arises when there are several maxima of comparable magnitude. What do we now mean by a best estimate, and how should we quantify its reliability? The idea of a best estimate and an error-bar, or even a confidence interval, is merely an attempt to summarize the posterior with just two or three numbers; sometimes this just can’t be done, and so these concepts are not valid. For the bimodal case we might be able to characterize the posterior in terms of a few numbers: two best estimates and their associated error-bars, or disjoint confidence intervals. For a general multimodal pdf, the most honest thing we can do is just display the posterior itself.
+
+### Simple Photon Counts: Best estimates and confidence intervals
+
+To compute these numbers for our example, you would run:
+
+sampper=np.percentile(samples, [2.5, 16.5, 50, 83.5, 97.5],axis=0).flatten()
+print("""
+F_true = {0}
+Based on {1} measurements the posterior point estimates are:
+...F_est = {2:.0f} +/- {3:.0f}
+or using credible intervals:
+...F_est = {4:.0f}          (posterior median) 
+...F_est in [{5:.0f}, {6:.0f}] (67% credible interval) 
+...F_est in [{7:.0f}, {8:.0f}] (95% credible interval) """\
+          .format(F_true, N, np.mean(samples), np.std(samples), \
+                      sampper[2], sampper[1], sampper[3], sampper[0], sampper[4]))
+
+`F_true = 1000` 
+`Based on 50 measurements the posterior point estimates are:` 
+`...F_est = 998 +/- 4` 
+`or using credible intervals:` 
+`...F_est = 998          (posterior median)`  
+`...F_est in [993, 1002] (67% credible interval)`  
+`...F_est in [989, 1006] (95% credible interval)`  
+
+In this particular example, the posterior pdf is actually a Gaussian (since it is constructed as a product of Gaussians), and the mean and variance from the quadratic approximation will agree exactly with the frequentist approach.
+
+From this final result you might come away with the impression that the Bayesian method is unnecessarily complicated, and in this case it certainly is. Using an MCMC sampler to characterize a one-dimensional normal distribution is a bit like using the Death Star to destroy a beach ball, but we did this here because it demonstrates an approach that can scale to complicated posteriors in many, many dimensions, and can provide nice results in more complicated situations where an analytic likelihood approach is not possible.
+
+Furthermore, as data and models grow in complexity, the two approaches can diverge greatly. 
+
+
+## Bayesian parameter estimation (multiple parameters, covariance)
+* multidimensional posterior pdf:s
+
+* nuisance parameters (e.g. background subtraction?)
+
+* corner plots, covariance, correlations
+
+* best example?
+
+
+
+
+
+## Bayesian model selection
+* Bayesian evidence
+
+* Occam's razor
+
+* Best example? How many spectral lines are there?
\ No newline at end of file
diff --git a/doc/src/LectureNotes/_build/jupyter_execute/chapter2.ipynb b/doc/src/LectureNotes/_build/jupyter_execute/chapter2.ipynb
index 74b415d93..caec56691 100644
--- a/doc/src/LectureNotes/_build/jupyter_execute/chapter2.ipynb
+++ b/doc/src/LectureNotes/_build/jupyter_execute/chapter2.ipynb
@@ -1094,27 +1094,27 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "2.1453055662574343\n",
-      "[[12.86665228 10.42534378 14.91184964  7.05197852 10.73905189  9.78942024\n",
-      "   2.99691286  8.42384711 18.16377739 12.166796  ]\n",
-      " [10.42534378  8.44724724 12.082487    5.71394166  8.70143262  7.93198333\n",
-      "   2.42828097  6.82551297 14.71739653  9.85827768]\n",
-      " [14.91184964 12.082487   17.28213796  8.17291407 12.44606006 11.34548129\n",
-      "   3.47328217  9.76284574 21.05097048 14.10074886]\n",
-      " [ 7.05197852  5.71394166  8.17291407  3.8650614   5.88587937  5.36540351\n",
-      "   1.64255353  4.61695766  9.95523663  6.6684    ]\n",
-      " [10.73905189  8.70143262 12.44606006  5.88587937  8.96326666  8.17066394\n",
-      "   2.50135015  7.03089888 15.16025642 10.15492225]\n",
-      " [ 9.78942024  7.93198333 11.34548129  5.36540351  8.17066394  7.44814941\n",
-      "   2.28016105  6.40917136 13.819667    9.25694395]\n",
-      " [ 2.99691286  2.42828097  3.47328217  1.64255353  2.50135015  2.28016105\n",
-      "   0.69804379  1.96209046  4.23072426  2.83390167]\n",
-      " [ 8.42384711  6.82551297  9.76284574  4.61695766  7.03089888  6.40917136\n",
-      "   1.96209046  5.51512534 11.89189544  7.96564848]\n",
-      " [18.16377739 14.71739653 21.05097048  9.95523663 15.16025642 13.819667\n",
-      "   4.23072426 11.89189544 25.64169777 17.17579438]\n",
-      " [12.166796    9.85827768 14.10074886  6.6684     10.15492225  9.25694395\n",
-      "   2.83390167  7.96564848 17.17579438 11.50500702]]\n"
+      "2.599907486910053\n",
+      "[[ 0.2319431   1.78702425  2.33555576  1.01612686  0.80805005  1.10075065\n",
+      "   1.70915039  0.60318841  0.62019593  1.49305147]\n",
+      " [ 1.78702425 13.76827179 17.99447691  7.82883097  6.22568649  8.48082175\n",
+      "  13.16828638  4.64731357  4.77834938 11.5033349 ]\n",
+      " [ 2.33555576 17.99447691 23.51792615 10.23191002  8.13667637 11.08403099\n",
+      "  17.21032448  6.07381797  6.24507555 15.03431202]\n",
+      " [ 1.01612686  7.82883097 10.23191002  4.45158225  3.54001199  4.82231328\n",
+      "   7.48767091  2.64252717  2.71703596  6.54095632]\n",
+      " [ 0.80805005  6.22568649  8.13667637  3.54001199  2.81510801  3.83482678\n",
+      "   5.95438729  2.10140515  2.16065644  5.20153566]\n",
+      " [ 1.10075065  8.48082175 11.08403099  4.82231328  3.83482678  5.22391907\n",
+      "   8.11124963  2.86259878  2.94331271  7.08569196]\n",
+      " [ 1.70915039 13.16828638 17.21032448  7.48767091  5.95438729  8.11124963\n",
+      "  12.59444676  4.44479575  4.57012136 11.00204953]\n",
+      " [ 0.60318841  4.64731357  6.07381797  2.64252717  2.10140515  2.86259878\n",
+      "   4.44479575  1.56864447  1.61287402  3.88281152]\n",
+      " [ 0.62019593  4.77834938  6.24507555  2.71703596  2.16065644  2.94331271\n",
+      "   4.57012136  1.61287402  1.65835067  3.9922914 ]\n",
+      " [ 1.49305147 11.5033349  15.03431202  6.54095632  5.20153566  7.08569196\n",
+      "  11.00204953  3.88281152  3.9922914   9.61098936]]\n"
      ]
     }
    ],
@@ -1673,15 +1673,15 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "0.021442867969978376\n",
-      "4.117524456004779\n",
-      "0.06378292739195897\n",
-      "0.9888063524440697 9.919191379006188 17.823611314267136\n",
-      "2.969136019500522 3.214525177223236 9.842379203904565\n",
-      "[[ 0.98880635  2.96913602  3.21452518]\n",
-      " [ 2.96913602  9.91919138  9.8423792 ]\n",
-      " [ 3.21452518  9.8423792  17.82361131]]\n",
-      "[25.24964689  0.08489291  3.39706925]\n"
+      "0.09525604390305954\n",
+      "4.231031843972674\n",
+      "0.04124568972977915\n",
+      "0.831883346005529 8.296268771337688 5.337726521457732\n",
+      "2.4701318674515136 1.7159602029423742 5.229477584776403\n",
+      "[[0.83188335 2.47013187 1.7159602 ]\n",
+      " [2.47013187 8.29626877 5.22947758]\n",
+      " [1.7159602  5.22947758 5.33772652]]\n",
+      "[12.99430696  0.07864499  1.39292669]\n"
      ]
     }
    ],
@@ -2296,7 +2296,7 @@
    "outputs": [
     {
      "data": {
-      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAESCAYAAAAMifkAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAATkUlEQVR4nO3dwW8bZ37G8edXBPAhaJaVFORiA1sm61x62NDyebMN3b0WWSX+BxomRa9axznuySut9txa/gMKx0rvjZXE58qiLz0FMJ2eFgusFCZI9lKgvx7mHc2Y5ugdUiRnOPx+gCCcl+TMy9cSH73vO/OOubsAADjPX1VdAQBA/REWAIAowgIAEEVYAACiCAsAQBRhAQCIIiwwNTPrmdldM9sysx0zexoe98zs4YLq0AnHezDDfT6d5/5LHL9tZg/NrDOuThfY79Pw/+4s9pffJ5qPsMBFnLr7h+5+IOmhpIG7H7j7vqS7ZXdiZr0L1OETSZ9K+uAC+3iOu78+z/2XOP5AUl/SWkGdXlCmDdN9uPuhpOE0dRs9TqxeaA7CAhfRn/K5M2bWkvThBerQcvehu0/15TemPm0z25rX/mdUp9Hno20Y20fJejx3nFnsE8uDsMDUwl/Ahc+Z2a0w5NELXyydMEzVMbOd8NK2pFZaXrS/sI9u+G8rlHUkrYX3tkde3w3DYq3csE535Ll0f3fDF6EknUraCe97Yf8F9cjv74GZ/eOY/XfSz130OfP7Du2Seq5OZdpwtE6S/i/dx0gbdcO/U3vkfS+025jjnOb3WaJtRtsaS+SlqiuAZkqHK8KQh8IX1kDSkbv3zUzh+b6ZDcNQVtG+tkb2tWNmg/DeU0mHo3/5u/uhmQ3C40GYQ2mNPDcIz3Ukbab7yb3vuf2fU490f6fu/l547l9G9+/u++Fz39RIzyt8Ibdy+76R+yxndQrvjbZhQZ2eC/f0WJIOzexY0rVIu71wnPS1JdrmhbYe80+NGqNngXm5JmmY/kWt5AvuY0nt8MU0ydDTDSVBkzpR8oVzUae5x2X+2j2vHmvuPjr0dlrweNyxRvc91oRtOK5ORdrxl5wr9m80aVujZggLzMtDKflrNHxh7ZtZz9133T0NkvQL6lQ6++t6nGM9/2X2uqTHE9ZnFhOxs6hHkSNJ13PbY79QL9CGMUVBNdpuRceZZ9ugBggLXFj44nhPyV+8W2bWSocqwnY6Bt/KbT/NzXk8sHPO5glnV7VyY+HHYUikE/Zb9N67knq5OYIPc3MRbUnvh/HzG5JujHnuuf3H6pF+hvP2r2QYadNG5lhCe52Efafvf6G+ZdvwvDqFlxymxwqveS/WbqPHye+zxL/RuLbAEjGWKAcAxNCzAABEERYAgCjCAgAQRVgAAKIICwBA1FJfwd1qtfyNN96ouhq18OOPP+rll1+uuhq1QFtkaIsMbZE5Pj7+s7u/Osl7ljosXnvtNT1+zHU/kvTo0SO9/fbbVVejFmiLDG2RoS0yZvY/k76HYSgAQBRhAQCIIiwAAFGEBQAgirAAAEQRFgCAKMICABBFWAAAoggLAEAUYQEAiCIsAABRS702VB3Yb63qKkiSvvrFV1VXAUCDERaYGYIzQ1ugaeY2DGVmnZHtLTPrmtmtScsAANWaS8/CzLqS7kp6PWx3JMndD82snQ+SWJm79+dRRwBYtLr0OKcxl7AIX/aDXNFNSQ/D44GkrqT1kmWEBYALO/7jsX75219WXY2ltaizoVqSTnPb6xOUAQAqtnQT3GbWk9STpFdffVWPHj2qtD57V/cqPX7qT6d/0h/+/Q+V1qEubfHDDz/wcxHUoS2O/3hc6fFTly9drs2/S9W2tT3xexYVFkNJa+FxS9JJeFy27Iy770val6Q333zTq75NYl26tXtX97T99eQ/AE301S++qvz2mXX5uaAtMvyOXMyiwuK+pM3wuC3pMDwuWwYAqNC8zobakrRpZlvufuDufTPbDGdJDdMznMqWFfnL//5lqc8uAIBlMa+zoQ4kHYyU7Y95XakyAEC1WBsKABBFWAAAoggLAEAUYQEAiFq6i/KAGJZ1AGaPngUAIIqwAABEERYAgCjmLIAGY/4Gs0LPAgAQRVgAAKIICwBAFGEBAIgiLAAAUYQFACCKsAAARBEWAIAowgIAEEVYAACiCAsAQBRhAQCIIiwAAFGEBQAgirAAAEQRFgCAKMICABBFWAAAoggLAEAUYQEAiCIsAABRhAUAIOqlRR3IzLYkDSW13X1/pKzj7rtFZQCAai2kZ2FmHUkDdz+UNDCzTihTKBsWlS2ifgCA8y1yGGon/L/t7n1JN5X0ICRpIKlbUAYAqNhCwiKEw8DMvpV0GopbuceStF5QBgCo2ELmLMyspaTHcEfSPTPrX2BfPUk9SdrY2NDe1b3ZVHLJXb50mbYIaIsMbZGhLTLb2p74PYua4O5JuuPuQzMbSEonsdfC8y1JJ+HxuLIzYXJ8X5KutK/49teTf+gm2ru6J9oiQVtkaIsMbXExCz911t0PlATFfUntUNyWdFhQBgCo2EJ6Fu6+a2a3Qq9iLXfq7KaZdSUNw7zG2DIAQLUWdp3FuGsm0tCIlQEAqsUV3ACAKMICABBFWAAAoggLAEAUYQEAiCIsAABR0VNnzezvlSzq11KysN+Bu38z53oBAGqk1HUW7v6NmR25+3Uz+7Wkb+ZbLQBAnZQZhrLQu/gibPsc6wMAqKEyYXEq6R8k3Qm9iuvzrRIAoG6iw1Du/kTSE0kKazuxuB8ArJhoz8LM3k0fh+B4Z641AgDUTmHPIgw53ZC0aWYfSjJJ3yo5M+o/FlM9AEAdFIaFu39mZoeSNt09ndyWmb2ykJoBAGrj3DkLd//OzE7N7E4oMklvSfrV3GsGAKiNMtdZdBVuYxpszakuAICaKhMWx+7+LN0ws4dzrA8AoIbKhMVtM9tRcr2FSfpbST+ba60AALVSJix2Ria4OXUWAFZM9DqLfFAET+dUFwBATZVZdfZOflPJRXks+QEAK6TMMJRJuhsetyUdza86AIA6KrM21O3c5rOwAi0AYIWUGYb6XMmy5OlyH0eSvpxzvQAANTLx2VAAgNVT6mwoM/uNmd03s+1FVAoAUC9llij/J0l9SbclPSEwAGD1lBmGepYbhnpmZvOsDwCghsqERdvMXMl9LNpKVp1lDgMAVkiZOYt7kq5J2pV0w9335l4rAECtlDl19gNJcvf3zewnZvauu3OnPABYIWWGoZ66+5fS2c2QpjqQmXWUDGPJ3Q9C2ZakoaSOu+8WlQEAqhUdhpJ0zczeNbOfm9m7mn5dqE9CSLTNrBPCQ+5+KGlYVDblsQAAM1RmzuL3ktYlfSSp7e6fTHqQ0Fs4Cvvbdfe+pJtKehBSMnneLSgDAFSszDBUOsl97wLHuS6dDUV1w/BSS8kNlVLrBWXPMbOepJ4kbWxsaO8q8+2SdPnSZdoioC0ytEWGtshsa/LL5UqFxYycuHvfzLqhpzEVd99XuCf4lfYV3/6aawQlae/qnmiLBG2RoS0ytMXFLCosTpQMK0nJMNP18P+1UNYKr1FBGQCgQmPnLMKE9iszPM6BwplQSkLgSNL9XFlb0mFBGQCgYkUT3Ovu/r2UBEf+iWnuZ+HuAyVnN22FfR+ESW6ZWVfS0N3748omPRYAYPaKhqFOzezT8LhtZjeV3M9CSpb7+NmkBwpzDVLSyxgtG/c6AEBNjA0Ld/9M0meSZGZvufuT9Dkze2tBdQMA1ESZ26o+MbPfSNqUdMTaUACwesrcz+IDcT8LAFhpZU6dHXA/CwBYbdzPAgAQxf0sAABRZdeG+v28KwIAqK8yS5QDAFYcYQEAiCpz6uz2jNeJAgAsmTI9i0G6TpQkmdnP51gfAEANlZng/sjMdpRcmGeacm0oAMDyKhMWd8NaUZIkM3tnjvUBANRQmbBYM7M7Sm5EtC/J51slAEDdlAmLp+5+L6w++z3LfQDA6ikTFtdCQLTCsh/XJH0511oBAGqlTFjsS/pEybpQ/8XV3ACwesrcz+I7SbfN7Kfu/s38qwQAqJsyF+W9Y2aPJe2Y2X9Ocw9uAMByKzMM1XL3zXTDzH49x/oAAGqozBXc30a2AQANN7ZnYWafKrmewpTc/OhE0neSWpKeirOhAGClFA1D3c9ftQ0AWG1jw2JkeY9XJG0q6VVIyd3y/nkBdQMA1ESZCe5dSce57fU51QUAUFNlwuKBu3+RbpjZwznWBwBQQ6VOnTWz+5IGSia835F0fa61AgDUSpmwaEu6nds+mVNdAAA1VSYsjt39WbrBMBQArJ4yF+XdNrOjsNTH55IeXOSAZnYr93jLzLqxMgBAtcr0LHZGJrinvlOemXUl3ZC0a2YdSXL3QzNrp9ujZe7en/Z4AIDZiPYs8kERPJ3RsW9KGobHA0ndgjIAQMWiPYtwS9WzTU15NlToJRya2cehqCXpNPeS9YIyAEDFygxDmaS74XFb0tGUx1qb8n0AgIqVuflR/rTZZ9PczyLtVYwUD5UFSEvZKbnjyvL76knqSdLGxob2ru5NWp1GunzpMm0R0BYZ2iJDW2S2tT3xe8oMQ32uZAVaKfmCP9Lkq862zaytJAjWwmT2fSVrTklJjyUNk3FlZ9x9X8mtXnWlfcW3v578QzfR3tU90RYJ2iJDW2Roi4uZ+Gyoabj7gXTWK2iFsr6ZbYYzpIbpWU/jygAA1Sq6n8W/Skonoo/CyrNn3P37aQ6W7xXktse9BgBQI0Wnzq4rGTJK/7+mZPXZZ5LeX0zVAAB1UTQM9YG7fydJZvaWpHtK5iraaTkAYHUU3fzoOzP7qZLexE8kvZdfHwoAsFrGDkOZ2e8kPZT0b+7+q5GFBCc+dRYAsNyKhqG6kj6SZCEcTMnpsybpd+J+FgCwUs6bs3gy7gkzuz2uHADQXGOHoYqCIjx3oWsuAADLp8z9LAAAK46wAABEERYAgCjCAgAQRVgAAKIICwBAFGEBAIgiLAAAUYQFACCKsAAARBEWAIAowgIAEEVYAACiCAsAQBRhAQCIIiwAAFGEBQAgirAAAEQRFgCAKMICABBFWAAAoggLAEAUYQEAiCIsAABRhAUAIOqlRR3IzHrh4evu/nEo25I0lNRx992iMgBAtRYSFmbWlXTo7gMzexC2TyXJ3Q/NrG1mnfT1+TJ37y+ijgCAYosahmpL6obHg7B9U0kPIi3rFpQBACq2kJ6Fu+/nNjuS7ku6ptC7CNYltcaUAQAqtrA5C0kKQ019d++b2bT76EnqSdLGxob2ru7NsIbL6/Kly7RFQFtkaIsMbZHZ1vbE71loWEjqppPbSoab1sLjlqST8Hhc2ZnQS9mXpCvtK7799eQfuon2ru6JtkjQFhnaIkNbXMxCz4bKnfHUVTIUtRmebks6DI/HlQEAKrSQCe4QDjtm9tTMvpWk9Cyn8NzQ3fvjyhZRPwDA+RY1wX0o6W/GlO+XKQMAVIsruAEAUYQFACCKsAAARBEWAIAowgIAEEVYAACiCAsAQBRhAQCIIiwAAFGEBQAgirAAAEQRFgCAKMICABBFWAAAoggLAEAUYQEAiCIsAABRhAUAIIqwAABEERYAgCjCAgAQRVgAAKIICwBAFGEBAIgiLAAAUYQFACCKsAAARBEWAIAowgIAEEVYAACiXqq6AqPMbEvSUFLH3Xerrg8AoGY9CzPrSJK7H0oaptsAgGrVKiwk3VTSq5CkgaRuhXUBAAR1C4uWpNPc9npVFQEAZMzdq67DGTO7K+muu/fNrCvphrt/PPKanqRe2Pw7Sf+94GrW1YakP1ddiZqgLTK0RYa2yLzp7n89yRvqNsE9lLQWHrcknYy+wN33Je1Lkpk9dvfNxVWvvmiLDG2RoS0ytEXGzB5P+p66DUPdl9QOj9uSDiusCwAgqFVYuHtfksIQ1DDdBgBUq27DUOkwU1mTvLbpaIsMbZGhLTK0RWbitqjVBDcAoJ5qNQwFTMrMtsysa2a3Iq8793mgqc67uLns74+0RGER+1CTfOhlV6IteuG/nUXXbZHKXvGfnoa9yLotWomfiU54zdai67ZoE3xX9MY93yThZ/9BwXMTrZixFGER+1CrtExIibboSjoMcz/tsN1UXPGv0j//n7j7gZKfiVX+/ehIGoTnB01uC+msHQYFT0/0+7MUYaH4h1qlL43YZ23nygbKTkVuougV/2bWCb8wTXbuz0ToTRxJkrvvNvwswzLfBWmPu93wtoiZaMWMZQmL2IdapWVCzv2s7r6fO6OsI2nii28aZi3+kqUX+/m/Lmk9DEU1fZg29vvRV9Kj+HbkdYhYlrDAhEL3ut/wv5zOveJ/RXoVZZ3krmNq/LxFETNrKfm5uSPpnpk1uecdE10xI29ZwiL2oSb60Euu7Gftjq6r1UBjr/gPXwhSMj6/FSYy1xo8Ph37mThRNm49VNLTaKpYW/Qk3Qn3yvlA0soFZ+73Y6IVM5YlLGJfCqu0TEisLWRmvfTGUU2e4D7niv8vwvMHYVJXSr44mir2M3GQe76lMH/RUNHfj1T42RiOljdJ6EVujvQm09+PiVbMWJqL8sJfhwMlk1LpQoLH7n6t6PmmOq8tcqfKnSr5C+s9hmKar+Tvx6mk603vcZZoi1vh+bWmf1fM0tKEBQCgOssyDAUAqBBhAQCIIiwAAFGEBQAgirAAAEQRFgCAKMICABBVu9uqAssuXC3bVnLh13Uly0s0+kphNB89C2CGzKw9sozEfYICTUBYADPk7umCfdeU3ISqyav+YoUQFsAM5Va2bbt7o+/aiNXCnAUwW91wj4SHYVFHbrCDRmAhQQBAFMNQAIAowgIAEEVYAACiCAsAQBRhAQCIIiwAAFGEBQAg6v8BnA4uazqvtooAAAAASUVORK5CYII=\n",
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAESCAYAAAAMifkAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAATiUlEQVR4nO3dwY8b533G8ecpDPgQ1GFXWxgFJCBlHPXSQ0ytznVqqrkWztpC7/U6Ra8bWz4FOSlS1udW6z8gULTpvdEm8blrrS89GRDlnooA0ZoOkmt/Pcw7yxFN7jvcXXKG5PcDGOa8Q868fEXy2XnfmXccEQIA4Cx/1nQFAADtR1gAALIICwBAFmEBAMgiLAAAWYQFACCLsMC52d6x/cD2tu17tp+mxzu2Hy+oDr20v0eXuM2n89x+jf13bT+23ZtUpwts92n6f/8ytlfdJlYfYYGLOImI9yLiQNJjSYOIOIiIfUkP6m7E9s4F6vChpF9IevcC23hBRHx7ntuvsf+BpGNJG1Pq9DV12rDcRkQcShqep27j+8nVC6uDsMBFHJ9z3SnbHUnvXaAOnYgYRsS5fvwm1Kdre3te27+kOo2vz7Zhbhs16/HCfi5jm1gehAXOLf0FPHWd7fdTl8dO+mHppW6qnu176aldSZ2yfNr20jb66b/tVNaTtJFe2x17fj91i3Uq3Tr9sXXl9h6kH0JJOpF0L73ua9ufUo/q9h7Z/scJ2++V73va+6xuO7VL6YU61WnD8TpJ+r9yG2Nt1E//Tt2x132t3Sbs56S6zRptM97WWCIvNV0BrKayuyJ1eSj9YA0kHUXEsW2l9ce2h6kra9q2tse2dc/2IL32RNLh+F/+EXFoe5AeD9IYSmds3SCt60naKrdTed0L2z+jHuX2TiLi7bTuX8e3HxH76X3f1tiRV/pB7lS2favyXk7rlF6bbcMpdXoh3Mt9STq0/UTSjUy7fW0/5XNrtM3X2nrCPzVajCMLzMsNScPyL2oVP3AfSOqmH6ZZup5uqQia0nMVPzgXdVJ5XOev3bPqsRER411vJ1MeT9rX+LYnmrENJ9Vpmm7+KWfK/RvN2tZoGcIC8/JYKv4aTT9Y+7Z3IuJ+RJRBUv5AnUinf11P8kQv/ph9W9KnM9bnMgZiL6Me0xxJullZnviDeoE2zJkWVOPtNm0/82wbtABhgQtLPxxvq/iLd9t2p+yqSMtlH3ynsvy0MubxyGeczZPOrupU+sKfpC6RXtrutNc+kLRTGSN4rzIW0ZX0Tuo/vyXp1oR1L2w/V4/yPZy1fRXdSFseG2NJ7fU8bbt8/dfqW7cNz6pTesphua/0nLdz7Ta+n+o2a/wbTWoLLBEzRTkAIIcjCwBAFmEBAMgiLAAAWYQFACCLsAAAZC31FdydTidee+21pqvRCn/605/0jW98o+lqtAJtMUJbjNAWI0+ePPl9RPzlLK9Z6rB49dVX9emnXPcjSZ988oneeOONpqvRCrTFCG0xQluM2P6fWV9DNxQAIIuwAABkERYAgCzCAgCQRVgAALIICwBAFmEBAMgiLAAAWUt9UR4A1PXkf5/oez/5XqN1iB8v7/2DCIsVwRehXfwTN10FSdJv/+63TVcBK4JuKABAFmEBAMgiLAAAWXMLC9u9seVt233b789aBgBo1lzCwnZf0qPKck+SIuJQ0tB2r27ZPOoHAJjNXMIi/dgPKkW3JQ3T44Gk/gxlAICGLWrMoiPppLJ8ZYYyAEDDlu46C9s7knYkaXNzUx/9/KOGa9QOV1++qr3re43W4ZNPPml0/6U//vGPjdel6X+L0u9Oftf4d+TGX91odP8lviMXs6iwGEraSI87kp6nx3XLTkXEvqR9SbrWvRa7n+/Oo75LZ+/6nppui/indlyU14bbZzZ9gWSJz8XIRz//iLa4gEWFxUNJW+lxV9Jhely3DADQoLmEhe1tSVu2tyPiICKObW+ls6SGEXGcnlerDABWQVumgTmPuYRFRBxIOhgr25/wvFplAIBmLd0AN5DThkkVgVXDdB8AgCzCAgCQRVgAALIICwBAFgPcuDRtOS2w6at08SI+F6uBIwsAQBZhAQDIIiwAAFmEBQAgi7AAAGQRFgCALMICAJBFWAAAsggLAEAWYQEAyCIsAABZhAUAIIuwAABkERYAgCzCAgCQRVgAALIICwBAFmEBAMgiLAAAWYQFACCLsAAAZBEWAIAswgIAkEVYAACyCAsAQNZLi9qR7W1JQ0ndiNgfK+tFxP1pZQCAZi3kyMJ2T9IgIg4lDWz3UplS2XBa2SLqBwA42yK7oe6l/3cj4ljSbRVHEJI0kNSfUgYAaNhCwiKFw8D2l5JOUnGn8liSrkwpAwA0bCFjFrY7Ko4Y7kr62PbxBba1I2lHkjY3N7V3fe9yKrnkrr58lbZIaIsR2mKEthjZ1e7Mr1nUAPeOpLsRMbQ9kFQOYm+k9R1Jz9PjSWWn0uD4viRd616L3c9nf9OraO/6nmiLAm0xQluM0BYXs/BTZyPiQEVQPJTUTcVdSYdTygAADVvIkUVE3Lf9fjqq2KicOrtluy9pmMY1JpYBAJq1sOssJl0zUYZGrgwA0Cyu4AYAZBEWAIAswgIAkEVYAACyCAsAQBZhAQDIyp46a/vvVUzq11Exsd9BRHwx53oBAFqk1nUWEfGF7aOIuGn7B5K+mG+1AABtUqcbyuno4tdpOeZYHwBAC9UJixNJ/yDpbjqquDnfKgEA2ibbDRURn0n6TJLS3E5M7gcAayZ7ZGH7rfJxCo4351ojAEDrTD2ySF1OtyRt2X5PkiV9qeLMqP9YTPUAAG0wNSwi4pe2DyVtRUQ5uC3bryykZgCA1jhzzCIivrJ9YvtuKrKk1yV9f+41AwC0Rp3rLPpKtzFNtudUFwBAS9UJiycR8axcsP14jvUBALRQnbC4Y/ueiustLOmvJX1nrrUCALRKnbC4NzbAzamzALBmstdZVIMieTqnugAAWqrOrLN3q4sqLspjyg8AWCN1uqEs6UF63JV0NL/qAADaqM7cUHcqi8/SDLQAgDVSpxvqVyqmJS+n+ziS9Js51wsA0CIznw0FAFg/tc6Gsv0j2w9t7y6iUgCAdqkzRfk/SzqWdEfSZwQGAKyfOt1QzyrdUM9sz7M+AIAWqhMWXduh4j4WXRWzzjKGAQBrpM6YxceSbki6L+lWROzNvVYAgFapc+rsu5IUEe/Y/qbttyKCO+UBwBqp0w31NCJ+I53eDOlcO7LdU9GNpYg4SGXbkoaSehFxf1oZAKBZ2W4oSTdsv2X7u7bf0vnnhfowhUTXdi+FhyLiUNJwWtk59wUAuER1xix+JumKpB9K6kbEh7PuJB0tHKXt3Y+IY0m3VRxBSMXgeX9KGQCgYXW6ocpB7o8vsJ+b0mlXVD91L3VU3FCpdGVK2Qts70jakaTNzU3tXWe8XZKuvnyVtkhoixHaYoS2GNnV7JfL1QqLS/I8Io5t99ORxrlExL7SPcGvda/F7udcIyhJe9f3RFsUaIsR2mKEtriYRYXFcxXdSlLRzXQz/X8jlXXSczSlDADQoIljFmlA+5VL3M+B0plQKkLgSNLDSllX0uGUMgBAw6YNcF+JiD9IRXBUV5znfhYRMVBxdtN22vZBGuSW7b6kYUQcTyqbdV8AgMs3rRvqxPYv0uOu7dsq7mchFdN9fGfWHaWxBqk4yhgvm/Q8AEBLTAyLiPilpF9Kku3XI+Kzcp3t1xdUNwBAS9S5repntn8kaUvSEXNDAcD6qXM/i3fF/SwAYK3VOXV2wP0sAGC9cT8LAEAW97MAAGTVnRvqZ/OuCACgvepMUQ4AWHOEBQAgq86ps7uXPE8UAGDJ1DmyGJTzREmS7e/OsT4AgBaqM8D9Q9v3VFyYZ51zbigAwPKqExYP0lxRkiTbb86xPgCAFqoTFhu276q4EdG+pJhvlQAAbVMnLJ5GxMdp9tk/MN0HAKyfOmFxIwVEJ037cUPSb+ZaKwBAq9QJi31JH6qYF+q/uJobANZPnftZfCXpju1vRcQX868SAKBt6lyU96btTyXds/2f57kHNwBgudXphupExFa5YPsHc6wPAKCF6lzB/WVmGQCw4iYeWdj+hYrrKazi5kfPJX0lqSPpqTgbCgDWyrRuqIfVq7YBAOttYliMTe/xiqQtFUcVUnG3vH9ZQN0AAC1RZ4D7vqQnleUrc6oLAKCl6oTFo4j4dblg+/Ec6wMAaKFap87afihpoGLA+01JN+daKwBAq9QJi66kO5Xl53OqCwCgpeqExZOIeFYu0A0FAOunzkV5d2wfpak+fiXp0UV2aPv9yuNt2/1cGQCgWXWOLO6NDXCf+055tvuSbkm6b7snSRFxaLtbLo+XRcTxefcHALgc2SOLalAkTy9p37clDdPjgaT+lDIAQMOyRxbplqqnizrn2VDpKOHQ9gepqCPppPKUK1PKAAANq9MNZUkP0uOupKNz7mvjnK8DADSszs2PqqfNPjvP/SzKo4qx4qFGAdLR6JTcSWXVbe1I2pGkzc1N7V3fm7U6K+nqy1dpi4S2GKEtRmiLkV3tzvyaOt1Qv1IxA61U/MAfafZZZ7u2uyqCYCMNZj9UMeeUVByxlGEyqexUROyruNWrrnWvxe7ns7/pVbR3fU+0RYG2GKEtRmiLi5n5bKjziIgD6fSooJPKjm1vpTOkhuVZT5PKAADNmnY/i3+TVA5EH6WZZ09FxB/Os7PqUUFledJzAAAtMu3U2SsquozK/2+omH32maR3FlM1AEBbTOuGejcivpIk269L+ljFWEW3LAcArI9pNz/6yva3VBxNfFPS29X5oQAA62ViN5Ttn0p6LOnfI+L7YxMJznzqLABguU3rhupL+qEkp3CwitNnLemn4n4WALBWzhqz+GzSCtt3JpUDAFbXxG6oaUGR1l3omgsAwPKpcz8LAMCaIywAAFmEBQAgi7AAAGQRFgCALMICAJBFWAAAsggLAEAWYQEAyCIsAABZhAUAIIuwAABkERYAgCzCAgCQRVgAALIICwBAFmEBAMgiLAAAWYQFACCLsAAAZBEWAIAswgIAkEVYAACyCAsAQBZhAQDIemlRO7K9kx5+OyI+SGXbkoaSehFxf1oZAKBZCwkL231JhxExsP0oLZ9IUkQc2u7a7pXPr5ZFxPEi6ggAmG5R3VBdSf30eJCWb6s4gijL+lPKAAANW8iRRUTsVxZ7kh5KuqF0dJFckdSZUAYAaNjCxiwkKXU1HUfEse3zbmNH0o4kbW5uau/63iXWcHldffkqbZHQFiO0xQhtMbKr3Zlfs9CwkNQvB7dVdDdtpMcdSc/T40llp9JRyr4kXetei93PZ3/Tq2jv+p5oiwJtMUJbjNAWF7PQs6EqZzz1VXRFbaXVXUmH6fGkMgBAgxYywJ3C4Z7tp7a/lKTyLKe0bhgRx5PKFlE/AMDZFjXAfSjpLyaU79cpAwA0iyu4AQBZhAUAIIuwAABkERYAgCzCAgCQRVgAALIICwBAFmEBAMgiLAAAWYQFACCLsAAAZBEWAIAswgIAkEVYAACyCAsAQBZhAQDIIiwAAFmEBQAgi7AAAGQRFgCALMICAJBFWAAAsggLAEAWYQEAyCIsAABZhAUAIIuwAABkERYAgCzCAgCQRVgAALJearoC42xvSxpK6kXE/abrAwBo2ZGF7Z4kRcShpGG5DABoVqvCQtJtFUcVkjSQ1G+wLgCApG1h0ZF0Ulm+0lRFAAAjjoim63DK9gNJDyLi2HZf0q2I+GDsOTuSdtLi30r67wVXs602Jf2+6Uq0BG0xQluM0BYjfxMRfz7LC9o2wD2UtJEedyQ9H39CROxL2pck259GxNbiqtdetMUIbTFCW4zQFiO2P531NW3rhnooqZsedyUdNlgXAEDSqrCIiGNJSl1Qw3IZANCstnVDld1Mdc3y3FVHW4zQFiO0xQhtMTJzW7RqgBsA0E6t6oYCZmV723bf9vuZ5525HlhVZ13cXPf7Iy1RWOTe1CxvetnVaIud9N+9Rddtkepe8V+ehr3Iui1ajc9ELz1ne9F1W7QZfit2Jq1fJemz/2jKuplmzFiKsMi9qXWaJqRGW/QlHaaxn25aXlVc8a/an/8PI+JAxWdinb8fPUmDtH6wym0hnbbDYMrqmb4/SxEWyr+pdfrRyL3XbqVsoNGpyKsoe8W/7V76wqyyMz8T6WjiSJIi4v6Kn2VY57egPOLurnhb5Mw0Y8ayhEXuTa3TNCFnvteI2K+cUdaTNPPFNytmI/+UpZf7/N+UdCV1Ra16N23u+3Gs4ojiy7HnIWNZwgIzSofXxyv+l9OZV/yvyVFFXc8r1zGt/LjFNLY7Kj43dyV9bHuVj7xzsjNmVC1LWOTe1ExvesnVfa/98Xm1VtDEK/7TD4JU9M9vp4HMjRXun859Jp5r1G89VHGksapybbEj6W66V867ktYuOCvfj5lmzFiWsMj9KKzTNCG5tpDtnfLGUas8wH3GFf+/TusP0qCuVPxwrKrcZ+Kgsr6jNH6xorLfj1L6bAzHy1dJOorcGjuaLL8fM82YsTQX5aW/DgcqBqXKiQSfRMSNaetX1VltUTlV7kTFX1hv0xWz+mp+P04k3Vz1I84abfF+Wr+x6r8Vl2lpwgIA0Jxl6YYCADSIsAAAZBEWAIAswgIAkEVYAACyCAsAQBZhAQDIat1tVYFll66W7aq48OumiuklVvpKYaw+jiyAS2S7OzaNxEOCAquAsAAuUUSUE/bdUHETqlWe9RdrhLAALlFlZttuRKz0XRuxXhizAC5XP90j4XGa1JEb7GAlMJEgACCLbigAQBZhAQDIIiwAAFmEBQAgi7AAAGQRFgCALMICAJD1/7mcMKjPm/vqAAAAAElFTkSuQmCC\n",
       "text/plain": [
        "
    "
       ]
@@ -2402,12 +2402,12 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "-0.04067115014325762 0.9243683507629109\n"
+      "0.054334900100894076 1.0092227336467725\n"
      ]
     },
     {
      "data": {
-      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAESCAYAAAAWtRmOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd5wV5bkH8N+LeEUQXMEoVmDRGBLrsiR2ja4Y2/UGFzVeE1uEhKioGIkae2zcxOj1GhUTo8aGYg0o6mI09riLmmiMbS1BwQIsRZC2z/3jmdd5Z96ZU3ZPGXZ/38/nfM708545M+8zb5k5RkRARETk6lHtBBARUfYwOBARkYfBgYiIPAwORETkYXAgIiIPgwMREXkYHKikjDH3VDsNnWGMqTPGvNvBdTu0XkoaxlRqX5Yq3dS1MDhQyRhjGgE0GmNqilxvTJmSVDQRmQWgNd93SEqziAwtUTLOAnA3gBNLtL2vlDnd1IUwOFAp9QcwFUDBmX2QCY8tW4rKICnNxpjaIDiWQo2ItIlIW4m2B6Ai6aYuhMGBSiLIeOYDuAF+BtRgjHnXGFMTZEaPG2Magtm1AGqMMY3GmDpnnTHBeg0283Km1dllU5azn9dgjLkn+FxvWrDsmcG0McaY2oTvZT/vCmd+UprnA7jC2W6+dDUYY26Il1CC7fUPtl2ba9/l217C/ipbuqnr6VntBFCXcbiITAYAY0x/Y0xdUEUDEWkyxrQGw63GmMcB1ATjs4wxbSIy1W7IZkoi0hSM24x5gIhMDoYnBtuJL9fqfN58ERkdbNabZqtYnPXvAWCXt8aKyGhjDABMDMa9NItIm/2OKel309Ua7Ic6APUAmpztzDLGzAfQFJQcWnPsu9TtBWkY6u4vERlbrnRT18OSA5XK0OCKtBGaaXSmqmg/AK3O+DwA+wJ4CdBMUkTGpixXHwz3t8HJEZ82HECbc2X9UkJaJgbfqT5hXjHpd9ef7wyX4go8aXsj4O+vfCqdbsowBgfqtCBjnSIiU4Or0hMBHJ5jlXgD6PxgO7aqqQVaBeIuPxOa4dnPrElZrrmIpNuSx6wgaEx2ZwbpmRh8J3s1bT8vnmZXZ9OVS6GNxy/B319A9dJNaxgGB+qUIDDciGimUhvMu8HJTG8AMMap6x7rZFj3GKcXTVA9ZdsJGgG0iMgkAPOC0kkDgNqU5WbZ+nV3m0nTbPWKu02nbv5whBlpHfRKub/zPe9J2j7C6rXUdAE4PPju+wHYL9ZOYJdxG/UT912u7QXfLbK/yplu6noMH9lNRERxZSs5OFc5SfMag6uTM8v1+URE1HFlCQ5BMTbx7k4bNIIeEW25gggREVVHWYJDkPG3psw+AoC9uacVQFLDGBERVVE1GqTtzVLWgCqkgYiIcmBvJSIi8lTjDuk2aJdAQEsR8+ILBF3txgBAzdprDx+6/faVSx0RURfQ0tLyuYh8raPrVyw4BH2v2wBMQXjXZS0SbsEP+ltPBoD6gQOluZn34RARFcMY80Fn1i9Xb6VGAPUm+rTHmcBXj0S2PZraEh5xQEREVZb5m+DqBw6U5rlzq50MIqI1ijGmRUSKeSZYBBukiYjIw+BAREQeBgciIvIwOBARkYfBgYiIPAwORETkYXAgIiJP9oNDxu/DICLqirIfHIiIqOIYHIiIyMPgQEREHgYHIiLyMDgQEZGHwYGIiDwMDkRE5GFwICIiD4MDERF5GByIiMiT/eDAx2cQEVVc9oMDERFVHIMDERF5GByIiMjD4EBERB4GByIi8jA4EBGRh8GBiIg8DA5ERORhcCAiIg+DAxERebIfHPj4DCKiist+cCAioorrWY6NGmMaAbQBqBORSTnm14rI5HKkgYiIOq7kJQdjTB0AiEgTgDY7HpvfGsxvjc8nIqLqK0e10hHQUgEAtAJoSFjmiuC9VkRmlSENRETUCeUIDjUA5jvjA9yZQTBoNcYsiC1HREQZUfEGaWNMDbRkcRmAG40xtQnLjDHGNBtjmr/88stKJ5GIqNsrR3BoA9A/GK4BMC82fwyAy4KG6hMBNMY3ICKTRaReROp79epVhiQSEVEu5QgOUwDY0kAtgCbgqxJDhIhMRdg+QUREGVHyrqwiMssYU2+MaQDQ5jQ4zwQwXEQmGWPONMa0AujPrqxERNlTlvsckjJ8ERnuDHv3PhARUXZk/w5pPj6DiKjish8ciIio4hgciIjIw+BAREQeBgciIvIwOBARkYfBgYiIPAwORETkYXAgIiIPgwMREXkYHIiIyJP94MDHZxARVVz2gwMREVUcgwMREXkYHIiIyMPgQEREHgYHIiLyMDgQEZGHwYGIiDwMDkRE5GFwICIiD4MDERF5sh8c+PgMIqKKy35wICKiimNwICIiD4MDERF5GByIiMjD4EBERB4GByIi8jA4EBGRh8GBiIg8PcuxUWNMI4A2AHUiMilhfh2AWgAQkanlSAMREXVcyUsOQcYPEWkC0GbHY84KgkJtynwiIqqiclQrHQEtNQBAK4AGd2ZQqngJAERkkojMyrk1Pj6DiKjiyhEcagDMd8YHxOaPADDAGFNnjDmzDJ9PRESdVK0G6Xm2xBCUJCKMMWOMMc3GmObly5dXPnVERN1cOYJDG4D+wXANgHmx+fOg1U122RHxDYjIZBGpF5H6ddZZpwxJJCKiXMoRHKYg6IkUvDcBgDGmJpg21Zlfg6D9gYiIsqPkwcGpLmoA0OY0OM8M5rdCezE1AhjArqxERNljJOO9geo32ECaFyyodjKIiNYoxpgWEanv6Pq8Q5qIiDwMDkRE5Ml+cMh4tRcRUVeU/eBAREQVx+BAREQeBgciIvIwOBARkYfBgYiIPAwORETkYXAgIiIPgwMREXkYHIiIyMPgQEREnuwHBz4+g4io4rIfHIiIqOIYHIiIyMPgQEREHgYHIiLyMDgQEZGHwYGIiDwMDkRE5Ck4OBhj+sXGB5c6MURElA09i1i2P4BFxpjLAQiAeQB+XZZUERFRVRUUHIwxQ0TkvWB0ioi8bIwZUsZ0ERFRFRUUHETkPWPMjtDSQ7OdVs6EOR9ekY8hIqJQoSWHfgCGAmgDcIQx5l0AGwBYICJPlDF9RERUBYW2OcwUkRF2uFyJISKibCi0t9KCsqaCiIgypdDgMNoYM6qsKSEioswotEF6IYD7ypwWIiLKiLLcIW2MaTTGNBhjzsyzXM75RERUHSUPDsaYOgAQkSYAbXY8YbkGAPuV+vOJiKjzylFyOALa5RUAWgE0lOEziIiojMoRHGoAzHfGB8QXMMbUBSULIiLKoGo9lbV/rpnGmDHGmGZjTPPKFSsqlSYiIgqUIzi0Icz8a6AP6PtKIaUGEZksIvUiUr/22muXIYlERJRLMU9lLdQUAPXBcC2AJgAwxtSISBuAWmNMLTSA9A+CxawypIOIiDqo5CUHm9EHvZHanIx/ZjB/qohMDabVlPrziYio84xk/Kmn9X37SvPixdVOBhHRGsUY0yIi9fmXTMa/CSUiIg+DAxEReRgciIjIw+BAREQeBgciIvIwOBARkYfBgYiIPNkPDhm/D4OIqCvKfnAgIqKKY3AgIiIPgwMREXkYHIiIyMPgQEREHgYHIiLyMDgQEZGHwYGIiDwMDkRE5GFwICIiT/aDAx+fQURUcdkPDkREVHEMDkRE5GFwICIiD4MDERF5GByIiMjD4EBERB4GByIi8jA4EBGRh8GBiIg8DA5EROTJfnDg4zOIiCou+8GBiIgqrmc5NmqMaQTQBqBORCYlzB8TDA4VkYnlSAMREXVcyUsOxpg6ABCRJgBtdtyZ3wCgSUQmA6gNxomIKEPKUa10BLTUAACtAOKZf60zrTUYJyKiDClHtVINgPnO+AB3ZlBisOoATClDGoiIqBOq1iAdVDfNEpFZCfPGGGOajTHNq1atqkLqiIi6t3IEhzYA/YPhGgDzUpZrSGuMFpHJIlIvIvU9e5alzZyIiHIoR3CYgrAdoRZAEwAYY2rsAsaYMbYXExukiYiyp+TBwVYTBZl+m1NtNNOZfoUx5l1jzIJSfz4REXVeWepsYo3Odtrw4L0JwAbl+FwiIiqN7N8hzcdnEBFVXPaDAxERVRyDAxEReRgciIjIw+BAREQeBgciIvIwOBARkYfBgYiIPAwORETkYXAgIiIPgwMREXmyHxz4+AwioorLfnAgIqKKY3AgIiIPgwMREXkYHIiIyMPgQEREHgaHQnz0EfD559VOBRFRxZTlb0K7nM031/dKdat97TVgyRJg550r83lERDEMDlm03Xb6Xqpg1NYG9O4N/Md/lGZ7RNTlrVnVSo88Arz/frVTURpLlwLGAJdfnr7Mp5/q+4cfAkcdBXz5Zcc+a4MNgFGjOrYuEXVLa1ZwOPBA4Otfr3YqfH//O/CvfxW3zvz5+n7ttenLjByp7yefDNx5JzBjRvFpW71a36dPL35dAPjjH4Gnn+7Yul3R3LlAc3O1U0FUdtkPDvGqlZUrS7Pda68FvvnNzm1DBPjf/wV22AEYNiw674svgP79gYcfTl83n3/+U99XrdL3tdYqPo1tbcWv4zr+eGDPPf3p8+frd+ysxx8Hnnii+PXmzAF+8Qugvb3zaSjG8OHAiBGV/UyiKsh+cEjzzjvJ05cvB37yE2DDDYEXXkhf/6STgDfe6FwaWlqA8ePT07dgAXDWWcnzbZCbOxe46KLkTM4GBXv135HgsGBB8esUYsAA4Fvf6vx2Ro4E9t03ed6KFcA//pE877jjgCuuAJ56SkuT997b+bQU4uOP9d3+Jp01e7ZWLz74YHT6smV6LBNVyZoTHNwr7XvuAbbeWtsg4m6+GbjhBmDePA0S+diTfM4cPUmfeSac196uJ2khaUrbbo+UXWxP/FWrgPPPB/761/Tt2231zNN/YMYM/Q6NjcCzz2oA+ugjnWdM+npTpkQz4Zkz9ao8nw8+yL9MoZL25e23A9tvr7933JIl+v7JJ8Dbb+t3NibMvK1XXgGuvjr5YuK224C33upYejtbIrNefVXfb7wxOr13b2Dw4NJ8RimsXg089lj0d7rxRq1SpS5pzQkO7pX1K6/o+6xZ/nJuBrpwYf7t2iv455/X9//5n3De+PF6kqZJy/iBsPE47Wo/flU4b17ycqtWaUYP5K9CsVfP994L7L478L3vAXvvrdNEkqu43nwTOPJI4Mc/Dqc1NOhVeSncc49mKvnYNhjX22/re1IJz+6L+P0nL78cDq9eDey0E3DqqcB++/nb+OEPNfikefVVDTgnnADssUd0XiElsmefDb9DkttuAw4+OExr3Ny5+T8D0GN4n32Au+8ubPmO2HdfYP/9o21XY8ZolWpHnXwycMstnU8blcWaExxsFQsArL22vie1P9TUhMMrVuTfrt1Gv376vmiRZlRLlmgJJMm11+pVbbxUYa8Cb70V2G03HXYDyNNPaylh6FA/c0m7Er3wwvBz8lUzxINHvC7/oIP8dWbP1vekjKizXWlbW4HDD9dMJb7dSy+NfuacOZqRnn9++Ln2N7e/t8tmpvZCIcnixeGw7fll2WPD3acnnhg9fh54QN9vuilaogRyB4enn9bfYvfdtcpr0SINkOPGRZc74YRwuDNtJ59/DvzlL8ARR4TTnnhCP7cQK1fm7uwgotV3gP6mdp3OeO014P/+Dzj22HBaSwsf0Z8h2Q8OK1ZoZp0UHOKZv3sQA+EB3N6u1STTpvnb/81vouMLF2p9+lZbpZ8AJ50EHH20fzW54476fvbZ4TQ3OOy5p7YvtLb6delvvaXf8b77otPd0pHbs+m004C99oou25EMxmaOSdVObvB75x2topoxI3oCr1qlAcxmlu3twJVXakls6NDkz2xpAc45BzjmmHDa3LmakV50kVYVAWHpy/3tLRscbAZuiQCHHKJVjm5wcEuAZ5wR3kvi+v3vo6XNeKnR/d5pweHhh/V3PvfccNr662uAvO666DHlBr3Vq/U4790bePJJf7vPPZfeS8r9nZ58Urez777A6NHJy8edey5wwAF+ALTc88z+JoV0Rpg6VYNAkvj+nzkTqK/P3XuPKmrNuAlujz305LDszVzxzPuuu4BrrgnHv/xSD+y//U2rSWbMCIvx1oUXaqPxhAk6bq+2bAZVrBdeiGYcaVVP8WqESZM0Y4p/rpsxPv54OHzVVeHwsmW6TzrSSBovjbz+ejjsZq5bbw0MGQK89170fouHHgIuuECv/K+/Xq+QJ0wANtkkut2PPgI220zfbWnFvbKdMyf6fYAwA0oK0va7xktcixfrRcC0adGMyU1z/IIgTTw42KtmID042HaYSy9Nnr94sfZiA/zg8OKL+t2T1rUlURF9ucHc/W777x+2tT32mC732WfaQSON/c0/+yycNnu2rtOrVzQQ2OPTtvnkYoNTvDQQr2ZcvDis8szXhiGix9nRRwN9++ZPQ7XMnq1BOle1ZcZlv+QAaJdON5O0GW685BC/QW7xYmCddcJ66PXXT97+DTeEB2UhVVG57LKL3uBmpbU53HabP23KlLB6yyqkvr53b2DbbYE//Sn/svETNX5j3bbbhsPxaon33tN3t4rGZs42A7GZphtYAH0EyZw5+v797/vpstt202S3kdQeY4NDPCC6wc1Nv83MCmlItvsoHhzcoGK/5/LlGrQLrQ5x0+QGh/b2cBvuBcWxxwI//3k4fvDB2o7icn/DFSu0e7XL3SdJbNWcMZq+Tz4BttgizNzd4PDvf+u7+/t+/LGWBO68M3n7++6rJcWWFg2A8WrGgw7S0iYQfvclS7Q9zHXNNXpD57hx6b0E0zz4YMfv9Vm82G8TW7jQr6p0DRvWufaYDFgzggMQDQ72JLrrLuCUU8KDKK03j20UHDBA182VoXQkOCRldtYzz2jDZLxqJKkxfeXK6BV0kvXXB+64Ixy336XQm/AefRT4wQ+AsWM1MzjqqPRl4xm85abRnjQ2o7MlkaQry5kzo+PuPjn//HDYlhzs5//ud5puQC8AVqxILyVdckk4HA9utnoxH1tSiVe1XXddOGyDw/jx2h3XVgXl6hUG6LH24Yc6HC852OPa3cYttwC//nU4Pn26tm1NnQrsuqs+fytXjzpAL1bSjuslS8KS3JIlwGGHAQMH6vi0aZomNzi4y1qDB2spLe1YeuIJLQ3V1yc/L8y9yfKhh/S4GDUK+MY3olWlp5wSVvvZdAD6lIHrrw/Hm5qiVWQiwH/9lwbWP/xB25AWLdL9f//9uszf/67VkcuXa35y9NHh8TNkiOYdrq22AjbeOPn7AuH+STuHli6NltSySERK/gLQCKABwJkdme++htuC9Ecf2QK1yMUXh8P2NXOmyJVX+tMBkdGj9X3ddUVGjvTnNzSEw+usk7wNQGThQpHVq/3pxx2Xvo59ffpp/mUKfdXWhsM/+Ulx6269dfL0QYNERKLT/vKX5GW/8Y1w+Be/iO7HSy9N/+zf/S46vu22ycs9+6ymZbfdotMXLdL3448X2Wab/N/17ruj459/LtLY6C9n2fG2NpF3382/fRGRHXbQ4e9/X+Tll0Wuu66w3+Hdd3Wf2/Gddxa5776OHQ9NTfmX6dtXPKtWRZe55hp/vVtuEWlu1uF+/cLj5Iknkj/n4ov9/Vns64ILRHr00OHbb9dz54ADosvsuad+xrPPhtOWLRO5667o73PdddH9bF+PPabvu+6qy40YoeO33ipy5pnJ+3annUTmzAmPQ/fYefNNPbZaWkRmzAjn33qryNtvi3znOyIffyzyyCMib70l8vWvh+uvXq3rrFoVbm/pUj23PvjA/90KBKBZpBP5eGdWTtwgUAegMRgeA6CumPnx11fB4YMPwh3u/nj2ddVV6Qfb7rt3/EB1X3vtJbJkiT/9tNPyr/v666VJAyBy0EGl25Z9bbmlSHt7dNo++5T2M37zm+j40KH6fvXV0el77KFp2X776PQ5c/S9f//o9CFDkj/vD3+Ijv/oR8nLLVpkzyZ9vfKKyLe+lf/7tLeLbLVVOG5P+EJet9wSDXAjRoj86U8d26+FHH+AyI036vv48Zr2+LF88cVhJmlfhx0mcscdOjxsmEhNje6rX/86/XM+/VTksssKT3+vXtHx0aM1mLnHQ3ydfv30wui7303f7vLl6fPOPz88Jl56qWP73X7XyZPT548bJ/LTn+rwhRf681et0oBhx6+9Vveve+y2t4usWCFy7706PmuWLjNvnsiLL+rFzLJlGmQyHhyuANAQDHulg3zzU4ODeyWXlDnmupK0mVApXp98ou9nnx1Ou+SS/Os99VTp0lCO15Zbinz5ZXk/I6nEB4hMnepPu+MOP9N/773k9eNXlYBmHr/9beFpe+edwpedNCk8FtZdt2P74qqrRPbbr3q/97PPinz2WXTaBhvkXmffffWKvr1d5PDD05cbP764tMQ/9+CDRTbeOBzfZJPSf/8+fXLP33xzkc02K81n7bKLvrs1FPb15psiJ5/sf/98+2vUKJH11/fnHXqo5k2//KV0NjiUo82hBoDbejOgyPkwxowxxjQbY8K+e279dFLDUlrdHpB+g1lH2PYJ9wGAhfSaqPafBeW7oerDD6N12+WQVDdujN6sF3fUUdFGakDrfpPU1vrbXLRIu/sWyt4sWAibjo03zl/f7zrssHB4/vzOd36Iu/PO5HtCkpx+evSc2WST3Pdu9Omjz5Vqb9f69HiXa1da99U08c+N99rL1w7XEfm64m68cdg1vRjxxnYgvMG2qcmft802YQ9Le6NmUpd714IFuv+TbvJ98EFt3/nVrwpPc4pMNkiLyGQRqReR+q8m5tthSTuqrk7f3R4qt97aucTts4++9+kTTuvVK/96pQxQHZGr8cz65S/Lm4akjHTwYN2Xl19e2ONOksSDw9e+Vvw23AbOfGyDbbE22ywc/u1vk+8KT+LemJfLbrtFj8u4Qw/V68sTTtBeQy0t4byke0lc++8f7ucjjsi9fLzjQZJcjwYp9inA48blf+7YVlv503I9jmbhQm2ULkZNTefuUr/7bu2AcNllOn788dH5bvd1QO9Qd734oj6RYPr00jyQsszVSo3IXa3kzU+tViq0XtV9vfWWP+3Pf46Ou8XXYl4PPxwO33JL4etNnNj5YmpHXiL5i6tJrz59RHr3zr1MrnrffK9DDgkrSf/97/TlNt88fd5tt0XHC2kv6Myro+1HX3yRPi9Xm9nOOxe2/bY2kU03Dcfj1Q62sfjpp3U83naT63X22SJ33pl7mZ49C9/eueeKPPqoyOzZ4bTnnosuE2+LSHuJiGy4YTi+0Ub+MsOGacO6rRIcOFA7KHz6qcjpp/vLX365bvdnP9O857XXovNfeEHk298Ox3fZRdsu3HY7t63ziiu08Xv2bO0okZQ3uZYvF1m5Mkzr8uU6fdkykQULRE44QWTuXG0jGjdOf/sYZLDNoQ7AmGD4TAQNzgBqcs3PGxzSMtUJE9IPmqTM5plnouPHHBMOP/544Qf3U0+J/PWvekBMmVL4eu7JUKnXaafp0WLbS4p5ufX8Rx4Z3d+HHabvhxzS8bRNnhwezbkyT9uol/R68MHo+N57+8vYzOOuu7Rh/NFHc28z/nIbs92ec8W8RNLnLVsWDsf35wUXRMdvuEHkhz9M3r7bG+3ll6Pzf/97Xebtt6PTp00Lh922NPf1xhvaI9Cddvvt+nk77aTjkyZpY37v3tEgNXWqZqbuuueco2lxM1ObGdtxY0QGD05Oz0MPaQeR447T9QYMCOfttZffK2277XS5V1/V8SFDojmpDRq9e2tDfXu7l9nKhx+G22tvjx4Hn30WLnf00Trtuuv0YvTAA7XxOM5Nnz1H4957T4NBB3Q2OJS8WklEZgGAMaYBQJsdBzAzz/zc0v7iMqlK5zvf0SJw0rwNNgiHd9gh+qiLhga9U7mQp2Gut57euV1frzfaFSpevSMS3v2aT/yxznH2Lu84e4NRUpVDXV30oXtxgweHj9S+4w7dP4B+Z/ukU/sf20nc/Z3EvTEx10MOc7XrrLdedNythtltN737dsstdbxPH61vHzky/z0JLrcuP/6dRDr++PLNNwf++791f44cqVU+hx4aXWbvvfUZVdawYenH6C67RNPlsvslfvf6uuvqPUOTJul9Irb6Ytdd9b13b73nIH7z3aGH6mNfbFWriJ5TX3yhD3MEdJuHHabnpLu+rZaK/wbuf2WIaLuTiN4M+tJL4RMODjhA7y256SYdHzs2XK9nT/9JvnY9+91/8IPo/AkTgB/9SJ9R1adP8rGxxRbhsDHARhvp8De/Gb0D3R4LW2yhnzt9enhXfBL7yJkkgwcXXq1Yap2JLJV4DU+6anBfF13kT1u6VENnUrfTjz+OXqloiI2Ox6/Ukl7/+le4vFvF5L7efdfvFSES7T0lIvLHP+b/vHxXnkB690Ir6R6NiRNz34MhIrJ4sfbmsf7xDy2VDRyoy9irsaTXUUflTvO0aRKxcGE4z91PSb+zfcVLg8OH+9/dliZmzAin/fOf0SvcXK+f/UyrtiZOjP4ODzyg42uvXdjvZ7sj2teRR4rn9tt13qBBWjpbsUKrJOw6r7zi98RqbNR1ly4NpzU3R6t6Xn/dP+aB8L4Sq71dr5KT+vO796rYq+uHHtLxJ58MlzvjDJ1mq2dEtNumvQ9mwoRw+gMPhN0z29uj3Tjjli3Tkk/c6tVhCei739VpU6bott9/P3oPwZw5XrfPgsXT1doaVvlYK1eKTJ+eXPpw2a7iZYKslRzK7rzzouNJpQN7lZc0L+0RGu6VWPxKNH53JBC9Ck9qDNt1Vy29JDWyTpwYHT/2WP9/CNK4jcb2qsnK11Ml6TlPPXvmb1Bfb73oQ/S23VaveEV0fPDg9Kvwm27SZxK99VZ4NemKl2bcx4e424w3HroNjKtWhXcdA8m9n+zv7t7ZO2xY+H8XcVdeqQ9YtHr10kdHuP/5PXRoeJU/dar+3iNH6hXyNttEt2dLfaNGac+13XfX8aTnRtnfY6eddLtrrx29A7hvXz1mdt9de8KIhFfK666rj6oA9Mr2+ee1F8zLL0f/+dB9Tlf8tzNGr3r79tVSgXsV/tOf+usdcoh+J/dBkPacsMeInXbssbrf3SfUHnpoWJmhT7gAAAieSURBVKowxm+Ije+bpMblHj3CY8nePX/44brtQYOi5+jAgbkft1+MIUP8Wo2ePfUvjfOVTB95pPAn51ZDZyJLJV5eySF+1+7VV/t3/boRO76+e1eo9fnnWsqw4ne5vvGGvx23DjHpDtVddtF5Z53lXz266bIWLy7sytNdN77Orbf665x8cvRy4oILtA7UvZq0DV+5PjPJq6/qzU4ieteyXcft++9yG11tY+jf/uZv99RT5asr37o6vdEsfreziMj992ud88qVOr5ihd75mrR/p0/XcbcEZD33nN4dPGFCeEOT3Y4tsdg6cmvu3OgxE9feHt7HMG6cP3/hQr256403/Hk2rd/7XnT6f/6nTv/kk/TPFdFj/P33cy8jIvLjH+v2nn8+/7Iu2xCbiz3uL7mkuG1bgwaF7QmF+uILbbxvbu7YZxaikPMiI5C1BulSv4YPGxbNFJ55RuS888Iqh+uvF5k/P9r6H91D4cueMPl+YPdO1aYmnWYbNC++WB+xYTMkkeTHTOy8s84777zCgoOI35PKfRnjr+tm6t/+tv84BFvlEWcb304/PXk/FRocXCtXasPZRx9psE1a32bwo0aJfO1r8lVwKsTq1dHeK/nce69m+B3hVlvYRyE8+mjx27FVMGPHFreebfi8+ebo9EWL9FgrlaVLtQorX/VH3Jdf6oVJLuecE54vXcmOO4qstVa1U1GQzgaH7FcrxYt/G22kj9luaNBxEW0g3Hrr/NsaNKiwz7TF+lGjwsZY23f+lFP0KZhuNUdSEVVE39Ma0pPkeopjUhF1rbW0OH/SSfo3o/HqrbTqolz/SV3IfkzSs6c2nG26aXI1HBA+dO2QQ8LPztUv39WjR7RBMJ9Ro6LVQsVwqy32319vyBo5svjt2O9YbJ/zLbbQ38j9vwtAq3mKuVkvn3XX1ZsNi2mYB7TxPF71GmfPiVL0t8+S5uZu89/e2Q8O7oE7bVqYedkTL9/NO+7ffnbGI4/oEx3jj9R20+KyJ0UxwcEu26dP9G7aNMZoRnzNNck9ptJO+lz/Sd3Soj02gOIy40JssYXWsR9zTBgU8t285LJtKml/elMutldKsWw7R1o7Vy6lqhOvFvv7FnP8rwnWWqu4Y3YNlv0/+3EzOPdvLm3G5j66edIkbcBznXFG9Hn4Hf3sQYPSG8q2204zAPcu7Xwlh0039e+atplfjx5+0Cv06m7iRG38vf/+9Ku7pJLD5Zdro6C9On3//eRA2Fn2d5s+Hbj55uhdw/msaRnN6NH62JITT6x2Sipv/Hht/C/2fxcoM9as4OCymYybif7858mB4L770p/Lk8RmqoU+hqFfP31ER79+4fNqbHBI698c/2MiIPyuPXr4xfF43/Q0l1+un/3669E/7nHZ6qvhw8Np8R5UhVbBddQ224SPCSiUvQ+is/9fXCk9enTfzLFXL+Dii6udCuqE7AeHNPaqt5C/xoz/Gc8LL+TO+PffX6tq3D8/L4TbldTWux94oF7xxh+yltTttF8/7Z541ln694LrrRf+u1ba//smMSY9MADhzUsdbV8oxNlnF/Y8p2Lcfrs+HNC9UYqIysKIvcLNqPodd5TmV1/VETetH36oDZszZhR+VV1um2yi1Qhnnw2cemo0AM2bp+nP9V++cStWaFtC377R/tC2hJHx346IqscY0yLuw0uLlP1Wr7RqpS231L9LzEpgAMJeLief7JdMBgwoLjAAWuL41a+A554rTfqIiAqU/WqlYrvZVdP99+udpx19pHMSe7er6+CDi6tmIiIqEoNDKW24of9Ar3L485/L/xlE1K2tudVKRERUNtkPDkREVHGZ761kjFkM4M1qpyMjNgRQ5T+jzgzuixD3RYj7IrSNiBTwB/fJst/mALzZme5YXYkxppn7QnFfhLgvQtwXIWNMp54zw2olIiLyMDgQEZFnTQgOk6udgAzhvghxX4S4L0LcF6FO7YvMN0gTEVHlrQklB+rmjDFnOsONxpiGfNOIujpjTF1svKBzo9DzJbPBobue8MaYMcHrCmdat80QjTENAPYLhusAQESaALQZY+qSplUtsWUUfNdGY0yjM61bHhfOdxyTMK1b7IvgvLjHGS/o3CjmfMlkcOguJ3xc8IM3ichkALXBgd1tM8QERwBoC4ZbATSkTOuKzhKRqdDjotMn/poq+E6twXds7a77wn5/Z1Kh50bB50smgwO6zwkfV4vwu7YG4902QzTG1AUngVUDYL4zPiBlWpcSlBZeAgARmSQis9CNjwsAtlRdy33xlULPjYLPl6wGhy5/wicRkclBqQEA6gA0o5tmiIGUv9HrdkYAGBBcEdsqkm55XATBoNUYswDhd+2W+6LcshocurWgCDwrOBG6pYRSA6BXgjZg1ACYlzKtK5pnjwe33aG7McbUQH/zywDcaIyprXKSsqLQc6Pg8yWrj8/oLid8mgYRsX/qnLYvuvr+qQ1O/P4A+gcBcwoA+2iEWgA2eCRN60rmIaxfboOWJLrrcTEGwGUi0maMaQXQiO67L1zFnBsFnS9ZLTlMgSYc6LonfCJjzBgRmRQMNyB5X3T5/SMiU4MGWEBPbjhXzg0A2kRkVtK0qiS4vKYi/L1roO0P3fK4cAXHRxu64b4ISo/1thRZ6LlRzPmS2Zvggm5qrdBGp25x16PTPW0+9KpntIg0Je2L7rh/urPg954PYIQtVXbX4yJod2kF0D/X9+4O+6KcMhsciIioerJarURERFXE4EBERB4GByIi8jA4EBGRh8GBqIOMMTdUOw1E5cLeSkRE5GHJgYiIPAwOREUyxtQG/7nRpR4DTeRicCAqXk3wzqfGUpfFNgeiDjDG3CMio6udDqJyYcmBqEjBY6Pns1qJujIGB6KOacu/CNGai9VKRETkYcmBiIg8DA5ERORhcCAiIg+DAxEReRgciIjIw+BAREQeBgciIvL8P3CaihM+LD2OAAAAAElFTkSuQmCC\n",
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAESCAYAAAAWtRmOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZgdVZkG8PdAWAVtEkCCitDAoMja6TgoOg7YGQ2yCNMQQMFRJo0gyuIYCHF52EkA2WRLhgFm2EJaFCIGTScIBBDpDgRiWGIaCIQtdOcmMdKddPqbP7461Kk6Vbfv7b5Ldff7e556bm237qm6t85X59Q5dY2IgIiIyLVJtRNARETZw+BAREQeBgciIvIwOBARkYfBgYiIPAwORETkYXCgkjLGzKp2GgbCGFNnjFnWz/f2630paWiq1LEsVbppaGFwoJIxxjQCaDTG1BT5vqYyJaloIrIQQHtf+5CUZhHZvUTJmAzgPgATS7S9D5U53TSEMDhQKY0E0Ayg4Mw+yIRPLVuKyiApzcaY2iA4lkKNiOREJFei7QGoSLppCGFwoJIIMp5OALfAz4AajDHLjDE1QWY01xjTECyuBVBjjGk0xtQ572kK3tdgMy9nXp1dN2U9+3kNxphZwed684J1JwXzmowxtQn7ZT9vqrM8Kc2dAKY62+0rXQ3GmFviJZRgeyODbdfmO3Z9bS/heJUt3TT0jKh2AmjIOE5EpgOAMWakMaYuqKKBiLQYY9qD8XZjzFwANcH0QmNMTkSa7YZspiQiLcG0zZhHicj0YPzcYDvx9dqdz+sUkWODzXrzbBWL8/5ZAOz61qkicqwxBgDODaa9NItIzu5jSvrddLUHx6EOQD2AFmc7C40xnQBagpJDe55jl7q9IA27u8dLRE4tV7pp6GHJgUpl9+CKtBGaaQykqmgcgHZnugPAVwE8A2gmKSKnpqxXH4yPtMHJEZ83BkDOubJ+JiEt5wb7VJ+wrJj0u+/vdMZLcQWetL2x8I9XXyqdbsowBgcasCBjnSkizcFV6UQAx+V5S/wGaGewHVvV1AatAnHXnwfN8Oxn1qSs11pE0m3JY2EQNKa7C4P0nBvsk72atp8XT7NroOnKp9Cbx8/AP15A9dJNgwyDAw1IEBhmIJqp1AbLbnEy01sANDl13ac6GdYs47SiCaqn7H2CRgBtIjINQEdQOmkAUJuy3kJbv+5uM2merV5xt+nUzR+HMCOtg14pj3T2c1bS9hFWr6WmC8Bxwb6PAzAudp/AruPe1E88dvm2F+xb5HiVM9009Bg+spuIiOLKVnJwrnKSljUGVyeTyvX5RETUf2UJDkExNrF3pw0aQYuIXL4gQkRE1VGW4BBk/O0piycAsJ172gEk3RgjIqIqqsYNadtZyhpVhTQQEVEebK1ERESeavSQzkGbBAJaiuiIrxA0tWsCgJrNNhuz+377VS51RERDQFtb2/siskN/31+x4BC0vc4BmImw12UtErrgB+2tpwNA/U47SWsr++EQERXDGPP6QN5frtZKjQDqTfRpj/OADx+JbFs05RIecUBERFWW+U5w9TvtJK3vvFPtZBARDSrGmDYRKeaZYBG8IU1ERB4GByIi8jA4EBGRh8GBiIg8DA5ERORhcCAiIg+DAxERebIfHDLeD4OIaCjKfnAgIqKKY3AgIiIPgwMREXkYHIiIyMPgQEREHgYHIiLyMDgQEZGHwYGIiDwMDkRE5GFwICIiD4MDERF5sh8c+GwlIqKKy35wICKiimNwICIiD4MDERF5GByIiMjD4EBERB4GByIi8jA4EBGRh8GBiIg8DA5ERORhcCAiIg+DAxEReUaUY6PGmEYAOQB1IjItz/JaEZmed2N8thIRUcWVvORgjKkDABFpAZCz07Hl7cHy9vhyIiKqvnJUK02AlgoAoB1AQ8I6U4PXWhFZWIY0EBHRAJQjONQA6HSmR7kLg2DQboxZFVuPiIgyouI3pI0xNdCSxWUAZhhjahPWaTLGtBpjWru6uiqdRCKiYa8cwSEHYGQwXgOgI7a8CcBlwY3qiQAa4xsQkekiUi8i9VtuuWUZkkhERPmUIzjMBGBLA7UAWoAPSwwRItKM8P4EERFlRMmbsorIQmNMvTGmAUDOueE8D8AYEZlmjJlkjGkHMLLPpqxERFRxZennkJThi8gYZ9zr+0BERNnBHtJERORhcCAiIk/2gwMfn0FEVHHZDw5ERFRxDA5ERORhcCAiIg+DAxEReRgciIjIw+BAREQeBgciIvIwOBARkYfBgYiIPAwORETkYXAgIiJP9oMDn61ERFRx2Q8ORERUcQwORETkYXAgIiIPgwMREXkYHIiIyMPgQEREHgYHIiLyMDgQEZGHwYGIiDwMDkRE5GFwICIiT/aDA5+tRERUcdkPDkREVHEMDkRE5GFwICIiD4MDERF5RpRjo8aYRgA5AHUiMi1heR2AWgAQkeZypIGIiPqv5CWHIOOHiLQAyNnpmMlBUKhNWU5ERFVUjmqlCdBSAwC0A2hwFwalimcAQESmicjCMqSBiIgGoBzBoQZApzM9KrZ8LIBRxpg6Y8ykMnw+ERENULVuSHfYEkNQkogwxjQZY1qNMa3d3d2VTx0R0TBXjuCQAzAyGK8B0BFb3gGtbrLrjo1vQESmi0i9iNRvscUWZUgiERHlU47gMBNBS6TgtQUAjDE1wbxmZ3kNgvsPqfj4DCKiiit5cHCqixoA5JwbzvOC5e3QVkyNAEaxKSsRUfYYyfiVef1220nrqlXVTgYR0aBijGkTkfr+vp89pImIyMPgQEREHgYHIiLyMDgQEZGHwYGIiDwMDkRE5GFwICIiT/aDQ8b7YRARDUXZDw5ERFRxDA5ERORhcCAiIg+DAxEReRgciIjIw+BAREQeBgciIvIwOBARkYfBgYiIPAwORETkyX5w4OMziIgqLvvBgYiIKo7BgYiIPAUHB2PMR2PTu5Y6MURElA0jilh3JIA1xpjLAQiADgBXliVVRERUVQUFB2PMbiLyajA5U0SeNcbsVsZ0ERFRFRUUHETkVWPMAdDSQ6udV86EERFR9RRacvgogN0B5ABMMMYsA7AdgFUiMr+M6SMioioo9J7DPBEZa8fLlRgiIsqGQlsrrSprKoiIKFMKDQ7HGmOOKWtKiIgoMwq9Ib0awP1lTgsREWVEWXpIG2MajTENxphJfayXdzkAPluJiKgKSh4cjDF1ACAiLQBydjphvQYA40r9+URENHDlKDlMgDZ5BYB2AA1l+AwiIiqjcgSHGgCdzvSo+ArGmLqgZEFERBlUraeyjsy30BjTZIxpNca0rt+woVJpIiKiQDmCQw5h5l8DfUDfhwopNYjIdBGpF5H6zTfbrAxJJCKifIp5KmuhZgKoD8ZrAbQAgDGmRkRyAGqNMbXQADIyCBYLy5AOIiLqp5KXHGxGH7RGyjkZ/7xgebOINAfzakr9+URENHBGMt6PoP6jH5XWNWuqnQwiokHFGNMmIvV9r5mMfxNKREQeBgciIvIwOBARkSf7wSHj90SIiIai7AcHIiKqOAYHIiLyMDgQEZGHwYGIiDwMDkRE5GFwICIiD4MDERF5GByIiMjD4EBERB4GByIi8mQ/OPDxGUREFZf94EBERBXH4EBERB4GByIi8jA4EBGRh8GBiIg8DA5ERORhcCAiIg+DAxEReRgciIjIw+BAREQeBgciIvJkPzjw2UpERBWX/eBAREQVx+BAREQeBgciIvIwOBARkWdEOTZqjGkEkANQJyLTEpY3BaO7i8i55UgDERH1X8lLDsaYOgAQkRYAOTvtLG8A0CIi0wHUBtNERJQh5ahWmgAtNQBAO4B45l/rzGsPpomIKEPKUa1UA6DTmR7lLgxKDFYdgJllSAMREQ1A1W5IB9VNC0VkYcKyJmNMqzGmtaenpwqpIyIa3soRHHIARgbjNQA6UtZrSLsZLSLTRaReROpHjCjLPXMiIsqjHMFhJsL7CLUAWgDAGFNjVzDGNNlWTLwhTUSUPSUPDraaKMj0c0610Txn/lRjzDJjzKoCNljqJBIRUR/KUmcTu+ls540JXlsAbFeOzyUiotJgD2kiIvIwOBARkYfBgYiIPAwORETkYXAgIiIPgwMREXkYHIiIyMPgQEREHgYHIiLyZD848PEZREQVl/3gQEREFcfgQEREHgYHIiLyMDgQEZGHwYGIiDwMDkRE5GFwICIiD4PDQH3wAbB4cbVTQURUUkMrOKxfD6xeXbrt9fbqNvP53veAffcFVvX9d9hERIPF0AoO48cDNTWl297ZZwNbbAFs3Ji+zoIF+rp2bek+t9z+/ndgw4Zqp6L/3n4bmDJFgzcRlcXgCQ7nngvss0/+debPL+1n3nijvvb0ROevXw8cfzywdCkwYoTOG0yZ7bbbAptvDsyeXe2U9M93vwtceinwxBPVTgnRkDWi2gnok3220rRp/Xv/QQcBhx0G/Pznxb/XGH2NlxwWLABmzgTefRfYbDOdFw8gg8GRR5bm2VW9vbr/m28+8G3FLVkC7L67luCsDz7Q13wlOiIakOyXHDZsAJ57Ljpv40bgvvvSqxXcTOPpp4Ff/ELH6+uBn/yk+DTEM/5NN9XX3t6w5NDXvYli3XsvsGKFfoYxwHnnRZfbDNL6xjeATfr4OnM5veoeqLY24NVXw+kzz9TMu9QPSVy5Evjc54DTTivtdtO8805lPodoEMh+cOjtBQ48MJzu7gauvx6YMAG4+26gtdU/qbu7k7fV1gZceWXhn21LDvHgYDPhjRvDkoP9zPXrgcMPB559tvDPcbW2Ag88AJxwgm7Hbveqq8LPPP10YOutgb/+NXzf73+vmfPGjcCDDyZn1FdeCdx+e//S5aqvB2prw+lf/UpfS121tmaNvv7pT8nL7fdTCs3NwOjRwKOPJi/v7gYuvBDo6irdZw4H3/pWaX5zVHHZDw5xzz8PtLfreEcHMHasfy8iLTj0VyElB5tp/PrXwEMPaZXNbbcV/1ljxwLf/KaOr1wZ7ovNCC+5BLjpJh1ftMh//9VXA0cdpemIK2VmmmSgwWHNmjAgAGF600okxZZURIArrvBLokB4/2LhwuT3XnedlkCvu664zyyXl18Gzjor+zfl7767NKVVqrjBFxw+/3ngD3/QcVvH3dERPWm7uoCLLiruhuv772srHldfJQc3ONhM/MQT9fXNN7WZ66JFWhLoy4IFwMMPR+dts40fHNyMLSmzf+ON8PPjbFBL0tMDXHONvj9+HNLMng385S/h9Dnn9K9q6eqrgRtuAD72MR3cNAG6zX320f2dPbv/Qe7QQ4FJk6IlUUD7qVxzjY6nZba2NVpWSg7HHANcey2wbJlOr1nTdzPunp704Adoqff++8PvcM0ava8Wt26dXrj0hf/FMqgNvuAAAK+8oq9uZnfmmeF4d7fegD7yyMK3ucMO2l8hSTw4uDeqbbVSWqZx4olaErAnseub3wT23lvHv/xlbYrr2nZbPzi4adlkE637/8c/wnn2mPT0AK+/Ht1evnsSN9+sTXd32UUDcCGOPBL4538Op6dP79+N+XPOAc44IzrvkUe0BAZoJmOr0H74w/5nOmnVUz/6UTje26vVlIcfHs0Y7X2svu7rfP/7+l0tXqwXLevW9S+tfbG/C3ssamvzN+Pu6gJ+/GNgzBjgz39OXuf884F//3fgscd0eo89gJ128terqwN23NGf//bbwCGHhIEjfl8M0OrWLAWNN94AfvrT7JfAqmBwBgfLzRRdSX0OLrooed033wwz3tde06snmymklRzsD6m1NcyM04LDkiX6mnQ1/sADwIsvpre6cUsOllt1I6KZwoQJ4Tyb1nvvBXbdFZg3L1yWL2Nz79u8+GL6en3ZsEGvYOvrw5LbY48Vf8P+0EM1aMS5LaJsJrNsmd9Lvbc3WqqJEwEaGzXzcwO3CHDXXRqYLr0UeOkl3Q/7nfcVHG65RV/33RfYfvv0C46+LFsWlgIB4IUXwu9eJEyzPQc6OvQ17be0225h6foLX9Bm2HH2GNqAllY6sBdncddeqwF4xgydjp+HDz6ogeXOO5PfXwmLF0d/KyeeqFW1L7yQ/p61a4HOzuRlc+YAt95a2jRmxOAMDjZDduunXW+/7c9La8o6Z050+swzNcN4//1wXk+PZvK2hY4bLGym39UF/OAH6WnOVx3hZgKumppoyUEkWvVk9+l3vwvn2QykrU1fbSc9IDljW7IEmDXLD7R2/0WA3/wm3OekexmuDRv06rCtTUsWv/0t8JWvaHXO/fcDn/hE8YHC/T6XLtWgDGjG3dWlV7j77hu9ALjoIi3V2HXjxo/XfXn33ej32dsbXi3/7W/AZz+r+2HT3FdwiHNbdRVjjz20FAdopr7ffnpDHIjea3J/p4Dei0gSb7SRVHqwQSF+ZW+rSvsSv0cUDw62SjQtjf21fHkYlPuy777RgG0z/Xwlh898Bhg1Kpz+r/8CjjtOxw87DPjP/ywuvYCeI8akB9oMGJzBwV4dpT2yYsWK/O8fP16vYm64wa+ntR3pdtghzNB7erRJpW2h42YmbrWS7TSXJP45bkujtDr+HXYITzBj/Aw86erv5pv11V5hr16tV3MtLckZ2+c+pz/0+LZ32EEztrvu0vrtyy7T+Y2NyWm11q2Ltvg5+mh9feYZrb55661oRuUGNit+wzh+o9um9fTTgYkTw/luSzRbNWJ/I/EranvfKr6stze5OuSXv9RX9xjOmAE0NITTM2f67ytGd3dyMLFVps8/r6/uxcS4cdpyzbIXBdaf/pR8j+bhh3W+e8zscY1n6vfcE467gaOnR4/XJZdoddXll0fXcX/XF1wQnk+33abpam4Oz6V33w2/syTvv6+ln6RWgIccotV5/XlKgQ36blqXLtVWVnbZW2/pq92vq67SC6okK1fmv/ixzfBt6Sp+cZolIpLpYYx+JcnDf/xH8vwttkh/T1/rHnusv86iReH43XeLzJ0bTh98sL5ec03+z5k1S6SnR2TZMpHOzuiyP/85+T2jR0enf/ObwvfLDmecEY5PnZq+3gkn5N/Ot78tItL35x1ySPL8nXcOx198Ubf1+uvF70++oaZGPlRfr/MeeUSn33kn/X3bbx+OT5ki8ulP6/hRR/nrXnFF+Bl2Xmdn/mMjIrJkicjXvy7S3Kzz5s8XWbNGpLc33J79Dj74IHzvf/93OP6974mcemr+Y3DJJSIdHSKbbipy0kkiX/pSYcfugANE9tor/J089ZS/DyL6+3fnf+c7/rYuvFDXfeyx6Pwf/chfd889RR59VF+B8HisWyfyt7+Fn/u//xv+TuPstp56SuSii0Tee0/k3nt1u4cdJrJ+vcjjj0ePq7XLLjo9e3Y471//VefNmxfd/ptvRqd7e6PbW78+PCbW2rUiGzbo+BNPiJx/vq5jjL4efbTIuefqetbUqSItLf5+FglAq0j/895+vzHvRoFGAA0AJvVnuTvkDQ6NjYX98IsZJk7057W1RafnzAnH/+VfCtvu6aeH462t0WUtLYVtw81gCx2+9a1wfMqU9PXGjet7Ww89VJpjPGeOBsslS0r73Y0aJbJxY3TetGl68tlMua9hs83C8aTf1y9/qWfeP/4RnS+Svs2kZfvso6+zZmlGNndueKHy3nvJ27GZVr7h7LNFxo8v7XG1+9DTk5zBxwcbHPrze1m9Wt/7b/+m0xs36vSNN+r0xIlh7jdjRvS9W26pr7vumvydfvaz4byf/ERkwoRw+q679DuYNCmcd+utYYYP6IXG3/8eTq9dG453dYk895yOG6MBoatLp086KflCIz709ur77PT224ucdlq4v1dfLfL73xcSF4KfXMaCA4A6AI3BeBOAumKWx4e8weFrXyvtCbDTTsmZ5NNPR6d/97twvNArM3eYNy86/cADhb1vxIiB7V/aVX21hj32KP0233ijdNs65hh/3qhRIjNn+vNnzUrfTr7A4Q72avKmm/xlm25a3e9q40aRhobC1p0yJQx+xQ7t7dEr8hUrNBP/wQ90+pxztDS1dq3IP/1T6fbv4ouT5194YTh+000it98eTr/6anTdH/9YX7ffXuTQQ4tPw4wZIt//vj//iCNELrggnH72WS11i4i88ILI4sVa2v3LX0Ruu00DWEaDw1QADcG4Vzroa3lRwaFSw223Raf/7//CcVt9UcwQL4rfdVf/0xa/Shruw+c/X7ptfeMbpdnOrbeWfj933FGrXip1XJ95pvB13Sv0YofWVpE//CGcvuqq6HJbjbv//lpNV6n9B7TqayAByQb/cg0f+5i+jhgh8vDDMtDgUI4b0jUA3HZfo4pcDmNMkzGm1RiT0tSkwuI9PE86KRxPaw2Tzx13RKcH0ha+0PbZI0dGp90buUNJvuarxSpVh7dTTinNdlxz5kRvRAPa9Llcxo4tfN2XXur/53R0aF8LK/4d2J7sixb5nUaTHH54/9Pi2morvVE9kNZFaf2uttkG2Guv/m/Xso1eenqAk08e8OYy2VpJRKaLSL2I1H8484ILqpiiMsvXs/VTn8r/3kKCw+OP+72Cy/EE1aHG7SNSbe5TaUW0v0BcVh7BLhKdPvXU6PRBB6W/92tfi7YcmjJlYGm59FJ/XjyoFpIx19enL3vxRW3dBGhrvr33Br7zHX8929rxIx/RwbryyuLPxylTgPfe01aA48aFv4dLLtEWdF/5SnHbS1CO4JADYC9TawB0FLk82ZZbliJt2hM5a/I9KXaXXcI21UlsM8ykk8D6xCe0aarLzWyKbbtfKf19eGGab387Om2vQm3/gSzbait93XXX5OUHHgjsvHPFklOUm2/Wpq7W6NHl/Tx7rABtqr16tT6Ycs89dV5bmwaw2bP1ScX2Aivtyv6II6K1Be658+ST2g/ihBN0+rTTtJm6+7DBlSu1xGFLX+PHawCcPFmDwmGHhZn7wQdr/xv7WJ6HHtKOuitW6BMPrrhCm+xefLGe0/X1wB//qPvU06O93OfO1eayA1WmG9JNwfgkBDecAdTkW97nPYfXXitNvZzbNBDQVhH77BPWZWZtOPTQ/Dc0P/5xff3jH9PXWbVKm2C68847Lxy/+urypf/ss0U+8pH863z5y8nzRUR2203Hf/az9Pdvvnn+7c+fr80QTzopnHfffXrjc/ny6A1QQGT33QvbtyefrMxv4I47wpvj778vEXad3l5tGlvMdk87TeTEE6MtdNIGY/rfICKeVtvq7q67St/i8IILtAkroI0HXE1NOv+tt6LzbaOSe+6JHnM3/R0d4fSxx+qN8pdfDrfR2+t/N3feqcfW6u7Wm/WtrTrd0xM2c12zRpu62qa8b70lsmCBDASydkNa04Qm6M3mJmdeW77lacMY9wtK+0Hst1/hPx73B+D+cNvbC3v/1lsX/ln77+83g40Pn/xkOL7jjv7yr39d0/fIIzp9ySXR5bW1+vroo/rjSvqMjRv95rM//3k4fvPNhe9TscMrr0Tbl9uhuVlk8mRt4fHVrya/VyTMqOfPF9l77+T1tt02Oj19ejh+1VV6EoqEweH225POpHCIt06LD5Mna3t/Ef+49nc44QTtT9PbK3LggdFlq1drC5SnnkpPtxXvnxAf3EDsOvpof93jjtPXxx8P13v44b779NjhggtEHnwwfO8VV0QvYlat0gz1lFN0Or7faUMup8fpV7/Sln9uCyIRzXAnThR56aXoPnZ3izz/vH8Mly0TOfNMbbZ68cUiS5fq/Pvu03PD2rBB5K9//bA1UNZlMjiUcigoOCQ1/UsbZs+OTltuxzSb4SYNI0cW/lk2A8m3zsSJYSuGpH4IRx0V/cZXr44uv+46fV28WJe/9pqf2YqIvP12dJ7bdM+eXPmaKhbSxj5peO01/8oc0ObAlm1hdMQRfrpt56jHH/f7FtjBttIAtE17b692fnrooeixKzQ4uFeJScONN0bf290dXZ6Wef761/68hQtFjj9eL04sty+BLTmmmTlTB8t22Bw9Wn8T69eHGf/kySIrV2q/jxNPjG7HrnPtteFnP/WUZoZJ7Dq2Jdbs2Voqdy+G0sydqxcF1htvaB+FtrbwYmDFiuRj+NOfRjsOimgQsOcmfWjoB4dtttHekSL643N7K8czyKTBLbaefXa0E4v7A3Y7Ty1e7K8H6Elre1SmDbffLnLWWWHnHZH0dZ98UjM82zP1ssv8dY47LvqNJ3W+Wr48uo5bTbDzzv7+ASKXXx6O26a5xx+fnlZ3P5L6dqQ18bNF+Pj8uXPD9O6/v8574QU/c7HH5umn04/nE09o229Aq5jS5AsObvPJDRu0t7X7GW7vchv0Xe66a9eGHZp6eqLpTDqucfPnh8snT07fnyQvv6zv+9SnwnmdnSI33OBnqi5bbdXcrFfkgP+7ctlSXG+vfmeupUu1o2N/LF0anu9f/KJ+xiOPaLXvq68mv2fDBu1kumxZ/z5ziBr6wWHMmKS9jg7u1U58OOus6Ekbf3/Sdl95JflqVyRaDZQ0PPusn17bOcb21PzkJ6PLbeaYdMV50knRdd0elGmZS2en1ievW5d+3NzqKdvT9LvfDdPqDvYq88orRbbbLjkYx6/67WDrYePzH3ssTNeSJSI//KEGMFu9ZzN522Z+4cJw/TffjHZAEglLRltskXxMRMLHrdxxR/Jyd3tvvBGWpOrqosE1LZO1y9evT54f7zSVFGQsW+13yinp6ySxx2+XXYp7n622XLFCp+O/nbg1a/SYl1Mu5wceKtjwDA7nnRc+fuH885PrtO3g1sHamz/ulVz0aOrw2mvRaTfTiD/vyA6bbKKv9rlBrq4urTr5n/8R76pOJKxrdXve2nlf+lJ0XTdoufWhhXDTazv2ffzjYWZ/+um63uOPh+sdfLC/HfexBTb42mffxAf7OIT4fLck4LIND0aP1mnb03bRovT9EdH6a0CrJ9K8955W433wQfJyW71lLVumgTz+e0iTFjwuvFDvdcUvOPLp7tZAne/qPYkNQHvsUdz7aMgZnsEhSby7vttSIX4ypp2cdn5aVYhI8k1jILxRnVb0FQmv6j796ej8ujqd/9vfho8EOftsfT3gAH878+drRlcsm9YHH9SMatEikXffDasRfvYzXW/5curN6doAAAW2SURBVJGtttK6aRtQXblcuK1cTrvti2ime9BBOt+2ULJXoPHj9dxzyWlct07vwdir6n33TV/f/V5sddtA6p0/+ED3Jw3gV/OlpSfNnXcWtl5/9fbqc4PiN2Np2GFwsNxHCbzwgmZ6lr3vEB61/MHBrQpxH08hokEnntFdf304P95MzrVsWbhNl30Ex8MP6zNTTjklbCa5556F7X8hLr1Un68U19WlgcGtSshXPy2imXG+QPjqq9pqyIofs6QSVhJb5ZZUXRf/Hi+8MLwxXy75jss994gcfnjf2zjooOKri4iKxOBgxR+f6+rpiWZ8fQUHWxWyZk30BrCIVgkBYQufpiadb+unV65MT6MNYLfdFp1vqzPs46VF9HHFQHhDebCLBwe3dU4+tpVUXyUHIooYaHAYMfBudBmR70/nN93U7zIff9aQy3Zl33ZbfX3xxfD9c+fqH32cfLL+Y5jtpt7cDDz1lP41ZJqtt9bsLM72UHa70G+3nb6m/RXqYDNnjv49o+0Nbv8kqS93361/DrPffv6yBQvS/yiJiAbESFJmlSH19fXSWujD7eJ/U5hm+XJ92FU8QLj/GW3/irQSvvhFDSwLFmj3eUC79H/mM9pNPt/jMwab0aP1n+BWrMju4x6IhgBjTJu4z6crUkYfqjMANnPNZ5ddkksO11+vzyupZGAAwpKD+xC9TTbR57EMpcAAAFOn6r6N8h7GS0QZMrSCw6pVA3uS5hln6JMOK81WWeWrGhsqTj5ZHxboPryMiDJn6NxzAICammqnoH9uvx249lrgC1+odkqIiAAMteAwWO28s1a3EBFlxNCqViIiopJgcCAiIk/mm7IaY9YCeLna6ciI7QG8X+1EZASPRYjHIsRjEdpLRLbt75sHwz2HlwfSVncoMca08lgoHosQj0WIxyJkjCmwg1gyVisREZGHwYGIiDyDIThMr3YCMoTHIsRjEeKxCPFYhAZ0LDJ/Q5qIiCpvMJQcaJgzxkxyxhuNMQ19zSMa6owxdbHpgs6NQs+XzAaH4XrCG2OagmGqM2/YZojGmAYA44LxOgAQkRYAOWNMXdK8qiW2jIJ9bTTGNDrzhuXvwtnHpoR5w+JYBOfFLGe6oHOjmPMlk8FhuJzwccEX3iIi0wHUBj/sYZshJpgAIBeMtwNoSJk3FE0WkWbo72LAJ/5gFexTe7CP7cP1WNj9d2YVem4UfL5kMjhg+JzwcbUI97U9mB62GaIxpi44CawaAJ3O9KiUeUNKUFp4BgBEZJqILMQw/l0AsKXqWh6LDxV6bhR8vmQ1OAz5Ez6JiEwPSg0AUAegFcM0Qwzk+bu+YWUsgFHBFbGtIhmWv4sgGLQbY1Yh3NdheSzKLavBYVgLisALgxNhWEooNQB6JWgDRg2AjpR5Q1GH/T249x2GG2NMDfQ7vwzADGNMbZWTlBWFnhsFny9ZfXzGcDnh0zSIyLnBeNqxGOrHpzY48UcCGBkEzJkA7KMRagHY4JE0byjpQFi/nIOWJIbr76IJwGUikjPGtANoxPA9Fq5izo2CzpeslhxmQhMODN0TPpExpklEpgXjDUg+FkP++IhIc3ADFtCTG86VcwOAnIgsTJpXlQSXVzPC77sGev9hWP4uXMHvI4dheCyC0mO9LUUWem4Uc75kthNc0EytHXrTaVj0enSap3VCr3qOFZGWpGMxHI/PcBZ8350AxtpS5XD9XQT3XdoBjMy338PhWJRTZoMDERFVT1arlYiIqIoYHIiIyMPgQEREHgYHIiLyMDgQ9ZMx5pZqp4GoXNhaiYiIPCw5EBGRh8GBqEjGmNrgPzeG1GOgiVwMDkTFqwle+dRYGrJ4z4GoH4wxs0Tk2Gqng6hcWHIgKlLw2OhOVivRUMbgQNQ/ub5XIRq8WK1EREQelhyIiMjD4EBERB4GByIi8jA4EBGRh8GBiIg8DA5ERORhcCAiIs//A5N4dKmwUBzSAAAAAElFTkSuQmCC\n",
       "text/plain": [
        "
    "
       ]
diff --git a/doc/src/LectureNotes/_build/jupyter_execute/chapter2_178_0.png b/doc/src/LectureNotes/_build/jupyter_execute/chapter2_178_0.png
index 80d8fd112..e1fe92955 100644
Binary files a/doc/src/LectureNotes/_build/jupyter_execute/chapter2_178_0.png and b/doc/src/LectureNotes/_build/jupyter_execute/chapter2_178_0.png differ
diff --git a/doc/src/LectureNotes/_build/jupyter_execute/chapter2_184_1.png b/doc/src/LectureNotes/_build/jupyter_execute/chapter2_184_1.png
index c7b66ed0b..c5fadabc8 100644
Binary files a/doc/src/LectureNotes/_build/jupyter_execute/chapter2_184_1.png and b/doc/src/LectureNotes/_build/jupyter_execute/chapter2_184_1.png differ
diff --git a/doc/src/LectureNotes/_build/jupyter_execute/chapter3.ipynb b/doc/src/LectureNotes/_build/jupyter_execute/chapter3.ipynb
index 713a567dc..b64d4a556 100644
--- a/doc/src/LectureNotes/_build/jupyter_execute/chapter3.ipynb
+++ b/doc/src/LectureNotes/_build/jupyter_execute/chapter3.ipynb
@@ -440,8 +440,8 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "[-0.36370241  0.57281693  0.31446735 -1.74797657 -1.47723222 -1.24113195\n",
-      " -0.714415   -1.26679543 -0.27301697  0.6793838 ]\n"
+      "[-2.57325284 -1.11383985  0.52678895 -0.47530983 -0.1807752   0.52810513\n",
+      " -1.50448048  0.33629261 -1.41402417 -0.39264297]\n"
      ]
     }
    ],
diff --git a/doc/src/LectureNotes/_build/jupyter_execute/chapter4.ipynb b/doc/src/LectureNotes/_build/jupyter_execute/chapter4.ipynb
index 90089505d..ade2177c1 100644
--- a/doc/src/LectureNotes/_build/jupyter_execute/chapter4.ipynb
+++ b/doc/src/LectureNotes/_build/jupyter_execute/chapter4.ipynb
@@ -1750,13 +1750,13 @@
      "output_type": "stream",
      "text": [
       "Training R2\n",
-      "0.9999854211447763\n",
+      "0.9999863998248437\n",
       "Training MSE\n",
-      "6.298065171189058\n",
+      "6.805697972778246\n",
       "Test R2\n",
-      "0.9999855160996954\n",
+      "0.9999773431421539\n",
       "Test MSE\n",
-      "7.1872710004915445\n"
+      "4.962804899252701\n"
      ]
     }
    ],
@@ -2041,7 +2041,7 @@
     {
      "data": {
       "text/plain": [
-       ""
+       ""
       ]
      },
      "execution_count": 16,
@@ -2353,27 +2353,27 @@
       "MSE before scaling: 0.00\n",
       "R2 score before scaling 0.99\n",
       "Feature min values before scaling:\n",
-      " [1.00000000e+00 1.00405815e-03 3.33637629e-03 1.00813277e-06\n",
-      " 3.34991581e-06 1.11314068e-05 1.01222393e-09 3.36351028e-09\n",
-      " 1.11765797e-08 3.71385616e-08 1.01633169e-12 3.37715992e-12\n",
-      " 1.12219360e-11 3.72892756e-11 1.23908217e-10 1.02045611e-15\n",
-      " 3.39086494e-15 1.12674763e-14 3.74406011e-14 1.24411055e-13\n",
-      " 4.13404436e-13]\n",
+      " [1.00000000e+00 4.73875395e-04 1.62035199e-03 2.24557890e-07\n",
+      " 7.67844938e-07 2.62554056e-06 1.06412459e-10 3.63862824e-10\n",
+      " 1.24417907e-09 4.25429987e-09 5.04262460e-14 1.72425639e-13\n",
+      " 5.89585850e-13 2.01600803e-12 6.89346326e-12 2.38957572e-17\n",
+      " 8.17082679e-17 2.79390227e-16 9.55336603e-16 3.26664263e-15\n",
+      " 1.11698369e-14]\n",
       "Feature max values before scaling:\n",
-      " [1.         0.99985941 0.99746704 0.99971883 0.99732681 0.99494051\n",
-      " 0.99957828 0.99718659 0.99480062 0.99242037 0.99943775 0.99704639\n",
-      " 0.99466076 0.99228084 0.98990661 0.99929723 0.99690622 0.99452092\n",
-      " 0.99214133 0.98976744 0.98739922]\n",
+      " [1.         0.99909628 0.99839582 0.99819339 0.99749355 0.99679421\n",
+      " 0.9972913  0.9965921  0.9958934  0.99519518 0.99639003 0.99569147\n",
+      " 0.99499339 0.9942958  0.99359871 0.99548958 0.99479165 0.9940942\n",
+      " 0.99339724 0.99270078 0.9920048 ]\n",
       "Feature min values after scaling:\n",
-      " [ 0.         -1.79147165 -1.7862625  -1.1584177  -1.16385875 -1.16903529\n",
-      " -0.9111456  -0.91736698 -0.92355665 -0.92969742 -0.77335697 -0.77892437\n",
-      " -0.78453895 -0.79019355 -0.79588019 -0.6835654  -0.68819847 -0.69289982\n",
-      " -0.69766733 -0.70249839 -0.70738995]\n",
+      " [ 0.         -1.71950803 -1.74066137 -1.12098766 -1.13008641 -1.13882035\n",
+      " -0.88585974 -0.89326197 -0.90038244 -0.90721064 -0.75127414 -0.7579701\n",
+      " -0.76446674 -0.77075409 -0.77682281 -0.6609642  -0.66708788 -0.67306513\n",
+      " -0.67888673 -0.68454393 -0.69002844]\n",
       "Feature max values after scaling:\n",
-      " [0.         1.702894   1.66035996 2.19865243 2.16709986 2.13442274\n",
-      " 2.59980117 2.57660557 2.55254863 2.52759568 2.94821185 2.93060502\n",
-      " 2.91238065 2.89350743 2.8739528  3.26264844 3.24876534 3.23444979\n",
-      " 3.21967888 3.20442837 3.18867266]\n",
+      " [0.         1.75646602 1.64607128 2.30045841 2.19871569 2.09765001\n",
+      " 2.75230652 2.65433203 2.55670018 2.45952257 3.14599634 3.05079367\n",
+      " 2.95561862 2.86057207 2.76575615 3.49778287 3.40509201 3.31215604\n",
+      " 3.21906542 3.12591311 3.03279417]\n",
       "MSE after  scaling: 0.00\n",
       "R2 score for  scaled data: 0.99\n"
      ]
@@ -3623,10 +3623,10 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "0.012893004091480358\n",
-      "4.036145788156587\n",
-      "[[0.84855665 2.43974599]\n",
-      " [2.43974599 8.11534493]]\n"
+      "0.13175867934658783\n",
+      "4.345411944186196\n",
+      "[[ 0.99370801  2.9978939 ]\n",
+      " [ 2.9978939  10.13547884]]\n"
      ]
     }
    ],
@@ -3667,10 +3667,10 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "0.08534074521320528\n",
-      "1.494502745126961\n",
-      "[[1.         0.54933577]\n",
-      " [0.54933577 1.        ]]\n"
+      "0.07991186451576213\n",
+      "1.435691712612069\n",
+      "[[1.         0.69854547]\n",
+      " [0.69854547 1.        ]]\n"
      ]
     }
    ],
@@ -3724,30 +3724,30 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "[[ 0.01162555 -0.25149407]\n",
-      " [ 0.32741231  1.04530907]\n",
-      " [-0.5034199  -1.57100552]\n",
-      " [ 1.67655824  6.4270278 ]\n",
-      " [-2.84730394 -8.72396496]\n",
-      " [ 0.71213034  4.19282793]\n",
-      " [ 0.6767474   0.58026128]\n",
-      " [ 0.05853864 -0.83340221]\n",
-      " [-0.2484568  -1.90992384]\n",
-      " [ 0.13616818  1.04436453]]\n",
+      "[[-0.35269644 -0.0359786 ]\n",
+      " [ 0.91157022  2.38965729]\n",
+      " [-0.68652791 -1.88459927]\n",
+      " [ 0.73533968  3.47606953]\n",
+      " [-0.59301581 -2.13696995]\n",
+      " [-0.37142517 -1.24766758]\n",
+      " [-0.12619024 -0.45164224]\n",
+      " [-1.74004988 -5.63952222]\n",
+      " [ 1.7194963   3.93948143]\n",
+      " [ 0.50349923  1.59117161]]\n",
       "          0         1\n",
-      "0  0.011626 -0.251494\n",
-      "1  0.327412  1.045309\n",
-      "2 -0.503420 -1.571006\n",
-      "3  1.676558  6.427028\n",
-      "4 -2.847304 -8.723965\n",
-      "5  0.712130  4.192828\n",
-      "6  0.676747  0.580261\n",
-      "7  0.058539 -0.833402\n",
-      "8 -0.248457 -1.909924\n",
-      "9  0.136168  1.044365\n",
-      "        0       1\n",
-      "0  1.0000  0.9645\n",
-      "1  0.9645  1.0000\n"
+      "0 -0.352696 -0.035979\n",
+      "1  0.911570  2.389657\n",
+      "2 -0.686528 -1.884599\n",
+      "3  0.735340  3.476070\n",
+      "4 -0.593016 -2.136970\n",
+      "5 -0.371425 -1.247668\n",
+      "6 -0.126190 -0.451642\n",
+      "7 -1.740050 -5.639522\n",
+      "8  1.719496  3.939481\n",
+      "9  0.503499  1.591172\n",
+      "          0         1\n",
+      "0  1.000000  0.970341\n",
+      "1  0.970341  1.000000\n"
      ]
     }
    ],
@@ -3789,37 +3789,37 @@
      "text": [
       "     0         1         2         3         4         5         6         7   \\\n",
       "0   0.0  0.000000  0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
-      "1   0.0  0.067129  0.071920  0.066488  0.067965  0.069289  0.058136  0.058890   \n",
-      "2   0.0  0.071920  0.077903  0.072255  0.074320  0.076177  0.063662  0.064746   \n",
-      "3   0.0  0.066488  0.072255  0.070120  0.072252  0.074165  0.064037  0.065174   \n",
-      "4   0.0  0.067965  0.074320  0.072252  0.074718  0.076940  0.066275  0.067613   \n",
-      "5   0.0  0.069289  0.076177  0.074165  0.076940  0.079448  0.068289  0.069815   \n",
-      "6   0.0  0.058136  0.063662  0.064037  0.066275  0.068289  0.060371  0.061607   \n",
-      "7   0.0  0.058890  0.064746  0.065174  0.067613  0.069815  0.061607  0.062970   \n",
-      "8   0.0  0.059633  0.065798  0.066265  0.068894  0.071275  0.062784  0.064269   \n",
-      "9   0.0  0.060376  0.066836  0.067327  0.070137  0.072691  0.063919  0.065523   \n",
-      "10  0.0  0.049985  0.054935  0.056831  0.058951  0.060864  0.054874  0.056079   \n",
-      "11  0.0  0.050516  0.055671  0.057601  0.059851  0.061888  0.055712  0.057004   \n",
-      "12  0.0  0.051063  0.056417  0.058372  0.060747  0.062904  0.056542  0.057918   \n",
-      "13  0.0  0.051629  0.057177  0.059149  0.061648  0.063923  0.057370  0.058831   \n",
-      "14  0.0  0.052218  0.057957  0.059939  0.062558  0.064951  0.058203  0.059747   \n",
+      "1   0.0  0.080229  0.080679  0.087554  0.086417  0.085175  0.083141  0.081602   \n",
+      "2   0.0  0.080679  0.082259  0.089654  0.089067  0.088307  0.086044  0.084862   \n",
+      "3   0.0  0.087554  0.089654  0.100625  0.100093  0.099390  0.098651  0.097358   \n",
+      "4   0.0  0.086417  0.089067  0.100093  0.099970  0.099642  0.098705  0.097743   \n",
+      "5   0.0  0.085175  0.088307  0.099390  0.099642  0.099662  0.098566  0.097913   \n",
+      "6   0.0  0.083141  0.086044  0.098651  0.098705  0.098566  0.098847  0.098003   \n",
+      "7   0.0  0.081602  0.084862  0.097358  0.097743  0.097913  0.098003  0.097449   \n",
+      "8   0.0  0.080151  0.083732  0.096133  0.096822  0.097279  0.097201  0.096917   \n",
+      "9   0.0  0.078794  0.082667  0.094987  0.095957  0.096680  0.096453  0.096419   \n",
+      "10  0.0  0.076701  0.079987  0.093012  0.093523  0.093835  0.094703  0.094277   \n",
+      "11  0.0  0.075255  0.078808  0.091666  0.092450  0.093019  0.093698  0.093521   \n",
+      "12  0.0  0.073932  0.077731  0.090440  0.091477  0.092285  0.092793  0.092846   \n",
+      "13  0.0  0.072727  0.076752  0.089333  0.090604  0.091633  0.091986  0.092253   \n",
+      "14  0.0  0.071634  0.075869  0.088337  0.089826  0.091062  0.091273  0.091739   \n",
       "\n",
       "          8         9         10        11        12        13        14  \n",
       "0   0.000000  0.000000  0.000000  0.000000  0.000000  0.000000  0.000000  \n",
-      "1   0.059633  0.060376  0.049985  0.050516  0.051063  0.051629  0.052218  \n",
-      "2   0.065798  0.066836  0.054935  0.055671  0.056417  0.057177  0.057957  \n",
-      "3   0.066265  0.067327  0.056831  0.057601  0.058372  0.059149  0.059939  \n",
-      "4   0.068894  0.070137  0.058951  0.059851  0.060747  0.061648  0.062558  \n",
-      "5   0.071275  0.072691  0.060864  0.061888  0.062904  0.063923  0.064951  \n",
-      "6   0.062784  0.063919  0.054874  0.055712  0.056542  0.057370  0.058203  \n",
-      "7   0.064269  0.065523  0.056079  0.057004  0.057918  0.058831  0.059747  \n",
-      "8   0.065687  0.067056  0.057224  0.058233  0.059231  0.060226  0.061224  \n",
-      "9   0.067056  0.068537  0.058326  0.059418  0.060497  0.061573  0.062652  \n",
-      "10  0.057224  0.058326  0.050807  0.051634  0.052448  0.053259  0.054070  \n",
-      "11  0.058233  0.059418  0.051634  0.052522  0.053398  0.054270  0.055142  \n",
-      "12  0.059231  0.060497  0.052448  0.053398  0.054336  0.055268  0.056202  \n",
-      "13  0.060226  0.061573  0.053259  0.054270  0.055268  0.056262  0.057257  \n",
-      "14  0.061224  0.062652  0.054070  0.055142  0.056202  0.057257  0.058313  \n"
+      "1   0.080151  0.078794  0.076701  0.075255  0.073932  0.072727  0.071634  \n",
+      "2   0.083732  0.082667  0.079987  0.078808  0.077731  0.076752  0.075869  \n",
+      "3   0.096133  0.094987  0.093012  0.091666  0.090440  0.089333  0.088337  \n",
+      "4   0.096822  0.095957  0.093523  0.092450  0.091477  0.090604  0.089826  \n",
+      "5   0.097279  0.096680  0.093835  0.093019  0.092285  0.091633  0.091062  \n",
+      "6   0.097201  0.096453  0.094703  0.093698  0.092793  0.091986  0.091273  \n",
+      "7   0.096917  0.096419  0.094277  0.093521  0.092846  0.092253  0.091739  \n",
+      "8   0.096634  0.096370  0.093871  0.093345  0.092885  0.092491  0.092163  \n",
+      "9   0.096370  0.096324  0.093496  0.093186  0.092926  0.092718  0.092564  \n",
+      "10  0.093871  0.093496  0.091863  0.091208  0.090630  0.090128  0.089699  \n",
+      "11  0.093345  0.093186  0.091208  0.090769  0.090391  0.090075  0.089820  \n",
+      "12  0.092885  0.092926  0.090630  0.090391  0.090200  0.090057  0.089963  \n",
+      "13  0.092491  0.092718  0.090128  0.090075  0.090057  0.090076  0.090133  \n",
+      "14  0.092163  0.092564  0.089699  0.089820  0.089963  0.090133  0.090330  \n"
      ]
     }
    ],
@@ -4405,10 +4405,10 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Runtime: 0.410282 sec\n",
+      "Runtime: 0.396214 sec\n",
       "Jackknife Statistics :\n",
       "original           bias      std. error\n",
-      " 100.139        100.129        0.148994\n"
+      " 99.9535        99.9435        0.150002\n"
      ]
     }
    ],
@@ -4559,10 +4559,10 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Runtime: 2.02233 sec\n",
+      "Runtime: 2.03201 sec\n",
       "Bootstrap Statistics :\n",
       "original           bias      std. error\n",
-      " 100.067  14.9691         100.07        0.150582\n"
+      "  100.07   15.041        100.069        0.149551\n"
      ]
     },
     {
@@ -4584,7 +4584,7 @@
     },
     {
      "data": {
-      "image/png": "iVBORw0KGgoAAAANSUhEUgAAArcAAAHjCAYAAAA5ajcLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3df6zV9WH/8dflXrgwvabWnSMOGUm3piwkFaPpis7LXBtA4RZza1PUiJ1f649ZrI66OWAaWl1piyFtjV2bzS7amg2p9gJz127TsjpsjGSV2dJkaZUpcZcLtoOr3Av33vP9o9tdkQr3yr3cy/s+HkkT7+e+D/f94X3POc8e3ud86mq1Wi0AAFCACaM9AQAAGC7iFgCAYohbAACKIW4BACiGuAUAoBjiFgCAYohbAACK0TDaE3izn/3s9fT3++jdNzvjjFOzd2/XaE+Do7BGY5v1Gfus0dhnjca28bI+EybU5fTTT3nL74+5uO3vr4nbt+DvZeyzRmOb9Rn7rNHYZ43GNutjWwIAAAURtwAAFEPcAgBQDHELAEAxxC0AAMUQtwAAFEPcAgBQDHELAEAxxC0AAMUQtwAAFEPcAgBQDHELAEAxxC0AAMUQtwAAFEPcAgBQDHELAEAxxC0AAMUQtwAAFKNhMIOefPLJ3HfffTlw4EAuvPDCrFq1Klu3bs1nP/vZ9PT05JJLLsltt92WJNmxY0dWrlyZ119/Peeff35Wr16dhoZB/RiAk1LTaVMyuXHwj3PdPb3Zv+/ACM4IYPw65qPxyy+/nLvuuiuPPPJIzjjjjFxzzTXZsmVL7rrrrjz00EM566yzcsMNN2TLli2ZO3dubr/99tx9992ZPXt2VqxYkfXr1+fKK688EecCMComNzakZXnboMdvundx9o/gfADGs2NuS/jHf/zHXHrppZk6dWomTpyYdevWZcqUKZkxY0amT5+ehoaGtLS0pL29Pbt27Up3d3dmz56dJGltbU17e/uInwQAACSDeOV2586dmThxYm688ca8+uqr+f3f//28+93vTqVSGRhTrVbT0dGR3bt3H3a8Uqmko6NjZGYOAABvcsy47evry3PPPZeHHnoov/Zrv5abbropkydPTl1d3cCYWq2Wurq69Pf3/8rjQ3HGGacOafx4Uqk0jfYUOAZrNLaNlfU5eKhvSHM5eKgvkybWj+CMxo6xska8NWs0tlmfQcTtr//6r2fOnDl55zvfmST54Ac/mPb29tTX/98DbWdnZ6rVaqZOnZrOzs6B43v27Em1Wh3ShPbu7Up/f21ItxkPKpWmdHbapTeWWaOxbSTXZ6hPJpMm1g95j+54+N1yHxr7rNHYNl7WZ8KEuqO+GHrMPbcXX3xxnn766ezbty99fX353ve+lwULFuTFF1/Mzp0709fXl82bN6e5uTnTpk1LY2Njtm3bliRpa2tLc3Pz8J0NAAAcxTFfuT3nnHNy3XXX5corr8yhQ4dy4YUX5oorrsi73vWuLFu2LD09PZk7d24WLFiQJFm7dm1WrVqVrq6uzJo1K0uXLh3xkwAAgGSQn3N7+eWX5/LLLz/s2Jw5c7Jx48Yjxs6cOTMbNmwYntkBAMAQuEIZAADFELcAABRD3AIAUAxxCwBAMcQtAADFGNSnJQAweoZyRbOeg31pnDT4q5l19/Rm/74Db3dqAGOOuAUY44ZyRbNN9y4e8tXPyr+eETCe2JYAAEAxxC0AAMUQtwAAFEPcAgBQDHELAEAxxC0AAMUQtwAAFEPcAgBQDHELAEAxXKEMKF7TaVOSZNCXsHVJWoCTl7gFije5sWFIl6T91ppFgw5hAMYWcQvwJpMm1g8phjfdu3gEZwPAUNhzCwBAMcQtAADFELcAABRD3AIAUAxxCwBAMcQtAADFELcAABRD3AIAUAxxCwBAMcQtAADFELcAABRD3AIAUAxxCwBAMcQtAADFELcAABRD3AIAUAxxCwBAMcQtAADFaBjtCQAweg4e6kul0jTo8d09vdm/78AIzgjg+IhbgHFs0sT6tCxvG/T4Tfcuzv4RnA/A8bItAQCAYohbAACKYVsCcNJpOm1KJjd6+ALgSJ4dgJPO5MaGIe8TBWB8sC0BAIBiiFsAAIohbgEAKIa4BQCgGOIWAIBiiFsAAIohbgEAKIa4BQCgGOIWAIBiiFsAAIohbgEAKIa4BQCgGOIWAIBiiFsAAIohbgEAKIa4BQCgGOIWAIBiiFsAAIohbgEAKEbDYAZdffXVee2119LQ8Ivhn/70p/P666/ns5/9bHp6enLJJZfktttuS5Ls2LEjK1euzOuvv57zzz8/q1evHrgdAACMpGNWZ61Wy0svvZSnnnpqIFK7u7uzYMGCPPTQQznrrLNyww03ZMuWLZk7d25uv/323H333Zk9e3ZWrFiR9evX58orrxzxEwEAgGNuS/jpT3+aJLn22mvzoQ99KN/4xjeyffv2zJgxI9OnT09DQ0NaWlrS3t6eXbt2pbu7O7Nnz06StLa2pr29fWTPAAAA/scxX7ndt29f5syZkz//8z/PoUOHsnTp0lx33XWpVCoDY6rVajo6OrJ79+7DjlcqlXR0dAxpQmecceqQxo8nlUrTaE+BY7BGjAcj+XvuPjT2WaOxzfoMIm7PPffcnHvuuQNfX3755fnSl76U8847b+BYrVZLXV1d+vv7U1dXd8Txodi7tyv9/bUh3WY8qFSa0tm5f7SnwVFYoxPHg/foGqnfc/ehsc8ajW3jZX0mTKg76ouhx9yW8Nxzz+WZZ54Z+LpWq2XatGnp7OwcONbZ2ZlqtZqpU6cednzPnj2pVqtvd+4AADAkx4zb/fv35/Of/3x6enrS1dWVxx57LH/8x3+cF198MTt37kxfX182b96c5ubmTJs2LY2Njdm2bVuSpK2tLc3NzSN+EgAAkAxiW8LFF1+c559/Ppdddln6+/tz5ZVX5txzz82aNWuybNmy9PT0ZO7cuVmwYEGSZO3atVm1alW6uroya9asLF26dMRPAgAAkkF+zu2tt96aW2+99bBjc+bMycaNG48YO3PmzGzYsGF4ZgcAAEPgCmUAABRD3AIAUAxxCwBAMcQtAADFELcAABRD3AIAUAxxCwBAMcQtAADFELcAABRD3AIAUAxxCwBAMcQtAADFELcAABRD3AIAUAxxCwBAMcQtAADFELcAABRD3AIAUAxxCwBAMRpGewIAnDwOHupLpdI06PHdPb3Zv+/ACM4I4HDiFoBBmzSxPi3L2wY9ftO9i7N/BOcD8Ga2JQAAUAxxCwBAMcQtAADFELcAABRD3AIAUAyflgAMu6bTpmRy4+AfXnxcFADDRdwCw25yY4OPiwJgVNiWAABAMcQtAADFELcAABRD3AIAUAxxCwBAMcQtAADFELcAABRD3AIAUAxxCwBAMcQtAADFELcAABRD3AIAUIyG0Z4AwMFDfalUmkZ7GgAUQNwCo27SxPq0LG8b9PhN9y4ewdkAcDKzLQEAgGKIWwAAiiFuAQAohrgFAKAY4hYAgGKIWwAAiiFuAQAohrgFAKAY4hYAgGKIWwAAiiFuAQAohrgFAKAY4hYAgGKIWwAAiiFuAQAohrgFAKAY4hYAgGKIWwAAiiFuAQAohrgFAKAYg47bz33uc7njjjuSJFu3bk1LS0vmzZuXdevWDYzZsWNHWltbM3/+/KxcuTK9vb3DP2MAAHgLg4rbZ555Jo899liSpLu7OytWrMj999+fxx9/PC+88EK2bNmSJLn99ttz55135oknnkitVsv69etHbuYAAPAmx4zbn//851m3bl1uvPHGJMn27dszY8aMTJ8+PQ0NDWlpaUl7e3t27dqV7u7uzJ49O0nS2tqa9vb2kZ09AGPawUN9qVSaBvW/JGk6bcoozxg42TUca8Cdd96Z2267La+++mqSZPfu3alUKgPfr1ar6ejoOOJ4pVJJR0fHkCd0xhmnDvk248X/PvgzdlkjONykifVpWd426PGb7l2cye5HY5rHubHN+hwjbh955JGcddZZmTNnTh599NEkSX9/f+rq6gbG1Gq11NXVveXxodq7tyv9/bUh3650lUpTOjv3j/Y0OApr9H88uHI83I/GLo9zY9t4WZ8JE+qO+mLoUeP28ccfT2dnZxYvXpz//u//zhtvvJFdu3alvr5+YExnZ2eq1WqmTp2azs7OgeN79uxJtVodhlMAAIDBOWrcfv3rXx/470cffTTPPvtsVq9enXnz5mXnzp05++yzs3nz5nz4wx/OtGnT0tjYmG3btuW8885LW1tbmpubR/wEAADgfx1zz+2bNTY2Zs2aNVm2bFl6enoyd+7cLFiwIEmydu3arFq1Kl1dXZk1a1aWLl067BMGAIC3Mui4bW1tTWtra5Jkzpw52bhx4xFjZs6cmQ0bNgzf7AAAYAhcoQwAgGKIWwAAiiFuAQAohrgFAKAY4hYAgGKIWwAAiiFuAQAohrgFAKAY4hYAgGKIWwAAiiFuAQAohrgFAKAY4hYAgGKIWwAAiiFuAQAohrgFAKAY4hYAgGKIWwAAiiFuAQAohrgFAKAY4hYAgGKIWwAAiiFuAQAohrgFAKAY4hYAgGKIWwAAiiFuAQAoRsNoTwA4OTSdNiWTGz1kMLIOHupLpdI06PHdPb3Zv+/ACM4IONl4pgIGZXJjQ1qWtw1q7KZ7F4/wbCjVpIn1g/49S37xu7Z/BOcDnHxsSwAAoBjiFgCAYohbAACKIW4BACiGuAUAoBjiFgCAYohbAACKIW4BACiGuAUAoBjiFgCAYohbAACKIW4BACiGuAUAoBgNoz0BAHi7Dh7qS6XSNOjx3T292b/vwAjOCBht4haAk9akifVpWd426PGb7l2c/SM4H2D02ZYAAEAxxC0AAMUQtwAAFEPcAgBQDHELAEAxfFoCjFNNp03J5EYPAQCUxTMbjFOTGxuG/BFKADDW2ZYAAEAxxC0AAMUQtwAAFEPcAgBQDHELAEAxxC0AAMUQtwAAFEPcAgBQDHELAEAxxC0AAMUYVNx+8YtfzKWXXpqFCxfm61//epJk69ataWlpybx587Ju3bqBsTt27Ehra2vmz5+flStXpre3d2RmDgAAb3LMuH322Wfz/e9/Pxs3bsy3vvWtPPTQQ/nxj3+cFStW5P7778/jjz+eF154IVu2bEmS3H777bnzzjvzxBNPpFarZf369SN+EgAAkAwibt/3vvflwQcfTENDQ/bu3Zu+vr7s27cvM2bMyPTp09PQ0JCWlpa0t7dn165d6e7uzuzZs5Mkra2taW9vH/GTAACAZJDbEiZOnJgvfelLWbhwYebMmZPdu3enUqkMfL9araajo+OI45VKJR0dHcM/awAA+BUaBjvwlltuycc//vHceOONeemll1JXVzfwvVqtlrq6uvT39//K40NxxhmnDmn8eFKpNI32FDgGawRjn/vp8fH3N7ZZn0HE7U9+8pMcPHgwv/M7v5MpU6Zk3rx5aW9vT319/cCYzs7OVKvVTJ06NZ2dnQPH9+zZk2q1OqQJ7d3blf7+2pBuMx5UKk3p7Nw/2tPgKE62NfIAyHh1Mt1Px5qT7XFuvBkv6zNhQt1RXww95raEV155JatWrcrBgwdz8ODB/PM//3OWLFmSF198MTt37kxfX182b96c5ubmTJs2LY2Njdm2bVuSpK2tLc3NzcN3NgAAcBTHfOV27ty52b59ey677LLU19dn3rx5WbhwYd75zndm2bJl6enpydy5c7NgwYIkydq1a7Nq1ap0dXVl1qxZWbp06YifBAAAJIPcc7ts2bIsW7bssGNz5szJxo0bjxg7c+bMbNiwYXhmBwAAQ+AKZQAAFEPcAgBQDHELAEAxxC0AAMUQtwAAFGPQVygDgJPdwUN9g76ASXdPb/bvOzDCMwKGm7gFYNyYNLE+LcvbBjV2072LU/61nqA8tiUAAFAMcQsAQDHELQAAxRC3AAAUwxvKoBBNp03J5EZ3aQDGN8+EUIjJjQ2Dfhd48ot3ggNAaWxLAACgGOIWAIBiiFsAAIohbgEAKIa4BQCgGOIWAIBiiFsAAIohbgEAKIa4BQCgGOIWAIBiiFsAAIohbgEAKIa4BQCgGOIWAIBiiFsAAIrRMNoTAICx6OChvlQqTYMe393Tm/37DozgjIDBELcA8CtMmlifluVtgx6/6d7F2T+C8wEGx7YEAACKIW4BACiGuAUAoBjiFgCAYohbAACKIW4BACiGuAUAoBjiFgCAYohbAACKIW4BACiGuAUAoBjiFgCAYohbAACKIW4BACiGuAUAoBjiFgCAYohbAACKIW4BACiGuAUAoBjiFgCAYohbAACKIW4BACiGuAUAoBjiFgCAYohbAACKIW4BACiGuAUAoBjiFgCAYohbAACKIW4BACiGuAUAoBjiFgCAYgwqbu+7774sXLgwCxcuzOc///kkydatW9PS0pJ58+Zl3bp1A2N37NiR1tbWzJ8/PytXrkxvb+/IzBwAAN7kmHG7devWPP3003nsscfy7W9/Oz/84Q+zefPmrFixIvfff38ef/zxvPDCC9myZUuS5Pbbb8+dd96ZJ554IrVaLevXrx/xkwAAgGQQcVupVHLHHXdk0qRJmThxYn7rt34rL730UmbMmJHp06enoaEhLS0taW9vz65du9Ld3Z3Zs2cnSVpbW9Pe3j7iJwEAAMkg4vbd7373QKy+9NJL+Yd/+IfU1dWlUqkMjKlWq+no6Mju3bsPO16pVNLR0TEC0wYAgCM1DHbgf/zHf+SGG27In/zJn6S+vj4vvfTSwPdqtVrq6urS39+furq6I44PxRlnnDqk8eNJpdI02lPgGKwRjG/j4TFgPJzjycz6DDJut23blltuuSUrVqzIwoUL8+yzz6azs3Pg+52dnalWq5k6dephx/fs2ZNqtTqkCe3d25X+/tqQbjMeVCpN6ezcP9rT4ChGe408oMHoK/1xerQf5zi68bI+EybUHfXF0GNuS3j11Vdz8803Z+3atVm4cGGS5JxzzsmLL76YnTt3pq+vL5s3b05zc3OmTZuWxsbGbNu2LUnS1taW5ubmYToVAAA4umO+cvvXf/3X6enpyZo1awaOLVmyJGvWrMmyZcvS09OTuXPnZsGCBUmStWvXZtWqVenq6sqsWbOydOnSkZs9AAD8kmPG7apVq7Jq1apf+b2NGzcecWzmzJnZsGHD8c8MAACGyBXKAAAohrgFAKAY4hYAgGKIWwAAiiFuAQAohrgFAKAY4hYAgGIM6vK7wInXdNqUTG50FwWAofDMCWPU5MaGtCxvG/T4TfcuHsHZAMDJwbYEAACKIW4BACiGuAUAoBjiFgCAYohbAACKIW4BACiGuAUAoBjiFgCAYohbAACKIW4BACiGuAUAoBjiFgCAYohbAACKIW4BACiGuAUAoBgNoz0BACjBwUN9qVSaBj2+u6c3+/cdGMEZwfgkbgFgGEyaWJ+W5W2DHr/p3sXZP4LzgfHKtgQAAIohbgEAKIa4BQCgGOIWAIBieEMZnCBNp03J5EZ3OQAYSZ5p4QSZ3Ngw5HdSA+Xy0WEwMsQtAIwCHx0GI8OeWwAAiiFuAQAohrgFAKAY4hYAgGKIWwAAiiFuAQAohrgFAKAY4hYAgGKIWwAAiiFuAQAohrgFAKAY4hYAgGKIWwAAitEw2hOAk1XTaVMyufHwu1Cl0jRKswEAEnELb9vkxoa0LG8b9PhN9y4ewdkAAIltCQAAFETcAgBQDHELAEAxxC0AAMUQtwAAFEPcAgBQDHELAEAxxC0AAMUQtwAAFEPcAgBQDHELAEAxxC0AAMUQtwAAFKNhtCcAABzbwUN9qVSaBj2+u6c3+/cdGMEZwdg0qLjt6urKkiVL8pd/+Zc5++yzs3Xr1nz2s59NT09PLrnkktx2221Jkh07dmTlypV5/fXXc/7552f16tVpaNDPAHC8Jk2sT8vytkGP33Tv4uwfwfnAWHXMbQnPP/98rrjiirz00ktJku7u7qxYsSL3339/Hn/88bzwwgvZsmVLkuT222/PnXfemSeeeCK1Wi3r168f0ckDAMAvO2bcrl+/PnfddVeq1WqSZPv27ZkxY0amT5+ehoaGtLS0pL29Pbt27Up3d3dmz56dJGltbU17e/vIzh4AAH7JMfcM3HPPPYd9vXv37lQqlYGvq9VqOjo6jjheqVTS0dExjFMFAICjG/KG2P7+/tTV1Q18XavVUldX95bHh+qMM04d8m3Gi6G8kQAARuJ5w3PR2GZ93kbcTp06NZ2dnQNfd3Z2plqtHnF8z549A1sZhmLv3q7099eGfLvSVSpN6ez01oCxxAMIMNYN9/OG56Kxbbysz4QJdUd9MXTIn3N7zjnn5MUXX8zOnTvT19eXzZs3p7m5OdOmTUtjY2O2bduWJGlra0tzc/PbnzkAAAzRkF+5bWxszJo1a7Js2bL09PRk7ty5WbBgQZJk7dq1WbVqVbq6ujJr1qwsXbp02CcMAABvZdBx++STTw7895w5c7Jx48YjxsycOTMbNmwYnpnBCdZ02pRMbvS5zABwMvNMDv9jcmPDkD8gHQAYW4a85xYAAMYqcQsAQDHELQAAxRC3AAAUQ9wCAFAMcQsAQDF8FBgAFOjgob4hXSa8u6c3+/cdGMEZwYkhbgGgQJMm1g/5s7v3j+B84ESxLQEAgGKIWwAAiiFuAQAohrgFAKAY4hYAgGKIWwAAiiFuAQAohrgFAKAY4hYAgGKIWwAAiuHyu5xUmk6bksmNg/u1dZ10ABh/xC0nlcmNDYO+VrrrpAPA+GNbAgAAxRC3AAAUQ9wCAFAMe24p1sFDfalUmkZ7GgDACSRuKdakifWDfvNZ8os3oAEAJzfbEgAAKIa4BQCgGOIWAIBi2HMLAAz6Tbj/O8ZVIBmrxC0A8LbehOsqkIxFtiUAAFAMcQsAQDHELQAAxRC3AAAUQ9wCAFAMcQsAQDHELQAAxfA5twDAiGs6bUomNw4uO1wgguMhbgGAETe5sWHQF4lwgQiOh20JAAAUwyu3jKqh/DMVAGPHwUN9qVSaRnsacARVwagayj9TJb/4pyoARt+kifUevxmTbEsAAKAY4hYAgGKIWwAAimHPLcPKG8QAgNGkQhhW3iAGAIwm2xIAACiGuAUAoBi2JQAAY8pQLxDR3dOb/fsOjOCMOJmIWwBgTHk7F4jYP4Lz4eQibseZoX6agf83DACcTMTtODPUTzP41ppFrh0OAJw0xC1H5drhAIx19ujyy8QtAHBSs0eXX+ajwAAAKIa4BQCgGOIWAIBi2HMLAIwr3oBWNnELAIwrQ30D2lA/FlMMjy5xe5Ib6kUZAICh8WkMJ5cRqaJNmzblK1/5Snp7e3PNNdfkqquuGokfQ4Z+UQafQwsAlGzY47ajoyPr1q3Lo48+mkmTJmXJkiX53d/93fz2b//2cP+ok8JQX1ntOdiXxkn1v/J7rhQGAHB0wx63W7duzfvf//684x3vSJLMnz8/7e3t+cQnPjGo20+YUDfcUzqmU0+dnMahBGhPb7q6ugc1dnJjQ/7f3d8Z9J/916vmDXl89fQpgx6fZFyNH0tzGW/jx9Jcxtv4sTSXk338WJrLeBs/lubydsYPpWeG3CFDfCFsKN1yMjjW321drVarDecP/OpXv5o33ngjt912W5LkkUceyfbt2/OZz3xmOH8MAAAcYdg/57a/vz91df9X1LVa7bCvAQBgpAx73E6dOjWdnZ0DX3d2dqZarQ73jwEAgCMMe9xecMEFeeaZZ/Laa6/lwIED+c53vpPm5ubh/jEAAHCEYX9D2ZlnnpnbbrstS5cuzaFDh3L55Zfnve9973D/GAAAOMKwv6EMAABGy7BvSwAAgNEibgEAKIa4BQCgGOIWAIBiiNsx4Gtf+1rmz5+flpaWfOUrX0mSPProo7n00kvT0tKSu+++O729vUfcbvfu3bn++utz2WWXZcmSJXnllVdO9NTHhbe7Pq+88kquuuqqLF68OFdffXV27dp1oqdevK6urixatGjgd3/r1q1paWnJvHnzsm7duoFxO3bsSGtra+bPn5+VK1f+yvXat29frr/++lxyySW56qqrDvu8bt6e4Vyfn/zkJwP3p49+9KPZsWPHCTuPkg3nGv2v//qv/8r73vc+z0nDZDjXqKurK8uXL89ll12Wyy67LD/84Q9P2HmcUDVG1b/+67/WFi1aVNu/f3+tt7e3dsMNN9S++tWv1i666KJaR0dHrVar1e66667aAw88cMRtr7nmmtrDDz9cq9VqtYcffrj2yU9+8oTOfTw4nvX51Kc+VfvmN79Zq9VqtQcffLC2fPnyEzr30v3gBz+oLVq0qDZr1qzayy+/XDtw4EBt7ty5tf/8z/+sHTp0qHbttdfWvvvd79ZqtVpt4cKFtX/7t3+r1Wq12p/92Z8NrMsvW716de2rX/1qrVar1R577DH3p+M03OuzZMmS2lNPPVWr1Wq1rVu31lpaWk7YuZRquNeoVqvV+vr6atdee21t9uzZtZdffvmEnUuphnuNVqxYUfvCF75Qq9VqtS1bttQuv/zyE3cyJ5BXbkfZj370o/ze7/1eTj311NTX1+eiiy7Kfffdl9mzZw9c2e3iiy/OP/3TPx12u9deey0//vGPs2TJkiTJhz/84dx6660nfP6le7vrk/ziUtRdXV1JkgMHDmTy5MkndO6lW79+fe66666Bddi+fXtmzJiR6dOnp6GhIS0tLWlvb8+uXbvS3d2d2bNnJ0laW1vT3t5+xJ/33e9+Ny0tLUmSRYsW5V/+5V9y6NChE3dChRnu9fnIRz6Siy66KEnynve8J6+++uqJO5lCDfcaJclf/dVf5YILLsjpp59+ws6jZMO5RrVaLd/5zndy/fXXJ0mam5vzF3/xFyf2hE4QcTvKZs2alaeffjo///nP09PTkyeffDKzZ8/O888/n1dffTV9fX1pb2/Pnj17Drvdyy+/nN/4jd/ImjVr8uEPfzi33HJLJk6cOEpnUa63uz5J8slPfjJ/8zd/k4suuigPPPBAPv7xj4/CGZTrnnvuyfnnnz/w9e7du1OpVAa+rlar6ejoOOJ4pVJJR0fHEX/eL49raGjIqaeemtdee20Ez6Bsw70+ra2tqa+vT5J86dHJP1cAAAQySURBVEtfygc/+MERnP34MNxr9MILL+T73/9+/vAP/3BkJz6ODOca7d27N5MmTcrDDz+cj370o1m6dGn6+vpG/iRGgbgdZXPmzElra2uuvvrqXHfddTnvvPMyZcqULF++PDfddFOuuuqqvOc97zkiXHt7e/OjH/0o73//+/Otb30rH/jAB3LHHXeM0lmU6+2uT5L86Z/+aT796U/ne9/7XlavXp1PfOITqblmyojp7+9PXV3dwNe1Wi11dXVvefxYarVaJkzwEDlchmN9arVaPve5z+X555/PihUrRnzO483xrNGBAweyevXq3H333e43I+h41qivry979uxJU1NT/u7v/i433HBDbr755hM29xPJb+Ao6+rqyrx587Jp06Y89NBDmTRpUs4888y8973vzbe//e387d/+bc4888xMnz79sNtVKpWccsopufjii5P84p9Rt2/fPhqnULS3uz6vvfZafvrTnw68ujR//vx0dnbmZz/72WicxrgwderUw94E1tnZmWq1esTxPXv2DPwT3y+rVqsDr8D39vbm9ddfzzve8Y6Rn/g4cbzr09vbm0996lP593//9zz44INpamo6IfMeT45njZ577rns3bs3N910UxYvXjzwhuef/vSnJ2z+48HxrNHpp5+ehoaGLFq0KEly4YUX5o033sjevXtPzORPIHE7yl555ZX80R/9UXp7e7N///5s2LAhixcvzsc+9rF0dXXl4MGD+cY3vpFLL730sNv95m/+ZqZOnZotW7YkSZ566qnMmjVrNE6haG93fU4//fQ0NjbmueeeS5Js27Ytp5xySt75zneOxmmMC+ecc05efPHF7Ny5M319fdm8eXOam5szbdq0NDY2Ztu2bUmStra2NDc3H3H7uXPn5tvf/naS5PHHH8/5559vq88wOt71+dznPpeurq488MADwnaEHM8aXXTRRXnyySfT1taWtra2VKvVfO1rX8u73vWu0TiVYh3PGk2aNCkXXHBB/v7v/z5J8oMf/CBTpkwpcn90w2hPYLybOXNm5s2blw996EPp6+vLxz72sZx33nm5+eab89GPfjS9vb1ZtGjRwBtdVq5cmT/4gz/IBz7wgXz5y1/OXXfdlS984Qs59dRTs2bNmlE+m/Icz/rcd999+cxnPpPu7u6ccsop+fKXvzzKZ1O2xsbGrFmzJsuWLUtPT0/mzp2bBQsWJEnWrl2bVatWpaurK7NmzcrSpUuTJF/84hdTrVZzxRVX5JOf/GTuuOOOLFy4ME1NTVm7du1onk5xjmd95s+fn29+85s5++yz85GPfGTgz2xraxuVcynV8d6HGHnHu0b33HNP7rzzzjz88MNpaGjIunXritxGUlezCRAAgEKUl+sAAIxb4hYAgGKIWwAAiiFuAQAohrgFAKAY4hYAgGKIWwAAiiFuAQAoxv8HDT/LCJkO33EAAAAASUVORK5CYII=\n",
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAArcAAAHjCAYAAAA5ajcLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dcazVd2H//9eFCxdWbmPbnVMcMhKnkYTE0rRxoutlnQawcKW5rSnSFV2/2topaofdOiA01bqh0hBrV6PZqmm1cYhtL7Du1m2tzA6dKZllVUwWbVnLusuF6uBW7oV77/n94XZ/pdRyT7mHe3nzeCRNuJ/7Ptz3h/c95zxz+j6f01Sr1WoBAIACTBjrCQAAwGgRtwAAFEPcAgBQDHELAEAxxC0AAMUQtwAAFEPcAgBQjOaxnsBL/fznL2RoyKV3G+W886blwIHesZ4GL2JNxifrMj5Zl/HJuoxPpa7LhAlNOeecs37t98dd3A4N1cRtg/n3HX+syfhkXcYn6zI+WZfx6UxcF9sSAAAohrgFAKAY4hYAgGKIWwAAiiFuAQAohrgFAKAY4hYAgGKIWwAAiiFuAQAohrgFAKAY4hYAgGKIWwAAiiFuAQAohrgFAKAY4hYAgGKIWwAAiiFuAQAohrgFAKAYzWM9AYDTXevZUzOlZeQPp339Azl08HADZwRw5hK3ACdpSktz2ld1jnj81tuX5lAD5wNwJrMtAQCAYohbAACKIW4BACiGuAUAoBjiFgCAYohbAACKIW4BACiGuAUAoBjiFgCAYohbAACKIW4BACiGuAUAoBjiFgCAYohbAACKIW4BACiGuAUAoBjiFgCAYohbAACKIW4BACiGuAUAoBjiFgCAYohbAACKIW4BACiGuAUAoBjiFgCAYohbAACKIW4BACiGuAUAoBjiFgCAYohbAACKIW4BACjGiOL2kUceSUdHR971rnfltttuS5Ls2LEj7e3tWbBgQTZu3Dg8dvfu3eno6MjChQuzZs2aDAwMNGbmAADwEieM22eeeSa33HJL7rrrrmzZsiU//vGPs3379qxevTp33XVXHnrooTz55JPZvn17kuSmm27KunXr8vDDD6dWq2XTpk0NPwkAAEiS5hMN+Id/+IdcdtllmT59epJk48aN2bNnT2bNmpWZM2cmSdrb29PV1ZU3vOEN6evry9y5c5MkHR0dueOOO7J8+fIGngJA2VrPnpopLSd8uE6S9PUP5NDBww2eEcD4dcJHyz179mTSpEn50Ic+lOeeey6///u/nze+8Y2pVCrDY6rVarq7u7Nv375jjlcqlXR3dzdm5gBniCktzWlf1TmisVtvX5pDDZ4PwHh2wrgdHBzM448/nnvvvTe/8Ru/kRtuuCFTpkxJU1PT8JharZampqYMDQ297PF6nHfetLrGU79KpXWsp8BLWJPxqZHrcrr+3eNB6ed3urIu49OZuC4njNvf/M3fzLx583LuuecmSd75znemq6srEydOHB7T09OTarWa6dOnp6enZ/j4/v37U61W65rQgQO9GRqq1XUbRq5SaU1Pj9d1xhNrMj7Vsy6v5smjnjWv9+8v+ffJ/WV8si7jU6nrMmFC0yu+GHrCN5Rdeumleeyxx3Lw4MEMDg7mu9/9bhYtWpSnnnoqe/bsyeDgYLZt25a2trbMmDEjLS0t2blzZ5Kks7MzbW1to3c2AADwCk74yu0FF1yQD3zgA1m+fHmOHj2at7/97Xnve9+b17/+9Vm5cmX6+/szf/78LFq0KEmyYcOGrF27Nr29vZkzZ05WrFjR8JMAAIBkBHGbJFdeeWWuvPLKY47NmzcvW7ZsOW7s7Nmzs3nz5tGZHQAA1MEnlAEAUIyRXTgRgFFz5OjgGfkOZoBTQdwCnGKTJ00c8XVrk19du7ZR6vmAiMSHRADjn7gFOIPV8wERiQ+JAMY/e24BACiGuAUAoBi2JQC8ROvZU5OcmR9bCXC6E7cAL/Fq9qECMD7YlgAAQDHELQAAxRC3AAAUQ9wCAFAMcQsAQDHELQAAxRC3AAAUQ9wCAFAMcQsAQDHELQAAxRC3AAAUQ9wCAFCM5rGeAACj58jRwVQqrWM9DYAxI24BCjJ50sS0r+oc8fitty9t4GwATj3bEgAAKIa4BQCgGOIWAIBiiFsAAIohbgEAKIa4BQCgGOIWAIBiiFsAAIohbgEAKIa4BQCgGOIWAIBiiFsAAIrRPNYTAGi01rOnZkqLhzuAM4FHe6B4U1qa076qc8Tjt96+tIGzAaCRbEsAAKAY4hYAgGKIWwAAiiFuAQAohrgFAKAY4hYAgGK4FBgAI3bk6GAqldYRj+/rH8ihg4cbOCOAY4lbAEZs8qSJdV8z+FAD5wPwUrYlAABQDHELAEAxxC0AAMUQtwAAFEPcAgBQDHELAEAxxC0AAMUQtwAAFEPcAgBQDHELAEAxxC0AAMUQtwAAFKN5JIOuueaaPP/882lu/tXwT37yk3nhhRfyl3/5l+nv78+73vWu3HjjjUmS3bt3Z82aNXnhhRdy8cUX59Zbbx2+HQAANNIJq7NWq+Xpp5/Oo48+OhypfX19WbRoUe6999689rWvzfXXX5/t27dn/vz5uemmm3Lbbbdl7ty5Wb16dTZt2pTly5c3/EQAAOCE2xJ+9rOfJUmuvfbavPvd787Xvva17Nq1K7NmzcrMmTPT3Nyc9vb2dHV1Ze/evenr68vcuXOTJB0dHenq6mrsGQAAwP86YdwePHgw8+bNy1/91V/lq1/9ar7xjW/kv/7rv1KpVIbHVKvVdHd3Z9++fcccr1Qq6e7ubszMAQDgJU64LeHCCy/MhRdeOPz1lVdemTvuuCMXXXTR8LFarZampqYMDQ2lqanpuOP1OO+8aXWNp36VSutYT4GXsCaUbLR/v91fxifrMj6dietywrh9/PHHc/To0cybNy/Jr4J1xowZ6enpGR7T09OTarWa6dOnH3N8//79qVardU3owIHeDA3V6roNI1eptKan59BYT4MXsSaNdyY+uI8no/n77f4yPlmX8anUdZkwoekVXww94baEQ4cO5bOf/Wz6+/vT29ubBx54IH/yJ3+Sp556Knv27Mng4GC2bduWtra2zJgxIy0tLdm5c2eSpLOzM21tbaN3NgAA8ApO+MrtpZdemieeeCKXX355hoaGsnz58lx44YVZv359Vq5cmf7+/syfPz+LFi1KkmzYsCFr165Nb29v5syZkxUrVjT8JAAAIBnhdW4//vGP5+Mf//gxx+bNm5ctW7YcN3b27NnZvHnz6MwOAADq4BPKAAAohrgFAKAY4hYAgGKIWwAAiiFuAQAohrgFAKAY4hYAgGKIWwAAiiFuAQAohrgFAKAY4hYAgGKIWwAAiiFuAQAohrgFAKAY4hYAgGI0j/UEAOrVevbUTGnx8AXA8Tw7AKedKS3NaV/VOeLxW29f2sDZADCe2JYAAEAxvHILQMMcOTqYSqV1xOP7+gdy6ODhBs4IKJ24BaBhJk+aWPcWkkMNnA9QPtsSAAAohrgFAKAY4hYAgGKIWwAAiiFuAQAohrgFAKAY4hYAgGKIWwAAiuFDHAAYN0byiWYv/r5PNANeStwCMG74RDPgZNmWAABAMcQtAADFsC0BGHOtZ0/NlBYPRwCcPM8mwJib0tJc9z5LAHg5tiUAAFAMcQsAQDHELQAAxRC3AAAUQ9wCAFAMcQsAQDHELQAAxRC3AAAUQ9wCAFAMcQsAQDHELQAAxRC3AAAUQ9wCAFAMcQsAQDHELQAAxRC3AAAUQ9wCAFAMcQsAQDHELQAAxRC3AAAUQ9wCAFAMcQsAQDHELQAAxRhx3H7mM5/JzTffnCTZsWNH2tvbs2DBgmzcuHF4zO7du9PR0ZGFCxdmzZo1GRgYGP0ZAwDArzGiuP3e976XBx54IEnS19eX1atX56677spDDz2UJ598Mtu3b0+S3HTTTVm3bl0efvjh1Gq1bNq0qXEzBwCAl2g+0YBf/OIX2bhxYz70oQ/lJz/5SXbt2pVZs2Zl5syZSZL29vZ0dXXlDW94Q/r6+jJ37twkSUdHR+64444sX768sWcAjDutZ0/NlJYTPrwAwKg74bPPunXrcuONN+a5555Lkuzbty+VSmX4+9VqNd3d3ccdr1Qq6e7urntC5503re7bUJ9KpXWsp8BLlLgm7as6Rzx26+1LGzgTSlfi/ed0ZB3GpzNxXV4xbr/5zW/mta99bebNm5f7778/STI0NJSmpqbhMbVaLU1NTb/2eL0OHOjN0FCt7tsxMpVKa3p6Do31NHiREtfkTHwwZeyUdv85HZX4OFaCUtdlwoSmV3wx9BXj9qGHHkpPT0+WLl2a//mf/8kvf/nL7N27NxMnThwe09PTk2q1munTp6enp2f4+P79+1OtVkfhFAAAYGReMW6/8pWvDP/5/vvvzw9+8IPceuutWbBgQfbs2ZPXve512bZtW6644orMmDEjLS0t2blzZy666KJ0dnamra2t4ScAAAD/p+53fLS0tGT9+vVZuXJl+vv7M3/+/CxatChJsmHDhqxduza9vb2ZM2dOVqxYMeoTBgCAX2fEcdvR0ZGOjo4kybx587Jly5bjxsyePTubN28evdkBAEAdfEIZAADFELcAABRD3AIAUAxxCwBAMcQtAADFELcAABRD3AIAUAxxCwBAMcQtAADFELcAABRD3AIAUAxxCwBAMcQtAADFELcAABRD3AIAUAxxCwBAMcQtAADFELcAABSjeawnAACv1pGjg6lUWkc8vq9/IIcOHm7gjICxJm4BOG1NnjQx7as6Rzx+6+1Lc6iB8wHGnm0JAAAUQ9wCAFAMcQsAQDHELQAAxRC3AAAUQ9wCAFAMcQsAQDHELQAAxRC3AAAUQ9wCAFAMcQsAQDHELQAAxRC3AAAUQ9wCAFAMcQsAQDHELQAAxRC3AAAUo3msJwCcHlrPnpopLR4yABjfPFMBIzKlpTntqzpHNHbr7UsbPBsAeHm2JQAAUAxxCwBAMcQtAADFELcAABRD3AIAUAxxCwBAMcQtAADFELcAABTDhzgAcMY4cnQwlUrriMb29Q/k0MHDDZ4RMNrELQBnjMmTJtb1SXuHGjwfYPTZlgAAQDHELQAAxRC3AAAUQ9wCAFAMcQsAQDHELQAAxRC3AAAUY0Rx+/nPfz6XXXZZFi9enK985StJkh07dqS9vT0LFizIxo0bh8fu3r07HR0dWbhwYdasWZOBgYHGzBwAAF7ihHH7gx/8IN///vezZcuWfOtb38q9996bn/zkJ1m9enXuuuuuPPTQQ3nyySezffv2JMlNN92UdevW5eGHH06tVsumTZsafhIAAJCMIG7f8pa35J577klzc3MOHDiQwcHBHDx4MLNmzcrMmTPT3Nyc9vb2dHV1Ze/evenr68vcuXOTJB0dHenq6mr4SQAAQDLCj9+dNGlS7rjjjtx9991ZtGhR9u3bl0qlMvz9arWa7u7u445XKpV0d3fXNaHzzptW13jqN9LPVefUsSYwPrlvjpx/q/HpTFyXEcVtknz0ox/NBz/4wXzoQx/K008/naampuHv1Wq1NDU1ZWho6GWP1+PAgd4MDdXqug0jV6m0pqfHp6WPJ6fLmpyJD5BwOtw3x4PT5XHsTFPqukyY0PSKL4aecFvCT3/60+zevTtJMnXq1CxYsCD/+q//mp6enuExPT09qVarmT59+jHH9+/fn2q1ejLzBwCAETth3D777LNZu3Ztjhw5kiNHjuSf/umfsmzZsjz11FPZs2dPBgcHs23btrS1tWXGjBlpaWnJzp07kySdnZ1pa2tr+EkAAEAygm0J8+fPz65du3L55Zdn4sSJWbBgQRYvXpxzzz03K1euTH9/f+bPn59FixYlSTZs2JC1a9emt7c3c+bMyYoVKxp+EgAAkIxwz+3KlSuzcuXKY47NmzcvW7ZsOW7s7Nmzs3nz5tGZHQAA1MEnlAEAUAxxCwBAMcQtAADFELcAABRD3AIAUAxxCwBAMcQtAADFELcAABRD3AIAUAxxCwBAMcQtAADFELcAABRD3AIAUAxxCwBAMcQtAADFELcAABRD3AIAUAxxCwBAMcQtAADFELcAABRD3AIAUAxxCwBAMcQtAADFELcAABRD3AIAUAxxCwBAMZrHegIAMB4dOTqYSqV1xOP7+gdy6ODhBs4IGAlxCwAvY/KkiWlf1Tni8VtvX5pDDZwPMDLiFs5QrWdPzZQWDwEAlMUzG5yhprQ01/2qFACMd95QBgBAMcQtAADFELcAABRD3AIAUAxxCwBAMcQtAADFELcAABRD3AIAUAxxCwBAMcQtAADFELcAABRD3AIAUIzmsZ4AAJTgyNHBVCqtIx7f1z+QQwcPN3BGcGYStwAwCiZPmpj2VZ0jHr/19qU51MD5wJnKtgQAAIohbgEAKIa4BQCgGOIWAIBiiFsAAIohbgEAKIa4BQCgGOIWAIBiiFsAAIohbgEAKIa4BQCgGOIWAIBijChu77zzzixevDiLFy/OZz/72STJjh070t7engULFmTjxo3DY3fv3p2Ojo4sXLgwa9asycDAQGNmDgAAL3HCuN2xY0cee+yxPPDAA3nwwQfzox/9KNu2bcvq1atz11135aGHHsqTTz6Z7du3J0luuummrFu3Lg8//HBqtVo2bdrU8JMAAIBkBHFbqVRy8803Z/LkyZk0aVJ+53d+J08//XRmzZqVmTNnprm5Oe3t7enq6srevXvT19eXuXPnJkk6OjrS1dXV8JMAAIAkaT7RgDe+8Y3Df3766afz93//9/nDP/zDVCqV4ePVajXd3d3Zt2/fMccrlUq6u7tHecrAy2k9e2qmtJzwLg0ARRvxM+F//Md/5Prrr8+f/umfZuLEiXn66aeHv1er1dLU1JShoaE0NTUdd7we5503ra7x1K9SaR3rKfASo7Um7as6Rzx26+1LR+VnAq9eSY/HJZ1LSc7EdRlR3O7cuTMf/ehHs3r16ixevDg/+MEP0tPTM/z9np6eVKvVTJ8+/Zjj+/fvT7VarWtCBw70ZmioVtdtGLlKpTU9PYfGehq8yGityZn4AAanu1Iejz23jE+lrsuECU2v+GLoCffcPvfcc/nwhz+cDRs2ZPHixUmSCy64IE899VT27NmTwcHBbNu2LW1tbZkxY0ZaWlqyc+fOJElnZ2fa2tpG6VQAAOCVnfCV27/5m79Jf39/1q9fP3xs2bJlWb9+fVauXJn+/v7Mnz8/ixYtSpJs2LAha9euTW9vb+bMmZMVK1Y0bvYAAPAiJ4zbtWvXZu3atS/7vS1bthx3bPbs2dm8efPJzwwAAOrkE8oAACiGuAUAoBjiFgCAYohbAACKIW4BACiGuAUAoBjiFgCAYohbAACKccIPcQAARt+Ro4OpVFpHPL6vfyCHDh5u4IygDOIWAMbA5EkT076qc8Tjt96+NIcaOB8ohW0JAAAUQ9wCAFAMcQsAQDHELQAAxRC3AAAUQ9wCAFAMcQsAQDHELQAAxRC3AAAUQ9wCAFAMcQsAQDHELQAAxRC3AAAUQ9wCAFAMcQsAQDGax3oCwMtrPXtqprS4iwJAPTxzwjg1paU57as6Rzx+6+1LGzgbADg92JYAAEAxxC0AAMUQtwAAFEPcAgBQDHELAEAxxC0AAMUQtwAAFEPcAgBQDHELAEAxxC0AAMUQtwAAFEPcAgBQDHELAEAxxC0AAMUQtwAAFEPcAgBQDHELAEAxxC0AAMUQtwAAFKN5rCcAAJzYkaODqVRaRzy+r38ghw4ebuCMYHwStwBwGpg8aWLaV3WOePzW25fmUAPnA+OVbQkAABRD3AIAUAxxCwBAMcQtAADFELcAABRD3AIAUAyXAoNTpPXsqZnS8vJ3uXquXQkA/HriFk6RKS3NdV+jEgCoz4i2JfT29mbJkiV59tlnkyQ7duxIe3t7FixYkI0bNw6P2717dzo6OrJw4cKsWbMmAwMDjZk1APCK/u8TzUb6X+vZU8d6yjAqTvjK7RNPPJG1a9fm6aefTpL09fVl9erVuffee/Pa1742119/fbZv35758+fnpptuym233Za5c+dm9erV2bRpU5YvX97ocwAAXsInmnGmOuErt5s2bcott9ySarWaJNm1a1dmzZqVmTNnprm5Oe3t7enq6srevXvT19eXuXPnJkk6OjrS1dXV2NkDAMCLnPCV209/+tPHfL1v375UKpXhr6vVarq7u487XqlU0t3dPYpTBQCAV1b3G8qGhobS1NQ0/HWtVktTU9OvPV6v886bVvdtqI935gPwck7m+cFzy/h0Jq5L3XE7ffr09PT0DH/d09OTarV63PH9+/cPb2Wox4EDvRkaqtV9O0amUmlNT49dVWPhTHyAAU4vr/b5wXPL+FTqukyY0PSKL4bW/SEOF1xwQZ566qns2bMng4OD2bZtW9ra2jJjxoy0tLRk586dSZLOzs60tbW9+pkDAECd6n7ltqWlJevXr8/KlSvT39+f+fPnZ9GiRUmSDRs2ZO3atent7c2cOXOyYsWKUZ8wAAD8OiOO20ceeWT4z/PmzcuWLVuOGzN79uxs3rx5dGYGAAB1qntbAgAAjFfiFgCAYohbAACKUfcbygCA8hw5OljXJQv7+gdy6ODhBs4IXh1xCwBk8qSJaV/VOeLxW29fmvKuoEoJbEsAAKAY4hYAgGLYlgCvUuvZUzOlxV0IAMYTz8zwKk1paa57fxoA0Fi2JQAAUAxxCwBAMcQtAADFELcAABTDG8rgf7n6AQCc/jyTw/9y9QMAOP3ZlgAAQDHELQAAxRC3AAAUQ9wCAFAMcQsAQDHELQAAxRC3AAAUQ9wCAFAMcQsAQDHELQAAxRC3AAAUQ9wCAFAMcQsAQDHELQAAxWge6wlAo7SePTVTWvyKA8CZxDM/xZrS0pz2VZ0jHr/19qUNnA0AcCrYlgAAQDHELQAAxRC3AAAUw55bAKBuR44OplJpHf76xX9+OX39Azl08HCjpwXiFgCo3+RJE+t+0+6hBs4H/o9tCQAAFEPcAgBQDHELAEAxxC0AAMUQtwAAFMPVEjittJ49NVNa/NoCnG5eeumwV+KyYZwMlcBpZUpL84gvPbP19qUNng0AI1XPpcNcNoyTYVsCAADFELcAABRD3AIAUAxxCwBAMbyhjDHl6gcAwGhSFYypeq5+kLgCAgDwysQtADCu1HNN3MR1cTmWuGVU2WYAwMmq55q4SfKt9UvEMMNUCKPKNgMATrV6Y9iHRJRN3AIAZxTbHsombgGAM4pXesvmOrcAABRD3AIAUAzbEnhFrn4AAJxOGlItW7duzRe/+MUMDAzkfe97X66++upG/BhOAVc/AABOJ6Met93d3dm4cWPuv//+TJ48OcuWLcvv/u7v5g1veMNo/6jTQr2vfPYfGUzL5IkNG3/k6OCIxwIA9V9dodHP5fWMr1Raz7irPYx63O7YsSNvfetb85rXvCZJsnDhwnR1deUjH/nIiG4/YULTaE9pTE1pac7/u+3bIx7/N2sXNHT8F//sHXXdQZOkes7U03b8eJrLmTZ+PM3lTBs/nuZyuo8fT3M508aPp7lMnjRxXD2Xv5rxL4ywr6ZNm5KWel6U6x9Ib2/fiMePhhO1YlOtVquN5g/80pe+lF/+8pe58cYbkyTf/OY3s2vXrnzqU58azR8DAADHGfWrJQwNDaWp6f8v6lqtdszXAADQKKMet9OnT09PT8/w1z09PalWq6P9YwAA4DijHrdve9vb8r3vfS/PP/98Dh8+nG9/+9tpa2sb7R8DAADHGfU3lJ1//vm58cYbs2LFihw9ejRXXnll3vzmN4/2jwEAgOOM+hvKAABgrPj4XQAAiiFuAQAohrgFAKAY4hYAgGKI20J8+ctfzsKFC9Pe3p4vfvGLSZL7778/l112Wdrb23PbbbdlYGDguNvt27cv1113XS6//PIsW7Yszz777KmeetFe7bo8++yzufrqq7N06dJcc8012bt376meepF6e3uzZMmS4d/zHTt2pL29PQsWLMjGjRuHx+3evTsdHR1ZuHBh1qxZ87JrdPDgwVx33XV517velauvvvqY63szcqO5Jj/96U+H7zdXXXVVdu/efcrOozSjuS7/57//+7/zlre8xfPMSRjNdent7c2qVaty+eWX5/LLL8+PfvSjU3YeDVfjtPcv//IvtSVLltQOHTpUGxgYqF1//fW1L33pS7VLLrmk1t3dXavVarVbbrmldvfddx932/e97321++67r1ar1Wr33Xdf7WMf+9gpnXvJTmZdPvGJT9S+/vWv12q1Wu2ee+6prVq16pTOvUQ//OEPa0uWLKnNmTOn9swzz9QOHz5cmz9/fu0///M/a0ePHq1de+21te985zu1Wq1WW7x4ce3f/u3farVarfbnf/7nw2vxYrfeemvtS1/6Uq1Wq9UeeOAB951XYbTXZNmyZbVHH320VqvVajt27Ki1t7efsnMpyWivS61Wqw0ODtauvfba2ty5c2vPPPPMKTuXkoz2uqxevbr2uc99rlar1Wrbt2+vXXnllafuZBrMK7cF+PGPf5zf+73fy7Rp0zJx4sRccsklufPOOzN37tzhT4e79NJL84//+I/H3O7555/PT37ykyxbtixJcsUVV+TjH//4KZ9/qV7tuiS/+hjr3t7eJMnhw4czZcqUUzr3Em3atCm33HLL8L/9rl27MmvWrMycOTPNzc1pb29PV1dX9u7dm76+vsydOzdJ0tHRka6uruP+vu985ztpb29PkixZsiT//M//nKNHj566EyrAaK/Je97znlxyySVJkje96U157rnnTt3JFGS01yVJ/vqv/zpve9vbcs4555yy8yjNaK5LrVbLt7/97Vx33XVJkra2tvzFX/zFqT2hBhK3BZgzZ04ee+yx/OIXv0h/f38eeeSRzJ07N0888USee+65DA4OpqurK/v37z/mds8880x+67d+K+vXr88VV1yRj370o5k0adIYnUV5Xu26JMnHPvaxfPWrX80ll1ySu+++Ox/84AfH4AzK8ulPfzoXX3zx8Nf79u1LpVIZ/rparaa7u/u445VKJd3d3cf9fS8e19zcnGnTpuX5559v4BmUZ7TXpKOjIxMnTkyS3HHHHXnnO000o9kAAAQpSURBVN/ZwNmXa7TX5cknn8z3v//9/NEf/VFjJ1640VyXAwcOZPLkybnvvvty1VVXZcWKFRkcHGz8SZwi4rYA8+bNS0dHR6655pp84AMfyEUXXZSpU6dm1apVueGGG3L11VfnTW9603HhOjAwkB//+Md561vfmm9961t5xzvekZtvvnmMzqI8r3ZdkuTP/uzP8slPfjLf/e53c+utt+YjH/lIaj5vZVQNDQ2lqalp+OtarZampqZfe/xEarVaJkzwkHoyRmNNarVaPvOZz+SJJ57I6tWrGz7nM8HJrMvhw4dz66235rbbbnP/GGUnsy6Dg4PZv39/Wltb87d/+7e5/vrr8+EPf/iUzb3R/KYVoLe3NwsWLMjWrVtz7733ZvLkyTn//PPz5je/OQ8++GC+8Y1v5Pzzz8/MmTOPuV2lUslZZ52VSy+9NMmv/tfqrl27xuIUivRq1+X555/Pz372s+FXnRYuXJienp78/Oc/H4vTKNb06dOPeRNYT09PqtXqccf3798//L8BX6xarQ6/6j4wMJAXXnghr3nNaxo/8YKd7JoMDAzkE5/4RP793/8999xzT1pbW0/JvEt3Muvy+OOP58CBA7nhhhuydOnS4Tcx/+xnPztl8y/VyazLOeeck+bm5ixZsiRJ8va3vz2//OUvc+DAgVMz+QYTtwV49tln88d//McZGBjIoUOHsnnz5ixdujTvf//709vbmyNHjuRrX/taLrvssmNu99u//duZPn16tm/fniR59NFHM2fOnLE4hSK92nU555xz0tLSkscffzxJsnPnzpx11lk599xzx+I0inXBBRfkqaeeyp49ezI4OJht27alra0tM2bMSEtLS3bu3Jkk6ezsTFtb23G3nz9/fh588MEkyUMPPZSLL77Ytp6TdLJr8pnPfCa9vb25++67he0oOpl1ueSSS/LII4+ks7MznZ2dqVar+fKXv5zXv/71Y3EqRTmZdZk8eXLe9ra35e/+7u+SJD/84Q8zderUYvZEN4/1BDh5s2fPzoIFC/Lud787g4ODef/735+LLrooH/7wh3PVVVdlYGAgS5YsGX7zy5o1a/IHf/AHecc73pEvfOELueWWW/K5z30u06ZNy/r168f4bMpxMuty55135lOf+lT6+vpy1lln5Qtf+MIYn015Wlpasn79+qxcuTL9/f2ZP39+Fi1alCTZsGFD1q5dm97e3syZMycrVqxIknz+859PtVrNe9/73nzsYx/LzTffnMWLF6e1tTUbNmwYy9MpwsmsycKFC/P1r389r3vd6/Ke97xn+O/s7Owck3MpycneV2iMk12XT3/601m3bl3uu+++NDc3Z+PGjcVsHWmq2cgHAEAhykh0AACIuAUAoCDiFgCAYohbAACKIW4BACiGuAUAoBjiFgCAYohbAACK8f8B30e+LYviLEgAAAAASUVORK5CYII=\n",
       "text/plain": [
        "
    "
       ]
diff --git a/doc/src/LectureNotes/_build/jupyter_execute/chapter4_278_2.png b/doc/src/LectureNotes/_build/jupyter_execute/chapter4_278_2.png
index a56a44f78..2abb9e89d 100644
Binary files a/doc/src/LectureNotes/_build/jupyter_execute/chapter4_278_2.png and b/doc/src/LectureNotes/_build/jupyter_execute/chapter4_278_2.png differ
diff --git a/doc/src/LectureNotes/_build/jupyter_execute/chapter6.ipynb b/doc/src/LectureNotes/_build/jupyter_execute/chapter6.ipynb
index f96ba6d3f..7d268f8b4 100644
--- a/doc/src/LectureNotes/_build/jupyter_execute/chapter6.ipynb
+++ b/doc/src/LectureNotes/_build/jupyter_execute/chapter6.ipynb
@@ -3116,7 +3116,44 @@
    "metadata": {
     "collapsed": false
    },
-   "outputs": [],
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      ":4: RuntimeWarning: overflow encountered in exp\n",
+      "  return 1/(1 + np.exp(-x))\n"
+     ]
+    },
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkIAAAJgCAYAAABmwww9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3gU1dvG8e/upkNCKCn0ojSF0AQB6UiLtCgoFkBQFGmKoqCCYkVBRQRFUBBQ+qtUEREQVIp0QaQL0kNCC6Rnd98/kt/CkEBQyC7J3J/r2gtm9uzOefKcJCfPnJm1OJ1OJyIiIiImZPV0B0REREQ8RRMhERERMS1NhERERMS0NBESERER09JESERERExLEyERERExLS9Pd0DEbN5++202btwIwIEDByhevDh+fn4AzJ492/X/7PTq1YvBgwdz++23X7XNmDFjKF26NB07drzxjmfo378/GzZsYNWqVfj7+9+09xUR8QSL7iMk4jnNmjVjzJgxVK1a1dNduS7R0dF06NCBatWq0aRJEx5++GFPd0lE5Ibo1JjILWTs2LE88cQTtGvXjkGDBhEbG0ufPn146KGHaNasGV27duX06dNA+iRqx44d/P7773Tp0oUXX3yRjh070rZtWzZv3gzAkCFDmDRpEgBVq1Zl7NixdOnShWbNmjFjxgwA7HY7I0aMoEWLFtx///0MHz6crl27Ztm/OXPmUK9ePaKiopg2bRqX/x31xx9/0LlzZ9q2bUtUVBTr1q275v6KFSty5swZ1+v/t/3777/Tvn17unTpQrt27UhJSeHtt9+mc+fOREZG0qZNG1d88fHxvPzyy7Rq1YrIyEg++ugjEhISqF27NgcPHnS99+OPP87y5ctvSo5EJG/RREjkFnPs2DHmzZvHBx98wPfff0/16tWZPXs2K1aswM/PjwULFmR6zfbt2+nZsyfz58/n/vvvZ/To0ZnapKSkULBgQWbNmsUnn3zCiBEjSE5OZu7cuezcuZPFixcza9Ysjhw5kmW/0tLSmDNnDu3bt6dZs2acPn2aX375BYDU1FT69u1L3759Wbx4MW+99RbvvvsuKSkpWe53OBzX/Brs27ePDz/8kEWLFrFz505OnTrF7NmzWbJkCVFRUXzxxRcAfPLJJyQnJ7NkyRLmz5/Pli1b2LFjBx07dmTu3LkAHD58mEOHDtG0adN/lQcRMQetERK5xVSvXh0vr/Rvze7du7Np0ya++uorDh06xL59+6hWrVqm1xQrVozKlSsDcMcddzBv3rws37t58+YA3HnnnaSkpJCQkMDq1avp0KEDvr6+ADz00EN8/fXXmV67YsUKHA4HDRs2xMvLi8jISKZNm0bjxo3Zu3cvVquVJk2aAFClShXXJCar/dkpWrQoxYsXB6BGjRoUKFDANUn7/fffyZcvHwBr167l5ZdfxmazYbPZ+OabbwAIDQ3lscceY+DAgcyePZtOnTphs9myPa6ImI8mQiK3mICAANf/R40axfbt23nggQe4++67SUtLI6tlfZcvsLZYLFm2AVyTHYvFAoDT6XRNuv7Has26UDxjxgySkpJo2bIlkF5hiomJYd++fdhsNtd7/s/evXuvur9cuXKGfSkpKYbty78Gq1at4p133qFHjx40b96ccuXKsXDhQgC8vLwM73/ixAn8/PwoW7YsFStWZMWKFSxevJg5c+ZkGZOIiE6NidzCfvvtN7p3707Hjh0pXLgwa9euxW6339RjNG7cmIULF5KSkkJaWlqW1aSDBw+yceNGvvvuO1auXMnKlSv57bffqF27NtOmTaNcuXJYLBbWrFkDwM6dO+nevftV9zscDgoVKsSOHTsAWLx48VX7t2bNGpo2bcojjzxClSpVWL58uetrUK9ePebNm4fD4SAlJYUBAwa4rsh75JFHGDlyJBEREYSFhd3Ur5mI5B2aCIncwvr27cvIkSNp164dzzzzDDVr1uTw4cM39Rj3338/ERERdOzYkS5duuDt7Z3psviZM2dy7733Urp06Uz9W7BgARcvXmTs2LGMGzeODh068PrrrzN27Fh8fHyuun/o0KG8+eabREVFceDAAUJCQrLsX5cuXdiwYQPt2rUjKiqKkiVLcvToURwOB/369cPb25sOHTrQsWNHGjdu7KpYNW3alISEBLp06XJTv14ikrfo8nkRk/vtt984ffo0HTp0ANLvc+Tr68uLL77o4Z7dmK1btzJ06FAWL16c6fSciMj/aCIkYnLR0dEMGTKE2NhYHA4HlSpVYvjw4QQGBnq6a//Z4MGD2bBhA6NHj6Z69eqe7o6I3MI0ERIRERHT0hohERERMS1NhERERMS0NBESERER08qVN1RsHdTD013wGEtQ7l3AejM4L1z0dBc8yuLn6+kuiKf4mjv3jsIFPN0Fj/px6xtuPZ7jZAW3Hcsavtdtx8ry+B49uoiIiIgH5cqKkIiIiOQcB9f+YOSbydMVGU8fX0RERMRjVBESERERA7vTfRUhT09EVBESERER09JESEREREzL0xUpERERucU4MM+nb6kiJCIiIqalipCIiIgYuPPyeU9TRUhERERMSxUhERERMbA7tUZIREREJM9TRUhEREQMdNWYiIiIiAmoIiQiIiIGdlWERERERPI+VYRERETEQGuERERERExAFSEREREx0H2ERERERExAFSERERExMM8njakiJCIiIiamiZCIiIiYlk6NiYiIiIFuqCgiIiJiAqoIiYiIiIHdPAUhVYRERETEvFQREhEREQNdPi8iIiJiAqoIiYiIiIEdi6e74DaqCImIiIhpqSIkIiIiBg5dNSYiIiKS96kilKFOqwh6vN4Jb18vDv55lNH9JpNwISnLti98/iSHdh7l27FLAQgI8mfguB6UrFAUi9XC8hlrmfvxEnd2/4bVbnYHPV5uh7ePFwd3HefjQTNJuHiV+Ec/yqHdJ/h2wkrXvvu6NaD1w/Xw8fNm/44jfDxoJqkpae7q/g2r0zKCHq8/gLevNwd3HmF0v6+unv/xT3Dor6N8O/ZH4LL8lw/HYrWyfOYa5n78gzu7f8Nq31uFHq92wNvHm4N/HeXjgd9cPf+fdOfQrmN8O365a9+sv0YRe+Ksa/vbz37i52835ni/bxYzx1+72R30GNw2/Xt/93E+fnEmCReTs2z7wkePpH/vT/wZgFc/f5yipUNcz4eXLMSO3w/wxhNfuqXvN0OdBuXp0f/e9Pj3RTP6jQUkxBvjbxYZQefu9+B0OklOSuWzkT+w76/j+Ph60W/IfVSsUhyLxcLuHUcZ9973pCTnnp99V6M1QiZToHAgz3/2BG91/ZQna73CiUMx9Hijc6Z2JSsU5b1FL9Gww12G/d2HRhF7/Cy96w5jQJM3aftEUyrXuc1d3b9hBQrl5/mPHuXtpybTq/E7nDx8mh4vt8vUruTtYYyY3Y8G91U37K/fJoL2PRrx8sOf0rvZCHz8vOnYq4mben/j0vPfMz3/d2Xkf3inTO3S8/9i5vy/GkXssbP0rvcaA5q+SdueTalcOxflv3B+nh/Tjbd7TqTXPcM5+U8sPYZ2zNSuZPlwRnz7HA3a1TDsL35bGBfOxdOv+buuR26ZBIC54y9QKB/Pf/Awbz89mV5N303/3h9yle/9mX1oEFnNsP+d3lPo12YU/dqM4pPBs7gYl8inQ//PXd2/YQUKBvDCGx1568XZPBk1lpNHz9JzwL2GNiVKF+bJ51ryat+v6dPlc2Z8+QuvffAQAA8/0Qibl5XeD46n94Of4ePnTZeeDT0RitwAt02EDhw4wGeffcZrr73G8OHD+eyzz9ixY4e7Dn9NNZvfyd4tBzl+IBqA7yetpFnnupnatXuqOUun/cKv840/5Ma/NIMvXp0NQKHwYLx9vYg/n5jzHb9JajauxN4/DnP8YAwAi6f9RtOouzK1a/t4Q36cuY5fF28z7G/+QB2+m/gzF88l4HQ6GTdkDitzyS8CgJrNMvL/9ykAvp/0c9b579WMpVOzyP/gGXwx9Ir8x+Wi/DepzN6thy7lf+ovNH2gTqZ2bXs05sfpa/h14RbD/jtql8NhdzBqwQt89vOrPPJ8JFZr7vlr0szx12yU8b1/KBaAxV+voWnHWpnate3WgB9nrefX7//I8n28vG28MPpRJr4xj9gT53K0zzdTzbq3sWfncY4fPgPA4rkbadYmwtAmNcXOx28u4EzsRQD27jxOwSL58fKysWPLP8z44hecTicOh5MDu08QWjTY7XHkBDsWtz08zS2nxqZPn86cOXNo1aoVVatWBSAmJoZhw4bRvn17evbs6Y5uXFVI8ULEHD3j2o45dpZ8BQIICPQznB75bNA3ANRqdmem93DYHbz0xVM06HAXaxdv5ui+Eznf8ZukSLFgYo5fKuvHnjhHviB/AvL7GU4PjM/4S69mo0qG15coF8rebf/w1jfPUDgsiD9//5tJ7yxwT+dvgpAShYg5dh35f3E6cI38T+yVkf8tuSz/BY35P36V/L+SPtmr2biy4fU2Lyvbft3N5Lfn4+Vl443pfUm4mMT8iSvJDcwcf5FiwcRcNnG59L3vazg9Nv61b4HM3/v/0+qhupyOPs/aH2+NP26vV0h4AWKjz7u2Y07FkS/Qj4B8vq7TY9EnzhF92dfo6RdasX71HtLS7GxZf8C1P7RoAaIercuYtxa5LwC5KdxSEZo2bRqzZs2iT58+dO7cmc6dO9OnTx9mzpzJnDlz3NGFa7JYLTidmZfI2+3/7t6aI3tN5MGy/QksmJ9HhnS4Wd3LcVaLhaw+aPh647d5W6nRqBIjen/FgMgPCAwO4PHBbW9yL3NOev4z7//X+X/qCx4sN4DAgvl4ZHD7m9S7nGe1WrPOv+P64l/6zRrGvzKH5IQU4uMSmff5Cuq3qZb9C28RZo7/6t/7/+6SoY5PNmbmJz/dpF65j9Vy/d/7vn7evDryQYqVLMToNxYanru9clE+nNSThbM28Puve3Oqu27lcFrc9vA0t0yEvLy8SEvLvHgsKSkJb29vd3ThmmKOnqFw0YKu7SLFCnLh7EWSE1Ku6/W1mlehUHh6OTQpPplVc9dze7XSOdLXnHDq+FkKhRVwbRcJL8CFc/EkJ15f/Gei41j7wx8kXEwiLdXOyu82UqlWmRzq7c0Xc+Q0hcMvlbP/ff7vNOb//37PXfk/eoZC4Zflv2gwF87GX3f8zTrVocwdxV3bFgukpeWeG/SbOf707/0g1/a//d4HuO3O4ti8rOxYvz8nupijTp08T+GQQNd2kdBALpxPIDkp1dAuJLwAH099EofDwUtPTSH+skph41ZVeG98NyZ/spxZk391W9/l5nHLRKh379507NiRoUOHMmbMGD755BOGDh1K586d6d27tzu6cE2bV/xJpdrlKHZbGAD39WzKuu+3XvfrG0XV5rGMCpC3jxcN76/DH7/sypG+5oQtq3dTqWZpipVNv/ojsmsD1v2LEvdv32+jYdsa+PilT2rrtY5g7x+Hc6SvOWHzyp3p+S8XCsB9PZuw7vtt137RZRpF1eGxIekVIG8fLxpG1c5l+d9FpVplL+W/e0PWLc16LUhWylQqRteX2mK1WvDx86Zdzyb8smBTDvX25jNz/Ft+2UOlGmUoVqYIAJGP3cO6ZX/+q/eoWvd2/lizLye6l+M2rztApaolKFaqEAD3darNulV7DG38A3wY9cXj/LZiFyOG/J/hirC7G1Wgz0tteLnP1/y8NHedFpRL3LJGqF27dtSpU4d169Zx6tQpHA4Hd911F/379ycsLMwdXbim87EX+KjPZIZO64OXjxcnDp5i1NNfUr5GGZ4b24O+DV6/5usnvjqL/h935/P1bwGwdvEW5n+We8rE509fZPQLM3h1Qk+8vG2c+CeWD577hvIRJXl21MP0azXymq9fPPVX8gcHMHbJi1htFvbvOMqXb853U+9v3KX898XLx8aJgzGM6p2R/08ep2/D4dd8/cRXZ9F/dDc+X/cmAGsXb2X+ZZdW3+rOx15g9LPTeHXSU5fy328K5auV4tmPHqNf83ev+frpH35PnxFdGL9qGDZvG78u2sLSb9a4qfc3zszxnz99kdGDZvDq5z3w8vbixOFYPnhuevr3/vtd6NdmVLbvUaxMEaIvW2OZm5w/G8+Hw+czbNRDeHnZOHH0DKOGzaP8HcUY+Fp7+nT5nPZd7ia0aDD3NKvEPc0urZEa/PRUeg1sBRYLA1+7dCp857YjfPre954I56a6FRYxu4vFmdXimFtc66Aenu6Cx1iCArNvlIc5L1z0dBc8yuLn6+kuiKf4mjv3jsIFsm+Uh/249Q23Hm/L4VJuO1bNUp49g6AbKoqIiIiB3US3GTRPpCIiIiJXUEVIREREDG6Fy9rdRRUhERERMS1VhERERMTATFeNqSIkIiIipqWKkIiIiBjYneapk5gnUhEREZErqCIkIiIiBg4T1UnME6mIiIjIFVQREhEREQNdNSYiIiJiAqoIiYiIiIGuGhMRERG5xSxatIjIyEhatmzJ9OnTMz2/evVq2rVrR7t27XjhhReIj4/P9j01ERIREZFbXnR0NKNHj2bGjBnMnz+f2bNns3//ftfzcXFxDBkyhNGjR7No0SIqVarE6NGjs31fTYRERETEwIHFbY/rtXbtWurWrUtwcDABAQG0atWKpUuXup4/dOgQxYoV4/bbbwegadOmLF++PNv31RohERER8Zi4uDji4uIy7Q8KCiIoKMi1ferUKUJCQlzboaGhbN++3bVdpkwZTp48ye7du6lUqRI//PADsbGx2R5fEyERERExsLvxhNHUqVMZN25cpv39+vWjf//+rm2Hw4HFcqmC5HQ6DdtBQUG8//77DBs2DIfDwYMPPoi3t3e2x9dESERERDyme/fuREVFZdp/eTUIIDw8nE2bNrm2Y2JiCA0NdW3b7XbCw8OZO3cuANu3b6dkyZLZHl9rhERERMTA7rS67REUFESJEiUyPa6cCNWvX59169Zx5swZEhMTWbZsGY0aNXI9b7FY6NmzJ9HR0TidTqZMmUJkZGS2sWoiJCIiIre8sLAwBg4cSLdu3ejYsSNt27YlIiKCXr16sWPHDqxWK2+++SZPPvkkrVu3JigoiCeeeCLb97U4nU6nG/p/U7UO6uHpLniMJSjQ013wKOeFi57ugkdZ/Hw93QXxFF9z595RuICnu+BRP259w63HW/B3dbcdq0O5bW47VlZUERIRERHT0mJpERERMbA79aGrIiIiInmeKkIiIiJi4M77CHmaeSIVERERuYIqQiIiImLgcJqnTmKeSEVERESuoIqQiIiIGGiNkIiIiIgJaCIkIiIipqVTYyIiImKgGyqKiIiImECurAjtHlfJ013wGJ98KZ7ugkelXCzu6S54VKGQC57ugsckp+bKH1c3TfxZf093waOerrPa010wFYeJ6iTmiVRERETkCub+E0tEREQyseuGiiIiIiJ5nypCIiIiYuBAV42JiIiI5HmqCImIiIiB1giJiIiImIAqQiIiImKgD10VERERMQFVhERERMTAoc8aExEREcn7VBESERERA60REhERETEBTYRERETEtHRqTERERAwcuqGiiIiISN6nipCIiIgY2PWhqyIiIiJ5nypCIiIiYqA1QiIiIiImoIqQiIiIGGiNkIiIiIgJqCIkIiIiBlojJCIiImICqgiJiIiIgV0VIREREZG8TxUhERERMXDoqjERERGRvE8VIRERETHQGiERERERE1BFSERERAwcTq0REhEREcnzNBESERER09KpMRERETGwm6hOoolQhqbFy/FSzcb4WG3sPhvD4HU/cDE1xdCmYnAR3qjTgkAfX+wOB6+s/5E/z0S7ni8aEMi8yK60WfQVZ5MT3R3CDWkSfjvPV2mGj9WLPeejeWXzIuLTjPFXCAplWPVWBHr7YXc6eG3L9+w8dxKA39u+wMnEOFfbL/euY9GRP90aw41w5d+Wkf+1V8n/3S0I9PbF7nTwyros8n9fV9oszH35bxBSgf4VW+JttbHvQjRv7phHfFqyoc3tgWG8dEdb8nv54sDJOzsWsCvuOEHe/rxyZ3sqBIWTaE9l4dEtzP5nvYci+W8ahZXnucr34mOzsfd8NMO2LcwUf/nAUF6JiCTQyxc7Tt7Ytoi/zp9gdO0HKZWvkKtd8YBgNsX+Q78NM90dxn/StMQVY3/NVcZ+3SvG/um8MfaPbIpnyzdnsKc6KVjah3v6heITYJwE/LP+IttmnQUL+Oa3Ur9PKEFFvQ1tVr53koBCNuo+FeLO7stNYJ4p3zUU8vVnVP1Inlk1n+YLvuTIxXMMrtnY0MbP5sXX9z7E5zt/577FUxi7Yy1jGrZzPX9/uTuZ0+oRwgMC3d39G1bQJ4ARtdrTf/3/0XrZZxyJP8egKs0NbfxsXkxu8Ahf7l1HxxVf8NmuX/mwThQAZfMX5lxKIh1WfOF65KZJUCFff0bdk5H/+V9y5MJV8t/iIT7/MyP/29cyptEV+W+dO/Mf7BPA8Ij7GbRlJvf/MoZjCWfoX7GloY2f1ZtPaz/O1L9/5ZE1n/HFvp95u3pnAF6oHEmCPYVOv3xC97UTuCekPA1DK3oilP+koE8Ab9foyHMbZ9N2xTiOJpzl+TvuNbTxs3nzRf2uTN63hk6rJ/D5ntW8X+sBAAZunMMDqz7ngVWf8/q2hVxITeLt7d97IpR/zTX2f55P83kZY79WFmO/5UN8vuN37ls0hbF/XDH2b7uTOW0eITxf7hv7SeftrBl7iqYvhXH/p6UIDPdm89enDW3Skh38+vEpmg4Op8PokpSsnY8Nk2INbXbMO0v0rtw1AcyOw2lx28PTNBECGhYry/bTJzl04SwA3+zZSoeydxraNCpWln8unmPVsb8B+OnIfvquXgBAqH9+WpYsT7flc9zb8ZukQVg5dpw9zj8XzwAw8+9NtC9V5Yo2t3Ek/iyrT+4HYMWJvTy7/lsAahQugcPpZEbj7iy89yn6VmqINRfdlTTL/JfLIv8XrpH/Urk3//WKlGfn+WMcSUj/BTD38AbaFKtmaFM35HaOJpxhTcxeAFaf2s2QrbMAqFygGN8f24YDJ2lOO7/F7KV5uPHrdyurH3obf549xuH49PE/6+Am7itR1dgmJH38/3pqHwA/n9zDC5vmGtp4W2y8WzOK9/5cysmkOHKDhsXLsj02m7FfPIuxv+qKsf9T7hz7x7YlUKS8H0HFfACo2DqIv3+5iNPpdLVxOsDphJQEBwBpSQ5s3pd+vp34M5FjWxKo2KqAezsvN41OjQHF8gVyIv7SD64TCRcI8vElv7ePq0RcNqggMYnxvF+vDZULhhCXmsyIzasAOJV4kd6r53ui6zdFUf8gTlx2WutkYhyB3n7k8/JxnR4rk78QMUnxvFOzLZWCw7iQmsTIHSsA8LJYWXvqbz74cyVeVisT6z/MxbRkpu7f4JF4/q1/lf/6GflPuSL/q3Jv/sP8ChCddN61fSrpf/n3dZ0eKp2vMKeTL/Ba1SgqBIVzITWJMbuXAvDnuaPcV7w6f5z9B2+rF83D7iTNafdILP9FUf8ChtO60VnEXyZ/YWKTL/Jm9fZULJAe/4c7fzK8z/2la3Aq6QIrTux2a/9vRLF8gZxIuGzsx2cz9gtljP1Nq4CMsf9z7h378bFpBBS+9GswX2EvUhMcpCY68QlIn+x4+1up1zuEJUOO4htow+mAyBHFAUg4k8aGSbG0GFaUPctyx+T3ejlMVCdxS6THjx+/5sPTLFhwZrHfftlfBV5WG02Ll2Pmvm20XzKNqbs3M6V5J3ysNvd1NIdYLRacWXwFHJfHb7HROPx2Zh/cwgMrJ/H1/o18cc/DeFttzDm0lbf++JFEeyoXUpP5at96WhSr5M4QbojFcp35L1GOmXu30f77jPzfm4fy78z8FbA7Ha7/e1ls3BNage8Ob+SxNeOZdWgdn9TuhrfVxke7fsCJkxkN+vJRrUdZf3o/qY7cMxFK//7PavxfFr/VSsPQ8sz9ZzMPrZ7I9L9/5/O6j+J9Wf673VaPCXt+cUufbxYLFrJI/dXH/uJpTN21mSkt8sbYxwmWLIrXlst+M579J5k/5pyh4yeleGhyGSI6BfPzyJM40pys/jCaOj0KE1BINYXczC3Ze/rppzl06BChoaGZfuBaLBZWrFjhjm5c1fH4OGqEFHNthwcEci45kcS0VNe+UwkX2X/+NNtiTwDp5eH36rWhZGAwB86fzvSeucnxhDgiChV3bYf5B3EuJZFE+2XxJ13gwIVYtp9Nn7iuOLGXdyxtKZWvIFUKFmX3uWj2xJ0C0nOamyoCx+PjqFEkm/wnZpH/+nkj/ycTz1EluIRrO9Q3iPMpCSRdlv+Y5AscuhjDn+ePAumnxl4jihL+hUi0pzBm94/Epaavkeh5W2PXabbc4ETieSIKXhr/oX6BnM9i/P99MYYdZ48B6afG3qzenpIBBfn7YiyVCoRjs1jZePqQu7t/Q677Z9+5K8a+JW+M/XxFvIjZm+TaTjidhk9+K95+l2ZCx7YmElrJz7U4ulKbAmz86jQxe5O4EJ3Khq9OA6dJPGfH6XBiT3VyT99Qd4dy09lvgbU7WVm0aBHjx48nLS2N7t278+ijjxqe37lzJ6+99hqpqakULVqUUaNGERQUdM33dEtFaObMmZQtW5aRI0eycuVKw8PTkyCAX08conqRYpQJLAjAoxWq89OR/YY2q479Tcn8BahSKAyAOqElcDqdHL1wzt3dvel+O3WA6oWKUzp/+pUvD5etxYrjewxtfjm5nxL5grkzOByAu4qUwumEI/FnKR8UyrN3NsGKBV+rF4/ddhdLjvzl9jj+q1+PH6J6yGX5r5hF/o9ekf+wvJP/dbH7qRpckpIBhQF4oHRtVp8ynt5Zc2ovxfwLUjko/ZdmzYJlcOLkWOJZOpWqwzPl0xfXF/LJR8eStfjh+Hb3BnED1p46QETBEq4rvx4qcxcrTxrj/y16PyUCCnJHgaIA1CpcGidwNCE9/7ULl+H32INu7ffNkOXYP5zFz77AAlQpfMXYv5j7x36x6v7E7E0m7nj6acA9P8ZRqk4+Q5vC5Xw4uTOJxHNpABzeEE/+UC/C7vDnwS/L0DGQLXEAACAASURBVGF0STqMLknFVkGUvSd/npgE3aqio6MZPXo0M2bMYP78+cyePZv9+43j9Z133mHAgAEsXLiQsmXLMmnSpGzf1y0Vofz58/P2228zd+5catWq5Y5D/iunkxJ4ce0SxjfuiLfVxj8Xz/L8b99TtXA479drTeTiKcQkxfPUz9/x9t0t8ffyJsVhp/fqeSTnolMAV3MmOYGXNy9i7N2d8LbaOBx/hpc2LqBKcFHeqdWWDiu+IDY5nj7r5jC8RiT+tvT4+62fS4rDzrhdq3mtehsWt3gaL6uVpUd3MefQVk+Hdd1OJyXw4poljG+Skf8Ll+W/fmsiF12W/7qX5X9V3sj/2ZR4hm//jlE1u+BttXE04QzD/viWygWK8VrVKB7+7VNOp1zkhc0zGFKlHf42H1IdaQzaMpMURxqTD6zmrWqdmNOwPxbg870r+Ov8MU+Hdd3OpMQzdOsCPq79IF5WG0fiz/LKlnncGVyMN6u354FVnxObfJH+G2YxrNp9+Nt8SHGk8dyG2aQ40n85ls5fiOMJuW9icDopgRd/W8L4ppeN/V8zxv49rYlcOIWYxHieWnnZ2Lfb6f3zPJLtuX/s+wd70aB/CD+PisaR6iQw3JuGz4YSuz+JNZ/G0GF0SYpGBFClYzBLhx7H6m3BN7+VZi8X9XTXc5w7r+aKi4sjLi7zGqugoCBDNWft2rXUrVuX4OBgAFq1asXSpUvp16+fq43D4SA+Ph6AxMREChTIfhG7xZnV4oBbXJlp73u6Cx7jky8l+0Z5WMpFH093waMKhVzwdBc8JjnV3Osw4s/6e7oLHvV0ndWe7oJHvXzHErce79mtD7vtWLf/Vp9x48Zl2t+vXz/69+/v2p4wYQIJCQkMHDgQgLlz57J9+3beeustV5tt27bRs2dPAgIC8Pf3Z86cORQsWPCaxzf3TxYRERHJxOF031Vj3bt3JyoqKtP+K9f2OBwOLJetbnc6nYbtpKQkXn31VaZMmUJERARfffUVgwcPZuLEidc8viZCIiIi4jFXngK7mvDwcDZt2uTajomJITT00pqsvXv34uvrS0REBAAPPfQQY8aMyfZ9zXOjABEREbkudixue1yv+vXrs27dOs6cOUNiYiLLli2jUaNGrudLly7NyZMn+fvv9Jt/rlixgqpVq17t7VxUERIREZFbXlhYGAMHDqRbt26kpqbSqVMnIiIi6NWrFwMGDKBq1aqMGDGC5557DqfTSeHChXn33XezfV9NhERERMTgVvgMsKy0a9eOdu3aGfZ98cUXrv83btyYxo0bX/mya9KpMRERETEtTYRERETEtHRqTERERAzcefm8p5knUhEREZErqCIkIiIiBo5/cVl7bqeKkIiIiJiWKkIiIiJiYL9FL5/PCaoIiYiIiGmpIiQiIiIGumpMRERExARUERIRERGDW/UjNnKCKkIiIiJiWqoIiYiIiIHuIyQiIiJiAqoIiYiIiIHWCImIiIiYgCpCIiIiYqD7CImIiIiYgCZCIiIiYlo6NSYiIiIGWiwtIiIiYgKqCImIiIiBbqgoIiIiYgKqCImIiIiB1giJiIiImIAqQiIiImKgipCIiIiICagiJCIiIgaqCImIiIiYQK6sCK1q9rGnuyDiEckm+ivtSt4Wp6e74FFhNh9Pd8GjjqaleroLpqKKkIiIiIgJ5MqKkIiIiOQc3VlaRERExARUERIREREDrRESERERMQFNhERERMS0dGpMREREDHRqTERERMQEVBESERERA1WERERERExAFSERERExUEVIRERExARUERIREREDpypCIiIiInmfKkIiIiJioA9dFRERETEBVYRERETEQFeNiYiIiJiAKkIiIiJioKvGRERERExAFSERERExMNMaIU2EREREJFdYtGgR48ePJy0tje7du/Poo4+6ntu1axdDhgxxbZ85c4YCBQqwePHia76nJkIiIiJyy4uOjmb06NF89913+Pj40KVLF+6++25uv/12ACpXrsyCBQsASExMpHPnzgwfPjzb99UaIRERETFwOi1ue1yvtWvXUrduXYKDgwkICKBVq1YsXbo0y7YTJkygdu3a3HXXXdm+rypCIiIi4jFxcXHExcVl2h8UFERQUJBr+9SpU4SEhLi2Q0ND2b59e6bXXbhwgTlz5rBo0aLrOr4mQiIiImLgzsXSU6dOZdy4cZn29+vXj/79+1/qk8OBxXKpX06n07D9PwsXLuTee++lcOHC13V8TYRERETEY7p3705UVFSm/ZdXgwDCw8PZtGmTazsmJobQ0NBMr1u+fDlPP/30dR9fEyERERExcDrdd6wrT4FdTf369Rk7dixnzpzB39+fZcuW8dZbbxnaOJ1Odu7cSY0aNa77+FosLSIiIre8sLAwBg4cSLdu3ejYsSNt27YlIiKCXr16sWPHDiD9knlvb298fX2v+30tTqc75303x6GjRT3dBRGPSDbRTc6u5G3JdT+qbqowm4+nu+BRR9NSPd0Fj6pY8rhbj1frh1fddqzNbd5x27GyooqQiIiImJbWCImIiIiBPnRVRERExARUEcrw+3ovvvrSl9RUKFvOwcBBieTLZ2yzYJ43C+f74OMLpUo56Dsgkf8tdF+0wJulS3xIToHy5e0MHJSETy46pX+j8XeOyk+RkEtrODo/mEyze9PcGMGNMXv+N663MW2SL2mpFkqXszPghSQCroh/8Txvvl/gg4+vkxKlHPTun0RgRvyP3p/PkP+oB1No0jz35H/DFfl/7ir5XzTfB19fKJmR///F/9AV4/+BXDT+f11nZewXXqSmQvlyTl57KZX8V8Q+6zsbs+fZ8PWBsqWdDHkulQJBYLfD+2O82PxH+t/UDe528NwzaWRxa5dbltnH/tWY6UNXVRECzp2z8OEoP4YNT2TS1HjCizqY/KWfoc22rTbmzPLlvQ8SGD8xntp3pzHmI38AfvvViwXzfRgxKp6Jk+JJTrEw79vc81vwRuM/csRKYKCT8RPjXY/c8ksAlP/z5yx88oEfL7+eyPgp6fFP/dJ4xcX2bTa+ne3DW6MSGDMhgbvqpPHp6PSv0dEjFgIDYcyEBNcjN/0iOHfOwkej/Bg6PJEvM/L/1RX5/2OrjbmzfBnxQQKfXpH/oxnj/9OJ8a5Hbhn/Z8/B8Pe9+eDNVOZ9nULxYk7GTjT+fbxxq5UpM7z4/MMUZk1K4Z66dt7+wBuA75fZOHTEwpzJ6c9t/sPK8tW559eK2ce+pHPbiF2+fDlff/01hw8fNuyfPXu2u7pwVVs22ahY0U7xEg4A2rZPYeUKb8N9FPbts1GjZhohGTP/Bg1S+X19+l9Ry5d580CnFIKCwGqFAc8l0fze3HOFw43G/9dOG1YbPP9sAL2fzMc303yw2z0RyX9j9vxv3WyjfAUHxUqkx9amXSqrr4j/wF4r1WraXX/51muQxoaM+HfvtGG1ORnynD/9ewUw6+vcl/8KV+T/52zyf08W43/QswE882Q+puei8b9uo5U7KzkolZH7zu3t/LDcZoh91x4Ld9dyEJZx37rmDR38ss5Kaio4HJCUZCElFVJTIDWNXFUJNfvYvxan030PT3PLROiDDz7gm2++4dChQzz88MOuT4cFmDVrlju6cE0xMVZDaTMkxElCvIWEhEttKlWys22bF9HR6eXCH3/0JjXVQlychWNHrZw7Z+GVIRkTgam+5M9/C2T3Ot1o/HY71Khp5933Evjg43g2b0qvkOQWZs9/7CkrRUIdru0iIU4SEiwkXhZ/hcoOtm+1cSoj/uU/epOWauFCnAW73UK1GnaGj0hkxOgEtmzyYvF8b3eH8Z/FxlhdExzIiD+L/P9xWf6XGeKH6jXtvP1eAqM+jmfLJi8W5pLxH33KQthlsYeGOLkYbyH+stir3OFg41Yrx0+mby/4wUZqqoVzcdCutZ3A/E5ad/Kl5QO+lCzupHF9B7mF2ce+pHPLGqHVq1czb948vLy86Nq1Kz179sTHx4c2bdpwK9zGyOEgy3PatsumiVUj7DzWNZk3XwvAYoVWrVMIDHTg7QVpdtiy2YvhbyXg4wMfvO/PV5N9eaZvsvuCuAE3Gn/kfanApQrI/Z1SWDDPh/sfSMn5zt8Eps//Vb4FrZfFf2dVO126pfDu6/5YrXBvq1QCA514eTlpdZ+x+tXxgRQWzfemwwO5oyrmcADZ5L9KhJ1Huybz1msBWK3QMiP/Xl7Q5orxH5Ux/qNywfh3OLMf+zUjnDzVPY1Bw3ywWKBDpJ0CQU68vWDiVC8KBsPyeckkJcMLQ735eraNrg/ljrKI2cf+tZjpqjG3TIQu/2C0MmXKMGHCBHr06EGhQoWy/MA0dwsNdbJ796V+xMZayB/oxM//UpuEBKhaLY3WkekDPDbGwtSvfAkMclK4sJMGDVNdiyub3ZvK9K99gdzxi/BG41/+kzflytkpd1vGX1ZOsOWiZfhmz39IqIO9uy4l7PRV4q8SkUbLNqmuNtOn+BIYBD//5EWZ2xyULZeefyfgZXNnBDcmNNTJnuvMf6vL8j8tI/8rMsZ/2Yzx73SCVy4Z/+GhTv7cdem3/qlYCAp04n9Z7PEJULOag473pU9uTsXA+MleFAiClb9YeenZNLy9wdsb2rZysHy1NddMhMw+9iWdW06NtW7dmq5du7J9+3YAypcvz5gxY3juuecyrRnyhFp3pbH7LxvHjqZ/Ob5f5EO9+sYZ/enTVl56Ph/x8enbM2f40qRZ+tURDRulsnqVN8nJ6T8E167xokLF3PGDAG48/kMHrUyb4ovdDsnJsHCBD42b5J6/iMye/xq17OzZZeP40fTJwA+LvLm7vnHB55nTFl59IYCEjPjnTPehYbNULBb455CVGVN8XPn/fr43DZrkngWjNa/I/5Is8n/mivzPumL8f33Z+F+0wIdGuWT816vtYMdfVg5n5P7bhV40vsc4dmNiLTz1nA8XM2Kf9I0XrZrZsVigUgUnP/2c/nVLTYPVa61E3JF7To2ZfexLOrd9xMa6desIDQ3ltttuc+07ceIEkydP5tVX/92tvHPiIzY2/O7F5C99SUuDokUdvDgkkZMnrIz+0J/xE9O/AxbM92bRAh+cDrizip2+A5Lw9U2/hHTmdB9W/+yNwwG3l3cwYGDmy29vZTcSf1ISfDrWj91/2UizQ8NGafR4IjlXXUKbW/KfUx+xsen3jEuI0yC8qJOBg9PjH/eRH2MmpC+YWDzfmyULvXE6LFSuksbT/ZPx9YXkJPh8rB97d1lJs1u4p1EqXXum3PT85+RHbGz43Yspl+V/0JBETpywMuZDfz7NyP/C+d4sXuCDIyP/fS4b/59ljH97xvjvngPjP6c+YuO39Zcuny9RzMlbr6Ry7LiFN0d5M2tS+um9Wd/ZmDM/fRF19aoOBj+bhp8vnDsP74/xZs8+C1Yb1KnpYOAz6RWimy2nPmIjN4x9cP9HbFRd+LrbjrWj/RtuO1ZW9FljIrmIPmvMvPRZY7mjypZTNBHKObnkTLaIiIi4i26oKCIiImICqgiJiIiIQe5bNPPfqSIkIiIipqWKkIiIiBiY6YaKqgiJiIiIaakiJCIiIgaqCImIiIiYgCpCIiIiYmCii8ZUERIRERHzUkVIREREDLRGSERERMQEVBESERERIxMtElJFSERERExLEyERERExLZ0aExEREQMtlhYRERExAVWERERExMCpxdIiIiIieZ8qQiIiImKgNUIiIiIiJqCKkIiIiBipIiQiIiKS96kiJCIiIga6akxERETEBFQREhERESNVhERERETyPlWERERExED3ERIRERExAVWERERExEhrhERERETyPk2ERERExLR0akxEREQMtFhaRERExARyZUWolFegp7sgIuJWh9IueLoLHhU5+0VPd8GjDgxy8wG1WFpEREQk78uVFSERERHJSVojJCIiIpLnqSIkIiIiRlojJCIiInJrWbRoEZGRkbRs2ZLp06dnev7vv/+ma9eutG/fnieeeILz589n+56aCImIiIiR042P6xQdHc3o0aOZMWMG8+fPZ/bs2ezfv/9Sl51OnnnmGXr16sXChQupXLkyEydOzPZ9NRESERGRW97atWupW7cuwcHBBAQE0KpVK5YuXep6fufOnQQEBNCoUSMAevfuzaOPPprt+2qNkIiIiBi58c7ScXFxxMXFZdofFBREUFCQa/vUqVOEhIS4tkNDQ9m+fbtr+/DhwxQpUoRXXnmFXbt2Ua5cOYYNG5bt8VUREhEREY+ZOnUqzZs3z/SYOnWqoZ3D4cBiuTRBczqdhu20tDQ2bNjAww8/zLx58yhZsiTvvfdetsdXRUhEREQMnG68aqx79+5ERUVl2n95NQggPDycTZs2ubZjYmIIDQ11bYeEhFC6dGmqVq0KQNu2bRkwYEC2x1dFSERERDwmKCiIEiVKZHpcORGqX78+69at48yZMyQmJrJs2TLXeiCAGjVqcObMGXbv3g3AypUrufPOO7M9vipCIiIiYnQL3kcoLCyMgQMH0q1bN1JTU+nUqRMRERH06tWLAQMGULVqVT799FOGDh1KYmIi4eHhjBw5Mtv3tTid7iyA3RyOkxU83QUREbcy+4eutphl9g9dfd6txys9KfsJxM3yzxMvue1YWdGpMRERETEtnRoTERERIzdePu9pqgiJiIiIaakiJCIiIgaWXLd6+L9TRUhERERMSxUhERERMVJFSERERCTvU0VIREREjHTVmIiIiEjep4qQiIiIGGmNkIiIiEjep4qQiIiIGKkiJCIiIpL3qSIkIiIiRqoIiYiIiOR9qgiJiIiIke4jJCIiIpL3aSIkIiIipqVTYyIiImJg0WJpERERkbxPFSERERExMlFFSBOhf8nphJdHQIVy0LOLp3vjXmaOHRS/4s+b8W9Y78VXX/qSmgplyzl4blAi+fIZ2yyY582i+T74+kLJUg76DkgkMCj9uYei8lMk5NJvzQceTKbZvWlujODGNClXlhcbNsDHZmN3TCwv/7iMiykphjYvN2lEZIUKnEtKAuDgmbMMWPw9+X18eK91S24rVAiLxcJ3O/9i4oaNnghDboAmQv/CgUPw1sewfVf6D0MzMXPsoPgVf96M/9w5Cx+N8uPDMQkUL+Fg0kRfvvrSj37PJrna/LHVxtxZvoweF09IiJMVP3kz5iN/hg5P5OgRK4GBTj6dGO/BKP67Qv7+jGzdigdnzOLQuXO81KghLzZqwOvLVxra1SxWjGcXf8+W4ycM+59vcA8nL1yk38LF+Ht7sfTx7mw8cpStJ4zt5NbmtonQoUOH8Pf3JywsjLlz57Jnzx5q1qxJZGSku7pww2bMh073QdEwT/fE/cwcOyh+xZ8349+yyUaFinaKl3AA0LZ9Cn2eyk/fAUlYMm4js2+fjRo10wjJqPrc0yCVjz/0IzUV/tppw2qDQc8GEB9voUGjVLo8moLN5qmI/p0GZUqz/eRJDp07B8D0bX/wffeuhomQj83GnaGhPFW7NqWCgzl49ixv/7yKExcu8ObKn7FlfKFC8+XHx2bjQkqyR2KR/y7bidDkyZMZN24cdrud4sWLU6FCBSpWrEjFihWpUKECJUqUyPYgU6ZM4euvv8bhcFC3bl1OnDhBixYt+Pbbbzl48CB9+/a9KcHktGHPpf+7ZpNn++EJZo4dFL/iT/83r8UfG2N1TXAAioQ4SYi3kJCA6/RYpUp2Fs7zITraQliYk2U/epOWauFCnAW7HarXtNPzySTS7PD6KwEE5IOoB1KucsRbS9HAQE7EXXBtn7xwgUBfX/L7+LhOj4Xmz8e6w0f4aM0a9saeplftu5jQsQPtv/4GALvTyYeRbWhToTzL9u3n7zNnPRLLzWamq8aynQhNmDCBkSNHEhERwZEjR9i7dy979uzhl19+Yd++fQCUL1+emTNnXvU9vv32W5YsWUJsbCxt27Zl/fr1+Pr60rlzZzp16pRrJkIiInmJwwFkcQNh22XXE1eJsPNo12Teei0AqxVatk4hMNCBlxe0uS8VSHW1jeqUwoJ5PrlmImS1WLJcE2x3Olz/P3o+jie+m+fa/mLjJvrWvZsSBYI4ej4OgBeW/MCwn5bzaft29K9XlzFr1+V01+UmynYilD9/fpo0aYKXlxehoaHUqlXL8PzRo0ddE6KrcTgc+Pj4ULx4cXr27Imvr6/rObvd/h+7LiIiNyI01Mme3ZdmQrGxFvIHOvHzv9QmIQGqVkujVWT6hCc2xsK0r3wJDEpfL1SunJ2yt6VPHJxO8MpFK0+Px12gWtFw13ZYYH7OJSaRmHppsXfFIkWoHBrC/L92ufZZLBbS7A4alinNnphYTsXHk5CayuLdu2lVobxbY8gx+oiNS5566inmzp171edLlChB06ZNr/keLVu25LHHHsNut9O/f38Adu/ezSOPPEKbNm3+ZZdFRORmqHlXGrv/snHsaPqvgiWLfKhXP9XQ5sxpKy89n4/4jPXQs2b40qRZGhYLHDpo5espvtjtkJwMixb40KhJ6pWHuWX99s8hahQtSpngYAAeqVaN5Qf2G9o4cfJas6aUKJB+mdyj1auxJyaGkxcvElmxIgPq1wPS1xJFVqzIusNH3BuE3LBs5+7vvfceqamp/PrrrzRs2JDKlStTsWJF/P39s3upy7PPPsvGjRuxXbaCzsfHh/79+9O4ceP/1nMREbkhwQWdDHwpiXfe8CctDYoWdTBoSCJ791gZ86E/n06Mp0RJBw8+nMzAfvlwOODOKnb6DEi/quzRbsl8NtaPZ57Mh90ODRul0Toy90yETickMnjpMsa1b4e3zcrhc+cZ9MNSqoaF8W6rFrSb9g17Y0/zxoqf+SKqI1aLhZMXLvLs4iUAvLtqNW+3aM4Pj3cDYNm+/UzZvMWTId08JlojZHE6ndcM9/Dhw+zZs8f12L17N8ePH6dEiRL8+OOP7uqngeNkBY8cV0TEUw6lXci+UR7WYtaLnu6CRx0Y9Lxbj1fu44/cdqy/n3NvbFfKtiJUqlQpSpUqRYsWLVz7EhIS2Lt3b452TERERDzERBWh//RZYwEBAVSvXv1m90VERETErXLR+n4RERFxBzPdR0ifPi8iIiKmpYqQiIiIGKkiJCIiIpL3aSIkIiIipqVTYyIiImKkU2MiIiIieZ8qQiIiImKgy+dFRERETEAVIRERETFyWjzdA7dRRUhERERMSxUhERERMdIaIREREZG8TxUhERERMdBVYyIiIiImoIqQiIiIGKkiJCIiIpL3qSIkIiIiBlojJCIiImICqgiJiIiIkSpCIiIiIreWRYsWERkZScuWLZk+fXqm58eNG0fTpk3p0KEDHTp0yLLNlVQREhERkVtedHQ0o0eP5rvvvsPHx4cuXbpw9913c/vtt7va/Pnnn3z00UfUqFHjut9XEyERERExcuOpsbi4OOLi4jLtDwoKIigoyLW9du1a6tatS3BwMACtWrVi6dKl9OvXz9Xmzz//ZMKECRw7dozatWszePBgfH19r3l8nRoTERERj5k6dSrNmzfP9Jg6daqh3alTpwgJCXFth4aGEh0d7dqOj4+ncuXKvPjii8ybN4+4uDg+++yzbI+vipCIiIgYuPPy+e7duxMVFZVp/+XVIACHw4HFYnFtO51Ow3a+fPn44osvXNs9e/bklVdeYeDAgdc8viZCIiIi4jFXngK7mvDwcDZt2uTajomJITQ01LV9/Phx1q5dS6dOnYD0iZKXV/bTHJ0aExERkVte/fr1WbduHWfOnCExMZFly5bRqFEj1/N+fn6MGjWKI0eO4HQ6mT59Oi1atMj2fTUREhERkVteWFgYAwcOpFu3bnTs2JG2bdsSERFBr1692LFjB4UKFeLNN9/kmWeeoXXr1jidTnr06JHt++rUmIiIiBjdojdUbNeuHe3atTPsu3xdUKtWrWjVqtW/ek9VhERERMS0VBESERERA33oqoiIiIgJ5MqKUKti1TzdBfEUi+bupuV0eLoHnmXysV/Ouc7TXfCsQW4+nipCIiIiInlfrqwIiYiISA5SRUhEREQk71NFSERERAx01ZiIiIiICWgiJCIiIqalU2MiIiJipFNjIiIiInmfKkIiIiJioMXSIiIiIiagipCIiIgYqSIkIiIikvepIiQiIiJGqgiJiIiI5H2qCImIiIiBrhoTERERMQFVhERERMRIFSERERGRvE8VIRERETFSRUhEREQk71NFSERERAx01ZiIiIiICWgiJCIiIqalU2MiIiJipFNjIiIiInmfKkIiIiJioMXSIiIiIiagipCIiIgYqSIkIiIikvepIiQiIiJGqgiJiIiI5H2qCImIiIiBxdMdcCNVhERERMS0VBESERERI60RkjqRNZmw7QMm7xrDsNnPExDof9W2L37Vl04vtHNj73KemeOvE1mDCVtHMvmv0QybPTCb2PvQ6fm2buxdzlP85h37YO78mz33ZqWJUBYKFAli0OQ+vNnpA3pWfpYTB6N54r1HM7UrVak4I5e/TsNOdT3Qy5xj5vgLFAlk0KRneLPzR/S8YyAn/o7miRGPZGpXqlJxRv40jIYP3O2BXuYcxW/esQ/mzr/Zc38li9N9D0/zyETovffe88Rhr1utlhHs3XiAY/tPArBo/DKaP9IwU7v2fVvzw6QV/Dp3vbu7mKPMHH+tltXYu+my2D//ieaPNMjUrn2flvwweSW//l/eiR0Uv5nHPpg7/2bPvZnl+Bqhl19+OdO+lStXcv78eQBGjBiR013410JKFiHmaKxrO+boafIVCCAg0J+EC4mu/eP6TwLgrhbV3N7HnGTm+ENKFCbmyGnX9lVjH/AVAHe1iHB7H3OS6eM38dgHc+ff7LnP5Bao1LhLjk+EgoODmT9/Pr179yYoKAiA9evXU6dOnZw+9H9mtVpwZjEIHHaH+zvjAWaOPz32zMGbIXZQ/GYe+2Du/Js992aW46fGBg8esHnD+gAAFkRJREFUzEcffcSSJUsoVqwYUVFRFChQgKioKKKionL68P/JqcOxFC5a0LVdpHgh4s5cJCkh2YO9ch8zx3/qSCyFi5kzdlD8Zh77YO78mz33ZuaWNUL16tVjwoQJzJgxg/fffx+73e6Ow/5nm5f9QeW65Sl+ezgAbXu3ZN2CjR7ulfuYOf7Ny7ZT+e7LYn+6BesWbvJwr9xH8Zt37IO582/23GfidOPDw9y2WDo4OJgxY8ZQrlw5QkJC3HXY/+RcTBwf9PyMYXNfYNLO0ZStUooJg6ZRoVY5Pt8yytPdy3Fmjv9cTBwfPDGeYXOeZ9KfH1G2aslLsW9+39Pdy3GK37xjH8ydf7Pn3swszqxOCN/iWlg7e7oL4ikW3fHBtJwmX6th9rFv8vz/5Pj/9u4/qKo6/+P466YwSSOhyeX2ze9OtVRWG2rOJFqr4A9IwB8lW8asUP4YWUVm6dc6jmZpuso4UcHOrjZJWtigu+JCJcJm62jQ7uC3r/Sdditz3dUV4SbZzVQk7v3+sdPdTmCAC+fA/TwfM3fGc+/1nPd73Gbf8zqfez47bb3eqKUFtl3rfwvzbLtWRwz/LwsAAJiMLTYAAIBVv7tXdPlIhAAAgLFIhAAAgEVf2PrCLiRCAACgX6ioqFBKSoqSkpJUUlJyye/98Y9/1KRJk7p0ThIhAABg1QcTocbGRhUUFGjXrl0KDw/XnDlzNHbsWMXGxlq+99lnn2nDhq4/7oFECAAA9Hk1NTWKj49XVFSUIiIilJycrMrKynbfW7FihXJycrp8XhIhAABgYecaIZ/PJ5/P1+79yMjI4B6lktTU1GR5ILPb7VZ9fb3l72zbtk233XabRo7s+qa4DEIAAMAxW7duVVFRUbv3c3JytHTp0uCx3++Xy+UKHgcCAcvxxx9/rKqqKr3yyis6depUl6/PIAQAAKxsTISysrI63IT922mQJHk8HtXV/XvvO6/XK7fbHTyurKyU1+vV7Nmz1draqqamJmVkZGj79u3fe30GIQAA4Jjv3gK7lPHjx6uwsFDNzc0aNGiQqqqqtGbNmuDnubm5ys3NlSSdOHFCmZmZnQ5BEoulAQDAd/XB3edjYmKUl5enzMxMzZo1S2lpaYqLi9PChQv1wQcfXHarbLqK/sX0jSdNZvimm8b/b9/wf3+7N129c5F9m67+zyY2XQUAAHAEa4QAAIAFW2wAAAAYgEQIAABYkQgBAACEPhIhAABg4ep/Pyi/bCRCAADAWCRCAADAypxAiEQIAACYi0QIAABY8BwhAAAAA5AIAQAAKxIhAACA0EciBAAALFgjBAAAYAASIQAAYEUiBAAAEPoYhAAAgLG4NQYAACxYLA0AAGCA/pkIuZjfAOPw373Z+Pe3F4kQAABA6OufiRAAAOg1rBECAAAwAIkQAACwCpgTCZEIAQAAY5EIAQAAC9YIAQAAGIBECAAAWJEIAQAAhD4SIQAAYOHyO12BfUiEAACAsUiEAACAFWuEAAAAQh+DEAAAMBa3xgAAgAUPVAQAADAAiRAAALBi01UAAIDQRyIEAAAsWCMEAABgABIhAABgRSIEAAAQ+kiEAACABWuEAAAADEAiBAAArHiOEAAAQOgjEQIAABasEQIAADAAiRAAALAiEQIAAAh9DEIAAMBYDEIAAMDCFbDv1R0VFRVKSUlRUlKSSkpK2n1eXV2t6dOnKzU1VcuWLdPFixc7PSeDEAAA6PMaGxtVUFCg7du3a/fu3SotLdWRI0eCn587d06rV69WcXGx3nzzTbW0tKisrKzT87JYGgAAWPntWy3t8/nk8/navR8ZGanIyMjgcU1NjeLj4xUVFSVJSk5OVmVlpXJyciRJERER2rdvn8LCwnT+/HmdPn3a8vcvhUToEu5KGa1N7+dry4cFWlmap4jBgy753SeKFyv90TQbq+t9Jvdvcu8S/dO/uf2b3LuTtm7dqsmTJ7d7bd261fK9pqYmRUdHB4/dbrcaGxst3wkLC9P+/fuVkJCgzz//XPfcc0+n12cQ6sDVwwbr8Zd/ptU/eU7zbstTw9FGzf9lRrvv/WDEdcqvXqkfzx7rQJW9x+T+Te5don/6N7d/k3vvUMC+V1ZWlt5+++12r6ysLEtJfr9fLpfr3yUGApbjb0ycOFF/+tOflJiYqKeffrrTVm0ZhOrr64N/rq2t1fr167Vx40YdPnzYjst325ikkfq47lP988gpSVLFb6o1OaP9VDljcZL2bNmnA799z+4Se5XJ/Zvcu0T/9G9u/yb37rTIyEgNHz683eu7t7U8Ho+8Xm/w2Ov1yu12B4/PnDmjgwcPBo+nT5+ujz76qNPr2zIIrVq1SpJUUlKidevWyePxaNiwYXrqqaf02muv2VFCt0QPv0be46eDx94Tp3XV1RHtYtKi3GK98/q7dpfX60zu3+TeJfqnf3P7N7n3jvTFX42NHz9etbW1am5u1vnz51VVVaUJEyYEPw8EAnriiSd08uRJSVJlZaXuvPPOTs9r62LpHTt2aNu2bRoyZIgkKT09Xenp6frpT39qZxmduuIKlwId7Lzrb/M7UI39TO7f5N4l+qd/c/s3uff+IiYmRnl5ecrMzFRra6vS09MVFxenhQsXKjc3V3fccYfWrFmjRYsWyeVyKTY2Vs8880yn57VlEPr666/l9/sVFRWl8PDw4Pvh4eG64oq+t0yp6fhnGjE2Nng87Lqh8jWf1YVzLQ5WZR+T+ze5d4n+6d/c/k3uvUMdDIV9wfTp0zV9+nTLey+99FLwz1OmTNGUKVO6dU5bppCoqCglJCTob3/7m9asWSPpX2uF5syZo3vvvdeOErrlUFW9bh17k66L9UiS0hZNVW15ncNV2cfk/k3uXaJ/+je3f5N7N50tidCrr74qSTp69GjwWQHh4eHKzc1VQkKCHSV0yxmvTxvn/1ordzyqsPCBOnn0lPKzfqWbx9yoRzcvUvaYXzhdYq8yuX+Te5fon/7N7d/k3jvS3Sc+92euQEc3Rfu4qQMedLoEAABsU91Wauv1EpM32Hatd/Y6O2TyZGkAAGDV7yKSy9f3VioDAADYhEQIAABYuPrfqpnLRiIEAACMxSAEAACMxa0xAABgZdADtUmEAACAsUiEAACABYulAQAADEAiBAAArMwJhEiEAACAuUiEAACAFWuEAAAAQh+JEAAAsHCZEwiRCAEAAHORCAEAACvWCAEAAIQ+EiEAAGDhYq8xAACA0EciBAAArFgjBAAAEPpIhAAAgJU5gRCJEAAAMBeDEAAAMBa3xgAAgIWLxdIAAAChj0QIAABYkQgBAACEPhIhAABgxRYbAAAAoY9ECAAAWPCrMQAAAAOQCAEAACsSIQAAgNBHIgQAAKxIhAAAAEIfiRAAALDiOUIAAAChj0QIAABY8BwhAAAAAzAIAQAAY3FrDAAAWHFrDAAAIPSRCAEAACsSIQAAgNBHIgQAAKxIhAAAAEIfiRAAALBiiw0AAIC+paKiQikpKUpKSlJJSUm7z//whz9o5syZmjFjhhYvXqwvvvii03MyCAEAAAtXIGDbq6saGxtVUFCg7du3a/fu3SotLdWRI0eCn589e1ZPP/20Nm/erPLyct1yyy0qLCzs9LwMQgAAoM+rqalRfHy8oqKiFBERoeTkZFVWVgY/b21t1apVqxQTEyNJuuWWW9TQ0NDpeVkjBAAArGz81ZjP55PP52v3fmRkpCIjI4PHTU1Nio6ODh673W7V19cHj4cMGaKpU6dKki5cuKDNmzdr7ty5nV6fQQgAADhm69atKioqavd+Tk6Oli5dGjz2+/1yuVzB40AgYDn+xpdffqklS5ZoxIgRuu+++zq9PoMQAACw8tuXCGVlZXU4sHw7DZIkj8ejurq64LHX65Xb7bZ8p6mpSfPnz1d8fLyWL1/epeszCAEAAMd89xbYpYwfP16FhYVqbm7WoEGDVFVVpTVr1gQ/b2trU3Z2tqZNm6bFixd3+foMQgAAwKoPPlk6JiZGeXl5yszMVGtrq9LT0xUXF6eFCxcqNzdXp06d0ocffqi2tjbt3btXkvSjH/1Ia9eu/d7zugKBPthtJ6YOeNDpEgAAsE11W6mt15t28y9su9aejzfYdq2O8PN5AABgLG6NAQAAq/53s+iykQgBAABjkQgBAAArEiEAAIDQRyIEAACsbHygotNIhAAAgLFIhAAAgFXA73QFtiERAgAAxiIRAgAAVvxqDAAAIPSRCAEAACt+NYa7UkZr0/v52vJhgVaW5ili8KBLfveJ4sVKfzTNxup6n8n9m9y7RP/0b27/JvduMgahDlw9bLAef/lnWv2T5zTvtjw1HG3U/F9mtPveD0Zcp/zqlfrx7LEOVNl7TO7f5N4l+qd/c/s3ufcOBQL2vRxm2yB04MAB+Xw+SdLu3bu1evVq/e53v7Pr8t0yJmmkPq77VP88ckqSVPGbak3OuKfd92YsTtKeLft04Lfv2V1irzK5f5N7l+if/s3t3+TeTWfLILR27Vpt2rRJLS0tev7551VeXq7Y2FhVV1fr2WeftaOEbokefo28x08Hj70nTuuqqyPaxaRFucV65/V37S6v15ncv8m9S/RP/+b2b3LvHTIoEbJlsXRNTY3Ky8s1YMAA7d+/X6WlpQoPD9eDDz6otLS+d4/1iitcCnTwj+NvM+MBUyb3b3LvEv3Tv7n9m9y76WxJhK688kqdPv2vSdvj8ejcuXOSpPPnz2vgwL73w7Wm45/pmv8aEjwedt1Q+ZrP6sK5Fgerso/J/Zvcu0T/9G9u/yb3bjpbBqElS5YoPT1dGzZs0PDhwzV37lytW7dODzzwgB555BE7SuiWQ1X1unXsTbou1iNJSls0VbXldQ5XZR+T+ze5d4n+6d/c/k3uvUMG3RqzZRCaNGmSSkpK5Ha71draqlGjRumqq67S+vXrdf/999tRQrec8fq0cf6vtXLHo3r5/57TDXf8tzY9vk03j7lRvzm0wenyep3J/Zvcu0T/9G9u/yb3bjpXoKObon3c1AEPOl0CAAC2qW4rtfV6065dYtu19jT8yrZrdYTnCAEAAGP1vZXKAADAWf3vZtFlIxECAADGIhECAABWJEIAAAChj0QIAABY+UmEAAAAQh6JEAAAsAgEzNljjUQIAAAYi0QIAABYsUYIAAAg9JEIAQAAK54jBAAAEPoYhAAAgLG4NQYAAKz8/HweAAAg5JEIAQAAKxZLAwAAhD4SIQAAYBFgjRAAAEDoIxECAABWrBECAAAIfSRCAADAik1XAQAAQh+JEAAAsArwqzEAAICQRyIEAAAsAqwRAgAACH0kQgAAwIo1QgAAAKGPQQgAABiLQQgAAFgE/AHbXt1RUVGhlJQUJSUlqaSk5JLfe/LJJ7Vr164unZNBCAAA9HmNjY0qKCjQ9u3btXv3bpWWlurIkSPtvpOdna29e/d2+bwslgYAAFY2Lpb2+Xzy+Xzt3o+MjFRkZGTwuKamRvHx8YqKipIkJScnq7KyUjk5OcHvVFRUaPLkycHvdEW/HISq20qdLgEAgJBV7d9p27UKCwtVVFTU7v2cnBwtXbo0eNzU1KTo6OjgsdvtVn19veXvLFiwQJJ06NChLl+/Xw5CAAAgNGRlZem+++5r9/630yBJ8vv9crlcweNAIGA5vlwMQgAAwDHfvQV2KR6PR3V1dcFjr9crt9v9H1+fxdIAAKDPGz9+vGpra9Xc3Kzz58+rqqpKEyZM+I/PyyAEAAD6vJiYGOXl5SkzM1OzZs1SWlqa4uLitHDhQn3wwQeXfV5XIBAwZ2c1AACAbyERAgAAxmIQAgAAxmIQAgAAxmIQAgAAxmIQ6oaubvYWqs6ePau0tDSdOHHC6VJsV1RUpNTUVKWmpio/P9/pcmz3wgsvKCUlRampqSouLna6HMds2LBBy5Ytc7oM282dO1epqamaOXOmZs6cqcOHDztdkq327dun+++/X9OmTdOzzz7rdDnoYTxQsYu+2ext165dCg8P15w5czR27FjFxsY6XZotDh8+rBUrVujYsWNOl2K7mpoaHTx4UGVlZXK5XFqwYIGqq6s1depUp0uzxZ///Ge99957Ki8v19dff62UlBRNnDhRN954o9Ol2aq2tlZlZWVKSEhwuhRbBQIBHTt2TO+8844GDjTv/zKOHz+uVatWaefOnbrmmmuUlZWl/fv3a+LEiU6Xhh5CItRF397sLSIiIrjZmyl27NihVatW9chTPPub6OhoLVu2TOHh4QoLC9MPf/hDnTx50umybHPXXXdp27ZtGjhwoE6fPq22tjZFREQ4XZatzpw5o4KCAmVnZztdiu2OHj0qSZo3b55mzJih1157zeGK7FVdXa2UlBR5PB6FhYWpoKBAI0eOdLos9CDzxvvL1JXN3kLZ2rVrnS7BMTfddFPwz8eOHdOePXv0+uuvO1iR/cLCwvTiiy9qy5YtuvfeexUTE+N0SbZ66qmnlJeXp4aGBqdLsZ3P59O4ceO0cuVKtba2KjMzUzfccIPuvvtup0uzxd///neFhYUpOztbDQ0NSkhI0M9//nOny0IPIhHqot7a7A39xyeffKJ58+bpySef1PXXX+90ObbLzc1VbW2tGhoatGPHDqfLsc3OnTt17bXXaty4cU6X4ojRo0crPz9fgwcP1tChQ5Wenq79+/c7XZZt2traVFtbq3Xr1qm0tFT19fUqKytzuiz0IAahLvJ4PPJ6vcHjntrsDf3DoUOH9PDDD+uxxx7rcJfkUPbpp5/qL3/5iyRp0KBBSkpK0kcffeRwVfZ566239O6772rmzJl68cUXtW/fPq1bt87psmxTV1en2tra4HEgEDBqrdCwYcM0btw4DR06VFdeeaWmTJli1N0AEzAIdVFvbfaGvq+hoUFLlizRxo0blZqa6nQ5tjtx4oRWrFihixcv6uLFi3r77bc1ZswYp8uyTXFxsd544w39/ve/V25uriZNmqTly5c7XZZtvvzyS+Xn56ulpUVnz55VWVmZMT8UkKTExEQdPHhQPp9PbW1tOnDggG6//Xany0IPMmes/w99e7O31tZWpaenKy4uzumyYIOXX35ZLS0tWr9+ffC9OXPm6KGHHnKwKvtMnDhR9fX1mjVrlgYMGKCkpCQjB0JTJSYm6vDhw5o1a5b8fr8yMjI0evRop8uyzciRI7VgwQJlZGSotbVVd999t2bPnu10WehBbLoKAACMxa0xAABgLAYhAABgLAYhAABgLAYhAABgLAYhAABgLAYhAABgLAYhAABgLAYhAF0ybdo0TZgwQZ988onTpQBAj2EQAtAlb7zxhq6//nrt3bvX6VIAoMcwCAHokgEDBmjMmDFGbbgKIPSx1xiALrlw4YLeeustsSsPgFBCIgSgSwoKCuR2u/WPf/xDX331ldPlAECPYBAC0Kn3339fe/bsUWFhoQYPHsyCaQAhg0EIwPdqaWnR8uXL9cwzzygqKkojRozQX//6V6fLAoAewSAE4Hu98MILGjVqlBITEyVJI0aMYME0gJDBIATgkurr61VZWanly5cH37v11ltJhACEDFeAn4AAAABDkQgBAABjMQgBAABjMQgBAABjMQgBAABjMQgBAABjMQgBAABjMQgBAABjMQgBAABj/T+RnfBv2RzQbwAAAABJRU5ErkJggg==\n",
+      "text/plain": [
+       "
    "
+      ]
+     },
+     "metadata": {
+      "filenames": {
+       "image/png": "/Users/hjensen/Teaching/FYS-STK4150/doc/src/LectureNotes/_build/jupyter_execute/chapter6_134_1.png"
+      }
+     },
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkIAAAJgCAYAAABmwww9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeZxO5f/H8de5ZzNjzAizkH0J2Zds2ZU1W+UbKaKvbyWEQmVJkRSiXySSrdBmz1oIZcpaJEuyZRnGMmZllvv8/rinexwzRZb7drvfz8fjfnDOfd3nXJ/7OufMdT7nOuc2TNM0EREREfFCNndXQERERMRd1BESERERr6WOkIiIiHgtdYRERETEa6kjJCIiIl5LHSERERHxWuoIidymRo4cSdu2bWnbti3ly5enWbNmzumLFy/+q2Xt3LmTYcOG/e37+/bto3Tp0kydOvVGqy0i4lEMPUdI5PbXuHFj3nvvPSpUqHBdn1+wYAGrVq1iypQp2b7/2muvkZiYyJYtW1izZg2+vr43Ul0REY+ho52IB/ryyy+ZN28edrud3LlzM3ToUEqUKMHWrVsZPXo0drsdgGeeeYaKFSvyf//3f8THx/PKK6/w1ltvWZaVkJDA0qVL+fLLL9m7dy+rVq2iVatWAKSlpTFmzBi+++47fHx8qFKlCq+99ho2my3b+VOmTOH8+fPO7NP777/vnH7yyScJDQ3l4MGDdOrUiQoVKjBmzBhSUlKIiYmhTp06jBo1CoB169YxYcIE7HY7QUFBvP7666xbt44DBw4wbtw4ALZu3crIkSNZtGiRq752EbkDqSMk4mE2b97MokWLmDNnDoGBgXz//ff06tWLFStW8P7779OtWzdatWrF3r17+fzzz2nWrBl9+vRh1apVWTpBAIsXL6Zo0aKUKFGCdu3aMXPmTGdHaO7cuezevZvFixfj7+9P//79Wb58OXFxcdnOv5qQkBBnuf79+9OnTx9q1qxJYmIiTZo04ddffyUyMpIBAwYwe/Zs7r33XlavXs3YsWN5++23adq0KbGxseTOnZsvvviCjh073twvV0S8jjpCIh7mu+++48iRI5ZOQFxcHLGxsbRo0YI33niDtWvXUqdOHfr373/V5X322Wf85z//AaBNmza8++677NixgypVqrBp0ybatm1Ljhw5AJgwYQIAzz77bLbz33///X9cV/Xq1Z3/Hz16NBs2bODDDz/k4MGDXLp0iaSkJLZv306pUqW49957AWjatClNmzYFoGHDhixevJh27drx/fff89prr13TdyYi8nfUERLxMHa7nbZt2zJgwADn9OnTpwkNDaVjx440atSIH374gY0bNzJx4kRWrlz5t8vaunUrv//+O9OmTWPGjBkA+Pn5MXPmTKpUqZJlrNCZM2ew2+1/O98wDC4fdpiammopFxQU5Pz/E088QenSpalXrx4tWrTgl19+wTRNfHx8MAzDWc40Tfbt20eZMmXo3Lkzw4cPx9fXl6ZNm5IzZ85/+e2JiFjprjERD1O3bl2WLVvG6dOnAZg3bx5du3YFoGPHjuzZs4eHH36YESNGEBcXR0xMDD4+PqSlpWVZ1rx582jbti3r169n7dq1rF27lg8//JBvvvmGEydOULt2bb7++mtSUlKw2+0MHz6cZcuW/e38u+66i927d2OaJgkJCaxbty7bGOLi4ti1axcvvfQSTZs2JTo6mqNHj2K326lUqRJ//PEHv//+OwBr1qxxdvqqVq2KzWbj448/1mUxEbkplBES8TB169alR48edO/eHcMwCA4OZuLEiRiGwUsvvcSoUaOYMGEChmHQq1cvChYsSHp6OpMmTaJXr15MnDgRgHPnzrF69Wrmz59vWX7t2rWpXLkyn3zyCS+99BLHjx/n4YcfxjRNatSowZNPPolhGNnOT05OZuPGjTRt2pSIiAhq1KhBdjemhoSE8L///Y/27dsTFBREREQEVatW5ciRI9SuXZuxY8cyaNAg0tPTCQ4OZvz48c7PPvzwwyxfvpwyZcrc2i9aRLyCbp8XEY+RlpZGr169aNOmDS1btnR3dUTkDqBLYyLiEQ4cOEDt2rW56667aN68uburIyJ3CGWERERExGspIyQiIiJeSx0hERER8VrqCImIiIjX8sjb55vn6eHuKriNWaKgu6vgVsah4+6uglsZlz2Q0OsEe3HsACmpVy9zB7OHenf7r9r2ukvXZ4++x2XrskXud9m6sl2/W9cuIiIi4kYemRESERGRW8eO3WXrcndGxt3rFxEREXEbZYRERETEIt10XUbI3R0RZYRERETEa6kjJCIiIl7L3RkpERERuc3Y8Z5f31JGSERERLyWMkIiIiJi4crb591NGSERERHxWsoIiYiIiEW6qTFCIiIiInc8ZYRERETEQneNiYiIiHgBZYRERETEIl0ZIREREZE7nzJCIiIiYqExQiIiIiJeQBkhERERsdBzhERERES8gDJCIiIiYuE9vzSmjJCIiIh4MXWERERExGvp0piIiIhY6IGKIiIiIl5AGSERERGxSPeehJAyQiIiIuK9lBESERERC90+LyIiIuIFlBESERERi3QMd1fBZZQREhEREY+wdOlSWrZsSdOmTZkzZ06W99evX0/r1q1p3bo1L774IomJiVddpjpCIiIiYmE3Xfe6VqdOnWL8+PHMnTuXRYsW8fnnn3PgwAHn+3Fxcbz88suMHz+epUuXUqZMGcaPH3/V5aojJCIiIm4TFxfHsWPHsrzi4uIs5TZt2kStWrXInTs3QUFBNGvWjJUrVzrfP3z4MAUKFKBkyZIANGrUiG+//faq69cYoQw1HqxAt2EP4+fvy6HfjjG+zyyS4i9mW/bFSd04vOc48yeuBsA/hx/Pj3mc0lWLYRgGe7cdZNKAuaRcTHVlCDekxv2l6PZ8E/z8fTj0+ynGj1xCUmKKpUzjFhXo8EQdTODSxVQ+GLuC3/ecdL4fFhHChOlP89zjHxJ3IdnFEdyYGg9WoNuQ9vgF+HJo93HGvzCLpIS/af+JTznaf9I3QEb7v/04pasWzWj/Q0wa5Fntf1/je+k26CHH9r/3BBMGzCMp4VK2ZV9893EO7z3J/KnrnPM++3kkZ05ecE7Pn7KWdYu23fJ63yz31S9Nt35NHfHvj2bCkAUkJf5N/KMe5fDv0cyf8T0ANptBzyFtqHBfMQC2bNjHtDErXFb3G3VfwzJ0e6mlY9/fd5IJr3z5923/zmMc3hfN/I/XO+e16lyb5v+pgX+AHwd2H2fCK1+QmpLuqurfVDXqlqJbrwfw8/Pl0IFTjH9jcZbtoHGLinTocj+maTqOg2NW8PueE26q8a3jyjFCs2bNYuLEiVnm9+rVi969ezunT58+TVhYmHM6PDycnTt3OqeLFi1KdHQ0e/fupUyZMqxYsYIzZ85cdf3KCAGheYPpP/EpRnSdzH9rDuXk4TN0G/ZwlnKF7olk9KIXqdemmmV+p/6t8PHx4bm6r/Nc3eEE5PDnsX4tXFX9GxaaO4gXh7VlxKAv+O+jk4g+Hkv3Xg9YyhQskpf/9nmQwX3m0LPzFOZ+vIFh7zzmfP+BlhUZO+Up8oWHuLr6Nyw0bzD9/68rI7p9yH9rDePkkZjs279UJKMX9qde6yvav19LfHxtPFf/DZ6r/zoBgX481teD2j9PTvqP7cTIZ6bTo9Eooo+epdvLrbOUK1Qygrfm9aRuy0qW+XcXDyc+NoleLcY4X57UCQq9Kyf933yEkX3n0qPVeKL/PEe3/s2ylCtUPIy3pj9N3ablLfMbt6nC3UXz8Vzb9+jZ/v+oUL0YdZuVz/L521Fonpz0f/sxRj4/mx5NxxB99BzdBrTMUq5QiXDe+uQZ6javaJlfp2l52jx5P690mcqzLcbhH+BHu271XVX9myo0dxAvvtaOEQM+57+PvE/0sfN0753NcbBvUwb3/oSej3/oOA6OfexvlijXqmvXrqxZsybLq2vXrpZydrsdw8jsoJmmaZkOCQnh7bffZujQoTzyyCOEh4fj5+d31fW7rCP0xx9/8MEHHzBs2DCGDx/OBx98wK5du1y1+n9UtVE59u84zImDpwFYNv07GneomaVc66cbsfKTjWxcbD3I74raz7xxyzBNE7vd5MDOo4QXzOuSut8MVWuVYN9vxznx5zkAvp6/hcbNK1jKpKakMWHkUs6dTQBg/54T3JU3GF9fG3nyBVO7YRle7fOpy+t+M1RtdC/7fz6S2f4z1tP40b9p/0+/Z+OSK9v/d+a9e1n77/qT8IJ5XFL3m6Fq/TLs/+UoJw47zpy+/uQHGrWrlqXcQ13qsuqzH9m47BfL/HurFcWebjLmq958sGogj7/QDJvNc+44qXp/Sfb/eowTR84C8PVnP9HoocpZyj3UqRar5m9l4yrrcctmM8gR5I+fvy9+/r74+vmQeinNJXW/UVXr3sP+nX9y4khG28+NolGbKlnKPfREHVZ98RMbV+y0zG/SvhoLpm8g4UIypmkycdh81npQJ/hyVWuXYN9vJzKPg19toXELa8cvNSWdCSMWc+5MxnHwt7+Ogz4ur++tlo7hsldISAgFCxbM8goJsZ5YR0ZGEhMT45yOiYkhPDw8s87p6URGRvLll18yf/58ypYtS6FCha4aq0s6QnPmzKF///4AVKhQgXLlygEwdOhQpk+f7ooq/KOwu+8i5vh553TMifPkDAkiKFcOS7kPBs3ju/mbs3x++7rfOP7HKQDCC+ah/bMPZOks3c7CIkI4cyrzWmzM6ThyBucgKKe/c96pkxfY/MPvzuln+jXjxw37SEuzc+5MAiMGfsHxo+dcWu+bJaxAHmKOZ9bd0f6BBAVf0f4v/037f/cbx/9wdKLCC+ah/TNNsnSWbmf5CuQm5mSsc/rMydiM+AMs5SYPm893i7dn+byPr42fv9/HkCenMKDD+1RtUJo2HpQVyBcZSkx05mW9M6fiyJkrB0E5r4j/zaV8d0UnEODbRdtJuJDMJ+teZs76Vzhx9Cw/fbf3ltf7ZsiX/4q2j75AzlzZtP3ri/hu6c9ZPl+wWBi58wYzYvp/+eDr/nTu05SEOM+6LP6XsIhQzly2HWQeBzO/i1MnY9n8/WXHwf7N+HH9PtLSPPNSoKepU6cOUVFRnDt3juTkZFavXk39+pnHGsMw6N69O6dOncI0TWbOnEnLllkznFdyyRih2bNns2jRIgIDAy3zu3XrRvv27enevbsrqvG3DJsN08w6dD09/d89W7NkpcIM+6QnS6atY/PqnVf/wG3CZhhkEz7p2fzYTEAOP156rS1hEaEM9tAM0JUMm5F9+9uvo/1n/dX+t0e281rYDIPsfmg6u/bPzsp5Pzr/fykZFn70HW271WfRZeNIbmc229/Ef43t37lnEy6cT+Tx+qPwD/Bl2PtP8vBTdVkw8/ubXNOb729jv8Zjn4+vjSr3l+KNZ2eScimNF995jKf6t2DKm0tuck1vPZthZPt769l9FwE5/Hjp9faERYQwuNedcRy8kt28/bK6ERER9OvXjy5dupCamsqjjz5KxYoV6dGjB3369KFChQq88cYb/Pe//yUlJYXatWvz9NNPX3W5LukI+fr6kpaWNVV88eLFa7p+d6vFHDtLmWrFnNP58ucm/nwil5JS/uFTVg0evo9eYzozaeDcbLMGt7PTpy5Qpvzdzul8YSHEX0jm0hWDfcMiQnjj3U4cPXyGgc/NIsVD0v9XE3P83I23f/v76PXO40z6m6zR7ez0ifOUrlLEOZ0vMpT42EQuJV9b/I0frs7B345zeK9j4LxhGB51hnz65AVKV8xMn+eLCCH+QhKXkq9tsHudB8sx+c2lpKWmk5aazreLt1O3aXmP6AidPhFL6UqFndP5IkKIj7322M+djmPTql+dg6vXLt7O470evCV1vdVOR1+gTPmCzul8Ybkc28GVx8HIUN4Y/zhHD8Uw8JmZd8xx0FP89Yygy3300UfO/zds2JCGDRv+q2W65NLYs88+S7t27RgyZAjvvfce//d//8eQIUPo0KEDzz77rCuq8I+2rfuNMtWLU6C441pjq24NiFqRNQ38d2o2q8hzb3Xk1UfGe9wfQYBtP/5BmfIFKVDIMa6l1SPVidpgTe0HBvkzZspTfL9uL28Nnn9H7fzb1v1GmWqXtf9T19H+ox7j1Q4TPLL9t2/YR5kqRSlQNB8ALZ+4n6jVv17z54uWzs+TL7bEZjPwD/Cjddd6bFi641ZV96bb/sPvlKlYmAJFHOP6Wj5Wg6i1e6758wd+O079jDF1Pr42ajUqy95f/rwldb3Ztm/cR5nKhSlQJKPtH69N1Le7r/nz36/YRb2WFfEPcJxT136wPPt3eUbsV9r24x+UqXDZcfDR+4hav89SxnkcXLuHt1796o46Dnozl2SEWrduTY0aNYiKiuL06dPY7XaqV69O7969iYiIcEUV/tGFM/G822sGQ2Y+i6+/LycPxTDmuY8pVbkIfd/ryvMN3vjHz/d4owMYBn3fyxzh/ttPB5g0cO6trvpNceF8EuPeWMzQ0R3w9fPh5LHzjBm+kFJl89NvSBt6dp5Cm//UIDwylPsbleH+RmWcnx3UczbxHnar/JUunInn3T4zGTL9GUf7H45hTM/pjvYf34XnG434x8/3eP1RR/uP7+Kc99vmA0waNO9WV/2muHA2gfEvzWXwh93w9fPl5NEzjO07h1IVC/HC2x3p1WLMP35+zviV9BzxKJO/GYSPrw8bl/1suVx2u7twLpHxQ75i8PjHHdv/n+cY+8qXlCp3Ny+MaE+vh7Pe1nu5qaOX0XNIG6Z+3Q+73c7PP/7BV9M3uKj2N+bCuUTGD/qCwROfdMR+9CxjB3xGqfIFeWFUB3q1+eeH0X09ZxPBuYN4f3FfbDaDA7uPM+2tpS6q/c114Xwi415fxNB3Hss4Dp5jzLCFlCpbgH5D29Dz8Q9p81hNwvPnznocfG6Wxx8Hr+RNP7FhmNkNjrjNNc/Tw91VcBuzRMGrF7qDGYeOu7sKbmUEBbm7Cu4T7MWxA6R4znOpbgV7qHe3/6ptr7t0fduPFr56oZukauGjLltXdvRARREREbFI96LHDHpPpCIiIiJXUEZIRERELG7H2+dvFWWERERExGspIyQiIiIW3nTXmDJCIiIi4rWUERIRERGLdNN78iTeE6mIiIjIFZQREhEREQu7F+VJvCdSERERkSsoIyQiIiIWumtMRERExAsoIyQiIiIWumtMRERExAuoIyQiIiJeS5fGRERExMKuwdIiIiIidz5lhERERMQi3YvyJN4TqYiIiMgVlBESERERC90+LyIiIuIFlBESERERC/3oqoiIiIgXUEZIRERELNJNPUdIRERE5I6njJCIiIhY6DlCIiIiIl5AGSERERGxsOs5QiIiIiJ3PmWERERExEJjhERERES8gDpCIiIi4rV0aUxEREQs9EBFERERES/gkRmhvSPKuLsKbmO765K7q+BWplnM3VVwK1//dHdXwW3SogPdXQW3sgfa3V0Ft+p7/zfuroJX0Y+uioiIiHgBj8wIiYiIyK2TrgcqioiIiNz5lBESERERCzu6a0xERETkjqeMkIiIiFhojJCIiIiIF1BGSERERCz0o6siIiIiXkAZIREREbGw67fGRERERO58ygiJiIiIhcYIiYiIiHgBdYRERETEa+nSmIiIiFjY9UBFERERkdvL0qVLadmyJU2bNmXOnDlZ3t+9ezePPPIIbdq04ZlnniEuLu6qy1RHSERERCzSMVz2ulanTp1i/PjxzJ07l0WLFvH5559z4MABS5k333yTPn36sGTJEooVK8bHH3981eXq0piIiIi4TVxcXLaZm5CQEEJCQpzTmzZtolatWuTOnRuAZs2asXLlSnr16uUsY7fbSUxMBCA5OZnQ0NCrrl8dIREREbFw5RihWbNmMXHixCzze/XqRe/evZ3Tp0+fJiwszDkdHh7Ozp07LZ95+eWX6d69O6NGjSIwMJAvvvjiqutXR0hERETcpmvXrrRv3z7L/MuzQeDI9hhG5qU00zQt0xcvXmTw4MHMnDmTihUrMmPGDAYNGsTUqVP/cf3qCImIiIjFvxm7c6OuvAT2dyIjI9m6datzOiYmhvDwcOf0/v37CQgIoGLFigA89thjvPfee1ddrgZLi4iIyG2vTp06REVFce7cOZKTk1m9ejX169d3vl+kSBGio6M5ePAgAGvWrKFChQpXXa4yQiIiImJxOz5HKCIign79+tGlSxdSU1N59NFHqVixIj169KBPnz5UqFCBt956i759+2KaJnnz5mXUqFFXXa46QiIiIuIRWrduTevWrS3zPvroI+f/GzRoQIMGDf7VMtUREhEREYv02zAjdKt4T6QiIiIiV1BGSERERCzsLrxrzN2UERIRERGvpYyQiIiIWGiMkIiIiIgXUEZIRERELOymxgiJiIiI3PHUERIRERGvpUtjIiIiYpHuRXkSdYQyNCpSjIG16uHv48PeszEMWruahNQUS5nB9zegZYl7uHDpIgAHz5+n1+qvyeXvz9uNmlHirjzYDIP5e3fz4Y4t7gjjujUqUIIBlRo54o89zcs/LiMhzRr/q1Wa0LJwGWJTMuKPO0ufHxYR6p+DEfc15967IkhKS+WrgzuZvX9rdqu5bTUqUIKBlRs64j9/mkE/Ls8S/+CqjWlRuExm+8efpff3iwn1z8HIGs0y4/9jJ7P2b3NHGNetYWRJXqzQCH8fX/bFnuLVrV9nif+ekDCGVWlOLr8A0k07Q7ctZ3dstPP9yMAQvmzyFG1Wf8T5lGQXR3BjGhUtxsDaGfv/mRgGrclm/6/bgBYl7+HCxYz2jz1P75VfE+DjyxsNm1ApIhID+PlUNMO+W8Ol9DQ3RPLvNS5UnIH31cff5sPeczEM3LgyS+xDajakZbHSxP617V84R6+1SwF4smxlOpauSA5fX3adOcXADStJsae7PI7rdWRrEj99Ekt6qkneov407JUX/yBrJ+DQj0lsmReLYUBALhsNeuYlNL+f8/2EmDQWDIqmw4T8BIb4uDoEuUHqCAF5cgQypnFzHl0wj8MXYnm5dj0G1a7H0A1rLOWqRRag9+plbI8+YZnfv+b9RCfG03PVUgJ9ffmm01NsPnGM7adOujKM65YnIIi3az3Ef76ZzeH48wyq3IiBlRsxbOsqS7mqYXfT54dFbD9z3DJ/SNUHSEpLoemyqfgYBlPqP8qxhFjWnjjgyjCuW56AQN6p3YoOqz/JiL8hA6s0YtiWK+LPV5A+3y/OEv/Qag+QmJrKg19/5Iz/z8QLrD3uIfH7BzH6vtY8tm4mRxLOM6BCY16q0JjhO1Y6y+Tw8WVG/cd5devXrI/+gyYF7mFczXY0X/UhAO2KVKBPuQZEBoa4K4zrlidHIO80aU6Hrxz7/6A69RhYpx7D1lv3/6qRBeizMuv+3+u+mvjaDFrMnYVhGIxv2pKe1Wsw/qdNrgzjuuTJEciY+s15ZOlcDsfF8vJ99Xn5vvoM2fStpVy18LvpvXYp205bY29etBRPlavKw0vnEnfpIpObtOXp8tWYvHOzK8O4bskX0ln3/lnavRVJ7gJ+/DjrPD/OPk/9Z/M6y6RdsrNm/Bk6TMhPaH4/flkSxw/TztNyaDgA+9YlsHVeLEnnPKfzdy00WNrL1CtchJ2nozl8IRaAT3/9hbb3lLWU8bf5UC5fOM9WuY+VHbswuXlrCgTnAuD1jet484f1AITnDMbfx4e4lEuuDeIG1MtfjF1nT3I4/jwAn/6+nbZFy1nK+Nt8KHdXJP8rW4sVLf/LB3UfpkCQ449e+Tz5WXjoV+ymSardzrrjf9C8cBmXx3G96uUvzk5L/DtoW/ReSxl/mw/l8kTwzL21WNHqaT6o1/6y+COt8Z84QItCpV0ex/WqG1mcXedPcCTBEf/cP7bRpkh5a5mI4hxNjGV99B8ArDmxnxd+XABAeI5gHixQmu4b5rq24jdJlv1/1y+0LZ3N/h8WzjPV7mNFpy580CJz/998/BgTt/yECdhNk99iTnN3Ls/oENa/uyg7Y6I5HJcR+56faVsy67Z/b95wnqlYg1UPP8WHTdpSIKcj9odLluOjXVu4cOkiJvDqD6tZcOA3V4dx3f78OZnwkgHkLuDI7tzbPBcHNiRimqazjGkHTEhJtAOQlmzHx8/RSUg8l8bhn5Jo9VqEy+suN486QkCB4BBOJsQ7p08mxBMSEECwn79zXnjOnGw6fpSxP31P889ms+PUST5q2c75frppMv6BFqzu2JUfjx/jYOx5l8ZwI/IHhXAyKc45HZ0URy7/HAT7XhZ/YDBRpw7z7s4NtFg+jR1njzOl/qMA/HL2OO2LlcfXsBHk60ezQqUJDwx2eRzXK39Qrizxh2QT/6boI4z7ZT0tln3Mz2dOMLXBIwD8fOaEJf7mHhZ/ZOAV7Z8cRy4/a/zFcuXlzMUERlV/iAVNujOzfmd8DMfh4/TFBJ6P+orDCedcXvebIX+uEE7GZ+7/0dnt/8E52XTsKOOivqfFvNn8HH2SqQ859v+Nfx7hUMb+fneuXHSrVJXlB/a7NojrlD9nLk4kXnbsS4wnxN8ae0RQMFEnjzJ22/c0WzCTHadPMO3B9gAUD72LvDmCmNXsUVY+/BT9qt7vUSeBiWfSCc6XeSkrOJ8PKUkmqcmZHSG/QBv1nsvDwpejmd3tGL8uj6dW19wA5MzjS7OXw8l9t1+WZXs6OzaXvdzNJTU4ceLEP77czTDAxMwyP920O/9/LD6Obl8vZP+5swBM3bGVwqGhFLzszK/ftyuoMv0DcufIwQv31b71Fb9JbBjZRO/o3P3lWOIFun/3BfsvxADw0Z6fKJzrLgrmDOXN7WswTVja4mmm1H+UH6IPkepBYwRshoGZzReQffxnAJh6Rfxg8nXL7kyp/wjfRx/2vPizmX95/L6GjQaRJfn84HYeXjOdTw5sYVq9jvjbPH88hO1a9v+4OLovzWb/D8nc/8uHhfP5Ix2ZvWsHaw8fvPUVvwmupe3/TLjAU6vms/+8Y9ufsmsLhUNyUyg4FF+bD/XuLsrza5fQetFsQgNyMKB6XRfV/saZdsjuJ7WMy/4ynj2cwrbPL/DY+wXoMmVwgAEAACAASURBVKMgVTuEsvrtGEvWSDybS8YIPfPMMxw+fJjw8PAsG49hGKxZs+ZvPukaJ+LjqRKR3zkdGRxM7MVkktMyBzuWyZuPsnnDWLh/j3OegUGa3U79QkXYe/YMp5MSSUpNZcnve2levJRLY7gRx5MuUClfAed0RGAuYi8lk5ye6pxXJncYZXJHsOjwr855BpBmt5PLL4DRP6/lQsYg6p731uFIvOdkxE4kxlE5b2b8kUHZx1/2rggWHso+/rd2rMuMv1xt52U2T3Ai6QKV8lze/iHEpljjP30xgT/iz/DLOceJy5oT+xlVvRWFcubmj/izLq/zzXQiPp7K17L/5wtj4b4r9v90R2fpoVKlGdGwCa+tX8uS/XtdV/kbdCIxnsrhl8WeM1dG7Jdt+3nCKJsnjIWXXfIyMEi1p3MqKYGVh/c7B1cvOvAbfap4zklgcJgPp3/PzGAlnk0nINiGX47MntCfPycTWTbAOTi6XItcbJp+novx9jt6YHS6xgjdXPPmzaNYsWK88847rF271vJydycIYOOfh6kckZ+ioY50Z+dylfjm0B+WMnbTZHi9xs4M0BPlK7H3bAzRiQm0KlmavjUcO7+/zYdWJe8h6vhR1wZxA74/eYgqee+maK67AOhcqirfHrOm9u0mvFb9QQrmDAXgiVJV2Rt7mujkeB4vVYV+FesDkC9HTv5TshJLDu92bRA3YOPJQ1TJlxn/46Wq8M2x3y1l7KbJa9UeuCL+GGf8/Z3xB/FYiUosOew54yS+P3WQynnvpkiwI/5Oxauy5ri1/defPEDBnLkplzsSgPvyFcY04c/EWJfX92bbePQwVSIz9//Hy1fim4NZ9//X6jd2ZoCeqFCJvWcc+3+TosV5rX5juiye71GdIIANxw5TJbwARUMyjn1lKrH6qHWQv900eb12EwoFO7b9J8tWZu/5GKKTElhxaB+tipchwMdxTt20SEl2xkTjKQpVDuTUvkvEnnB0/H5bFU/RGoGWMmHF/Tnx60WSYh1Z3sM/JZEr3PeO7gR5G8N0UX5v586dfPnll4wYMeKGl1V00ribUCOrhkWKMahWXfxsPhyJi6X/tyspHBLK242b0vLzTwBod09ZnqtaAx/D4GRiAoPWruJEguOa+psNH+CePPkAWHXwd8Zv3pRtyvlG2e66NdffGxYowYBKDfGz+XA04TwvRi2lcHBu3qrZiodWfAxA26LlePbeOvgYBtFJ8bz80zJOJMWR09efcbVbUyRXHgxg8m+bWHyLOkLmLTpLaVigBAMrN3C0f0IsL25yxD+6ZktarZgOQLui5Xi2XG1n/IN+XO6M/906rSmS6y5H/LujWHSL4vf1vzWX3BpEluDFCo3xz2j/AZsXUyj4LkZVb0Wbb6YBjs7PoIpNCPT1I8Wezsgdq9l29k/Lcn7vMIQai8fdktvn06IDr17oOjUsUoyBdTL2/wuxvPjNSgqHhjK6cVNafZax/5cuy7PVHPt/dEICg9Y49v81T3QjNEcOTiUkOJe37eSJLHed3Sh7oP3qha5Do4LFHLfP+ziOff3WL6dwrlDerteclgtnAdC+5L08V7EGPjYbJxPjGbhhJScS47EZBr0r16Z18dLYDBu/nj3Fq99nffTAzdD3/m9u+jIBjmxNZvOn50lPMwmJ9KPxC3mJO5XG+oln6TDBkSn9dXk8vy6Pw8fXICDYRt3/5SFPYX/Lcj5sd4Suswvesg5Sv7Krb8ly/84LOzq5bF3vVZnnsnVlx2UdoZvpVnSEPMWt6gh5ilvVEfIUt6oj5AluZUfIE9yqjpCnuFUdIU+hjtCto+cIiYiIiIXddP/dXK7iPZGKiIiIXEEZIREREbFIz+65AncoZYRERETEaykjJCIiIhb6rTERERERL6COkIiIiHgtXRoTERERC90+LyIiIuIFlBESERERC7tunxcRERG58ykjJCIiIhbpun1eRERE5M6njJCIiIhY6K4xERERES+gjJCIiIhY6Cc2RERERLyAMkIiIiJioecIiYiIiHgBZYRERETEQmOERERERLyAMkIiIiJioecIiYiIiHgBdYRERETEa+nSmIiIiFhosLSIiIiIF1BGSERERCz0QEURERERL6CMkIiIiFhojJCIiIiIF1BGSERERCyUERIRERHxAsoIiYiIiIUyQiIiIiJewCMzQu+1+MTdVRA3yWNLcHcV3CoVH3dXwW3SvehHILPTJDDd3VVwq7P2RHdXwavcrhmhpUuXMnnyZNLS0ujatSudO3d2vrdnzx5efvll5/S5c+cIDQ3l66+//sdlemRHSERERLzLqVOnGD9+PAsWLMDf35+OHTtSs2ZNSpYsCUDZsmVZvHgxAMnJyXTo0IHhw4dfdbnqCImIiIiFK58sHRcXR1xcXJb5ISEhhISEOKc3bdpErVq1yJ07NwDNmjVj5cqV9OrVK8tnp0yZwn333Uf16tWvun51hERERMRtZs2axcSJE7PM79WrF71793ZOnz59mrCwMOd0eHg4O3fuzPK5+Ph4vvjiC5YuXXpN61dHSERERCxcOUaoa9eutG/fPsv8y7NBAHa7HcPIrJdpmpbpvyxZsoQHHniAvHnzXtP61RESERERt7nyEtjfiYyMZOvWrc7pmJgYwsPDs5T79ttveeaZZ655/d59G4aIiIh4hDp16hAVFcW5c+dITk5m9erV1K9f31LGNE12795NlSpVrnm56giJiIiIhd00XPa6VhEREfTr148uXbrQrl07HnroISpWrEiPHj3YtWsX4Lhl3s/Pj4CAgGtermGapvmvvyE3W3ywsrurIG6i5wjpOULeSs8R8u7nCIUVOO7S9T34XT+XreubhuNdtq7saIyQiIiIWNyuD1S8Fbz7FEtERES8mjJCIiIiYqGMkIiIiIgXUEZIRERELExlhERERETufMoIiYiIiIUrf3TV3ZQREhEREa+ljJCIiIhY6K4xERERES+gjJCIiIhY6K4xERERES+gjJCIiIhYaIyQiIiIiBdQR0hERES8li6NiYiIiIUGS4uIiIh4AWWERERExEKDpUVERES8gDJCIiIiYmGa7q6B6ygjJCIiIl5LGSERERGxsKMxQiIiIiJ3PGWERERExELPERIRERHxAsoIZdizOZ0VM1JJS4X8xQw69PUnR05rj/jXH9JZ/Wkqhg2Cchk82sePvAUcfcnhjyUTmi+zfINHfKna2HO+Xm+Pf+dPMH+GjbRUKFgMnupnJzCntcz2H2DxJzZsBuTMBV362gkvAPZ0mDPJYP8uR/wV7jPp0MPE8KATql0/weIZkJoR/xP9yBL/zz/A15+AYUBQLniiL4QVgKkjIOZEZrkz0VCqIvR83bUx3IhffzJZMsMkLRXuLgaP9zMIvGL7/+UHk2WfmBg2R/t3esEgrICjzIalJptWmqSmQOGSjs/7+XvGBvBdFIyfCimpULo4jBwEwVe0/afzYc5CyBEAxQvD0H6QO8Tx3tyF8NUyuHQJypWGkQPB39/1cVyvTVE+TJnmT0qqQYnidl4ZcJGcV8T/1QI/5i/yI8DfpEgROy++cImQEEhIgNFjcnDkqIFpGjRvlsoTnVLdE8hNpucIeZmEWJMv3k3hySH+DJyWg7yRNlbMsG7MqZdM5o1JoctQf/pNykHZmj4s/tBR5vQxO0G5DPpNyuF8eVInwNvjj4+FGeNs9Bxq582P7YRFmsyfbj0IpFyCaW87yrw22U7FWibzJjt2n6g1BqeOGbz+oeO9/bsMtm10RyTXJz4WZo+D/w2F1z+GfJGwaLq1TMolmPG2o8zgyVCxFnwx2fHeX/MGT4bOfSEoGDo+7/o4rld8rMmn75r8d6jBsI9t5M0PS2ZY7x1OuWQy6x2THsMMXvnARvmaBl9NdpT5+XuT9UtMeo82GDzFICUF1i10RyT/3rlYGDwa3hsBKz6FggVg3BRrmZ+2w7R5MONdWPgx1K8Fr411vLd6A8xZANPfhaWz4OIlmPWl6+O4XudjYdQ7AYx8/SLzZidRIL+dyVMDLGW27/Bhzjw/3huXzMxpydSumc4743IAMG26P2Fhdj6ZkcxHk5NYtNiPX3frz6qnUYsB+7enU+geG2F3O76OWg/5sGNdOuZlD1Kw2x3/Xkx0/JuSbOKbcdZz5Dc7Nht88NIl3n3uIt/MScWe7jkPYfD2+HdvNyhaGiLudkw3fMjkp7WG5Tkaf8WfnOT491Iy+PllvnfpoiObkpbx8vWgM+I926FoaQjPiL/+Q7B5LVniN7HG7+tnXU5aKswaCx2ehTzhLqn6TbF3OxS5B8LvdnR+67Uy2LIWy/Zv/hV/xvZ/KTmzjTevMWnysEHOXAY2m0HH3gY1mrg4iOv0wxYoXwaKFnRMd2oLX39rbfvd+6F2NYjMaNMH68O6TY4M0pJV8NRjjuyQzQbDX4Q2TV0fx/XassWXsqXtFCroCLh921S+WeNriX/vfhvVq6UTHuaY2aBeGj9E+ZCaCi/0TuH551IAOHvOIDUVcub0nGPfPzFN173czWWn7d9++y0nT56kQYMGFC5c2Dn/888/57HHHnNVNbJ14YxJaFhmBiA0n8HFJLiUBDkyUqQBgQYP9/JjUv9LBIU4Dow9xznOHOzpULKyjZbd/UhPg+mvpZAjKJ167T0jK+Lt8Z+LgTz5MvfGu8IgOcnxHfx1eShHIDzR22R0Pxs5czk6Bi+/6+gd3f+gydYNBgM620hPh3JVoXItd0Ryfc7HwF35Mqdzh8HFJLLE/3hvGNsPZ/wvvWtdzg8rITQPVL7fdXW/Gc7HOGL+S3bxBwQadOwN7/Y3CcplYtqh/7uOfeb0cYi/AJMG27lwFkqUh3b/9YzLCtGnIf9lndaIMEhINEhMMp2XxyqWdVwaOx4Nd0fCwhWQmmoQG2dy+E+oUAZ6DIDTZ6BaRXjpWffEcj1OxRiEh2fu+2FhJomJBklJOC+PlSubzlcL/IiONoiMNFm+0o/UVIMLcQb58pr4+sAbbwbw3Xpf6tVLo3Ch2+Avu/wrLskIjR07lk8//ZTDhw/TqVMnFi9e7Hzvs88+c0UV/pFpJ9snJth8Mv9/8pCdb+em8dKUAIbOCaRxRz8+GZmCaZrUbOFLu57++OcwCAw2qN/el183pbus/jdK8ZPtF3B5/McOwdI5Bm9MtTNunp1WnUwmj7BhmrDkU4NcuU3e/czOmDl2EuNh1Vee8YcQMrJdV4n/+CFYPgeGTYXR86B5J8fYoMvP5tYuhJaP3/Lq3nSmSbbjuazxm6yYYzJ4isGouTaadTSYNsLENE3S02DvdpPurxoMfN8gKR6WzvSMP4Z/ZTqvZLvsL0P1StDzKeg9BB79n+O7Cg0x8fOF1DTYtBXGD4cvp8KFOJgwzRU1vzlM+9+0/WXxV6pop3uXFF4dloOnnwnEMExCQkz8fDPbeNjgS3y9OJH4OIOZsz0oHfwPTNNw2cvdXNIRWr9+PdOmTWPo0KHMmTOH9957jxUrVgDW9LO75A43iDuXWY+4MyaBweCfI7OB9m9Lp+i9Nufg4DoP+RB9xCQpDratSePkocwjimmCj2ckQwDFnyccLpzNjDX2DAQFmwTkyCyze6tByXIm4QUc041bmxw/AglxsP0Hg7pNTXz9ICgn1HnQZN8v7t+5r5Uj/sxpR/xY4v9tKxQv5xgcDdCwNZw4Aolxjuk/D0B6umOQtKe5K8wa/wVn/JltuGfbX/E75tW/LP7QvFDpfsfgal8/g/saGxza4+oork/+CIi5LPZTZyA0l0lQYOa8xCS4rxIsmAZfTYUmdR3zc4dAeD7HpbLgnODvB62bwi+7XRvDjYiIMDlzJrOdz8QY5MplEnhZ/ElJULlyOtOnJvPxlGTq3e84yQsJgZ82+zg/HxQIDzRJY9/vGnHiaVzSYqZpYmR0u4sWLcqUKVN48803+emnn5zz3emeqj4c3Wsn5rjjj/mPy9MpV9vHUubukjYO7rITf97RYdgdZSdPhEHOUINTh01Wf+IYF5N6yWTT0jQq1ffJsp7blbfHX66ayR974dRxx/R3ywwq17Z20AuXMtm/y+DCecf0jk2QLwJyhUKRkiZbNji247Q0+PlHg+Jl3d/Bv1Zlq8GhvY5LPAAbl0Gl2tYyhUrB77sgLiP+nzPiDw51TO/fCaUrZ392fbsrWw0O74XTxx1ttnGZSYUr4y8JB3ZCXMb2/0sU5I2A4FCDynUNdmwwSbnkyBDtjDIpco+ro7g+998Hv/wGh485pj9fAo2vuLR5+gx07QsJGeOjPvwEWjVxtHWzBrBynWOQtGnCmo2OMUeeokb1dHbvsfHnMceGu2ipH/XuT7OUOXPGoHffQBIz4p/1qT8PNE7FMGDtd75Mn+WPaUJKimO6WhXPyYaLg2G6ICUzceJENm3axMsvv0zFio5Txm3bttGrVy9SUlLYtm3bv1re4oOVb3od92xOZ+XMVNLTIE9+g44v+XP2pMlX76XQb5Lj1HjT0jQ2LU3DxxcCcxm06+lHZBEbKRdNFn2QytG9dtLToWI9H5p39b0tOnnXylPiz2NLuOnLBNi5GRZMt5GWBuH5ofsAO2eiYdZ4G69NdnQQ1y4xWLvEwNfXMU7m8eft3F3UkRWaO8ng6AEDmw3KVDH5Tw8zy2DimyGVW9PB/HWz406x9DTIlx+eGuC4Df7T8Y67wQC+WwLrlziyfTlzwWPPQ4GijvfmTXSMD7qVl8bSzVt33rZ7c8bt8xnxdxlgcOYkzJ1g8soHjvWuX2KyYamJr6/j8QH/6WmQv6iBPd1k5TzYvsHEnu7oNHXsk/X2+xvVJPDW/IFd/6Pj9vnUVCh0N4x+FY6dgKFjHHeJgePOsLkLwW5C1QowtK/jVvr0dEfHaMVaSLfDvaXg9Zey3n5/M5y1J978hQJRP/rw4Uf+pKUZ3F3AzpBXLnLipI3RYwKYOS0ZgPkL/ViwyA+7CRXLp9P/hUsEBEB8Aox9N4CDhxzbSP166Tz9VIrl0trNElbg+M1f6D+osOQ1l61rVxv3PmvDJR0hgKioKMLDwylRooRz3smTJ5k+fTqDBw/+V8u6FR0h8Qy3qiPkKW5VR8gT3MqOkCe4VR0hT3GrOkKeQh2hW8dlIzlq166dZV7+/Pn/dSdIREREbi09UFFERETEC3jQvT0iIiLiCrfBDd0uo4yQiIiIeC1lhERERMTidnjQoasoIyQiIiJeSxkhERERsVBGSERERMQLKCMkIiIiFl5005gyQiIiIuK9lBESERERC40REhEREfECygiJiIiIlRcNElJGSERERLyWOkIiIiLitXRpTERERCw0WFpERETECygjJCIiIhamBkuLiIiI3F6WLl1Ky5Ytadq0KXPmzMny/sGDB3nyySdp06YNTz/9NBcuXLjqMtUREhEREQvTNFz2ulanTp1i/PjxzJ07l0WLFvH5559z4MCBy+ps8txzz9GjRw+WLFlC2bJlmTp16lWXq46QiIiI3PY2bdpErVq1yJ07N0FBQTRr1oyVK1c639+9ezdBQUHUr18fgGeffZbOnTtfdbkaIyQiIiJWLrxrLC4ujri4uCzzQ0JCCAkJcU6fPn2asLAw53R4eDg7d+50Th89epR8+fLx6quvsmfPHooXL87QoUOvun5lhERERMRtZs2aRZMmTbK8Zs2aZSlnt9sxjMwOmmmalum0tDQ2b95Mp06dWLhwIYUKFWL06NFXXb8yQiIiImLhyrvGunbtSvv27bPMvzwbBBAZGcnWrVud0zExMYSHhzunw8LCKFKkCBUqVADgoYceok+fPlddvzJCIiIi4jYhISEULFgwy+vKjlCdOnWIiori3LlzJCcns3r1aud4IIAqVapw7tw59u7dC8DatWspV67cVdevjJCIiIhY3YbPEYqIiKBfv3506dKF1NRUHn30USpWrEiPHj3o06cPFSpUYNKkSQwZMoTk5GQiIyN55513rrpcwzQ977FJiw9WdncVxE3y2BLcXQW3SsXH3VVwm3TTuxPYTQLT3V0FtzprT3R3FdwqrMBxl66v+NxRLlvXwcdfddm6sqOMkIiIiFjot8ZEREREvIAyQiIiImLlcYNmrp8yQiIiIuK11BESERERr6VLYyIiImKhwdIiIiIiXsAjM0IPBJ53dxXcJslMc3cV3MqG95ylZMfPi89dgm053F0Ft7Jjd3cV3Kr2py+5uwpudWCgi1eowdIiIiIidz6PzAiJiIjIreQ92XdlhERERMRrKSMkIiIiVhojJCIiInLnU0ZIRERErJQREhEREbnzKSMkIiIiVnqytIiIiMidTxkhERERsTA1RkhERETkzqeMkIiIiFgpIyQiIiJy51NHSERERLyWLo2JiIiIlW6fFxEREbnzKSMkIiIiFoYGS4uIiIjc+ZQREhEREStlhERERETufMoIiYiIiJXuGhMRERG58ykjJCIiIlYaIyQiIiJy51NGSERERKyUERIRERG58ykjJCIiIlbKCImIiIjc+ZQREhERESs9R0hERETkzqeOkIiIiHgtXRoTERERC0ODpUVERETufMoIiYiIiJUXZYTUEcqwMcrG+x/5kpoKpYqbDBuYSnBOa5nPFvjw+UIfAvyhWBGTl/umEhpiLfPiUD/C8pq83DfNdZW/CTZF+TBlmj8pqQYlitt5ZcBFcl4R/1cL/Ji/yI8Af5MiRey8+MIlQkIgIQFGj8nBkaMGpmnQvFkqT3RKdU8g1+mHKB8+nOZPaiqUKG7n1QGXssT/5QLfy+I3eemy+EeNCeDIURumCS2apfGkh8X/fZSNSdP8SMnY/ocMSMmy/X++wIcvFvlmbP92Br6QdfsfMMyfsLwmA1/wrPi/i4LxUyElFUoXh5GDyBL/p/NhzkLIEQDFC8PQfpA7I/65C+GrZXDpEpQrDSMHgr+/6+O4HuujYMJUGympcE9xGDHIniX2OfMN5i40CAiA4oVNhvQznbHf38ZGRFhm2e4dTR560HP+ijYsXoyX6t+Pv68P+06f4ZWV35CQkpJt2QdKlmDsQ82pPGESAKE5AnijaRPKhoeRnJrKV7t+45PtP7uy+nIT6NIYcD4Whr/tx9g3Uln4SQp3FzB5f6q1j7hlh42Zc335cFwKn32cwv210hk51s9SZuY8H3bs9Lyv9HwsjHongJGvX2Te7CQK5LczeWqApcz2HT7MmefHe+OSmTktmdo103lnXA4Apk33JyzMziczkvlochKLFvvx627P+R7Ox8Kb7wQw6vWLfDY7mQL57Xww1fpXbNsOG5/O8+P/xl1k1rSL1K6Zzuhxju9o6nR/wsNM5sxI5uPJySxc7MsuD4v/jXf8efv1FObPvsTd+e1MnGrdtrfusDF7nh8fjLvE3GmXuL+mnVHjrN/R7Hm+/OyB2/+5WBg8Gt4bASs+hYIFYNwUa5mftsO0eTDjXVj4MdSvBa+Ndby3egPMWQDT34Wls+DiJZj1pevjuB7nYmHIaBsTRthZ9qmdggVM3p1ivW36p+3w8TyDj9+1s+BjO/VrwfCxjnY+dBRCc8GCj+3Olyd1gvIEBvJ2i6Y8v/hrmk6bxdELFxjQoG62ZYvclZtXGtXn8m9ncOOGJKWk0vzj2Tz6yWc0KFaURiWKuaLqchO57Kh1+PBhTp06BcCXX37JyJEjWb58uatW/4+ittgoV8ZO4YKOHbhDm3RWfOuDedn+vGefQc1qdiLCHdNN6tnZEGUjNePEd+sOG5s223i0TbqLa3/jtmzxpWxpO4Uy4m/fNpVv1vha4t+730b1aumEhzlmNqiXxg9RPqSmwgu9U3j+OccZ1NlzBqmpkDOn5xwMN2/xoWzpdGf8D7dNY/UV8e/bb+O+y+JveFn8/Xqn0OuK+IM9KP4ft/hwb+nM7f+RtumsXHPF9p8R/19n/o3qpbPxsu1/2882orbYeLiNZ2VCAX7YAuXLQNGCjulObeHrb7HEv3s/1K4GkRn7/4P1Yd0mRwZpySp46jFHdshmg+EvQpumro/jemzaYlC+DBTJiL1jW5Nl3xqW2H/bb1CrmumM/YH6Jt9lxP7zrwY+Nniyl4323Wx8MNMg3YMOgXWLFWFndDRHzscCMHfHTtrcWyZLuRy+voxr1ZxR69Zb5pePCGfR7j3YTZNUu53vDh6i+T2lXFJ3uXmu2hGaPn06VatWpVKlSrRs2ZK+ffsyefJk1q5dy7Fjx65pJTNnzuTpp5+mY8eOvPLKKyxbtoxixYoxf/58Jk2adMNB3KhTpw0iwjL3/PAwk4REg8SkzDLl77WzZYeNE9GO6cUrfEhNNYiNg5gzMGaiL28OScXmeSfEnIoxCA/PjD8szCQx0SDpsvjLlU1n2w4foqMd50PLV/qRmmpwIc7AMMDXB954M4Au3YKoXDmdwoU8pyNwKsZGxFXiv7esnW07fDiZEf+ylb5Z4h/+ZgBPdAukSmW7h8VvWOIPz4jfsv2XtbN1h80Z/9KVPhnxO7b/ce/7MWJwCj4euP1Hn4b84ZnTEWFk2f8rlnVkRo5n7P8LV+Dc/w//CefOQ48B0LYbTJwBuYJdG8P1OnkaIi9r++xjN/lpu+E89i1cYTjbPi0dalUzmTLGzqz/s/PDFoM5CzznQXz5c+XiZHyCczo6Pp5cAQEEX3Fdc2SzB/jsl13sPX3GMv+Xk9G0K1cWX5uNID8/mt1TkvArryt6KMN03cvdrnrYmjJlCu+88w7ffPMNI0aMoGbNmpw6dYqPPvqIdu3aUb16dTp16vSPy5g/fz7Lly/n008/ZeXKlUyZMoXOnTszefJkVq1addOCuV52E4xs9t3LD+pVK5r8r2saLw31p/P//LHZIDTExGbAKyP8efH5NMLyuq7ON5Npzz7+yzt1lSra6d4lhVeH5eDpZwIxDJOQEBM/38yteNjgS3y9OJH4OIOZsz1kgATXFn/lina6d0nllWEBdH8mB4ZBlviHD77E8sVJxMXBjNl+ardrwAAAIABJREFUWRd4m/q7+C/f/qtUtNOjSxoDhvnT5ZkADMOx/RsGDB7hT7/nU8nnodu/3Z79/Mvbv3ol6PkU9B4Cj/4PZ/x+vpCaBpu2wvjh8OVUuBAHE6a5ouY3zrRDdt2Wy2OvVgl6PmXSZ4iN//zPhu2y2Du0Nhnc1yQoEEJyQdf/2Fmz0XM6QjYDTDPrX+J0M3Oj6Fy5Iml2O1/t2p2l3Kh1GzAxWdK18/+3d+dhUdX9/8dfw6ZgIJoCJvUtl1Ir3FLRck9U0NSkO7PS2y3NLak08+dSmt5uSaVlamlUWOmdmlYiluZdQYtWkmnlnigKSoq7yMzvD2z0BMpocAY4z8d1zXV1hg/nfF4xjG/e53PmaG63+/T1nj+UnXOZFxSKrQIXS1933XVq1aqVvLy8FBQUpIYNGxq+npqaqu3bt19xH3a7XT4+Pqpatar69u2rMmUurj/JKQZ91JAgh7Zsu/ibn35YCvB3yNf34piTp6QGde3qGpU73/QMae5CL6Wm2bT/gE2zXs39X3kk06Ycu3TunDR+VMk4TRAc7NDWbRffvA5n2OT/t/ynTkn16uWoU1RupowMm95YJAUESN9+56nq1eyqVCn3DfHetuf1xf9Kzjr84GC7ftl2cb4Z+eQ/eUqqXy9HnS/Jv+BC/m8u5K98IX+7tjn64n+eZse4ZsHBxtd/RoYt/9d/Pbu6RJ2VlPv6n7fIW/vTPLQ/zabY13ILvyOZNtkvvP7HjiwZC6arBEsp2y5uHzoslffP/Vn+5eQpqVFdKTrqwpgM6ZWFuafDgirlnir7qxHQOUKaG2fe/P+J3Ow2/XWJ0F/vfX/Pflddh7pH5Y45lCHNXmhT+QBp5Rqbbqvh0G3Vc8c6HJJXyfnV14Gs46pbpYpzO9j/Oh09fUansy++d99/5+3y9fLSyt4Py8fTU2Uv/Hf/D1fI0+ahaV98qWNncn8vHg9vpL1Hj5qeo0hwi42LHnvsMS1devmVf6GhoWrduvUV9xEREaFHHnlEOTk5GjZsmCTp119/Vc+ePdWxY8ernHLha9rIrp+3euiP1Nwf/IcrvdTybmOBlnHYpsdG+OjEydztN9/1Uvs2Oap7u0Orl57V+2/mLqLufl+OIlrnlJgiSJIa35WjX7Z5aN+F/CtWeav53cb5Hz5s07ARvjp5IX/cuz66t022bDZp3RdeWhjnI4cj9x/AdV94qWF99xe4rsrN73lJfq988w8ZUdaZ/613vdWuzfkL+T21MM77kvyeJSp/+F052rLtktf/Kk+1yOf1P2hEGefrf+G73opok6Ow2+36ZEnuAurFb5xV9/vOq13rnBJTBEnS3Y2kzVulPRfO9H+wUmpzt3FM+mGp9wg587/+jhTVNrcz1L6llLA+d5G0wyF9/mXumqOSoFkjh1K2Snud2W1qc7exQ5J+WOozwsOZfd47NkW2ze0Gbt8tzVnooZyc3PzvLfdQh9bF4FyHi77cs1f1bgjR/1UIlCT1rBemz3bsNIzp/s57ilz0ju6Li1e//67QmfPndV9cvNJPnFTPemEacU8zSdL1fn56IOxOrdz6q+k58M/YHPn1BS9Rv359ZWdnq0WLFmrevLlq166t2267Tb6X/rnogu+//16NGjVybu/atUv79u1Ty5Ytr3rSJ9P+76q/pyBffXPx8vnQGxyaNCZb+w/YNHGGt95/M3ch7PvLPLVkRe4i0np32vXME+dV1nhxlV5f5KWjx1Rkl8+fchTNfpO/8dTrC3x0/rxNVW+wa+yzZ3QgzUNTZ5TRW2+cliR9uNxby1Z4y+6Qwu7I0ZNPnFWZMtLxE9LMWWW0a3duXd2ieY76/ftckayX8si3kf/PJX3jqdcXeCv7Qv7xz57V/jQPTZ3ho7g3zkiS/rs89/J5x4X8Tz1xzpl/+iX5WzY/r/7/Lpr1Yt5FdH3D19946NUF3so+n/v6f+7Zc9qfZtMLM3y0+I3cv3aXLPfU0hVesjukenfYNfKJ7Dyv//lveenoMVuRXD5/nUfZQt/nXzZ8k3v5fHa2dGNVaeoYKfWANG5G7lViUu6VYYuX555Kb3CnNG5E7qX0OTm5hdHqdVKOXapTU3r+6byX3/9TdhXNKZf/fSPFzvfQ+QvZp4yxK/WANH6Gh5a9mXvM+GU2vb/cdiF77umwsmWk02ekyS/ZtHmrTefPS+1bOfTEAEe+p1r/qVpvDyn8nUpqWe1mPd3iHnl7euiPo8c08pME3RgYqCnt79V9cfGGsVUDAvRp30dV98Ll8+V8vDUzqoP+LzBQNptNr3/znT4qokJox6iYItnv5VR7aZZpx9o14knTjpWfAguhP/74Q7/99pvz8euvv+rAgQMKDQ112/qeoiiESoqiKoRKiqIqhEqKoiqESoKiLIRKgqIqhEqKoiqESgoKoaJT4Nncm266STfddJPatWvnfO7UqVP6/fffi3RiAADATUrOGc5/7Jr+vPTz81O9evUKey4AAACmsm6fHQAA5Ku4fo7QqlWrFBkZqYiICMXHx+f5+pw5c9S6dWt16dJFXbp0yXfM35WgCx0BAIBVHTp0SLGxsVq2bJl8fHzUo0cPNWnSRDVq1HCO2bJli2bNmqX69eu7vF86QgAAwMhh4sNFSUlJCg8PV2BgoPz8/NS+fXslJCQYxmzZskXz5s1T586dNXHiRJ09e7bA/VIIAQAAt8nKylJqamqeR1ZWlmFcenq6Kleu7NwOCgpy3sNUkk6ePKnatWtr5MiRWr58ubKysvTaa68VeHwKIQAA4DZxcXFq27ZtnkdcnPEj2u12u2yXfEiVw+EwbJcrV04LFixQ9erV5eXlpb59+2rDBuONcvPDGiEAAGBk4uXzvXv3Vrdu3fI8HxAQYNgOCQnRxo0bndsZGRkKCrp4x+QDBw4oKSlJ0dHRknILJS8X7vlCRwgAALhNQECAQkND8zz+Xgg1a9ZMycnJyszM1OnTp5WYmKgWLVo4v162bFnNmDFD+/btk8PhUHx8vOEzEC+HjhAAADC42svazRAcHKyYmBj16tVL2dnZio6OVlhYmAYMGKDhw4frzjvv1MSJE/X4448rOztbDRo0UJ8+fQrcb4G32CiOuMWGdXGLDes2cbnFBrfYsDKzb7FRY4Z5t9jYMbKY32IDAABYjMM6f3Ra989LAABgeXSEAACAUYlbNHPt6AgBAADLoiMEAAAMiuNVY0WFjhAAALAsOkIAAMCIjhAAAEDpR0cIAAAYsEYIAADAAugIAQAAIzpCAAAApR+FEAAAsCxOjQEAACNOjQEAAJR+dIQAAIABl88DAABYAIUQAACwLAohAABgWawRAgAARqwRAgAAKP3oCAEAAAOuGgMAALCAEtkRuv+We9w9Bbex+fi4ewpuZT9z1t1TgLs47O6egVs5cnLcPQW3qu6z0d1TcK9RJh+PjhAAAEDpVyI7QgAAoAjREQIAACj96AgBAAADrhoDAACwAAohAABgWZwaAwAARpwaAwAAKP3oCAEAAAMWSwMAAFgAHSEAAGBERwgAAKD0oyMEAACM6AgBAACUfnSEAACAAVeNAQAAWAAdIQAAYERHCAAAoPSjIwQAAIzoCAEAAJR+dIQAAIABV40BAABYAIUQAACwLE6NAQAAI06NAQAAlH50hAAAgAGLpQEAACyAjhAAADCiIwQAAFD6UQgBAAAjh4mPq7Bq1SpFRkYqIiJC8fHxlx33xRdfqE2bNi7tk1NjAACg2Dt06JBiY2O1bNky+fj4qEePHmrSpIlq1KhhGHf48GFNmzbN5f3SEQIAAAY2Ex+uSkpKUnh4uAIDA+Xn56f27dsrISEhz7ixY8dq6NChLu+XjhAAAHCbrKwsZWVl5Xk+ICBAAQEBzu309HRVrlzZuR0UFKSUlBTD97z99tuqU6eO6tat6/LxKYQAAICRiVeNxcXFac6cOXmeHzp0qIYNG+bcttvtstku9pAcDodh+/fff1diYqLeeustHTx40OXjW7oQatyhnvpMelDeZby0++d9ih20QKeOn3ZpjH+Fchr2Sl9Vq3uTzpw8q8S3/6eVcxMlSbc2rKZBMx9VWb8y8vD00JIXV2nde1+7I+JVadw+TH0mROdm3ZKq2KELder4mXzHPvV6f+35JVUfzr7YlixX3lczVz+rWUMWavuPe0ya9bVr3LGe+r7Qw/mznfXY/Lw//8uM8fCwacjLfRTWopYk6bvVP2nB6MWSpLot6+ix6Q/L08tTWUeO6/Wn39GulD9Mz3clVs4uSY071lffyT3k7eOt3T//oVmPzcsnf/5jPDxsGvJKX4U1ry1J+i7hJy145l1Jkn+Fchrych/dVLuqypT10eKpK/R5/Jem5ytI48gG6jelp7zLeGt3yl692H9u3vyXGTNuyVOqWiPEOS7kliClbNiq8V2n6abaoYqZN1C+15WVw+HQm8/Ga2PiZrPjFYj3/uKld+/e6tatW57nL+0GSVJISIg2btzo3M7IyFBQUJBzOyEhQRkZGerevbuys7OVnp6unj17avHixVc8vmXXCJWv5K+n5j+mST1eUv+wkTq4O119X3jQ5TEDZzyi0yfP6LF6ozSixQQ1al9XTTrWlySNe/8JvTPpvxrcZIzGdpmugdMe1g3Vg03PeDXKX++vJ1/rp0mPvqr+DccobU+G+jz/QJ5xN95aRVNXjVLzLncZnm8UEaaX141T1Zoheb6nOCpfyV9PLxioiQ++pH53PK203YfUb3IPl8e0fbi5bry1igbWf0aDGj6rsBa11bx7E/kF+Gr8khgtGL1YgxqO1uxhi/T/Fg+Xt0/x+ZvDytmlC9neGKSJ/4pVvzueVNrudPWb8pDLY9o+0uJC/pEa1PAZZ35JevrNx5WRmqnBjZ7VMx0ma3Bsb1WqWtH0jFdSvlKAnl44WBOjZ6pv7Sdyf7ZTH3Z5zKR/vahBDUZqUIORmvXY6zpx9KRmD31DkjT81f5KWLROgxqM1Mx+r2nsB0/Kw7N4/TPDe79rbA7zHgEBAQoNDc3z+Hsh1KxZMyUnJyszM1OnT59WYmKiWrRo4fz68OHDtWbNGn300UeaP3++goKCCiyCJDcVQlOnTnXHYQ0a3Hunftu0Swd2HpIkfbzgM7XpcbfLY2rWv0WfL/5KdrtD57Nz9N3qn3TP/Y3lXcZb705eph/X/SJJOrw/U8cOH1fl0OtNTHf1GrS9Xb//sNuZ9ZM316nNA+F5xnV+rK0S3v6fvlzxveH5LoPu1fQBC/TnwWOmzPefatguTL9t3KUDO3Lbpx/P+0xtHrrb5TGenh4qW66MvMt4y7uMl7x8vJR9JltVa4To5LFT+ml97s9/328HdCrrtGqH1zQx3ZVZObv0V7adl2RbqzYP3ePymNz8ZfPk969QTg3uDdO7k/4rKfd3f/jd43Q884SJ6QrWMCJMv3+/U/svZFs1N1Fteza/6jFe3l4a9dZQzY15SxmpRyRJHp4e8q9QTpLk5++rc2fOFXWcq8Z7f8kVHBysmJgY9erVS127dlWnTp0UFhamAQMG6Oeff77m/Rb5n2rPPvtsnufWrVunY8dy/8H8z3/+U9RTyFfl0Ot1ODXTuZ2Rmqly5f3k5+/rbJFeacyv3+9U25736Jek3+Vdxkv3dGuk89k5yj6brTVvbXB+T8d+reXrX1bbvt1uXrhrULlqRWVcmnX/nxeyljWcHnvt6dxTAA3b3G74/rH3zzJnooWkcmhF55u3dLmf/+XHJL69Qc27N9HiPa/K08tDmz77Wd988oP8/H1VtlwZNbz3Tm367Gfd2rCa/q9OqCpWCTQ94+VYObuU+3ttzHYk39/9y41JjPsiN//e1+Tp5alNa1P0zSc/6LZG1ZWZ9qe6j4hSow715F3GS/+d9bH2b08zPeOVVL6xkjJSDzu3883vwpgO/droyIFMfb3iO+e42UPf0IzPJ+j+EZ0UGFReUx6KlT3HblIy1/De76Ji+snSnTt3VufOnQ3PLViwIM+40NBQrVu3zqV9FnlHKDAwUF988YVq1aqlxo0bq3HjxvLz83P+t7t4eNjkcOT9Sedc8kt7pTHzn4mXw+HQa99O1nNLn9QPn2/R+XPnDeP+9XRnPTq2uybc/6LOncku/BCFyObC/4/SxObhoXziGt60rzTmkXHddexwlh4MHaSetwyVf4Vy6j4iUqeOn9Zz0bPU45kumrvxP7r3keb6af0veV4b7mTl7FJh5I/WscPH9WDVgep582D5V7xO3UdEydPbU1WqBevU8dOKaTlBUx5+RQNn9lLNBrcUYZqrl/u+lvd5e573viuP6T4iSvGTP3Rue5fx1tj3YzSjz6vqedMgPdVyvJ54fWCx64jw3o+/K/JC6JlnntGsWbP06aef6oYbblC3bt1Uvnx5devWLd/FUWZJ33dE11ep4NyuVLWijmee0NlTZ10a4xfgqzfHvKeBDUdrdOR/ZLPJ2Ub19vHS6LeHqPW/miqm1XPa9XPxWyz6dxmpmcasN1TQ8T9P6Oyp4tfaLgwZ+w7r+hsudioqVa2orMwTOnPJz/9KY+7p2khr3tqg89k5OpV1Wmvf+VJ1W9aRzWbTmZNnNLLdC3r8rmf1WkycQm+t4nxtFAdWzi5dyPa33+t8819mzD1dG2vNovWX5P+f6ra6XZkH/pQkrXnrC0m57we/fP2bbmtk/LA3d0v/o+D8BY2pXu9meXp5KmXDVueYW+64UWX8yujbT36QJG37drv2/rJPtZoUr1OjvPfj70xZI9S0aVPNmzdPixcv1rRp05STk2PGYa9o02c/q1bjGs6FbFED2ir5400uj+k0oK16jY+WJAUGBahDn9Za/0GSJGnUosHy8/fViFbP69DewyoJNn2+RbUaVbuYtW9rJX/yo5tnVXQ2rf1ZtRvX1A0Xrn7p9FhbJa/a5PKY7T/uUYvo3DVUnl6eatq5gX79boccDode+GiUswvQ8oFwnTuTXayunLJydknatDZFtZvUuCTbvUpetdHlMdt/2q0WDzSVdCF/p4b69dvtOrgnQ9t/2KV2vVpKkgKDyqtO01v1+6adZkVzyabEzaodXtN55VenQRFK/uj7qxoT1rKOflq/xfA9+3ccVLnyfqrT9FZJUpVqwbqpTqh2/Li7KONcNd77XVRMb7FRFGyO/Pp/RWjp0qVavXq1Fi5ceM37aF/24YIHuaBR+7rqO+lBefl4KW1Xumb0m6uQW4IUM3eABjcZc9kxx/88Kd/rymrUwsd1Q/Vg2Ww2vT9jpda997VqN6mhlzY8r32/H9C50xdbom/+v/e06bNrX8z1F5uPzz/ex+U0ighTnwndc7PuTteMgW+oys2VNWJ2Hw25Z4Jh7FNz+2nP1v2Gy+clKe7nGXqh16tFdvm8/czZgge5qFGHeur7woPy9vHSgZ2HNKNv7s//yXkD9HijMZcdc/zPk/KveJ2Gvvxv1ah3s3Jy7Ppp3RbNfyZe57NzdGfzWnr8xV7y8vFSZtpRvTT4DR3cnV5o8y4MJTK7o/BO0zbqUE99Jz8kb28vHdh1SDP6vKqQasF6ct5jevyu0ZcdczF/n4v512/R/FHv6nx2jirfeL2GvdJXIbcEycPDQ8tnf6pPFnxeKHN2FOIfkI071lffKT2dP9vpveeoSrUgPbngcQ1qMPKyY47/mbvwe9icfjqS9qcWT15m2G/dVrdrwLRH5FPWRznnc/TOxKVK+luRda08CvG9ryS+9685c/n7ahWFusNjTTvW5ldiTDtWfkwvhApDYRVCJVFRFkIlQWEWQihhCrEQKokKsxAqiQqzECqJzC6E6g0zrxD6abZ7C6Hi9QEPAAAAJipen3QGAADcr8SdK7p2dIQAAIBl0RECAAAGNjpCAAAApR8dIQAAYERHCAAAoPSjIwQAAAxYIwQAAGABdIQAAIARHSEAAIDSj44QAAAwoiMEAABQ+lEIAQAAy+LUGAAAMODyeQAAAAugIwQAAIzoCAEAAJR+dIQAAICBzWGdlhAdIQAAYFl0hAAAgJF1GkJ0hAAAgHXREQIAAAZ8jhAAAIAF0BECAABGdIQAAABKPzpCAADAgDVCAAAAFkBHCAAAGNERAgAAKP0ohAAAgGVxagwAABiwWBoAAMACSmRHyH7unLun4D5Wzg7Asiz9vu8OdIQAAABKvxLZEQIAAEWHNUIAAAAWQEcIAAAYOazTEqIjBAAALIuOEAAAMGCNEAAAgAXQEQIAAEZ0hAAAAEo/OkIAAMDAZnf3DMxDRwgAAFgWHSEAAGDEGiEAAIDSj0IIAABYFoUQAAAwsDnMe1yNVatWKTIyUhEREYqPj8/z9bVr16pz586KiorS6NGjde7cuQL3SSEEAACKvUOHDik2NlaLFy/WihUr9MEHH2jHjh3Or586dUoTJ07UokWL9Mknn+js2bNavnx5gfulEAIAAEYOh3kPFyUlJSk8PFyBgYHy8/NT+/btlZCQ4Py6n5+f1q1bp0qVKun06dM6cuSIAgICCtwvV40BAAC3ycrKUlZWVp7nAwICDIVMenq6Kleu7NwOCgpSSkqK4Xu8vb21YcMGjRo1SkFBQbrnnnsKPD4dIQAAYGDmGqG4uDi1bds2zyMuLs4wJ7vdLpvN5tx2OByG7b+0bNlS3377rVq3bq3nnnuuwKx0hAAAgNv07t1b3bp1y/P8309rhYSEaOPGjc7tjIwMBQUFObePHj2qLVu2OLtAnTt3VkxMTIHHpyMEAACMHOY9AgICFBoamufx90KoWbNmSk5OVmZmpk6fPq3ExES1aNHi4pQdDo0cOVIHDhyQJCUkJKhBgwYFRqUjBAAAir3g4GDFxMSoV69eys7OVnR0tMLCwjRgwAANHz5cd955pyZNmqSBAwfKZrOpRo0aev755wvcr83huIol28VEO48H3D0FAABMs9a+1NTjNe8207Rjfbn8adOOlR9OjQEAAMvi1BgAADAqeSeLrhkdIQAAYFl0hAAAgMHV3gOsJKMjBAAALIuOEAAAMKIjBAAAUPpRCAEAAMvi1BgAADBgsTQAAIAF0BECAABGduu0hCxdCDWObKB+U3rKu4y3dqfs1Yv95+rU8dMujRm35ClVrRHiHBdyS5BSNmzV+K7TdFPtUMXMGyjf68rK4XDozWfjtTFxs9nxCkR+6+a3cnaJ/OS3dn4YWfamq+UrBWjBllmKuWes9u84qP5TH5avv69mD3njqsZI0q13Vdf4pU8ppvk4ZaQe0cx1z2ntOxu0ZtF6Va93s15c/7zur9RH9hz7P553YSG/dfNbObtEfvKXzPxm33S1ZeR004614dNRph0rP6asEUpJSXH+d3JysqZOnaqZM2dq82b3VcoNI8L0+/c7tX/HQUnSqrmJatuz+VWP8fL20qi3hmpuzFvKSD0iSfLw9JB/hXKSJD9/X507c66o41w18ls3v5WzS+Qnv7XzIy9TTo1NmDBBy5cvV3x8vN5//311795dkjR+/Hg98MADeuSRR8yYhkHlGyspI/Wwczsj9YjKlfeTn7+vs0XqypgO/droyIFMfb3iO+e42UPf0IzPJ+j+EZ0UGFReUx6KLVZ/EUnkt3J+K2eXyE9+a+d3lZWuGjN1jdCSJUv09ttvq0KFCpKk6OhoRUdHu6UQ8vCw5Xtz3UtftK6M6T4iSrED5zm3vct4a+z7MZrR51V9+8kPqt2kpiauHK3fvt/p/KuhOCC/dfNbObtEfvJbOz/yMuXU2Pnz52W32xUYGCgfHx/n8z4+PvLwcM8V/Ol/HNb1VSo4tytVraiszBM6c+qsy2Oq17tZnl6eStmw1TnmljtuVBm/Mvr2kx8kSdu+3a69v+xTrSY1izrSVSG/dfNbObtEfvJbO7/LHA7zHm5mShUSGBioVq1aaffu3Zo0aZKk3LVCPXr0UIcOHcyYQh6bEjerdnhN5+r/ToMilPzR91c1JqxlHf20fovhe/bvOKhy5f1Up+mtkqQq1YJ1U51Q7fhxd1HGuWrkt25+K2eXyE9+a+dHXqacGnvnnXckSbt27VJWVpak3G7Q8OHD1apVKzOmkMfRjCzN7Puaxi19St4+Xjqw85Cm956jWxtW05MLHtegBiMvO+YvoTWr6OCedMN+Tx47pefun6HBL/WRT1kf5ZzP0UsD5ylt1yGzI14R+a2b38rZJfKT39r5XWWlNUKWvXweAICSwuzL51u3n2basdaveca0Y+XH0h+oCAAA8lHiWiTXjnuNAQAAy6IjBAAADGwlb9XMNaMjBAAALItCCAAAWBanxgAAgFHJvDPINaEjBAAALIuOEAAAMGCxNAAAgAXQEQIAAEbWaQjREQIAANZFRwgAABixRggAAKD0oyMEAAAMbNZpCNERAgAA1kVHCAAAGLFGCAAAoPSjIwQAAAxs3GsMAACg9KMjBAAAjFgjBAAAUPrREQIAAEbWaQjREQIAANZFIQQAACyLU2MAAMDAxmJpAACA0o+OEAAAMKIjBAAAUPrREQIAAEbcYgMAAKD0oyMEAAAMuGoMAADAAugIAQAAIzpCAAAAxcuqVasUGRmpiIgIxcfH5/n6Z599pi5duui+++7T4MGDdezYsQL3SSEEAACMHA7zHi46dOiQYmNjtXjxYq1YsUIffPCBduzY4fz6iRMn9Nxzz2n+/PlauXKlbrvtNs2ePbvA/VIIAQCAYi8pKUnh4eEKDAyUn5+f2rdvr4SEBOfXs7OzNWHCBAUHB0uSbrvtNqWlpRW4X9YIAQAAIxM/RygrK0tZWVl5ng8ICFBAQIBzOz09XZUrV3ZuBwUFKSUlxbldoUIFtWvXTpJ05swZzZ8/X48++miBx6cQAgAAbhMXF6c5c+bkeX7o0KEaNmyYc9tut8tmszm3HQ6HYfsvx48f15AhQ1SrVi1169atwONTCAEAAAMzP0eod+/e+RYsl3aDJCkkJEQbN250bmdkZCgoKMgwJj09Xf369VN4eLjGjBnj0vEphAAAgNv8/RSSLDDuAAAM80lEQVTY5TRr1kyzZ89WZmamfH19lZiYqEmTJjm/npOTo0GDBqljx44aPHiwy8enEAIAAMVecHCwYmJi1KtXL2VnZys6OlphYWEaMGCAhg8froMHD2rr1q3KycnRmjVrJEl33HGHJk+efMX92hyOkvepSe08HnD3FAAAMM1a+1JTj9eh7jjTjpWweVLBg4oQl88DAADL4tQYAAAwKnkni64ZHSEAAGBZdIQAAIARHSEAAIDSj44QAAAwMvEWG+5GRwgAAFgWHSEAAGBg5i023I2OEAAAsCw6QgAAwIiOEAAAQOlHRwgAABjZ6QgBAACUenSEAACAEWuEAAAASj8KIQAAYFmcGgMAAEacGgMAACj96AgBAAAjOkIAAAClHx0hAABgxAcqAgAAlH50hAAAgJHD7u4ZmIaOEAAAsCw6QgAAwIirxgAAAEo/OkIAAMDIQleNWboQahzZQP2m9JR3GW/tTtmrF/vP1anjp10aM27JU6paI8Q5LuSWIKVs2KrxXafpptqhipk3UL7XlZXD4dCbz8ZrY+Jms+MViPzWzW/l7BL5yW/t/DCyORwl70RgO48H/vE+ylcK0IItsxRzz1jt33FQ/ac+LF9/X80e8sZVjZGkW++qrvFLn1JM83HKSD2imeue09p3NmjNovWqXu9mvbj+ed1fqY/sOcVnFT75rZvfytkl8pO/ZOZfa1/6j/dxNTre+IRpx1q972XTjpUf09YIffnll8rKypIkrVixQhMnTtSHH35o1uHzaBgRpt+/36n9Ow5KklbNTVTbns2veoyXt5dGvTVUc2PeUkbqEUmSh6eH/CuUkyT5+fvq3JlzRR3nqpHfuvmtnF0iP/mtnR95mXJqbPLkydq2bZtiY2P10ksvKSUlRffee6/Wrl2rbdu2aezYsWZMw6DyjZWUkXrYuZ2RekTlyvvJz9/X2SJ1ZUyHfm105ECmvl7xnXPc7KFvaMbnE3T/iE4KDCqvKQ/FFqu/iCTyWzm/lbNL5Ce/tfO7rOSdLLpmphRCSUlJWrlypTw9PbVhwwZ98MEH8vHx0YMPPqhOnTqZMYU8PDxs+f6cL33RujKm+4goxQ6c59z2LuOtse/HaEafV/XtJz+odpOamrhytH77fqfzr4bigPzWzW/l7BL5yW/t/MjLlFNjZcuW1ZEjuS+EkJAQnTp1SpJ0+vRpeXm5Z712+h+HdX2VCs7tSlUrKivzhM6cOuvymOr1bpanl6dSNmx1jrnljhtVxq+Mvv3kB0nStm+3a+8v+1SrSc2ijnRVyG/d/FbOLpGf/NbOj7xMKYSGDBmi6OhoTZs2TaGhoXr00Uc1ZcoU/etf/1KfPn3MmEIemxI3q3Z4Tefq/06DIpT80fdXNSasZR39tH6L4Xv27ziocuX9VKfprZKkKtWCdVOdUO34cXdRxrlq5Ldufitnl8hPfmvnd5nDYd7DzUxpx7Rp00Y1a9bUZ599pr1796pevXoqV66cpk6dqrCwMDOmkMfRjCzN7Puaxi19St4+Xjqw85Cm956jWxtW05MLHtegBiMvO+YvoTWr6OCedMN+Tx47pefun6HBL/WRT1kf5ZzP0UsD5ylt1yGzI14R+a2b38rZJfKT39r5kZdlL58HAKCkMP3y+SpDTDvW6rRXTTtWfrjFBgAAsCxLf7I0AADIR8k7WXTN6AgBAADLoiMEAACM6AgBAACUfnSEAACAkZ2OEAAAQKlHRwgAABg4HCX0ZrHXgI4QAACwLDpCAADAiDVCAAAApR8dIQAAYMTnCAEAAJR+FEIAAMCyODUGAACM7Fw+DwAAUOrREQIAAEYslgYAACheVq1apcjISEVERCg+Pv6y40aNGqVly5a5tE86QgAAwMBRDNcIHTp0SLGxsVq2bJl8fHzUo0cPNWnSRDVq1DCMmTBhgpKTkxUeHu7SfukIAQCAYi8pKUnh4eEKDAyUn5+f2rdvr4SEBMOYVatWqW3bturYsaPL+6UjBAAAjExcI5SVlaWsrKw8zwcEBCggIMC5nZ6ersqVKzu3g4KClJKSYvie/v37S5I2bdrk8vEphAAAgNvExcVpzpw5eZ4fOnSohg0b5ty22+2y2WzObYfDYdi+VhRCAADAyMSbrvbu3VvdunXL8/yl3SBJCgkJ0caNG53bGRkZCgoK+sfHpxACAABu8/dTYJfTrFkzzZ49W5mZmfL19VViYqImTZr0j4/PYmkAAGDksJv3cFFwcLBiYmLUq1cvde3aVZ06dVJYWJgGDBign3/++Zqj2hyOkvepSe08HnD3FAAAMM1a+1JTj9e+7MOmHWvNmct/HpAZODUGAAAMHCauEXI3To0BAADLoiMEAACMrmLtTklHRwgAAFgWhRAAALAsTo0BAAADFksDAABYAB0hAABgZKHF0iXyAxUBAAAKA6fGAACAZVEIAQAAy6IQAgAAlkUhBAAALItCCAAAWBaFEAAAsCwKIQAAYFkUQgAAwLIohAAAgGVRCF2FVatWKTIyUhEREYqPj3f3dEx34sQJderUSampqe6eiunmzJmjqKgoRUVFafr06e6ejulefvllRUZGKioqSosWLXL3dNxm2rRpGj16tLunYbpHH31UUVFR6tKli7p06aLNmze7e0qmWrdune6//3517NhRL7zwgrung0LGvcZcdOjQIcXGxmrZsmXy8fFRjx491KRJE9WoUcPdUzPF5s2bNXbsWO3Zs8fdUzFdUlKSvvrqKy1fvlw2m039+/fX2rVr1a5dO3dPzRTfffedvvnmG61cuVLnz59XZGSkWrZsqWrVqrl7aqZKTk7W8uXL1apVK3dPxVQOh0N79uzR+vXr5eVlvX8y9u3bpwkTJmjp0qW6/vrr1bt3b23YsEEtW7Z099RQSOgIuSgpKUnh4eEKDAyUn5+f2rdvr4SEBHdPyzRLlizRhAkTFBQU5O6pmK5y5coaPXq0fHx85O3trerVq+vAgQPunpZpGjdurLffflteXl46cuSIcnJy5Ofn5+5pmero0aOKjY3VoEGD3D0V0+3atUuS1LdvX913331699133Twjc61du1aRkZEKCQmRt7e3YmNjVbduXXdPC4XIeuX9NUpPT1flypWd20FBQUpJSXHjjMw1efJkd0/BbWrWrOn87z179mj16tV677333Dgj83l7e+uVV17RwoUL1aFDBwUHB7t7SqYaP368YmJilJaW5u6pmC4rK0tNmzbVuHHjlJ2drV69eumWW27R3Xff7e6pmWLv3r3y9vbWoEGDlJaWplatWmnEiBHunhYKER0hF9ntdtlsNue2w+EwbKP02759u/r27atRo0bp5ptvdvd0TDd8+HAlJycrLS1NS5Yscfd0TLN06VJVqVJFTZs2dfdU3KJ+/fqaPn26/P39VbFiRUVHR2vDhg3unpZpcnJylJycrClTpuiDDz5QSkqKli9f7u5poRBRCLkoJCREGRkZzu2MjAxLniayqk2bNunf//63nnrqKXXr1s3d0zHVzp07tW3bNkmSr6+vIiIi9Ntvv7l5Vub59NNP9fXXX6tLly565ZVXtG7dOk2ZMsXd0zLNxo0blZyc7Nx2OByWWitUqVIlNW3aVBUrVlTZsmV17733WupsgBVQCLmoWbNmSk5OVmZmpk6fPq3ExES1aNHC3dOCCdLS0jRkyBDNnDlTUVFR7p6O6VJTUzV27FidO3dO586d0+eff66GDRu6e1qmWbRokT7++GN99NFHGj58uNq0aaMxY8a4e1qmOX78uKZPn66zZ8/qxIkTWr58uWUuFJCk1q1b66uvvlJWVpZycnL05Zdf6vbbb3f3tFCIrFPW/0PBwcGKiYlRr169lJ2drejoaIWFhbl7WjDBm2++qbNnz2rq1KnO53r06KGHHnrIjbMyT8uWLZWSkqKuXbvK09NTERERliwIrap169bavHmzunbtKrvdrp49e6p+/frunpZp6tatq/79+6tnz57Kzs7W3Xffre7du7t7WihENofD4XD3JAAAANyBU2MAAMCyKIQAAIBlUQgBAADLohACAACWRSEEAAAsi0IIAABYFoUQAACwLAohAC7p2LGjWrRooe3bt7t7KgBQaCiEALjk448/1s0336w1a9a4eyoAUGgohAC4xNPTUw0bNrTUDVcBlH7cawyAS86cOaNPP/1U3JUHQGlCRwiAS2JjYxUUFKQ//vhDJ0+edPd0AKBQUAgBKNCPP/6o1atXa/bs2fL392fBNIBSg0IIwBWdPXtWY8aM0fPPP6/AwEDVqlVLv/76q7unBQCFgkIIwBW9/PLLqlevnlq3bi1JqlWrFgumAZQaFEIALislJUUJCQkaM2aM87natWvTEQJQatgcXAICAAAsio4QAACwLAohAABgWRRCAADAsiiEAACAZVEIAQAAy6IQAgAAlkUhBAAALItCCAAAWNb/B72xHrZgB/RnAAAAAElFTkSuQmCC\n",
+      "text/plain": [
+       "
    "
+      ]
+     },
+     "metadata": {
+      "filenames": {
+       "image/png": "/Users/hjensen/Teaching/FYS-STK4150/doc/src/LectureNotes/_build/jupyter_execute/chapter6_134_2.png"
+      }
+     },
+     "output_type": "display_data"
+    }
+   ],
    "source": [
     "# visual representation of grid search\n",
     "# uses seaborn heatmap, you can also do this with matplotlib imshow\n",
@@ -3178,7 +3215,632 @@
    "metadata": {
     "collapsed": false
    },
-   "outputs": [],
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n",
+      "  warnings.warn(\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  1e-05\n",
+      "Lambda =  1e-05\n",
+      "Accuracy score on test set:  0.18333333333333332\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n",
+      "  warnings.warn(\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  1e-05\n",
+      "Lambda =  0.0001\n",
+      "Accuracy score on test set:  0.18611111111111112\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n",
+      "  warnings.warn(\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  1e-05\n",
+      "Lambda =  0.001\n",
+      "Accuracy score on test set:  0.13055555555555556\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n",
+      "  warnings.warn(\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  1e-05\n",
+      "Lambda =  0.01\n",
+      "Accuracy score on test set:  0.24444444444444444\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n",
+      "  warnings.warn(\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  1e-05\n",
+      "Lambda =  0.1\n",
+      "Accuracy score on test set:  0.23333333333333334\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n",
+      "  warnings.warn(\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  1e-05\n",
+      "Lambda =  1.0\n",
+      "Accuracy score on test set:  0.12777777777777777\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n",
+      "  warnings.warn(\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  1e-05\n",
+      "Lambda =  10.0\n",
+      "Accuracy score on test set:  0.1527777777777778\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n",
+      "  warnings.warn(\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  0.0001\n",
+      "Lambda =  1e-05\n",
+      "Accuracy score on test set:  0.9111111111111111\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n",
+      "  warnings.warn(\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  0.0001\n",
+      "Lambda =  0.0001\n",
+      "Accuracy score on test set:  0.8888888888888888\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n",
+      "  warnings.warn(\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  0.0001\n",
+      "Lambda =  0.001\n",
+      "Accuracy score on test set:  0.8722222222222222\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n",
+      "  warnings.warn(\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  0.0001\n",
+      "Lambda =  0.01\n",
+      "Accuracy score on test set:  0.8305555555555556\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n",
+      "  warnings.warn(\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  0.0001\n",
+      "Lambda =  0.1\n",
+      "Accuracy score on test set:  0.8888888888888888\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n",
+      "  warnings.warn(\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  0.0001\n",
+      "Lambda =  1.0\n",
+      "Accuracy score on test set:  0.8805555555555555\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n",
+      "  warnings.warn(\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  0.0001\n",
+      "Lambda =  10.0\n",
+      "Accuracy score on test set:  0.8944444444444445\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n",
+      "  warnings.warn(\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  0.001\n",
+      "Lambda =  1e-05\n",
+      "Accuracy score on test set:  0.975\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n",
+      "  warnings.warn(\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  0.001\n",
+      "Lambda =  0.0001\n",
+      "Accuracy score on test set:  0.9777777777777777\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n",
+      "  warnings.warn(\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  0.001\n",
+      "Lambda =  0.001\n",
+      "Accuracy score on test set:  0.9805555555555555\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n",
+      "  warnings.warn(\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  0.001\n",
+      "Lambda =  0.01\n",
+      "Accuracy score on test set:  0.9861111111111112\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n",
+      "  warnings.warn(\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  0.001\n",
+      "Lambda =  0.1\n",
+      "Accuracy score on test set:  0.9805555555555555\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n",
+      "  warnings.warn(\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  0.001\n",
+      "Lambda =  1.0\n",
+      "Accuracy score on test set:  0.9777777777777777\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n",
+      "  warnings.warn(\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  0.001\n",
+      "Lambda =  10.0\n",
+      "Accuracy score on test set:  0.9444444444444444\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/Users/hjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n",
+      "  warnings.warn(\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  0.01\n",
+      "Lambda =  1e-05\n",
+      "Accuracy score on test set:  0.9861111111111112\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  0.01\n",
+      "Lambda =  0.0001\n",
+      "Accuracy score on test set:  0.9888888888888889\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  0.01\n",
+      "Lambda =  0.001\n",
+      "Accuracy score on test set:  0.9888888888888889\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  0.01\n",
+      "Lambda =  0.01\n",
+      "Accuracy score on test set:  0.9861111111111112\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  0.01\n",
+      "Lambda =  0.1\n",
+      "Accuracy score on test set:  0.9888888888888889\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  0.01\n",
+      "Lambda =  1.0\n",
+      "Accuracy score on test set:  0.9722222222222222\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  0.01\n",
+      "Lambda =  10.0\n",
+      "Accuracy score on test set:  0.9527777777777777\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  0.1\n",
+      "Lambda =  1e-05\n",
+      "Accuracy score on test set:  0.8777777777777778\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  0.1\n",
+      "Lambda =  0.0001\n",
+      "Accuracy score on test set:  0.8388888888888889\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  0.1\n",
+      "Lambda =  0.001\n",
+      "Accuracy score on test set:  0.8916666666666667\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  0.1\n",
+      "Lambda =  0.01\n",
+      "Accuracy score on test set:  0.9111111111111111\n",
+      "\n",
+      "Learning rate  =  0.1\n",
+      "Lambda =  0.1\n",
+      "Accuracy score on test set:  0.9166666666666666\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  0.1\n",
+      "Lambda =  1.0\n",
+      "Accuracy score on test set:  0.9083333333333333\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  0.1\n",
+      "Lambda =  10.0\n",
+      "Accuracy score on test set:  0.7944444444444444\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  1.0\n",
+      "Lambda =  1e-05\n",
+      "Accuracy score on test set:  0.09166666666666666\n",
+      "\n",
+      "Learning rate  =  1.0\n",
+      "Lambda =  0.0001\n",
+      "Accuracy score on test set:  0.11388888888888889\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  1.0\n",
+      "Lambda =  0.001\n",
+      "Accuracy score on test set:  0.07777777777777778\n",
+      "\n",
+      "Learning rate  =  1.0\n",
+      "Lambda =  0.01\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Accuracy score on test set:  0.14444444444444443\n",
+      "\n",
+      "Learning rate  =  1.0\n",
+      "Lambda =  0.1\n",
+      "Accuracy score on test set:  0.11944444444444445\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  1.0\n",
+      "Lambda =  1.0\n",
+      "Accuracy score on test set:  0.1361111111111111\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  1.0\n",
+      "Lambda =  10.0\n",
+      "Accuracy score on test set:  0.13055555555555556\n",
+      "\n",
+      "Learning rate  =  10.0\n",
+      "Lambda =  1e-05\n",
+      "Accuracy score on test set:  0.07777777777777778\n",
+      "\n",
+      "Learning rate  =  10.0\n",
+      "Lambda =  0.0001\n",
+      "Accuracy score on test set:  0.21388888888888888\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  10.0\n",
+      "Lambda =  0.001\n",
+      "Accuracy score on test set:  0.10555555555555556\n",
+      "\n",
+      "Learning rate  =  10.0\n",
+      "Lambda =  0.01\n",
+      "Accuracy score on test set:  0.07777777777777778\n",
+      "\n",
+      "Learning rate  =  10.0\n",
+      "Lambda =  0.1\n",
+      "Accuracy score on test set:  0.125\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  10.0\n",
+      "Lambda =  1.0\n",
+      "Accuracy score on test set:  0.1527777777777778\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Learning rate  =  10.0\n",
+      "Lambda =  10.0\n",
+      "Accuracy score on test set:  0.17777777777777778\n",
+      "\n"
+     ]
+    }
+   ],
    "source": [
     "from sklearn.neural_network import MLPClassifier\n",
     "# store models for later use\n",
@@ -3211,7 +3873,36 @@
    "metadata": {
     "collapsed": false
    },
-   "outputs": [],
+   "outputs": [
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkIAAAJgCAYAAABmwww9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd5gTVdvH8e8k22GXuksHRam6iCCKCCgoIL0IiigqKIo0QcWKiqLCA0gRxIKAFKnSi4rCKypVQQWRpsKDSNulLWy2JJl5/wjPssMuoggJ2fw+17UXzORk5txJZubkPudMDMuyLERERERCkCPQFRAREREJFDWEREREJGSpISQiIiIhSw0hERERCVlqCImIiEjIUkNIREREQlZYoCsgEmpef/11vvvuOwB+++03SpUqRVRUFACzZs3K+v/5dOvWjWeffZarr776nGVGjx5NuXLlaNOmzb+v+Gm9e/dmw4YNfPXVV0RHR1+07YqIBIKh+wiJBE7Dhg0ZPXo0iYmJga7K33Lo0CFat27Nddddx2233ca9994b6CqJiPwr6hoTuYyMGTOGhx9+mJYtW/L000+TnJxMjx49uOeee2jYsCGdO3fmyJEjgK8RtWXLFtavX0/Hjh3p378/bdq0oUWLFmzcuBGA5557jgkTJgCQmJjImDFj6NixIw0bNmT69OkAeL1eBg8eTKNGjWjXrh0DBw6kc+fOudZv9uzZ3HzzzbRt25YpU6aQ/XvUTz/9RIcOHWjRogVt27Zl7dq1f7m+UqVKHD16NOv5/1tev349rVq1omPHjrRs2ZLMzExef/11OnToQLNmzWjatGlWfKmpqTz//PM0adKEZs2aMWLECFwuF7Vq1WL37t1Z237ooYf48ssvL8p7JCJ5ixpCIpeZP//8k/nz5zN8+HCWLl1K9erVmTVrFitWrCAqKoqFCxfmeM7mzZvp2rUrCxYsoF27dowcOTJHmczMTAoVKsTMmTN5++23GTx4MBkZGcyZM4etW7eyZMkSZs6cyR9//JFrvTweD7Nnz6ZVq1Y0bNiQI0eO8PXXXwPgdrvp2bMnPXv2ZMmSJQwaNIg333yTzMzMXNebpvmXr8GuXbt46623WLx4MVu3buXw4cPMmjWLZcuW0bZtW8aPHw/A22+/TUZGBsuWLWPBggVs2rSJLVu20KZNG+bMmQPA3r172bNnDw0aNPhH74OIhAaNERK5zFSvXp2wMN+h+eCDD/L9998zadIk9uzZw65du7juuutyPKdkyZJUqVIFgKpVqzJ//vxct3377bcDcM0115CZmYnL5WLVqlW0bt2ayMhIAO655x6mTp2a47krVqzANE3q1atHWFgYzZo1Y8qUKdx6663s3LkTh8PBbbfdBsC1116b1YjJbf35lChRglKlSgFw/fXXU6BAgaxG2vr168mXLx8Aa9as4fnnn8fpdOJ0Opk2bRoACQkJ3H///fTr149Zs2bRvn17nE7nefcrIqFHDSGRy0xMTEzW/4cNG8bmzZu56667uOmmm/B4POQ2rC/7AGvDMHItA2Q1dgzDAMCyrKxG1/84HLkniqdPn056ejqNGzcGfBmmpKQkdu3ahdPpzNrm/+zcufOc68uXL29bl5mZaVvO/hp89dVXvPHGG3Tp0oXbb7+d8uXLs2jRIgDCwsJs2z9w4ABRUVFceeWVVKpUiRUrVrBkyRJmz56da0wiIuoaE7mMffvttzz44IO0adOGIkWKsGbNGrxe70Xdx6233sqiRYvIzMzE4/Hkmk3avXs33333HfPmzWPlypWsXLmSb7/9llq1ajFlyhTKly+PYRisXr0agK1bt/Lggw+ec71pmhQuXJgtW7YAsGTJknPWb/Xq1TRo0IBOnTpx7bXX8uWXX2a9BjfffDPz58/HNE0yMzPp06dP1oy8Tp06MXToUKpVq0axYsUu6msmInmHGkIil7GePXsydOhQWrZsyeOPP06NGjXYu3fvRd1Hu3btqFatGm3atKFjx46Eh4fnmBY/Y8YM7rjjDsqVK5ejfgsXLuTUqVOMGTOGsWPH0rp1a1555RXGjBlDRETEOdcPGDCA1157jbZt2/Lbb78RHx+fa/06duzIhg0baNmyJW3btqVMmTLs27cP0zTp1asX4eHhtG7dmjZt2nDrrbdmZawaNGiAy+WiY8eOF/X1EpG8RdPnRULct99+y5EjR2jdujXgu89RZGQk/fv3D3DN/p0ffviBAQMGsGTJkhzdcyIi/6OGkEiIO3ToEM899xzJycmYpknlypUZOHAgsbGxga7aBXv22WfZsGEDI0eOpHr16oGujohcxtQQEhERkaBx6tQpOnbsyHvvvUfp0qVtj23bto0XX3yR1NRUbrjhBl599dUcE0LOpjFCIiIiEhR++ukn7r33Xvbs2ZPr4/379+fll1/m888/x7KsvzVjVA0hERERCQqzZ8/mlVdeISEhIcdjf/75J+np6Vnd4e3ateOzzz477zZ1HyEREREJmJSUFFJSUnKsj4uLIy4uzrbujTfeOOd2Dh8+bJt9Gh8fz6FDh867/6BsCN1Z/eVAVyFgjNT0QFchsE6lBroGAZVZqVSgqxAw4UdC+733bt0Z6CoEVFi5MoGuQkB9unuEX/dnHqzot31NntObsWPH5ljfq1cvevfu/be3Y5qmbYaoZVl/a8ZoUDaEREREJG948MEHadu2bY71Z2eDzqd48eIkJSVlLScnJ+fahXY2NYRERETExuSvfxj5YsqtC+xClCpVisjISDZu3EjNmjVZuHAh9evXP+/zNFhaREREgla3bt2yfq5n+PDhDB48mDvvvBOXy8UDDzxw3ucrIyQiIiI2Xst/GaELaYisXLky6//jx4/P+n/lypX55JNP/tG2lBESERGRkKWGkIiIiIQsdY2JiIiIjUno/PqWMkIiIiISspQREhERERt/Tp8PNGWEREREJGQpIyQiIiI2XktjhERERETyPGWERERExEazxkRERERCgDJCIiIiYuNVRkhEREQk71NGSERERGw0RkhEREQkBCgjJCIiIja6j5CIiIhICFBGSERERGxC55fGlBESERGREKaGkIiIiIQsdY2JiIiIjW6oKCIiIhIClBESERERG2/oJISUERIREZHQpYyQiIiI2Gj6vIiIiEgIUEZIREREbLwYga6C3ygjJCIiIiFLGSERERGxMTVrTERERCTvU0botBvrVaRL7zsIjwhj966DjBy4EFdqhq1Mw2bVaP9gXSwsMtLdvPufZez6ZX/W40WLxTFq6qP0uHscKcdd/g7hX6l1W2W6PNXUF/+OA4x6YQ6uUxm5ln3qP3ezZ+dB5k74GoAXx9xPiXJFsx4vXroQWzbs5tXuH/mj6hdFrduvocsLrX3xb/uTUU9+jOtUeq5lnxrdmT3b9jP3vRU5HhswoRtHDp7g3RdnX+oqX1Q31b6KR7o1IDzcye+/H2b40KW4XJm2Mnc0uoa776mNBWSkuxk7Zjk7dxwEoFXrGjRrXp2IyDB27TzI8KFLcbu9AYjk37uxXiW69G1MeLjTdy54eX7Oc0GL62j/UD0sCzLSM3l38FJ2/fJngGr8z93YrAYPv9mJ8Mhwdm/+L2898i6uk2l/q4zD4aDX2IepVr8qABs+3cQH/acCULtFTfp/1IukvclZ2+lX/yXSznEsXQ5qNahCl2ea+4797fsZ9eysc5/7ht/Lnh0HmDv+q6x1ze+vw5331CYiKpxff97HqGdn4s4Mzs9+dhojFGIKFIrhyVfbMOjpmTzS5m0O7DtGlyca2cqULleER/o1YUDPKfS8511mjF/FS291zHr89hbXMXziwxRNiPN39f+1AoXz8eSQu3m911S6NRnGwT+O0OXppjnKlbkqgcFTHqXundVs69/oPY1erUbRq9Uo3n7xE06lpPPOwPn+qv6/VqBIfp4c1ZnXHxlPt3qvcfC/yXR5sXWOcmUqFGPwnD7UbXF9rttp3+MOrr3pqktd3YuuQIEY+j/bgoEvz+WhB97nwP7jPPJoA1uZ0mUK82j323numZk89sgEpk1dzcDX7gKgbr1KtGl3A/2fms7DD31AREQYd3W4MRCh/GsFCsXw5KB2DOo3nUdajfKdC/o2sZUpfUVRHnmyKQO6T6Znh7HM+OArXhrVKUA1/ucKFI3j6Yk9eK39cLpWeYIDuw/x8JD7/naZOzrXp3TFkjxa7Skeq/401epXpX772gBUrVOJT95aRPca/bP+LudGUIHC+XhyaEdef/wjut0+hIN7j9LlmRY5ypW5KoHBHz9O3ab2c1+dJom0erAez9//Ht0bDyUiMpw2XW/1V/XlIvFbQ+i3335j3LhxvPzyywwcOJBx48axZcsWf+3+L9W4+Wp2bt3P/r1HAVg65zsanvWBd7u9jHptIUeTTwGwc+t+ChXNT1iYk8LxsdRpUIUXH5/i97pfDDXqVmTnlj/Y/1/ft7gl09fRoFXOi32L+27m8zkb+OazzbluJyzcyVND7+GDNxaRfPDEJa3zxVTj1irs/PG/7N+dBMCSyd/QoF2tHOVaPHQrn09fwzeLf8jxWOLNFajZoCpLp3x7yet7sd1Q60p2bD/An38eA2DRok3cfsc1tjJut5e3hi3l6NFUAHbuOEDhwvkJC3PQuMm1fDJ7PSdPpmNZMGrEZ3y5/PI4tv+pGnUqsHPrn+zfewSApbPW07D5dbYy7kwPo16Zz9HkkwDs3Ppn1rkgGNRsXI2d3/3Gn7/6snmL313O7Z3q/e0yDqeDqHyRhEeGER4ZTlhEGJnpbgCuubkS1Rtcy3s/DGPEqtdIrFfFj5H9czXqVWLn5j/Yv+f0uW/aahq0rpGjXIsH6vL5rPV8s+wn2/rb293AvA+/4tQJF5ZlMXbAHFbO3+iXul9qXgy//QWaXxpCH3/8MU8++SQAiYmJXHON7yT70ksvMXHiRH9U4S/FFytAUrYLd9KhFPLFRhGTLzJr3aH9x9nwzc6s5ceevpN1X+3A4/FyNOkkg56ayZ+nT57BpmjxAiQdOBN/8sET5IuNJiZ/pK3cu68t5KvFP55zO0061OLIoRTWfLH1ktX1UihasiBJ+49lLScfOE6+uGhi8kfZyr374my+mv99jucXLlaA7oPaM7TnR5je4LsNWXxCHElJKVnLSUkp5M8fRUxMRNa6QwdPsH7db1nLj/e8g7VrduHxmJQuXYSCBfMxeOg9jJ/wCA8+VI9T5+hauNzFF/+754IdWcuP9W/Guv/bjscTHN0h8WWKkrTvTNdV0r4j5CsQQ0xs9N8qs/yjrzh1LJUZ+95n1v4P2P/rQdYt8V38U46cZMn7X9D9+v5MeGE6A+f1p2ipwv4L7h8qWqIgSQeOZy0nHzxx+tg/69z3yjy+WrQpx/NLXxlPwSL5GfTRo4z79Gnu69uEUylpOcrJ5c0vDaEpU6Ywc+ZMevToQYcOHejQoQM9evRgxowZzJ4d+LEUhsPAsnIOkffmclGLjArnxWF3U6JMYUa9ttAf1bvkHA4D/mb8f6XNQ/WYMS7nuJnLncPhILcfWv478TvDHDz3bhc+eGUuxw6nnLf85chhGLm9/Zi5TBuJigrn5YFtKVmqEMOHLQV8r0HNG65k0MD5PP7YRGLjouj6SHB2DxjGOc4FZi7nguhwXnyro+9cEERdwQ7HOd7vbJ/3vyrT+ZUOnEhK4e7i3bi3THdiC+en/ZO+7qRX2w/nm7nrANi6ejtb1+ygZqNqOTd0mTj3ue/vTZlyhjm5vm4lBveaTJ9WI4ktEMNDTze72NUMCNMy/PYXaH5pCIWFheHxeHKsT09PJzw83B9V+EtJB45TJD42a7loQiwnT7jIOJ3u/Z/44gUYObkbXq/Fs90mkXry8u37/icO7z9O4Wxjm4oWi+PkcRcZae6/eJbdVVVL4gxzsGXD75eiipfU4T+PUrhYgazloiUKcvJYKhlpmX/xLJ8K15WjeNmidBt4F2O/eJ5mD9Tj1tY1eGJ48IwZOXz4BEWK5M9aLlo0lpSUNNLP+vwnJMTx9tgHME2Lp/p+TOrprM+R5FN88/UOXK5MPB6TL7/4mapVS/k1hosl6eBximQ/FhLifOeCtFzOBVMf850LHp4QVOeCw3uTKVKiUNZy0VKFSTl6inRXxt8qc0vbG/ls0ko8bg+uFBdfTFnFdbddS74CMdz7fFvbvgzDwHMZD5o/vP+4/dgvXuD0ue/8xz7A0cMprPl8M65TGXjcXlYu2EjlGuUuVXXlEvFLQ6h79+60adOGAQMGMHr0aN5++20GDBhAhw4d6N69uz+q8Jc2rv2NytXKULKsL4XbvH0t1n613VYmOiaCoR92YfXKXxjy3BwyM3I27ILVpm93Url6WUqenvnV7N7arF3xz7q3Em8sz09rfzt/wcvQpq+2UbnmFZS8Mh6AZg/UZe3nuY+DOtv2jbt54IYB9Go0mF6NBrNsyjesWriJ0U9Pv5RVvqi+/243VauWolQp34WvZasarFm901YmOjqCt0bdxzff7OD11xaQmXnm8//1qu3c1qAKERG+Sai31K3Ijh0H/BfARbRxza+nzwVFAGh+942s/b9ttjLRMREMnfQIq7/8hSHPzAq6c8HG5T9RpXYFSl1dHIAW3RuzduF3f7vMr5t2c+vddQBfRuTmljewbf1O0k6m06rHndRtdxMAV1W/gko3Xs13n527Oz3QNn2zg8rXl6PkFafPfZ3qsPaLn//287/99CfqNa9ORKTvC/3NjRPZufmPS1JXuXT8Mn2+ZcuW3Hjjjaxdu5bDhw9jmiY33HADvXv3plixYv6owl86cSyVEa/MZ8CwjoSFOzmw7yjDBsyjQtWS9H2lNT3veZdWHW8ioURB6jSsQp2GZwYAPvfoR5w8Edx9wieOpjLyuTm8OOZ+wiKcHNh7lOH9Z1Lh2tI88WZ7erUadd5tlCxXlEN/HvVDbS++E0dOMbLvNF4c/whhEWEc2JPE8D5TqHBdWZ4Yfh+9Gg0OdBUvqePHXQz9zxJeebWd7/O//xhD3lxMxUrFeap/cx57ZAJt2takWLEC1K1Xibr1KmU9t/+T01m0cCOxcVG890FXHA6DXbsO8t64TwMY0YU7cTSVES/NZcCIe32vxR9HGfbCJ1SoWoq+r7alZ4extLq3tu9ccHtV6txeNeu5zz0yISjOBceTUhjedRwvzXmK8Igw9v92iKEPjqVizfI8Of5xutfof84yAO8++RG9xzzMhF9GYXpNfli5hdlDF2GaJq+0+Q89336YBwbejekxeaPjSFKOnAxwxOd24sgpRvafyYvjHvK93/9NZvhTM6iQWJonhtxDr+Zv/eXzl0xdTf4CMYxZ3A+H08GvP+/jwzfyxpCJy2EQs78YVm4d4pe5O6u/HOgqBIyRGjwp+EviVGqgaxBQmZWCs8vpYgg/EtrvvXfrzvMXysPCypUJdBUC6tPdI/y6v017y/ptXzXK7vXbvnKjGyqKiIiIjTeEbjMYOpGKiIiInEUZIREREbG5HKa1+4syQiIiIhKylBESERERm1CaNaaMkIiIiIQsZYRERETExmuFTp4kdCIVEREROYsyQiIiImJjhlCeJHQiFRERETmLMkIiIiJio1ljIiIiIiFAGSERERGx0awxERERkRCghpCIiIiELHWNiYiIiI2pwdIiIiIieZ8yQiIiImLjDaE8SehEKiIiInIWZYRERETERtPnRUREREKAMkIiIiJiox9dFREREQkBygiJiIiIjdfSfYRERERE8jxlhERERMRG9xESERERCQHKCImIiIiNqfsIiYiIiOR9ygiJiIiIjcYIiYiIiIQANYREREQkZKlrTERERGx0Q0URERGREBCUGaHnF8wMdBUCJt0Kyrfsokm3IgJdhYBKN8MDXYWAKRN+JNBVCKiTZlSgqxBQXrYFugohRT+6KiIiIhICQju9ICIiIjl4dUNFERERkbxPGSERERGxMdGsMREREZE8TxkhERERsdEYIREREZEQoIyQiIiI2OhHV0VERERCgDJCIiIiYmPqt8ZERERE8j5lhERERMRGY4REREREQoAaQiIiIhKy1DUmIiIiNqZuqCgiIiKS9ykjJCIiIjZe/eiqiIiISN6njJCIiIjYaIyQiIiISAhQRkhERERsNEZIREREJAQoIyQiIiI2GiMkIiIiEgKUERIREREbrzJCIiIiInmfMkIiIiJiY2rWmIiIiEjep4yQiIiI2GiMkIiIiEgIUEZIREREbExLY4RERERE8jw1hERERCRkqWtMREREbLwhlCdRQ+i0zeth/iTwuKHUlfBgP4jOZy+zciH83yIIj4ASZaFTT8gXd+bxo4dhSF946V2ILeDf+l9sW9dbLJ5k4nFDySvh3n4OovPZ+4xXLTT5ZpFFeCQUK2PQoZdBvtjg7FfetsHLp5PceNxQ4kqDDn0jiDor3p9Xe1k+zY3hgJhYg/Z9wilS0oHrpMW8sW72/2YSEQW1GoVxS+vgOrR2bHDz5eQMPG4ofoWD1n2jiYqxx//LGjf/93EGhgHRsQat+0RTuIQD02ux9L109mzxAlDhhjCaPByJYQTPZ+Gn9QafTHTicUPpKy26PunNcfx/ucDBikUOwiOgZFmL+3t5yR8HmRkwdayT3dsNLKB8ZYvOvbxERAYklH8t1I797LauN1k6yXs6doOO/Zw5zgNfL/Ty7SIzK/a7ejnzROyhLHSafH/h5HGY/BZ0fwkGTYD44jBvor3M9h/hs9nw5BB4+V24thZMHX3m8bVfwLCn4fiR4D8gTh63+HiESdeXHAyY4KRICYPFkyxbmZ0/WXw5x6LnEAfPjnNStRbMHG0GqMb/zqnjFrNHZNJ5QATPfBhFkeIOPp3ktpVxZ1jMGJbJAy9F0O+dKKrc5GThe74yi993ExkFT78fSa+RkWz/3ssv672BCOWCpJ4wWTAqnY4vRPPEB/kpVNzBF5PSbWXcGRZzh6fR8cUYeozNT6Ubw1j6nq/MTyvdJO8z6flOPnqMzceenz1s/dYTiFAuSMpxmDDcSc+XPQye6CG+hMWcCfZT47YfDZbNdtD/Px5ee89DtRtNJo9yArB4ugPTC6+972HQex7cGbB0ZnCeWkPt2M/u1HGLmSO8dHkpjBcmhFOkhMGSSfa4dv1ksnKOSY8hYfQfF06VWg5mjw6eY/2fMC3Db3+BFpxH60X2yyYoVwmKlfIt39oC1q8EK9vxv3cXVLkeCsX7lmvU9WWRPG44fgR+XAt93/R/3S+F7ZssylaEhFK+D2jd5gbfr7Swsr0gf+yyqFTdoFC8r8x1dQ1+Xg8et5XrNi9nOzd5KVPRQXwp3+FQu4WTH/7Pa4vXPH0+TE/1/ZuZZhEW4fv/vl9NatzuxOE0CAs3qHKjky3fBs/J8ddNXkpWcFKklO/CXqt5BJu/cucaf0aqb11mui8z+r/HMtMtPG7f8eD1kPXaBIOtGw2urGRR/PTx37CFybqVDtvxv2eXQdXrLQqfPv5r3mLx43oDjxsqJVq07OTF4QCHE8pebZF8KPAn9wsRasd+djs2WZSpaBB/OvZbmjvYuNK0xb5vl0XF6gYFT8dera7B1vVW0Mce6tQQAo4mQeGiZ5YLxUO6yyDddWbdlZV9WaEjh3zLqz8Hj9vgVAoULAKPvwzFSvu33pfK8SSyTnIABeMh3YXt9biissGunyyOHvKdANYvt/C6ITXF37X9904kWxTIFm+Bor73PiNbvJHRBu16hfPOkxkMui+NNYs9NOsaDkDZSg42rfDi9VhkpFlsWe3l5NHgOTGeSDJt8ccVNchwQUbamTKR0QYte0Yx/ulUhnU+yfolmTTqEgXA9XeEE53fYPgDJxnW+SRFSjiofFO4v8O4YEeTDArHn3m/CsVD2lnH/1WVLbb9aJB8+vj/Zrkj6/i/9gaL4qeP/eRD8MU8B7XqB2eGJNSO/eyOJVlZDRyAAqdjz34eKHtW7BuWm3ki9tyYOPz2908sXryYZs2a0bhxYz7++OMcj2/dupW77rqLVq1a8dhjj5GScv43xy8DGfbv3/+Xj5csWdIf1Tgny4TchjM4nGf+XyERWt4P414DhwG3NIF8sRZhwXO+/9ssC3L7mZnsr8dV1xrceZ/Bh6+ZGA6o3dggJpagfD0sM9dwbfEe2G3y5XQPT78fSZGSDr5d6GHq65n0fSeSFt3CWfKhm1G9MogtZFDhegf//SV4LoSWdY7Pf7bz06E9Xr6akUHv9/JTuISDdYsymPmmix5j8vF/0zPIV8DBMx/H4MmE6YNcrJ6XwS3tgmOQzN+Jv2KiRev7vYx5NQyHYVGviZXj+N+zE8a8GsbtrU2q1w6ehnB2oXbsZ3euz4Fhi91Bk/tg4mseDIfBTXkk9mBx6NAhRo4cybx584iIiKBjx47cdNNNXH311Vll3njjDfr06cOtt97KkCFDmDBhAv369fvL7fqlIfTYY4+xZ88eEhISbGlGAMMwWLFihT+qcU6FE2D39jPLx5MhJr9FZNSZdekuqJgIde/0LR9LhoWTIV+sf+vqD4XiYc/2M+/TiWSIyQ+RUWfOEukui6urGdx8p+9qcTzZYukUi5ggfD0KJhjs3XGm4ZKSbBGdHyKyxbtzo5crqjooUtIXb50WThZ/4MaVApkZFs0fDifm9IDJlTPdFCkZPF0jBeMN/txx5v0+eSRn/Ls2eihbNYzCJXzx39g8gk/HZ+BKsdi21kOzx6IICzcIC4frbw9n62oPt7TzeygXpHC8xW/bz7R6jiX7vuRERp8pk+aCStUs6jf1ZJWZN9mRdfyv/z+DqWOd3NfTy80Ng7MRBKF37GdXKN5g7/Yz54FzxX5VNYPad/paPseTLT6dYgZ97LnxXgZjd862Zs0aateuTcGCBQFo0qQJn332Gb169coqY5omqam+MQxpaWkUKHD+mUt+6RqbMWMGV155JUOHDmXlypW2v0A3ggCq1oTft8OhP33Lq5ZC9ZvtZY4fgeHPQNrpMSLLZkCt23L/BhHsKtc0+O92OPyn74T47VKLxJvtgZ44Am8/Y5J2eszI8pkWNW8zgmqm0P9UrOFk73aTpD99J8F1y7xcc7PTVqbU1Q5+32Jy8pgv3q1rTQoXM8hXwGDdMi/Lp1v2gkkAACAASURBVPoGTp88ZrHhcy/X3xY8s8auqhHGHzu8HPnTN67pu2WZVK5t/4pb8mone7Z4OHXM9xptW+ehUDGDfAUclLjKydZvffF7PRbb13soXdn++l3Orq1p8fs2g4Onj///W+Lg+pvtjZnjR+A//cOyjv/F0x3c1MDEMODHtQYfj3Py1ODgbgRB6B372VWqabBnu0XS6djXLDW59qzYU47AO894SD8d+5czTWrc5gj62AMtJSWFffv25fg7u1vr8OHDxMfHZy0nJCRw6NAhW5nnnnuOAQMGULduXdasWUPHjh3Pu3/DOjtFc4ls3ryZOXPmMGjQoH+9rVV7Kl6EGtlt2QDzJ4LHA/EloGt/SDoIU0b6ZomBb/r8V4t9KdSrr4F7e5JjiuyjTQzemm1dsunz6ZZ/LrBbN/im0Ho9ULQE3N/fwZEDMGOUybPjfBe5rxeZfLPYwjKh/DUG7XsaRERe2hNCunVpRuFu2+Dls4/ceD1QuIRBx6cjOHLA4pPRmfR7x5caXLPYw5rFHpxhvunjbXqEU7ycg3SXxazhmSTvt8CCBveEUaPhpXmf0s1Lk4Pf+Z2bLyZn4HVD4RIO2j0VzbGDJgtHp9FjbH4A1i/JZP3iTJzhEJ3foMXjUSSUc+JKMVn6bjoHfvd1lZS/zjd9Piz84n4WyoQfuajby+6nDQZzT0+fTyhp8Uh/L0kHDSaNcPLae74s0JcLHaxc5BtEXeEa3/T5iEh4vmsYp05CoSJntlfhGpPOvS9u9+hJM+r8hS6Cy/XY98d9bX7ZcHr6vAeKljDo1N/JkQMWs0Z56T/Od+x9s8jLt4vNrNjb9XRe8tgBml358yXfR3ZP/HCv3/Z19bd1GDt2bI71vXr1onfv3lnL7777LhkZGfTt2xeA2bNn8/PPP/Paa68BkJ6ezl133cXgwYOpVq0akyZNYu3atXzwwQd/uX+/NYQupkvREAoW/moIXa4uVUMoWFyqhlAwuJQNoWDgr4bQ5SqUbvCXm7zcEBp01fu5DmqOi4sjLu7Mzfrmz5/P999/zxtvvAHAO++8g2VZWV1jmzdvZuDAgcybNw8Al8tFnTp1+PHHH/9y/6F9VRUREZEcTMt/Dc+zGzznUqdOHcaMGcPRo0eJjo5m+fLltl6mcuXKcfDgQX7//XfKly/PihUrSExMPO921RASERGRy16xYsXo168fDzzwAG63m/bt21OtWjW6detGnz59SExMZPDgwfTt2xfLsihSpAhvvnn+G/ypayzIqGtMXWOhSl1j6hoLZf7uGuux6X6/7WtcjWl+21duQvuTJSIiIiEttNMLIiIiksPl8Btg/qKMkIiIiIQsNYREREQkZKlrTERERGz8OX0+0EInUhEREZGzKCMkIiIiNiYaLC0iIiKS5ykjJCIiIjZeTZ8XERERyfuUERIREREbzRoTERERCQHKCImIiIiNfmJDREREJAQoIyQiIiI2uo+QiIiISAhQRkhERERsNEZIREREJAQoIyQiIiI2uo+QiIiISAhQQ0hERERClrrGRERExEaDpUVERERCgDJCIiIiYqMbKoqIiIiEAGWERERExEZjhERERERCgDJCIiIiYqOMkIiIiEgIUEZIREREbJQREhEREQkBQZkRuiXKDHQVAigz0BUIKAeeQFdBAiYoT1cXjdtKD3QVAuqYGdrx+5syQiIiIiIhILS/YomIiEgOurO0iIiISAhQRkhERERsNEZIREREJASoISQiIiIhS11jIiIiYqOuMREREZEQoIyQiIiI2CgjJCIiIhIClBESERERG2WEREREREKAMkIiIiJiYykjJCIiIpL3KSMkIiIiNvrRVREREZEQoIyQiIiI2GjWmIiIiEgIUEZIREREbDRrTERERCQEKCMkIiIiNhojJCIiIhIC1BASERGRkKWuMREREbHRYGkRERGREKCMkIiIiNhosLSIiIhICFBGSERERGwsK9A18B9lhERERCRkKSMkIiIiNiYaIyQiIiKS5ykjJCIiIja6j5CIiIhICFBG6B+yLHhhsEHF8tClYwgNqye0Ywdf/M8PhorloWvHQNfG/xR/3oz/67UGo8c7yXRDxfIWrz7jJX8+e5np8xzMmO8gKgKuLGfxYl8vBeLgRAq8PtLJ9l8NoqOgTVOTTu3MwARygdauc/LBh5G4Mw3Kl/fybP908p0V/9x54cxfEEFkpEXZsib9nkgnLs732PyF4SxdGk5GJlSqaPLM0+lERPg/jotN9xGSXP22B7r2c/DFqtD5gPxPKMcOvvi79IPlqwJdk8BQ/Hkz/qPH4aX/OBnxmofFUz2ULmkx6gP7ZWHDDwYTpzsY/5aHORM81Ktt8upwJwBD33ESEw0LPvLw8TgP3643WLUmeM4Rx48bDBkaxaCBaUybkkrJkibvj4+0ldn0g5MZMyMY8ZaLCeNd1L7Jw/ARUQB8/XUY8+aHM2K4i8kTXWRkwJxP8kArKMT4rSH05ZdfMnXqVPbu3WtbP2vWLH9V4V+bscDgruYWjW8LvWxIKMcOMH0BtG8OTW4LdE0CQ/HnzfjXfmdwbWWLcqV9y3e3Mln2pcN2D5lfdhjUrmlRPMG3fHs9i1VrDdxu32MtGpk4nRAeDvVrW3yxKni+X3/3vZPKlUxKl/YF3LqVmy9XhNvi37nTQc2aXhLifSvr1/OwZm0Ybjd8/kUY93RwExcHDgc81S+Dxo3cgQjlorMs//0Fml8+scOHD2fatGns2bOHe++9l4ULF2Y9NnPmTH9U4aIY0NeiRaPL4F0LgFCOHeClvtCiUaBrETiKP2/Gf/CwQfH4M8d1sXg4lWqQ6jpTJrGqxYYfDPYf9C0v/NSB221wPAWqVbVY8oUDtwdcLvjia4Pko34O4l84fNhBQsKZrrz4eIvUVANXtvirVDHZ9IOTgwd9ma5PPwvH7TZISTH4Y5+DY8cN+j8bTZdHYpg0OYL8+UP3PBms/DJGaNWqVcyfP5+wsDA6d+5M165diYiIoGnTpliXQ3NQRCQEmRbkdrsYR7avyDWrWXR/0Evfl8JwGBZtmlkUiLMID4OnHvfy1rtO7n4kjKKFLW6+weLHn4Ona8w8x+Une/zXVfPy0AOZDHg5GocDmjZ1ExdnERZm4fEYfL/RyZuD0oiIgDeHRPHhhEh698rwTwCXUCjNGvNLQ8iyLAzD96JeccUVvP/++3Tp0oXChQtnrRcREf8qkWCxZduZq/7hZIiLtYiJPlMm1QU3XGfRrrkHgENJ8M5EBwXi4OBheLK7b+A0wPhpDsqWCp4vt8USTLZtO3MZTE4yiI21iM4Wv8sF113noXkzX5dXUpLBxEmRxMVB0SIm9et6sgZXN27kZvIU+xgjufz5pWvszjvvpHPnzmzevBmAChUqMHr0aPr27ZtjzJCIiPjHzbUsNv9i8N99vuU5ixw0uMXekDmcDF37hnEq1bc8fpqDpg1NDANmL3LwzkTfZeTIUZi31EGzO4KnIVTrBi+/bHOyb5/vC/mixeHcUsdjK5OcbNC3Xwypp+Of+nEEtzdwYxhwa30P/7cqjIwM31iXb74No3Ilr7/DkH/JLxmhXr16UbNmTfJlm5NYs2ZN5s2bx8SJE/1RBREROUuRQjDoWS9PveIb/FumpMUbL3jZut1g4DAncyZ4uLIsPNzJ5L7HwzAtqJFo8fwTvov9I/eZvPCGk7YP+S4lPbt4ubZy8DSEChWyeK5/Oi8PjMbtgVIlLV54Lo3tOxwMGx7FhPEuypa16HRvJt17xmCZBomJHvr28XV9tWnt5uRJg27dYzC9BhUqeOn5eHqAo7o4QqlrzLCCcJCO5+DVga6CBIhDd3yQEOW2POcvlIcdM/NGA+NCFS+136/7S1z0it/2taXVq37bV250Q0URERGx0Q0VRUREREKAMkIiIiJiE3yDZi6cMkIiIiISspQREhEREZtQmjWmjJCIiIiELGWERERExEYZIREREZEQoIyQiIiI2ITQpDFlhERERCR0KSMkIiIiNhojJCIiIhIClBESERERuxAaJKSMkIiIiIQsNYREREQkZKlrTERERGw0WFpEREQkBCgjJCIiIjaWBkuLiIiI5H3KCImIiIiNxgiJiIiIhABlhERERMROGSERERGRvE8ZIREREbHRrDERERGREKCMkIiIiNgpIyQiIiKS9ykjJCIiIja6j5CIiIhICFBGSEREROw0RkhEREQk71NDSEREREKWusZERETERoOlRUREREJAUGaEHGq/iUiIcRqhfd5zh9Lo3ctBCL3coX1kiYiISEhTQ0hERETOYvjx7+9bvHgxzZo1o3Hjxnz88cc5Hv/999/p3LkzrVq14uGHH+bEiRPn3aYaQiIiInLZO3ToECNHjmT69OksWLCAWbNm8euvv2Y9blkWjz/+ON26dWPRokVUqVKFDz744LzbDcoxQiIiInIJ+XGMUEpKCikpKTnWx8XFERcXl7W8Zs0aateuTcGCBQFo0qQJn332Gb169QJg69atxMTEUL9+fQC6d++e63bPpoaQiIiIBMzkyZMZO3ZsjvW9evWid+/eWcuHDx8mPj4+azkhIYHNmzdnLe/du5eiRYvywgsvsG3bNsqXL89LL7103v2rISQiIiJ2fswIPfjgg7Rt2zbH+uzZIADTNDGMM2OKLMuyLXs8HjZs2MC0adNITExk1KhRDBkyhCFDhvzl/tUQEhERkYA5uwvsXIoXL87333+ftZyUlERCQkLWcnx8POXKlSMxMRGAFi1a0KdPn/NuV4OlRURExM4y/Pf3N9WpU4e1a9dy9OhR0tLSWL58edZ4IIDrr7+eo0ePsn37dgBWrlzJNddcc97tKiMkIiIil71ixYrRr18/HnjgAdxuN+3bt6datWp069aNPn36kJiYyDvvvMOAAQNIS0ujePHiDB069LzbNSzLCrr7R5oHKwa6CiIifmViBroKAXXAmxroKgRUmVIH/Lq/chPP34C4WP7b9Rm/7Ss36hoTERGRkKWuMREREbELur6iC6eMkIiIiIQsNYREREQkZKlrTEREROz+wbT2YKeMkIiIiIQsZYRERETExtBgaREREZG8TxkhERERsVNGSERERCTvU0ZIRERE7DRrTERERCTvU0ZIRERE7DRGSERERCTvU0ZIRERE7JQREhEREcn7lBESERERO2WERERERPI+ZYRERETETvcREhEREcn71BASERGRkKWuMREREbExNFhaREREJO9TRkhERETsQigjpIbQP2RZ8PxgqFgeunYMdG38K5RjB8Wv+PNm/KvWwqgPHGS6fbENetYkfz57mY/nGkyfbxAZCeXLWgzoZ1EwDo6nwKARBtt/NYiOgrZNLe67K7iuoOvWhTHhw0jcmVC+vMlT/dPId1b88+eFs3BBBJGRULasSe8n0oiL8z22cGE4ny6NIDMTKlT08tTT6URE+D8OuXDqGvsHftsDXfrB8lWBron/hXLsoPgVf96M/+hxGDDEwahBJkunmZQuaTHiffu06fWbYMIMgwkjTOZNMKlfGwYO9106/jPWICYaFk02mf6uyTfrDb5aE4hILszx4wbDh0bxysA0PpqSSomSJh+Oj7KV+fEHJ7NmRjLsLRfvj0/lxps8jBwRDcA3X4excH4EQ4en8uHEVDIyDOZ+olZQsPFbQ2jPnj0cOnQIgDlz5vD666+zbNkyf+3+opi+ANo3hya3Bbom/hfKsYPiV/x5M/413xlcWxnKlfYtd2xtsfRLAytbUueXnQa1a1oUT/At31Hf4qs1kOn2PdaysYXTCRHhUP9mi+Wrguf+Mxu/d1KxkpfSpU0AWrbKZMWKcFv8O3c6qVHTQ3y8b2Xdem7WrQ3D7YYvvginfYdM4uLA4YC+/dJp1MgdiFDkXzhv19jEiRMZO3YsXq+XUqVKUbFiRSpVqkSlSpWoWLEipUuXPu9OPvroI6ZOnYppmtSuXZsDBw7QqFEj5s6dy+7du+nZs+dFCeZSe6mv79/V3we2HoEQyrGD4lf8vn/zWvwHDkPxhDNX/WLxcCrVINVFVvdYtSoW0+Y62H/QomRxmP+pgdttcCLF99ji5QbXJ1pkZsIXqwzCwiBYBpgcPuwgIVv88fEWrlQDl4us7rEqVbzMnx/BoYMGxYpbfP5ZOG63QUqKwb59Do4fN3ju2RiOHDFITPTS7dH0AEVzcYXSrLHzNoTef/99hg4dSrVq1fjjjz/YuXMnO3bs4Ouvv2bXrl0AVKhQgRkzZpxzG3PnzmXZsmUkJyfTokUL1q1bR2RkJB06dKB9+/ZB0xASEclLLBNyy984svUV1LwOejxk0WeAA4fDNw6oQJxFeBj072Ex/F2D9o84KFoY6txg8cPPwZMRss5xsc8ef2I1Lw88kMErL8fgcECTppnExpmEhYHXAxs3hvHaIBcRETB0SDSTJkTSo1eGfwKQi+K8DaH8+fNz2223ERYWRkJCAjVr1rQ9vm/fvqwG0bmYpklERASlSpWia9euREZGZj3m9XovsOoiIvJvlCgGm7cZ/C+DczgZ4mItYqLPlEl1wQ3XWdzV3FfmUBKMmWhQIM6XUXqyu0XBON9jH0w1KFs6eFIJCQkW27adabglJxnExlpEZ4vf5YJq13lo2sydVeajSZHExVkUKWJRt647K3t0eyM306ZEAnmgIaSf2Djj0UcfZc6cOed8vHTp0jRo0OAvt9G4cWPuv/9+vF4vvXv3BmD79u106tSJpk2b/sMqi4jIxVCnlsXmX+C/+3zLsxYZNLzF3pA5nAxd+jo4lepbfn+qQbPbLQwDZi80GDvRd8FMPgpzlxo0vz14GkI1b/CwbZuTfft8l8LFiyOoU8c+xudIsoOn+uUj9XT8H38cScMGHgwD6tV3s2pVOBkZvuzS6m/DqFRJX+6DzXkzQkOGDMHtdvPNN99Qr149qlSpQqVKlYjO3mQ+jyeeeILvvvsOp9OZtS4iIoLevXtz6623XljNRUTkXylSCF5/zqTvyw48bihTCt58weTn7fDyMAfzJphcWRYe7mRxb3cHpgU1Ei1e7Otr7HS73+K5Nxy0fsg3wLpXV4vEKgEO6h8oVMiif/90XhsYjccDJUqaPPtcGjt2OBgxPJr3x6dSpqxJx3sz6N0zH6YJ1yZ66d3HNw6oVWs3J08aPN49H6YXKlQw6f54WoCjukiCpz37rxmWda5eUp+9e/eyY8eOrL/t27ezf/9+Spcuzeeff+6vetqYBysGZL8iIoFiYga6CgF1wJsa6CoEVJlSB/y6v/KjRvhtX7/3fdJv+8rNeTNCZcuWpWzZsjRq1ChrncvlYufOnZe0YiIiIhIgIZQRuqD7CMXExFC9evWLXRcRERERv9JPbIiIiIhNKN1HSD+xISIiIiFLGSERERGxU0ZIREREJO9TQ0hERERClrrGRERExE5dYyIiIiJ5nzJCIiIiYqPp8yIiIiIhQBkhERERsbOMQNfAb5QREhERkZCljJCIiIjYaYyQiIiISN6njJCIiIjYaNaYiIiISAhQRkhERETslBESERERyfuUERIREREbjRESERERCQHKCImIiIidMkIiIiIieZ8aQiIiIhKy1DUmIiIiduoaExEREcn7lBESERERG02fFxEREQkBagiJiIhIyFJDSEREREKWxgiJiIiIncYIiYiIiOR9ygiJiIiIjWaNiYiIiISAoMwIrU43A12FgNmRWSLQVQioetG/BboKAeXFCHQVAiY8lL6iSg4zjtcKdBUC6uVSft5hCB1uygiJiIhIyArKjJCIiIhcQsoIiYiIiOR9ygiJiIiITSgNyVNGSEREREKWGkIiIiISstQ1JiIiInbqGhMRERHJ+5QREhERERsNlhYREREJAcoIiYiIiJ0yQiIiIiJ5nzJCIiIiYqeMkIiIiEjep4yQiIiI2GjWmIiIiEgIUEZIRERE7JQREhEREcn7lBESERERO2WERERERPI+ZYRERETERrPGREREREKAGkIiIiISstQ1JiIiInbqGhMRERHJ+5QREhERERsNlhYREREJAcoIiYiIiJ0yQiIiIiJ5nzJCIiIiYqeMkIiIiEjep4yQiIiI2BiBroAfKSMkIiIiIUsZIREREbELoTFCagidtnk9zJ3kwOOG0lfCQ/1MovPZy6xYaLBykUFEBJQoa9Gpp0X+OMjMgI/HGuzeYYAFV1a2uK+XRURkYGK5EL99l8HXU07hdUP8FWHc2SeWyBh7wnDn2gxWTz+FYRhExxo07hVLoRK+j9APS11sXp6OJ9Oi2NVh3NknjrDw4EmufrfOyZQJkXjcBuXKe+nzVDoxZ73/S+aHs3RhBBGRFqXLmnTvnU5snL3MmwOjKFzEonvvDP9V/iL4fp2TaRMicLsNypU36ZVL/Evnh7NsYXhW/I/2zsgR/5CBURQuYvJo70z/Vf4i+G6dk8kfRuJ2G1xR3ssTT+eMf/H8cJYs8L3/ZcqaPN7nzPvfqW0+isSfuXK0uzuTBnd4/BjBhQvl2AH+3HiKH6cl4fVYFCoXSe0exQmPcdrK/LH+JJtnJmM4DCLyO7np8WLEFo8gM9XLunEHSfkzE8uC8rfFcU3bIgGKRC6UusaAk8dh0lsOerxk8sYEk/jiFnMn2i/i23+Ez2YbPDXE5JV3TRJrwdTRvpdv6QwD0wsD3zMZ+J6JOxOWzQyeRoDrhMlno1No83wBHnmvCAWLO/n6o1RbGXeGxbK3TtDm+QI89HZhrqoVycoPTgGwc006m5akcffrBen6TmE8GbBxgSsQoVyQE8cN3h4exfOvpPHuR6kUL2Ey+UN7K3bzj07mzopg0DAXo993ccONHt4ZGWUrM3dWBL9ssZ9Ag8GJ4zBmeCTPvJLOOx+5KF7CZOpZ8W/50cn8WeG8OiyNke+nUfNGL++eFf/8WeFsC8r4DUYNi+L5gWm8P9n3/n909vv/g5NPZkbwxnAXYz5wccNNHsaM8MW/7w+D/LEw5gNX1l+wNARCOXaA9BMe1o49SL3+pWg1pjz5i0Xww7RkWxlPhsnq0Qeo/0wpmr11BaVuyMf3Ew4DsHlmMjFFwmkx6kqa/qccuz4/TtKOtECEctEZlv/+Ai0gDaEhQ4YEYrfntHWTwRWVoFgp3/JtLSzWrzSwsr1B/91lUOV6i8LxvuUadS1+Wg8eN1RMtGjeycLhAIcTylwFRw77P44LteeHTIpXCKdQSV92p3rTaH5ZlY6V7QWwTAvLggyXb11muoXzdMZn68p0arWJITrWgeEwaNwzlqoNo3Lu6DL1w0YnFSqalCzti61pSzerVoTb3v/fdjq4roaXoqe/+d5c18OGdWG43b7Ht/zoZNN3Tu5s4fZ39f+1HzeG2eK/s6Wbr1eE5Yi/Wrb4a9f18N06Z1b8P5+Ov0kQxr/peycVKpmUOh1/s1Zuvjrr/f91l4Pq2eKvk+3937bVicNp8cwT0fR6JIYZUyLwegMRyT8XyrEDHPjJRZGro4grGQFAhSYF2fNNylnnPsCCTJcJgCfdzDr31eyaQI0HfReFtGMevG6LiBjlF4LNJe8ae/7553OsW7lyJSdOnABg8ODBl7oK53U0CQoXPfPBLxQPaS6DdBdZ3WNXVrZYsdDBkUMWRYrB6s8NPG6DUylwTc0z2zpyCL6cb/DAE6afo7hwJ5O8xBY9c/DGFnWQ6bLITLOIjPEd8BHRDhr1jGV6/2NExTmwTItO/ykEwLH9XlwnTOa8cpxTR72UrhrBrV3yBySWC5F82EHRhDPvV9F4C5fLIM1FVhdBxSomi+dHcPiQQUIxiy8/D8fjNjiZ4mswjx8XycDBaXy2JDxAUVy45MMGRRLOfP6L5Bq/l6Xzw7PiX5ktfiyYMC6ClwanszwY409yUDT+rPc/9az4K9vf/y+yxe/1GlSv4eWhRzLweOHVF2KIyWfR+q7Lv1EYyrEDuJLdxBQ9cxmMKRKG22XiSTOzusfCox3c+Fgxlr+wl8hYB5YJjd8oC4BhGBhOWD16P3vXnqLMjfmJPd2oCnqXQabGXy55Q6hgwYIsWLCA7t27Exfn61Ret24dN95446Xe9d9mmeQ6V9CRLctfMRFa3m/xzmsODAPqNrHIF2sRlu28v2cXjHvVQcNWFtfVvuTVvmgsCzByvgCG48y6pD0e1s500WVcYQqVCGPjIhcLB6fw4NuF8Hpgz4+ZtB1QgLBwg2WjUvh26ikadov1YxQXzjzHAe/I9sXumkQvHR/I5M1XonE44I4mbmJjLQwHDBsUxcOPZ1C4SHCeOSwr96my2eOvmmhyzwOZ/OeVKAwH3N7ETf5YXxZ02KAoujyeGbzxm7l+/G3xX1vNy72dM3nj5WgMBzS60/f+h4VZ3NncftFv0z6TxfPDg6IxEMqxw7k/+9nPfcf+m8GWOUdoMfoKYotHsH3pMb4etp9mb5XDOP3i3fJESW581OSbYX/y85wjVOtY1E8RyMVwyRtCzz77LPXr12fUqFE8+eST3HTTTUyePJm2bdte6l3/bYUTYPd2g/81gY8nQ0x+i8hsvTvpLl8XWL07fWWOJcOCyQb5Tl/rN3xlMG2MwX09LW5qGFwXhNh4Jwd2njlxnTxiEpXfICLqzMlg96ZMSlUJzxocfX3zaP5vwinSUizyF3ZQ8ebIrMHVVW+LYu1M+xijy1l8gsnObWcOhSPJBvljLaKiz5RxueDaah4aN3Vnlfn4o0gOHTA4dMDBxPd84yqOHTUwTXBnQu+ngmPAdNEEi53bzrzXucWf5oJrqnm5o6knq8z0j+DgAYNDBwwmvef7Fnz8qIFpGrgzDXoGSfzxCSY7tv+N9/+6/2/v3uN7rP8/jj8/O2GazexEfFPxtQ7O5XzOeU6Vb0k5zCkhhaIkKiWkJoeEEgrF1ymnOSR9xaoflVGhQszGNqeZMTt8fn98arlstLJd17brcb/dPreb69r7uq7X+/b+fOy11/t9XZ90tW7vGv/EBIc+mQM7pAAAIABJREFU+qCYfEpJWzd76NbbMnXr7a7KitMpuReS21Ds3HdJKhngoVM/X8raTjmVLq+b3ORR/M9MMO77CwoMLSGfENd7/N9t/fTt/Hilns/Q6V9T5XdLMXn7e8izhJtuaVRKx746b3o/cGNMmcysX7++Zs+ercWLF2vSpEnKKGCTyHfVdurX/dLJ467tbescqlHfmMycPSVNGemmi7//fl+3xKE6zZxyOKTvv5KWvOPQ8NczC10SJEkVa3op9kCazsS6fsnt2XBRleoaF0wG3+6hY/su68IZ1394P3+VKt9gd3n7uqlKw2La/2Wq0lKdcjqd+uWrVIVULjxTJDVrZ+jAT+6KjXElAxvWeKpuA+OCz9OnHHphhLdSfh//pYu81LhFmkLvzNS8JRf09mzXIuq2HdLUqFl6oUmCJKlG7Qwd/Mktq/8b13iqTg79f3FEiaz+//eK/r+3JEURs12LqNt0SFfDZmmFJgmSpJr3ZOjAj+46/nv/16/xVL0c+v/88CvGf7GXmrRIk8Mh/XbYTYvmu9bGpKZKa1d7qnGzwrFg2M59l6SyNUoq8eBFJcW67nL8edNZlb/XOK3vf1sxxf+QootnXf2K+SZZJYM8VbyUh47uPK+9SxPldDqVkZapozvPK7iqt+n9yBdOE18WczivXBVmgmXLlmnDhg2aN2/ePz7H9iOV8jAil+hvpBXz3JSeLgWVlfo8m6nEE9KCCDeNm+X65b91tUNb17jWhFS+y3X7vFcx6YW+brpwXip9xV2Tle5y3UKf1w5cLpvn55SkQ7tS9b8FF5SR7pRfiLvaDy+lcycyFDn9vHpP85ckfbsuRd+tvSh3D4eK+zjU8nEfBdziocwMp6KWpujA9kvKzHQlTa0HZ7/9Pi80LvFrnp9TknZ9/fvt8+lSSFmnho26qBNxbprxVnG9Pdt1B9zaVZ5a/6mnnJkO3XF3uh5/MlXFrnpEwuIFXkpKcuTb7fMZ+fS8191f/377fLpDIWUz9dSoSzoZ56aZbxVTxGzXXTDrs/ov3XF3hvrn0P+PF3gpKUn5cvu8Zz7eXvJ/X7tuIU9Pl8qWdWr4c67xn/ZmcU2f4xr/Nas8tW61a/zvvDtdA4e6+n/pkvTu9OI68KOb0jMcatQkTT37Xs5xyqkgKix9X3L2nrw/qX6/fX5RojLTnbopxFMNniyr5JNp+nrWCbV/s6Ik6cCGMzq44azcPBwqdpOb7ukXLL9/FdPlCxn6ZvZJnT3q+rxXqOujag+XMUyt5ZWxd3+a5+e8nupDI0y71p5pw0y7Vk5MT4TyQn4kQoVFfiVChUV+JUKFRX4lQoVBfiZCKPjyKxEqLMxOhGo8aV4i9P303CdCa9as0axZs5Senq5evXrp0UcfzbHdtm3b9Morr2jr1q1/ec5CNJsLAADs6uTJk4qIiNCKFSvk5eWlbt26qW7duqpUyVgcSUxM1KRJk3J9Xh54AAAAjExcI5SUlKSYmJhsr6SkJENIO3fuVL169eTn5ydvb2+1adNGkZGR2UIfM2aMhgwZkuuuUhECAACWWbBggWbMmJFt/5AhQ/Tkk09mbcfHxyswMDBrOygoSNHR0YZjFi5cqDvvvFPVq1fP9fVJhAAAgIGZS/J69eqV4yN1/nj24B8yMzOznt0kSU6n07B98OBBbdq0SfPnz9eJEydyfX0SIQAAYJlSpUplS3pyEhISol27dmVtJyQkKCgoKGs7MjJSCQkJevDBB5WWlqb4+Hh1795dixcvvu55WSMEAACMCuBzhBo0aKCoqCidPn1aFy9e1KZNm9SkSZOsnw8dOlQbN27U6tWrNWfOHAUFBf1lEiSRCAEAgEIgODhYw4YNU8+ePdWlSxd16NBB1apVU//+/bV3795/fF6mxgAAgEFBfWxXx44d1bFjR8O+uXPnZmtXvnz5XD1DSKIiBAAAbIyKEAAAMCqgFaH8QEUIAADYFhUhAABgREUIAACg6CMRAgAAtsXUGAAAMCiot8/nBypCAADAtqgIAQAAIypCAAAARR8VIQAAYOBw2qckREUIAADYFhUhAABgZJ+CEBUhAABgX1SEAACAAc8RAgAAsAEqQgAAwIiKEAAAQNFHRQgAABiwRggAAMAGqAgBAAAjKkIAAABFH4kQAACwLabGAACAAYulAQAAbKBQVoReDa1rdQiWcWbaKE3PwTL3ilaHYCm3kECrQ7CU89QZq0OwjMPP1+oQLOVMSbE6BGslmnw9G/2qoSIEoFCwcxIEIP8UyooQAADIP6wRAgAAsAEqQgAAwMhpn5IQFSEAAGBbVIQAAIABa4QAAABsgIoQAAAwoiIEAABQ9FERAgAABo5MqyMwDxUhAABgW1SEAACAEWuEAAAAij4SIQAAYFtMjQEAAAMeqAgAAGADVIQAAIARX7oKAABQ9FERAgAABqwRAgAAsAEqQgAAwIiKEAAAQNFHRQgAABiwRggAAMAGqAgBAAAjniMEAABQ9FERAgAABqwRAgAAsAEqQgAAwIiKEAAAQNFHIgQAAGyLqTEAAGDAYmkAAAAboCIEAACMMu1TEiIRuoY6bWsofPzD8izmocN7jyli4FylnL+YY9tn3ntcR/Yd03+nrjc5yhtTp10N9Xm1W1Yf3xowJ1sfr9XGp3RJPTmjj26vfosuXUjVpgVfaPU7m/SvO27W8wsHZx3v5u6mW+/+l15+KEI7Vv2f2V38R+q0ra7wVx6SZzFPHd73x9hfyrHtM3MH6MgPMYVu7K92b/M7FD4yTJ5eHjq8P1ZTR32ilOTUHNuOmPKIjhyI0/K527L2hT3WQG0friev4p76ZV+Mpo76WGmXM0yK/sbVaVNN4eO6ut7n+2IUMWTeNcd8xLv9dOSHGC2fHilJ8iruqcFv9lCV2rfK4XBo/65DmjniQ12+lGZmF/6xe1vcqfBRHf4c+2eXXHvs3+quI/vjtHzO51n7wno0VNtu9eRV3Eu/7D2mqSOXFK6xb1VV4WPud439D8cV8dQCpSRfY+xn9NaRn45r+czNkiRvnxIa9nZPVagcIoebQ1s+jtKy6RvNDB95gKmxHPgG+GjEnAEa322q+lV7VicOx6vPqw9na1ehSjlNihytxvfXsSDKG+Mb4KNn5j6uVx6eqr53P6O4wyfV97VuuW4zcEoPXUq+pP7VntVTjcbq3jY1VLd9TR396bieuHd01mv35r3a+vGOQpME+Qb4aMTsARr/yDT1qz7SNfbjrzH2G54vlGN/NV//kho+uZtefWK++t83USeOnlb4yA7Z2lW4PUivL3pCjdpVM+xv0KaqOvVqrOcfe1cDW0+WVzFPdenT1Kzwb5hvGR8Nf6evxveYqX61RyvuSILCX/5PtnYV/l1WE9eMVOPO9xj2P/JsR7l7uOmJ+mP1RP0XVayEpx4eEWZW+DfE17+khk95RK8+Pk/9m0/QiaOnFP5cx2ztKlQK1utLBqlR++qG/Q3aVlOn3k30fPdZGthyoryKe6pLv2YmRX/jfMvcpOHTeml8+LvqV2+s4n5LUPjYB7K1q1A5RBNXDlfjjrUN+3s930mJsWc0sPHLGtpqgjqEN9Ud99xmVvj5y2niy2KmJELR0dFZ/46KitLEiRM1ZcoU7dmzx4zL/221WlbVgd2HFPvrSUnS2rlb1KJbw2ztOg1spcgPtul/K74xO8QbVrtVNR3YdUixv5yQJK2dvUUtHmmY6zaVa92qLYu+VGamU+lpGfp6w3dq/IAxKbi7YRU1fqCOpg2eZ0KP8ka2sZ/zmVp0a5CtXaeBLRU5v3CO/dVqNa6ig9HHFHskUZK09qMdat65VrZ2HXo20sZPvtb29cbP7X0P3KMV721T8rkUOZ1OzRizTFtX7jYl9rxQ6767dPDbw1ljvu79rWrxn3rZ2nUccJ8iF/5P269K6vfuOKAlb6yR0+lUZqZTv+w5qqAKAabEfqNqNQnVwT1H/xz7D3eoeZfa2dp16NlIGz/+StvXXTX2D96rFXM//3PsRy/V1uW7TIk9L9RqfqcOfv+bYg/FS5LWffCFWnStm61dx77NFfnRl9r+qfF9PWv0J5o77r+SJP9gX3kW89SFpJxnDlBwmZIIjRs3TpK0aNEiTZgwQSEhIQoICNDYsWP10UcfmRHC3xJYvowSY05nbSfEnFZJX295+5QwtJs5bIE+/2Sn2eHlicDy/kqIOZW1nVMfr9dm/ze/qOWjjeTu4a7iJYup8f115B9S2nCN/hO7a/64pdecUiyIAsv7K/HKPh//o8/FDe1mDluozz+JMju8fBFQ1k8JcWezthNPnFPJUiXkfVMxQ7tZ41Zo26ffZju+/K2B8itzk8bPH6B3NjyjR59uo+RC9Msg8GZ/JVz5eT9+Jscxf+eZj7Rt2VfZjv926w86/osriQqqUEb3D2qVLVkqqALKXTX2cWdzHvuxy7VtdQ5jf9vvY7/wcb2zcaQeHda2cI19OX8lHL9i7GPP/N7/q8b+uSXatjznP3oyMzI1clYfzd7+kqJ3HFDM7384FnYOp3kvq5k6NbZ06VItXLhQvXv3Vu/evbVo0aICmQi5uTnkzOGbdzMyMi2IJn843Nxy/HLhzCv6eL02s0cuktMpzfq/CXrpvyP07Wd7lZ6WntXmznqV5RtQSluXFK5E0c3hyLHPGRkF4NOaT9zcHDl+03Ru++zu4a6ajaro9SELNLRThHx8vdX7mfZ5HWa+ceTR571SjVs0JfJ5fTrnM30TWTCr3VdzczhynJr4W2PfuIpeHzRfQzu8KR+/kuo9snBMC0rXGfvMvzf2k5+Yp4eqDJePX0l1fzb7tDIKNlMSofT0dGVmZsrPz09eXl5Z+728vOTmVvCWKcUfO6UyZf+sbgTc7K/zp5OVmpLzAsLCKOFYosqU88vaDrjZX0mnk3Xpij5er413qRJ67/nFGlBzlJ5rN0FyOLKm0CSp6X/qa8ui7Tn+J1OQucb+yj6XLnJjf7X42LPyD/bN2g4I8dX5sylKvXg5V8efjk/Szo3RSklOVXpahrau2q3QWrfkV7h5LiHmtPHzXq60zp9JVmpK7vovSU0frKPXVz+jeS/9V5+8uS4/wswX8bFn5B9cKmvbNfYXcj/2J89pZ+QVY79yl0JrVcynaPNewvHTKhNyxee9rJ/On7mQ67Gv3fxO+Ye4PjuXLqRq24pvVKnav/IlVtM5nea9LGZKFuLn56dmzZrp8OHDGj9+vCTXWqFu3bqpbdu2ZoTwt+zeslehdSqp3O3BkqSw/vcpam3hWfOQG7s379UddSqrXKUQSVKHAfcpas3uXLfpMKCleo5zLSj1CyqlduHNtfXjP6s/1ZqE6rutP5jRlTy1+7N9xrHvd5+i1mafEihKvt1+QKE1b1G5iq51Le27N1DU5n25Pv7LDXvUOKyGvIp5SpLqt66qg9HH8iXW/LD7s30Kvfe2P8e8T3NFrfsu18fXbVtdT0x+VKO7vJnj1FlB9u3/Dii0ZsU/x/6xhora9DfGfv0eNe5wxdi3qaqDe47mS6z5YffnPyq09m0qd1uQJCmsd1NFbfg+18c36XyPHnvWtbjc08tDjTvfoz3bD+RLrMg/ptw+/+GHH0qSDh06pKSkJEmuatDQoUPVrFkzM0L4W84lJOnNAbP14pKn5OHlobhD8Xqj7yxVrnWrhs3qr0F1R1sd4g07m5CkKf1n68WPn5Knl4difz2pN/q4+jh8dn89ce/oa7aRpI8nrdao+YM057tJksOhha8s08Hdh7LOf3OlEJ38LcGq7v1j5xKS9Objc/Xi4qHy8HJ3jX2/2a6xf6evBtUbY3WIee7cqWRFPPuxXnintzw83RX3W6KmjFiiylXL66mJD2tI2JvXPX7thzt0k6+3pq8ZJjd3N/2yL0bvvbbapOhv3LnE83pr0DyNWTjI9Xk/HK83Hn9PlWtW1NPTwzW40bjrHt//tYclh0NPTw/P2vfj1z9r5oiCN+1/tXOnkhXxzGK98G64PDw9FHc0UVOeXqTK1SroqUndNKTdG9c9fu3CL3WTn7emrx8hN7ffx378KpOiv3HnEs/rraHzNWbe466xP5KgNwbNU+Uat+jpiJ4a3Hz8dY+fM3aZnnzzMb273fUe2bnuO62a/ZkZoee7grB2xywOZ2Gbu5DUpvijVodgGaeNHnKVE4e7u9UhWMotJNDqECzjPHXG6hAs5fDz/etGRZgzJcXqECwVmTjH1Os1bzPJtGt9vnGUadfKCQ9UBAAARjb6m7vgrVQGAAAwCRUhAABg4Ch8q2b+MSpCAADAtkiEAACAbTE1BgAAjIrOFyn8JSpCAADAtqgIAQAAAxZLAwAA2AAVIQAAYGSfghAVIQAAYF9UhAAAgBFrhAAAAIo+KkIAAMDAYZ+CEBUhAABgX1SEAACAEWuEAAAAij4qQgAAwMDBd40BAAAUfVSEAACAEWuEAAAAij4qQgAAwMg+BSEqQgAAwL5IhAAAgG0xNQYAAAwcLJYGAAAo+qgIAQAAIypCAAAARR8VIQAAYMRXbAAAABR9VIQAAIABd40BAADYABUhAABgREUIAACg6CucFSGHffM3tzsqWh2CpZwHDlkdgqUyTyRYHYJlHCWKWx2CtdwcVkdgKYe3t9Uh2AsVIQAAgKKvcFaEAABA/uE5QgAAAEUfFSEAAGDAc4QAAABsgEQIAAAUCmvWrFH79u3VunVrLVq0KNvPt2zZos6dO6tTp04aNGiQzp0795fnJBECAABGTqd5r1w6efKkIiIitHjxYq1atUqffPKJfvnll6yfJycn66WXXtKcOXP06aefqkqVKpo+ffpfnpdECAAAFHg7d+5UvXr15OfnJ29vb7Vp00aRkZFZP09LS9O4ceMUHBwsSapSpYri4uL+8rwslgYAAEYmLpZOSkpSUlJStv2lSpVSqVKlsrbj4+MVGBiYtR0UFKTo6Ois7dKlS6tVq1aSpEuXLmnOnDnq0aPHX16fRAgAAFhmwYIFmjFjRrb9Q4YM0ZNPPpm1nZmZKYfjzyesO51Ow/Yfzp8/r8GDBys0NFT333//X16fRAgAABiZWBHq1atXjgnLldUgSQoJCdGuXbuythMSEhQUFGRoEx8fr759+6pevXoaPXp0rq5PIgQAACxz9RTYtTRo0EDTp0/X6dOnVaJECW3atEnjx4/P+nlGRoYGDhyodu3aadCgQbm+PokQAAAwKoBfsREcHKxhw4apZ8+eSktLU9euXVWtWjX1799fQ4cO1YkTJ/Tjjz8qIyNDGzdulCTdfffdeu211657XhIhAABQKHTs2FEdO3Y07Js7d64kqWrVqtq/f//fPieJEAAAMOArNgAAAGyAihAAADCiIgQAAFD0URECAABGmVSEAAAAijwqQgAAwIg1QgAAAEUfiRAAALAtpsYAAIARU2MAAABFHxUhAABgREUIAACg6KMiBAAAjHigIgAAQNFHRQgAABg5M62OwDRUhAAAgG1REQIAAEbcNQYAAFD0URECAABGNrprjEToGuq0ra7wVx6SZzFPHd53TBED5yrl/KUc2z4zd4CO/BCj/05db3KU+adO438rfGgreXp56PDBE4p4aZVSLqQa2rQIq66uvRrJ6XQq9VKaZk1ap59/jLUo4rxTp20NhY9/WJ7FPHR47x9jfzHHts+897iO7DtWtMbehu/9Oq2qKnzsA673+48xihi64Jp9HjEzXEd+Oq7lMzZJkrx9SmjY9F6qUDlEDjeHtiyJ0rJpkWaGf0PubX6nwkeGufq+P1ZTR32slOTUHNuOeLO7juyP1fK527L2hT3WUG271ZNXcU/9sveYpo76WGmXM0yK/sbd2+JOhY/q8Gf/n11y7f6/1V1H9sdp+ZzPs/aF9fij/16u/o9cUqj6D6bGcuQb4KMRswdo/CPT1K/6SJ04HK8+4x/O1q5ClXKatOF5Nb6/jgVR5h/f0t4a/sr9Gj9iifp1fltxx88o/KlWhjblbwlQv2FtNGbQAg1++B0tmbtNL771iEUR5x3fAB+NmDNA47tNVb9qz7rG/tVrjH3k6KI39jZ87/uWuUnDZ/TW+F6z1K/ui4o7kqjwsQ9ka1fh3yGauGqEGneqbdjfa3RnJcae0cCGL2nofa+pQ5+muuPe28wK/4b4+pfU8De66dUnPlD/+17XiWOnFD6qQ7Z2FW4P0uuLB6lRu2qG/Q3aVFWn3o31/KOzNLDVJHkV91SXvs1Miv7G+fqX1PApj+jVx+epf/MJOnH0lMKf65itXYVKwXp9ySA1al/dsL9B22rq1LuJnu8+SwNbTnT1v18zk6LPZ06neS+LmZYIbd++XUlJSZKkVatW6ZVXXtHy5cvNuvzfUqtlVR3YfUixv56UJK2d85ladGuQrV2ngS0VOX+b/rfiG7NDzFe16lfSwX3HFXv0tCRp3dJv1OKq/wDS0tI19eVVOp2YLEk6+GOsSgfcJA8Pd9PjzUvZxn7uFrXo1jBbu04DWynygyI49jZ879dqfpcOfndEsYfiJUnr5m1Ti//UzdauY9/mivxwu7av3m3YP+v5jzX3xWWSJP9gX3l6eehCUs4VxIKmVuMqOhh9TLFHEiVJaz/aoeada2dr16FnI238+CttX7/HsP++B+/VirmfK/lcipxOp2a8sExbV+wyJfa8UKtJqA7uOfpn/z/coeZdrtP/dX/R/9FLtXV54ek/XEyZGnvttdf0008/KSIiQlOnTlV0dLRatmypzZs366efftKYMWPMCCPXAsv7KzHmVNZ2wvHTKunrLW+f4oZy+cxhCyVJte6ranqM+SkwxFcJJ89lbSecTFJJn+LyLlksa3rsZOxZnYw9m9Xm8Wfa6attB5SeXrhLwoHlyygx5nTWdkLMH2NfwjA9NnPYAkmuxKEoseN7P/Dm0ko4fiZrOyH2jEqWyt7nd0YtkSTVbn5XtnNkZmRq5Lt91ahTbe1c951ifj6R/4HngYBypZUQ9+fnODHunEqWKiHvm4oZpodmjVshSarVpIrh+PK3BupgGR+NXzBAZYJ8te//Dun919eYE3weCCjnd1X/z+bc/7GuP9prNQk1HF/+tkAd/P4mjV/4uMoE+2rfN7/q/QmFp//XVQAqNWYxpSK0c+dOLViwQIGBgfriiy/07rvvqnv37po5c6Z27NhhRgh/i5vDkeN7ICPDHm8Mx7X6n5n9AVvFSnjqhTceVtkK/pr68ioTostfbm4OOXPofEaGPR4uZsf3vsPNLU/GfPLA9/VQ5WHyKV1S3Udmn14piNwcjhx/4eV2vN093FWz8b/1+uAFGtrpLfn4eav3s+3zOsx84+p/9v1/r/9V9Pqg+Rra4U35+JVU75FheRwl8pspiVDx4sV16pTrr8yQkBClpKRIki5evCgPj4K3Xjv+2CmVKeuXtR1wc2mdP52s1JScF9AVNQknzqlMoE/WdkCQj86fS1HqxTRDu8AQX0UsGKCMTKdG9ZunC9dYXFqYuMa+dNZ2wM3+thp7O773E2JOqUzIFX0u66fzZy4oNeVyro6v3eIu+Yf4SpIuXUjVtuXfqFK1f+VLrHktPvaM/INLZW0HhPjq/NkLSr2Yu76fjj+nnZF7lZKcqvS0DG1duVuhtSrmU7R574b7f/KcdkZGX9H/XYWq/3AxJREaPHiwunbtqkmTJql8+fLq0aOHJkyYoIceekjh4eFmhPC37P5sn0LrVFK524MlSWH97lPU2m8tjso8u6N+UWi1Cir3L39JUth/6ihq235DmxLeXpr8fh/t+OxHTRy1VJdT060INc/t3rLXOPb971PU2t1/cVTRYcf3/u7Pf1ToPbep3G1BkqSw8KaK2vB9ro9v0uUePfZ7BcjTy0ONu9yjPdv3/8VRBcO32w8otEZFlasYIElq/2gDRW3el+vjv1wfrcZhNeRVzFOSVL/13Tq452i+xJofvv3fAYXWvKL/jzVU1Ka/0/89atzhiv63qVqo+n9dNlosbUo5pkWLFqpcubK2bNmi3377TTVq1FDJkiU1ceJEVatW7a9PYLJzCUl68/G5enHxUHl4uSvuULze6DdblWvdqmHv9NWgegVrTVNeO3f6gt4au0JjpjwiD093xcWc1hsvLFflO8vp6XFdNPjhd9SpWz0FlfVTgxZ3qEGLO7KOfW7ABzp/rnAsFM3JuYQkvTlgtl5c8pQ8vDxcY993lmvsZ/XXoLqjrQ4xX9nxvX8u8bzeGvKBxswf6Brzwwl644n3VbnGLXr67V4a3PSV6x4/Z8xSPfnWY3p3x0uSpJ3rvtOqdz8zIfIbd+5UsiKeXaIXZvWWh6eH4n5L1JThi1W5agU9NelhDWk/5brHr/3wS93k563pa4fLzc1Nv/wQo/deW21S9Dfu3KlkRTyzWC+8G+7q/9FETXl6kSpXq6CnJnXTkHZvXPf4tQt/7//6Ea7+74vRe+ML/xIBu3E4c5ocL+DalOhhdQiWcfy7otUhWMp54JDVIVjLYd8nXjhKFLc6BEs5St1kdQjWstED/nKy4ehUU6/Xruxg0661IW6madfKiX3/VwUAALZX8FYqAwAAaxW+yaJ/jIoQAACwLSpCAADAiIoQAABA0UdFCAAAGNnoLj0qQgAAwLaoCAEAAAOn0x7fryhREQIAADZGRQgAABixRggAAKDooyIEAACMeI4QAABA0UciBAAAbIupMQAAYJTJ7fMAAABFHhUhAABgxGJpAACAoo+KEAAAMHCyRggAAKDooyIEAACMWCMEAABQ9FERAgAARnzpKgAAQNFHRQgAABg5uWsMAACgyKMiBAAADJysEQIAACj6qAgBAAAj1ggBAAAUfSRCAADAtpgaAwAABiyWBgAAsAEqQgAAwMhGi6UdTqeNvmIWAADgCkyNAQAA2yIRAgAAtkUiBAAAbItECAAA2BaJEAAAsC0SIQAAYFskQgAAwLZIhAAAgG2RCAEAANsiEfob1qxZo/bt26t169ZatGiR1eGYLjl92+h1AAAFkElEQVQ5WR06dFBMTIzVoZhuxowZCgsLU1hYmCZPnmx1OKZ7++231b59e4WFhemDDz6wOhzLTJo0Sc8995zVYZiuR48eCgsLU+fOndW5c2ft2bPH6pBMtXXrVj3wwANq166dXn31VavDQR7ju8Zy6eTJk4qIiNCKFSvk5eWlbt26qW7duqpUqZLVoZliz549GjNmjI4cOWJ1KKbbuXOnvvzyS61cuVIOh0P9+vXT5s2b1apVK6tDM8U333yjr776Sp9++qnS09PVvn17NW3aVLfddpvVoZkqKipKK1euVLNmzawOxVROp1NHjhzR559/Lg8P+/3KOHbsmMaNG6dly5apTJky6tWrl7744gs1bdrU6tCQR6gI5dLOnTtVr149+fn5ydvbW23atFFkZKTVYZlm6dKlGjdunIKCgqwOxXSBgYF67rnn5OXlJU9PT91+++2KjY21OizT1KlTRwsXLpSHh4dOnTqljIwMeXt7Wx2Wqc6ePauIiAgNHDjQ6lBMd+jQIUlSnz591KlTJ3300UcWR2SuzZs3q3379goJCZGnp6ciIiJUvXp1q8NCHrJfev8PxcfHKzAwMGs7KChI0dHRFkZkrtdee83qECxTuXLlrH8fOXJEGzZs0JIlSyyMyHyenp6aNm2a5s2bp7Zt2yo4ONjqkEw1duxYDRs2THFxcVaHYrqkpCTVr19fL774otLS0tSzZ0/deuutatiwodWhmeK3336Tp6enBg4cqLi4ODVr1kxPP/201WEhD1ERyqXMzEw5HI6sbafTadhG0ffzzz+rT58+GjlypCpWrGh1OKYbOnSooqKiFBcXp6VLl1odjmmWLVumsmXLqn79+laHYomaNWtq8uTJ8vHxkb+/v7p27aovvvjC6rBMk5GRoaioKE2YMEGffPKJoqOjtXLlSqvDQh4iEcqlkJAQJSQkZG0nJCTYcprIrnbv3q3evXtrxIgRuv/++60Ox1S//vqrfvrpJ0lSiRIl1Lp1ax04cMDiqMyzfv167dixQ507d9a0adO0detWTZgwweqwTLNr1y5FRUVlbTudTlutFQoICFD9+vXl7++v4sWLq2XLlraaDbADEqFcatCggaKionT69GldvHhRmzZtUpMmTawOCyaIi4vT4MGDNWXKFIWFhVkdjuliYmI0ZswYXb58WZcvX9Znn32m2rVrWx2WaT744AOtXbtWq1ev1tChQ9WiRQuNHj3a6rBMc/78eU2ePFmpqalKTk7WypUrbXOjgCQ1b95cX375pZKSkpSRkaHt27frrrvusjos5CH7pPU3KDg4WMOGDVPPnj2Vlpamrl27qlq1alaHBRO8//77Sk1N1cSJE7P2devWTY888oiFUZmnadOmio6OVpcuXeTu7q7WrVvbMiG0q+bNm2vPnj3q0qWLMjMz1b17d9WsWdPqsExTvXp19evXT927d1daWpoaNmyoBx980OqwkIccTqfTaXUQAAAAVmBqDAAA2BaJEAAAsC0SIQAAYFskQgAAwLZIhAAAgG2RCAEAANsiEQIAALZFIgQgV9q1a6cmTZro559/tjoUAMgzJEIAcmXt2rWqWLGiNm7caHUoAJBnSIQA5Iq7u7tq165tqy9cBVD08V1jAHLl0qVLWr9+vfhWHgBFCRUhALkSERGhoKAgHT16VBcuXLA6HADIEyRCAP7Sd999pw0bNmj69Ony8fFhwTSAIoNECMB1paamavTo0Xr55Zfl5+en0NBQ7d+/3+qwACBPkAgBuK63335bNWrUUPPmzSVJoaGhLJgGUGSQCAG4pujoaEVGRmr06NFZ++644w4qQgCKDIeTW0AAAIBNURECAAC2RSIEAABsi0QIAADYFokQAACwLRIhAABgWyRCAADAtkiEAACAbZEIAQAA2/p/BXoYN16axtQAAAAASUVORK5CYII=\n",
+      "text/plain": [
+       "
    "
+      ]
+     },
+     "metadata": {
+      "filenames": {
+       "image/png": "/Users/hjensen/Teaching/FYS-STK4150/doc/src/LectureNotes/_build/jupyter_execute/chapter6_138_0.png"
+      }
+     },
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkIAAAJgCAYAAABmwww9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3gUVdvH8e9sSQNCTSCAVJUmKFVAAQUFEQTrY8eGihoREV4r2BBQkCIoHWz0B+mgKChFilQLUkVEeiIlkoRkd2fePzZPkpEgCmSXzf4+17UXzOzZ2XPvzG7uvc+ZWcOyLAsRERGRMOQIdgdEREREgkWJkIiIiIQtJUIiIiIStpQIiYiISNhSIiQiIiJhS4mQiIiIhC0lQiIXqD59+tCxY0c6duzIZZddRps2bbKXT548+a+29cMPP9C7d+/T3r9t2zaqVavG6NGjz7XbIiIhxdB1hEQufC1btmTo0KHUrl37rB7/2Wef8cUXXzBq1Kg873/11VdJTU1l7dq1LF68GJfLdS7dFREJGfq0EwlB06dPZ/LkyZimSbFixejVqxdVq1Zl3bp19O/fH9M0AXj88cepU6cO7733Hn/++Scvvvgi/fr1s23rxIkTzJ07l+nTp7N161a++OIL2rVrB4DX62XAgAF88803OJ1O6taty6uvvorD4chz/ahRozh69Gh29WnYsGHZy/fffz9FixZl165d3H333dSuXZsBAwaQmZlJUlISTZs2pW/fvgB8/fXXDBkyBNM0iYmJ4fXXX+frr79m586dvPvuuwCsW7eOPn36MGvWrEC97CJSACkREgkx3333HbNmzWLixIlER0ezYsUKEhMTWbhwIcOGDeOhhx6iXbt2bN26lalTp9KmTRu6du3KF198cUoSBDB79mwqVapE1apVufnmm/nwww+zE6FJkyaxefNmZs+eTUREBN27d2fBggWkpKTkuf5MYmNjs9t1796drl27cuWVV5KamkqrVq346aefKFOmDD179uTjjz+mZs2aLFq0iIEDB/L222/TunVrjh07RrFixZg2bRp33XXX+X1xRSTsKBESCTHffPMNv/32my0JSElJ4dixY7Rt25Y33niDJUuW0LRpU7p3737G7U2ZMoX//Oc/AHTo0IFBgwaxceNG6taty8qVK+nYsSNRUVEADBkyBIAuXbrkuX7YsGF/+1wNGjTI/n///v1ZtmwZI0eOZNeuXWRkZJCWlsaGDRu45JJLqFmzJgCtW7emdevWAFxzzTXMnj2bm2++mRUrVvDqq6/+o9dMROR0lAiJhBjTNOnYsSM9e/bMXj58+DBFixblrrvu4tprr+Xbb79l+fLlDB8+nM8///y021q3bh07duxg7NixTJgwAQC3282HH35I3bp1T5krlJycjGmap11vGAa5px16PB5bu5iYmOz/33fffVSrVo1mzZrRtm1bvv/+eyzLwul0YhhGdjvLsti2bRvVq1fn3nvv5bXXXsPlctG6dWsKFSr0L189ERE7nTUmEmKuvvpq5s+fz+HDhwGYPHkyDzzwAAB33XUXW7Zs4dZbb+XNN98kJSWFpKQknE4nXq/3lG1NnjyZjh07snTpUpYsWcKSJUsYOXIkX375Jfv376dJkybMmzePzMxMTNPktddeY/78+addX7x4cTZv3oxlWZw4cYKvv/46zxhSUlL48ccf6dGjB61bt+bgwYPs2bMH0zS5/PLL+eWXX9ixYwcAixcvzk766tWrh8PhYNy4cRoWE5HzQhUhkRBz9dVX8+ijj/Lwww9jGAaFCxdm+PDhGIZBjx496Nu3L0OGDMEwDBITEylfvjw+n4/333+fxMREhg8fDsCRI0dYtGgRM2bMsG2/SZMmXHHFFXzyySf06NGDffv2ceutt2JZFo0aNeL+++/HMIw816enp7N8+XJat25N6dKladSoEXmdmBobG8tjjz3GLbfcQkxMDKVLl6ZevXr89ttvNGnShIEDB/L888/j8/koXLgwgwcPzn7srbfeyoIFC6hevXr+vtAiEhZ0+ryIhAyv10tiYiIdOnTgxhtvDHZ3RKQA0NCYiISEnTt30qRJE4oXL84NN9wQ7O6ISAGhipCIiIiELVWEREREJGwpERIREZGwpURIREREwlZInj7ftuZLwe5C0Fi5LjQXjozkI8HuQlB5ql8U7C4EjeE1g92FoHLt3BfsLgRXdFSwexBUC3cPPnOj88g8eGnAnstRZnvAnivP5w/qs4uIiIgEUUhWhERERCT/mASuAhvsikywn19EREQkaFQREhERERufFbiKULATEVWEREREJGwpERIREZGwFeyKlIiIiFxgTMLn17dUERIREZGwpYqQiIiI2ATy9PlgU0VIREREwpYqQiIiImLjszRHSERERKTAU0VIREREbHTWmIiIiEgYUEVIREREbHyqCImIiIgUfKoIiYiIiI3mCImIiIiEAVWERERExEbXERIREREJA6oIiYiIiE34/NKYKkIiIiISxpQIiYiISNjS0JiIiIjY6IKKIiIiImFAFSERERGx8YVPQUgVIREREQlfqgiJiIiIjU6fFxEREQkDqgiJiIiIjQ8j2F0IGFWEREREJGypIiQiIiI2ps4aExERESn4VBHK0rB5NR56tjXuCBe/bj/IkFc+Iy01I8+2z/W9nd07DjJjwgoAHA6DJ1/pQO2GlQFYu2wbYwcsDFjfz4dGzavxULfWuCOc/Lr9IIN7zfyb+G9j9/ZDzPjQH3/hotE83asDVasncDLdw6KZ65kzaXUgu3/OGl53GQ+93BF3hJtff97LkGc/Je3EyTzbPvfeA+zeso8ZI74CIKZIFM8Ovp/yl5TBYRh8NW0104cvCmT3z9mVjavyyGPX4nY72bXrMO++PZ+0tExbm1bX1+I/dzXGsiAjw8P77y1i+7aDtjavvnkbfyT/yfChoRV/oyYX80iXa/3v/52HeLffvFPjb30Zd9zTBLA4edLDB0MWsX3rAWIKRfLci+25qGJJHIbBlwt/YOrEVcEJ5CyE+7Hf8NqaPPR/7fz7fut+hjw/hbQTp/nse/cedm/dz4wx32Svm7LhTZIPHs9enjFqCV/P3pDf3c53miMUZooWL0T3t26jT7dJPNpuMAd/P8JD3duc0u6iKnH0G/8IV7e+zLa+ZYe6lKtUiic6DuXJW96jdoPKXN3mslMef6EqWjyG7n1u5c1uk+jcfggH9h49bfz9xz9Ms+vtsT3+/I2cTMvksQ5D6XbPSBo0u5RGLaoFqvvnrGjJwnQf2ok+D4/m0ate4+BvyTz0ys2ntLvokjL0m9GNq2+qa1vf6YUOJB84xhMt3qTrDf1p90BzqjeoHKjun7OiRWPo8UJ7Xu81g4fuH8WB/cfo/Pi1tjblLyrBY0+04sWeU+jSeRwTP/6W1968zdbmP3c3pnadiwLZ9fOiaLEYerx8E2+8/F8evnsEB/Yf45EnWtralK9QgkefasVLz02my4NjmfTRCl5963YAHny0BclJKTx2/2gSO4+n/S31qVGrXDBC+dfC/tgvUYjuA+6izxMTeLRVPw7+/gcPPd/+lHYXVY2n36QnubptHdv6clXi+PNYGok3Dsy+FYQkKNwELBH65Zdf+OCDD+jduzevvfYaH3zwAT/++GOgnv5v1bvqYrb/tJf9v/0BwLwpa7i2/RWntGt/d2O+mLGO5V/Y++1wGETFROCOcOGOcOFyO/FkeAPS9/OhXtNL2P7TPvbv8cc/f8oaWra7/JR2N93dmM9nrGP5op9s6y+pWY7FczdhmhZej4+1y7bRrHXoJIL1rqnB9o272f9rEgDzPlrGtbc1OqVd+4da8MXEb1k+x/5BN/LlaYx5bQYAJeKL4o50kZaSnv8dP0/qN6zM9q0H2LfvKABzZ2+g1XW1bG08Hh+D3pnPkSOpAGzfdoDiJQrjcvk/Qi6/ogING1VhXgj+EajfqArbt+xn396s+Geup9Vfjl9Ppo9B/edz5I8TAGzfcoDiJf3xfzBkEaOG+yskJUoWxu12knqaauqFJtyP/XrNqrH9h9/ZvzsZgHmffsu1Heuf0q59p6v5Yspqli/43ra+Zv3KmKbFgGlP88HCntzTtTUOR8GopPgwAnYLtoAMjU2cOJFp06bRpk0bateuDUBSUhK9evWiQ4cOPPzww4HoxmmVKlOUpFylzeRDKRQqEkVMoUjb8NCIt+YCUK/pxbbHfzVrA83a1OaTr1/A6XKw4dsdrPlma2A6fx7EJdjjTzpN/B9kxV+/6SW2x2/74Xda3XQFmzf+hjvCxVXX18LnDZ3LcZUqW5yk/Uezl5P3H6NQbDQxhaNsQwQjXpoKQL0WNU7Zhukz6fn+g1zdvh4rF25i785D+d/x8yQ+PpbDh1Oyl5OSUihUOIqYmIjs4aFDB49zKNcx0uWp61j17Q68XpOSJQvz5NPX82LPKbTvUC/g/T9XcfGxJP3L+B/vej2rVmzHm3Wcmz6L53t3pPk1Nfh22Tb2Zn2puNCF+7Ffqmxxkg4cy15OPnA8K/5I2/DYiFc/A6Bec3ul2+l0sGnFdsa/PQ+X28Hr4x8j7cRJZo1fFpgA5LwISEXo448/ZsqUKTz55JPccccd3HHHHTz55JNMnjyZadOmBaILf8vhMMjrh3Z95j/7Y37vk604fjSVe5r35f5r+1OkaAy3Pnj1ee5l/jEMA8s69QX4p/GPHrAQy4L3/5vIq+/dy8aVO/F4fOe7m/nG4XCc0/7/nwFPfcidNXpSpFgh7nmu3XnqXf4zTvMN1szjtJGoKDe9Xr+FcuWK8+6A+TidDl7qfTMjhn+VXS0KNQ6HQR6H/+njf/NWypUvzqD+82z3vf3GbG5r9y5FYqO476Fm+dXd8yrcj32HYZDXzvf9wx/a+nzKaka89hkZ6Zmkppxk5thvaNqm9vnuZlCYlhGwW7AFJBFyuVx4vacOFZ08eRK32x2ILvytwweOUyK+SPZyqdKx/Hk8jYx0zz96fNPra7Hos/V4PT7STmTw1ewN1GlUJb+6e94lHThGyfjY7OVS8f8u/pjCkYwd9Dldbn6PFztPAMPIHmYLBYf3HqFEmaLZy6USivHn0VQy/jJZ9nTqXVODEqX9jz+ZlsE3M9dycQjNlTl86DglSxbOXi5VqggpKemcPGnf//HxsQx9vxOmz+K5bhNJPZHBpdUTSChbjC5PXcfIsY/QvkNdrmlZk+49bwx0GGft8MHjlCyVO/7YPOOPKx3LkJEP4jMteiR+SmpWxaBBoyrZjz+Z7uHrrzZz8aVlAhfAOQj7Y3//UUqUzvXZV6Yofx5LJSP9n8Xf8pYGVKqekL1sGAZeT+hUw8UvIIlQly5duPnmm3nllVcYOnQo7733Hq+88gp33HEHXbp0CUQX/taGb3dQvU4FylYsCcCNdzZi1ZIt//jxO3/eR/Mb/N8CnC4Hja+twdbvf8+XvuaH9St3Ur3ORZSt4I+/3b+Mv91/GtEp8ToAipUsxA23NeCb+d+f4VEXjg1Lt1C9fmXKVo4D4MYHmrHq83/e/+Yd63NvD/+3YHeEi+Yd6rNpxbZ86Wt+WL/2V2rULEe5csUBuKlDPVZ+u93WJjo6gneH3suKZdt4641ZZGb6v9hs2byPe+4YTpfO4+jSeRzz5mzkmyU/M2jAgoDHcbbWf7eLGrXKUa68P/72t9Rj1fK/xB8TwbvD7mfF0q30fXVmdvwAzVvW5L6HmgPgdjtp0bImmzbsDlj/z0W4H/sblm+j+hWVKFupFAA33tuUVV/+dIZH5ahUrQz3d2+Lw2EQEenmpgeuZtm8jfnVXcknhpXXmEg+OHToEKtWreLw4cOYpkmZMmVo0qQJpUuX/tfbalvzpfPev4bNL+XBbm1wuZ0c+P0IA1+cTkL5Ejzz5i0k3jrc1rb7W7fx285D2afPFykazZOvdKBqjbKYpsmm1b8wdsBCvPkwPGQZ+VNGbNjsUh56tjUulz/+AS/9l4TyJej2xi08dZs9/ufeuo3dO3JOn4+OiaBn/zsoW6EkhgFTxyxlybz8SYSM5CP5st2GrWrx4Ms3+/f/b8kMTPyQhIqleGbQfSS26mtr231oJ37buj/7FOJCsdE8PeAeKlYvC8DKBZv49J15eQ43nitP9fz5tt3oyqo88tg1/vj3HeXtvnNJKFuM7j3b0aXzOO6+twkPPtKCX3cl2R73f90nkZJrcmynB5sRWzQ6X06fN/Jx3lmjJlV5+PGWuN1O9u87yjtvziahXHG6v9COLg+O5a77m/Lgo9ewe9dh2+N6dp2IaZo80/NGKlXxJxPfLtvGx2OX5jncdi5cO/ed3w1mCZVjn+io879NoOE1NXjw+Xa43C5//N0nkVChJM+8fSeJNw60te0+8G5+23Yg+/T5yCg3T75xG9XrVsTpcrJ8wSY+yqcvAQt3D86X7Z7Ohj0VAvZc9SrsCdhz5SVgidD5lB+JUKjIr0QoVORXIhQq8isRCgX5mQiFgvxKhEJGPiVCoUKJUP7RBRVFRETExhdGlxkMn0hFRERE/kIVIREREbG5EE5rDxRVhERERCRsqSIkIiIiNhfCT18EiipCIiIiErZUERIREREbnxU+dZLwiVRERETkL1QREhERERszjOok4ROpiIiIyF+oIiQiIiI2OmtMREREJAyoIiQiIiI2OmtMREREJAwoERIREZGwpaExERERsTE1WVpERESk4FNFSERERGx8YVQnCZ9IRURERP5CFSERERGx0enzIiIiImFAFSERERGx0Y+uioiIiIQBVYRERETExmfpOkIiIiIiBZ4qQiIiImKj6wiJiIiIhAFVhERERMTG1HWERERERAo+VYRERETERnOERERERMKAEiEREREJWxoaExERERtdUFFEREQkDIRkRei1hZOD3YWg+cMsFOwuBFWqGRnsLgTVb5m7gt2FoKkZtS/YXQgqjxWSH9fnTYyREewuBNnggD6bfnRVREREJAyE91cMEREROYVPF1QUERERKfhUERIREREbE501JiIiIlLgqSIkIiIiNpojJCIiIhIGVBESERERG/3oqoiIiEgYUEVIREREbEz91piIiIhIwaeKkIiIiNhojpCIiIhIGFAiJCIiImFLQ2MiIiJiY+qCiiIiIiIFnypCIiIiYuPTj66KiIiIFHyqCImIiIiN5giJiIiIhAFVhERERMRGc4REREREwoAqQiIiImKjOUIiIiIiYUAVIREREbHxqSIkIiIiUvCpIiQiIiI2ps4aExERESn4VBESERERG80REhEREQkDqgiJiIiIjWlpjpCIiIjIBWXu3LnceOONtG7dmokTJ55y/+bNm7ntttvo0KEDjz/+OCkpKWfcphIhERERueAdOnSIwYMHM2nSJGbNmsXUqVPZuXOnrc1bb71F165dmTNnDpUrV2bcuHFn3K6GxkRERMTGF8A6SUpKSp6Vm9jYWGJjY7OXV65cSePGjSlWrBgAbdq04fPPPycxMTG7jWmapKamApCenk7RokXP+PxKhLJsWmMwfbwTj8fgosoWnbt7iS5kb7NoloOv5jiJiLAoW8GiU6KPwjn7iD8OwxvPuOkz0kORM7/2F5TNa0zmT/Dh9UDZygZ3PeskqpB9jPiHb00+/8SH4YCYIgZ3PuOkVFmD1D8t/jvMx75fLCKioFFrB807OoMUydnZ9p2XRR9m4vNYlK7s4JZuUUTF2OP/eaWXxZ9mYjggurDBzc9EUjLBweS30vnjgJXd7uhBk8q1ndz3anSgwzhre9alsfaTo/g8FiUqRdA8sRQRMfYPwl9Xp7Jh8jEwILKIk+ZPliQ2wU1mqsmy4ckc2+fBMuGSloW54tbQegOE8/G/5TsfCyd48HogobLBHd0iTon9p299LPrUkx377V3dlCzrwPRZzPrAw64fTQCqN3TSrrMLwwid+SU/rbGYM8HC64FyleGeZw2i/xL/999azP/EwnBAoSJw9zMGcVn7fuowi72/QEQUNG5tcE3H0In9QvHRRx8xfPjwU9YnJiby9NNPZy8fPnyYuLi47OX4+Hh++OEH22NeeOEFHn74Yfr27Ut0dDTTpk074/NraAxIOQZjBrp4ureXd8Z7iE+wmDrO/kH28yaD+dOcvPC2hz4jvVzeyGLCkJw8csWXDt56zs3RP0LvTXDimMWUQT4e6uXipXFuSiYYzJtg2tpkZlhMfMfHQ71d9PzATa0rDT4b4QNg1igfkVHwwmgX3Ya42LrOYvMaM6+nuiClHrf4bHAGd78cRbcxhShRxsGiCRm2Np4Mi+kDTnLPK1EkDo+h+pVO5o/0t7n75WgSh8eQODyGm7tGElXIoP2TkcEI5aykH/exdFgy1z0fz38+KE+R0i6++/iorY03w+Sbwclc90I8tw0pR8WG0awcewSAdZOOUqikk9vfK8fNAxPYsjCFQ1tPBiOUsxLOx/+JYxbTBmVy/ysR/N/YKEqWcbBwgsfWxpNhMXlAJp16RfDs+1HUuNLJ7JH+NhuW+EjaZ9F9RCTPfhDJrh99/LgiNGIH+POYxaeDLDr3Mug9zkHJBJgzwbK1ycyw+Ogdi0d7G7z4gYPLrjT47wh/mxmjLCKj4JXRBj2GGPy8zuLHNVZeTxVyTMsI2O2BBx5g8eLFp9weeOABe59M05ZkW5ZlWz558iQvv/wyH374IStWrOCee+7h+eefP2OsSoSAn9Y7qFLNokw5/3LL9j5WLXFg5Tqed+8wqFXXpERWMtrgKpONawy8Hjj6B6xf6aBnP8+pGw8B2zZYXHSpQVw5/wF1VTsH65eYWLleAMsECzjprziSmQ7uCP//9+6waNDKgcNp4HIb1Gxo8P3y0Pkw3LHBS7lLHZQq5387NGrn5vuvvbb4zaxwTqb612Wmg8tt347XYzHj3ZO0ezySYnGh89batymduIsjKVrWH1DNG4qwc9mJU/e/BZmp/hfCk27hdPuPlyadS3DlQyUASDvqw+eFiEKhE384H//bN/i46FIHcVnHfuP2TjZ+7TvNse//NzPdwhWRc1/mSX81xesBn/fU98WFbOsGqHgpxGft+2btDNYuIc99n54Vf0Y62fH/vgMatjKy932thgablheMRCiQYmNjKV++/Cm33MNiAGXKlCEpKSl7OSkpifj4+Ozl7du3ExkZSZ06dQC48847+e677874/BoaA/5IghJxOQdviThITzM4mUb28FjV6haLZjlJPuSjVGlYtsiB12NwIgWKl4RnXvUGqffn7miSRbG4nKy6aBycTIOMNIjKij8y2uCOp50M7e6lUBH/B2DXQf7Dp2I1g3WLTSrX8ieG339r4QydkQGOJ1kULZUTf2wpg4w0/wdeVIx/XWS0QYfESEY/l05MrIFpwmMD7UNf6xd5KVLSQc2mofW2OpHspVCpnB1WqJQLT5qFJ90iImt40B3t4OonSjLnhQNEFXFimRY39U8AwDAMDCd8PTiJX1emUqlxoeykKhSE8/F/PNmiaO7YSxl5xn5ropv3u2cQE+tPDJ5811/xbHCdkx+W+3jr/pP4fHBpPQc1G4dI8MDRJCiWM9JCsax9n/uzPzLa4K6nYVB3i5giFpYJ3Qf5X7NK1WDtYouqtcje947QCf9vmRdgnaRp06YMGzaMI0eOEB0dzaJFi3jzzTez769YsSIHDx5k165dVKlShcWLF1O7du0zbjcgn9j79+//2/vLli0biG6clmVBXkPajlzHQbXaFrfc52Po6y4MA5q3MSlUxMIZOp/3p3W6+I1cb+j9v1osmujjhVEuSpU1WDbLx4dveunxgYuOjzmZPcbHwKe8FCluUK2uwa9bQudb0T/Z/wd/9fH1pEy6joqhZIKDVbMzmfzWSZ4aHp1dml05M5OOXUNnSOx/LJM853QYueI/sjuTjVOPccewcsQmuPlpXgpfvX2YWweXzX7stc/GcXWXknz59mE2TjtG/buLByqEcxLOx79lkucvSuX+Y37gV5OvJnnpMSqSkmUdrJjt5ZM+mXR7P5IvJ3opXNSg16QoPJnw0RuZLJ3hocVtofHBeNr3fq749/1qsXCixcuj/POCvpllMfZNixc+gFseM5g5xqL/UxaxxQmpfR+KSpcuzbPPPkunTp3weDzcfvvt1KlTh0cffZSuXbtSu3Zt+vXrR7du3bAsi5IlS9K3b98zbjcgidDjjz/O7t27iY+Pt5Ucwf8BvHjx4kB047RKxsEvW3PeDUeToVARi8hcX/jT06B6HZMWbf114iPJMOMjJ4WLBLq351/xOIM9W3NK+ceTIaYwREblvCbb1ptUqmVQqqx/3dU3OZg12iQ1BTwZcFNnJ4WK+O/7coovu10oKBZnsHdbznGZkmwRXRgicsW/c4OPCjWdlEzwZwdXtnezYEwmaSlQqCjs/8WHaULl2qH3dbBwnIukHTlzolL/8BFZ2IE7KicT2rspndI1IolNyBo+a1uE1eOPkPGnSdIvmZSo6KZQCRfuaAcXNyvEr6vSAh7H2Qrn479YvMGebTmx53Xsb1/vo1JNByXL+o+Hpu2dzB3tIS0Fflrpo+MTblxuA5c7q0K0wkeL2wIeylkpHge7t+Ys57Xvt6yHKrUgLmufNr8JZoyG1BTIzICbOxvZ+/6LKRZxwf1ef974LtALKt50003cdNNNtnVjxozJ/n+LFi1o0aLFv9pmQGpfkydPpnLlyrzzzjssWbLEdgt2EgRQu77JL1sMDu7zLy+Z56ReE/sY/7E/oG9Pd/Y48ZxJTppca+b5bSLUVKtvsHurRdI+fzKwcr7JZU3sgZW/2OCXHyz+POpv8+Mqi5KloXBRg5XzTT7/2P96/XnUYvXnJvWvCZ0X5uJ6Tn7fapK8zx/D2gUeqje2f0dIqOpg948+Thz1t9myykfx0gaFivrj3P2jjyp1nCF1tsz/lL8imsPbMji+3z/HbcsXf1KxUYytTckqERz46SRpx/wThH9bk0aReBdRsU52rUhlw5RjWJaFz2Ox69s0ytaJCngcZyucj/9L6znZs9UkKevYX73AR60m9mS+3MUOdv1oZse+eZVJiaxjv9zFDn5Y5j8mfF6Ln1f7qFj9whtSOZ0a9f2J0OGsfb98vkXtJvY2F10MO3+AlKz4v19F9r5fMd9i/uERvHgAACAASURBVMf+9SlHLVZ+btEgRPa95DCsv5Zo8skPP/zA9OnTbeN5Z2vNb5XPQ4/svv/OYNp4J16PQXxZi8d7ejl80GD8ICd9Rvrn/3w523/6vGXBpbVMOiX6iPjLSEin1hG8Pz0z306f/8MsdOZGZ+Hn77JOH/ZCqQSDe3o6+eOAxdQhPnp+4K8CrJjjY/lcE5fLfwrtrU86SahkcDLNYuIAH8n7LSwLrrvTSYNW+fNhmGrmz9DTtrVevvwwE5/XokQZB7f1iOLoAZOZ72WQONyfFKyem8maeR6cLoPoIgbtn4igdEX/H42572dQuITBtXdH5Ev//ue3zFL5st0969JY++lRTC8UKePimmdK8echL8uGJ3PbEP9ZBJsXpPDzgj9xuCCysJOmj5WgRIUIMk74WDHyD47u8SdSla6Mof7dxTAc5/cPQs2ofed1e7mFwvHvsfKngL/lOx+ff+jB54USCQZ39YjgjwMW/x2aybPv+xPalXO9rJzrxemC6CIGNz/ppkxFB6kp/tPn9/9i4nDAxVc4aNfZXyE632KMjDM3Ogubv8s6fd4LpRKgU0+D5AMwaYjFix/49+PSORbL5lpZ+x7+86SRve8/HmCRtN8/zNb6ToNGrfInEbq+8pZ82e7pPLPx7oA919C6kwP2XHkJWCJ0PuVHIhQq8isRChX5lQiFivxKhEJBfiZCoSC/EqFQkV+JUKhQIpR/wvudJSIiIqcwrdAZ4jxX4ROpiIiIyF+oIiQiIiI2vjwvrFAwqSIkIiIiYUsVIREREbExL9DrCOUHVYREREQkbCkREhERkbCloTERERGx0enzIiIiImFAFSERERGxMXX6vIiIiEjBp4qQiIiI2Ph0+ryIiIhIwaeKkIiIiNjorDERERGRMKCKkIiIiNjoJzZEREREwoAqQiIiImKj6wiJiIiIhAFVhERERMRGc4REREREwoAqQiIiImKj6wiJiIiIhAElQiIiIhK2NDQmIiIiNposLSIiIhIGVBESERERG11QUURERCQMqCIkIiIiNpojJCIiIhIGVBESERERG1WERERERMKAKkIiIiJio4qQiIiISBgIyYpQnYhg9yCYUoPdgaBykBbsLgSVu9Cfwe5C0Hgsb7C7EFQmVrC7EFSHfCeD3YWwooqQiIiISBgIyYqQiIiI5B9dWVpEREQkDKgiJCIiIjaaIyQiIiISBpQIiYiISNjS0JiIiIjYaGhMREREJAyoIiQiIiI2qgiJiIiIhAFVhERERMRGFSERERGRMKCKkIiIiNhYqgiJiIiIFHyqCImIiIiNfnRVREREJAyoIiQiIiI2OmtMREREJAyoIiQiIiI2OmtMREREJAyoIiQiIiI2miMkIiIiEgaUCImIiEjY0tCYiIiI2GiytIiIiEgYUEVIREREbDRZWkRERCQMqCIkIiIiNpYV7B4EjipCIiIiErZUERIREREbE80REhERESnwVBESERERG11HSERERCQMqCKUZdkqg2FjnGR6DC6pYvHa/3kpXMjeZvJnDqbMdBIZYVGlosWL3XwUjYUevV3s2ZfTbv9Bg/qXWwzt6w1sEOfgXOL3+aD/UCfrv/fn1VdfafLsEz6MEPpCsWyVwdAxTjI9cGkVi9f/z3dK/JM+czB5poOoCKhc0eLlXPH3Heqwxf/cE2ZIxf/NKhg8GjI9UK0K9HmeU+L/dAZMnAlRkVClAvR6ForFwrEUeH0QbN0J0VFwa1u477bgxHG2wnn/h/t7f81qFxPGRuLxQOUqJs/2SKfQX+KfPdPNnFkRRERChQomT3VNJzbW3uaNV6MpUdIisevJwHU+H+k6QmHmyDF49W0XA9/wMvsTD+XLWgwd7bS1WbvRYMIkJ6Pf9TBtnJerG1u8OdCfRw58w8u0cf5b754+ihSGF7uFThJ0rvHPW+Rg9+8G08d7mDrOw7rvDb5cGjpvoiPHoNfbTga94WXuJ17Kl7UYMtr+1vhuo8H4SQ7GvOtl+jgvzRqbvD7Q/xrNW2Sw+3eDGeP9960Pwfhf7g9D34SFn0L5svDuKHubNRtg7GSYMAhmjoPmjeHVgf77+g+HmGiY9xFMGQHL1sDXKwMfx9kK5/0f7u/9Y8cM3h0QRa/X0hn3USplEkzGj42ytdm00cm0KZH0H5jGiNGpNLzSy9BB0bY206ZE8NOP9tdNQocSIWDVWge1qltULO9fvqODj4VfOWzXUfh5m8GV9U1Kx/uXWzUzWbrKwOPJaePxQO9+LnomeikTH7j+n6tzjd80If2kv5rgyQSvFyIjAh/H2Vq11uCyXPH/p4PJgjzib1zfyt6vrZpZ2fH7/hK/x2sQEULxf7sWLqsOlbLiv7sjzPvKfh2RzduhSX2y47++uT/ZyfT47+vYGpxOiHBDiyawaGng4zhb4bz/w/29v2Gdk2rVfJQrbwLQvkMmSxa7bfHv2OGkbj0vcXH+lVdf7WHNalf2Z//3m5ysW+ui3U2ev24+pFlW4G7BFrBE6KuvvuKTTz5hz549tvVTp04NVBdO69BhKBOXszdKx8GJVIPUtJw2tWtarN3oYP9B//LshQ48HoNjKTltZi5wEFfSomWzC2DP/gvnGn+HG0xiC0Pr291cd5ubi8pBi6ah8xocPGz8o/i/22jkGX/HGyxiC8N1t7toeZuLi8pZXBNS8UNCrsQ9r/jr1PBXhfZlxT9zIdnx16kBsxeBxwupafDlUkj6I7AxnItw3v/h/t5PSnJQKlf8cXEWaakGabnir17dx6ZNLg4d8le6vvjCjcdjkJJi8EeywYj3o3j+pXQcKiuErIDsuoEDB/Lpp5+ye/du7r77bmbPnp1935QpUwLRhb9lWuQ5pu3M9erUq2Px+AM+uvdycc9jLgwHFI21cOeaZfXpdCed7/flf4fPs3ONf9RHTooXs1gy08MX0z0cT4GPp4bOp4JpQV6XzMj9wVa/jkWXB3x06+XirsectvhHfuSgRDH4ZqaXL6d7SUmBj0IpfjPv9bnjb3A5PPkgPP0K3P6Y/3j5X/zPP+lfvrUzJL4CTRtge19c6MJ5/4f9e988c/y16/i47/4M3ugdQ+IThXAYUKSIicOAfm9F0+XJk5QsGTrJ3z9lWUbAbsEWkI+rpUuXMnPmTFwuF/fffz8PP/wwERERtG3bFusCqIslxMNPW3J2xuFkiC1iEZ1rGDg1DepfbnJLO/9fjUNJ8MF4J0WzJsxt3WHg80GDK4Ifz791rvEvXmbwwjM+3G5wu+GmNiZfLXXQ6c7T/IW9wCTEW/y4JeeT73/xx/wl/gaXW9zazj/361ASvD/eQdFY+GqZgxdzxd+hjcmXSx08cGegIzk7CaXhhy05y4eSoWge8Te8HG5vl9UmCd4b758sfeAw9Oji/z/AqE+gQvnA9f9chfP+D/f3fny8xdatOfEnJxsULmIRlSv+tDSofbmXG270D30lJxl8NCGSAwccHDjgYNQI/5yio0cMTBM8mVE826NgTJgOFwFJ3S3LwshKuytVqsSoUaN46623WLNmTfb6YGrS0OSHnw1+2+tf/u8cJ9dcZX8jJyVD525uTqT6l8d+6uSGljlnhqzbZNCobuicKZLbucZf41KLRV/7DyWPF5audFCnZmh8EAI0aWjZ4p8+x8G1V9kT2sPJ8HA3V3b8Yz510DZX/F987d/xHi98s9JBnZqhkxBf1RC+/xl2Z8U/dQ60vMre5nAyPNCN7PhHfgLtWvm/TU+dDcPG+9cnH4H/zof2rQLX/3MVzvs/3N/79Rt42fqzk317/THMnxtBk6b2uT5//OHg/7oXIjUr/smTIrmmpZeatXxMnHKCEaNTGTE6lXY3eWh+jVdJUAgyrACUZIYPH87KlSt54YUXqFOnDgDr168nMTGRzMxM1q9f/6+2l36g8nnv4/LV/lNIPR6D8mUt+rzkZe9+g9cHOJk2zv8tcMpnDqbOcmJaULe2yQvP+IiK9D++7xAncSUsHu0UOh8CuZ1L/MeO+0+h3brDgcNpcWU9i+5P+L8lnm+OfLrs+/LV/tOnPR64qKzFWy/52Lvf4LUBTqZnxT/5MwdTZjkwLahX2+LFXPH3Hepk6w4DpxOurOc/fTo/4ncb+VPEXbraf/q8xwMXlYP+L8He/dBrgP8sMYCJn8GkmWTFD726+U+lT02D59+C3/b5Jz4+di90aH3+++ix8u9MzFDY/yb581EdKu/9Q778STC+W+Ni/NhIvF5ISDDp+UI6Bw84GPxuNCNG+7Of2bPczJ0dgWVCrct8PNX1JJGR9u188lEkx48b+Xb6fKXyB/Jlu6dTe86rAXuuHzu8HrDnyktAEiGAVatWER8fT9WqVbPXHThwgPHjx/Pyyy//q23lRyIkoSG/EqFQkV+JUCjIz0QoFORXIhQq8isRChVKhPJPwD5VmzRpcsq6hISEf50EiYiISP7SBRVFREREwkD41tlFREQkTxfACd0Bo4qQiIiIhC1VhERERMTmQrjQYaCoIiQiIiJhSxUhERERsVFFSERERCQMqCIkIiIiNmF00pgqQiIiIhK+VBESERERG80REhEREQkDqgiJiIiIXRhNElJFSERERMKWEiEREREJWxoaExERERtNlhYREREJA6oIiYiIiI2lydIiIiIiBZ8qQiIiImKjOUIiIiIiYUAVIREREbFTRUhERESk4FNFSERERGx01piIiIhIGFBFSEREROxUERIREREp+FQREhERERtdR0hEREQkDKgiJCIiInaaIyQiIiJyYZk7dy433ngjrVu3ZuLEiafcv2vXLu6//346dOjAI488wvHjx8+4TSVCIiIicsE7dOgQgwcPZtKkScyaNYupU6eyc+fO7Psty+KJJ57g0UcfZc6cOdSoUYPRo0efcbsaGhMRERGbC3Gy9MqVK2ncuDHFihUDoE2bNnz++eckJiYCsHnzZmJiYmjevDkAXbp0ISUl5YzbVSIkIiIiQZOSkpJnwhIbG0tsbGz28uHDh4mLi8tejo+P54cffshe3rNnD6VKleKll15iy5YtVKlShV69ep3x+UMyEYo03MHugogEmNsIyY+r8ybdygh2F4IqwrjwKhQFWgAnS3/00UcMHz78lPWJiYk8/fTT2cumaWLkOg4sy7Ite71evvvuOz799FNq167NkCFD6N+/P/379//b5w/vTxYREREJqgceeIBbbrnllPW5q0EAZcqUYd26ddnLSUlJxMfHZy/HxcVRsWJFateuDUD79u3p2rXrGZ9fk6VFRETkL4yA3WJjYylfvvwpt78mQk2bNmXVqlUcOXKE9PR0Fi1alD0fCKBu3bocOXKErVu3ArBkyRJq1ap1xkhVERIREZELXunSpXn22Wfp1KkTHo+H22+/nTp16vDoo4/StWtXateuzfvvv88rr7xCeno6ZcqU4Z133jnjdg3LskLusknmwUuD3QURkYAK9zlCx01PsLsQVGXL7Q/o81X66O2APdfuB54P2HPlRUNjIiIiErY0NCYiIiJ2ITdWdPZUERIREZGwpYqQiIiI2F2AV5bOL6oIiYiISNhSRUhERERsQu988rOnipCIiIiELVWERERExE4VIREREZGCT4mQiIiIhC0NjYmIiIidTp8XERERKfhUERIREREbQ5OlRURERAo+VYRERETEThUhERERkYJPFSERERGx01ljIiIiIgWfKkIiIiJipzlCIiIiIgWfKkIiIiJip4qQiIiISMGnipCIiIjYqSIkIiIiUvCpIiQiIiJ2uo6QiIiISMGnREhERETClobGRERExMbQZGkRERGRgk8VIREREbELo4qQEqEs36yCwaMh0wPVqkCf56FwIXubT2fAxJkQFQlVKkCvZ6FYLBxLgdcHwdadEB0Ft7aF+24LThxnS/ErfsUfnvEvX+Vg2BgXHg9cUsWi9/95Tol9ymdOps50EhkBlStavNDNQ9FY6Nnbze/7cs4u2n/QoN7lJkP6egIcxdlbtdrJ2LGReDINqlTx0bPnSQr9Jf7PPnMza1YEEZEWFSuYPPPMSWJj/fd1vLkQcXE5WcOdd2Zy/XXeAEYg50pDY8CRY/Byfxj6Jiz8FMqXhXdH2dus2QBjJ8OEQTBzHDRvDK8O9N/XfzjERMO8j2DKCFi2Br5eGfg4zpbiV/yKPzzjP3oMXnvbzcA3PMz8JJNyZS2GjbZ/P1670cGHk1yMfDeTKeMyuaqxjz4D3QAMeMPDlHH+9b16eihcGF7oFjpJ0LFjBu+8E8Xrr6Xz8cepJJQ1GT0m0tZm40Ynk6dE8O67aYwdk8aVV3p5d1AUAHv2GMTGwtgxadk3JUGhJ2CJ0O7duzl06BAA06dPp0+fPixYsCBQT/+3vl0Ll1WHSuX9y3d3hHlfgZWrNLh5OzSpD2Xi/cvXN/d/2GV6/Pd1bA1OJ0S4oUUTWLQ08HGcLcWv+BV/eMa/aq2DWtVNKpT3B3tHBx8Lv3LaYt+yzeDK+ials2Jv1cxk2SoHnlz5jscDvfu56ZHoyX6NQsHadU6qVTMpnxV/xw4eFi922+Lfvt1B/fq+7KpPs2ZeVq3yV9A2b3bicFh0fSaaRzrH8NHHEfh8wYhEzsUZE6Hx48dTr149Lr/8cm688Ua6devGiBEjWLJkCXv37v1HT/Lhhx/yyCOPcNddd/Hiiy8yf/58KleuzIwZM3j//ffPOYhzdfAwJOR685aOgxOpBqlpOevq1PB/K9x30L88cyF4PAbHUvz3zV4EHi+kpsGXSyHpj8DGcC4Uv+JX/DnL4RT/ocMGpXMN68THWafEfllNk7UbHezPin32Qmd27P8za4GTuJIWLZuZAer5+ZF02EF8fE6f4+IsUlMN0nLFX6OGycaNTg4e9A8Bfv65G4/HICXFwOczqF/Px9v90xk6JI21a13MnOkOdBj5wrACdwu2M84RGjVqFO+88w516tTh999/Z/v27Wzbto1ly5axY8cOAC655BImT5582m3MmDGDBQsWkJycTPv27Vm9ejWRkZHccccd3H777Tz11FPnL6KzYJ7mvevIlSY2uByefBCefsW//ta2UDTWwu2C55+Ed0bArZ2hVAlo2gA2/hSQrp8Xij/v9Yo/5/+Kv2DGb1pg5HEBYWeu2OvVsXjsAS89ekVgGNDxRl927P8zcbqTV54LvSEh04K8rp+ce9/XqeOjU6dMeveOxnBA27YeYmMtXC6L9u3tw4B33JHJZ5+5uf320BkelH+QCBUuXJhrrrkGl8tFfHw89evXt92/d+/e7ITodEzTJCIignLlyvHwww8TGZkzBuu7AOqICaXhhy05y4eSoWgRi5jonHWpadDwcri9XVabJHhvvH+y5IHD0KOL//8Aoz6BCuUD1/9zpfgVv+LPWQ6n+MvEW/y0Jeev/uFkiC1iEf2X2OtdbnJzO/9n9eEkGDHeRdGseLfuMPD5oP4VoVUNAigdb7JlS86fwaQkgyJ/iT8tDa643Eu7Gz3ZbSZMiCQ2FhYtclG1qknVqlmxW+AqKKcg6Sc2cjz22GNMnz79tPeXL1+ea6+99m+30bp1a+677z58Ph9PP/00AFu3buWee+6hbdu2/7LL599VDeH7n2F31kjf1DnQ8ip7m8PJ8EA3OJHqXx75CbRr5f82NXU2DBvvX598BP47H9q3Clz/z5XiV/yKPzzjb9LQ5MefHezZ6/+jN2OOixZX2b+cJiUbPNYtIjv2cZ+6aNPSl11JWr/JQcO6Zp6VpQtdgwY+tmxxsjcr/rlz3VzV1F7ZSk426PZsDKlZ8X86MYKW13owDPh1t4MJH/rnBWVkwMxZbq69JvQqY+HOsCzrb0fo6tati8fjoXnz5jRr1owaNWpQrVo1onOnzP/A2rVradiwYfbyrl27+P3332nRosW/7rR58NJ//ZgzWbraf/qsxwMXlYP+L8He/dBrgP8sEYCJn8Gkmf5yar3a0Kub/1Ta1DR4/i34bZ9/guVj90KH1ue9i/lK8St+xX9hx59uZZz/jQIrVuecPl++rMWbL3nYt9/gjQFupozLBPynz0+b5Z9EfUVtk+ef8RKVVdjvN8RFXAmLzp3yt7p/3Myf4abVq52MGRuJ1wtly1q8+EI6Bw44GDAwirFj/JOFZs50M2u2G8s0uKy2l2e6ZhAZCSdPwtD3otiyxYHXa9CihYfOj2TmS1JYttz+87/Rv1FlyKCAPdeubt0D9lx5OWMitGfPHrZt25Z927p1K/v376d8+fJ88cUXgeqnTX4kQiIiF7L8SoRCRX4lQqFCiVD+OeNoZoUKFahQoQLXX3999rq0tDS2b9+erx0TERGRILkAzuYKlLO6jlBMTAxXXHHF+e6LiIiISEAVlPntIiIicp5cCNf3CRT9xIaIiIiELVWERERExE4VIREREZGCT4mQiIiIhC0NjYmIiIidhsZERERECj5VhERERMRGp8+LiIiIhAFVhERERMTOyodfjr1AqSIkIiIiYUsVIREREbHTHCERERGRgk8VIREREbHRWWMiIiIiYUAVIREREbFTRUhERESk4FNFSERERGw0R0hEREQkDKgiJCIiInaqCImIiIgUfEqEREREJGxpaExERETsNDQmIiIiUvCpIiQiIiI2On1eREREJAwoERIREZGwpURIREREwpbmCImIiIid5giJiIiIFHyqCImIiIiNzhoTERERCQMhWRGalVoo2F0Imt89JYPdhaCqHrk/2F0IqmKO9GB3IWgc4fQVNQ8eyxnsLgTVZ8caB7sLQTWwXICfMIzebqoIiYiISNgKyYqQiIiI5CNVhEREREQKPlWERERExCacpuSpIiQiIiJhS4mQiIiIhC0NjYmIiIidhsZERERECj5VhERERMRGk6VFREREwoAqQiIiImKnipCIiIhIwaeKkIiIiNipIiQiIiJS8KkiJCIiIjY6a0xEREQkDKgiJCIiInaqCImIiIgUfKoIiYiIiJ0qQiIiIiIFnypCIiIiYqOzxkRERETCgBIhERERCVsaGhMRERE7DY2JiIiIFHyqCImIiIiNJkuLiIiIhAFVhERERMROFSERERGRgk8VIREREbFTRUhERESk4FNFSERERGyMYHcggFQREhERkbClipCIiIjYhdEcISVCWbZ+5+OLDzPxeqBMZQe3dYsgKsZeHNy80stXn3owHBBd2ODWZyIomeDA9FnMGeHh1x99AFRr6KTtI24MI3SKi7vXpbP64+OYHouSldxc+3QJImLsBcNdq9L4bnIKhgOiCju45qkSFE2wH0IL+yVTqIST5o8XD2T3z9nmNRZzJ5h4PVC2Mtz9rIPoQvb99/23Fgs/MTEcEFME7nrGQVxZg9Q/LaYNs9j3i0VEFFzZ2qBFx9Aqtm5aYzB9vBOPx+Ciyhadu3uJLmRvs2iWg6/mOImIsChbwaJToo/CsTn3/3EY3njGTZ+RHooUDWz/z9XGNQ6mjXPi8UCFyhadn/MSk0f8i2Y7iYiAshUsHnzaS+FYyMyAD4e52LXNwLKganX/fRGRwYnl3/p+jcF/xzvxeqB8ZYuHu/tO2fdfzXKweI4Dd1bs9+Wx7/s84+KNkd6Q2/eHNhxny6QDmB6L2IpRXN6lAu4YZ/b9vy89wq55h7OXPWkmJ49kcv2IWhgugx/H7OX47nScUQ4qXFOCym3jghGGnIPQ+rTOJyeOW/x3cAb3vhzJc2OiKVHG4PMJHlsbT4bF1AGZ3PtKJF2HR1PjSidzR2YCsHGJj6S9Js98EEXX96PY9aPJTyt8wQjlrKQf9/H1e0e44YWS3DMigdgyLlZ9fMzWxpth8tXgI7R9sSR3DilDpYbRLB9z1NZm42cpHPg5I5BdPy/+PGYxcZDJw70cvDLOSckEg7kT7F+HMjMsPnnH5JHeDp7/wMllVxrMGGECMHOURWQUvDTaQfchDrass/hpTeh8nUo5BmMGuni6t5d3xnuIT7CYOs5pa/PzJoP505y88LaHPiO9XN7IYsKQnCR4xZcO3nrOzdE/Qif5/5//xf9Mby8DJ/wvfnuC//Mmg7lTXbz4joe+ozxc0chkXFb8syc5MX3Qd5SHfqM8ZGbAnMnOvJ7qgpNyDMYNdPJUby/9xnuJS7CYPs7+Z2HLJoMF0xz0fNvLGyO91Glk8tGQnPi+/dKg/3MujoXgvs9I8bLpg99p8FxlWg6tQUx8JFsm7be1uahFCVoMqE6LAdVp1q8aUcVcXPZweSKLudn84T6cUQ6uHVydZm9dwuFNKRxafzxI0ZxfhhW4W7AFJRHq379/MJ72tHZs8FH+UgelyvlfjsbtXGz62otl5ewh0/83j4zUrH/TweU2su/LPAlej//m81q4IgIawjn5feNJ4i6OoFhZNwC1bijMjqVptvgtE7AgI9W/znPSxBWR88G378eT7Nlwklo3FA5o38+HrRssKlwK8eX88VzdzmDdEuuU+C0gPdf+d2ft4993WDRsZeBwGrjcBjUbGmxafgG8u/+hn9Y7qFLNokw5/3LL9j5WLXGQK3x27zCoVdekRNaX3QZXmWxcY+D1wNE/YP1KBz37eU7deAj4cb2DypealCnvD7jVTT5WLrbH/+sOB5fVNSn5v/ivNtm42oHXA9Vrm3S814fDAQ4nVLrYIvlwaCQFm9cbVLbte5PVeez7mnWt7H1f/yqLTbn2/YaVDp7r5w1858+DpO9TKFY1hsIJ/vJdpdYl2bf8qO29n9vO2YeIKOqi0vWlADj+azrlmxfHcBg4XA7i68ayf/WxPB8rF658Hxp78cUXT1m3ZMkSjh/3Z839+vXL7y6c0fEki6KlcnLC2FIGGWn+P3ZRMf51kdEGNydGMOK5k8TEGlimxeMDowCof52TH1d46dcpHdMHl9R1UuPK0Bl1PJHso3CpnG94hUs5yUyz8KRbRGQND7qjHTT///buOyyKaw0D+LsVBCkizRobig0VYo1i14i9JBqNBVtQCYr92mMvKNhjL1Fjib2jITsLlgAAIABJREFUscRCiiXijdhBRTooiCACO/ePzV2cgBEVdmD3/T0Pz+PMntn9jmd299vvnNkdVgz7JkTD1EIOQQN0W2APAHgZn4mL65+jw3Q7/HUyWZI+fIznsUAxu6w3Lms74FWK9u//UwQmRWTo+a0M/qM1MLfQJkajlmjPmU+qyPDHzwIqVNcmwjcuCVAUjoIAACA+FrCxy3rht7EDUlNkov5XdBYQeECBuOhM2DoAvwTKkZEuQ3ISUKw4MHJ64XwjBID4WJkuwQGy+p+aAt30WEVnDQL3qxAXDW3/T2r7/yIJqPlp1v9dXDRwYp8Cg3wLx/9HQqxMNPbF3jL2pw/IdX2/8I+x/3Z64al+/1NqfDqKFFfptk2Lq5GRqkFGqkY0PQZoq0cPj8SiyfzKun3WlcwQ/ssz2FQpCk2GBpG/JUKuLBxJ8DsV0M9yhw8fxurVq5GRkYH+/fujT58+ObY7d+4cZs6ciTNnzrzzPvP93dra2hoHDhyAl5cXLC21k8q//vor6tWrl98PnWuCAOS0nEf+Rr0sKlSDn3ekw3eNKYqXkOPSwXRsn5MGnxWm+HlHOswtZZi8vQgyXgM/zErDhX3paNJNlf1OC6C39V/2Rv/jw17jyq4kfLWiBKxKKBF8+AVOzI9Hj8UOOLU4Hp8Nsoa5TSF693+DICDHa0Xlb3QnIlTAie0CJq3Rrgs6f0CDDbM0mLBKji5DZTiwTsDCERpYFAOq1JEhNKSAvorkIDfnf5WaArp+nYml3ykhkwHubTUwtxCgKByn+L8SNO/uv3NNAV37ZsB/hgpyGeD+eSaKWghQvtH/0LsyBMxQoXXnTNRpoMn/wPNAbsa+ck0Bnb/OxPLvlJDLBDRpK8D8H30vtDTI8bkvy2Gu5PHpeDh8aglzh6zFX9X7lcStHyLwy4Q7MLFWws7FAs/uvMy/eI1cdHQ0/P39sW/fPqjVavTq1Qv169dHpUqVRO3i4uKwYMGCXN9vvidCEyZMgLu7OwICAjB69GjUr18fW7ZsQdeuXfP7oXPN2k6GJ3eyXriS4gQUKQqoTbOeIXevZeKTanIUL6F9hjTsoMTRdelISQL+upyJjl5qKFUyKFWAa0slbl7MKDSJUFE7BaLvZq3teRmfCZOicqhMs14NHl9/hRJV1brF0TU8iuLSxueIvvMaiVEZuLRRWw5OeZYJQQNkvhbQ/Fsb/XbkAxWzA8JuZyUuiXGAWVHA5I3xD7kqoEJ1GexKavc16SjDvrUCXiZpF8t2HiyDuYX2/ytwp0bXrjAobgc8uJ0V77M4wNxCgEmRrDapKYCziwZN22mfJwlxwN4tChS10He0ea+4vZBj/01z6H+zN/u/Gbr+B52VY/NyJfp7Z6BRi8KRBAHaSuCD21nP87eNfRUXAe7tMnRt9m2Rw9wAxr6IrQrP7mclLq8S0qEyV0Bpmv1DXUTQM1T3LC3al56qQdWvS0JdVPu6eG9fNMwcC8kq+QIkKSkJSUlJ2fZbWlrqCigAcPnyZTRo0ADW1tYAgLZt2+LEiRPw9vYWHTdlyhR4e3tj8eLFuXp8vawRatiwIdasWYMdO3ZgwYIFyMwsWKVUJ1cFntzORNxT7QvYb8cyUK2B+IlQqqIcoTc1ePFM+4Z5KygTxRxkMLeSoWRFOW5e0PYpM0NAyG+ZKOtceKojZWqbIvrOazyP0K7x+O+JZJSvZypqY1dRjYi/0pDyXNvP0N9SYWGvRMnqJui/sSR6BjiiZ4Ajqn9eFJUamxWaJAgAnN1keHQbiHmqHduLRwXUbChOZMpUkuF+sICkv8c/OAgo7gAUtZLh0lEBx7Zq9yc9ExB0QoBbs8KTCNV00+BBiAxRT7XbZ44o4NpQ/Gb+PB6YO06lWyN1aIcCDZtrcqwmFDY13TS4HyJHVLi2Mz/n0P9n8TLMGatGyt/9P7hDqev/tSA5flilxIR56YUqCQKAGm4CHr4x9mePyFGnobia+TweWDBOqRv7wzvkqG8gY29XywLP7qUgOVL7QfDRqTg41s1+2dvr5Ay8jHoNm8riy+keBcbhzq4oAEDa83Q8PhOP0o0L1xWzbyXo72/Lli1o2bJltr8tW7aIQoqJiYGdXdY8tr29PaKjo0Vttm7dimrVqqFWrVq57qreFrJYW1tj6dKl2LNnD+7cuaOvh82VotYydPc1wfa5acjMAGwcZfhyrAnC72Zi37LX8FlRBBVrK+DeXYl1E19BoQTMLGToN02b+bcfqsah1a+xZGgqZHJo2/YoPGuEzKwVaOFjg5ML4pGZIcDKUYmWo2wQc+81zq5MQM8AR5R2MUXtLhY4MDkGCqUMJkXl8JhsK3XoecLCWobeo+XYOFuDzAzAtgTw9Tg5Ht8V8GOABhNWKVC5tgwtesiwfLzm7/EHhkzXfo5o3VOGHxZpMO+bTAgC4NFXhk+qFJ53CctiwJCxGVg+S4mMdBnsSwr4ZlwGHt6VYeMSBWZ/n4ESZYAOPTMxw0cFQQAqV9egn3fB+kDzoayKAUPHZmDZLCUy0gH7kgK8xmfg4R0Z1i9RYu6adJQsI6BjzwzM+FYFjQBUqSGgv7e2QrJjrQKCAKxfkvWcr1xdwACfgr9OyLIYMHBsJla90ffB4zIReleGTUsUmPn32Hv01GCWjxKCADhV114+bwhMrFSoPawsri4JhSZDgJmDCep4l8XzBym48f1jNF3kDABIiXoNE2tltvU/Tl0dcG35I5wbcxuCAFT50hHWlcyk6Eqh1r9//xxnid6sBgGARqMRfS2NIAii7bt37yIwMBCbN29GVFRUrh9fJrxteXwBtu9BHalDkMyT9OJShyApZ5OIdzcyYNbyVKlDkIy8IFxnK6F0wbi/7WTf87pShyApv1q79Pp4tb/119tj/bncN1ft9u/fjytXrmDOnDkAgJUrV0IQBN3U2LJly3D06FGYmpoiPT0djx8/houLC3bs2PGv92vczywiIiIqFBo1aoSgoCAkJCQgNTUVgYGBcHd3193u4+ODkydP4uDBg1i7di3s7e3fmQQBTISIiIjon/S4Rii3HBwc4Ovri379+qFLly7o0KEDXFxcMGTIENy8efODu1p4FrIQERGRUevYsSM6duwo2rdu3bps7UqXLp2r7xACmAgRERHRPxjTkjxOjREREZHRYkWIiIiIxFgRIiIiIjJ8rAgRERGRCNcIERERERkBVoSIiIhIjBUhIiIiIsPHihARERGJsSJEREREZPiYCBEREZHR4tQYERERifDyeSIiIiIjwIoQERERibEiRERERGT4WBEiIiIiEZlgPCUhVoSIiIjIaLEiRERERGLGUxBiRYiIiIiMFytCREREJMLvESIiIiIyAqwIERERkRgrQkRERESGjxUhIiIiEuEaISIiIiIjwIoQERERibEiRERERGT4mAgRERGR0eLUGBEREYlwsTQRERGRESiUFaHvq1aVOgTJyM3NpA5BUkcSi0kdgqSUJUtIHYKkhJRUqUOQjMxELXUI0tJopI5AWtF6fjxWhIiIChZjToKIKP8UyooQERER5R+uESIiIiIyAqwIERERkZhgPCUhVoSIiIjIaLEiRERERCJcI0RERERkBFgRIiIiIjFWhIiIiIgMHytCREREJCIzoi/yZkWIiIiIjBYrQkRERCTGNUJEREREho+JEBERERktTo0RERGRCL9QkYiIiMgIsCJEREREYvzRVSIiIiLDx4oQERERiXCNEBEREZERYEWIiIiIxFgRIiIiIjJ8rAgRERGRCNcIERERERkBVoSIiIhIjN8jRERERGT4WBEiIiIiEa4RIiIiIjICrAgRERGRGCtCRERERIaPiRAREREZLU6NERERkQgXSxMREREZAVaEiIiISExjPCUho06E6rWrjYGze0FlokTozSdYMnQtUl6k5qqNRTFzfLtiICrW+gSvXqYhcMt5HFwVCACo7FYBwxb3ham5CeQKOXb7HcbPOy5J0cX3Uq9NTXhO6w6VWonQv8Lh77MZKS9e5dh2zKqBCLsVjr0rAnX7zC2LwO/YBCzx3oR7fz7SV9gfrJ6HKwbN7Q2ViQqhwY+wePDq7OP/ljZTd49BqUqOunaO5e0RfP4WpnVZgLJVS8N3zTcoUtQUgiBgw3+240rgDX13773UbVENnv/pqB37kAgEjP0RKclvGXv/Pgi7HYm9a85ku23KukGIj07E6ik/5XfIeape65rwnNZN2/9b4fD32fL2c3+lJ8JCnurOfbWpCiMW9UYV1/KQyWS4ffUhVo7bgdev0vXZhQ9Wt2V1eE7q/PfYP0XA6O1vH/ulfREWEoG93/+c7bYpG4YgPioRqyfvzu+Q81TdVjXgObkzVGoVQm+FI8B329v7v6y/duxXnwYAmFmYwte/L0o7OUIuk+H07l+x543XRCocjHZqzMrWAmPXfYOZPQMwqMZYRIZGY9CcXrlu4+XXF6+SX2GIyziMbDwNddvWRn2POgCAabtGYevMvRhWdxImd1yIbxZ+jZJvvGkWRFbFi2L0Ck/M6rcKg+tNQeSjWHhO756tXZnKJTD/4Bg06eQm2l+3dU0sPT0ZpSo56Cvkj2Jla4mxG4djZg8/DKw6Uju28/vkus2sLxfDy3UcvFzHYcnQ75H8/CWWe68HAPisHIwTm87Ay3Uc/AatwpRdoyFXFNynmpVNUYxe0gezh27EkKZzEPU4Hp7/6ZitXZlKDpi3yxuN29fO8X56DGuJGvUq5ne4eU577g/ArP6rMbj+VESGxcFzWrds7cpUdsT8A9nP/a9Gt4dCocCwxt9hWOMZMDFVo6dvO32F/1GsihfF6IC+mD14HYY0mYmoR3HwnNw5W7syTg6Yt8cHjTvUyfF+egxvhRr1C+nYL+2H2QPXYshnM7T9n9IlW7syTo6Yt3cUGncU97/fxE6Ii3yOYU1nwefz+Wjf3x3On5bXV/j5S9Djn8T08uocHBys+3dQUBDmz58PPz8/3Lgh3adkt9YuuHPlISLuRwEAjqw5jRZffZbrNk6u5XF6+0VoNAIy0jPx2/HraNKtHlQmKmybvQ/Xz/wXABD3NAHP417ArpSNHnv3/lxbVMfd62GIeBgDADi64RxafFE/W7uOg5vjxA8XcOHgFdH+zkNbYuE36/EsOlEv8X4stzYuuPvHAzz9e2wPrw5Ey95N3ruNUqXE+M3eWO27GbHh8QAAuUIOi2LmAAAziyJ4/ep1fnfno7g2dcbdG48RERoLADiy9SKad/00W7sOA5rg5I9BuHDkz2y31WxYCW7NquLoDxfzPd685tr8H+f+xrec+4P+f+5fFe2/GXQXPy4+CkEQoNEIuB/8GPali+sl9o/l2rQq7v75KGvst1xA8251s7XrMKApTu64jAuHr2e7rWZDJ7g1r4ajWwvh2Derqh17Xf9/QfPu9bK16+DZFCe3X8KFQ9dE+7+fvBvrZuwFANjYW0FlokRKUmq246lg00siNH36dADA9u3bMXfuXDg6OsLW1hbTpk3Dtm3b9BFCNnalbXRvXAAQG54AcyszmFkUyVWb27/fR6s+jaFQKmBqboImXevBxrEY0tPScWLzOd0xHoNawKyoKUJ+u6eXfn0ou1I2iH2aoNuOjXgGc0szmFmYitqtGr8D5376PdvxU74IwN3rYfkdZp6xK2OL2PA43XZseHz28c9Fm88HtUB8RAIuHcj6P1nuvR69JnbFjsffY8GpaVg2fB00mZp87tGHsy1pjdiIZ7rtuMjnMLcsArOi4rFfPeUnnDtw9Z+Hw8bBEl7fdcfCb7dCUwjXFdiVKobYp1n9f+u5P+FHnNub/dy/dvYWnj6IBgDYl7ZBV69W2ZKlgirXYz95N87tv/LPw2HjYAWvWT2wcMTmAn2Ov41tyWLi/ke8pf+TduHcvj9yvA9NpgbjVg7A9+enIvjyXYTfj87XmPVFJujvT2p6rdfv3r0bW7duxYABAzBgwABs375dskRIJpfn+OO6bz6Z/63NmvHbIQjA6j/mYsZPY3Dt55vISM8Qtes5riP6TuuOad38Cvx6AZlclmNfMwvhi1tuyN/S3zfHPzdtuo9qj+1z9uq2VSYqTNnpi0WeK9G7rBfGNJ2Gkd9/A7sCXCGQy2Q5lqdzM/YKpRwTVw7A2hn78CwmKR+iy3/a53n2/4D3Pfcr1SoLv2PjcWj9WfweGPzuAwoAuVz+cWO/2hNrp+8ttGP/1v5r3m/sF43YjJ5Vx8HC2hy9x7TPo+hIX/SyWDojIwMajQbW1tZQq9W6/Wq1WnsiSiD2SRyc31jPYFvKBkkJyXiVkparNnZlimP9f3bgxbOXAIBeEzrrptBUaiXGbvBC2aqlMMp9OqIfZVUVCqrY8AQ4u2XNbduWtMaLZy+RllKwp3U+VMzjODjXc9Jt5zT+72pTsXY5KJQKBJ+/pWtTvkYZmJiZ4Lej2hJ6yG/38OivJ3Cu7ySqLhYkMRHPUKVOOd22raMVXjx/ibTUd4+9k0tZOJYtjiHTuwIAitlZQqGQQ22iwtJxP+ZXyHkqNjxefO6XeP9zv2m3uvBe1Acrx+/IsWpUUMU8TRCP/f/7npuxr/UJHMvaYsgM7VrCYvaWUChkUJsosXTsjvwKOU/FhCegims53fb7jr1rs6oIC4lAQnQiXqWk4dz+P966jqrQyelToIHSSxZibW2NZs2aITQ0FLNmzQKgXSvUq1cvfP755/oIIZurp26iaj0n3SLmDkNbIujw1Vy36TC0FfpN/wIAYG1viXaezXFm52UAwIQtI2BmUQS+7jMKRRIEAFfP/AXnTyuiZAV7AEB7z2YIOpZ9LYihuBp4A1UbOOmu/Org1QZBB/94rzYuTavhz7P/FR3z9H4UzK3MUK1hZQBAiQoOKFutNO5fD83P7nyUa+dvw9n1E5QsbwcA8OjbGEEnb+bq2NvXwtCv3nR4t10I77YLcWzbJZw/fK3QJEEAcPXsLTh/WuGNc78pgo7n/tyv39YFw+b1wqTu/oUqCQKAa+dC4OxWLmvs+zVG0MncVbNuXw1Fv0+nwLv1PHi3nodjWy/g/MFrhSYJAoBr50Pg7FY+q//9myDoRO7Xrrp3dkOfsdoKkEqthHsnN/x58U6+xEr5Ry8VoR9++AEA8PDhQyQlaUuoarUaPj4+aNasmT5CyOZ5bBL8hqzB1J0joVIrEfEgGosGroaTa3mMXjMEw+pOemsbANi54CAmbB6OtdcXADIZts7cg7tXH6JqfSe4d6+PJ3cj4H9+uu7x1k/aiaunCm65PDHuBZZ4b8KULcOgVCkRGRaDRV4b4VT7E4xa1h8j3GdKHWKeeh6bBL+BqzB1zxjd2C7svwKV3Spg9Lph8HId99Y2/1faqQSiwmJE9/syMQUzui3C8ABPqE3VyMzIRMA3axD5sOCuG0iMT4b/mB2YvGYglCoFIh/FwW/UNji5lMHIRV/Bu+1CqUPMV7pzf7MXlGolIkNjsWjYBu25v7Q/RjT993N/yMwvAJkMo5b21+279dt9rBxf8BOCxPhk+I/ahsnrBmv7HhYLP5+tcKpVFiP9+sC79TypQ8xXiXEv4D9yKyZvGJp17ntv1vZ/ydfwbjn3X49fN30vvl3UG6vPTwUAXD72Jw6uPauP0PNdQVi7oy8yIafJ8QKujbq31CFIRm5uJnUIkspMLBxXpeUXZckSUocgGSHFuK/GkZmo393IkL3nuh1Dczx6tV4fr3nbBXp7rLMnJ+jtsXJi1F+oSERERDkodCWSD1dwv+WNiIiIKJ+xIkREREQissK3auaDsSJERERERouJEBERERktTo0RERGRmBFdpMeKEBERERktVoSIiIhIhIuliYiIiIwAK0JEREQkZjwFIVaEiIiIyHixIkRERERiXCNEREREZPhYESIiIiIRmfEUhFgRIiIiIuPFihARERGJcY0QERERkeFjRYiIiIhEZPytMSIiIiLDx4oQERERiXGNEBEREZHhY0WIiIiIxIynIMSKEBERERkvJkJERERktDg1RkRERCIyLpYmIiIiMnysCBEREZEYK0JEREREho8VISIiIhLjT2wQERERGT5WhIiIiEiEV40RERERGQFWhIiIiEiMFSEiIiIiw1coK0JCRrrUIUhGqFRG6hAkJQ8x3rEHgIyISKlDkIyyeHGpQ5CWiVrqCKRVxFTqCIwLK0JEREREho+JEBEREYlp9Pj3Hg4fPgwPDw+0adMG27dvz3b76dOn0blzZ3Tq1AnDhw9HYmLiO++TiRAREREVeNHR0fD398eOHTtw4MAB7Nq1C/fv39fdnpycjBkzZmDt2rU4dOgQqlSpguXLl7/zfpkIERERkYhMEPT2l5SUhPDw8Gx/SUlJopguX76MBg0awNraGmZmZmjbti1OnDihuz09PR3Tp0+Hg4MDAKBKlSqIjHz3uspCuViaiIiIDMOWLVuwYsWKbPu9vb3x7bff6rZjYmJgZ2en27a3t0dwcLBuu1ixYmjdujUA4NWrV1i7di369u37zsdnIkRERESS6d+/P7p27Zptv6WlpWhbo9FAJpPptgVBEG3/34sXLzBixAg4OzvneL//xESIiIiIxPR4+bylpWW2pCcnjo6OuHLlim47NjYW9vb2ojYxMTEYNGgQGjRogEmTJuXq8blGiIiIiAq8Ro0aISgoCAkJCUhNTUVgYCDc3d11t2dmZsLLywvt2rXD5MmTc6wW5YQVISIiIhIrgF+o6ODgAF9fX/Tr1w/p6eno0aMHXFxcMGTIEPj4+CAqKgq3bt1CZmYmTp48CQCoUaMG5syZ86/3y0SIiIiICoWOHTuiY8eOon3r1q0DANSsWRO3b99+7/tkIkRERERiBbAilF+4RoiIiIiMFitCREREJPaeP31RmLEiREREREaLFSEiIiISkXGNEBEREZHhY0WIiIiIxFgRIiIiIjJ8rAgRERGRmIYVISIiIiKDx4oQERERiXGNEBEREZHhYyJERERERotTY0RERCTGqTEiIiIiw8eKEBEREYmxIkRERERk+FgRIiIiIjF+oSIRERGR4WNFiIiIiMQEjdQR6A0rQkRERGS0WBEiIiIiMV41RkRERGT4WBEiIiIiMSO6asyoE6F6Hq4YNLc3VCYqhAY/wuLBq5HyIjVXbabuHoNSlRx17RzL2yP4/C1M67IAZauWhu+ab1CkqCkEQcCG/2zHlcAb+u7ee6n3mRM8R7SESq1A6L1o+M8+hJSXr0VtWrSriS++bgQBQNqrdKzyO457IZG62+0cLBGwcRCG9f4eSYmpKEzqta0Fz+++gEqtROhfT+A/fANSXrzKse3YNUMQ9lc4flp2XLfP3MoMficnYcmw9bh3PUxPUX84nvtidVvXgOfkLlCZKBF66ykCRv6AlOScx3/Miv4IuxWBvatOAQDUpiqMWNALleuUg0wmw51roVg5YSdev0rXZxc+WN3m1eA5vr323L8dgYAJO5GSnJZj2zGLeyPsdgT2rjun27fz2izERSXqtveuOYOzB6/ld9h5pm7TKvD0bavt/50oBEzZi5SXb+n/vC8QdjcKezddAADI5TIMn9oJNT8tDwD445c7WL/oeI7HUsFltFNjVraWGLtxOGb28MPAqiMRGRqNQfP75LrNrC8Xw8t1HLxcx2HJ0O+R/PwllnuvBwD4rByME5vOwMt1HPwGrcKUXaMhVxTc/2orazOMmdYZsybsxuAeKxH19DkGercStSn9SXEM9mmNyT7bMbzPGuzY8AumLeypu72Vhwv81gyArb2lvsP/aFa2Fhjz/WDM6rMcg10nIio0FgNnfpmtXZkqJbDg6AQ06VJXtL9uGxcsPTsNpZ0csx1TEPHcF7MqXhSjl/bD7IFrMaThDESFxcFzatds7co4OWLevlFo3MFVtL+XbzvIFQoMbzobw5vOgtpUjZ4jP9dX+B/FysYcoxf1wuxhmzCk5TxEPYmH54QO2dqVqWiPeTuGo3E7F9H+UhXs8OJ5Crw9/HR/hSkJsipmjtFzemD2yO0Y4rEEUeEJ8ByTfezKVLDDvE2D0bhNDdH+Fp3qoFQ5OwzrvBTDuy5DzboV0LhtjWzHF0qCoL8/ientFerChQtISkoCABw4cAAzZ87E3r179fXw2bi1ccHdPx7g6f0oAMDh1YFo2bvJe7dRqpQYv9kbq303IzY8HgAgV8hhUcwcAGBmUQSvX4krKwWNa4OKuHPrKSKeJAAAjuz9Ay0+rylqk/46AwGzDyMhPhkAcDckAsWKF4VSKYeNbVE0bOaMST7b9B57XnBtUQN3rj5ExINoAMCR9WfQ4suG2dp1GtoKJzafxy/7fxft7zKsNRYOXoOEqOd6ifdj8dwXc21WDXf/fISIhzEAgCObf0HzHvWyteswqBlObruEC4fFb/T/DbqHnUuOQRAEaDQCHtx8AvsyNnqJ/WO5NqmCu8FPEBEWBwA4su0Smnd2y9auQ7/GOLnzV1w4Jq7uVXMrD41GwKLd32LV8XHo7dMGcrlML7HnBdfPnHD3v+GIeKQ9f4/8+Cuad6idrV2H3g1x8qc/cOHkTdF+uUIO0yJqqNRKqNRKKFUKpKdl6CV2yjt6mRqbM2cOQkJC4O/vj4CAAAQHB6NVq1Y4deoUQkJCMGXKFH2EIWJXxhax4XG67djweJhbmcHMoohuiiA3bT4f1ALxEQm4dCDrzXG593os+nk6uo3qAGt7K8z9yh+azIL7nQx2DpaIi07SbcfGJMG8qCnMzNW66bHoyERER2aVv7/xbYtff7mDjAwNEuKSMWv8br3HnVfsStsg7mmCbjv2acLf42wqmh5bOeYHAIBrS/EnvsldF+sn0DzCc1/MtlQxxD59ptuOi3gGc8siMCtqKpoeWz1xJwDAtXk10fHXzoXo/m1f2gZdvmmBZaO353PUecO2ZDHERmYl8HGRiX/33UQ0PbZ6+j4AgKt7FdHxCoUcf168i40LjkCpkuO7jUORkvwKBzb+op8lT27yAAAMJ0lEQVQOfCRbRyvEvvG6FhedBHMLU5iZm4imx1bPPgRAmzi96fT+q2jStiZ+OPcfKJRyXLt0D7+du62f4PNbAajU6IteEqHLly/j0KFDUCgUOH/+PHbt2gW1Wo2ePXuiQ4fsZVh9kMtlOY7zmy/auWnTfVR7+H+zRretMlFhyk5fLPJcid+OXkPV+k6YeWgi7vzxQPepuaCRy3LuZ2Zm9p0mpiqMnd4Zdg5WmFxIK0D/9LZxzizgb+Afiue+mFwuy/FFP1PzfuNfyaUspm7xwuEN5/D7qZvvPqAAkMve0vccnvs5ObHzV92/01KB/evPobNnk0KTCGmrVx8+9n1GtETis5fo3WQO1CZKTFvRF90GNMa+zRfzOFLKT3qZGjM1NUV8vPaF0NHRESkpKQCA1NRUKJXSrNeOeRyH4iWK6bZtS9kgKSEZr1LSct2mYu1yUCgVCD5/S9emfI0yMDEzwW9HteXzkN/u4dFfT+BcX/xJoiCJiU5Ecbuium1bO0u8SExF2j8We9o5WCJgw0BoNALGD9uCl29ZUFnYxDxJQHFHa922bclieJGQjLSUgj+t8yF47ovFhCfA5s3xL2GNF89evtf4N+3yKeb+NBKbZu3HroAT+RFmvoiJeAYbh6x1fbaOVnjx/CXSUnPX9xZdP0U55xK6bZlMhoz0wvMBIibyOWzs3ui/gyVePE9BWmruFro3al0dgXuvICM9EynJaTh94Bpc6lfMr3Apn+glERoxYgR69OiBBQsWoHTp0ujbty/mzp2LL7/8Ep6envoIIZurgTdQtYGT7uqXDl5tEHTwj/dq49K0Gv48+1/RMU/vR8HcygzVGlYGAJSo4ICy1Urj/vXQ/OzOR7n66wM41yiNkn+va2jf/VME/SIu7xYxU2PRmgG4ePY25k3ei9cGNA9+9cxNONeriJIVHQAA7Qe1QNDR6xJHlX947otdOxcCZ7fyKFnBHgDgMcAdQSdyf6Vb/TY14TX3S0z+YinO7fvj3QcUINcu3IFz7XIoWc4WAODRpxGCTv33HUdlKVfFEX1Ht4NcLoPaRIWO/RvjlyOF57lz7dI9ONcqg5KfFAcAePSsj6Azt95xVJb7tyLg3k67nlKhlKNBi6q4feNxvsSqd0a0WFov5ZgWLVrAyckJp0+fxqNHj1C7dm2Ym5tj/vz5cHFxefcd5IPnsUnwG7gKU/eMgUqtRMSDaCzsvwKV3Spg9Lph8HId99Y2/1faqQSiwmJE9/syMQUzui3C8ABPqE3VyMzIRMA3axD5MFrfXcy1xGcpWDzzIKbO/wJKlQKR4c+waMZ+OFUtAd8pnTC8zxp0+rIe7B2t8FlzZ3zW3Fl37IThW/GikF0q/0+JsS+w2Gs9pm7zhlKtROTDGCwauhZOdcrBd+VADG80TeoQ8xTPfbHEuBfwH7kVkzcMhVKtQGRYLPxGbIZTrbIYGdAX3s3n/Ovxg7/rDplMhpEBfXX7bv3+AKsm7Mzv0D9aYnwy/Mf9iMmrB0CpUiLyURz8Ru+AU80yGLmgJ7w9/P71+O0BJzF8ZnesPjkeCqUCF479KZouK+gSE17Cf/JeTA7oo33te5IAv4m74VS9FEbO6gbvbsv/9fi1849i+JROWHvUFxqNgD+DHuCnDYVjWpCyyAShAKRj76m1/AupQ5CM3M1ALs38UCEPpY5AUpq/p5WNkbJ4calDkJZ5EakjkFYRU6kjkNTxkHl6fbx2JUbo7bGOR67U22PlpGB/wQcRERFRPjLqb5YmIiKiHBS+yaIPxooQERERGS1WhIiIiEiMFSEiIiIiw8eKEBEREYlpWBEiIiIiMnisCBEREZGIIBSen0r5WKwIERERkdFiRYiIiIjEuEaIiIiIyPCxIkRERERi/B4hIiIiIsPHRIiIiIiMFqfGiIiISEzDy+eJiIiIDB4rQkRERCTGxdJEREREho8VISIiIhIRuEaIiIiIyPCxIkRERERiXCNEREREZPhYESIiIiIx/ugqERERkeFjRYiIiIjEBF41RkRERGTwWBEiIiIiEYFrhIiIiIgMHytCREREJMY1QkRERESGj4kQERERGS1OjREREZEIF0sTERERGQFWhIiIiEjMiBZLywTBiH5iloiIiOgNnBojIiIio8VEiIiIiIwWEyEiIiIyWkyEiIiIyGgxESIiIiKjxUSIiIiIjBYTISIiIjJaTISIiIjIaDERIiIiIqPFROg9HD58GB4eHmjTpg22b98udTh6l5ycjA4dOiA8PFzqUPRuxYoVaN++Pdq3b4+FCxdKHY7eLV26FB4eHmjfvj02bdokdTiSWbBgASZOnCh1GHrXt29ftG/fHp07d0bnzp1x48YNqUPSqzNnzqBbt25o164dZs+eLXU4lMf4W2O5FB0dDX9/f+zbtw9qtRq9evVC/fr1UalSJalD04sbN25gypQpCAsLkzoUvbt8+TIuXryI/fv3QyaTYfDgwTh16hRat24tdWh68fvvv+PXX3/FoUOHkJGRAQ8PDzRt2hQVKlSQOjS9CgoKwv79+9GsWTOpQ9ErQRAQFhaGs2fPQqk0vreMJ0+eYPr06dizZw+KFy+O/v374/z582jatKnUoVEeYUUoly5fvowGDRrA2toaZmZmaNu2LU6cOCF1WHqze/duTJ8+Hfb29lKHond2dnaYOHEi1Go1VCoVKlasiIiICKnD0pt69eph69atUCqViI+PR2ZmJszMzKQOS6+eP38Of39/eHl5SR2K3j18+BAAMHDgQHTq1Anbtm2TOCL9OnXqFDw8PODo6AiVSgV/f3/UqlVL6rAoDxlfev+BYmJiYGdnp9u2t7dHcHCwhBHp15w5c6QOQTJOTk66f4eFheH48eP48ccfJYxI/1QqFZYtW4aNGzfi888/h4ODg9Qh6dW0adPg6+uLyMhIqUPRu6SkJDRs2BBTp05Feno6+vXrh/Lly+Ozzz6TOjS9ePToEVQqFby8vBAZGYlmzZph1KhRUodFeYgVoVzSaDSQyWS6bUEQRNtk+O7du4eBAwdi/PjxKFeunNTh6J2Pjw+CgoIQGRmJ3bt3Sx2O3uzZswclSpRAw4YNpQ5FEnXq1MHChQthYWEBGxsb9OjRA+fPn5c6LL3JzMxEUFAQ5s6di127diE4OBj79++XOizKQ0yEcsnR0RGxsbG67djYWKOcJjJWV69exYABAzBmzBh07dpV6nD06sGDBwgJCQEAFClSBG3atMGdO3ckjkp/jh07hkuXLqFz585YtmwZzpw5g7lz50odlt5cuXIFQUFBum1BEIxqrZCtrS0aNmwIGxsbmJqaolWrVkY1G2AMmAjlUqNGjRAUFISEhASkpqYiMDAQ7u7uUodFehAZGYkRI0bAz88P7du3lzocvQsPD8eUKVPw+vVrvH79Gj///DPc3NykDktvNm3ahCNHjuDgwYPw8fFBixYtMGnSJKnD0psXL15g4cKFSEtLQ3JyMvbv3280FwoAQPPmzXHx4kUkJSUhMzMTFy5cQPXq1aUOi/KQ8aT1H8nBwQG+vr7o168f0tPT0aNHD7i4uEgdFunBhg0bkJaWhvnz5+v29erVC1999ZWEUelP06ZNERwcjC5dukChUKBNmzZGmRAaq+bNm+PGjRvo0qULNBoNevfujTp16kgdlt7UqlULgwcPRu/evZGeno7PPvsM3bt3lzosykMyQRAEqYMgIiIikgKnxoiIiMhoMREiIiIio8VEiIiIiIwWEyEiIiIyWkyEiIiIyGgxESIiIiKjxUSIiIiIjBYTISLKlXbt2sHd3R337t2TOhQiojzDRIiIcuXIkSMoV64cTp48KXUoRER5hokQEeWKQqGAm5ubUf3gKhEZPv7WGBHlyqtXr3Ds2DHwV3mIyJCwIkREueLv7w97e3s8fvwYL1++lDocIqI8wUSIiN7p+vXrOH78OJYvXw4LCwsumCYig8FEiIj+VVpaGiZNmoTvvvsO1tbWcHZ2xu3bt6UOi4goTzARIqJ/tXTpUtSuXRvNmzcHADg7O3PBNBEZDCZCRPRWwcHBOHHiBCZNmqTbV7VqVVaEiMhgyAReAkJERERGihUhIiIiMlpMhIiIiMhoMREiIiIio8VEiIiIiIwWEyEiIiIyWkyEiIiIyGgxESIiIiKjxUSIiIiIjNb/APJ4FZEd9b+SAAAAAElFTkSuQmCC\n",
+      "text/plain": [
+       "
    "
+      ]
+     },
+     "metadata": {
+      "filenames": {
+       "image/png": "/Users/hjensen/Teaching/FYS-STK4150/doc/src/LectureNotes/_build/jupyter_execute/chapter6_138_1.png"
+      }
+     },
+     "output_type": "display_data"
+    }
+   ],
    "source": [
     "# optional\n",
     "# visual representation of grid search\n",
@@ -3297,7 +3988,16 @@
    "metadata": {
     "collapsed": false
    },
-   "outputs": [],
+   "outputs": [
+    {
+     "ename": "SyntaxError",
+     "evalue": "invalid syntax (, line 1)",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[0;36m  File \u001b[0;32m\"\"\u001b[0;36m, line \u001b[0;32m1\u001b[0m\n\u001b[0;31m    pip3 install tensorflow\u001b[0m\n\u001b[0m         ^\u001b[0m\n\u001b[0;31mSyntaxError\u001b[0m\u001b[0;31m:\u001b[0m invalid syntax\n"
+     ]
+    }
+   ],
    "source": [
     "pip3 install tensorflow"
    ]
diff --git a/doc/src/LectureNotes/_build/jupyter_execute/chapter6_134_1.png b/doc/src/LectureNotes/_build/jupyter_execute/chapter6_134_1.png
new file mode 100644
index 000000000..eccdd272b
Binary files /dev/null and b/doc/src/LectureNotes/_build/jupyter_execute/chapter6_134_1.png differ
diff --git a/doc/src/LectureNotes/_build/jupyter_execute/chapter6_134_2.png b/doc/src/LectureNotes/_build/jupyter_execute/chapter6_134_2.png
new file mode 100644
index 000000000..e3bbd6804
Binary files /dev/null and b/doc/src/LectureNotes/_build/jupyter_execute/chapter6_134_2.png differ
diff --git a/doc/src/LectureNotes/_build/jupyter_execute/chapter6_138_0.png b/doc/src/LectureNotes/_build/jupyter_execute/chapter6_138_0.png
new file mode 100644
index 000000000..cae5147e9
Binary files /dev/null and b/doc/src/LectureNotes/_build/jupyter_execute/chapter6_138_0.png differ
diff --git a/doc/src/LectureNotes/_build/jupyter_execute/chapter6_138_1.png b/doc/src/LectureNotes/_build/jupyter_execute/chapter6_138_1.png
new file mode 100644
index 000000000..f889a46a4
Binary files /dev/null and b/doc/src/LectureNotes/_build/jupyter_execute/chapter6_138_1.png differ
diff --git a/doc/src/LectureNotes/_build/jupyter_execute/chapter8.ipynb b/doc/src/LectureNotes/_build/jupyter_execute/chapter8.ipynb
index 1f63eb927..f91d8f826 100644
--- a/doc/src/LectureNotes/_build/jupyter_execute/chapter8.ipynb
+++ b/doc/src/LectureNotes/_build/jupyter_execute/chapter8.ipynb
@@ -94,27 +94,27 @@
       "MSE before scaling: 0.01\n",
       "R2 score before scaling 0.94\n",
       "Feature min values before scaling:\n",
-      " [1.00000000e+00 2.52277631e-04 1.58998839e-03 6.36440033e-08\n",
-      " 4.01118504e-07 2.52806307e-06 1.60559584e-11 1.01193226e-10\n",
-      " 6.37773764e-10 4.01959093e-09 4.05055916e-15 2.55287874e-14\n",
-      " 1.60896055e-13 1.01405288e-12 6.39110289e-12 1.02186547e-18\n",
-      " 6.44034203e-18 4.05904756e-17 2.55822858e-16 1.61233230e-15\n",
-      " 1.01617794e-14]\n",
+      " [1.00000000e+00 3.86972479e-03 3.66528972e-03 1.49747700e-05\n",
+      " 1.41836625e-05 1.34343487e-05 5.79482386e-08 5.48868704e-08\n",
+      " 5.19872323e-08 4.92407803e-08 2.24243736e-10 2.12397083e-10\n",
+      " 2.01176282e-10 1.90548268e-10 1.80481726e-10 8.67761543e-13\n",
+      " 8.21918258e-13 7.78496845e-13 7.37369358e-13 6.98414609e-13\n",
+      " 6.61517814e-13]\n",
       "Feature max values before scaling:\n",
-      " [1.         0.99869632 0.999475   0.99739435 0.998172   0.99895027\n",
-      " 0.99609407 0.99687071 0.99764796 0.99842581 0.99479548 0.99557111\n",
-      " 0.99634735 0.99712419 0.99790164 0.99349859 0.99427321 0.99504844\n",
-      " 0.99582426 0.99660069 0.99737773]\n",
+      " [1.         0.99791107 0.99418827 0.99582651 0.99211148 0.98841032\n",
+      " 0.9937463  0.99003903 0.9863456  0.98266595 0.99167043 0.98797091\n",
+      " 0.9842852  0.98061323 0.97695496 0.9895989  0.98590711 0.98222909\n",
+      " 0.9785648  0.97491417 0.97127716]\n",
       "Feature min values after scaling:\n",
-      " [ 0.         -1.70327945 -1.70487638 -1.10269333 -1.11031464 -1.11779074\n",
-      " -0.8675412  -0.87387845 -0.88025384 -0.88666051 -0.73621263 -0.74081963\n",
-      " -0.74548023 -0.75019465 -0.75496285 -0.65064103 -0.65385832 -0.65711132\n",
-      " -0.66040159 -0.66373067 -0.66710007]\n",
+      " [ 0.         -1.66745992 -1.72823447 -1.09228638 -1.1114811  -1.1313648\n",
+      " -0.86506245 -0.87590574 -0.88698187 -0.89829265 -0.73687981 -0.74440021\n",
+      " -0.75205856 -0.75985475 -0.76778823 -0.65190491 -0.65755533 -0.66330693\n",
+      " -0.66916042 -0.67511627 -0.68117476]\n",
       "Feature max values after scaling:\n",
-      " [0.         1.7648157  1.69590827 2.29614193 2.24159129 2.18630067\n",
-      " 2.72674862 2.67767959 2.62822832 2.57838874 3.10180456 3.05384136\n",
-      " 3.0057178  2.95743601 2.90899738 3.44157349 3.39262419 3.3436382\n",
-      " 3.2946235  3.24558749 3.196537  ]\n",
+      " [0.         1.76330702 1.70952132 2.29506297 2.25330559 2.21033632\n",
+      " 2.73427311 2.69911557 2.66307432 2.6261167  3.11835914 3.08796041\n",
+      " 3.05688927 3.02512212 2.99263434 3.46490261 3.4379033  3.41038453\n",
+      " 3.38232999 3.35372259 3.32454443]\n",
       "MSE after  scaling: 0.00\n",
       "R2 score for  scaled data: 0.97\n"
      ]
@@ -326,13 +326,7 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Test set accuracy: 0.95\n"
-     ]
-    },
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
+      "Test set accuracy: 0.95\n",
       "Test set accuracy scaled data: 0.96\n"
      ]
     },
@@ -829,10 +823,10 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "0.07206099142503472\n",
-      "4.125937788943556\n",
-      "[[ 1.03889255  3.1254397 ]\n",
-      " [ 3.1254397  10.77395298]]\n"
+      "0.11455861677725063\n",
+      "4.414311328416008\n",
+      "[[1.04460586 2.98508511]\n",
+      " [2.98508511 9.60711152]]\n"
      ]
     }
    ],
@@ -873,10 +867,10 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "0.07865526085644693\n",
-      "1.4346285243728323\n",
-      "[[1.         0.63566281]\n",
-      " [0.63566281 1.        ]]\n"
+      "0.07650519724946082\n",
+      "1.5077307406836722\n",
+      "[[1.         0.63582252]\n",
+      " [0.63582252 1.        ]]\n"
      ]
     }
    ],
@@ -931,30 +925,30 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "[[ 0.53680052  2.50858953]\n",
-      " [-0.62753667 -1.74895806]\n",
-      " [ 0.16570918  0.64706639]\n",
-      " [-0.27228985 -0.08345099]\n",
-      " [ 0.85643181  1.55737697]\n",
-      " [-0.329196   -2.18391809]\n",
-      " [ 0.92272798  2.91370357]\n",
-      " [-0.37864859 -0.25719203]\n",
-      " [-0.55226917 -1.4496257 ]\n",
-      " [-0.32172921 -1.9035916 ]]\n",
+      "[[-0.21054401 -1.67179046]\n",
+      " [-0.27924545 -1.98111498]\n",
+      " [-1.24089109 -3.51965915]\n",
+      " [ 2.17042981  6.77708985]\n",
+      " [-0.36489938 -0.63821317]\n",
+      " [-1.61222872 -5.67668711]\n",
+      " [-0.67184873 -0.62798419]\n",
+      " [ 1.21860807  4.36876959]\n",
+      " [ 0.0351748  -0.93789716]\n",
+      " [ 0.95544469  3.90748678]]\n",
       "          0         1\n",
-      "0  0.536801  2.508590\n",
-      "1 -0.627537 -1.748958\n",
-      "2  0.165709  0.647066\n",
-      "3 -0.272290 -0.083451\n",
-      "4  0.856432  1.557377\n",
-      "5 -0.329196 -2.183918\n",
-      "6  0.922728  2.913704\n",
-      "7 -0.378649 -0.257192\n",
-      "8 -0.552269 -1.449626\n",
-      "9 -0.321729 -1.903592\n",
+      "0 -0.210544 -1.671790\n",
+      "1 -0.279245 -1.981115\n",
+      "2 -1.240891 -3.519659\n",
+      "3  2.170430  6.777090\n",
+      "4 -0.364899 -0.638213\n",
+      "5 -1.612229 -5.676687\n",
+      "6 -0.671849 -0.627984\n",
+      "7  1.218608  4.368770\n",
+      "8  0.035175 -0.937897\n",
+      "9  0.955445  3.907487\n",
       "          0         1\n",
-      "0  1.000000  0.907143\n",
-      "1  0.907143  1.000000\n"
+      "0  1.000000  0.972249\n",
+      "1  0.972249  1.000000\n"
      ]
     }
    ],
@@ -997,37 +991,37 @@
      "text": [
       "     0         1         2         3         4         5         6         7   \\\n",
       "0   0.0  0.000000  0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
-      "1   0.0  0.076168  0.080935  0.078268  0.081921  0.085682  0.070727  0.073798   \n",
-      "2   0.0  0.080935  0.086925  0.081766  0.086005  0.090410  0.072900  0.076302   \n",
-      "3   0.0  0.078268  0.081766  0.085075  0.088305  0.091557  0.079623  0.082640   \n",
-      "4   0.0  0.081921  0.086005  0.088305  0.091884  0.095514  0.082116  0.085368   \n",
-      "5   0.0  0.085682  0.090410  0.091557  0.095514  0.099557  0.084565  0.088068   \n",
-      "6   0.0  0.070727  0.072900  0.079623  0.082116  0.084565  0.076371  0.078937   \n",
-      "7   0.0  0.073798  0.076302  0.082640  0.085368  0.088068  0.078937  0.081684   \n",
-      "8   0.0  0.077019  0.079892  0.085767  0.088751  0.091725  0.081567  0.084509   \n",
-      "9   0.0  0.080401  0.083686  0.089005  0.092271  0.095547  0.084258  0.087409   \n",
-      "10  0.0  0.062426  0.063699  0.072024  0.073913  0.075721  0.070361  0.072486   \n",
-      "11  0.0  0.065031  0.066507  0.074745  0.076802  0.078786  0.072797  0.075066   \n",
-      "12  0.0  0.067773  0.069474  0.077585  0.079825  0.082002  0.075324  0.077748   \n",
-      "13  0.0  0.070658  0.072612  0.080551  0.082990  0.085376  0.077944  0.080532   \n",
-      "14  0.0  0.073697  0.075932  0.083645  0.086302  0.088919  0.080656  0.083422   \n",
+      "1   0.0  0.062851  0.073533  0.065769  0.068538  0.069947  0.059538  0.060320   \n",
+      "2   0.0  0.073533  0.087625  0.078561  0.082559  0.084744  0.071705  0.072983   \n",
+      "3   0.0  0.065769  0.078561  0.072769  0.076696  0.079022  0.068407  0.069803   \n",
+      "4   0.0  0.068538  0.082559  0.076696  0.081211  0.083977  0.072511  0.074206   \n",
+      "5   0.0  0.069947  0.084744  0.079022  0.083977  0.087102  0.075127  0.077070   \n",
+      "6   0.0  0.059538  0.071705  0.068407  0.072511  0.075127  0.066156  0.067782   \n",
+      "7   0.0  0.060320  0.072983  0.069803  0.074206  0.077070  0.067782  0.069583   \n",
+      "8   0.0  0.060735  0.073737  0.070715  0.075353  0.078424  0.068928  0.070878   \n",
+      "9   0.0  0.060967  0.074217  0.071359  0.076188  0.079434  0.069792  0.071870   \n",
+      "10  0.0  0.052371  0.063245  0.061909  0.065807  0.068404  0.061247  0.062897   \n",
+      "11  0.0  0.052613  0.063714  0.062491  0.066554  0.069297  0.062003  0.063759   \n",
+      "12  0.0  0.052756  0.064027  0.062920  0.067118  0.069987  0.062592  0.064441   \n",
+      "13  0.0  0.052853  0.064260  0.063261  0.067574  0.070552  0.063079  0.065009   \n",
+      "14  0.0  0.052933  0.064453  0.063553  0.067966  0.071040  0.063502  0.065504   \n",
       "\n",
       "          8         9         10        11        12        13        14  \n",
       "0   0.000000  0.000000  0.000000  0.000000  0.000000  0.000000  0.000000  \n",
-      "1   0.077019  0.080401  0.062426  0.065031  0.067773  0.070658  0.073697  \n",
-      "2   0.079892  0.083686  0.063699  0.066507  0.069474  0.072612  0.075932  \n",
-      "3   0.085767  0.089005  0.072024  0.074745  0.077585  0.080551  0.083645  \n",
-      "4   0.088751  0.092271  0.073913  0.076802  0.079825  0.082990  0.086302  \n",
-      "5   0.091725  0.095547  0.075721  0.078786  0.082002  0.085376  0.088919  \n",
-      "6   0.081567  0.084258  0.070361  0.072797  0.075324  0.077944  0.080656  \n",
-      "7   0.084509  0.087409  0.072486  0.075066  0.077748  0.080532  0.083422  \n",
-      "8   0.087542  0.090669  0.074644  0.077376  0.080221  0.083181  0.086261  \n",
-      "9   0.090669  0.094040  0.076827  0.079721  0.082739  0.085887  0.089170  \n",
-      "10  0.074644  0.076827  0.065759  0.067867  0.070042  0.072284  0.074590  \n",
-      "11  0.077376  0.079721  0.067867  0.070098  0.072403  0.074782  0.077234  \n",
-      "12  0.080221  0.082739  0.070042  0.072403  0.074845  0.077371  0.079979  \n",
-      "13  0.083181  0.085887  0.072284  0.074782  0.077371  0.080053  0.082828  \n",
-      "14  0.086261  0.089170  0.074590  0.077234  0.079979  0.082828  0.085781  \n"
+      "1   0.060735  0.060967  0.052371  0.052613  0.052756  0.052853  0.052933  \n",
+      "2   0.073737  0.074217  0.063245  0.063714  0.064027  0.064260  0.064453  \n",
+      "3   0.070715  0.071359  0.061909  0.062491  0.062920  0.063261  0.063553  \n",
+      "4   0.075353  0.076188  0.065807  0.066554  0.067118  0.067574  0.067966  \n",
+      "5   0.078424  0.079434  0.068404  0.069297  0.069987  0.070552  0.071040  \n",
+      "6   0.068928  0.069792  0.061247  0.062003  0.062592  0.063079  0.063502  \n",
+      "7   0.070878  0.071870  0.062897  0.063759  0.064441  0.065009  0.065504  \n",
+      "8   0.072303  0.073408  0.064109  0.065065  0.065830  0.066473  0.067035  \n",
+      "9   0.073408  0.074615  0.065054  0.066094  0.066934  0.067645  0.068268  \n",
+      "10  0.064109  0.065054  0.057780  0.058594  0.059248  0.059799  0.060281  \n",
+      "11  0.065065  0.066094  0.058594  0.059478  0.060193  0.060800  0.061333  \n",
+      "12  0.065830  0.066934  0.059248  0.060193  0.060965  0.061622  0.062202  \n",
+      "13  0.066473  0.067645  0.059799  0.060800  0.061622  0.062326  0.062949  \n",
+      "14  0.067035  0.068268  0.060281  0.061333  0.062202  0.062949  0.063612  \n"
      ]
     }
    ],
@@ -1454,10 +1448,10 @@
      "output_type": "stream",
      "text": [
       "          0         1\n",
-      "0  3.884269  1.937050\n",
-      "1  1.937050  1.951359\n",
-      "[[3.88426936 1.93705016]\n",
-      " [1.93705016 1.95135877]]\n"
+      "0  3.963873  1.979586\n",
+      "1  1.979586  1.976796\n",
+      "[[3.96387325 1.97958566]\n",
+      " [1.97958566 1.97679551]]\n"
      ]
     }
    ],
@@ -1486,13 +1480,13 @@
      "output_type": "stream",
      "text": [
       "Centered covariance using own code\n",
-      "[[3.88426936 1.93705016]\n",
-      " [1.93705016 1.95135877]]\n"
+      "[[3.96387325 1.97958566]\n",
+      " [1.97958566 1.97679551]]\n"
      ]
     },
     {
      "data": {
-      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAD6CAYAAAC8sMwIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOy9e3yU5bnv/b2fOeREEsNkDJkQjUmAKDQYOVmESqELtagV9+pqt5a9oNYDa/vy7mV3d5f6vv+8+6PdXV2l7+bTbkSK8FZl9Si2SkV2oSAHhYCRNJEAMzGYZEKcDHEScpyZ537/eOZ5mJkcCBAhgfv7+fAhc7qfmcnkmuu57t/1u4SUEoVCoVCMX7Sr/QQUCoVCcXmoQK5QKBTjHBXIFQqFYpyjArlCoVCMc1QgVygUinGOCuQKhUIxzhmVQC6EuEEI8XshRJ0Q4rgQ4sujsa5CoVAoLox9lNb5n8AOKeXfCyGcQPpwd87NzZVFRUWjdGiFQqG4Pjh69GiblNKdfP1lB3IhRBbwFWAlgJSyH+gf7jFFRUUcOXLkcg+tUCgU1xVCiNODXT8apZViIABsFkJUCSF+KYTIGOQJPCGEOCKEOBIIBEbhsAqFQqGA0QnkduAOYL2UsgLoAv4l+U5SypellLOllLPd7gFnBgqFQqG4REYjkDcBTVLKQ7HLv8cI7AqFQqG4Alx2IJdSngEahRDTYlctAT6+3HUVCoVCMTJGS7XyfwCvxxQr9cCqUVpXoVAoFBdgVHTkUsqPYvXvcinlQ1LK9tFYV6FQKL4oXtrr46CvLeG6g742Xtrru0rP6NJRnZ0KheKKMNYCZ/nkbJ7eWmU9p4O+Np7eWkX55Oyr8nwuBxXIFQrFFWGsBc75Jbn8/JEKnt5axdqdJ3h6axU/f6SC+SW5V+X5XA6jVSNXKBSKYYkPnN+ZdxOvHfr0qgfO+SW5fGfeTazb7WXN4tJxGcRBZeQKheIKEh84vzPvpqseOA/62njt0KesWVzKa4c+HVD6GS+oQK5QKK4YYylwmqWdnz9SwTNLp1lnC+MxmKtArlAorghjLXBWN4USSjtm6ae6KXRVns/lIKSUV/ygs2fPlso0S6G4vnhpr4/yydkJ5ZSDvjaqm0I8dXfJVXxm4wchxFEp5ewB16tArlAoFOODoQK5Kq0oFIohuRjt91jTiV9PqECuUCiG5GK032NNJ349oUorCoViWMyAPBLt98XcV3HxqNKKQqG4JC5G+z2S+6oSzOijArlCoRiWi9F+j+S+qgQz+qgWfYVCMSTx2u/5JbncWeIa0pNkqPveMz2PB2Z6EvTaqxcV89iWI9xx0w1UN4fYsGKWdft4kiSOFUmlysgVCsWQXEzTzMvv1bN6UfGAgP1xS8eADHz9nnrunZHHAV+QcFS31hhv2flYObtQm50KxTXGlcoSk49z0NfGk68e5f7yfH70cHlChg4kbIKuXlTM+j31fGfeTWw+2ADAqvlFF9wgXbn5MHeVunh84fnXsXGfjwPeIFtWzR2113YxXMkNXrXZqVBcJ1xOlngxG5HJxzF5u7plgC1s/Cbo3VPdrN9Tb7Xqb1gxi3BUH9Fm6l2lLl7cXsfGfcbz2bjPx4vb67ir1HXB1/ZFMRaMwFQgVyiuMYbz2b5QoL6YL4HBjrNhxSxWzS8aENTiN0F31JxJKMEAOGwa80tcF9xMfXxhCc8tK+PF7XX8w0sHeXF7Hc8tK0vI0K80Y8IITEp5xf/NmjVLKhSKL5afvlsnb/7h2/Kn79ZZ1x3wBmTF/7NTHvAGBr0cf91P360bcNuFjjPYY4c75kiez2B8c/0BefMP35bfXH/gkt6b0eJSn/+lAhyRg8RUVSNXKK5BhqvbjqSmu3bnCdbt9jK/xMXTSQMX4uvt8WuZtW5TgWLelqxaiV8DuOh6vllOmVOUQ2VD+1XNyK+0akXVyBWK64QL2cVeqKYbXyr4W3OIJ189OmipJfk495fnJ6xjll5udmVQ3RRKKDnML8m1yjXzS3JZufmwVfeeX5LLU3eXsHGfj5WbDyesaQbx55aV8dun5ltlFvOxV5qn7i4Z8P6Zz/9KogK5QnGNcSHJ4HA13eTgvGHFLACefPXogHp78nF+9HA5G1bMSpAmmkHNrL0/+0Y1B31tA74QNMGINjEPeIMJGbhZMz/gDX5B7+b4QJVWFIprhJGc5ic37SRfHmqNX+z2csAXZM3iUp5ZOu2Cx3z5vXqe+ErihubGfT5+8u5JNGFsbppfEmb5RROw9VCjVTJ5ZF4hhRMzLpjdjpWmnCuBKq0oFGOcy/UgGYni5ELZ+mClAoDjZzoHzeCHOuZdpa5Bm4Cm3DiB3rBOOKrzgS/IY1uOcN+MPADeqWll6qQJHG5oJ3eCk3dqWoeVTJrvV/xzOOhr49k3qsdVU9FoMGoZuRDCBhwBmqWU9w93X5WRKxQDuVC2fDFrXGpzysU0+Qy3eVrdFMKmYTX9bNz3CfNLJrLvVBBNg6guCUclRa40Tgd7eG5ZGQ1tXbx+qNF6Lo/OK+SF5eVDZtfJDUdPvnqUcFS3sv1r0XXxSmTk/ydwfBTXUyiuK4bTf48EM3OP38iMv34kJGfYbx3zA/DATE/Cc0yugydvnpZPzmb9nnrunupm3W4v2Wl2dtUF+ObsAr6/dCrhqJFANgR7eGReIT9+5wSvH2pEE8aaMzxZbD3UyPJf7B+Rjv2DWKt/b1hn1fyiazKID8eomGYJISYDy4AXgGdGY02F4nokPiiuSZL9wfD14PLJ2Tz56lEA1iwuZfPBBjYfbLBq0RfCXNsMjt+ZdxNvV7dwf3l+wvHMTs3448dvnt5Z4qK6KcR9M/J4/VAj0yZN4MSZc9gE/LayCQRoAnQJRa503qlpJTPVTnt3GLtNY9mX8nmzqhmAY00hXvvePKt+b9MgqpOQnZdNymTdbi+pDi3hOVxPwXy0MvL/F/hvgD7UHYQQTwghjgghjgQCgVE6rEJxbXGhLsFLab9/65h/RLV3c204n9WHo7qVjcfz0l4fz75RzcZ9PuvM4c4SF7cXZvPYliM0nu1i66FGZniyOHHmHDM8WegSIrGSii5hQamLjt4I983Io6svil0T9EeMEGK3CSRgt50PUTbNULaYV5lln48aPyfVoeGwadxZ4hogt7weuOwauRDifuDrUsp/EkIsAv6rqpErFBfPSGvkQ9XBzYz6A1/QyujvLHHx1jE/79a2Wvd79o1q3q5uGdQ61szqTUdCh03j/vL8AQ09pgIFCT+4dyr1gS7erGomqsM3ZxfQ/HkvNgG76gIU5qTR2N5DUW46DW3dgBGU0512HpyZz9ZDjdxemI030MXMyTew39uG0yZYVp7P9uozpDg0y1BrMLMt8/kBCTVzpVq5OO4CHhRCNAC/BhYLIV4bhXUViuuKkVrGJtek45ttav0hXjv0KcsrCti47xNq/SFudmUk1N7frm5JWC85qzdrzU8sLGbDilm8Xd2S0BT07BvVrNvl5Qf3TCXFofHTnSf53ZEmesI635xdwDs1rRTckMruugAzPFk0tvfgnuCkoa0bASyv8JDutBOJ6vz+aDN3leZSd+Yca5aUcqzpc5w2gRCCb84uZMt35yQYaj2+sMR67eUF2WxYMYsfPVxulXvM9+tqNOVcTUZVR64ycoVidBisFv7sG9WAsfEY3xY/pyiHjxqNmvTWQ408Mq+Qd2parctmA43Zdm9m6oNl9Wa2Hm8p+9YxP60dvXzUGLKOGYnqPFRRgHtCCut2ewGj3h3s6udrt+axraqZGZ4savwdFLnSaQh2YxNGqWTzqjmAoTK5aWI6tf4OllcUsPdkwGrnByO7Xr2omHW7vHypIJu6M50JGfmFVDnXor5c6cgVijHEcJpxc1Mvvs67cZ+PPxxt5o8f+Vn5SiWrFxVzZ4nR9VjZ0M59M/L4TWUTd5W62HqokbununmnppXnlpUR1QfW3oEBWf3GfT7erW1lw4pZPLN0mjXFp7Wjl/2ngtyWb2wqfu3WPCJRyW8qG/nFHqPO7rRr+D/vobsvwraqZqZNmkCNvwO7Jgh29bO4zI1N0/j7WQVWHX7NklLqznQy3ZNlOSL+6OFywDg7Wb2omLU7T7FhxSy2Pn4nqxcV8+L2OlYvKh7UeiCZsTL04UowqqPepJR7gD2juaZCcbUYzYwueS2zFj3cEIbVi4p5emsVd09182ZVM88tK6M+0MUfP/Lz050nLb10rT/E2p2neGCmh21VzcwtymFbVTNrFpcS1bG+FMzsNTPNzsrNlaTYz6s8Vi8q5sfvnOBbcyZb9fn1e+p5+A4PNf4OhID93iC5E5xsq2rGphmqE6RkhieLnnAUX6ALMBQpJ86cQxMgMQL24wsN75S1O0/xzNIpvHXMzxsfNmMT8OnZbp5ZOoX1e+ppaOvijQ/9PFRhZOWbVs623rO3q1u4vTAbc6CQOYHo5ffqE35H8e+1GezvnupmR82ZhPWuJVRGrlAMwWhmdMMNYXh04wc8+erRhCEMZjZa6s5gW1UzD1UU8PjCEn70cDnfW3BLQnfk+j31PLN0CntPBlhe4aGyoZ3lFQW8duhTbBqs3XnK8v8+6Gtj3S4vNmFsEpqZ7fo99XxrzmS2Hmpk2bp9PPnqUVYvKuadmlbuL8/Hpgk0AW3n+gFDAigwgnaNv4NP2owg7slONQI8RqD/9pzJrN9Tz0FfG9M92TxU4TEux7xRUhw21iwpZf2eejzZqbx+qJFnlk7hgZke3q1ttd6ng7426gNd1LZ0YtOMYG1+MRTckGqd4ZhnM49tOcJBXxvzS3K5e6qbbVXN3Dtj0jUZxEF5rSgUwzKaY7wGW8tUmKQ6NF5ZOSdBrXJbfib7vUHmFuXgDXQNGJn28r56esM6yys87D3ZZtWPh/rfPO5wtrJvHfNT6++wVCUbVszil/vqee+kYWzVHz0fL6Z7svAFztEbNlLkSVkpnOnow66Bpmn0R3RuL8zm1vwsAEs5E/+av790Kuv31OPOdHLizDkWlLp47Xt3Dvl+1fpDlpnWAW+QxWVudtcFrH2B5Necn53Kx/4OHorV4Me7omWoGrkK5ArFBYjfJIw3jLrcteI3HE0p3ZcKsvlbc4gHZ+bz+qFGFpTmcqzpc8tIKqpLvnG7hwdmeiyZoK4bEsCobmT+Znt8/OUTZzqtcov5GuJLEGYm+7P/fYru/igAdg2mTcrkY7+RBQshkBJ0Ka2sO9Whcd+MSfzxI791HRgNScGuPl4/1MiMgiz8n/cmtNJPTHfyWWcvdptGQXYqda3nKMxJo+1cf0L5I/79Sk+xczrYRePZHvZ728id4KTtXD95WSm0dvSxvMLDX45/ZpWr/vk3VWyr8uOwCf6/7861jg2M2xZ+tdmpUFwCoznGK36tzQcbrHJK/NzKg74gfeEovzvazKPzCjnW9DmRqE5lQzsl7gwiUUmxO8Maq/bKyjn8h1kFrN9TbwVlsz3evGzTYFtVs5Htxr0Gmwb/aZPhA374kyAvbK8jEtVxxPrkIzrU+ju5MSuFSKyU8sP7ppFiPx82+iM6O2rO4Ihr3HHYBC/vq+dPx1qYUZBFTXMH+dmp1PpDViC12wSRqORcbyQWxFNpbO8hJ92RYIAV/94f/iTIb480sd/bxrRJmVaZp7WjD/cEJ9uq/FYD00FfGztqWlle4SHVYePJV4/ygc8o5yR3ql4LjOpmp0JxLZHckGNm0JdSXkleK3Cub4Ce22HTuOOmHA59chYbEldGCmBI9mZOzuaD+rP86rG5A/TmpsLD1E+b7fGPbTnCvTPyeLPKz5IyNwd9Z/lyyUSefPWoVZf+1pzJvLC9jpx0IxT0R6VRGom10IMRKDVhZOR/rm6hL3K+gVuX0BPWcdoEKXaNybHmn96wji7hdLCb5RUFbKtq5nhLByl2jU0r57C92m9tjqbaNRrbewFoO9fHgim5/N9v1tDY3sOWVXOs937V5kqisSd14kxnwnsXONdvbcDW+kOs31NvZfYHfW18d0vlqJ1VjUVURq5QDMFIG3QuZa34IQxmkDdldv9y3zT6o5J1u72sml/E1269kf3eINMmTRiw7kFfG6eDXazb5eWdmhYO+to4/EmQrYcamXVzDtuq/NyQ7uD9eiOIVza00xuO8uN3TnDP9DyWlXtw2jXauyPkTnACRiauS8hw2qzj6BL6IjpVjSHcmSnW9amx7DwjxY7TrnFn8UT6IzpFrnT6IzpzinL42bduZ0GpC11COCr5+W4vv64873LYG/tiEJz/8mhq78EWM9B6aa+PWr/xnpe6M4Z8j6M6fHNWQcLmrslIhzuPV1SNXKEYAcn1ZFO5Ej+78lI30Aazjn1sSyXhqEQTRvB7KFb/BaxsOifdgf/zHp5ZOpV1u7xEdUl/JIoujaAYjf1v/oV7slOJ6pLWzj4AllcU8Oe/tRCJSmbdfAOHG9oTMnGnXSMS0dEBmzDWi8fc3HRPcBI410+RK53TwW4emVdI8+e9FNyQyhsf+nlm6RTW7jxFiTuDGn+H9fjB1ixypdPa0cczS6cw3ZNtSQe3VTWzuMxNXlYqvz3SZGXmYJw9aELgsAkeqijggZmeATNFL8caeCyhauQKxWUQLx80NeBPvno0YXblSGWJyc1AZvA3r396axWbVs7hgZke+qMSh81oVzddDNft8nL3VDe+QBc9YZ0fv3OCNUtKueOmHCI6SHk+QJrhThPgD/VaQdweq5v3RXRuSLdz/Ewndk2gS5iQYmTi/RGdb1R4SHdoAwIuwJmOPvKyUgic6yfVJmgIdnNjVgpvfOjnrlIX79S0Mm3SBH6y4yTPLJ3C6bPd2E2fWgYGcYCGYDf3zpjE+j311PpDZKbY2VbVjNOu8b4vyO9iQVwTxmsA44vngZkeNq2cY0kWzfd0NM+qxjIqkCsUIyDZ+9rkA1/wojO84fTpZuAB2FFzhuUVBaQ4bLx1zM/8klw2rJjFlwqyrcYfMBwFf/TnOvZ72yhypTPYObYuh74c7AqTmWInzWmj1J3BuT5DtaIJeLPKT3dYJ8UmSEZg1M/tGvRGJTZhXC65MYN1u7zcMz2PW/OzEAIa2rqIRHUisSA8FDnpDv78txYmpjt4cXsdzZ8bJltISapDIxJ74s9+vYybXRnW8/zTMT+1fmOv4P/aVmNtlgJWJm66PV6LPixqs1OhuADxpQ+zrX15hYfPOvoG9Q1PLpWY0r4D3qA1x9Jsf793xqSEjsN4vxPzumffqOZ3R5po7ejlewuLqTvTyYLSXPZ725juyeR4S6dVTvGHekf0mvS4souZrafaNbyxDUiA7DQH7d1hBNA3SPpsXmPufUYlZKfZqWnuwKYJq8Tx97MKeP1QI3ZN4LRrllVtMp7sVPyhXgTgDXShCWNt07OlvzuCJrBUM41ne6wN1qxUOy9sryPFrnFXqYvHtlQSlbBl1ZwBXbPXIiojVygugJlBb9znizkLenizys+Hn7aPaI6l6aNdcEOqtc76PfWxzciBHYcft3QQ1aXlI17sziCiS/5aF2DV5kpunpjGfm8bKXaNk2fOWdm1xCiHDJfxxiOBJWVuq9zRmxRg27vDAJTEbTBeaOlQT8RYW0q2V/s5/EmQ3x1pJifdQUSXgwZxc01/qNdq689Jd6BLQ8de4++wXtOdxS42rZzD2p2n+ObsApx2jZZQL8fPdOKwCfoiOv0RSW9YxyYu7YxpPKI2OxWKEbBxn48Xt9fxUEUBfzneSiSqY0+aBB8fLJJdBE0Xv5smpvOxv4O7Sl3s9wZZUOri45ZO67Hf+MV+Ah19tHf3I4RAl0bwiy+FmEHNoQn6opJ0h0Z3eMiZLheNAIRILL84NEFYl6TYNbJS7QRiGu6hcGU4CHYZ2bwWa04ymeC00dUfTajfm8cSwI2xBh/Tx3xCio1zfVGWVxSwo+aMtRFq+qevfOWw1XE6bVImJ850srzCQ2FO+jUnORxqs1OVVhSKERDV4aGYHnp+iYunF5cC51Ur8T7YgGVgZQaS6Z5swlGdWn8H0yZNiAXxXD5u6bDMsVYvKqa2uYOILnHajMJH/yAlDTPomeWOSHIB/BIwyxdgZMRSQopNWMeYkGonzWHDH+q9YBAHo+5urhUfxDMcGuf6o2hAmtNGil2zMn8B2DVBa0cfqQ6NxvYeUmyCc31RKgqzmTYpk9s8mby4vY6vlrn53sJiav0h6z3SMPTlk3PS2FFzBrvtvClYZpp9wIi4awkVyBWKEVA+OZuX36tPsIGNn12ZPMcSsLodX95Xz8Z99cjYeLP93iAzPFl83NLBzRPT+Mm7J5lblMOP3znBV6bmsu9k26ABPBmzxh0ewX2T0Uicy2hazsZ/KcTXxdu7w7QTvujjxOPQoCusWxn2HTfdwIeffm7dLoTxD7D8W/qikorCbE6f7UnYd/jJjpPs9wbpj+ikOTRK3BOsEkygs4++iE4acGeJi8w0Oy9ur+O5ZWWX9fzHMiqQKxTDYG5Urt9Tn2AD+9iWI0Naoho68CM8s3QKnT0Ra/CCK8PBfm/Q0lOXujOoajRkcPu9QYpc6QCER5hhS4yGnOTa9kjQwfIoMRlhaX1IBtOFxxPWwakZGfaMgiz2e4O4MhzckOagJdSLLhnwBTZtUianz/ZYdrUAjy8sSXhf87JSqfF3sKTMzeGGdm7MTKEl1IuUkl/s9nL8TKfly36tomrkiuuWkfiNxwflxxeebzBZvah40FP1lZsPown4comLn+w4id0muDEzhYagIaMzNxbjjafiSbFrIOUAlYgmjHJH/LWpDs3KXK80yRn9xeLKcJCT7kxQySQzOSeN5vYeHplXSOHEDEu/b05EctiE9UW0vKKAn33rdjbu81nqoF/s9nLAF7wuauRKtaK4bhmJ3/j8klw2rZzN+j31rN15wtrUfHxhyaD11rtKXeyuC/CBL4jdJujuj9IQ7MauYZUuIvrgQRyMNvjBpH56LIgLjI5K4AsL4qXuDNIdw4eGSz2yM6ZHD3aFLa8VE5F0v3+cfzOpDo2thxoxPbmiumR3XYDb8jPp6Alb93239gzf3XKY9XvqeeIrxQAcP9M5KmZn4wEVyBXXLfFNPvFBOlkTDnDrpExrLBoYmffGfT7rdjC+CKI6PDqvkF11AVwZTus2ieEKmJd13qfkQpgdliZCwO2F2ZyJK4d8EXgDXaOqgoknvnSS/HUlgTSHhtMmKJyYzrpdXr5c4sJhE7z2/qc8+epRpuRNYIbHKMtEJTy/rAybJugNR/lrXYDVi4wgbv4uRzISbrixe+MFFcgV1zXJE+mTa97m6fyHn7aT6tDYuK+eJ189SsENqby4vc7KFOOz+ReWlzNtUiaN7T3WOlHdqOW2XkQQNjssTexCUNUYuuxa9lgkXvu+YEouN8f2CwA0TXD6bDc9/VECHX2xgc5p9Ed0PvZ3Eo4aZzgPVXiI6gPb8uPdIU3iA/W1MNtT1cgV1zXmH+2tkzKpbg4lDBx49o1qDn9yFv/nPdhtGl+79Ua2VfnRhKFIuavURWVDuzUkYdPKObx1zE9dS4e1iTlaaEm67gttLI5HHp1XyJ+OtVga/TVLSlm3y0t3f8SYPRr3mp9fVsbHfmNYBjBAjx/PSIyzRnMS1BeJ0pErFEkkt26bRlhmk8/b1S309kex2wRfuzWPbVXNltVq6Y0T+N7CYg76Kjl91tjI3F7t5w9Hm87rmsVAj5NLJXmdayWIp9oFvRHjxegS5sT8Y/KyUlm3y0tXXwRdYjksOm2Cb842PNRtMa2iXRP801cNXf9g5bH4EtpQgTr+zCzZcmE8oEoriuuW+FNw05CqP6LzwvbjPL21ivvL8/lv901Dl4ZToC3WPi5ik99r/YlljtcPNSZMykmxada0HcVA3BOc9EYkZXkTcNo1Wjt6qWxop7KhnQdmenBlOC1PmLZz/ZaBWEuoF7smiErJtEmZ/OqxuTy9tQpgSGfDC5XQRnMS1NVABXLFdY3pZwLGH/ucohxq/cZosgdmeli786Q1EScqz8sAe8NRXtheRzgqrWAugK7+KNlpdgTQE9GJXoXS5VhkEPNEAuf6meHJ4vTZHn5wz1QqG9qtjHzFpkM0BLut2vkj8wrZezLAmiWl7DvVRkSXzC3K4eSZTmr9oYTO2sHURMMF6vgzs5Fsjo5FVCBXXLeUT842/FA2V7Jxn8/SIGtArb+DF7YfHyDxS7FrVBRmW92UEV1aDoISSLELQj0RJMaQhNEqrYx3hioF1fg7ePgOD+t2eYlEdfKyUvnarTdazTtfLnbx3LIy3vjQz30z8njfFyQclSwuc/Nf/m4qXy1z8+L2Omr9Qw/1uFCgvhY8yy97s1MIUQj8CpiEIS99WUr5P4d7jNrsVIwVzGk8PbGAnebQePgOw3YVEifswOCbjLYkUyjF+elByZiNRKamXhNGsK483c70/EzK8rP4w4fNSCljAyQEv3psLrX+EGt3nsJzQyoA355baHXb1vpDvF3dwn0z8gcN5iNp/BovfJENQRHg+1LKW4E7gf8shLhtFNZVKC6ZkWqD55fk8vjCYuvylwqy+dOxFtKdtgF2sOb4tPjLTptgsFzoeq+MD6V1N7/vzCD+jds9HPAFyUl3UNUY4g9Hm0DCP8yeTKrDRkSXPLalkvd9QcomTaA+0EV+dhovbq/jvhl5VDeFmO7JpjHmxTIYT91dMqAmfq0Nl7jsQC6lbJFSfhj7uRM4DhRc7roKxeUwnDY4Psgf9LWx+WADTruGAA43tHOuN8LyCg+35mchMbLHDKeW2B5v16zJ8oMFclVRMfBkpw55m00IS87Z2tGHK8OBBObeksPrhxpZXuHh0XmF9EV09pwIUNUY4mZXOvu9bdxVmsvWQ42cONNxXfiNX4hR1ZELIYqA94AZUsqOpNueAJ4AuOmmm2adPn161I6rUJjEn0abwfvuqW5rCk91U8gywTI9wvsjOlFdDvA/MTNy8zqzrHKpRlWKoSlypfPp2W7LT8Y09DLPbOKj1NyiHCob2rnZlUZDsOea8lK5EF+414oQYgLwB+C/JAdxACnly1LK2VLK2W63e7QOq1AkYGbiz75RDWBNYC92Z1DrD3E62GUF8X979yR2zZgq47AJ/uPcQgRYPiO6TAhAXWEAACAASURBVNRvR6XxB9MX0clKtQ08uGLE2OMiz+ScNJo/70nwkzE7YCWJQXxBqYvDDe3YNEFDsIflFZ4RywWvhVb8oRiVQC6EcGAE8dellG+MxpoKxaVgtmO/Xd3CP75ymDermllQmsvJ1k5e2F5HsTuDnz9SYagkdJ327jB5mSk8s3QqLywvp9idMcBnxJTO5aQ7EJpAAh290YEHV4yY+BOaMzHLWTAmEYlBNhicNoFDE+z3Bg0NuS5ZUuamvTs8IrmgaUccf7+N+3w8tuXIuGrFH4rLDuRCCAFsAo5LKdde/lNSKEaGaVwVT+PZLv51xwlmTjYkgpoGR06fNSbe2DXW7fLygS9IXzhKVDeywe5wlJ+8e5KN+3x8e27hgONEJdbMSaGq36NGRWE2MzxZlhvkkjI30UHsfdMcGv1Rafm0F+Sk8ci8QnbXBbir1DUiuWD55GzrTOzprVX8828+4sXtdTyzdMo1UVsfDfnhAmAf8DfOb0o/J6X881CPUfJDxeVg1sFr/SFr8st0Tza/3FfPX+sCLC5zs6suwLRJEzhx5hwAyys8/OX4Z9w0MZ3a2EgzuwZpTjtrlpSydudJesI6di0xW1SMPhWF2Xy9PJ/N+xvwh3qtMXPJgy7iMWeAOmwCmyZwxLxYLmZ8m7lnUurO4HBDu+VhPp74wmrkUsr9UkohpSyXUt4e+zdkEFcoLhezDj7dk81zy8p4cXsdKzYd4q91AR6ZV0hVY4jlFQWcOHMOp02Q6tD4y/HPeHBmPrVxE9mddhtrlpSyfk89D99RgGBkQbw0bqq84uL5qDHEL/d9gj/Ui12D55bdyoJSlxXETftegZGNe7JTCXaFKXVn4MlOozesE47qTPdk89TdJRclNb17ai6HG9qZW5TD3pOBcdW9ORyqs1Mx7og3QfprXQAhjIacOUU5FE7M4L4Zefz5by2kOjRSHDa+v3Qqc4oMSVteZgq6hOmeTKSU/OuOE2Sl2vl1ZRMAGc7zm5jJpVrTRiV5IILi4pBgDVjWNI3HtlSy3xu0bnfGumcl4LkhjamTMqkozMbX1sVnnb3ML3HhsGm8dcwPjNyGduM+H29W+Vle4cEb6LLKLNdCMFeBXDGuMLMv0wTpoC+ILiF3gtMwXPokyNZDjUzOSaPghjQenJnP+j315GWlUlGYTWtnHxOcNj4920M4KolEJQ3BbqKxVvvu/vObmMlFR7N7U1XJR4fesM7CUpfVVasJw54WBGX5WUbwDnRZI+GcNo2oDk8vLmXDilm8W9tqfRYuNCDkoK+NtTtP8dyyMn72rQp+/kiFVTMfT634Q6ECuWJcYWZfhjdKvXV9V1+Eu0pd7IrVyP/7QzNoCfXy+qFG7puRR2tHr+URfq4/ys0T0y2flHhUkL6y7K4LWD+bm5w/f6SC1o5ePmoMMSkrhW1VzaTaNfoiOj+4dypw3h/FDMIXcjesbgqxaeVsHl9YYt3/549UXFSNfSyjBksoxjQv7fVxOtjFAzM91h/n89uqef1QIxNSbOgS/vnvpvDjd04Q0SULSl3YbRpbVs1l4z4f/7rjhOVQKDGGF/z74UZlZjUGMf1XllcUsKPmDDnpDvyhXut3N8OTxY1ZKbzvO8umlbMHZNyD+Y1fSz4roIYvK8YppkPhyphD4UFfG78/2oxdM0ah3V54A9M92ThsArsmONYU4omvFHPQ18b6PfVMzcsEsFrti3IzBuiUr3dflCvNYBbtFYXG71AThvd7iTsjIYgLDKfE3XWBAZLB4dwNr4UxbiNBZeSKMU+8Q6EmjIBst2no0nDHk1LyUEUBD8z08OSrR+npj+K0a/zz303h3941/MSTXQwVV48JKbaEeaSFOak0tp8P2tlpdkI9EUrdGXgDXQmTljQBr31v3oAB2cNl3eNljNtIUBm5YtwS71CoS2MSu5SSV1bO4c7iiVaAn1+Sy6r5RUR0SXd/lBe319EX0S0zJsXVxaYZGXfyUOnG9l7jB2F0cIZ6IiwodeENdDGjIOu8141mNHWZahVz4zve3dCUHca7G16ofn4toAK5YsxjOhSmOjSrLBKV8IEvyEeNIR6dV8gbH/pjG6CfUBSbwG4OfOjsjVy9J6+wiOqJXZsOTVCYk2ZdtglIcdhYs7iUyoZ2St0ZnDjTCRiSRKdN48slLkutMtKyyXgf4zYSVCBXjGkO+tpYtbmSSFTn+0unMiHFjtMm6I/orNvtpWxSJsvKPTyzdAovbq9j1s030BDsth5vZvCKq4NdE1QUDqxHawKiUtLY3kNKzMwmosOGFbO4s8SFlBJvoItwVPLovEK2rJqD3aZR2dDOfTPyePm9+hHLDsf7GLeRoGrkijHNS3t9HP4kyPu+IHabxv3l+XzW0cuuuoCVwUkpmTopk+LcCWyrasZpE0SlkQEqxiYiNvs0J91Be3c4QZly8rNzRHUdpLHZffpsD6sXFVMf6KK1o3eAamXtzhOs2+0d1M72elGtqECuGBd84xf7+bi5g2mTMqnxd/DovEKKcjMsvw6nTaDL87Mzk0/jFWMDc3i1xGi/B8HDd3j43ZFmIrpu/A4xbn9uWRmPLyxh4z4fL26v46EKD3tPtiVk3dfSRuZIUJudinHDYN4Z95fnE5WSGn8HMzxZ/OlYC//jnRP4Q70sKXOjS2IuehK7JlQQHyM4kiJMZord2LvAKHnNL5nIOzWtfHN2AVpsA0QH7irNZf2eetbuPMH6PfU8VOFhW5U/YbPyeimbjAQVyBVXhIsx9R9sE2vdLi+pDhtOm6DG38G53ghRXeKwGcayNk2Q5tCI6Fgt30MxmI5ZMfoIIPlXEeqN4J7gJMWhkZViZ1ddgIpCY06qLfaLsWuC/d42bsvPZN1uL3dPdbP3ZNuAzUqzu9MM7COxs71WUaUVxRUhPnuaX5LLs29U88aHzfzXe6YS1bGUBm8d83OzKwObBmt3nuLxhbfw0t56JJL/du80OnsirNvtBSDVLojoRiYOhi1t8lQfxdjCrIWbdsELSl287zuLEMbvcXGZmy+XuCz9/4JSFwe8Qb5a5mbuLS5rTJ/5ORrP9e5LQdXIFVed+Hrm5oMNRKI69piv9LpdXqK6xKaJmD/4KfKyUmgIdjPdk0V94Bz9Ed0aB2YS3yxi1wTZaXaCXeGr8voUg2M2AAnAbhPW/kWRK51gV7/lEb+8wsNtnix+suMkdpvgzuKJBLv6OXmm0/qcmEZX5pf/9TZ4WdXIFVeUwUopAGWTjNPlVfOL2LRyDgA/3WlkX939Ub52642s3XmK+SUTaQh2Y9fg07PdlE3KJBoXxJ0xyZoZxPOyUojoUgXxMYJZvTIbgPKyUhACwjEp6O2F2WSnO4jqMhbEC/jL8c9Yt8vLgikubJpghiebxrM9bFo5hw0rZhHVsVwLu/si110QHw4VyBVfCIPVuZ989Sh/aw5ZtU6AVfOL6A3r9Ed05hblsK3Kz6ybb2B3XYAlZW4iutHQYzoXmiRrw4eaLKO4ssRPva8ozMauaaQ5NKZ7sshIsVtnUMvK8/nhvWXYNEG600ZrR6+1xvcWFrNqflFCJ6bZqXk9dGleCvar/QQU1ybxzRq3Tsrkw0/bsds0NqyYxfySXDLT7KzaXIkmINWhoUti47c87Khp5ZF5hbxT00phThqN7T3WujZhdHUOdVlxdYmvgVc3d/Av9xm6bkM+WMBfjrcypyiH9XvquWd6HhtWzOIDX9DSgd9Z4uKtY37erW21vvDvLHElKFXiuzTjb7ueURm5YlSJL6mY2dMBX5C+iM6aJaXWBtW6XV6klEQlfH/pVIgNR/7L8c94ZukU3qlpxWETCUEcBgbtqFTuhWMF9wQn8SKVIlc6Ud0YyHFXqYttVc2sml/EKyvn8vNHKrjZZYzMiw/Mtf4Q79a2DiopVHLDoVEZueKSGKpj7p2aFn7xVy8bVswCYPPBBpw2w2d67c6TdPZEeO3Qp9xfnk9rRy95WalEdfjBvVNZt8vLnKIcorpxWr4rbujAcKiE/MpjNmBF4iRCgXP91pDkiA7fnltoNfQc8AZZXlEwIIuOr3PfWeLisS1HEmxqkyWFQ8kNr/esXKlWFJdEspzQvLx6UTHrdnmJRHXLR9pu03hwZj6vH2oEsE6hn95axT3T83hgpofqppAlOSxxZ1Dj7yDdodE9jCY8xS7oi6gwfjUwJ95nOG10xcbjmfVvc0BEutPGPdPzeLPKb3Vpxn9uqptC11T7/JVAyQ8Vo85Q7dEHfW1855eH0KVR/15eUUCxO4N/e/ck4aiO066hCcE3bvfwwEwPT2+tIifdQUuol+JcI4jnZaUQPNefkPElEy89VFx5BEbWvfdEAH+o19KGO2yC6Z4s2rvDnA52s7zCw8++VWE9TgXrS0fJDxWXxYXkhKaC4KW9Pn65r/683awu+ayjlxe21xHVJd+4vYDesCE1LHZnML8kl3um53E62EV3f5QafwfuCU5aO/qGDeKggvjVxm4TFLsz6OyLYNdAIlhe4SESlXzs7+BsVz9rFpey92Rbwmcn3itcMTqMSiAXQtwrhDghhPAKIf5lNNZUjC0GkxOufKWSjxo/T2idbjzbxe66ANFY1144Kq1at00TvFPTYmnA3/cFASh2ZxCJq6AEzvVf2RenuGg0YOEUww+l1J1BVIcHZxqmVo/MKyQclcwpyiE9xc7thdk8+erRhM/Os29UD7BnuBgbB0Uil11aEULYgJPA3wFNQCXwH6WUHw/1GFVaGZ8kd2aanZjmxqY5Zs10IATQdcN3Oi8rhfbuMP0RnTWLS8lMs/Pi9jpui3Vt9oZ1tWk5TjA7MgHWLCnlgDdITrqTbVXNrFlcSnqKHZuG1X355KtHiUT1hHF8gCVFNRlq30U1/ZzniyytzAW8Usp6KWU/8GvgG6OwrmKMEd+MUV6QzS//cTYbVszi6a1V/GK3scE5vSCLLd+dy9S8TMJRSVRKpk3KJNDZR39Ep8iVzv/aY2RYt3myqPV30BPWuX2Q4QOKsckN6Q42rJhFJKrzdnULd5W6eLvab6lSTge7mO7JpnxyNtVNIe4vz0eX8LsjTXx3SyVgfAEkm1uNZFCEYnBGI5AXAI1xl5ti1ymuMeKbMY7HRnDFa8Ul8MN7ywA42dppPe5Ua6dVz/6ss49vzZnMi9vrOHmm09KAJ3duKsYmNs1wHaz1hxBCEOjo48XtdfzD7MnsPRlg9aJi3viwmZWvHObJV49SPjmbYncGfRGdiC7pDet87dY81u+pH3SSvercvDRGI5AP1o8x4CxZCPGEEOKIEOJIIDAyfbBi7BB/mpueYmf1omKe3lrFxn0+Xjv0KcsrPOgSHttSycrNlYSjkiVlbsuR0OTO4oksK/fgsAnCuiTVPvKPoGr8uXpownj/BQK7Jnhhex3hqE7gXB9fLXPzwvJyywdlTtFE+qOSSFTnA1+QtTtPWuvYNcG2qmZWLyoeNEhfD/M1vwhGI5A3AYVxlycD/uQ7SSlfllLOllLOdrvdo3BYxZUk3vv5dLCLdbu83Dcjj7U7T7F6UTF/Of4ZhTlp9EcM35TlFQXcmJVKfPgVwL5Tbfzzbz7CbtNw2AQ9keG9w+NRNfSrgybAYdN4ZF4hEV3SH5WG13jU2CP53sJi4Hw2vd/bxvKKAiSwbreXnrBOmkMzFC26MfjjJ++eHHRjU3VuXhqjEcgrgSlCiFuEEE7g28CfRmFdxRjiqbtLqG4KcdDXxgMzPQD87kgzeVkp/PidE4CRbZv2pNurW/j90SbrD9ccABGOSlo7+ugNR7FrgryslKv4qhQjQZfw8B0FLCv34IydQUnOZ+kmB31tbD7YwPwSF3853powM3V+iYu9J9tYXuEhqkum52cOqJGrQRGXzmUHcillBHgaeBc4DvxWSll7uesqxh6mBBEMxQECGoLdRHTJgzPzeeNDP4vL3JwOdpOT7iAcNYK4JgT3l3sS1tIl9EV05Vo4RkkODHUtHazYdBhd17HHJvloQhCO6jy68RDPb6u21ChfLXPTF44SjkqWV3iwaYJddQFuy89k78k2nltWhjfQxelgV8IxTHfDeJTmfGSMio5cSvlnKeVUKWWJlPKF0VhTMfaIVxW8sP04/bGyiNMm+N2RZrJSjdFdN2al0NrZR15mClFd4rQbdVGzSSjZSzwn3XE1Xo5iGMyClzkWr6oxhEAS0UGX0iqTRHSj7v3vhxuJRHU2rJjF+74g/bE9kmmTsrh7qhGc93uDlE3KZLpHKZRGG9XZqbgo5pfkcvdUN7X+DgCWVxSQ4rAR0XVaO/vQhOENrglo7TSy7XN9hheHlIYZVrKXeHu3GgYxVtGE4NF5hdg0QVHMrVCX0BxzpbRrgvtneqzJTR/4grzvO8uj8wqpajS8VL63sJiUWEnmSMNZnnz1KBtWzOJHD5dfrZd1zaECueKiOOhrY3t1C067hl0T7KhpwZXhtLJrPWYra142jbNMlMxw7BL/ezI2OI29jT8da+HBmR4+CXaxuMzN5Jw0Dje0M7cohx/eN40dNWdYXlFAJCpZt9vL4wtvsVQsT756lO9uqcRpNzx3+qOScHTkG9yKkaECuWLEmKqC/zCrgC2r5vDD+6bRE9ZpCHYnBIFkdYk55V4xton/vZlfxM8snUo4qrOtqhmHTSM/O5Xm9h5smjEIZO3OkzyzdAp/OW74x6c6NDYfbOCgr435JbmUF2THtOM3svdkgDWLS3HYNN46NkDYprgMlB+5YsQkqwrAOLU2sm7JcErCqHK4GvOY033iL//sf59CjymPdF2y9VAjzy0rY7onm/+06TA9YZ2f7DhBisNmzWB965jfsjQ+fqaT5RWeBCtb08L4gZltquFnlFCBXDFiTPXAys2HCXX34w108avH5lqjuhTjG/PMSQBOu8a0vAn8rbmDiC4NF8NTAU6e6WS6x/AQ/6dFJazb7SXdaed/feeOBNlgsdvH2p2n2LRyNtVNIZ5blsX6PfXWY9VAiNFFBXLFiDGnAt1V6uKF7XVoAjbtq2fvSdWwMV6xa4I7iyey3xtEYJw5mYM/Vr5i1Lb/acEtlt88YOm64zswk4nqsGnlbGtwMsB0T7YVvOOvV1w+arCEYsSYNfJ7puehCayJP8mogQ/jA3OSD0BepiEZXVCay8ctHdakpzlFOcy9xWX1EJjBXLkUXh3UYAnFiBnKF9qskb9d3cIfPmwe0vukfHK2pRVPRn3grizDbTF/qSDL+jlwrg9PdioHvG2W9cKDM/M54A1y+JNgQjnkrWN+7pmepzowxxDq70oxgMGGSDy9tcqar7hqfpHlHx4fKPKyUlhc5qa6KcTEDOegwVwJz64sEmPARzICQwpaUZiNw2Z0365aUESqQ+P3R5u5d8Ykfn+0GU1AZUP7eRXK5GzerW21bBpMVAfm1UXVyBUDiO/gHGwe58Z99dZ9Tc8NXRqNQF+7NRV3ZgpnOvqsTk7F1SMz1cah+rOA8XtKsWtkpTqsZq1jTSEyUuzWkIfpnmy+u6UyJjcUbF41FzBKKXdPzWVHTatV+1aMHVRGrhiUwXyhzcx86qRMKgqzMR1ozSYggK2HGi3/FCnPt3gPxlDlF8XooAmsM6c0h8azXy9DCEFrZ5/1+9IlrJpfNGhgDkcltf5QrJs3l21Vfu6dkaeC+BhEZeSKAby018fhT4JUNrRbqoTMNDtvV7dwz/Q8HphpTL6fXpDNp8Fu2rvDlnTN1IubWfpwm57JrfqK0UVKuGliOv/9oRmAMYovHNWxCdBiMsP+qGTjvnruLHFZ93HYNJ5YWMzGffW8uL2OvScCHPAGWV7hsQYpq2A+tlAZucLC3OS0abC7LkB/RCczzU5OuoMXttfxccxfpbopxO2F2XT1RmjvDltZd3zTj1KtXH0k8O25hZbU76aJ6YaHuE0j1WFjy3fn8vyyMnRpBPBfxkpmG1bM4pml09i0cg42zTC7eqiigJ99q0J5hI9RVCBXWJibnG9Xt1AaG8/1wvY6y27U8BLvpXxyNge8QbyBLjzZqSpoj2HW7/HF9jV81Po7yEl3oOvGhPuX36unoa2Lh+8osOZqPjgzn5ffO78HYtM0pnuy2HsyYGXiSqEy9lA6ckUCB31tPPnqUfpik36ScdoEmibQJSwsdbGrTo3tu5LYBAxXkUrW8GsC7DaN/ohu/ByzVEh12Ch1Z1DVGOLReYW8sLycjft8vLi9zmrBV1rxsYfSkV/nDKUNf2mvL+E6U144WBBfUJpLf9QYoPvUV4rZtHLuRc3cVFw+QwXx+M3LGZ4s6zpdkvC77I8N+5g5OZuqxhBLytxsPdTIP7x00Arijy8sUdN6xhnqr/A6YThteDzmuC7HIIqS/d42HDaB067x0t56nt9WTe9FzNxUXB7mb8SmCRxJf7kTM84P56jxd/DovELSnLaE++jSOKP6UkE2+2Obl5tWzmVOUQ6HG9qZU5TD4wsNLbia1jO+UIH8OiFeG75254lBT5PN4H5/eT7TPVmDrnPzxHRS7Boi1qKfYlcSwiuFELBmcSkgCcd9f2oCgl3nN501Ab+pbKK3P5rweC1WljG9xPeebOP5bdVUxi5XNrSzcV/iGZpifKAC+XXEYNrweKqbQtwzPY9idwbHWzpJc2g8v6wMT3YqYAQCb6CLB2fm8/AdBZS4M+iLyGHbwBWjh8OmEezqw5zL8Oi8QlLs2vkhHtKY2CSloSCKP1ea4clCj12vCfioyejqfP1QI4/MK+S3T83nuWVlvLi9TgXzcYgK5GOUC9W0R1rzTr493rEu+fFP3V3CAzM9rN15irtKXWxaOYf6QBedfREWlLrQJRS50nj9UCPVTZ9zOthNRWH2gEESii+Gwpw0fl3ZhMDY9NQlPPmVYuv2m11p7D0Z4JF5hdbvxMzS6850WpdTHTYWlBpT7ZeUuSmcaIxwe3xhCc8tK+OAN3gFX5ViNFCBfIxyoZr2SGveZsCPdy7MTLNzz/Q86/HxXwDVTSGeWTqFjxpDfOAL8oejzfSGoxz+pJ27Sly0xlrva/2dlNxoqB40ATlpaoDyF0H8VsWnZ7uxCcMr/F++Xsbb1S2s3+uz1CgNwR6+M+8mdGl0ct48MZ2MFDsLSl1EdIl7gpOMFDvfuN3DKyvn8qvH5jLnFhdP3V1ifU4eX1jCllhb/rNvVPPsG9UJz+dCyYLi6qDkh2MYM/gm+52M9Pb4+5gdmbX+UILE7Mc76jju7+QH904lqoNNg3W7vLgynDQEuylypdMQ7CbFrvH1L+Wzrao5YX0R+6e2PL8YnHaNcERPOOtZXOYmPzuVfz/ciC4NNdHR02cJRyUOm8Bu0yzvFFNSOHXSBE6cOcfyigJ+9q3bBxwnWV5oylABay0lQbz6KPnhOORCNe0L3W7e5+ePVPBubSsf+IKs31PPc8vKWL+nng98QeoDXdhtgnW7vFYQ7+2P0hDsJjvNTkOsfCIEA4I4GN2DKoiPPnlZKTjthv47fhPCrsEBb5DXD50P4vu9bQgh+OF905g6KdO670FfG+v31PPIvEI+DfawvKKAN6uaB62BD7YZvmHFLDasmDXsBrlibKC8VsYwyTXtO0tcAzLy4W43iQ/4axaX8vjCEjp7ItblwLk+/viRn5/uPEkkKq3T8MC5fuya4G/NHQnrJc92VFw+8e9pkSudYFe/sXuJ9Z9hMiYEfTHJp10T6FKS7rRZw63vm5FP+eRsS++9elEx6/fUW46Ft3kyWbvzlDVyLZ7kz4l5+2DXKcYWlxXIhRA/AR4A+gEfsEpK+floPLHrneTTWHNgrVkigfNTWsAYDDBUxpQc8DPT7AmXVy8qRpdGow9AhlMjcK6fsrwJnD7bTU/serNrcLAgroL75RH/3rWEeglHdXRpZOatHX04bIK5txgj2eD89Qd9QdYsLiUzzW7NyIwfo/bSXl9CI8/jC0uskWtg7ImY2vDBEgNgRMmC4upyWTVyIcRSYLeUMiKE+DGAlPKHF3qcqpFfGHM+ZnIG/tYxP+/Wtg4Z0OP/MM3HxAf4+DbsxxeWWLXQ/ohOX6yNW5dG00m600aJO4OPGo0/enMwb0SZq4waKTZBX1y7pic7lTMdvbHfgZGN35qfhfezc/RFdJx2DZuAL5e4eN8XpCes47AJUh02q5Y9GIPVwC90WdXIxx5D1cgvKyOXUu6Mu/gB8PeXs57iPIN10JmZ1gMzjT8o94SUAZucg2nD42+P6lgSs+me7Nh1EilhRkEWNc0daMI4jY/q0src0hyalZnHozLxy8MM4hWF2VQ3h/CHenFoAvcEpzX8ISfdSV/ECNg/uGcq7/uC/LUuwHPLyvjY38G2Kj9C6Lx1zD9kgB1uWAgM/JzML8nl/vJ86+f4NcwByoqxw6ipVoQQbwG/kVK+NsTtTwBPANx0002zTp8+PSrHvV5Zu/OEVbd8Zum0i358vJoFsIYpz/BkUevvID87FaddoyHYjU0TfHvO5CGHLSsuD01ARoqdNUtK+cmOE/RHJdM9WdQHzhGNeaUsryjgm7Mn8/J79TSe7abxbDcLpuTyUWOI2/Kz2O9to8Sdwa7vLxr2WJf7uVFcXS5ZtSKE+IsQomaQf9+Iu8/zQAR4fah1pJQvSylnSylnu93uS30dCi7c2DMU8U1E1U0hVi8q5u3qFqqbQmw91Mij8wq5f6aH55aV4Q/10hDsxq4JnDZhZH1J66mOzsvHNLYKR3X+WhcgxWFjeuzL9N4Z+aTYNe4qcbG9uoVf7qvnia8U81lnH0IIdtcFyEq1s9/bRopd47POvmE/C5f6uVGMfS4YyKWUX5NSzhjk3x8BhBD/CNwPPCqvhij9OiO+TvnM0mkXZfQf30RUPjnbkBqGo9T6O7jZlcbvjzZj02C6J9saw6ZLSVaqg+7+KLcXZieMZ5Og3A8vEwlM9xiSwYO+IF+7NY9Pz3bjtGtsq2pmzZJS/vPiUuw2I3DX+kNsWDHLGuLREDTuu3nVjL1mogAAIABJREFUHDasmDWkO+HlfG4UY5/L3ey8F1gL3C2lHLExtdrsvHSG2gRN3uQc6nGAVSfd8F49fRHdMkzSNHDaNL5c4uKD+rNEoro1jm1BaS6HG86SYte4aWIatf5Oa+0Mp42uJIMmhSEPHMnGsDPWxBOJ6oSjkueWlVEf6OKPH/nRpcQRa/Cp9YdYu/MUjy+8hfV7fYRjv5tUh8YrK+cMW7e+1M+NYmwxVGnlcgO5F0gBTHOGD6SUT13ocSqQX1le2uvDpsH6PfX8/JEKPvAFWbfbC5zf4FxeUcCOmhZrQ9NhE2gxzbJNE+i6tDI/gO/88pCaDDQM5sR6c/hxPGa3rF07P9d0cZmbvKxUWjt6qYxZyvaGdQ76gswvcbH18TuB8zVuMLo+NQGaENg0MaxqRXFt8IV0dkopS6WUhVLK22P/LhjEFSPjUkyxhqJ8cjbr99SzelExKzdX8os9RiCwCaht7mBBqYu3q/3ML3FZde9wVNIX0Vlc5uZf7puGENAX0dle7Wd7tV8F8QugS+iJBXHTuEoAyys8dPRGeHReIdMLsrklNwOHTVDZ0M4DMz18b2ExUV3y3sk2/tYcYs3iUurOdFqeOJsPNljr/eCeqbyyco419PqtY/6r9XIVVxnltTJGuZDO92J59o1q/viRn95wNOZimM5nnX30hqNICTfnptPQ1m0FCTNQPx/zZHnrmJ9fH25McNWTQzQHXc8sKHVZTTtwvuxkfkF+tczY6K9saE/wQ/nXHSeQEuw2Q6tv+qVUN4WwafBv755EE/BQRQGtHb3WGveX5/PATA9vHfNzsytDlUmucb4QHbnii+NCut+L5YGZHn5TafhzFOak0hDstixNb8xMsYK4GcDNn//Hn+tIT7EzpygHTcPywlYZ+UAEcCxps7GrP0qGQ6M3Kvn2nMnU+DuoD3TR0x+1ptav2+VFAGHdsEeY7snk+WW3Ud0U4vAnQd73BXFlOGnr6kcTsP9UkAVTXNYx4js5FdcnSnIwhhmJKdZIqfWHkNII0K2d/VagznDaOB3sJi8zBZsQcVNmBHlZKUQlSF3yvi9oBfFkslNtg99wnSGBzt4IABNSzr8nXWGdW1zpvFPTym35WaxZUorTrrG7LsDKzZX0xc6SwNggrfV3UusPUT452+rebO/uZ27RxJhZlrQy+h89XH4VXqlirKEC+RhmtHS/B31trN15iq+WuZlf4qI/Yvh4ZKfZCZzrB2DSDam4M1MsRz0JtHb0oQk41x+1NkEH046HepViJZlzfVFSYlJN48uzj9WLjCEQ6/fUc8/0SYDR7NMfMyp7flkZv3psLulOGy9sr+N3R5qw2zTSHBrhqGS/tw2bMCwSVs0vUlm4wkIF8jHKaOp+q5tCbFo5my+XuDjgDWKPpd2hHiN7lMCnwW78oV4yU+109YUx9050mfghuZYqKskDjC+GwR6qJX3L9UWNMXgOm8aaJaVEdfjRw+WsXlTMm1WGZt9kQamLxxcaA49/+Y+zme7JYltVM6vmF3HvjHxLxhiVxoapauhRxKMC+RhlMO+LeBe7wRhK6WKyfk89xe4MbNr5EsqC0lyKXOm0d4fJy0qhszdCVWPICODmxmfSccwvAtsg6bnG+On4DOvGa7iY52sThmQz+T1Jc2jc5EofcH+JMaKtPtDFU3cbJmXrdnmt/QaHTeC0a+z3BhN8wltCvaxZXMrGffVsq2q23vMUu8Zfjn/G6kXFqqFHYaEC+RjlqbtLBvWLHk6VMNz4t5ffq+f2wmy+PbeQiC5jm55pHPC20RDsZoYni9aOPtLj0lTXBOegx/lSQRae7FTiTPsSgv54ytqjI1TemMFel1Af6EoI/k6bobdvaOsmJz1x5J1DE3gDXbxZ1Ww14JS6M6wgnuqw8YN7prK4zM3anafYuM9nnYndWeIiEnuTvzI1l+eXleG0a0R1SX2g64Jf7IrrByU/vMaIH/+2cd8nPLN0imVXu2pzpeVzHd91aNPg1f+/vbOPjro+E/3nmZm8kJAghBgyEMUkQDTcYATBIixUWBDxBdxr28XrrWtF5LTL3eW0pwrn7Nmzntptu2vP8dhl1VtfLrVdd/cKbdUKKxarUFFiIIIGSFIwJAFDiAkQSDIz3/3j98JkmCEhmUzens85czKZ33fm9+SZyTPP7/k+L9+aw5NvfMaB+lay0pOtwQYReGzv1THgqUlWwYvTAdEpdBmuhGf1wMXOj05v8JzMFDwIX5y5QNBY4ZIz7QEqG84waewomts6WTA1m63ldeRnp/PEiulA1zbEz/3B6qcyt2A8//puNceazpFvG37Ho9dUw5GLph+OEMIzXVaW+tm0s8adBjOvMIsdlY22Abpokb5xcx4/eauSA/WtTPdnuhPXHUrzxnCDP5NX9tR2Kcd3BlEYe015bQvpyR7OdQyu4W+RBjgSxyDPnjyWD482x1zj83pI9Qmt9ubuX99WyHtHGimvbWFsWhKdQcONeZmcPRpg8fU5vHXgBOuXTKGm8RyzrxtH9ugUnn6nimSv8MSK6e4Vl+NZR16FXa6VsaKEo4Z8mBGZ6eLEUosmZFB2rNktyQ+GLAPn9Qgf1JzmT6fOcf+cPCaPT+fgG5V4BPLHp9PWEaS8toUb/JnkjR1FbfP5LudL9grGQHltC2NG+dwN1HiT4hPaA727euyJEfd5pIsR9wh4Rei0n+z0Av/hm5WAVR7/LzurCISsOZrXZKVxZ0lul6Ed982yro7WLSrk6R1VdAZDpCZ5SPJ2jWiqcVb6isbIhxHRMl027axhwdRsdlc3ETKGIyfPuutDBr42axK1p8+TmuRleYmfX+2p5baibJK81ubd7scXsagom1f21HYx4k6M2Of1MH/qeAT6zYg7Z8xIvXK/w/mAh+d1O6T6PIxO9VGaN8adfuQQMjAubI+gI2j4yVuHSE3ycltRNhiDPTqT7y8r4tffnucO7XDy7Z0N6prGc3QGQ1zoDPHI/Hx3oLFuVCrxQg35MCJapsvahfm8deAEK0v9bv8Un8fyMAFe/eg4982ayD03+vnOL8uZkXcV71Q2ckNuBvtqW3j+vWp2V18sOfd57PJ8rJh4IGjYVdXUZcMwM44FQl6xztkeCHGhs+f56s7mq8H6kJ9tDzLdn8m0CaPdNRcCIQqz01k6PZfvLp1Kir3Rm5OZYuV+t7aT7BV+uXoOG5cXWRujxjBmVJLbFTLZ56Gm8RxghUJWzy/oEhJx3oskr8e9SgJ0o1KJK2rIhxGRMVanEGj9kilMm5BJaZ7VxjYQgqk5GWxcXgTAf5ZZm29FEzLYUl5HktfKtFgwNZsn36gkNckyzF471uzzCNP9mRxtamP+lCw6g9asz5zMFAA3hhwPMlOT8Hg8lOaNIWRMl6yQtG4SwRcVZTM2PQkDjE1L4kB9K4dOnCU7zNPeV9vC+0ca+adth11j+2Vbp3vcY38jFPvH8LVZk8jPHs2W8nqSvMK62wpJ8Xl4vaIhpne9u/oU2w6e5NkHZnapB4DoMfDLEc9GasrwQg35EKC3/8AVx1tYv2QKm3bWUDJpDEW5maTYw3s/rW8F4L5Zk7ghN4MfvFHJnj+ddntjXzMujS3ldUwam0pzWycrS/08dkcRIWOFGaobz7JxeRF/OHKKkIHH7yhiz4bFrjGPRk/ztT0C98/JA6D5fCezJ49jy7fn8e2FhTTbRnbMKB9tUWaIdtVRE+c7gqQmeTh74WLYp/FsB0l2KOXa8Wm8X9VEIBji2QdmupPjnQrXJK+Hh1/ey5rNZeRnp3Pki7Nu2uAtBVk8+8BMgJidB3tTDxCLy6WXKiMbNeRDgJ7+A0ca/EcXFFDsH8PS4hzWbC5jS3kdyT4Pmx+ew4blRTz5RiW1p9sor23BZ7dCNVjDmD+1M1hqmy8w3Z/Ju4dPUdN4jpWlfsBqc3vmfABBKMhOp9g/ht3Vp+gMmqiFQhA7XzvyQxgysKfmNGB50h/UWMUyL+4+SrLdHbDlfOCSnO1IAiFDZ9DQETR0hgzT/Zm4A40Eiv2ZbrOwFPuq47f760n2ebh/Th5lx5pZfP3VtHUEKchOZ9POGv7ipom8/NBsN84N1pT5a7PSo8rQm3qAWIQ3Untq+yGdaK+4aB75ECE8PzxWJ8TLtb792TtV7Kpu6jJ0929fLWdLeb2b/x2efjfdn8mB+lbun5PHax/Xc+9Nfn6zvwGAv5o72Z1Qs+62Qm4pyGLN5jIA7p6R22VIc6pXuBBWOSR2+9twnMyRJI9gsAqRTra2s6gom/LaFpZNz+GXe2q5MW8MlSfO0B4IIULMJl7Oa6bYee5wMbUwNcnDsukT2FJuedBJXuHlh2YD8K2X9vKVgnE8PN/K43aGOBT7M8jOSHXzu8P1PRATdnSA8silXwZLKImjJ50QY3lsAJ+dONOl+dbu6lO8e/gUK0v9HGtqY15hFh8ebabYn4nXIxyot6YG/WBlCeuXTOFXH9Zy9kKAmyePJWOUj4DdR+RnO6v57f567izJpSMQ4pU9tQi4nnO4ER+bloQxXUv7nY3T++fk8fK3ZuPxCCdb25lXmMXN12XxzKpSXvu4nlsLs/ikrpWvFGTx+B1FpCX7SE++dFN1kd3v22Dlufs8Vgn8h0eb8Xkcb77TlSFkpxfOLRjP+iVT+GO1dSXgDHFITfLw+enzlxhxsMImkVdF/R2z1gHKSjTUkA8RevoPHGnwgUtSEtdsLmPN5jKeWVXKtAmZrJqTx66qJlaWTuTz021gLCO97eAJdlefothvGSsD/L6ykSffqGTVnDy3XPzX++q5a4afq0ZZoY6rM1K4b9YkvGFdpJw+LqNTvG7VozNBfro/k7ovL3CwvoWOQIicjGQ+bTjjzphcPf863q9q4q4ZueyrbaHYP4a7Z+RGnRP6+0MXR8d6xPK4rcwXsUMtId6pbMTn9ZDsFTweYc3mMnZXn2L1/AJ+/uAs1mwu46GXPgLghQdvjpkumOiYtQ5QVmKhoZUhwJVMC4oMwSwtzuGuGf4u6x5/rQKwOvE562+0M1o+OtpMwC7jd+Z1JnsFEaEjECRgFxIl+zxuV7+nd1Rxzbg0Pq1vZUXY7E+fx8qQcUInY9OSaG7rtNMfPXQEQm65vxPeSfF5uPemieRnp7sZN5t21rh/z9qF+Ty1/TDnO0NuxWa0ys1ifyY1jWfxXzWKL860s/j6q/nt/gZCxuARax7pSw9Z80fXbC7jzpJct7f3quc/YHe19cX206/f6Oo1Wml8T0Je8UIHKCsaWhnC9DTzIZrHtu3gyUte74f3lrhG67k/WLM8H56fzx+rT7NuUSHrl0xl0thRgLXx2R4IYYxhVLKP2ZPHEjJW2OKv5k5m9fwCFl9/NQfrW1lR6uenX7+Rr9iZH4HQxd7mgNVhMSOF7y8r4nxniMxRFzcrnYlFAJUNrTy9o4p7b/Lz1PYjrF2YT+PZdpYW57BpZw2ZqdbzvB4hNcmDz+O5pOCnvTNIIGhsI57DlvJ67pqRS97YNAIhw6ML8t2KyvDNyt3Vp6g8cYaVpRPZWl7XpSPhtoMnL/G24zn8ozviuXGqDC/UIx9G9MZjc4z/0uIc8rPT3VJyT5gHDpbRfGzZNJ7eUcWFziCddox8w/Iintp+hNun5/Du4VPuF8y2Aw1U1LUSDBmyM5JpPNPh9uYu9mew/3iLneLXdcals8maluzlnhv9rkxgZYccrG/hn7YdRmwv/HtLp/JmRQPltS2MTvFytj3oev5JXuFrsybx2sf13D49h63l9aQmeVg9Pz+q9/z4axW8XtHQZZbmD96opNifQUNLe4+ugDSLROlPYnnkasgVdlefYs3mMqszYshK13PCIsleq9+IsQuBRiV7KchOZ19tixvSCB/QvO3gSXeDdc3mMto6ggRDxl3rGOoUn4c7/oeVPeLEr5018wrHs//4l3TaaSlJ9iBiwP3SuWuGv8s5SiZmUnniLNeOG0XlybNMzkrD6xFqGs+xonQib392kkAw5A41dl4r3PBGGvLd1ad48MWP6AiEomaIxHtAtqJ0hxryEUS4Z+7cB1zP3PHSAXedk9IGFye/J3uFl+zUvPuf3+MOSahtPk9aspdpOaNpvRBg9nXjuhjw3+6vZ2u5lbL46kfH3Xa52aOTaTxrzQv9+s1Wwc9rH9fREQhxa6E1vcgaeiEsL7mYIugY0WhXHOHxfiedsihnNMdOn2dFqZ8LnSG2lNcxtyCL79xWGFUPsWLeL+4+CljpltG8bY1ZK4lGY+TDnPBiICeb4vn3rH7WTpZKyaQxXTIrwte9uPsoyXa1zLmOIJOz0twimbkF49mwvAgBapvPM3vyWP72z6ewr7aFkklXuUbciTlfm5XO+iVT+I+9VgHSutsK8XqExrMdZKb68Hk9rked7POwak4eZce+ZMPyIkYleyn2Z7Cl3CrMSU3y8OLuo+yuPhU1RuzE+59/r5qt5fWsLPXzxdkO1i+ZwusVDbz92UnW3VbotuYNjylHiy+Hx7w77WrP9UumuUVV4RkiJZPGXLJPoTFrZSCIiyEXke+KiBERvZ4cIMJT4SqOW0U0T75R6TaaCgRD/Oydqi6X/k5TrSffqOSacWl4xRpZlprkoelcB+sWFV5iqLwe+PBoM09tP8yK0olsKa/rssnnDEMAqyDH6xFe/ajWDa8YrLj2ms1lfNrQyrMPzCRvXDo/f3AWq+cXsG5RIQcbzpCW7OUvbprICw9ezCy5XD+Tp7YfYcPyIn769VKeWVXK0zuqCIYMd5bkXlGqnpPmObcgq0u7WeeLxynF1/J4ZTDR537kIpIH/Dnwed/FUXpLxfEWt/f4gqnj2Vpebw/wrWdeYRZ7jzW7lZ1gGdxHFxQQDOEa5NQkj2s4f/RWJU/vqOLZB2by/HvVbu740VNt7Kpu4nxniDc/aXDz2p0eJceazvF6RQPtgRDfWzqVVz743J0a9Jez8wgZeGr7YQIhww25mZf04t5V1cS8wiy3uhJg3aJCXq9ooOJ4S9TYszNcOjyr586SXAA3Oyc80ydW/Lq7mLeTT549+pBubCqDij7HyEXkP4EngF8Ds4wx3VYnaIw8/jhGZ8HUbLaU17mbik5MO9krPLqgwI37hm/oOYMnPqlr6fK4k1/d0HKBiVeluiX6i6+/mi3l9XgEfvHwHAC3RN/ZSFyzuYz2zqDb7jXF5yHZ52Hx9TlsLa/jq0XZvPDg7Jh/x0BsIPYk5q3l8cpA0i8xchG5G6gzxuzvwdpHRGSviOxtbGzsbrnSA5y4uFMSvnZhPlvL68gencyB+lZyMlOobT6Pz2M1uYqc7hNuJH+5+pYuFYwVx1tYt6jQyp2eOIbf7G8gEAxxZ0ku0yZksnF5EekpPtZsLuMDu1/5nSW5roe9+Poc14ivLPWT7PPQ1hFgS3kdK0onRjXiMLCNobrL09byeGWw0q1HLiJvAxOiHNoIbACWGGNaROQo6pEnFMcQr12Yz9M7qggEQ27HP6/dzbDYn8nnp9uYMekq3q865Ta5qjjewgc1TdxamMXq+Rc35x566UOaznXw/duLunj4OZkptHUEXY/dOf8z71SxO6IZlzPo2RiDxyMkeT3u+SeNHUVbR7Bb4zzYPF9NNVQGA732yI0xi40x0yNvQA1wHbDfNuKTgI9FJJrRV/oBx3vdtLOGa8alcd6eaO/0QLl/Th53zfCzblEhu6qsBlnOhJpHFxTwyJ/ls2lnTZdeIR8dbXYn3jgevtcDX7S2s25R4SVGq9JuxvX8e3+ypwlZIZlkn4fv3T6NaRMyuNAR5P2qU8wrHE9bR5Bl03P41kt7L7t5+Ys9n3NrQZabsRJ+bCAGKcSzr7iixJu45ZGrR55YwuO5jvfqFNTMLcjiq0XZbNppld87P4Ohi9ktkZ5leGUi4BYIBW0Pf2Wp363cjOaROhuiXy3KJiczlXy7f/ey6Tm8sqeWwux0/uesPLwe3I3TvHHpl6Tqhb+uIwcQs4hHUUYSmkc+zAjPAf/Fns+ZnJXmlrxXnjhDsX8Mz6wqZVdVE8+sKnVnSUZ6kuF509dPyHAfK5k4xu3lfWtBFu8ePsXahfnu8yI91GAIVs3J44/Vp8keneJ+edR9eYGNy4s43dZJW3uATTtr2LC8KKoRj3xdJ1ME4JmI1ElFUS6ilZ0JJp7VgI4X7FRFrpqTx+8OnHS98HCjF+u8Tll9eCWj09Hwgt14aoNdgn85QxqZNbOydCLvHm501/cl5j3Y4uWKMlCoRz5IiGcPaysH3M/7VU2sKPXzg5UlPLOqlGDo0int0c67ZnMZr1c0uN0Sn31gJoFgyC0QSk3ysmF5EZt21gCXn/zuFBdtLa9j9uSxbC2vY+3CfDcM09tsD80UUZTuUY98AIhXx7wrfZ2e9iqvON7CwfpW1wPuyRXDRY98PFvsUnknHBN+dXAl2R6aKaIoXVGPfBARjx7WvZkWE3neH95bcsm575rhp6HlQhcPuCf9Q5zK0ncPWymOjhF3YvS9yfbQTBFF6SHGmITfZs6caUYyu6oaTek/bDf/vK3SlP7DdrOrqvGKX2PTzqpLnrerqtFs2lnV6/M6x53HI3/vyd/Um+cqitIzgL0mik1VQ55gBsrg9eS8vflyiMdzFUXpGbEMucbIE8xA9bDW3tmKMvTRwRKKoihDHN3sVPqN8KEWDgNVSq8oIxE15EqfiWduvKIoV06fB0soSnjrWZ0mryiJRz1yJS7EIzdeUZTeoYZciQtaSq8oA4cacqXP9KbKVFGU+KGGXOkzWkqvKAOL5pEriqIMETSPXFEUZZiihlxRFGWIo4ZcURRliKOGfISiZfWKMnxQQz5C0bJ6RRk+aIn+CEXL6hVl+KAe+QhGy+oVZXighnwEo2X1ijI8UEM+QtGyekUZPvTZkIvIX4vIIRE5KCI/jodQSv+jZfWKMnzo02aniHwVuAcoMca0i8jV8RFL6W+izemcWzBe4+SKMgTpq0e+FvhHY0w7gDHmi76LpCiKolwJfTXkU4H5IrJHRN4VkZtjLRSRR0Rkr4jsbWxs7ONpFUVRFIduQysi8jYwIcqhjfbzxwK3ADcD/y4i+SZKS0VjzHPAc2B1P+yL0IqiKMpFujXkxpjFsY6JyFrgNdtwfygiIWA8oC63oihKguhraGUrcBuAiEwFkgHNX1MURUkgfS3RfwF4QUQOAB3AN6OFVRRFUZT+o0+G3BjTAfyvOMmiKIqi9IJhVdmprVkVRRmJDCtDrq1ZFUUZiQyrNrbamlVRlJHIsPLIQVuzKooy8hh2hlxbsyqKMtIYVoZcW7MqijISGVaGXFuzKooyEpGBqN+ZNWuW2bt3b8LPqyiKMpQRkTJjzKzIx4eVR64oijISUUOuKIoyxFFDriiKMsRRQ64oijLEUUOuKIoyxBmQrBURaQSO9cNLj2fw90NXGePDYJdxsMsHKmO8SKSM1xpjsiMfHBBD3l+IyN5oqTmDCZUxPgx2GQe7fKAyxovBIKOGVhRFUYY4asgVRVGGOMPNkD830AL0AJUxPgx2GQe7fKAyxosBl3FYxcgVRVFGIsPNI1cURRlxqCFXFEUZ4gwpQy4i94nIQREJicisiGOPi0iViBwSkaUxnj9ORP5LRI7YP8cmQOZXRWSffTsqIvtirDsqIp/Y6xLWGlJE/l5E6sJkvCPGuttt3VaJyGOJks8+909EpFJEKkRki4hcFWNdwnXYnV7E4mn7eIWI3JQIucLOnycivxeRz+z/nf8TZc1CEWkJ+wz8XSJltGW47Hs3CPQ4LUw/+0SkVUT+JmLNwOnRGDNkbsD1wDRgJzAr7PEbgP1ACnAdUA14ozz/x8Bj9v3HgB8lWP5/Bv4uxrGjwPgB0OnfA9/tZo3X1mk+kGzr+oYEyrgE8Nn3fxTrfUu0DnuiF+AO4HeAALcAexL8/uYCN9n3M4DDUWRcCLye6M/elbx3A63HKO/7CazinEGhxyHlkRtjPjPGHIpy6B7g34wx7caYPwFVwOwY6162778MrOgfSS9FRAT4GvCrRJ0zjswGqowxNcaYDuDfsHSZEIwx240xAfvXD4BJiTp3N/REL/cA/89YfABcJSK5iRLQGNNgjPnYvn8G+AyYmKjzx5EB1WMEi4BqY0x/VKf3iiFlyC/DRKA27PfjRP+w5hhjGsD6gANXJ0A2h/nASWPMkRjHDbBdRMpE5JEEygXwHfty9YUY4aae6jcRPITlmUUj0TrsiV4Gje5EZDJQCuyJcvgrIrJfRH4nIsUJFcyiu/du0OgR+AaxHbIB0aMvUSfqKSLyNjAhyqGNxphfx3palMcSllfZQ5n/kst747caY+pF5Grgv0Sk0hjzh/6WD9gEPIGlryewwj8PRb5ElOfGVb890aGIbAQCwCsxXqbfdBiDnuhlQD+brhAio4H/D/yNMaY14vDHWGGCs/YeyVZgSoJF7O69Gyx6TAbuBh6PcnjA9DjoDLkxZnEvnnYcyAv7fRJQH2XdSRHJNcY02JdlX/RGxki6k1lEfMC9wMzLvEa9/fMLEdmCddkeFyPUU52KyPPA61EO9VS/vaYHOvwmcCewyNgBySiv0W86jEFP9NLvuusOEUnCMuKvGGNeizwebtiNMW+KyL+IyHhjTMKaVfXgvRtwPdosAz42xpyMPDCQehwuoZXfAN8QkRQRuQ7rW/DDGOu+ad//JhDLw483i4FKY8zxaAdFJF1EMpz7WJt7BxIhWESccWWM834ETBGR62yP5BtYukwIInI78H3gbmNMW4w1A6HDnujlN8D/trMubgFanPBeIrD3Zn4OfGaMeSrGmgn2OkRkNpZdaEqgjD157wZUj2HEvLIeUD0OxA5rb29YhuY40A6cBLaFHduIlUFwCFgW9vj/xc5wAbKAHcAR++e4BMn9EvBoxGN+4E37fj5WxsN+4CBWOCFROt0MfAJUYP2z5EbKZ/9+B1bGQ3Ui5bPPXYUVH91n3/51sOgwml6AR53R1ND7AAAAd0lEQVT3Gysk8DP7+CeEZVslSL55WCGIijD93REh43dsne3H2kyem2AZo753g0mPtgxpWIZ5TNhjg0KPWqKvKIoyxBkuoRVFUZQRixpyRVGUIY4ackVRlCGOGnJFUZQhjhpyRVGUIY4ackVRlCGOGnJFUZQhzn8DY+w+bSRIuBwAAAAASUVORK5CYII=\n",
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAD4CAYAAADxeG0DAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOy9e3hT95nv+/ktSb7bipEVYxkH13LAAWricEsdKBQykA5JJ7Cn0zmkmUIupTmTwz5lT2d2kj/Pk2Q67TD78GSGpCSFmVzmdHeKMw00CRMSKMQJtxhcuxGO5TjYlnFk4cj4Lmmt88fSWkiyfAEM2PD7PE+eIFlaa+n2rne9v+/7fYWmaUgkEolk6qJc7wOQSCQSyZUhA7lEIpFMcWQgl0gkkimODOQSiUQyxZGBXCKRSKY41uux07y8PK24uPh67FoikUimLCdPnuzUNM2ZeP91CeTFxcWcOHHieuxaIpFIpixCiC+S3S9LKxKJRDLFkYFcIpFIpjgykEskEskURwZyiUQimeLIQC6RSCRTHBnIJRLJNeHFQ16qvZ1x91V7O3nxkPc6HdGNgwzkEonkmlA+w86Tb9SYwbza28mTb9RQPsN+nY9s6nNddOQSieTmo9KdxwsbKnjyjRq+v+Q2Xjt6lhc2VFDpzrvehzblkRm5RCK5ZlS68/j+ktvY/n4j319ymwziE4QM5BKJ5JpR7e3ktaNn2bKylNeOnh1WM5dcHjKQSySSa4JRE39hQwVbV882yywymF85MpBLJJJrQm1rMK4mbtTMa1uD1/nIpj7ieszsXLhwoSZNsyQSyWTnxUNeymfY42r51d5OaluD/Gi5+5ofjxDipKZpCxPvlxm5RCKRjMBUkUxK+aFEIpGMwFSRTMqMXCKRSEZhKkgmZSCXSCSSUZgKkkkZyCUSiWQEpopkUgZyiUQiGYGpIpmUgVwikcRxqS6FN7Kr4Y+Wu4fVxCvdeddFejgaMpBLJJI4LlVyN1UkejcysiFIIrlBuZJmFiMYj1dyd6mPl1wesiFIIrnJuJJM+VIldxMt0buRyzVXAxnIJZIblNhmlm37z/DkGzWsmZs/7HHJAuSlSu4mWqInyzWXhgzkEskNTGKm/MB815gB8lIld1dDopfsJCTLNSMzYYFcCGERQtQIIfZO1DYlEsmVYWTKlW4Hu6qbAcwA+eNfneLR3SeGBchLldxdLYmeLNeMn4nMyP878OkEbk8ikcQwnkAU+xgjU35iRQkzHRkAbNp1nHpfkOWznFTVtHHfvOnm8wx+tNxNbWswbl+V7jzKZ9iTBr3LkeiN9VpePORl52FvXLlm52HvJQXdxH2Uz7Cz+dWTPLWn1tzfjVKumZBALoSYAawFXp6I7UkkkuGMp24c+5ja1iBPrChhx8EmALasKkUR8PzvPFTVtDGvMIff/aGdza+epHyGPS6QJtvXo7tPYEmIGJeb0Y71Wo59HuC5fR6eWFHC1tWzeWJFCc/t83Ds88Bl78Ngb237DVeumaiM/H8BfwuoIz1ACPFDIcQJIcQJv98/QbuVSKYul3qpP566sRG8n3yjhr7BMDsONvHEihIAdhxsYsHMaagaKALq2rox5Mf1vmBcIE22r62rb2fHwaYJWYAc67Xk56SRZlPYfqCRbfvPsP1AI2k2hfyctCvax0sPL2BTZfGkNsC6HK44kAsh7ge+1DTt5GiP0zTtF5qmLdQ0baHT6bzS3UokU57LUWaMVTcun2Fnx8Emls/KY/v7jSyflWdm5N+el8+Rxk7K8rNQo+0jqgb33nEr2/Z/NuykkLivx5e5L3sBMtlJC+CO6dlJX8vz68t5ZeMiQhGV7e83EoqovLJxEc+vLx9zX6O9X8CkN8C6HCbCj/we4DtCiD8F0oAcIcRrmqZ9fwK2LZHcECRrzgFYMzf/khtvYgPR3W7HsOBrlCEWF+fyZo2Pp9eWAfDcPg9LSx0caQxgVQRhVSOialTV+FhXUUilO49qbydvnfaZ23u3vmPYvozAuGVlqbnvZK/PqEU/v77cPGk9saKEiHqxXg2wZWUpOw9/Tp0vyGPLSsxt1PuCDIVHvMgfF7Hv167qZnZVN/PSwwuodOdxt9txw5RXrjiQa5r2FPAUgBBiBfA3MohLJPEYgcwIGrGSPWdW6rDAmIzY54wUiKq9new42MQcVw7HmrtYWupgx8Em1szNZ2WZkwMeP4qAsKpRUWSnpkVXllTVtNHc2cOZjh40TUMIgUURbFlVyoeNAbNc88SKEl47epZ1FYXsPPy5GdyN12fo1EucmeytbTf/3eTv5c4iO8/t8/BghYt//qARwAyq2elWnt3n4eOm87z8g4XU+4I8u88DwFxXDk3+Hja/etJ8/Hg6VBPfL3/PoHlMEK+umeqBfEJb9GMC+f2jPU626EtuRpK1sQOjZuSxma7xb4Bf/L6JH35Tr30bMj+LAh82Brin1MH2A430D0UIqxoPLSmiPTjABx4/QujllHmuHOp93WxYUkRVjY++oYi5z1Srwvq7ClEE7PnEx/q7XHH/N2rlxkJq7Mlp86snGQhFCEe0uCsBm0WQarNw7x23UlXjo9Lt4MmEE9fOw16e/50Hq0UhHFFRNXgmuo2fv9uAIuDBikJTCz9aJv3iIS9fBHp5YL4r7iT31mkfMx2Zk870arxckxZ9TdMOjhXEJZKblWT12rEaaWLr6EbwefKNGhSBWZr40XI3FkUPmMG+IXYcbGLLqlLSUyxYFHj9aAsHz/hJsSpkplpZWuqgztfNyjInRdMyWTN3etxxDoZVPvIGeONoC5XuaWbw/vXJNkqcmWbwNmrmxomk0p3HpspiQhENq0Xwj/sb+Mf9DdgsgqGIxr135HOooZMtK0vxnLvAW6d9ca/18WVuKt0OhsJ6EF9XUchcl17zv6fUAcAXgT7zPQNGXBgun2Hn3foO87ZxEn1gvmvKBvHRkKZZEsk1IjEjXzM3Py5jfPGQF4sCERUz2BhZ5Lv1HXx/yW3sPPw5W1ffzlyXnY2/PI7VIlgzN599fziHJVoymZWfTXtwgBc2VPDrE61U1bQBkGZT+B+rZ7HjYBPfnpfPG0dbmOnIoDnQh1WBkcrRz6wto8nfy3+cbCUU0bjH7WDZLCfHPg+Qn5NmLkDuPOzl5+82sGBmLp+c7WIgpJr7/fa8At6saWPDkiKKpmXG1ciNcsnOw16e3ecx6/cpVoVUq8JLDy8A4JHdxxkIqWxZWTqu+vaNaOQlTbMkkqvIWFLCZG3ssRkjXFScGOWT2CzSyOTvmzfdVKEsvd1B31CEqhofmqaxdfUsBFDv6zYz/vc+7SDFIgAYCqts29/Amrn5qJpee24O9AFgURTuLEqulvmjr5s9n7QRiujBtbYtSMv5Xj7w+Hmzpo1qbyc7D3t5bp8HRcC3ypyImOerqkaaTWHDkiLeONqCRdGz95ceXsD95QVml+mz+zxkpFj4t0cXs66ikKGwymDM2cVmUUizKfzicBObXz05ZmCeCrM2JwoZyCWSCWAsKeF42thH0lbDRcncoQa/ufAYq+jQNPjZO2cIRTTWVRSyq7qZx/5Vv+rd/chi1lW4UDXoD6kc//w8ez5po87XjSJAoJdTTkcXPm3RwJ+ZYgGgqsbHYFglNZoh33tHPq8fbWFlmROrReGR3cf56dtnsFkE33Dr9XmrRWFlmZOKIjuhiMZvTrby29PtPL22jEj0sGtbg+ZJqqqmjbmuHH78J7fz1mkfhxr8rKsoBA1ePtxkasB/uKyEgZBKKDK2mmUyzdq82vYAMpBLJBPAWA0u421jj80iy6ZnU+8L8ujuE3Edjtv2f4bLnhaVEeqLnGFVYyiisWFJEf/0vTu5v7yAwZDKd+YXAHCooZN1FS4UoNHfa2a6igCjuKoBuek2whGNeYU59A5F4jLrRcW5bKosNrtCD38W4LZp6QyEVMKqhqrpjTz3lxfw0sML+OXGxVT99VIerChkKKLx9UI7jy9zm6/ZKK/sqm5my8pSzp7vY9v+BvbWtvPChgr+6Xt3svuRRXzkPW82NRmB2WZR4mSSiUy2WZtX281xInTkEslNxWgDG8qiDS6xUsKRpHKJ23nxkJeW873s+cRn6p6PfX6eW7NT2X5Al+vtONhEpXsaBzx6d7RFUVA1jUg0HH/cdJ4XD3l5fn05Jc5M/uGdM/z6RBu7H1kEwHuffknPQNgM3kZSLwAhoKs/RKkzk68X2vm0vZuIqgf726ZlcKQxwPGopPFIYwCbIqj3XUCJKmEUAQ/Md8VdZVR7O83s+p26c1R7O0ctcaiabiUQe+XyysaFvHXaF6eQMWrkD8xPvr3RroCuR4kl9kR/NWr2MiOXTGmuxiXrWNscKbuyKPCHtiBpNoVd1c1UeztHzbwSt9NyvpfXj7aw/i4XW1fPZsuqUiKqRqB3iIFQhOf2echJs3LA48eqCFaVORkMq4QiGsWOdKwKeP29tJzvpdrbSZO/F4HAahF87A3w5Bs1bFlVSlaalbyslLhj0YDpOWkoQs/Y99efQ4sGZ1WDe0od2BTBYFjl6OfnsSoQUjUE+t+XluaRarOw+dWTWBRdWbPzsNfUnh9q8LN19e1xr7e2NRjXMr+psphdmxaRWDWpdOcx05F5SQ6Lk3HW5tWs2UvVimRKk9j0kXj7am0zURERq6kGXRoYiqjYLIqpyhhpX4/uPsF986bzTt05U6u9YGYuHzcF+N6iGRz5LMAX5/sQXCyDPLSkiPc9fvwXBsysOsUiUDUtmh0LFAH/bcEMs+FoaamDk198xfq7XLx+tGXE11/qzKTR34sA0lMsrKvQH29VAHRFSaXbgdffQ0f3IIuLc/lDWzfr73JR5+tmTkEOAL8+0crs6dmcPd9HiTOT+8sLaPL3mjruREXOWFnqlYyumwxMhIpGqlYkNyRXYwDBeLaZmF1FVMyuRkNPPRBSuW1axqi+3JXuPHLSraal7LPrylkwM5cjjZ2kWRV+fbIN31f9wMUgDro2PNWqEFbBqkRVKRENNRrUjZq5ImBXdTNzXTkcaQyQkaLwv0+0YlUEFUV2c2HTIMOm0OjvRRGQYlWwKAJHZiopFkFYBU3T2LKylJqzXXzZPcjSUgefnP2KSvc03jjawjxXDm/W+PC0dxNWNep93YQiKnmZKTy7z0OJ82IQ3/zqSbMePp469lSeGnS1a/YyI5fcEGzbf8asTW9dPfuqbzNZdvXWaR+/+aSN7y4o5O26DtPz22VPY/lsZ5zhk5FJls+w89i/njA7K5eW5nGk8eKP28jCjRJHIobmuig3nZau/mH3g65CCUX0MohFEURUDbczk887e4kk2aaxT5tFMCs/i3rfhbjtPbSkiP84qWvTU6wK35lfwBtRFUu19zxuZyZ1vm5zO8ZzH1pSxJ5PfNw3T28MMnT0gJlVj5VhT1Vt+ERdTciMXHJDkThAQff/cLHz8OcTkuWMJl0bKbsqcWYi0LPlOQXZvPdpB6lWBV9wgP88dbGL8ak9tWYt+dHdJ/jxn9zOQ0uKAMwgPi3TRlaqxczCVU3PlhMJq3rW3dLVjxKTXCsCiqPDJELRzFxDD9LuaOkkWRAH/XHzXDmEIpq5mGkE4mfWlvH60RYWFeeSYlW4v7yAommZbFhSRLX3PF8vzKHO121ux2VPI6xqWBVBcV4mJc4Mqmp8uOxpzHRkApjrCy8e8o5Zx56q2vCrXbOXgVwyJTEus+MX1DqHLahdDmNdBicqImpbg9xZZKfJ38v6uwoRwJHGABei6pBn1pYxOz+Lv3rlGNv2nzGNm/bWtrP+LhfbDzSy55O2OKnfzGkZ9AxG4o6rLxS/CpimF63NTN24uC52ZDAU0WgO9JnbVIReSgmpGo3+3jHfAyMYG9tfWurgt6fb+cgbMMs0myqLTWfDt+s6yM9J5VhzF/Z0XQwnAF9wgJw0K2FV46dvn+Hs+X7ys1Op83Wzt9ZnfnbbDzTyRWDs45pM2vDJhCytSKYssQuFhxr8cYuTl7MAZlz+GiWP2G0Z9yfbplHvBV1r/YHHb2bS81w5WC2CUy1B5rn0bLXS7eBbZU5++vYZQM+eh5Kkxy57Gr7gwLiOPSXqZ5JiEfzkvtn8w9tnCEUjfH5OKh3dg6M+P3YhNZH8nFT6hiIMhVUiqkZ6ioVNlcWmzQDAsc/P4405QeSkWegeuHgiMsorS0vzOPnFefqjJ6XZ07PwfaW/RsNp8YffLInLXg073FizLMBcKJ0q5ZWJYKTSigzkkinNRNXGDZ+TWK3yzsNetu3/jFc2LhwzUOgnleNmgErEqHHbLHqNWhGxCpOLWfXSUgfnggOjZs2J9XJnVgr+niFAD5hGDdzYhpGVj/VLXxW1uTWeY5RihNAz+rCqkW5T2Lp6Fk3Rbe+tbTdb6Uc7GQhgZl4GzZ192CwCRQizKSnFqvCTNbNGdVQEuL+8wKypxwb0qaJamQhkjVxywzGRl9mGz4nR/v7jX53iuX0etq6+fdggYmPfsVr12tZg3BiyFKtCTtrFfjtzIo+qoWmYC4dbVpbGBeUjjQFSbZZRjzVx0dPfM0Ruhg2Bvl0NXQtuKFI0xg7iAswgnpmixHV7qtHjNTxbfv5uA2/WtNHRPcCi4lymZaZgU0TSfaRaFdZVFKIBzZ26OVcoorG01BHnAfOzd86wZm5+0ilELz28gJceXsC79R2mHt4I9MbVUiwT2fo+VZCBXDIluVI5V2LTjzFZ56dvn+HWrBSqatp4sKKQx5e5OfZ5gI2/PM5Te2rNJp/Nr57ki0CvGTQsCjQH+nTZnkUQDqt0D4RJtV6sfFsUiGjELWD++7GzgB5IbRaBVRHUx9Snx0tXXyiuxg6YSpWxUBIe1DukkpthG/a4oYhGf9TnJKxqfOTVOz0fWVqMxZJ8T4oA/4WL5aEZuRlm5i+EwBltTBqKaHR0D5ja8gJ7qjmqzpBvliUZCzeVJYkTiSytSKYkl1vPNhip6cdlT6PO101ZfhZf9gzxxIoSfvZOgz6QQdUQQpfwCSG4u2Qax5u7WFScy/HmLiKqFtVdp5iugjZFmLXqRIwSiT3NigrkZ6fGlVRiSygKySebG7Vx0I9LjWbk48US3Ufic0aSO8b+3WZR+PMFhbzv8RPsDzEUjsRZ4SqCqOJFP1HdE5VWVhTZ+WP7BcKqGtfFWVFkp9Hfa5ZqYodf/PZ0OxFV486iW/CcuzBqc9aNXDOXpRXJDYUh54rNyGJvj5WRJWv6+fa8fOp93RQ7MvB09OCyp/HsPg9zXdmmBM/ISvuHInzcdB64aBT18g8Wcu8dt9Ic6CPNpuDItBFSNWyJKW8UI1AGB8JmNyVcbPDRRgnixiZjF0mNE8l4MH74kYQgPteVEzecebRjvz0/i9ePtnAuOMBgKAIJ+b9ektH/LQQUTUvnoSVF1LQEuf3WLDOIz3XlkG5T+ENb98UgHpUx3pqdyutHWxgKq1gUwZMrS3liRQmP7j5hZuFTVZI4kchALpnSXElnZ2wAmFOQw55PfHyrzMlDd9+GInQJXn5OKjUtQQZCalzA02vHGi89vIDn15ebzT7v1HUwz5XDQEjlfG8IRTAsI09LogevibGQ/btvzzZ13/Z0a9JMfKRsOaxqlDozx3ztc1w5w+6b58rhs44LeDp6zPtGupoAqGvrxmVPM2vwAl1pk4zyGXZ+c7KNqhqfOaHIqggsAhrOXeC+edMJqxrT7WksLc2jrq2b2dOz6LgwiEA/SRkDJnYcbDLXLkBKEkG6H0puAEaa7D4W+kCEz81uyoeWFLG23MXGXx4z1SSGbC9ZOItEg9zOw1721rbTcr7fVLg8U1XL60dbSKxcKgJzck5st6SBVRH86ljLxWy9Pzz+NyLKperEk90Xe4UwGu3BAR5aUsSvT7YRjqh0XBgulxTA2vICpmWm8IHHz5HGgDnNyKpARNOoqvGxrsLF3tPtfBHoY2lpHh82dlKWn4WnowcN+NgbGFY6GW0gdWzZzWAqebNcCjIjl0wZRnIlfGpP7SVnZEYA2Lr6dv7Y3s1D0ek1z/3uU1OPXXhL+qjbCEU0Nuw8yrP7PLQE+qgosvPy4SY27jrGs+vKcWanDHtObNw2Gmdi6Q+p4wqgE0V2mnVYBm84ICbrJDWI9iJxa04qRdMy+dE3S1A1fUxdYnnHogi2H2ikIJq9g17SMernqqarWxo6eswrgJNfnGfDkiI8HT2mpDJZ6WQ0u9qbaSFULnZKpgzJFigT5z6O5X64cdcx7il1EFExs7Vnqmp542gLeVE99tJSB119Q9T7LiQ9DotgWHt7bAb70JIi3qk7R6A3NLFvwAQzXn15Ikazj6EbL3Vm8kWgb9QyjMueRmfPEKFIfIlKEbBitpN5Ljvb329kriuHFIugzteNRdH15rPzs6jzdaNpuhvjaG6SidxoC6GyIUhyQzDWAGPjMUb9NPHS2ih5rCxz8tiyEl4+3MQHHj+3Rrsfc9Ks9A1FhpU8LpfRhhobjKZsuRrYFF1xktjyfykoQl/ATFSdnD3fZ57AinLTafuq/6LyJokSJrYZ6PtLbmNXdTOgD7Ko93WbypWn15Yx16Vf8XzkPT+uJi2Dq2Godr2QqhXJNeFqzSY0thtbD18+y8lMR6b5g9646xg7D180XjKcBf/sn4+Yx/F2XQcPLSniA4+fh185yvvRIQ0d3YMoAroHwmYQL3VmDtNYj4U15gl6g87Yz7mWQVzf33DflvEQ+15oWnwQT7MpnGoJxg1Lbova7xpPi32Zhk3uUFjlH945Y/YDvPTwAiJR+9vZ07Op83WzsszJ48v0mvapliBbV9/OL37fNK7v2c2yECoDuWRCuVp1yViTLN3psJA3a9qwxHyDFQHP7vOw87D+Y673BekbilDXFoxTtKga5GWlmGPMItGr0jsKLio5nFkptH01QOEtyVUYIxGbyV/7a92rh5FNG0E59vwmwFT19AxGsCq61YCq6QE/2fuQZrPwkzWzsFnEsAVhiyKY68rhzLkLLC118L7Hz49/VWN+fo8vc/PDb5aM+T2bbHM7ryaytCKZcCaqLpno4bzzsJfn9nmY48qhPTiQ1JfD8PZeXJzLseYuUq16u/lQWGXLylLudjt4+OWjRDTdsOnMOV1ql+yyv9iRYTb2jOYjcjNjT7cOU9YY76VhEmaQahUMhjUsApbPdgKYtgYzHZmUz7Dzi983cU+pwyy1vHb0LHMKsjnSGBhWGhnrezbVJwolQ5ZWJNeMiWrQSMzum/y9WC16C3vZ9GzmuuzD5jbenp+FAI41d2FVBH+zZhZq1LP7Xw562bTruLlQ+VlHj1kuiA3iqVFJhhHEQa8H34wklpYSbycG8TSr0KWbQEO0qQr0k+JgWKPYkUFEg7OBPj7w+ClxZppWuE++UWMG8TVz87nb7eCJFSV82BhgXYWLXdXNphMijP09m4xzO68WMpBLJpyJqksmNvv85pM2rIpgy8pS/tAWZPOrJ6n3XWwKefKNGvIyU2ImxGv8/N0GM0MMqxqhiEqpMxOrogecZOXpiBpfP64osse1yt9MxL4/liRXLYkMhDWcWSmoQG6GjVnTs1lV5qSje5B1FS6+CPQxz5VDc6CPp9eWseNgU1zZK6LCCxsqeGC+i82vnmT7gUaeXltGWtRIbG9t+7CBIjd6/Xs8yIYgyYQyWoPG5WTmsVmXzSKwWi3c7XZwt9vBo7uP89w+Dw9WuPjF75v49rx8c5alqmmEIhqDYRWbIki36SWWUFilOdCHOkpJMXGB8nRrEKui13UThz3cTIw0USgWw/vcqgg6Lgxy7y1pvF3XEaMyEVTVtLEuakh2oT8c18gV+x25v7yAvbXtXOgP8259h9nZaVyBTeT3bKpzxQmGEKJICPGBEOJTIUS9EOK/T8SBSaYmozVoXA5G1lXpdpBms7BlVSlPvlHDx94AVovCrdmpVNX4+P6S26jzdZORYuFv1szCZb/YzBNSNbaunoXbmUVEY0RpYWLZIM2qmNl4WIXemySIX6pSJ5aO7kFSrQrfXTiDh5YU8e/HWnhiRYm5hnGowc+6ikLeqTvHI7uP8dLvm1hXUWhm1NXeTv7sn4/w1J5anl9fzqbKYtMF8Re/bwL0konxPYOLI+JG+55dLTXVZOGKFzuFEAVAgaZpnwghsoGTwIOapv1xpOfIxU7JeBjJodAYaryuopB9te0g9Lp2iTOTea4cqmp8DIQiZhnAGI6garquO6LqC5fJGnskV45VEXxzVh6/b+hk+aw8bs1J44/t3TT5e/nO/ALavhrgnlIHz+7zoAjITLWyZVUp2w80Eo5qGq0WhS2rStlxsInls/J4s8bHhiVFvF3XMez7MJ4sfKTv0lTL4K9ZQ5AQ4j+BFzRN+6+RHiMDuWQkYpUGxr/h4uX0u3Xt/LH9An/69QKqatqoKLJT19YNQvc+MYJ3uk2h0u0whyUYjGXPKrkyDHWPReiL1TUtQTJSLNxdMo0jn3USimg8vbaMJn8vb9a08Q23g+PNXYQi+hg5IQS7Ny2i3heMls0KOdTgNxVKxv8vRxF1I3R5XhPVihCiGKgAjib52w+FECeEECf8fn/inyUSIF6pYqgLDH2wRdEbQhShN6CsKnNS0xJEQx+gEJuB3zdvOseau8wyQWaKRffrjgniN6kQ5api6Pojmu7oqAjoG4rQMxAmFNFIsylmzfuVjYv45cbFbKosZiCkEopo/Cg6rzOiwoMVhVTVtPH9JbeZk4MiKpetiLqR7W4nLJALIbKA3wD/t6Zpw6zVNE37haZpCzVNW+h0Oidqt5IbDKPW+ejuE/z4V6fiLn+b/L2UODMRQl8w+9AbIMWijxgz6t5WRWC1CKpqfAyGVTJTrcyenkXvUARN0+K7E6/PS7yhSKynh9V4K9s/u7PQ1PQvKs7l8WUlbH9fN9ACPUveVd2MzaJb2r4U7dgsn2HnUIM/TpFi+M1frlLlRla5TEhpRQhhA/YC72qatm2sx8vSys3HpTZn/PhXp6iqaWNpqYPXHrvbvCz+9rx8/j3G5nVpqYMjjQFAb07pH4rEDVtYVebkSGOAUEQ1n5OVenOrTyaa2MapWKwKWBSFwbDK4ugUpTSbwn3zpvNmjQ8h9BOvMXFJ0zSzPBZrjmV89skawEbOchgAACAASURBVGSNXGciVCsCeAX4dDxBXHJzcimt+4a6wQjS33/5Y/OH/OuTbWZJRBGYQVwIuNAfNoO4MXPygMdPqlVgi+nll0H8yomd6dkc6MOdZJjF1wvtDIZVMlIs/MncfNJsCgMhlTmuHJ5eW4aq6ROOQhF9stHW1bNItVmY6cjg/vKCYcqnDxsDl62Immg11WRjIlQrS4HDwB+4OJHqaU3TfjfSc2RGfnMynsWm2EzpF79v4vTZr+jqD1GWn0VbcIC+wTAR7aJe2WBVmZNDDZ2EVQ2Lont8xNbDrYrAZhH0X4Hjn0THIowJSfptRcC3ZjvNheVYe9xVZU4eXVZitt5/5A2ganB3iYMjn/nNE/Hi4lw+PXeB+8sLzGlLl8qN2JKfyFXLyDVNO6JpmtA0rVzTtDuj/40YxCU3L+NZbIrNnIJ9Q3T165aono4eMlIsRKLGTbpeWQ8Z03NSWfQ1B99dOCM6hGC4MiWsajKITwBKVLJpvL/FjgysijCDuM0iyEqzsmFJEaBn65XuPArsaWw/0Mhjy0rYvWkxFgVzUpBVERxr7mIwFOGB+a7LPrabaZBEIjdbx7HkOjLaYlNiw0a1txPPuQtxi2lGBm6zCKyKbsA0e3o257oHo7LEbmbkjj7VR3JlJJ4gg32huDWJJ5a7eenhBaZdsO+rAbbtP8Pe2nbzMdXeTrbtbwCI1sf1+4cimmm5cDlcyfzWqY50P5RcE6q9nTy6+wRbV9/O48vcZrZ0Z5HddMDbW9vOouJcPvIGcN2STmtXP+GIOqxpRxF66eTBCheHGjqZlmG7puPRJPGkWARDEd0Qq6N7kFc26lf+ta1B+gYvtuAbbfRl07M5+UUXf76gkN980sZASGVdhYva1iAt5/vZ/ciiuJmcl1oauZEGSSQi3Q8l15XaVn0gwLb9n/HI7mMAprPdmzVteNq7GQyrHPmsk/6QitffO2wsmIGqwZ1Fdr67sMgM4qPNl5RcXYYiGqvKnHQPhPmGe5o5fs+QCq6rcPEvB73U+4J8f8ltVHsDbP5mCe3BAVQNtqws5VBDJ3+5uIhUm8Jbp33A5ZVGbmSJ4WjIjFxyyVzJotLOw16e3echxaqQatXbsLftb6A/pA7zr45ltL8Z2CyCkOy5v+akWRUGwqo5Pu+xfz1BgT2Nrr6QKRk0DM0yUiw8tvRrvHzkc/qGIjyztizuCm0iOjenusRwNGRGLrlkRjIa+iLQe0mLSi8e8vLUnlqqvZ08vszNuopChsIqPQNhfvr2GawWhaWledT5upMaNq0qc9LwZc+wae+xuOxpMohPELEfQVaqZdTHOrNSGAirCAGe9gv89B0PFkXQ2tXP8llOU/e9ttxFRoqFcPQzCkc0MlIszHXp3xmjvn0lnZs3usRwNGRGLhmR0TIcwJQS7jz8Od9wT+OxZSXUtgbNgP7WaR8zHZlYFPjZuw1mBr79QCO9g2Fz4WxdRSHvfdpB/yhDj5MNgJBcXcY7FSnWvybdpvDKxkV87A3E1amNq7jY++92O4Zdxd0IfihXE5mRSy6Z0VQAsVLC++ZN53hzF5tfPYlFgc2vnuSxfz3B3tp2LArsONjET9bMIhxReXafh/4h3ZnQCM5VNW3kZ6eSnmIhLVrrTkzMRxoCIbl6jPZ256RdzNRjP5f75hUADKtTG8E69n4gaRC/GWZsTjQykEtGZSTtd+yi0qEGP1tWlQLwj/sbGAyr9A1FuPeOfHNs11yXnbtuywV0TbdVEXwtT5/UI4BGfy+lzkwGolpvjSvzxZZcPmkjLBynWi5+ID2DkWGfz7qKQvae9rFp1/FhwXjnYe+YQfpmLo1cKXJCkGRUElUA/p5BSpyZbNv/mSklzE63sm3/ZyyYmcuRRv2Hubg4l6qaNvMSevOrJwlH1OiINY2wqsvVvrwwyEAoQlaqlZqWiz9YaTd7/RgIqUm92kOqRkp0QTnZZ/P7hi9RNQ1rzIDTel+Qomnpo7bXG/clWyhPnBokSY4M5BKTRDVKtbeTza+e5P7yAraunm0G5MGwyncXFLLjYBMfeQMcb+6i0j2NAx4/KRbdKvZYc5c5+SU73Uo4ojIQUnkwWg/vHQxzwONnVZmTg2f8dPWF4o5FBvHrS7J1Y1WDxV9z8HFTwFzLEOhqobCqEejVP8P1dxVS2xo0PcWfjipTIP47Zvx3o7XRXw9kaUViktjibOh5jbbpSnceLz28gP92VyFv13WwfFYe73v8ZlBOt+kjvsKqhkXAvj+0c2eRnZ+/20DZ9GzmuHKoqmnj3jvyWTHbiUXRTa0uV2wSe6kvmRisCREh8R0+0tjJ1wtzzG7MVJvC2vICVE1v13c7M3n9aAuvHG7i2ZggXu3t5Kk9tZeseJKMDxnIJSaJi5vGwNvYS9tKdx7Pry/n+0tuo6rGx+LiXDN7zs9J4z9OtlHqzCSiwdyCbABCEZXTrUE+6+hhXXRYgACKHSPLCcfDoJQbTjixg6dtyvAFT5siONUSJM2qsGVlKaqqUVXjY12Fi+6BMP/Pg/Moy8/C3zOERcBcl928sttb284D8103bRv91USWViRxxC5uGpPNEzHq5usqXLxZ49MXuWp9NAf6sCjgCw6QblM409GDqmkIoc/JnJWfxaEGvZxywOPHkWlLcgSSyUIyj7GQqg/neLCikEDvIEMRvW6eZrNEB4IcZyCkUpafhaejh7965RhWi24jHJsU3DE9e9h3TJZYLh+ZkUviGKvFObYD71BDJ0+vLWNfbbspJ4yo+mivNJuFvqEIAyEVTYOsVIU6XzcRVeWjpvOUOjPNmqpkamCNylRUDf7QFuT1oy2sKnPyk/tm0x4coN4XpD+kUuLM5J0fL2dpqYOwqjEQUrn3jvy4gP3J2S6simBXdTPV3k5ZYrlCZEYuMUlsALrb7YgzugJdInZnkZ29te28sKGCt07rk160iIbKRZ/w2MVLVYOeQRUBBPvDgB7spTJlcpKTZqF7IDKsIchY2AxFNOrauslKtXCkMUC1N8CDFYXsrW03bWyrvZ2cbg1GbYU13qxpY44rm7kuO5tfPYnVorB1td4c9sju48MydsmlIQP5FOFamOYn0/EaRldzXXZqW4NYFDje3GU+p6N7gLCqoqIPHOjsGRpx+7FBwRccAMbfPSi5dnQPREi1iOFrEAJuSbfhj37GAyHVVK8MhFQ+6+ghFNGY58oxjbP+7dHFADy6+zjP7fMwx5UDYAbtC/26O+Jdt+XKIH4FyNLKFOFqmebH+qn8aLmb2tYgOw97efGQF4DHl7n5hnsam3Yd58y5Czy3z8OWVaVsWVXKQzuP8vuGTlRVXwRz3ZJO5BJTbA1krXwSkhjEhYBQRMPfM0RmtGEo1k6hqqaNvqEIDy0pos7XTYkz0wzWle48Xtm4iDmuHOp93WyqLDZlh0YZz3PuguzgvAJkIJ8iXC3T/MQThEWB5/Z5MEZcVns7Od7chaZpVNW08WBFIdsPNPKP+xtQFF0/PNORQUjVaOnqv6xjkLXyyYMtiaRTESBi4vpAJPmkJUXAnk/aaPL3MqcgZ9jfz57vo9Lt4LWjZ8fV6SkZP7K0MoUYj6LkcrZp/IgMo6Kn15ax42ATF/rDesYUNbpSFJW369qJqPrA3DSbwjdKHGY3ZzIULg5yTYYsrUwOjE7OZA6SWal6zTw3w0ZXXwjDZy/xs1M16A+pPLN6FnNd9jiDNaPU8uRK3crBWHsZrdNTMn5kRj5FePGQl52HvXGKktgSyJVsF+KtQ+e67KY8bPmsPHYcbOKlhxfww2UlDIRUQhENW1RStnz2xR9dsvacsaZkyiB+/UmzKUmnMBl0D0QodmQghKCiyI6m6dbCI/GRVx+ovGZuPk++UcM/v98IkFBqWUhiYl/pzpPSw8tEZuSTlMTFTYsCz+7zsLLMydbVs8lOt5rtz1dC+Qy7mS1tWVnKrupmXj7yORZFsGVlKTsPf87W1bcDsKu6mRSrwlBYZa4rh7+9r4yHXj4KDPdGSbpYJpmUDMQIxp1ZKfh7hlC1i/8W6EOUt6wsJSPVyp+WF9Dk70URAo2LvitWRaAIONXyFZtfPclLDy/AmZXK9vcbWVdRGJdtSw+ViUVm5JOUxNp1k7+XjBQLx5u72Lb/DDsONvH02rJhWc2V0jsYpm8owpZVuo/0KxsX8rN3zvDwK8e4v7yA3ZsW8czaMk61BHl6zx/My+zENc5hi2UTe5iSq0Buho3zfSFs0XTc3zPEvKjKZK4rh9eOnsWi6N/Fd+s7+J9/OtvUlluVaH1dCM73hghHVF4+3MSu6mbWVRTyZk0bFkVfc7nSq0jJcGQgn6Qka5d/+QcL2VRZbJZAHl/mvuJL0drWIC89vMDcrj1dV5AYl8f1viBDEY2MFIUH5usDcue67CwtddAc6CM3wzauIH1LhlSmTEZiJwDp2n+NUPSsnGHTm7hWljlp8vdSUWTnuX0eOroHeGFDBR95A0Q03b42PcVKiTMrerWWzTfcDj7w+BkIRXjv0w6eXlvG9gONbH71pGz6uQrIQD6JSfQCh+GG/VdKouF/OGpV+oHHz1+8WM1z+zyk2xS2rLqdR3ef4J26dh7dfZwPGwMI9B//eAooie6GkslBz2Ak7rZxhSfQfVdWlTmp9p5nwcxbOODxs2FJEYu/5qDeF+QDj5//+e3Z/NP37mTLqlL+6OtmXYWL9uAg+TlpPFjhIhTRCEVULkQbwe4vL5AllauADOSTmFid7a7qZja/enLC5VqJU1leengBqTYLGroVrRDwysZFPL7Mzfq7XJxq0duwNeRC5Y1E4lWVIuAn983ilY2LuTUnlSONAZaWOtjziY++QX3Waokz03Q2NEp9s6fn8MKGCvbWtvPep1+yJapS2f5+I5sqi3l+ffm1f3E3ATKQT1ISA+z95QVxf5+o6SnJujm/M//ivlQN9tX6qPZ28tvT7aaaQda8py7JPrvEk7IQsP1AIzsPezkX7cI90hhgwcxctr/faPYPGN3FL2yo4PFlbspn2E374/vLC7jb7cBmUUizKaavimTikcOXJykT1ZI/2nYA82/G4/bV+nj9aAvpNiU68UevlVsVSLFasCgC1y1pnDnXk3R/aTYlTgUhmVwYWnADRUCKNflnVurMpNHfS4pVAU1jKGYBO92msHX1LHYcbBo2nHvN3HzTwz5WS/7WaR/v1ndI29orYKThyzKQ3+AkGmHF3gaGNW30DoYRAr5eaMfr78WRmUJzoA/Qf/TlM+ycakl+FWBIEGUwnxok+5wUATaLwmDUmNwa7d5dV+Hit6d9RFQoyk3np39ebrpg7jjYlHTq/bXwB7rZuKqBXAhxH/D/AhbgZU3T/n60x8tAfm0xgnfij+3FQ15zyv0d07P55GwXoYjGjNw0mgP9Sd0JZSfmjYEQEPvT1+WDSvRvgluzU8zvwJPfKuVfDnoJqxqzp2dz5twFnllbZhqp9Q2GzW7jratnX6dXdHMwUiC/4hq5EMIC/DPwbWAO8H8IIeZc6XYlE0ei+sXIkMpn2NlxsInls/L4MColEwKaA/3kZ6cmtZjV0A2y0keYtJ4xwv2SyYHx6WxYXGSaX6VYBGEVyqZns3X1LO4umYYvOIhVia6R/EG3ZUixCHxf9fPQkiKe2+eh3hekfIZ9wpVUkktnIn51i4FGTdOaNE0bAv4/4M8mYLuSKLEOhQZP7anlqT21cfeN1Gwx0rCI2tYg356Xz5vRkW1DYb393p5upePCIGKEFc1pWSlxznex9MmSyqQi8SNUgeK8DH59opXekIoj08ZQRKM4L4NTLUF+9s4Zjjd3sbRUX6RMtSp4/b3MdeWQarNwf3kBz64r5+m1ZeytbZfGV5OEiQjkhUBLzO3W6H1xCCF+KIQ4IYQ44ff7J2C3Nw/JLGz31razt7Z9TFvbRPVL7I/NosAbR1u4pzRPlxpGnxPsD2NVBCNV3Tq6B5OaK1mllGXSYXxKxY4MQL+aau7sYyiiUerM1IO4I4Pmzj7uLLIT0WAgFKHaG8BqUdi1aRHrKlzURe1nDfng48vcfHtewTDF00QoqSSXzhXXyIUQ3wXWaJr2WPT2w8BiTdP+r5GeI2vkl06yOjeQtPYNFxeaaluDfBHoBaDEmUlE1U8MP33HQ89AmJw0K6daghTmptMataEtdmSYC5zjIStVoWdQZuKTkVj3SWN6U06ahZ7BCKlWBWt0Ms++Wh//+0Qr95cXUFWjywcr3Q6+VeaMW8x8YkUJERW5WHmduGo1cvQMvCjm9gzANwHbveFJVjIZqTySrM49Uu0bLmbxXwR6KXFm8mZNm+kzXu8LUt/Wjdffi+fcBea4cswgrgho7dJr5OOlb0gG8clK7CfT0T1IRZGdCwMRbpuWgapddCR8dl05f3vfbPbVniPNpuu+T37RxbP7PDyxooStq2fzxIqSOK96yeRhIj6S48DtQoivCSFSgL8EfjsB272hMRQjsSWTnYe9PLr7RFIvimR17tEGJde2BnliRQl7a9v5x/0NCCGwWQR///YZnt3nIcWq4HZmomrw2Ze6JlwRMH+GnYiq0XFhENDVDGMh525ODVz2NH5yXxlzXDk0B/oovTXLLINUezvZfqCRVJvCLzcu4pcbF6FpkGpV2H6g8aoatUmunImSH/4p8L/Q5Ye/1DTt2dEeL0sr8dPoDeXImzU+nl5bZg46TnxsrBbcsJ41Mqqn9tSyt7bdvG085rZpGdT7urFZ9Jq3sUi5rsLFHFcOz+7zADDPlUOdrzvpscohyTcGhs2s1aIQjugL2xuWFFE0LROALwK9PDDfFTft/q3TPs4G+vjQG5DywknA1SytoGna7zRNm6VpmnusIC7RMRaGdhxsotSZSVWNjwcrXMOCOMAvft/EEytKqG0NUu3tpNKdx63ZqdwaLX+8eMjLA/NdhCMqP33HYz4vHFH5o6+bxcW5hCIaYVXDquiZ+Zs1PvbVtpMSHe01UhCX3DiEVb07czAcIdVmYcOSIt442oJF0Wvez68vjyvPVbrzeGC+i0/PXZDywkmOrHZdRyrdeSyf5eRYcxeLi3M51NCZ9Ifyw2/qWbtRitl52MuXFwZpDw7E2YJaLQpN/l627T/D5ldPEtHgnlIHx2Km3isClt2eh80iONUSxHVLuukpPRIyG5/aJH66ERXs6TberusYtVQymuJJMrmQLfrXkZ2HvTy3z8ODFYUcavCbZZZkXhTV3k4e3X2CBTNv4cPGAA9WuHjv0y+JqBp3Ft2C59wF1szNZyCkUlXTRrEjg3tKHbx+tMX8IWenWRgMa2gazHVl0x4c4Fy3XguXHZs3NsnKY0tLHbz22N3m7cT2edliP/kYqbQiR71dJ6q9nWzb/5lZE4+tmScbQFvpzuO+edOpqmlj9vRsqmp8rKso5O26dqqj9cu6qEf0vMIc6tq6afuq33QrVDV99mJFkZ2O7kFOtwZRNaOrT5NZ9w1MTpo+PNk4WadaFSKqxpHGAM9U1VI0LdO0ajBkrUbWnex7KA2vJh+ytHKdqG0N8srGhWZN3KiZj6TRfWpPLe992sHSUgdnzl2gLD+Lqpo2Iqpm+pV/5A2QZlP4ItBHfnYqoYiefRtBep4rh/r2C5zrHjDvy0y1xgXxkbo5JZODMapgJoZVgs0iCEX03gAN3elw/V2F/Nuji6kosvP60RaOfObnuajM0Fg4T5zkI0e0TW5kIL9O/Gi5O2m2kyyIG52cEVXjdGuQpaUOPB26ZNCqCO52O7i/vACrRbcW7R+KmPJBDb1sUpyXQZ2vm8XF0+ICt2FpagQITZNe45OZ2M/OkIZaFGF2bipCH5rcF1JxZKZgVRTW3+Wio3uQe9wO0PTGsEp3HlV/vZR1FS6ONAaY48phx8Emtu0/w97a9rh9jtQ1LJk8yNLKFMCYq/nWaR//ecrHiS/0yT0C2Lp6FrWtQZ5fX06JM5O9te0oMQVRi4CIBs2dfZQ6MznS2Ik9zUpwIBy3D1XT27dDssYyadGza/3zsSr6KDajW/Ps+YuduP6eIZxZKXT2DLGyzMkbR1t4OupW+PLhJp6LSk7nuuy89+mXpNkUzp7v49478k0Xw7vdjhG7hiWTD7nYOYWo9nbyV68cI6xqzHXl0OTvwWpR2LKqlCZ/L2/WtBGKaCgCcwiAoR02bmfYFGlsNQVJtQgGE/xtjCBuYFgrpFgEP7lvNtv2N9AfUllV5uTWnDRzqEO9L8jP321ARL3HX3p4AfW+YNzC+wsbKvjYG5D2tJOMq6ojl1wb3jrtQ6DXOZv8emllMBTh798+w69PtBJWNWwWwR2uHJ5ZW8a6ChdhVeP2/CxWljlJs8YHcZtFFlGmColBPDfDFhfEnVkpfBHo46ElRex+ZDERVZ+1mm5TaPyyN24yz+PL3CyYmctASGVTZTGA2bXZ1TfEEytK2PzqSXZVN5vrL4lOm5LJhQzkU4AXD3nZedjLmzU+/mLRDFKtCqoGAyGVoYhGRNUQAv58wQy2rp6FIzOFvbXtHGroZF2FiyZ/HwIYCKtkpliA+Mt0ydQjdlxbfk4qnT1DbFhSxNt1HcDFNZjHl5Xwxfm+OC+eam8nnpgmn7dO+8yZmz/8ZgnbDzQSjqjmzE0gzmlTMvmQNfJJjuHJsm3/Z6y/y8XbdR0sLs7lgCfeCjgc0fC0d7PnkzYGwyopFsHS2/No6uyl8JY0Dnj8rCpz8uiyEn7wyjEZxKcAxvrGaCgCvuoL8fTaMnYc1DuA3zrto7Z1+NAHIyjH2j0YtXBjxmalO4/7ywvYW9uOMyuVJ9+o4aWHFwAklcVKJgcyI79GJDodGll2rKSr2tvJxl3H4h5XPsPO9gONfMM9jbfrOphTkM0Bjz9OJmjog2taggyGVUqdmVgUwcdN52k4dwGvv5eKIjuLvuag3heUC5qTlFhpYX5O6rgatFQNNE1jrsvOCxsqaPLrZRSjC9joylwzN5/Nr540s2/Dj6e2NTjMQ/z59eVsqiwe5rQZq6i6FOdOydVHBvJrROJwCItCnCWoIfFShD4EOfZHEo6oHGkMRFUnAZTovEVF6NPMbQkWhY3+XoYiGpqm8WBFIfOL7NS0BNl72sdP3z6Dy56W9BjHq1GWXB1iz68d3YPj+jwU9IXsR3cf52NvgDdrfMycls7e2va4gN3RPcBQdKBy7BBuo3MzNkiP5qppkGzYiZQoXj9kaeUaYTT8xEq6jMvhC/1hXjt6ljVz8ylxZnK8uYvNr55kU2UxOw83oWqwuHiaLh1MtxLsD1PsyCDQO8R35hfw+tGWYfuLqBqaRVesGNlWna8be5oFX3Ag6THKRH1yER5DXPTQkiKqanwMhiMMhlW2v9/I0lIHHzbqjWGgN5L95ykfFkXwN2tmseNgEwOhU7xTd45XNi5MagWRrPSSKD9M9n2WEsXrh8zIryGJgyAeX+aOu/3AfBc7DjaxZVUpoYj+w+wPqdEfZydLS/MI9oepKLLTPRCOC+Iue5rpZAjRcosGrx9tibtEDw5Eru2LllwVFAG/Pd3Oj//kdpbPcqJpMCM3nQ8bA2xYUoTVovDI7uP85pM2+oYibFlVyuPL3Cyf5aSqpo375k1PGnSNUst4xreNNthEcm2RGfk1JPGSNTvdymtHz3KP28Gu6mbudjt4YUMFm189aV4GWxU40hgwXepmOtJ542gLG5YU0fbVANlpVnoHw3T1DWG16IO9hiIaGjAY3YamQZpVYWCEFM+ZlYK/Z+gavQsSA4FuiXCpV0Kmt3xE5SNvgFMtQR6sKKSqpi3qv9PBvXfkU1XTBuje8zsONvFHXzdvRj16DjX4TUtkuGiQlVhiMQyykgXpxO/z3W6HDObXCZmRXyMSLUGNsVlPrCjhr1eWAnptvN4XpH8ojKrB7OnZWBQFRehdeD9a7jYnmO/5xEcorBKOqFgVhW+4HfzZnS6+u3BGUlvakYI4IIP4dUJjeBCP/eQqiuzD6uRLSx38+YIZfHfhDGZNzybQq+u+DzX42bKylPc+7WDmtHR+94d2c2Tbe59+yZyCHNPz/p++d+cwS9pLrXlLi9vJhQzk14jES9aIipllV7rzTInXv3zgJazC0tI8/BcG+Zs1s7BaFN46rY9BffGQl7kuO48v+xofegNowHcXFvKR9zy1rUFeP9qCIuTC5WQnO+3ixfD0nIvzUTX0Zh+jTKZqmD4quRk2PmwMUOLM5Pn15fzdfWU0+XvZfqCRNXPzyU63Eo6o1LQEUQT8j9WzWFdRGHU61Etzhud9Yskktua9bf+ZpHXxWC6lBCO5+sgW/UnEj39VQ1WNz/SJjrW2NVwRq72dbNp1HEVAxW25nGg+TyiiMSM3nZboAGWLAn+xsAhPezc1LfKHNRkpdmRw9nwfmqYHb2NyU4kzi8++7EGg9wb85eIZ/PZ0u37lZVH4zvwC9nziMxcqjRF/995xK2/W+PhWmdPcx6kWfW7r9gONLCrOZfHXHGbmPVKQ3rb/jGzLn8RIP/JJTrW3k3fqOlhX4YrLmtbMzafJ38vz68vNx2qaRn9Yw/dVP0MRDQVo6eo3FS0RFd73fEkgoWSSlWqhZzAih0hMApoDfVgVgVAwx/B9c1Yejy0rAWDjL4+TalM4Fe0N2L1pEXDR/thoznl+fTnOrFS2v99ofndiVSSG4Vps0DbuT6ZYkTXvqYksrVwDxmqeMDLvWdOzmOPKias3KgJ+dbyFnYf1x7512keqzcK8Qn0SOoAKZKZY6O4PY1H0OmtH96A5aNmgZ/CiYmWs8W6Sq4/ujaNQ6XaQkWLhI2/AHKq9+5FFfC0vk3pfN2u/Pt0sWRgLj8bV2VN7as3ge6ihk+WznHEqkvHaJcua99RGBvJrwFgLSUa98f7yAp7b56Hep99++XATbxxt4VuznTy3z8OPf3WKd+s7+M78AurbuinKTTf30TsUQUQbhcbKtjUYFuQlV59Yj7KsVN3zxqIInlxZyss/WIjVorCosgYgGwAAIABJREFUOJcn36jh1yda+aOv28yyjU7Np/bUUu3Vr9g2v3qSvbXtPLGihIxUK0+sKOHNqHLlUgcly5r31EbWyK8RRvAeq3nCmOO5qDiX481d5ii4H//qFFU1bZTlZ3Gmo4cNS4rY84mPnHRrnAueZPJjUQQRVTO9bwyJnyH3O3Pugikl/Kfv3Rm3VrL9QCOhiIrNonB/eQElzkzTY8X4f0RlzFq4ZGoibWyvM+Ntnnh8mZtFxbkca+5iRm46c112qr2dHGrws7g4F09HD7fmpPJ2XQdbV99Od38o6XYk14506/h/Rpkp+rxMlz3NXIg2yhyV7jzKZ9h5p+7cMK23MQZwU2UxAyGVUETFmZVqztmMqJgOhkY5RWbUNw9ysfMaMdpCUuy08p2HvRxv7qIsPwtPRw+bdh0nxaoPj9h+oBGbRdDRPUhFkZ2PvAGsFoVVbscwN8RYkk1Ql0wMioDByHCNvk2B2PkdxkSf3iEViwI//4v5QLyjoJF5G4qUxHZ50MsrW1aW8ovDTaa6ZKSByHJQ8s2DDOTXgLH8K8pn2Hl09wkq3dN43+M3faVXlTk54PGjqhr/uL8Bm0XhXx9ZTL0vyM/ebQDguwsK+e3p9lH3L4P41cNmUcwOWtAXkcOqFhfEc9KsdEdH6wlAVaHeF+TxZfELkWPVqY3vDMCu6mbz/1JdIpE18mtAbMZtENv+DHpt/Nl9Hua5cvAFB8ya57fn5bOv9hxf9YfitL3V3k7eOu3jP062YlEEA3J826RgaamDj5sCpuFVTpqF7hh/m3UVLua4cnhun4dvlTn55cbF49qu8R2C+ID+1mlf3PQfyY2N1JFfRxKlXjD8svfxZW7+6OumqsbHjNx0th9o5P7yAtqDA0Q0jXUVhew8/DnZ6Vaa/L3MdOjdfS3n+zjSGBh1/1I3PvHorjY6M3LTae3qZ/b0LNNm2KB7IEK6TaE/pFLsyOCdug7muHJIsykEesdvjWB8h1485B2WtT8wv1MOfbjJkYF8kqAvaHayOLrQmRK1oH3f48ci4L1PO5g9PYtn93mwKYK87FQsChz7vIvcDFvc6K9Y5LDlq4OK3kEbUaG1q5+lpQ6ONAb00okGxXkZBHqGGAhF6A+pVEQ94ZeWOnhun8dUI10q40kKJDcfV6RaEUL8TAjhEULUCiGqhBC3TNSB3QwYjUJGDT03w8bp1iD52akMRTR+dbyV3AwbEQ0uDITNWmlI1RDAs/s8aGhU3HbLiIOUZRC/fDJso/88/nJRERkpFhQBHzYGcNnT0IhOs+/sY3FxLgLBnUV21swrYF1FIUcaAzxYUUiS9VGJ5LK5UvnhfwHzNE0rBxqAp678kG4eDK3vW6d9PLGihNaufgbDKq5b0shIsRBRNbr6QqTFGCgZGMMhFCH4wOOXMzgnEOOcONZJ8AOPn5d/sJCvz7DjdmbS1RfioSVFdHQPMs+VwwGPn5/cN4s3/3op5TPspkPhoQa/nKQjmVCuKJBrmrZf07Rw9ObHwIwrP6Qbh7Fa8w1Vwrv1HRz0+EmxKqwqc1LTEkSge4qnWRUGEoK0MyvF/PdgWI2rf8vG+ysnoul6b4A0mxLnJmmM1yt1ZvLlBb0Ra05BDi1d/XzDPY215S7um5dPna+bYkcGTf5entpTm7T93ejSjEXOvZRcDhPZEPQI8PZIfxRC/FAIcUIIccLvH1nzfCMxHo9no1HoQ2+ATZXFvLJxMUW56fQO6dlgMh/xkfzDM2zKiCUWyaXRO6TizEohFNFItSqsmO1kZZmT7y3Sp+8s+to0vrtwBm+d9vHAfBcpVoWPm87zV68cparGh80i+PLCIHtrdWloMlkhMKFzL+VA5JuXMeWHQoj3gOlJ/vSMpmn/GX3MM8BCYL02Dj3jZJcfjkcuOF5Ga81/8ZAXiwI7DjaZf68osnPA4x8mWxuP8kQ3zBLSR+UKMN5nAVgtAmdWKv6eQdJsFtNFsNrbyS9+34Qi4Hhzl+kl//2Xj5rlr3UVLt779EuAYe6DsYzXumE8JPYrJGsokkxtLlt+qGnavWNs+AfA/cCq8QTxqUCiT0XsDyIZYwV+ozXf6MIzsCjEKRjqfUEOePyUOjPx+nuZ58qhztcNgM2qmOPfRkJViRZkJJeLhj7/9NacVE61BPEFB7AqUOLMjBvCAJgj+Tb+8jiLvzbNDOKKgKoaH1tWlnK32zGqNDDWuiHx+3GpyIHINy9Xqlq5D/g74DuapvVNzCFdfy51WspoJZTE1vzYS19jStCOg01s23+GQw2dVBTZaQ708fTaMp5ee4cpQzSCeGx93GBpqYNSZ6YM4ROELzjAqZZgjNWv4O/uK4v7nGtbg3xnfgGDYd335Eij/rkWOzJQNd0Ya1d1M2+d9g0rlSRaGL929CyV0bmtsd+PyymLyIHINydXWiN/AcgG/ksIcUoI8eIEHNOk4FJ+ECMFfiDpIpfxY/3RcjePL7uYsf+fK9ysmVfAvz26mLkuPWjs/v/bO/fgqMs033/evuR+MXQygQ7RmMQQDRMNCCgXYcUFHXRGOFtnZnE4og4iZzycHc7Mzg5UbW2dLZ2dszVOleUsIl44g3qOu7XCjjKOHHVgEJSbkRgkQDoEQzrEJMQkkFun+z1//C75dac76ZCQdML7qUolnd+vu99+u/v5Pe/zfp/neXwufz0313yu6RmJ2C0ZJynxNj6ubuHrju5rNBMTn0i7BvaQT39oiXaJJMFpIzFOKzlrfZ9PX+zgzcN15LmSgi6g3m+6SHTaiLML5uRl8G5FA+t3Ho94kTc+H09b+rZaJanDjZcP5jgoJi8jVa0USilzpZR36D9PjdbAxpvhfiHCGX5r7QzDs7LWzghtDPD64a/MEE3FhTaWl2QDUP9NN1tWFLNlRTEnLrTht8TAL/donnp7tx93esK1mIpxZ6Q9MEJXKgsLXdgEQVrume40vneHO+g8fwAemDmNbWtmmxdg433eVV5Pibu/uQdoF4xev2TTsiI2LSsiILX4+IOl08Ku7qyfD2vf1t9+VH1VsW3VHOL6RdVaCcPVbBoNtWkV7jGNbjDb1sym4kIbdhs8t/csm5bdwrpFBWw/4DFv+wOaB/nMnioA4uyCW7JTOOntuPYTMokIt2lstMgzsAmtGJaUkl6/5JF5uawodVNxoY3S6ems33mcG6ckcdLbblaWNH47bIJ7ijL5vC64AFa0vTBH0jNzNDfpFbFJpM1OZcjDMNwvRLSGP9TYLy/J5qHb3UH3eWBmNm8dvcD350znvcpGHpiZzRuH65iZk8aphnbiHXZ+tPBmXvxzjRk3n+xp+HahabuvFfGO/gqGM91pnL+kedkblxbyh4oGyuva2LKimBJ3unnxfbB0mtnc2kjPd9gEASlx2G3seGzOgPK0Q21AjqaCRTE5UY0lhkG0fQ4Nom2TFRp++eWq0gH3ea+ykekZibxxuI7bpqXyXmUjM3PSqKxvR0p4+dE7SU10mEbcJoaXhp84RNp5LOKXwW3SrpaMJGfQbRuwZUUxff5+I151sYONSwvZtmY2/gDs+vFCtqwo5rm9Z3nho2pAW0E9dLub85e6eGReLsfPf8OWFcU47ML0zg2iDXeosIhiJEy8b/U4MFSixXAa3A4WdzcMfW1LJzYBH1e38K2UOCrr281iTP/yJ63cbWqCQwsTDNNT7YoRzz1umJZ5NDzy1k4f2Wnx5mZxANh/uomAhLLcdB683c3vnpjL1n01QH+BqnWLCli36GYO6Ulb1vh27pRkXll7JyXudLORstNu450TXiD6i7zqmakYCZPKkF+rzLZoMjSHItTjWl6SHaRmAK0m+fYD51hZloOUWjy3qvEyNgEpCQ59Cd9MniuJy9193J4b/Px2EVmhkZ0WP+B/o5UE6ryK3chev8RhA6ddMJJFQllu+gBP24rDJijOTjFvp8Q7SIqzk+C04bBhFrHa9eOFEVukhbsAW7vZg6ZO2rZmNm+uu4tta2bz/snGoPOshLvID3cVqFBYmVSGfDQMbjiGqysPR6jH9dDtmkLC8NyMpsurZrnZf0brEmQ4oQEJhVnJHKxuoTg7hdqWTu4tzuJUQwc2yzvol5GzP40GzVbjPVpxZ59FRTOcEgF9AchKiSfSIsFq4K0hoekZiebfX9S3RyzhC/DtnDRqmjtNL7y2+Qrb1szmfywrwm6zkeC08cGpxqALqtWARhPyUN60YryZdJudo71hZN34NBQFK8vczJiaNmJvyTrW7QfOmeqUI+da+MSjNYuYdWMGh89pHWeMuuMLC10crG7hBv220djAYLDNwbLcdCoutF2TzcMEp23ITkVOmwgy/JEwelxa+406bHpylF/itAsCAYlf9rdXs97PSpzDRrxDK3R1sqGDny0vMpsWv3PCS2N7d5DKxLqxHbrxbZRV8AcIMvZKGaIYC66bzc7RzmwzvPztBzy8fvgrVpa52V3u5ci5lhGHcaxjvX/mVLMDOmix7LsLXLyx7i7uKcoCtBjvjKmpHKxuMcum5rmSaNKr8BkMZqTL666NEQcGNeJG+MVqxBMs3ecFwauFvoCmJjHi1wD+gDQNttNuIy8zmTi7IDXBYT6G8fCJTpsZcuntC1Cak86uHy9kx2NzOFjdYhrth25383ldGxuW5FNxoW3AKi405FE6PZ2t+2rM46O16lMoRsKkM+Sjndk2vyCTDUvyeXZPFYuLsth/ppnNK4o5Wts6IGPviR3HBmQLDmbcjbGWuNPYXV7P4zuOcMjTTMuVXhw2rVnBT976nI/Ptpghi9MXO4h32KhuuoLTJlhQ6Apq/huKTWgGcSyThdITgkv4pCU46AvIoPIC8Q4RVNlRol2AZuakmf/r6QvgsAkykuOIswtTFTLTnUafX9La2UuvX/KNHlqRBGu627p8poLks69azffqrnyXabSNMMjWfTWcvtjOEzuODbqKq7igGX1rmM24CCgU48WkavU2VLf6q8UfQNsQK69n472FrFtUQE3TFRrbu3n6zXJunZpKRX0bq2a5eW7vWUrcmndmbYw72FgBnthxlI+qmjhwthmhP2fxtBR2ldeT50qisb0bh03g80vTAGalxlPpbacwK5nqpisRx+/zB2i63BPx+GjT1t0XdDsrNZ6CrGTK6/qNXU9fv2duDQV5vr5Mdlq8GdPvC0g+qmrCoYczFhZmcrC6mRJdkgmaAc/L1LrygGbEtYuAdp9H5uXy+xMNPLHjKA67zcygtH42Fhdlsqvcy8oy96CfFWOFtrgoSw+z5ZhhGoVivJhUHvm12HQyYqJGd5fXD3/F9gOah/15XRuLizI56Gmh2+fn9yca2LTsFtbvPM7jO47ybkVDUIq+4Z0bMVWrJ/dwWQ5Ou2aoe/2SErdmqGZMTeW8nga+aVlR0Ni8bd1U1rdR3XTFbEsWutVohBqMDkJ5rqSrnouhECFPnpag1SjxNF0JMuJWUuLt+KX2G7TxN7b3DFCi9AXgNncqr/9oHg+X5VBZ305yXP99DCMO2l6CTWj3WVCYydufebnv1mwCUkvkMVLijY3Ln7z1Obt1I77/TPOgqzhjhba7vJ65eRnsLq9nw5J8lbijGFcmlSEfqYQrnHyx7tIVntlTxYYl+WxaNsMMs+RnJetfaC9z8zLo80v6/AE6uvrw+QN0+wKm5hj6PTm7DfP31n015u38rGRTWWETUOltpzg7hdMXO7jNnYYQguf2nsFpFyQ4tc076K8X0tUXMMMI1vAEBLeIi4arlSWG7psXZKVQmJUc9D9biETyco9WI+Zyj5+MJCedvgCFWcl80+kLSqwRQE3TFbYf8LD/TBMLC11c6fWT7LQFKXUcNs0bj3fYKHGncbC6mdk33cCu8nrW35PPL1eVmudqnngWu8rrebgsh998v2zIRJxDnma27qvh4TI3R2pbebjMzdZ9NSpxRzGuTCpDPlLCyRff/szLI/NyzVKzW/fVsHlFMTVNV/QvdI7+hc5BAs/r2X+hMXprLHZxUZYec89k674aNizJ5/kPq3HaNeNjxHlPN15mZVkOX13qxG7TwgyvrJ3Dq2vnEKo2khLuzneZ3qpNhDfI1iJPkZBgKeF69ZTXtQ0I+QRCJJIp8XaaL/diF9pmbm5GAl+1dmmqFNlv9J12gT8geWZPFTdNSeTEhTYcNrhi2WDVWrIJOnv9+PySry51snpeLgerW8hzJbHtz8EGd/sBD++c8LKyLIf9Z5qCYubWVZz1Am+spD449TXzC1zsP9OsYuSKcWdSxchHSrjC/K+svZP5BZm4kvuLGa1bpMnSNizJZ+u+GjbeW8hrh2qRuhTOabdxV4HLjNFba6oYKpUZU1PN5gM1urHbuLSQ5z+sNmt/lOWms6eigYCUSCQCwUlvG7XNV+jVQyWGJBHgaO0lhB7fcNpt9GmdJnDaBTOmppoxZdAUI/EO24B4tlG3JSAlcXZhPk8krNK/q6HbF9D2BCTkZiRS19pFdlo8nb1+piTbudjew8ycNM40XiYzOQ5vWzfeb7r1tHotmSgA3Do1lUpvO36JGWN3JTt5+zMv9xZncaC6BbvQysRuXFrIJ54W/lTVZDb1CN1fCVWqGMeMolmAWXp2sKYjCsVYoDzyEMLJF8MpYQwZ2gury7irwGXe/+cPzDDLngJBdckNCePCQhenL3awsDCT1w9/BWDW9niwdBoLCl0kxdlJjHPgsAuk1Iz4t3PSeHZPFW8crgM0g9Xa6WNpcRbxDhu9fklPX4BH5uWyoNCFP6ApPPwBSWV9uxl3TnDY6O4LDDDioNVtsdFfknUorEY8kg8/mG/fF5D4ApKy3HQutHbpG7s9ZKdqxtioM1P0rRRaO32sLHPT2NFDflYK9xRlkhBnZ+cTc9m84lbuLc7CBuZ5tS1dFGQl81FVEz9bXsQra+fQ5w/wq/dO8/HZFtOIG+97pP0U6wX+t5Z6K9ZYu/LIFeOJMuQhhBrt7Qc8YTP73jnhDaorvW3NbF57bA7+QLBRmF+QyS9XlZqx9dumaTpwrdhSKw/MzGZ3uZeTXi2hJD8rmU88l5h1YwaHPC0sL8kmKd6B0y44dbG/ZG1WShyN7T0sLc7ikOeSKU+cqYdm9p1uwpXs5MuGdjOU0drp0xUx/cbXYYM8VyJWlhRnkapvVEbClewMMtDxDhubVxTjCPOJGupyIICTDR2snpfL+ZZOstPiqW66wk2uJLzfdLOwMJNKbzsFWcn8sbKRlWVuvtRb4FkVKHcXuEiIs/OfZmnx7pVlOVR621lQqIWwPvW0INEuHk8tzjeNOBBUN8fAKh0NbZJt9dhVKr1ivJl0mZ0jIVw52id2HDPrg1vPiyaTz5oV+OJ+D6cvtrOr3EuJO5U9G+8x642vmuXm7c+8bFp2C1v31TAlyUl10xWyUuJovtzL5hXFfOntYFd5PdAf/lhY6OLLhg7c6QlUettNw3pHbjqV3nZ8folAy7gELclopjuNk952UhLs9PgCZCQ5aezoHdG82YUWfqhuukJvX4CfLi9i/+lms/3ZYKwsc7Or3IvDJkiMs3Pfrd9iV7mXrBQnTZd9A17jI/NyeWZlqVnS4GE9vm1cOI35Nt7LxUVZ/LHyIvfPnMqu8noSnDaeXJQ/IOvXON8Ig0FwyMSQkqoSs4rxRNUjj4LRLswfqhVfv/M4Pn8Ap65lDjY4mewu97JA10nb9VTzOLtACDEg6cdI0y/RGzQbBlqENDoArZmwt63bjEEbMjuj1rlBvMNGgkPQ1u0P+3qcejzc+MTYBfz4L7T9gR5fgNwpifxgbi4l7nTWvnaUPn9ggGIm0WkLqsC4xXKRynMl0d7dx7T0BL70tpuvbWFhJkdrL3HbtFTOX+oyjehP3vrc1PZbmzCEXpANo++0C+Kd9rA6cuN+oe+R8b4BA94zZcwVY40y5GNE6MXgkKeZH/3vY/T2BUiMC29EjBouc/MyOFLbysJCFycutOHzBzRZo24N4x02U5LnsNuYm5fBh1VNzHSn4W3rJi3BEaRKKXGn8XV7N02Xe0lLcNDR3ceCwkyOn29lfsEUPqxqMrXrCwszOXKuZdC4uNE5x6cb6KQ4Oy8/qn2m1u88zpy8DI7WtgJaka/yOq3rkb7napLnSqL+my4tDATUXeritmmpfFzdwtLiLI7UttLnD+CX4L4hgdrmTuLsgh2PzwUwPe/QmjrG/62eubHqubtgCtlpCTx0uzvo4hp6kTbeC6vnbt2sNlD1VRTjgTLk1xjDgMPAJfm/HbtAX0AGeY6GIejPFAz2yDevKKajq4/nP6o225PNL3CZSomXD9TwieeSGZZZNcsd5F0LML3zZKeNK74A7vQEGtq6ubc4iw+rmkx1x9y8DL6ob6PLF8AutIuEEP21U0rcWnciQw5YmptuxuKNzNWT3jYOVrcwLT2BdysamJIUR0NbF3abJhv0+TVPPtFp4+GyHPKzknn+Q12qubSQdysaEGhJVkZrtbWvHjHL3dptNl7Tu+784u0K3q1oCPKQjQtJaPGr4YTGrEXMXjpQQ7cvcFUt1xSKa0UkQ67kh6OEVaL2wuoyc4kugMQ4O4/Nz+P1w19xV4HL3CA1koHuyE3ng1Nfs3peLm8dvcDqebk8t/cMftkfQy7Ru9cA5v1/tEjLKMzLTObZPVWmwTfCKlIPsfgCmsLF29ZNdlo8hzyXKMtN19uUZXLiwjf06AlFf/edYqC/N6gQ2n1rmrSQiE1oypqt+2pYXpLNhiX5A0oRnG/p5JCnhTiHTVOl+KWuaxd0+QK8/Vk9cQ4bc/IyAEyDvm3NbE5623hu71k6e/3m6uC/LikMKrcQiey0BJaXJLB+53Fzvl9Zq33mX9zvMQ13qLwQBobBXjtUa/423jOFIlZRHvkoEs6jS3DaeHXtnAGxVcD0FmuarrC7vB6H3Rakcb4jN53zl7pMvbrxOzQ2u/a1I5xqaKexvYeFhS5m3ZhhJibdW5xFjy/AQU+LuYE4Y2oKpy9eNjcOf/F2Bf/xuRcpJXcXuPi8ro0Eh43Gjm7sQtOSJzptrJqlqUDOXLzMqllu/u1YPehFuTYuLTRfR0DC+nvy2brPgy8gzfDK5hXaReKf/3hGL4DVX8nQ8K6hP7zhtAs2LC4wQydG7fZfrioN2lv4Y2Wjqfc/5Gnm8R1HTW862no7g62ojIuUMuaK8ea6KWM7nlg16AAL9LZf1uNWWeIra+9k674aslLicejndXT18YnnEptXFLN85jReWF3GukUFvLC6DH+AsJrlBYUu3YhncuJCGy9/fI4Ep42kODvnWzqpqG9jZZmb5staGdzTFy+T50rimZVauvovV5Xy8qN38nBZDp94LrG4KIuOnj6S4hzckp0KQH5WCu9VNvLz+4vZtOwW3v7My9ybp9DbF6Cr18+v957h349foNunqVbu0qWAdtFfdGzrvhpK3OnseHwOd+TeQLdvYCmDQ55mXjtUS4LTRoLTzl0FLlPymZ+VbHbesRa6un9mdpCRddq1hhEvHahh/c7jURlho7yDtV6PIR1VOnFFrKM88lEkkuphMI/Q8D436rFv4+9o47KGZ2qk+ff0BejtC7BF936tao3v3j6NNw/XmY0RrAkx4cZT6W3jT1VN5ibmZr2TvBEWenZPFUVTUzmth3wcNsHPH5hhZr7abVrY5Ns56VRd7GDDknz8AczsSJ9eKCZ0riJJAIP3FLLYXV7Pw7oCx5p49cLqMj71tJiblsaKSKGY6KjNzhEQjSxxKB2y4dGFKlrW7zzOt3PS+aJeO27EdqNdylvH9sj2TznoaTE7GAHYbfBuRQNnLnYg0TYrjc3G5/aeNUMSxnie2HGM+2dO5Q9fNJj1XIxqjDVNlwlI+KneYccoOGXE5OMcNnY8NjCMZL1tXHCAAfK+B0unRaUOMWSHK8ty+M3374g49z+cdyOvHarlwdJpQcWyFIqJijLkIyCSUbIa2+EYe6uuHPprrABRefGDjTFSwoph5MMpZ55aXBBkaLfuq+GmKYmU17WZ8kSjpsq9xVnMvdllets9Pq1A1YJCF0fOtRLv1LxrqxTwvcoGHiydRok7nZf+XINNQOuVXtKT4tjx2FwOeZp554SXm1zJQ8r5rBcbIxHI2p7NuumsNN+KyYYy5CNktHqBGo9jNKOwGj0gyLBGq1MOldlZjbIRyhhq7NYLkbFS6Or10xeQ5uao4XFDv9rmaG0rG5cWBlVxtHrAho6+s9fPFj00Y9w2Nlutr2Ow1zxaF1SFYqKiDPlVEr75cg4zpqYOq875+ZYrZtjAeJwSdxoP3e6O6nEGM1CAWd/c2Jiru3QlKO0/1KhHSqAxHvOdE15O6jXRqxovMzcvg1MXO3iwdBo3uZIj3s/4f2jNEsN4G579I/Nyea+ycVieszLSiuuda6paEUL8VAghhRCTbu06sPlyDrvL6wf05hzqMd6taGD9zuNsP+DhtUO1xDlsnPS2R/044WqlGz1Cn1pcYCpb1u88zjsn6nnzcB2blt2CP4AZLjGM4PKSbN454Q1qdqF1NTrC02+WU3fpCl/qaf9VjZdZWOiiuukKG5cW8v7JRs63RG4rF475BZn8aOHNgFawam5eBs+sLDXVKEbvy6FWOSNtHKJQTFZGbMiFELnAXwJfjXw4sUdw8+VM9p/RalgPpyvM/IJMtq2ZTbfPzzN7qswONltWFPP8h9X84u2KqB4j1PAZ3rZ1HD5/gJPeDrNzTWdP3wDt+UO3u3n/ZCOgyRmf/7Cabp+fP1U1cdu0VN48XMfqebl4mq7wyLxcvmzoCNKyQ3+XI+MiZ70d2lH+kKeZlz8+B2jKliO1rWw/4AlbMngwwnVwGqy5tUJxvTAaHvlvgL9l6GqlExZNB61lWP5w3o2m9zscbfH8gkyKdE12b1+A0pz0Aecc8jSz9rUjEY1VqOEzxmEY9/U7j+O029h4byH7zzSbDYJDjaT1ovCppwWfP4DPL5mTl8HH1S0sKHSROyVaZpsLAAAMh0lEQVSZV9beyTMrtRK8B6tbTC27oa0O1+0o1KsOjZH/7om5JMXZeWZPFVt2VQyo8z4Y4VYl4S4cCsX1xogMuRDiu0C9lPJEFOc+KYQ4JoQ41tTUNJKnHXNKp6ez/0xzkMGJZklv9SAPeZr56lKnWa/703MtPLunio1LC4MyFRcUuiIaq3ANLqzG3ecPsG3NbDYtm8EduemmRM84N1x9bSN5aWWZm6N6wa6D1S3YbZix6637anjynvyg12zcf1d5PXPyMsyLXKhXXXGhjVuyU9iia9bnF2Ty8qN3ckduOm8dvTCgzvtQjY+HG45RKK4Hhqy1IoT4AJga5tAWYDOwLJonklK+BLwE2mbnMMY4roRuwjVd7mH9zuNBKeWRNtwMD9JQc/T5AyTGObh9ejof68by+Q+r6ejqC1KTlLgHqkwgWJJopJ5vWKJV6Jtf4DK16Ic8zRytbSUpzk6C02bGzqFf3mjNoBTAB6e+NkNGq+fl8uyeKr70tpvJNuFqk1j3DFaWuYNqyRg8tbhgwLzML8jk/pnT+Nv7i81zQ7NeI2G9AG28t1AZcYWCKDxyKeV9UsqZoT9ADXAzcEIIUQtMBz4TQoQz+hMWa8o2YCacGHU/BlveG8bpub1nuXFKkllL5cuGDlaWubHbbNw4JWlA+CNc7Dh0HEbs/rm9Z3lhdRlvrrvLbDH3zgkv29bM5uVH7+T9k4186mkBtISb+QWZfO+3H/PEjqM8WDqNV9fO4eGyHHp8ft6taOCF1WXkTkkOCiVFKjC1YUm+uWdgNCEeyqs2uNqNy3CrEoXieueqQytSyi+klN+SUuZJKfOAC8AsKeXFURtdDBBqcIyNy/dPNg5Y3ofbjAOYdeMNnPS2c9+t2WYc+TffL+Nny4v40tseFP4AzVhtP3DO9HIPeZpNA2fd2PMHCMrMNC4cN7mSzVohxgXhsfl5prbblRxHty9AfpZ2Xn5WMj6/xJUcx/yCzLChJCvGRcWo/TJULZjRwro6ijYco1BcD4yajlz3yu+UUg75rZpIOvJIWGuSWDMlQxNWjJDGY/Pz2H7g3JBJO1Z1iNEG7r3KxqD/+wNEnShkhGiszw2YXXNyMhKpb+2K2E0+ljIjlY5ccb1zzeuR6175dUHo8t6IC1s344w6H9DfIsyIaxuFp0KN4wury3jpz/3Kj088Lbx5uI4FhS6e23uWTctuMTMnoxmj1QCnJjp4Vq8xvm5RASXudGw2uNDaxdy8DNPAhwvhRBO7HgvCGetwtcUViusNldk5TKLxWA1v3ejoc7Ue5CFPs9klx8isBII2WiMRzns12p6tW3Qz2/V66dbKhqGVEGMN5ZErrndUPfJRYjCPFYK9daOjj5XhZiLGO+3YbXCktpVun39QI26N0RuxfavkcN2iAtYtupnnP6qmyxdg84pi/vWp+WxeUcyze6rYfiC2E2uUjlyhCI8y5MNkMLXFaG7GGY+1cWmh2ZzC55ec9EbeSBzK0BkXmZumJJHotFHi1v6/blEBm1cUc7C6ZdjjHEuUjlyhCI8KrYwio7n0f3G/hyPnWvi05hJ2m9A3S2voC0j+avb0iPW1I1VpjOVNzOESbqNZobgeUKGVMWA0izoZ9+ns9bNxqWawNi0rwueXNLZ3R7xfpPolQ4WEJgpKR65QDGTUVCuK0WfuzS7uLnCxdV+Nmf25ZUUxeoe0sERS1EwGxUfoKiLaxsoKxWRHGXILsaaKMJ6zo6vPDCUMpiyJdUM30vmNZWmkQjGeqNCKhVhURQwnlBDr4ZORzq+qR65QhEdtdoYwWi3dRnMsk2GD0iCW5lehmGiozc4oGW6zg2tJrHvYV0Msza9CMVlQhjyE8VJFhBbcenG/B7uNIKM9GbIYlepEoRh9lCG3MJ7V9ULjx3YbPLunyuzpGQvx+pGiqhcqFNeGCREjHys1yVirVkKfz6iWWJqTzqmL/X0yJ0s8OdZUQQrFRGNCx8jHSk0y1qqI0NcFWvPkg54WsyfnZIonK9WJQnFtmBA68tDysLHmnV6tpxmu7K3TbuPJRVr7ttRER9jkHoVCobAyITxyiG21w0hWDJGaJ29Yks+ze6rYsCR/yHhyuM5E1qqHCoVicjNhDHksqx1GUpXPeF3zC1xmlUPQ2rhttqTjDyY9jMVEJoVCMYZIKcf8Z/bs2XI4HKxukmX/c688WN0U9nas8Ov3q+RNP39X/vr9qqjOH83XZdz31+9XxeTcKBSKkQMck2Fs6oTwyCdCYszVrBhG83XFcuhJoVBcWyaE/DDWiYVUepX6rlBMfia0/DDWGe8Vg0q0USiub5QhHyGxkOQy3hcShUIxvihDPkJiQTGiEm0UiuubCZEQFMvEerKSQqGY/CiPfBRQihGFQjGeKEM+CsRyspJCoZj8KEM+QpRiRKFQjDfKkI8QpRhRKBTjzYgTgoQQ/w14GugD9kgp/3ao+0y2hCCFQqEYCyIlBI1ItSKE+Avge0CplLJHCPGtkTyeQqFQKIbPSEMrG4B/klL2AEgpvx75kBQKhUIxHEZqyIuARUKIw0KI/UKIOZFOFEI8KYQ4JoQ41tTUNMKnVSgUCoXBkKEVIcQHwNQwh7bo988A7gLmAP8qhMiXYQLvUsqXgJdAi5GPZNAKhUKh6GdIQy6lvC/SMSHEBuBt3XAfEUIEgExAudwKhUIxRow0RX83cC+wTwhRBMQBQwqojx8/3iyEOD/C546WTKIYUwyixj22TNRxw8Qduxr38Lkp3D9HJD8UQsQBrwJ3AL3AT6WUH131A14DhBDHwsl1Yh017rFloo4bJu7Y1bhHjxF55FLKXuCHozQWhUKhUFwFKrNToVAoJjjXgyF/abwHcJWocY8tE3XcMHHHrsY9SoxLz06FQqFQjB7Xg0euUCgUkxplyBUKhWKCM+kMuRDiLSHE5/pPrRDi8wjn1QohvtDPG/dSjEKIfxBC1FvG/p0I590vhDgthKgWQvzdWI8zzHj+WQhRJYSoEELsEkLcEOG8mJjvoeZPaDyvH68QQswaj3GGjClXCPEnIcQpIcRJIcR/D3POEiFEm+Xz8/fjMdZwDPXex+icz7DM5edCiHYhxN+EnBM7cy6lnLQ/wK+Bv49wrBbIHO8xWsbzD2g6/MHOsQMeIB8t+eoEcNs4j3sZ4ND//hXwq1id72jmD/gO8B4g0EpPHI6Bz8Y0YJb+dypwJsy4lwDvjvdYr+a9j8U5D/O5uQjcFKtzPuk8cgMhhAD+M/B/xnsso8hcoFpKWSM1Df//RSsjPG5IKfdKKfv0m58C08dzPEMQzfx9D/id1PgUuEEIMW2sB2pFStkgpfxM/7sDOAXkjOeYRpmYm/MQlgIeKeVYZaMPm0lryIFFQKOU8myE4xLYK4Q4LoR4cgzHNRhP60vLV4UQGWGO5wB1ltsXiK0v9ONonlU4YmG+o5m/mJ5jIUQeUAYcDnP4biHECSHEe0KIkjEd2OAM9d7H9JwDPyCyQxgTcz7SWivjwmAVGaWU/6H//dcM7o0vkFJ69WYY/08IUSWl/PNoj9XKEJUktwL/iPah/0e0sNDjoQ8R5r7XXD8azXwLIbagdYl6I8LDjPl8hyGa+RuXOY4GIUQK8O/A30gp20MOf4a29L+s76/sBm4Z6zFGYKj3PpbnPA74LvCLMIdjZs4npCGXg1RkBBBCOIBVwOxBHsOr//5aCLELbdl9TQ3LUOM2EEJsB94Nc+gCkGu5PR3wjsLQBiWK+X4UeBBYKvXgYZjHGPP5DkM08zcuczwUQggnmhF/Q0r5duhxq2GXUv5BCPEvQohMKeW4F6WK4r2PyTnXeQD4TErZGHogluZ8soZW7gOqpJQXwh0UQiQLIVKNv9E27CrHcHzhxmSNCa4k/HiOArcIIW7WPYUfAL8fi/FFQghxP/Bz4LtSys4I58TKfEczf78H/ouupLgLaJNSNoz1QK3o+z2vAKeklM9FOGeqfh5CiLlo3+2WsRtleKJ872Nuzi1EXNnH0pxPSI88CgbEtIQQbuBlKeV3gGxgl/4eOIA3pZR/HPNRBvO/hBB3oC0pa4H1EDxuKWWfEOJp4H20nfRXpZQnx2vAOi8A8WhLZoBPpZRPxeJ8R5o/IcRT+vEXgT+gqSiqgU7gsbEeZxgWAGuAL0S/nHYzcCOY4/4rYIMQog/oAn4QaXU0xoR97yfAnCOESAL+Ev27qP/POu6YmXOVoq9QKBQTnMkaWlEoFIrrBmXIFQqFYoKjDLlCoVBMcJQhVygUigmOMuQKhUIxwVGGXKFQKCY4ypArFArFBOf/AxZabHVtcBnZAAAAAElFTkSuQmCC\n",
       "text/plain": [
        "
    "
       ]
@@ -1581,20 +1575,14 @@
      "output_type": "stream",
      "text": [
       "Eigenvalues of Covariance matrix\n",
-      "5.082577137078103\n",
-      "0.7530509884519185\n",
+      "5.185256249406441\n",
+      "0.7554125101643117\n",
       "First eigenvector\n",
-      "[0.85042593 0.5260948 ]\n",
+      "[0.85104821 0.52508756]\n",
       "Second eigenvector\n",
-      "[-0.5260948   0.85042593]\n"
-     ]
-    },
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
+      "[-0.52508756  0.85104821]\n",
       "Eigenvector of largest eigenvalue\n",
-      "[-0.85042593 -0.5260948 ]\n"
+      "[-0.85104821 -0.52508756]\n"
      ]
     }
    ],
diff --git a/doc/src/LectureNotes/_build/jupyter_execute/chapter8_77_1.png b/doc/src/LectureNotes/_build/jupyter_execute/chapter8_77_1.png
index 50f345c7f..1bc84b602 100644
Binary files a/doc/src/LectureNotes/_build/jupyter_execute/chapter8_77_1.png and b/doc/src/LectureNotes/_build/jupyter_execute/chapter8_77_1.png differ
diff --git a/doc/src/LectureNotes/_build/jupyter_execute/chapter9.ipynb b/doc/src/LectureNotes/_build/jupyter_execute/chapter9.ipynb
index 0d3781755..50f4bc92b 100644
--- a/doc/src/LectureNotes/_build/jupyter_execute/chapter9.ipynb
+++ b/doc/src/LectureNotes/_build/jupyter_execute/chapter9.ipynb
@@ -478,7 +478,7 @@
      "output_type": "stream",
      "text": [
       "\r",
-      " 1/12 [=>............................] - ETA: 0s - loss: 2.5817 - accuracy: 0.2500"
+      " 1/12 [=>............................] - ETA: 0s - loss: 3.0150 - accuracy: 0.1250"
      ]
     },
     {
@@ -486,7 +486,7 @@
      "output_type": "stream",
      "text": [
       "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
-      "12/12 [==============================] - 0s 917us/step - loss: 2.8078 - accuracy: 0.1139\n"
+      "12/12 [==============================] - 0s 958us/step - loss: 2.9441 - accuracy: 0.0722\n"
      ]
     },
     {
@@ -495,7 +495,7 @@
      "text": [
       "Learning rate =  1e-05\n",
       "Lambda =  1e-05\n",
-      "Test accuracy: 0.114\n",
+      "Test accuracy: 0.072\n",
       "\n"
      ]
     },
@@ -504,7 +504,7 @@
      "output_type": "stream",
      "text": [
       "\r",
-      " 1/12 [=>............................] - ETA: 0s - loss: 3.0782 - accuracy: 0.1875"
+      " 1/12 [=>............................] - ETA: 0s - loss: 3.3024 - accuracy: 0.0312"
      ]
     },
     {
@@ -512,7 +512,7 @@
      "output_type": "stream",
      "text": [
       "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
-      "12/12 [==============================] - 0s 932us/step - loss: 3.3640 - accuracy: 0.1500\n"
+      "12/12 [==============================] - 0s 957us/step - loss: 3.0967 - accuracy: 0.1389\n"
      ]
     },
     {
@@ -521,7 +521,7 @@
      "text": [
       "Learning rate =  1e-05\n",
       "Lambda =  0.0001\n",
-      "Test accuracy: 0.150\n",
+      "Test accuracy: 0.139\n",
       "\n"
      ]
     },
@@ -530,7 +530,7 @@
      "output_type": "stream",
      "text": [
       "\r",
-      " 1/12 [=>............................] - ETA: 0s - loss: 2.7005 - accuracy: 0.0312"
+      " 1/12 [=>............................] - ETA: 0s - loss: 2.9008 - accuracy: 0.2188"
      ]
     },
     {
@@ -538,7 +538,7 @@
      "output_type": "stream",
      "text": [
       "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
-      "12/12 [==============================] - 0s 933us/step - loss: 2.7588 - accuracy: 0.0944\n"
+      "12/12 [==============================] - 0s 1ms/step - loss: 3.4843 - accuracy: 0.1028\n"
      ]
     },
     {
@@ -547,7 +547,7 @@
      "text": [
       "Learning rate =  1e-05\n",
       "Lambda =  0.001\n",
-      "Test accuracy: 0.094\n",
+      "Test accuracy: 0.103\n",
       "\n"
      ]
     },
@@ -556,7 +556,7 @@
      "output_type": "stream",
      "text": [
       "\r",
-      " 1/12 [=>............................] - ETA: 0s - loss: 3.9994 - accuracy: 0.1250"
+      " 1/12 [=>............................] - ETA: 0s - loss: 3.7695 - accuracy: 0.1250"
      ]
     },
     {
@@ -564,7 +564,7 @@
      "output_type": "stream",
      "text": [
       "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
-      "12/12 [==============================] - 0s 1ms/step - loss: 4.2041 - accuracy: 0.1028\n"
+      "12/12 [==============================] - 0s 968us/step - loss: 3.6647 - accuracy: 0.1417\n"
      ]
     },
     {
@@ -573,7 +573,7 @@
      "text": [
       "Learning rate =  1e-05\n",
       "Lambda =  0.01\n",
-      "Test accuracy: 0.103\n",
+      "Test accuracy: 0.142\n",
       "\n"
      ]
     },
@@ -582,7 +582,7 @@
      "output_type": "stream",
      "text": [
       "\r",
-      " 1/12 [=>............................] - ETA: 0s - loss: 12.9177 - accuracy: 0.1250"
+      " 1/12 [=>............................] - ETA: 0s - loss: 12.4141 - accuracy: 0.0938"
      ]
     },
     {
@@ -590,7 +590,7 @@
      "output_type": "stream",
      "text": [
       "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
-      "12/12 [==============================] - 0s 939us/step - loss: 12.8693 - accuracy: 0.1056\n"
+      "12/12 [==============================] - 0s 960us/step - loss: 12.3084 - accuracy: 0.1028\n"
      ]
     },
     {
@@ -599,7 +599,7 @@
      "text": [
       "Learning rate =  1e-05\n",
       "Lambda =  0.1\n",
-      "Test accuracy: 0.106\n",
+      "Test accuracy: 0.103\n",
       "\n"
      ]
     },
@@ -608,7 +608,7 @@
      "output_type": "stream",
      "text": [
       "\r",
-      " 1/12 [=>............................] - ETA: 0s - loss: 92.4325 - accuracy: 0.1250"
+      " 1/12 [=>............................] - ETA: 0s - loss: 92.7441 - accuracy: 0.0938"
      ]
     },
     {
@@ -616,7 +616,7 @@
      "output_type": "stream",
      "text": [
       "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
-      "12/12 [==============================] - 0s 2ms/step - loss: 92.4491 - accuracy: 0.1111\n"
+      "12/12 [==============================] - 0s 959us/step - loss: 91.7224 - accuracy: 0.1306\n"
      ]
     },
     {
@@ -625,7 +625,7 @@
      "text": [
       "Learning rate =  1e-05\n",
       "Lambda =  1.0\n",
-      "Test accuracy: 0.111\n",
+      "Test accuracy: 0.131\n",
       "\n"
      ]
     },
@@ -634,7 +634,7 @@
      "output_type": "stream",
      "text": [
       "\r",
-      " 1/12 [=>............................] - ETA: 0s - loss: 514.5186 - accuracy: 0.0312"
+      " 1/12 [=>............................] - ETA: 0s - loss: 514.1083 - accuracy: 0.0625"
      ]
     },
     {
@@ -642,7 +642,7 @@
      "output_type": "stream",
      "text": [
       "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
-      "12/12 [==============================] - 0s 939us/step - loss: 514.2589 - accuracy: 0.1028\n"
+      "12/12 [==============================] - 0s 958us/step - loss: 513.9800 - accuracy: 0.1111\n"
      ]
     },
     {
@@ -651,7 +651,7 @@
      "text": [
       "Learning rate =  1e-05\n",
       "Lambda =  10.0\n",
-      "Test accuracy: 0.103\n",
+      "Test accuracy: 0.111\n",
       "\n"
      ]
     }
diff --git a/doc/src/LectureNotes/_build/jupyter_execute/testbook/_build/jupyter_execute/chapter2.ipynb b/doc/src/LectureNotes/_build/jupyter_execute/testbook/_build/jupyter_execute/chapter2.ipynb
index 95900d0c8..8e5d4fc24 100644
--- a/doc/src/LectureNotes/_build/jupyter_execute/testbook/_build/jupyter_execute/chapter2.ipynb
+++ b/doc/src/LectureNotes/_build/jupyter_execute/testbook/_build/jupyter_execute/chapter2.ipynb
@@ -2152,8 +2152,8 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "[-1.17029256  1.16440383 -0.52324284  0.91883189 -0.05688344 -0.80125589\n",
-      "  1.85759035 -1.11063616  0.67761635  0.09188807]\n"
+      "[ 1.8413241  -1.44931127  0.91189378  1.82553587  0.69940044 -0.65726179\n",
+      "  1.72526513  1.82825747 -1.34003257 -0.5817835 ]\n"
      ]
     }
    ],
diff --git a/doc/src/LectureNotes/_build/jupyter_execute/testbook/chapter2.ipynb b/doc/src/LectureNotes/_build/jupyter_execute/testbook/chapter2.ipynb
index f033fa132..ba853211e 100644
--- a/doc/src/LectureNotes/_build/jupyter_execute/testbook/chapter2.ipynb
+++ b/doc/src/LectureNotes/_build/jupyter_execute/testbook/chapter2.ipynb
@@ -2152,8 +2152,8 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "[-0.15903306 -1.03373868  0.31582494 -0.65395262  0.96308377 -0.89760255\n",
-      "  1.62466988  1.27946914 -0.71331923 -0.2179017 ]\n"
+      "[-1.04569783  1.68833083  1.59472197 -0.22027464 -1.12477388  0.78480985\n",
+      "  0.8799874  -0.43301111 -0.53140431  0.2976373 ]\n"
      ]
     }
    ],
diff --git a/doc/src/LectureNotes/_toc.yml b/doc/src/LectureNotes/_toc.yml
index 5ad15dc40..abf629534 100644
--- a/doc/src/LectureNotes/_toc.yml
+++ b/doc/src/LectureNotes/_toc.yml
@@ -9,5 +9,7 @@
   - file: chapter6.ipynb
   - file: chapter7.ipynb
   - file: chapter8.ipynb
-  - file: chapter9.ipynb    
+  - file: chapter9.ipynb
+  - file: chapter10.ipynb
+  - file: chapter11.ipynb      
 
diff --git a/doc/src/LectureNotes/chapter10.ipynb b/doc/src/LectureNotes/chapter10.ipynb
new file mode 100644
index 000000000..be47c97e1
--- /dev/null
+++ b/doc/src/LectureNotes/chapter10.ipynb
@@ -0,0 +1,3815 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Recurrent Neural Networks\n",
+    "\n",
+    "[Overview video](https://www.youtube.com/watch?v=SEnXr6v2ifU&ab_channel=AlexanderAmini).\n",
+    "See also lecture on Thursday October 22 and examples from [week 42](https://compphysics.github.io/MachineLearning/doc/pub/week42/html/week42.html).\n",
+    "\n",
+    "[IN5400 at UiO Lecture](https://www.uio.no/studier/emner/matnat/ifi/IN5400/v20/material/week10/in5400_2020_week10_recurrent_neural_network.pdf)\n",
+    "\n",
+    "[CS231 at Stanford Lecture](https://www.youtube.com/watch?v=6niqTuYFZLQ&list=PLzUTmXVwsnXod6WNdg57Yc3zFx_f-RYsq&index=10&ab_channel=StanfordUniversitySchoolofEngineering)\n",
+    "\n",
+    "## Recurrent neural networks: Overarching view\n",
+    "\n",
+    "Till now our focus has been, including convolutional neural networks\n",
+    "as well, on feedforward neural networks. The output or the activations\n",
+    "flow only in one direction, from the input layer to the output layer.\n",
+    "\n",
+    "A recurrent neural network (RNN) looks very much like a feedforward\n",
+    "neural network, except that it also has connections pointing\n",
+    "backward. \n",
+    "\n",
+    "RNNs are used to analyze time series data such as stock prices, and\n",
+    "tell you when to buy or sell. In autonomous driving systems, they can\n",
+    "anticipate car trajectories and help avoid accidents. More generally,\n",
+    "they can work on sequences of arbitrary lengths, rather than on\n",
+    "fixed-sized inputs like all the nets we have discussed so far. For\n",
+    "example, they can take sentences, documents, or audio samples as\n",
+    "input, making them extremely useful for natural language processing\n",
+    "systems such as automatic translation and speech-to-text.\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "## Set up of an RNN\n",
+    "\n",
+    "\n",
+    "Text to come.\n",
+    "\n",
+    "\n",
+    "## A simple example"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "%matplotlib inline\n",
+    "\n",
+    "# Start importing packages\n",
+    "import pandas as pd\n",
+    "import numpy as np\n",
+    "import matplotlib.pyplot as plt\n",
+    "import tensorflow as tf\n",
+    "from tensorflow.keras import datasets, layers, models\n",
+    "from tensorflow.keras.layers import Input\n",
+    "from tensorflow.keras.models import Model, Sequential \n",
+    "from tensorflow.keras.layers import Dense, SimpleRNN, LSTM, GRU\n",
+    "from tensorflow.keras import optimizers     \n",
+    "from tensorflow.keras import regularizers           \n",
+    "from tensorflow.keras.utils import to_categorical \n",
+    "\n",
+    "\n",
+    "\n",
+    "# convert into dataset matrix\n",
+    "def convertToMatrix(data, step):\n",
+    " X, Y =[], []\n",
+    " for i in range(len(data)-step):\n",
+    "  d=i+step  \n",
+    "  X.append(data[i:d,])\n",
+    "  Y.append(data[d,])\n",
+    " return np.array(X), np.array(Y)\n",
+    "\n",
+    "step = 4\n",
+    "N = 1000    \n",
+    "Tp = 800    \n",
+    "\n",
+    "t=np.arange(0,N)\n",
+    "x=np.sin(0.02*t)+2*np.random.rand(N)\n",
+    "df = pd.DataFrame(x)\n",
+    "df.head()\n",
+    "\n",
+    "plt.plot(df)\n",
+    "plt.show()\n",
+    "\n",
+    "values=df.values\n",
+    "train,test = values[0:Tp,:], values[Tp:N,:]\n",
+    "\n",
+    "# add step elements into train and test\n",
+    "test = np.append(test,np.repeat(test[-1,],step))\n",
+    "train = np.append(train,np.repeat(train[-1,],step))\n",
+    " \n",
+    "trainX,trainY =convertToMatrix(train,step)\n",
+    "testX,testY =convertToMatrix(test,step)\n",
+    "trainX = np.reshape(trainX, (trainX.shape[0], 1, trainX.shape[1]))\n",
+    "testX = np.reshape(testX, (testX.shape[0], 1, testX.shape[1]))\n",
+    "\n",
+    "model = Sequential()\n",
+    "model.add(SimpleRNN(units=32, input_shape=(1,step), activation=\"relu\"))\n",
+    "model.add(Dense(8, activation=\"relu\")) \n",
+    "model.add(Dense(1))\n",
+    "model.compile(loss='mean_squared_error', optimizer='rmsprop')\n",
+    "model.summary()\n",
+    "\n",
+    "model.fit(trainX,trainY, epochs=100, batch_size=16, verbose=2)\n",
+    "trainPredict = model.predict(trainX)\n",
+    "testPredict= model.predict(testX)\n",
+    "predicted=np.concatenate((trainPredict,testPredict),axis=0)\n",
+    "\n",
+    "trainScore = model.evaluate(trainX, trainY, verbose=0)\n",
+    "print(trainScore)\n",
+    "\n",
+    "index = df.index.values\n",
+    "plt.plot(index,df)\n",
+    "plt.plot(index,predicted)\n",
+    "plt.axvline(df.index[Tp], c=\"r\")\n",
+    "plt.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## An extrapolation example\n",
+    "\n",
+    "The following code provides an example of how recurrent neural\n",
+    "networks can be used to extrapolate to unknown values of physics data\n",
+    "sets.  Specifically, the data sets used in this program come from\n",
+    "a quantum mechanical many-body calculation of energies as functions of the number of particles."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "\n",
+    "# For matrices and calculations\n",
+    "import numpy as np\n",
+    "# For machine learning (backend for keras)\n",
+    "import tensorflow as tf\n",
+    "# User-friendly machine learning library\n",
+    "# Front end for TensorFlow\n",
+    "import tensorflow.keras\n",
+    "# Different methods from Keras needed to create an RNN\n",
+    "# This is not necessary but it shortened function calls \n",
+    "# that need to be used in the code.\n",
+    "from tensorflow.keras import datasets, layers, models\n",
+    "from tensorflow.keras.layers import Input\n",
+    "from tensorflow.keras import regularizers\n",
+    "from tensorflow.keras.models import Model, Sequential\n",
+    "from tensorflow.keras.layers import Dense, SimpleRNN, LSTM, GRU\n",
+    "# For timing the code\n",
+    "from timeit import default_timer as timer\n",
+    "# For plotting\n",
+    "import matplotlib.pyplot as plt\n",
+    "\n",
+    "\n",
+    "# The data set\n",
+    "datatype='VaryDimension'\n",
+    "X_tot = np.arange(2, 42, 2)\n",
+    "y_tot = np.array([-0.03077640549, -0.08336233266, -0.1446729567, -0.2116753732, -0.2830637392, -0.3581341341, -0.436462435, -0.5177783846,\n",
+    "\t-0.6019067271, -0.6887363571, -0.7782028952, -0.8702784034, -0.9649652536, -1.062292565, -1.16231451, \n",
+    "\t-1.265109911, -1.370782966, -1.479465113, -1.591317992, -1.70653767])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Formatting the Data\n",
+    "\n",
+    "The way the recurrent neural networks are trained in this program\n",
+    "differs from how machine learning algorithms are usually trained.\n",
+    "Typically a machine learning algorithm is trained by learning the\n",
+    "relationship between the x data and the y data.  In this program, the\n",
+    "recurrent neural network will be trained to recognize the relationship\n",
+    "in a sequence of y values.  This is type of data formatting is\n",
+    "typically used time series forcasting, but it can also be used in any\n",
+    "extrapolation (time series forecasting is just a specific type of\n",
+    "extrapolation along the time axis).  This method of data formatting\n",
+    "does not use the x data and assumes that the y data are evenly spaced.\n",
+    "\n",
+    "For a standard machine learning algorithm, the training data has the\n",
+    "form of (x,y) so the machine learning algorithm learns to assiciate a\n",
+    "y value with a given x value.  This is useful when the test data has x\n",
+    "values within the same range as the training data.  However, for this\n",
+    "application, the x values of the test data are outside of the x values\n",
+    "of the training data and the traditional method of training a machine\n",
+    "learning algorithm does not work as well.  For this reason, the\n",
+    "recurrent neural network is trained on sequences of y values of the\n",
+    "form ((y1, y2), y3), so that the network is concerned with learning\n",
+    "the pattern of the y data and not the relation between the x and y\n",
+    "data.  As long as the pattern of y data outside of the training region\n",
+    "stays relatively stable compared to what was inside the training\n",
+    "region, this method of training can produce accurate extrapolations to\n",
+    "y values far removed from the training data set.\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    ""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "# FORMAT_DATA\n",
+    "def format_data(data, length_of_sequence = 2):  \n",
+    "    \"\"\"\n",
+    "        Inputs:\n",
+    "            data(a numpy array): the data that will be the inputs to the recurrent neural\n",
+    "                network\n",
+    "            length_of_sequence (an int): the number of elements in one iteration of the\n",
+    "                sequence patter.  For a function approximator use length_of_sequence = 2.\n",
+    "        Returns:\n",
+    "            rnn_input (a 3D numpy array): the input data for the recurrent neural network.  Its\n",
+    "                dimensions are length of data - length of sequence, length of sequence, \n",
+    "                dimnsion of data\n",
+    "            rnn_output (a numpy array): the training data for the neural network\n",
+    "        Formats data to be used in a recurrent neural network.\n",
+    "    \"\"\"\n",
+    "\n",
+    "    X, Y = [], []\n",
+    "    for i in range(len(data)-length_of_sequence):\n",
+    "        # Get the next length_of_sequence elements\n",
+    "        a = data[i:i+length_of_sequence]\n",
+    "        # Get the element that immediately follows that\n",
+    "        b = data[i+length_of_sequence]\n",
+    "        # Reshape so that each data point is contained in its own array\n",
+    "        a = np.reshape (a, (len(a), 1))\n",
+    "        X.append(a)\n",
+    "        Y.append(b)\n",
+    "    rnn_input = np.array(X)\n",
+    "    rnn_output = np.array(Y)\n",
+    "\n",
+    "    return rnn_input, rnn_output\n",
+    "\n",
+    "\n",
+    "# ## Defining the Recurrent Neural Network Using Keras\n",
+    "# \n",
+    "# The following method defines a simple recurrent neural network in keras consisting of one input layer, one hidden layer, and one output layer.\n",
+    "\n",
+    "def rnn(length_of_sequences, batch_size = None, stateful = False):\n",
+    "    \"\"\"\n",
+    "        Inputs:\n",
+    "            length_of_sequences (an int): the number of y values in \"x data\".  This is determined\n",
+    "                when the data is formatted\n",
+    "            batch_size (an int): Default value is None.  See Keras documentation of SimpleRNN.\n",
+    "            stateful (a boolean): Default value is False.  See Keras documentation of SimpleRNN.\n",
+    "        Returns:\n",
+    "            model (a Keras model): The recurrent neural network that is built and compiled by this\n",
+    "                method\n",
+    "        Builds and compiles a recurrent neural network with one hidden layer and returns the model.\n",
+    "    \"\"\"\n",
+    "    # Number of neurons in the input and output layers\n",
+    "    in_out_neurons = 1\n",
+    "    # Number of neurons in the hidden layer\n",
+    "    hidden_neurons = 200\n",
+    "    # Define the input layer\n",
+    "    inp = Input(batch_shape=(batch_size, \n",
+    "                length_of_sequences, \n",
+    "                in_out_neurons))  \n",
+    "    # Define the hidden layer as a simple RNN layer with a set number of neurons and add it to \n",
+    "    # the network immediately after the input layer\n",
+    "    rnn = SimpleRNN(hidden_neurons, \n",
+    "                    return_sequences=False,\n",
+    "                    stateful = stateful,\n",
+    "                    name=\"RNN\")(inp)\n",
+    "    # Define the output layer as a dense neural network layer (standard neural network layer)\n",
+    "    #and add it to the network immediately after the hidden layer.\n",
+    "    dens = Dense(in_out_neurons,name=\"dense\")(rnn)\n",
+    "    # Create the machine learning model starting with the input layer and ending with the \n",
+    "    # output layer\n",
+    "    model = Model(inputs=[inp],outputs=[dens])\n",
+    "    # Compile the machine learning model using the mean squared error function as the loss \n",
+    "    # function and an Adams optimizer.\n",
+    "    model.compile(loss=\"mean_squared_error\", optimizer=\"adam\")  \n",
+    "    return model"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Predicting New Points With A Trained Recurrent Neural Network"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "def test_rnn (x1, y_test, plot_min, plot_max):\n",
+    "    \"\"\"\n",
+    "        Inputs:\n",
+    "            x1 (a list or numpy array): The complete x component of the data set\n",
+    "            y_test (a list or numpy array): The complete y component of the data set\n",
+    "            plot_min (an int or float): the smallest x value used in the training data\n",
+    "            plot_max (an int or float): the largest x valye used in the training data\n",
+    "        Returns:\n",
+    "            None.\n",
+    "        Uses a trained recurrent neural network model to predict future points in the \n",
+    "        series.  Computes the MSE of the predicted data set from the true data set, saves\n",
+    "        the predicted data set to a csv file, and plots the predicted and true data sets w\n",
+    "        while also displaying the data range used for training.\n",
+    "    \"\"\"\n",
+    "    # Add the training data as the first dim points in the predicted data array as these\n",
+    "    # are known values.\n",
+    "    y_pred = y_test[:dim].tolist()\n",
+    "    # Generate the first input to the trained recurrent neural network using the last two \n",
+    "    # points of the training data.  Based on how the network was trained this means that it\n",
+    "    # will predict the first point in the data set after the training data.  All of the \n",
+    "    # brackets are necessary for Tensorflow.\n",
+    "    next_input = np.array([[[y_test[dim-2]], [y_test[dim-1]]]])\n",
+    "    # Save the very last point in the training data set.  This will be used later.\n",
+    "    last = [y_test[dim-1]]\n",
+    "\n",
+    "    # Iterate until the complete data set is created.\n",
+    "    for i in range (dim, len(y_test)):\n",
+    "        # Predict the next point in the data set using the previous two points.\n",
+    "        next = model.predict(next_input)\n",
+    "        # Append just the number of the predicted data set\n",
+    "        y_pred.append(next[0][0])\n",
+    "        # Create the input that will be used to predict the next data point in the data set.\n",
+    "        next_input = np.array([[last, next[0]]], dtype=np.float64)\n",
+    "        last = next\n",
+    "\n",
+    "    # Print the mean squared error between the known data set and the predicted data set.\n",
+    "    print('MSE: ', np.square(np.subtract(y_test, y_pred)).mean())\n",
+    "    # Save the predicted data set as a csv file for later use\n",
+    "    name = datatype + 'Predicted'+str(dim)+'.csv'\n",
+    "    np.savetxt(name, y_pred, delimiter=',')\n",
+    "    # Plot the known data set and the predicted data set.  The red box represents the region that was used\n",
+    "    # for the training data.\n",
+    "    fig, ax = plt.subplots()\n",
+    "    ax.plot(x1, y_test, label=\"true\", linewidth=3)\n",
+    "    ax.plot(x1, y_pred, 'g-.',label=\"predicted\", linewidth=4)\n",
+    "    ax.legend()\n",
+    "    # Created a red region to represent the points used in the training data.\n",
+    "    ax.axvspan(plot_min, plot_max, alpha=0.25, color='red')\n",
+    "    plt.show()\n",
+    "\n",
+    "# Check to make sure the data set is complete\n",
+    "assert len(X_tot) == len(y_tot)\n",
+    "\n",
+    "# This is the number of points that will be used in as the training data\n",
+    "dim=12\n",
+    "\n",
+    "# Separate the training data from the whole data set\n",
+    "X_train = X_tot[:dim]\n",
+    "y_train = y_tot[:dim]\n",
+    "\n",
+    "\n",
+    "# Generate the training data for the RNN, using a sequence of 2\n",
+    "rnn_input, rnn_training = format_data(y_train, 2)\n",
+    "\n",
+    "\n",
+    "# Create a recurrent neural network in Keras and produce a summary of the \n",
+    "# machine learning model\n",
+    "model = rnn(length_of_sequences = rnn_input.shape[1])\n",
+    "model.summary()\n",
+    "\n",
+    "# Start the timer.  Want to time training+testing\n",
+    "start = timer()\n",
+    "# Fit the model using the training data genenerated above using 150 training iterations and a 5%\n",
+    "# validation split.  Setting verbose to True prints information about each training iteration.\n",
+    "hist = model.fit(rnn_input, rnn_training, batch_size=None, epochs=150, \n",
+    "                 verbose=True,validation_split=0.05)\n",
+    "\n",
+    "for label in [\"loss\",\"val_loss\"]:\n",
+    "    plt.plot(hist.history[label],label=label)\n",
+    "\n",
+    "plt.ylabel(\"loss\")\n",
+    "plt.xlabel(\"epoch\")\n",
+    "plt.title(\"The final validation loss: {}\".format(hist.history[\"val_loss\"][-1]))\n",
+    "plt.legend()\n",
+    "plt.show()\n",
+    "\n",
+    "# Use the trained neural network to predict more points of the data set\n",
+    "test_rnn(X_tot, y_tot, X_tot[0], X_tot[dim-1])\n",
+    "# Stop the timer and calculate the total time needed.\n",
+    "end = timer()\n",
+    "print('Time: ', end-start)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Other Things to Try\n",
+    "\n",
+    "\n",
+    "Changing the size of the recurrent neural network and its parameters\n",
+    "can drastically change the results you get from the model.  The below\n",
+    "code takes the simple recurrent neural network from above and adds a\n",
+    "second hidden layer, changes the number of neurons in the hidden\n",
+    "layer, and explicitly declares the activation function of the hidden\n",
+    "layers to be a sigmoid function.  The loss function and optimizer can\n",
+    "also be changed but are kept the same as the above network.  These\n",
+    "parameters can be tuned to provide the optimal result from the\n",
+    "network.  For some ideas on how to improve the performance of a\n",
+    "[recurrent neural network](https://danijar.com/tips-for-training-recurrent-neural-networks)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "def rnn_2layers(length_of_sequences, batch_size = None, stateful = False):\n",
+    "    \"\"\"\n",
+    "        Inputs:\n",
+    "            length_of_sequences (an int): the number of y values in \"x data\".  This is determined\n",
+    "                when the data is formatted\n",
+    "            batch_size (an int): Default value is None.  See Keras documentation of SimpleRNN.\n",
+    "            stateful (a boolean): Default value is False.  See Keras documentation of SimpleRNN.\n",
+    "        Returns:\n",
+    "            model (a Keras model): The recurrent neural network that is built and compiled by this\n",
+    "                method\n",
+    "        Builds and compiles a recurrent neural network with two hidden layers and returns the model.\n",
+    "    \"\"\"\n",
+    "    # Number of neurons in the input and output layers\n",
+    "    in_out_neurons = 1\n",
+    "    # Number of neurons in the hidden layer, increased from the first network\n",
+    "    hidden_neurons = 500\n",
+    "    # Define the input layer\n",
+    "    inp = Input(batch_shape=(batch_size, \n",
+    "                length_of_sequences, \n",
+    "                in_out_neurons))  \n",
+    "    # Create two hidden layers instead of one hidden layer.  Explicitly set the activation\n",
+    "    # function to be the sigmoid function (the default value is hyperbolic tangent)\n",
+    "    rnn1 = SimpleRNN(hidden_neurons, \n",
+    "                    return_sequences=True,  # This needs to be True if another hidden layer is to follow\n",
+    "                    stateful = stateful, activation = 'sigmoid',\n",
+    "                    name=\"RNN1\")(inp)\n",
+    "    rnn2 = SimpleRNN(hidden_neurons, \n",
+    "                    return_sequences=False, activation = 'sigmoid',\n",
+    "                    stateful = stateful,\n",
+    "                    name=\"RNN2\")(rnn1)\n",
+    "    # Define the output layer as a dense neural network layer (standard neural network layer)\n",
+    "    #and add it to the network immediately after the hidden layer.\n",
+    "    dens = Dense(in_out_neurons,name=\"dense\")(rnn2)\n",
+    "    # Create the machine learning model starting with the input layer and ending with the \n",
+    "    # output layer\n",
+    "    model = Model(inputs=[inp],outputs=[dens])\n",
+    "    # Compile the machine learning model using the mean squared error function as the loss \n",
+    "    # function and an Adams optimizer.\n",
+    "    model.compile(loss=\"mean_squared_error\", optimizer=\"adam\")  \n",
+    "    return model\n",
+    "\n",
+    "# Check to make sure the data set is complete\n",
+    "assert len(X_tot) == len(y_tot)\n",
+    "\n",
+    "# This is the number of points that will be used in as the training data\n",
+    "dim=12\n",
+    "\n",
+    "# Separate the training data from the whole data set\n",
+    "X_train = X_tot[:dim]\n",
+    "y_train = y_tot[:dim]\n",
+    "\n",
+    "\n",
+    "# Generate the training data for the RNN, using a sequence of 2\n",
+    "rnn_input, rnn_training = format_data(y_train, 2)\n",
+    "\n",
+    "\n",
+    "# Create a recurrent neural network in Keras and produce a summary of the \n",
+    "# machine learning model\n",
+    "model = rnn_2layers(length_of_sequences = 2)\n",
+    "model.summary()\n",
+    "\n",
+    "# Start the timer.  Want to time training+testing\n",
+    "start = timer()\n",
+    "# Fit the model using the training data genenerated above using 150 training iterations and a 5%\n",
+    "# validation split.  Setting verbose to True prints information about each training iteration.\n",
+    "hist = model.fit(rnn_input, rnn_training, batch_size=None, epochs=150, \n",
+    "                 verbose=True,validation_split=0.05)\n",
+    "\n",
+    "\n",
+    "# This section plots the training loss and the validation loss as a function of training iteration.\n",
+    "# This is not required for analyzing the couple cluster data but can help determine if the network is\n",
+    "# being overtrained.\n",
+    "for label in [\"loss\",\"val_loss\"]:\n",
+    "    plt.plot(hist.history[label],label=label)\n",
+    "\n",
+    "plt.ylabel(\"loss\")\n",
+    "plt.xlabel(\"epoch\")\n",
+    "plt.title(\"The final validation loss: {}\".format(hist.history[\"val_loss\"][-1]))\n",
+    "plt.legend()\n",
+    "plt.show()\n",
+    "\n",
+    "# Use the trained neural network to predict more points of the data set\n",
+    "test_rnn(X_tot, y_tot, X_tot[0], X_tot[dim-1])\n",
+    "# Stop the timer and calculate the total time needed.\n",
+    "end = timer()\n",
+    "print('Time: ', end-start)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Other Types of Recurrent Neural Networks\n",
+    "\n",
+    "Besides a simple recurrent neural network layer, there are two other\n",
+    "commonly used types of recurrent neural network layers: Long Short\n",
+    "Term Memory (LSTM) and Gated Recurrent Unit (GRU).  For a short\n",
+    "introduction to these layers see \n",
+    "and .\n",
+    "\n",
+    "The first network created below is similar to the previous network,\n",
+    "but it replaces the SimpleRNN layers with LSTM layers.  The second\n",
+    "network below has two hidden layers made up of GRUs, which are\n",
+    "preceeded by two dense (feeddorward) neural network layers.  These\n",
+    "dense layers \"preprocess\" the data before it reaches the recurrent\n",
+    "layers.  This architecture has been shown to improve the performance\n",
+    "of recurrent neural networks (see the link above and also\n",
+    "."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "def lstm_2layers(length_of_sequences, batch_size = None, stateful = False):\n",
+    "    \"\"\"\n",
+    "        Inputs:\n",
+    "            length_of_sequences (an int): the number of y values in \"x data\".  This is determined\n",
+    "                when the data is formatted\n",
+    "            batch_size (an int): Default value is None.  See Keras documentation of SimpleRNN.\n",
+    "            stateful (a boolean): Default value is False.  See Keras documentation of SimpleRNN.\n",
+    "        Returns:\n",
+    "            model (a Keras model): The recurrent neural network that is built and compiled by this\n",
+    "                method\n",
+    "        Builds and compiles a recurrent neural network with two LSTM hidden layers and returns the model.\n",
+    "    \"\"\"\n",
+    "    # Number of neurons on the input/output layer and the number of neurons in the hidden layer\n",
+    "    in_out_neurons = 1\n",
+    "    hidden_neurons = 250\n",
+    "    # Input Layer\n",
+    "    inp = Input(batch_shape=(batch_size, \n",
+    "                length_of_sequences, \n",
+    "                in_out_neurons)) \n",
+    "    # Hidden layers (in this case they are LSTM layers instead if SimpleRNN layers)\n",
+    "    rnn= LSTM(hidden_neurons, \n",
+    "                    return_sequences=True,\n",
+    "                    stateful = stateful,\n",
+    "                    name=\"RNN\", use_bias=True, activation='tanh')(inp)\n",
+    "    rnn1 = LSTM(hidden_neurons, \n",
+    "                    return_sequences=False,\n",
+    "                    stateful = stateful,\n",
+    "                    name=\"RNN1\", use_bias=True, activation='tanh')(rnn)\n",
+    "    # Output layer\n",
+    "    dens = Dense(in_out_neurons,name=\"dense\")(rnn1)\n",
+    "    # Define the midel\n",
+    "    model = Model(inputs=[inp],outputs=[dens])\n",
+    "    # Compile the model\n",
+    "    model.compile(loss='mean_squared_error', optimizer='adam')  \n",
+    "    # Return the model\n",
+    "    return model\n",
+    "\n",
+    "def dnn2_gru2(length_of_sequences, batch_size = None, stateful = False):\n",
+    "    \"\"\"\n",
+    "        Inputs:\n",
+    "            length_of_sequences (an int): the number of y values in \"x data\".  This is determined\n",
+    "                when the data is formatted\n",
+    "            batch_size (an int): Default value is None.  See Keras documentation of SimpleRNN.\n",
+    "            stateful (a boolean): Default value is False.  See Keras documentation of SimpleRNN.\n",
+    "        Returns:\n",
+    "            model (a Keras model): The recurrent neural network that is built and compiled by this\n",
+    "                method\n",
+    "        Builds and compiles a recurrent neural network with four hidden layers (two dense followed by\n",
+    "        two GRU layers) and returns the model.\n",
+    "    \"\"\"    \n",
+    "    # Number of neurons on the input/output layers and hidden layers\n",
+    "    in_out_neurons = 1\n",
+    "    hidden_neurons = 250\n",
+    "    # Input layer\n",
+    "    inp = Input(batch_shape=(batch_size, \n",
+    "                length_of_sequences, \n",
+    "                in_out_neurons)) \n",
+    "    # Hidden Dense (feedforward) layers\n",
+    "    dnn = Dense(hidden_neurons/2, activation='relu', name='dnn')(inp)\n",
+    "    dnn1 = Dense(hidden_neurons/2, activation='relu', name='dnn1')(dnn)\n",
+    "    # Hidden GRU layers\n",
+    "    rnn1 = GRU(hidden_neurons, \n",
+    "                    return_sequences=True,\n",
+    "                    stateful = stateful,\n",
+    "                    name=\"RNN1\", use_bias=True)(dnn1)\n",
+    "    rnn = GRU(hidden_neurons, \n",
+    "                    return_sequences=False,\n",
+    "                    stateful = stateful,\n",
+    "                    name=\"RNN\", use_bias=True)(rnn1)\n",
+    "    # Output layer\n",
+    "    dens = Dense(in_out_neurons,name=\"dense\")(rnn)\n",
+    "    # Define the model\n",
+    "    model = Model(inputs=[inp],outputs=[dens])\n",
+    "    # Compile the mdoel\n",
+    "    model.compile(loss='mean_squared_error', optimizer='adam')  \n",
+    "    # Return the model\n",
+    "    return model\n",
+    "\n",
+    "# Check to make sure the data set is complete\n",
+    "assert len(X_tot) == len(y_tot)\n",
+    "\n",
+    "# This is the number of points that will be used in as the training data\n",
+    "dim=12\n",
+    "\n",
+    "# Separate the training data from the whole data set\n",
+    "X_train = X_tot[:dim]\n",
+    "y_train = y_tot[:dim]\n",
+    "\n",
+    "\n",
+    "# Generate the training data for the RNN, using a sequence of 2\n",
+    "rnn_input, rnn_training = format_data(y_train, 2)\n",
+    "\n",
+    "\n",
+    "# Create a recurrent neural network in Keras and produce a summary of the \n",
+    "# machine learning model\n",
+    "# Change the method name to reflect which network you want to use\n",
+    "model = dnn2_gru2(length_of_sequences = 2)\n",
+    "model.summary()\n",
+    "\n",
+    "# Start the timer.  Want to time training+testing\n",
+    "start = timer()\n",
+    "# Fit the model using the training data genenerated above using 150 training iterations and a 5%\n",
+    "# validation split.  Setting verbose to True prints information about each training iteration.\n",
+    "hist = model.fit(rnn_input, rnn_training, batch_size=None, epochs=150, \n",
+    "                 verbose=True,validation_split=0.05)\n",
+    "\n",
+    "\n",
+    "# This section plots the training loss and the validation loss as a function of training iteration.\n",
+    "# This is not required for analyzing the couple cluster data but can help determine if the network is\n",
+    "# being overtrained.\n",
+    "for label in [\"loss\",\"val_loss\"]:\n",
+    "    plt.plot(hist.history[label],label=label)\n",
+    "\n",
+    "plt.ylabel(\"loss\")\n",
+    "plt.xlabel(\"epoch\")\n",
+    "plt.title(\"The final validation loss: {}\".format(hist.history[\"val_loss\"][-1]))\n",
+    "plt.legend()\n",
+    "plt.show()\n",
+    "\n",
+    "# Use the trained neural network to predict more points of the data set\n",
+    "test_rnn(X_tot, y_tot, X_tot[0], X_tot[dim-1])\n",
+    "# Stop the timer and calculate the total time needed.\n",
+    "end = timer()\n",
+    "print('Time: ', end-start)\n",
+    "\n",
+    "\n",
+    "# ### Training Recurrent Neural Networks in the Standard Way (i.e. learning the relationship between the X and Y data)\n",
+    "# \n",
+    "# Finally, comparing the performace of a recurrent neural network using the standard data formatting to the performance of the network with time sequence data formatting shows the benefit of this type of data formatting with extrapolation.\n",
+    "\n",
+    "# Check to make sure the data set is complete\n",
+    "assert len(X_tot) == len(y_tot)\n",
+    "\n",
+    "# This is the number of points that will be used in as the training data\n",
+    "dim=12\n",
+    "\n",
+    "# Separate the training data from the whole data set\n",
+    "X_train = X_tot[:dim]\n",
+    "y_train = y_tot[:dim]\n",
+    "\n",
+    "# Reshape the data for Keras specifications\n",
+    "X_train = X_train.reshape((dim, 1))\n",
+    "y_train = y_train.reshape((dim, 1))\n",
+    "\n",
+    "\n",
+    "# Create a recurrent neural network in Keras and produce a summary of the \n",
+    "# machine learning model\n",
+    "# Set the sequence length to 1 for regular data formatting \n",
+    "model = rnn(length_of_sequences = 1)\n",
+    "model.summary()\n",
+    "\n",
+    "# Start the timer.  Want to time training+testing\n",
+    "start = timer()\n",
+    "# Fit the model using the training data genenerated above using 150 training iterations and a 5%\n",
+    "# validation split.  Setting verbose to True prints information about each training iteration.\n",
+    "hist = model.fit(X_train, y_train, batch_size=None, epochs=150, \n",
+    "                 verbose=True,validation_split=0.05)\n",
+    "\n",
+    "\n",
+    "# This section plots the training loss and the validation loss as a function of training iteration.\n",
+    "# This is not required for analyzing the couple cluster data but can help determine if the network is\n",
+    "# being overtrained.\n",
+    "for label in [\"loss\",\"val_loss\"]:\n",
+    "    plt.plot(hist.history[label],label=label)\n",
+    "\n",
+    "plt.ylabel(\"loss\")\n",
+    "plt.xlabel(\"epoch\")\n",
+    "plt.title(\"The final validation loss: {}\".format(hist.history[\"val_loss\"][-1]))\n",
+    "plt.legend()\n",
+    "plt.show()\n",
+    "\n",
+    "# Use the trained neural network to predict the remaining data points\n",
+    "X_pred = X_tot[dim:]\n",
+    "X_pred = X_pred.reshape((len(X_pred), 1))\n",
+    "y_model = model.predict(X_pred)\n",
+    "y_pred = np.concatenate((y_tot[:dim], y_model.flatten()))\n",
+    "\n",
+    "# Plot the known data set and the predicted data set.  The red box represents the region that was used\n",
+    "# for the training data.\n",
+    "fig, ax = plt.subplots()\n",
+    "ax.plot(X_tot, y_tot, label=\"true\", linewidth=3)\n",
+    "ax.plot(X_tot, y_pred, 'g-.',label=\"predicted\", linewidth=4)\n",
+    "ax.legend()\n",
+    "# Created a red region to represent the points used in the training data.\n",
+    "ax.axvspan(X_tot[0], X_tot[dim], alpha=0.25, color='red')\n",
+    "plt.show()\n",
+    "\n",
+    "# Stop the timer and calculate the total time needed.\n",
+    "end = timer()\n",
+    "print('Time: ', end-start)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Solving ODEs with Deep Learning\n",
+    "\n",
+    "The Universal Approximation Theorem states that a neural network can\n",
+    "approximate any function at a single hidden layer along with one input\n",
+    "and output layer to any given precision.  \n",
+    "\n",
+    "\n",
+    "\n",
+    "## Ordinary Differential Equations\n",
+    "\n",
+    "An ordinary differential equation (ODE) is an equation involving functions having one variable.\n",
+    "\n",
+    "In general, an ordinary differential equation looks like"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation} \\label{ode} \\tag{1}\n",
+    "f\\left(x, \\, g(x), \\, g'(x), \\, g''(x), \\, \\dots \\, , \\, g^{(n)}(x)\\right) = 0\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "where $g(x)$ is the function to find, and $g^{(n)}(x)$ is the $n$-th derivative of $g(x)$.\n",
+    "\n",
+    "The $f\\left(x, g(x), g'(x), g''(x), \\, \\dots \\, , g^{(n)}(x)\\right)$ is just a way to write that there is an expression involving $x$ and $g(x), \\ g'(x), \\ g''(x), \\, \\dots \\, , \\text{ and } g^{(n)}(x)$ on the left side of the equality sign in ([1](#ode)).\n",
+    "The highest order of derivative, that is the value of $n$, determines to the order of the equation.\n",
+    "The equation is referred to as a $n$-th order ODE.\n",
+    "Along with ([1](#ode)), some additional conditions of the function $g(x)$ are typically given\n",
+    "for the solution to be unique.\n",
+    "\n",
+    "\n",
+    "## The trial solution\n",
+    "\n",
+    "Let the trial solution $g_t(x)$ be"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation}\n",
+    "\tg_t(x) = h_1(x) + h_2(x,N(x,P))\n",
+    "\\label{_auto1} \\tag{2}\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "where $h_1(x)$ is a function that makes $g_t(x)$ satisfy a given set\n",
+    "of conditions, $N(x,P)$ a neural network with weights and biases\n",
+    "described by $P$ and $h_2(x, N(x,P))$ some expression involving the\n",
+    "neural network.  The role of the function $h_2(x, N(x,P))$, is to\n",
+    "ensure that the output from $N(x,P)$ is zero when $g_t(x)$ is\n",
+    "evaluated at the values of $x$ where the given conditions must be\n",
+    "satisfied.  The function $h_1(x)$ should alone make $g_t(x)$ satisfy\n",
+    "the conditions.\n",
+    "\n",
+    "But what about the network $N(x,P)$?\n",
+    "\n",
+    "\n",
+    "As described previously, an optimization method could be used to minimize the parameters of a neural network, that being its weights and biases, through backward propagation.\n",
+    "\n",
+    "\n",
+    "\n",
+    "## Minimization process\n",
+    "\n",
+    "For the minimization to be defined, we need to have a cost function at hand to minimize.\n",
+    "\n",
+    "It is given that $f\\left(x, \\, g(x), \\, g'(x), \\, g''(x), \\, \\dots \\, , \\, g^{(n)}(x)\\right)$ should be equal to zero in ([1](#ode)).\n",
+    "We can choose to consider the mean squared error as the cost function for an input $x$.\n",
+    "Since we are looking at one input, the cost function is just $f$ squared.\n",
+    "The cost function $c\\left(x, P \\right)$ can therefore be expressed as"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "C\\left(x, P\\right) = \\big(f\\left(x, \\, g(x), \\, g'(x), \\, g''(x), \\, \\dots \\, , \\, g^{(n)}(x)\\right)\\big)^2\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "If $N$ inputs are given as a vector $\\boldsymbol{x}$ with elements $x_i$ for $i = 1,\\dots,N$,\n",
+    "the cost function becomes"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation} \\label{cost} \\tag{3}\n",
+    "\tC\\left(\\boldsymbol{x}, P\\right) = \\frac{1}{N} \\sum_{i=1}^N \\big(f\\left(x_i, \\, g(x_i), \\, g'(x_i), \\, g''(x_i), \\, \\dots \\, , \\, g^{(n)}(x_i)\\right)\\big)^2\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The neural net should then find the parameters $P$ that minimizes the cost function in\n",
+    "([3](#cost)) for a set of $N$ training samples $x_i$.\n",
+    "\n",
+    "\n",
+    "## Minimizing the cost function using gradient descent and automatic differentiation\n",
+    "\n",
+    "To perform the minimization using gradient descent, the gradient of $C\\left(\\boldsymbol{x}, P\\right)$ is needed.\n",
+    "It might happen so that finding an analytical expression of the gradient of $C(\\boldsymbol{x}, P)$ from ([3](#cost)) gets too messy, depending on which cost function one desires to use.\n",
+    "\n",
+    "Luckily, there exists libraries that makes the job for us through automatic differentiation.\n",
+    "Automatic differentiation is a method of finding the derivatives numerically with very high precision.\n",
+    "\n",
+    "\n",
+    "\n",
+    "## Example: Exponential decay\n",
+    "\n",
+    "An exponential decay of a quantity $g(x)$ is described by the equation"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation} \\label{solve_expdec} \\tag{4}\n",
+    "  g'(x) = -\\gamma g(x)\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "with $g(0) = g_0$ for some chosen initial value $g_0$.\n",
+    "\n",
+    "The analytical solution of ([4](#solve_expdec)) is"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation}\n",
+    "  g(x) = g_0 \\exp\\left(-\\gamma x\\right)\n",
+    "\\label{_auto2} \\tag{5}\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Having an analytical solution at hand, it is possible to use it to compare how well a neural network finds a solution of ([4](#solve_expdec)).\n",
+    "\n",
+    "\n",
+    "\n",
+    "## The function to solve for\n",
+    "\n",
+    "The program will use a neural network to solve"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation} \\label{solveode} \\tag{6}\n",
+    "g'(x) = -\\gamma g(x)\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "where $g(0) = g_0$ with $\\gamma$ and $g_0$ being some chosen values.\n",
+    "\n",
+    "In this example, $\\gamma = 2$ and $g_0 = 10$.\n",
+    "\n",
+    "\n",
+    "## The trial solution\n",
+    "To begin with, a trial solution $g_t(t)$ must be chosen. A general trial solution for ordinary differential equations could be"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "g_t(x, P) = h_1(x) + h_2(x, N(x, P))\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "with $h_1(x)$ ensuring that $g_t(x)$ satisfies some conditions and $h_2(x,N(x, P))$ an expression involving $x$ and the output from the neural network $N(x,P)$ with $P $ being the collection of the weights and biases for each layer. For now, it is assumed that the network consists of one input layer, one hidden layer, and one output layer.\n",
+    "\n",
+    "\n",
+    "## Setup of Network\n",
+    "\n",
+    "In this network, there are no weights and bias at the input layer, so $P = \\{ P_{\\text{hidden}},  P_{\\text{output}} \\}$.\n",
+    "If there are $N_{\\text{hidden} }$ neurons in the hidden layer, then $P_{\\text{hidden}}$ is a $N_{\\text{hidden} } \\times (1 + N_{\\text{input}})$ matrix, given that there are $N_{\\text{input}}$ neurons in the input layer.\n",
+    "\n",
+    "The first column in $P_{\\text{hidden} }$ represents the bias for each neuron in the hidden layer and the second column represents the weights for each neuron in the hidden layer from the input layer.\n",
+    "If there are $N_{\\text{output} }$ neurons in the output layer, then $P_{\\text{output}} $ is a $N_{\\text{output} } \\times (1 + N_{\\text{hidden} })$ matrix.\n",
+    "\n",
+    "Its first column represents the bias of each neuron and the remaining columns represents the weights to each neuron.\n",
+    "\n",
+    "It is given that $g(0) = g_0$. The trial solution must fulfill this condition to be a proper solution of ([6](#solveode)). A possible way to ensure that $g_t(0, P) = g_0$, is to let $F(N(x,P)) = x \\cdot N(x,P)$ and $A(x) = g_0$. This gives the following trial solution:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation} \\label{trial} \\tag{7}\n",
+    "g_t(x, P) = g_0 + x \\cdot N(x, P)\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Reformulating the problem\n",
+    "\n",
+    "We wish that our neural network manages to minimize a given cost function.\n",
+    "\n",
+    "A reformulation of out equation, ([6](#solveode)), must therefore be done,\n",
+    "such that it describes the problem a neural network can solve for.\n",
+    "\n",
+    "The neural network must find the set of weights and biases $P$ such that the trial solution in ([7](#trial)) satisfies ([6](#solveode)).\n",
+    "\n",
+    "The trial solution"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "g_t(x, P) = g_0 + x \\cdot N(x, P)\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "has been chosen such that it already solves the condition $g(0) = g_0$. What remains, is to find $P$ such that"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation} \\label{nnmin} \\tag{8}\n",
+    "g_t'(x, P) = - \\gamma g_t(x, P)\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "is fulfilled as *best as possible*.\n",
+    "\n",
+    "\n",
+    "## More technicalities\n",
+    "\n",
+    "The left hand side and right hand side of ([8](#nnmin)) must be computed separately, and then the neural network must choose weights and biases, contained in $P$, such that the sides are equal as best as possible.\n",
+    "This means that the absolute or squared difference between the sides must be as close to zero, ideally equal to zero.\n",
+    "In this case, the difference squared shows to be an appropriate measurement of how erroneous the trial solution is with respect to $P$ of the neural network.\n",
+    "\n",
+    "This gives the following cost function our neural network must solve for:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\min_{P}\\Big\\{ \\big(g_t'(x, P) - ( -\\gamma g_t(x, P) \\big)^2 \\Big\\}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "(the notation $\\min_{P}\\{ f(x, P) \\}$ means that we desire to find $P$ that yields the minimum of $f(x, P)$)\n",
+    "\n",
+    "or, in terms of weights and biases for the hidden and output layer in our network:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\min_{P_{\\text{hidden} }, \\ P_{\\text{output} }}\\Big\\{ \\big(g_t'(x, \\{ P_{\\text{hidden} }, P_{\\text{output} }\\}) - ( -\\gamma g_t(x, \\{ P_{\\text{hidden} }, P_{\\text{output} }\\}) \\big)^2 \\Big\\}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "for an input value $x$.\n",
+    "\n",
+    "\n",
+    "## More details\n",
+    "\n",
+    "If the neural network evaluates $g_t(x, P)$ at more values for $x$, say $N$ values $x_i$ for $i = 1, \\dots, N$, then the *total* error to minimize becomes"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation} \\label{min} \\tag{9}\n",
+    "\\min_{P}\\Big\\{\\frac{1}{N} \\sum_{i=1}^N  \\big(g_t'(x_i, P) - ( -\\gamma g_t(x_i, P) \\big)^2 \\Big\\}\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Letting $\\boldsymbol{x}$ be a vector with elements $x_i$ and $C(\\boldsymbol{x}, P) = \\frac{1}{N} \\sum_i  \\big(g_t'(x_i, P) - ( -\\gamma g_t(x_i, P) \\big)^2$ denote the cost function, the minimization problem that our network must solve, becomes"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\min_{P} C(\\boldsymbol{x}, P)\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "In terms of $P_{\\text{hidden} }$ and $P_{\\text{output} }$, this could also be expressed as\n",
+    "\n",
+    "$$\n",
+    "\\min_{P_{\\text{hidden} }, \\ P_{\\text{output} }} C(\\boldsymbol{x}, \\{P_{\\text{hidden} }, P_{\\text{output} }\\})\n",
+    "$$\n",
+    "\n",
+    "\n",
+    "## A possible implementation of a neural network\n",
+    "\n",
+    "For simplicity, it is assumed that the input is an array $\\boldsymbol{x} = (x_1, \\dots, x_N)$ with $N$ elements. It is at these points the neural network should find $P$ such that it fulfills ([9](#min)).\n",
+    "\n",
+    "First, the neural network must feed forward the inputs.\n",
+    "This means that $\\boldsymbol{x}s$ must be passed through an input layer, a hidden layer and a output layer. The input layer in this case, does not need to process the data any further.\n",
+    "The input layer will consist of $N_{\\text{input} }$ neurons, passing its element to each neuron in the hidden layer.  The number of neurons in the hidden layer will be $N_{\\text{hidden} }$.\n",
+    "\n",
+    "\n",
+    "## Technicalities\n",
+    "\n",
+    "For the $i$-th in the hidden layer with weight $w_i^{\\text{hidden} }$ and bias $b_i^{\\text{hidden} }$, the weighting from the $j$-th neuron at the input layer is:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\begin{aligned}\n",
+    "z_{i,j}^{\\text{hidden}} &= b_i^{\\text{hidden}} + w_i^{\\text{hidden}}x_j \\\\\n",
+    "&=\n",
+    "\\begin{pmatrix}\n",
+    "b_i^{\\text{hidden}} & w_i^{\\text{hidden}}\n",
+    "\\end{pmatrix}\n",
+    "\\begin{pmatrix}\n",
+    "1 \\\\\n",
+    "x_j\n",
+    "\\end{pmatrix}\n",
+    "\\end{aligned}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Final technicalities I\n",
+    "\n",
+    "The result after weighting the inputs at the $i$-th hidden neuron can be written as a vector:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\begin{aligned}\n",
+    "\\boldsymbol{z}_{i}^{\\text{hidden}} &= \\Big( b_i^{\\text{hidden}} + w_i^{\\text{hidden}}x_1 , \\ b_i^{\\text{hidden}} + w_i^{\\text{hidden}} x_2, \\ \\dots \\, , \\ b_i^{\\text{hidden}} + w_i^{\\text{hidden}} x_N\\Big)  \\\\\n",
+    "&=\n",
+    "\\begin{pmatrix}\n",
+    " b_i^{\\text{hidden}}  & w_i^{\\text{hidden}}\n",
+    "\\end{pmatrix}\n",
+    "\\begin{pmatrix}\n",
+    "1  & 1 & \\dots & 1 \\\\\n",
+    "x_1 & x_2 & \\dots & x_N\n",
+    "\\end{pmatrix} \\\\\n",
+    "&= \\boldsymbol{p}_{i, \\text{hidden}}^T X\n",
+    "\\end{aligned}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Final technicalities II\n",
+    "\n",
+    "The vector $\\boldsymbol{p}_{i, \\text{hidden}}^T$ constitutes each row in $P_{\\text{hidden} }$, which contains the weights for the neural network to minimize according to ([9](#min)).\n",
+    "\n",
+    "After having found $\\boldsymbol{z}_{i}^{\\text{hidden}} $ for every $i$-th neuron within the hidden layer, the vector will be sent to an activation function $a_i(\\boldsymbol{z})$.\n",
+    "\n",
+    "In this example, the sigmoid function has been chosen to be the activation function for each hidden neuron:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "f(z) = \\frac{1}{1 + \\exp{(-z)}}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "It is possible to use other activations functions for the hidden layer also.\n",
+    "\n",
+    "The output $\\boldsymbol{x}_i^{\\text{hidden}}$ from each $i$-th hidden neuron is:\n",
+    "\n",
+    "$$\n",
+    "\\boldsymbol{x}_i^{\\text{hidden} } = f\\big(  \\boldsymbol{z}_{i}^{\\text{hidden}} \\big)\n",
+    "$$\n",
+    "\n",
+    "The outputs $\\boldsymbol{x}_i^{\\text{hidden} } $ are then sent to the output layer.\n",
+    "\n",
+    "The output layer consists of one neuron in this case, and combines the\n",
+    "output from each of the neurons in the hidden layers. The output layer\n",
+    "combines the results from the hidden layer using some weights $w_i^{\\text{output}}$\n",
+    "and biases $b_i^{\\text{output}}$. In this case,\n",
+    "it is assumes that the number of neurons in the output layer is one.\n",
+    "\n",
+    "\n",
+    "## Final technicalities III\n",
+    "\n",
+    "\n",
+    "The procedure of weighting the output neuron $j$ in the hidden layer to the $i$-th neuron in the output layer is similar as for the hidden layer described previously."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\begin{aligned}\n",
+    "z_{1,j}^{\\text{output}} & =\n",
+    "\\begin{pmatrix}\n",
+    "b_1^{\\text{output}} & \\boldsymbol{w}_1^{\\text{output}}\n",
+    "\\end{pmatrix}\n",
+    "\\begin{pmatrix}\n",
+    "1 \\\\\n",
+    "\\boldsymbol{x}_j^{\\text{hidden}}\n",
+    "\\end{pmatrix}\n",
+    "\\end{aligned}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Final technicalities IV\n",
+    "\n",
+    "Expressing $z_{1,j}^{\\text{output}}$ as a vector gives the following way of weighting the inputs from the hidden layer:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\boldsymbol{z}_{1}^{\\text{output}} =\n",
+    "\\begin{pmatrix}\n",
+    "b_1^{\\text{output}} & \\boldsymbol{w}_1^{\\text{output}}\n",
+    "\\end{pmatrix}\n",
+    "\\begin{pmatrix}\n",
+    "1  & 1 & \\dots & 1 \\\\\n",
+    "\\boldsymbol{x}_1^{\\text{hidden}} & \\boldsymbol{x}_2^{\\text{hidden}} & \\dots & \\boldsymbol{x}_N^{\\text{hidden}}\n",
+    "\\end{pmatrix}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "In this case we seek a continuous range of values since we are approximating a function. This means that after computing $\\boldsymbol{z}_{1}^{\\text{output}}$ the neural network has finished its feed forward step, and $\\boldsymbol{z}_{1}^{\\text{output}}$ is the final output of the network.\n",
+    "\n",
+    "\n",
+    "## Back propagation\n",
+    "\n",
+    "The next step is to decide how the parameters should be changed such that they minimize the cost function.\n",
+    "\n",
+    "The chosen cost function for this problem is"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "C(\\boldsymbol{x}, P) = \\frac{1}{N} \\sum_i  \\big(g_t'(x_i, P) - ( -\\gamma g_t(x_i, P) \\big)^2\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "In order to minimize the cost function, an optimization method must be chosen.\n",
+    "\n",
+    "Here, gradient descent with a constant step size has been chosen.\n",
+    "\n",
+    "\n",
+    "## Gradient descent\n",
+    "\n",
+    "The idea of the gradient descent algorithm is to update parameters in\n",
+    "a direction where the cost function decreases goes to a minimum.\n",
+    "\n",
+    "In general, the update of some parameters $\\boldsymbol{\\omega}$ given a cost\n",
+    "function defined by some weights $\\boldsymbol{\\omega}$, $C(\\boldsymbol{x},\n",
+    "\\boldsymbol{\\omega})$, goes as follows:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\boldsymbol{\\omega}_{\\text{new} } = \\boldsymbol{\\omega} - \\lambda \\nabla_{\\boldsymbol{\\omega}} C(\\boldsymbol{x}, \\boldsymbol{\\omega})\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "for a number of iterations or until $ \\big|\\big| \\boldsymbol{\\omega}_{\\text{new} } - \\boldsymbol{\\omega} \\big|\\big|$ becomes smaller than some given tolerance.\n",
+    "\n",
+    "The value of $\\lambda$ decides how large steps the algorithm must take\n",
+    "in the direction of $ \\nabla_{\\boldsymbol{\\omega}} C(\\boldsymbol{x}, \\boldsymbol{\\omega})$.\n",
+    "The notation $\\nabla_{\\boldsymbol{\\omega}}$ express the gradient with respect\n",
+    "to the elements in $\\boldsymbol{\\omega}$.\n",
+    "\n",
+    "In our case, we have to minimize the cost function $C(\\boldsymbol{x}, P)$ with\n",
+    "respect to the two sets of weights and biases, that is for the hidden\n",
+    "layer $P_{\\text{hidden} }$ and for the output layer $P_{\\text{output}\n",
+    "}$ .\n",
+    "\n",
+    "This means that $P_{\\text{hidden} }$ and $P_{\\text{output} }$ is updated by"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\begin{aligned}\n",
+    "P_{\\text{hidden},\\text{new}} &= P_{\\text{hidden}} - \\lambda \\nabla_{P_{\\text{hidden}}} C(\\boldsymbol{x}, P)  \\\\\n",
+    "P_{\\text{output},\\text{new}} &= P_{\\text{output}} - \\lambda \\nabla_{P_{\\text{output}}} C(\\boldsymbol{x}, P)\n",
+    "\\end{aligned}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## The code for solving the ODE"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "import autograd.numpy as np\n",
+    "from autograd import grad, elementwise_grad\n",
+    "import autograd.numpy.random as npr\n",
+    "from matplotlib import pyplot as plt\n",
+    "\n",
+    "def sigmoid(z):\n",
+    "    return 1/(1 + np.exp(-z))\n",
+    "\n",
+    "# Assuming one input, hidden, and output layer\n",
+    "def neural_network(params, x):\n",
+    "\n",
+    "    # Find the weights (including and biases) for the hidden and output layer.\n",
+    "    # Assume that params is a list of parameters for each layer.\n",
+    "    # The biases are the first element for each array in params,\n",
+    "    # and the weights are the remaning elements in each array in params.\n",
+    "\n",
+    "    w_hidden = params[0]\n",
+    "    w_output = params[1]\n",
+    "\n",
+    "    # Assumes input x being an one-dimensional array\n",
+    "    num_values = np.size(x)\n",
+    "    x = x.reshape(-1, num_values)\n",
+    "\n",
+    "    # Assume that the input layer does nothing to the input x\n",
+    "    x_input = x\n",
+    "\n",
+    "    ## Hidden layer:\n",
+    "\n",
+    "    # Add a row of ones to include bias\n",
+    "    x_input = np.concatenate((np.ones((1,num_values)), x_input ), axis = 0)\n",
+    "\n",
+    "    z_hidden = np.matmul(w_hidden, x_input)\n",
+    "    x_hidden = sigmoid(z_hidden)\n",
+    "\n",
+    "    ## Output layer:\n",
+    "\n",
+    "    # Include bias:\n",
+    "    x_hidden = np.concatenate((np.ones((1,num_values)), x_hidden ), axis = 0)\n",
+    "\n",
+    "    z_output = np.matmul(w_output, x_hidden)\n",
+    "    x_output = z_output\n",
+    "\n",
+    "    return x_output\n",
+    "\n",
+    "# The trial solution using the deep neural network:\n",
+    "def g_trial(x,params, g0 = 10):\n",
+    "    return g0 + x*neural_network(params,x)\n",
+    "\n",
+    "# The right side of the ODE:\n",
+    "def g(x, g_trial, gamma = 2):\n",
+    "    return -gamma*g_trial\n",
+    "\n",
+    "# The cost function:\n",
+    "def cost_function(P, x):\n",
+    "\n",
+    "    # Evaluate the trial function with the current parameters P\n",
+    "    g_t = g_trial(x,P)\n",
+    "\n",
+    "    # Find the derivative w.r.t x of the neural network\n",
+    "    d_net_out = elementwise_grad(neural_network,1)(P,x)\n",
+    "\n",
+    "    # Find the derivative w.r.t x of the trial function\n",
+    "    d_g_t = elementwise_grad(g_trial,0)(x,P)\n",
+    "\n",
+    "    # The right side of the ODE\n",
+    "    func = g(x, g_t)\n",
+    "\n",
+    "    err_sqr = (d_g_t - func)**2\n",
+    "    cost_sum = np.sum(err_sqr)\n",
+    "\n",
+    "    return cost_sum / np.size(err_sqr)\n",
+    "\n",
+    "# Solve the exponential decay ODE using neural network with one input, hidden, and output layer\n",
+    "def solve_ode_neural_network(x, num_neurons_hidden, num_iter, lmb):\n",
+    "    ## Set up initial weights and biases\n",
+    "\n",
+    "    # For the hidden layer\n",
+    "    p0 = npr.randn(num_neurons_hidden, 2 )\n",
+    "\n",
+    "    # For the output layer\n",
+    "    p1 = npr.randn(1, num_neurons_hidden + 1 ) # +1 since bias is included\n",
+    "\n",
+    "    P = [p0, p1]\n",
+    "\n",
+    "    print('Initial cost: %g'%cost_function(P, x))\n",
+    "\n",
+    "    ## Start finding the optimal weights using gradient descent\n",
+    "\n",
+    "    # Find the Python function that represents the gradient of the cost function\n",
+    "    # w.r.t the 0-th input argument -- that is the weights and biases in the hidden and output layer\n",
+    "    cost_function_grad = grad(cost_function,0)\n",
+    "\n",
+    "    # Let the update be done num_iter times\n",
+    "    for i in range(num_iter):\n",
+    "        # Evaluate the gradient at the current weights and biases in P.\n",
+    "        # The cost_grad consist now of two arrays;\n",
+    "        # one for the gradient w.r.t P_hidden and\n",
+    "        # one for the gradient w.r.t P_output\n",
+    "        cost_grad =  cost_function_grad(P, x)\n",
+    "\n",
+    "        P[0] = P[0] - lmb * cost_grad[0]\n",
+    "        P[1] = P[1] - lmb * cost_grad[1]\n",
+    "\n",
+    "    print('Final cost: %g'%cost_function(P, x))\n",
+    "\n",
+    "    return P\n",
+    "\n",
+    "def g_analytic(x, gamma = 2, g0 = 10):\n",
+    "    return g0*np.exp(-gamma*x)\n",
+    "\n",
+    "# Solve the given problem\n",
+    "if __name__ == '__main__':\n",
+    "    # Set seed such that the weight are initialized\n",
+    "    # with same weights and biases for every run.\n",
+    "    npr.seed(15)\n",
+    "\n",
+    "    ## Decide the vales of arguments to the function to solve\n",
+    "    N = 10\n",
+    "    x = np.linspace(0, 1, N)\n",
+    "\n",
+    "    ## Set up the initial parameters\n",
+    "    num_hidden_neurons = 10\n",
+    "    num_iter = 10000\n",
+    "    lmb = 0.001\n",
+    "\n",
+    "    # Use the network\n",
+    "    P = solve_ode_neural_network(x, num_hidden_neurons, num_iter, lmb)\n",
+    "\n",
+    "    # Print the deviation from the trial solution and true solution\n",
+    "    res = g_trial(x,P)\n",
+    "    res_analytical = g_analytic(x)\n",
+    "\n",
+    "    print('Max absolute difference: %g'%np.max(np.abs(res - res_analytical)))\n",
+    "\n",
+    "    # Plot the results\n",
+    "    plt.figure(figsize=(10,10))\n",
+    "\n",
+    "    plt.title('Performance of neural network solving an ODE compared to the analytical solution')\n",
+    "    plt.plot(x, res_analytical)\n",
+    "    plt.plot(x, res[0,:])\n",
+    "    plt.legend(['analytical','nn'])\n",
+    "    plt.xlabel('x')\n",
+    "    plt.ylabel('g(x)')\n",
+    "    plt.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## The network with one input layer, specified number of hidden layers, and one output layer\n",
+    "\n",
+    "It is also possible to extend the construction of our network into a more general one, allowing the network to contain more than one hidden layers.\n",
+    "\n",
+    "The number of neurons within each hidden layer are given as a list of integers in the program below."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "import autograd.numpy as np\n",
+    "from autograd import grad, elementwise_grad\n",
+    "import autograd.numpy.random as npr\n",
+    "from matplotlib import pyplot as plt\n",
+    "\n",
+    "def sigmoid(z):\n",
+    "    return 1/(1 + np.exp(-z))\n",
+    "\n",
+    "# The neural network with one input layer and one output layer,\n",
+    "# but with number of hidden layers specified by the user.\n",
+    "def deep_neural_network(deep_params, x):\n",
+    "    # N_hidden is the number of hidden layers\n",
+    "\n",
+    "    N_hidden = np.size(deep_params) - 1 # -1 since params consists of\n",
+    "                                        # parameters to all the hidden\n",
+    "                                        # layers AND the output layer.\n",
+    "\n",
+    "    # Assumes input x being an one-dimensional array\n",
+    "    num_values = np.size(x)\n",
+    "    x = x.reshape(-1, num_values)\n",
+    "\n",
+    "    # Assume that the input layer does nothing to the input x\n",
+    "    x_input = x\n",
+    "\n",
+    "    # Due to multiple hidden layers, define a variable referencing to the\n",
+    "    # output of the previous layer:\n",
+    "    x_prev = x_input\n",
+    "\n",
+    "    ## Hidden layers:\n",
+    "\n",
+    "    for l in range(N_hidden):\n",
+    "        # From the list of parameters P; find the correct weigths and bias for this layer\n",
+    "        w_hidden = deep_params[l]\n",
+    "\n",
+    "        # Add a row of ones to include bias\n",
+    "        x_prev = np.concatenate((np.ones((1,num_values)), x_prev ), axis = 0)\n",
+    "\n",
+    "        z_hidden = np.matmul(w_hidden, x_prev)\n",
+    "        x_hidden = sigmoid(z_hidden)\n",
+    "\n",
+    "        # Update x_prev such that next layer can use the output from this layer\n",
+    "        x_prev = x_hidden\n",
+    "\n",
+    "    ## Output layer:\n",
+    "\n",
+    "    # Get the weights and bias for this layer\n",
+    "    w_output = deep_params[-1]\n",
+    "\n",
+    "    # Include bias:\n",
+    "    x_prev = np.concatenate((np.ones((1,num_values)), x_prev), axis = 0)\n",
+    "\n",
+    "    z_output = np.matmul(w_output, x_prev)\n",
+    "    x_output = z_output\n",
+    "\n",
+    "    return x_output\n",
+    "\n",
+    "# The trial solution using the deep neural network:\n",
+    "def g_trial_deep(x,params, g0 = 10):\n",
+    "    return g0 + x*deep_neural_network(params, x)\n",
+    "\n",
+    "# The right side of the ODE:\n",
+    "def g(x, g_trial, gamma = 2):\n",
+    "    return -gamma*g_trial\n",
+    "\n",
+    "# The same cost function as before, but calls deep_neural_network instead.\n",
+    "def cost_function_deep(P, x):\n",
+    "\n",
+    "    # Evaluate the trial function with the current parameters P\n",
+    "    g_t = g_trial_deep(x,P)\n",
+    "\n",
+    "    # Find the derivative w.r.t x of the neural network\n",
+    "    d_net_out = elementwise_grad(deep_neural_network,1)(P,x)\n",
+    "\n",
+    "    # Find the derivative w.r.t x of the trial function\n",
+    "    d_g_t = elementwise_grad(g_trial_deep,0)(x,P)\n",
+    "\n",
+    "    # The right side of the ODE\n",
+    "    func = g(x, g_t)\n",
+    "\n",
+    "    err_sqr = (d_g_t - func)**2\n",
+    "    cost_sum = np.sum(err_sqr)\n",
+    "\n",
+    "    return cost_sum / np.size(err_sqr)\n",
+    "\n",
+    "# Solve the exponential decay ODE using neural network with one input and one output layer,\n",
+    "# but with specified number of hidden layers from the user.\n",
+    "def solve_ode_deep_neural_network(x, num_neurons, num_iter, lmb):\n",
+    "    # num_hidden_neurons is now a list of number of neurons within each hidden layer\n",
+    "\n",
+    "    # The number of elements in the list num_hidden_neurons thus represents\n",
+    "    # the number of hidden layers.\n",
+    "\n",
+    "    # Find the number of hidden layers:\n",
+    "    N_hidden = np.size(num_neurons)\n",
+    "\n",
+    "    ## Set up initial weights and biases\n",
+    "\n",
+    "    # Initialize the list of parameters:\n",
+    "    P = [None]*(N_hidden + 1) # + 1 to include the output layer\n",
+    "\n",
+    "    P[0] = npr.randn(num_neurons[0], 2 )\n",
+    "    for l in range(1,N_hidden):\n",
+    "        P[l] = npr.randn(num_neurons[l], num_neurons[l-1] + 1) # +1 to include bias\n",
+    "\n",
+    "    # For the output layer\n",
+    "    P[-1] = npr.randn(1, num_neurons[-1] + 1 ) # +1 since bias is included\n",
+    "\n",
+    "    print('Initial cost: %g'%cost_function_deep(P, x))\n",
+    "\n",
+    "    ## Start finding the optimal weights using gradient descent\n",
+    "\n",
+    "    # Find the Python function that represents the gradient of the cost function\n",
+    "    # w.r.t the 0-th input argument -- that is the weights and biases in the hidden and output layer\n",
+    "    cost_function_deep_grad = grad(cost_function_deep,0)\n",
+    "\n",
+    "    # Let the update be done num_iter times\n",
+    "    for i in range(num_iter):\n",
+    "        # Evaluate the gradient at the current weights and biases in P.\n",
+    "        # The cost_grad consist now of N_hidden + 1 arrays; the gradient w.r.t the weights and biases\n",
+    "        # in the hidden layers and output layers evaluated at x.\n",
+    "        cost_deep_grad =  cost_function_deep_grad(P, x)\n",
+    "\n",
+    "        for l in range(N_hidden+1):\n",
+    "            P[l] = P[l] - lmb * cost_deep_grad[l]\n",
+    "\n",
+    "    print('Final cost: %g'%cost_function_deep(P, x))\n",
+    "\n",
+    "    return P\n",
+    "\n",
+    "def g_analytic(x, gamma = 2, g0 = 10):\n",
+    "    return g0*np.exp(-gamma*x)\n",
+    "\n",
+    "# Solve the given problem\n",
+    "if __name__ == '__main__':\n",
+    "    npr.seed(15)\n",
+    "\n",
+    "    ## Decide the vales of arguments to the function to solve\n",
+    "    N = 10\n",
+    "    x = np.linspace(0, 1, N)\n",
+    "\n",
+    "    ## Set up the initial parameters\n",
+    "    num_hidden_neurons = np.array([10,10])\n",
+    "    num_iter = 10000\n",
+    "    lmb = 0.001\n",
+    "\n",
+    "    P = solve_ode_deep_neural_network(x, num_hidden_neurons, num_iter, lmb)\n",
+    "\n",
+    "    res = g_trial_deep(x,P)\n",
+    "    res_analytical = g_analytic(x)\n",
+    "\n",
+    "    plt.figure(figsize=(10,10))\n",
+    "\n",
+    "    plt.title('Performance of a deep neural network solving an ODE compared to the analytical solution')\n",
+    "    plt.plot(x, res_analytical)\n",
+    "    plt.plot(x, res[0,:])\n",
+    "    plt.legend(['analytical','dnn'])\n",
+    "    plt.ylabel('g(x)')\n",
+    "    plt.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Example: Population growth\n",
+    "\n",
+    "A logistic model of population growth assumes that a population converges toward an equilibrium.\n",
+    "The population growth can be modeled by"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation} \\label{log} \\tag{10}\n",
+    "\tg'(t) = \\alpha g(t)(A - g(t))\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "where $g(t)$ is the population density at time $t$, $\\alpha > 0$ the growth rate and $A > 0$ is the maximum population number in the environment.\n",
+    "Also, at $t = 0$ the population has the size $g(0) = g_0$, where $g_0$ is some chosen constant.\n",
+    "\n",
+    "In this example, similar network as for the exponential decay using Autograd has been used to solve the equation. However, as the implementation might suffer from e.g numerical instability\n",
+    "and high execution time (this might be more apparent in the examples solving PDEs),\n",
+    "using a library like  TensorFlow is recommended.\n",
+    "Here, we stay with a more simple approach and implement for comparison, the simple forward Euler method.\n",
+    "\n",
+    "\n",
+    "## Setting up the problem\n",
+    "\n",
+    "Here, we will model a population $g(t)$ in an environment having carrying capacity $A$.\n",
+    "The population follows the model"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation} \\label{solveode_population} \\tag{11}\n",
+    "g'(t) = \\alpha g(t)(A - g(t))\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "where $g(0) = g_0$.\n",
+    "\n",
+    "In this example, we let $\\alpha = 2$, $A = 1$, and $g_0 = 1.2$.\n",
+    "\n",
+    "\n",
+    "## The trial solution\n",
+    "\n",
+    "We will get a slightly different trial solution, as the boundary conditions are different\n",
+    "compared to the case for exponential decay.\n",
+    "\n",
+    "A possible trial solution satisfying the condition $g(0) = g_0$ could be\n",
+    "\n",
+    "$$\n",
+    "h_1(t) = g_0 + t \\cdot N(t,P)\n",
+    "$$\n",
+    "\n",
+    "with $N(t,P)$ being the output from the neural network with weights and biases for each layer collected in the set $P$.\n",
+    "\n",
+    "The analytical solution is\n",
+    "\n",
+    "$$\n",
+    "g(t) = \\frac{Ag_0}{g_0 + (A - g_0)\\exp(-\\alpha A t)}\n",
+    "$$\n",
+    "\n",
+    "\n",
+    "## The program using Autograd\n",
+    "\n",
+    "The network will be the similar as for the exponential decay example, but with some small modifications for our problem."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "import autograd.numpy as np\n",
+    "from autograd import grad, elementwise_grad\n",
+    "import autograd.numpy.random as npr\n",
+    "from matplotlib import pyplot as plt\n",
+    "\n",
+    "def sigmoid(z):\n",
+    "    return 1/(1 + np.exp(-z))\n",
+    "\n",
+    "# Function to get the parameters.\n",
+    "# Done such that one can easily change the paramaters after one's liking.\n",
+    "def get_parameters():\n",
+    "    alpha = 2\n",
+    "    A = 1\n",
+    "    g0 = 1.2\n",
+    "    return alpha, A, g0\n",
+    "\n",
+    "def deep_neural_network(P, x):\n",
+    "    # N_hidden is the number of hidden layers\n",
+    "    N_hidden = np.size(P) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n",
+    "\n",
+    "    # Assumes input x being an one-dimensional array\n",
+    "    num_values = np.size(x)\n",
+    "    x = x.reshape(-1, num_values)\n",
+    "\n",
+    "    # Assume that the input layer does nothing to the input x\n",
+    "    x_input = x\n",
+    "\n",
+    "    # Due to multiple hidden layers, define a variable referencing to the\n",
+    "    # output of the previous layer:\n",
+    "    x_prev = x_input\n",
+    "\n",
+    "    ## Hidden layers:\n",
+    "\n",
+    "    for l in range(N_hidden):\n",
+    "        # From the list of parameters P; find the correct weigths and bias for this layer\n",
+    "        w_hidden = P[l]\n",
+    "\n",
+    "        # Add a row of ones to include bias\n",
+    "        x_prev = np.concatenate((np.ones((1,num_values)), x_prev ), axis = 0)\n",
+    "\n",
+    "        z_hidden = np.matmul(w_hidden, x_prev)\n",
+    "        x_hidden = sigmoid(z_hidden)\n",
+    "\n",
+    "        # Update x_prev such that next layer can use the output from this layer\n",
+    "        x_prev = x_hidden\n",
+    "\n",
+    "    ## Output layer:\n",
+    "\n",
+    "    # Get the weights and bias for this layer\n",
+    "    w_output = P[-1]\n",
+    "\n",
+    "    # Include bias:\n",
+    "    x_prev = np.concatenate((np.ones((1,num_values)), x_prev), axis = 0)\n",
+    "\n",
+    "    z_output = np.matmul(w_output, x_prev)\n",
+    "    x_output = z_output\n",
+    "\n",
+    "    return x_output\n",
+    "\n",
+    "\n",
+    "def cost_function_deep(P, x):\n",
+    "\n",
+    "    # Evaluate the trial function with the current parameters P\n",
+    "    g_t = g_trial_deep(x,P)\n",
+    "\n",
+    "    # Find the derivative w.r.t x of the trial function\n",
+    "    d_g_t = elementwise_grad(g_trial_deep,0)(x,P)\n",
+    "\n",
+    "    # The right side of the ODE\n",
+    "    func = f(x, g_t)\n",
+    "\n",
+    "    err_sqr = (d_g_t - func)**2\n",
+    "    cost_sum = np.sum(err_sqr)\n",
+    "\n",
+    "    return cost_sum / np.size(err_sqr)\n",
+    "\n",
+    "# The right side of the ODE:\n",
+    "def f(x, g_trial):\n",
+    "    alpha,A, g0 = get_parameters()\n",
+    "    return alpha*g_trial*(A - g_trial)\n",
+    "\n",
+    "# The trial solution using the deep neural network:\n",
+    "def g_trial_deep(x, params):\n",
+    "    alpha,A, g0 = get_parameters()\n",
+    "    return g0 + x*deep_neural_network(params,x)\n",
+    "\n",
+    "# The analytical solution:\n",
+    "def g_analytic(t):\n",
+    "    alpha,A, g0 = get_parameters()\n",
+    "    return A*g0/(g0 + (A - g0)*np.exp(-alpha*A*t))\n",
+    "\n",
+    "def solve_ode_deep_neural_network(x, num_neurons, num_iter, lmb):\n",
+    "    # num_hidden_neurons is now a list of number of neurons within each hidden layer\n",
+    "\n",
+    "    # Find the number of hidden layers:\n",
+    "    N_hidden = np.size(num_neurons)\n",
+    "\n",
+    "    ## Set up initial weigths and biases\n",
+    "\n",
+    "    # Initialize the list of parameters:\n",
+    "    P = [None]*(N_hidden + 1) # + 1 to include the output layer\n",
+    "\n",
+    "    P[0] = npr.randn(num_neurons[0], 2 )\n",
+    "    for l in range(1,N_hidden):\n",
+    "        P[l] = npr.randn(num_neurons[l], num_neurons[l-1] + 1) # +1 to include bias\n",
+    "\n",
+    "    # For the output layer\n",
+    "    P[-1] = npr.randn(1, num_neurons[-1] + 1 ) # +1 since bias is included\n",
+    "\n",
+    "    print('Initial cost: %g'%cost_function_deep(P, x))\n",
+    "\n",
+    "    ## Start finding the optimal weigths using gradient descent\n",
+    "\n",
+    "    # Find the Python function that represents the gradient of the cost function\n",
+    "    # w.r.t the 0-th input argument -- that is the weights and biases in the hidden and output layer\n",
+    "    cost_function_deep_grad = grad(cost_function_deep,0)\n",
+    "\n",
+    "    # Let the update be done num_iter times\n",
+    "    for i in range(num_iter):\n",
+    "        # Evaluate the gradient at the current weights and biases in P.\n",
+    "        # The cost_grad consist now of N_hidden + 1 arrays; the gradient w.r.t the weights and biases\n",
+    "        # in the hidden layers and output layers evaluated at x.\n",
+    "        cost_deep_grad =  cost_function_deep_grad(P, x)\n",
+    "\n",
+    "        for l in range(N_hidden+1):\n",
+    "            P[l] = P[l] - lmb * cost_deep_grad[l]\n",
+    "\n",
+    "    print('Final cost: %g'%cost_function_deep(P, x))\n",
+    "\n",
+    "    return P\n",
+    "\n",
+    "if __name__ == '__main__':\n",
+    "    npr.seed(4155)\n",
+    "\n",
+    "    ## Decide the vales of arguments to the function to solve\n",
+    "    Nt = 10\n",
+    "    T = 1\n",
+    "    t = np.linspace(0,T, Nt)\n",
+    "\n",
+    "    ## Set up the initial parameters\n",
+    "    num_hidden_neurons = [100, 50, 25]\n",
+    "    num_iter = 1000\n",
+    "    lmb = 1e-3\n",
+    "\n",
+    "    P = solve_ode_deep_neural_network(t, num_hidden_neurons, num_iter, lmb)\n",
+    "\n",
+    "    g_dnn_ag = g_trial_deep(t,P)\n",
+    "    g_analytical = g_analytic(t)\n",
+    "\n",
+    "    # Find the maximum absolute difference between the solutons:\n",
+    "    diff_ag = np.max(np.abs(g_dnn_ag - g_analytical))\n",
+    "    print(\"The max absolute difference between the solutions is: %g\"%diff_ag)\n",
+    "\n",
+    "    plt.figure(figsize=(10,10))\n",
+    "\n",
+    "    plt.title('Performance of neural network solving an ODE compared to the analytical solution')\n",
+    "    plt.plot(t, g_analytical)\n",
+    "    plt.plot(t, g_dnn_ag[0,:])\n",
+    "    plt.legend(['analytical','nn'])\n",
+    "    plt.xlabel('t')\n",
+    "    plt.ylabel('g(t)')\n",
+    "\n",
+    "    plt.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Using forward Euler to solve the ODE\n",
+    "\n",
+    "A straightforward way of solving an ODE numerically, is to use Euler's method.\n",
+    "\n",
+    "Euler's method uses Taylor series to approximate the value at a function $f$ at a step $\\Delta x$ from $x$:\n",
+    "\n",
+    "$$\n",
+    "f(x + \\Delta x) \\approx f(x) + \\Delta x f'(x)\n",
+    "$$\n",
+    "\n",
+    "In our case, using Euler's method to approximate the value of $g$ at a step $\\Delta t$ from $t$ yields"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\begin{aligned}\n",
+    "  g(t + \\Delta t) &\\approx g(t) + \\Delta t g'(t) \\\\\n",
+    "  &= g(t) + \\Delta t \\big(\\alpha g(t)(A - g(t))\\big)\n",
+    "\\end{aligned}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "along with the condition that $g(0) = g_0$.\n",
+    "\n",
+    "Let $t_i = i \\cdot \\Delta t$ where $\\Delta t = \\frac{T}{N_t-1}$ where $T$ is the final time our solver must solve for and $N_t$ the number of values for $t \\in [0, T]$ for $i = 0, \\dots, N_t-1$.\n",
+    "\n",
+    "For $i \\geq 1$, we have that"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\begin{aligned}\n",
+    "t_i &= i\\Delta t \\\\\n",
+    "&= (i - 1)\\Delta t + \\Delta t \\\\\n",
+    "&= t_{i-1} + \\Delta t\n",
+    "\\end{aligned}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Now, if $g_i = g(t_i)$ then"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation}\n",
+    "  \\begin{aligned}\n",
+    "  g_i &= g(t_i) \\\\\n",
+    "  &= g(t_{i-1} + \\Delta t) \\\\\n",
+    "  &\\approx g(t_{i-1}) + \\Delta t \\big(\\alpha g(t_{i-1})(A - g(t_{i-1}))\\big) \\\\\n",
+    "  &= g_{i-1} + \\Delta t \\big(\\alpha g_{i-1}(A - g_{i-1})\\big)\n",
+    "  \\end{aligned}\n",
+    "\\end{equation} \\label{odenum} \\tag{12}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "for $i \\geq 1$ and $g_0 = g(t_0) = g(0) = g_0$.\n",
+    "\n",
+    "Equation ([12](#odenum)) could be implemented in the following way,\n",
+    "extending the program that uses the network using Autograd:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "# Assume that all function definitions from the example program using Autograd\n",
+    "# are located here.\n",
+    "\n",
+    "if __name__ == '__main__':\n",
+    "    npr.seed(4155)\n",
+    "\n",
+    "    ## Decide the vales of arguments to the function to solve\n",
+    "    Nt = 10\n",
+    "    T = 1\n",
+    "    t = np.linspace(0,T, Nt)\n",
+    "\n",
+    "    ## Set up the initial parameters\n",
+    "    num_hidden_neurons = [100,50,25]\n",
+    "    num_iter = 1000\n",
+    "    lmb = 1e-3\n",
+    "\n",
+    "    P = solve_ode_deep_neural_network(t, num_hidden_neurons, num_iter, lmb)\n",
+    "\n",
+    "    g_dnn_ag = g_trial_deep(t,P)\n",
+    "    g_analytical = g_analytic(t)\n",
+    "\n",
+    "    # Find the maximum absolute difference between the solutons:\n",
+    "    diff_ag = np.max(np.abs(g_dnn_ag - g_analytical))\n",
+    "    print(\"The max absolute difference between the solutions is: %g\"%diff_ag)\n",
+    "\n",
+    "    plt.figure(figsize=(10,10))\n",
+    "\n",
+    "    plt.title('Performance of neural network solving an ODE compared to the analytical solution')\n",
+    "    plt.plot(t, g_analytical)\n",
+    "    plt.plot(t, g_dnn_ag[0,:])\n",
+    "    plt.legend(['analytical','nn'])\n",
+    "    plt.xlabel('t')\n",
+    "    plt.ylabel('g(t)')\n",
+    "\n",
+    "    ## Find an approximation to the funtion using forward Euler\n",
+    "\n",
+    "    alpha, A, g0 = get_parameters()\n",
+    "    dt = T/(Nt - 1)\n",
+    "\n",
+    "    # Perform forward Euler to solve the ODE\n",
+    "    g_euler = np.zeros(Nt)\n",
+    "    g_euler[0] = g0\n",
+    "\n",
+    "    for i in range(1,Nt):\n",
+    "        g_euler[i] = g_euler[i-1] + dt*(alpha*g_euler[i-1]*(A - g_euler[i-1]))\n",
+    "\n",
+    "    # Print the errors done by each method\n",
+    "    diff1 = np.max(np.abs(g_euler - g_analytical))\n",
+    "    diff2 = np.max(np.abs(g_dnn_ag[0,:] - g_analytical))\n",
+    "\n",
+    "    print('Max absolute difference between Euler method and analytical: %g'%diff1)\n",
+    "    print('Max absolute difference between deep neural network and analytical: %g'%diff2)\n",
+    "\n",
+    "    # Plot results\n",
+    "    plt.figure(figsize=(10,10))\n",
+    "\n",
+    "    plt.plot(t,g_euler)\n",
+    "    plt.plot(t,g_analytical)\n",
+    "    plt.plot(t,g_dnn_ag[0,:])\n",
+    "\n",
+    "    plt.legend(['euler','analytical','dnn'])\n",
+    "    plt.xlabel('Time t')\n",
+    "    plt.ylabel('g(t)')\n",
+    "\n",
+    "    plt.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Example: Solving the one dimensional Poisson equation\n",
+    "\n",
+    "The Poisson equation for $g(x)$ in one dimension is"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation} \\label{poisson} \\tag{13}\n",
+    "  -g''(x) = f(x)\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "where $f(x)$ is a given function for $x \\in (0,1)$.\n",
+    "\n",
+    "The conditions that $g(x)$ is chosen to fulfill, are"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\begin{align*}\n",
+    "  g(0) &= 0 \\\\\n",
+    "  g(1) &= 0\n",
+    "\\end{align*}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "This equation can be solved numerically using programs where e.g Autograd and TensorFlow are used.\n",
+    "The results from the networks can then be compared to the analytical solution.\n",
+    "In addition, it could be interesting to see how a typical method for numerically solving second order ODEs compares to the neural networks.\n",
+    "\n",
+    "\n",
+    "## The specific equation to solve for\n",
+    "\n",
+    "Here, the function $g(x)$ to solve for follows the equation"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "-g''(x) = f(x),\\qquad x \\in (0,1)\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "where $f(x)$ is a given function, along with the chosen conditions"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{aligned}\n",
+    "g(0) = g(1) = 0\n",
+    "\\end{aligned}\\label{cond} \\tag{14}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "In this example, we consider the case when $f(x) = (3x + x^2)\\exp(x)$.\n",
+    "\n",
+    "For this case, a possible trial solution satisfying the conditions could be"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "g_t(x) = x \\cdot (1-x) \\cdot N(P,x)\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The analytical solution for this problem is"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "g(x) = x(1 - x)\\exp(x)\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Solving the equation using Autograd"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "import autograd.numpy as np\n",
+    "from autograd import grad, elementwise_grad\n",
+    "import autograd.numpy.random as npr\n",
+    "from matplotlib import pyplot as plt\n",
+    "\n",
+    "def sigmoid(z):\n",
+    "    return 1/(1 + np.exp(-z))\n",
+    "\n",
+    "def deep_neural_network(deep_params, x):\n",
+    "    # N_hidden is the number of hidden layers\n",
+    "    N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n",
+    "\n",
+    "    # Assumes input x being an one-dimensional array\n",
+    "    num_values = np.size(x)\n",
+    "    x = x.reshape(-1, num_values)\n",
+    "\n",
+    "    # Assume that the input layer does nothing to the input x\n",
+    "    x_input = x\n",
+    "\n",
+    "    # Due to multiple hidden layers, define a variable referencing to the\n",
+    "    # output of the previous layer:\n",
+    "    x_prev = x_input\n",
+    "\n",
+    "    ## Hidden layers:\n",
+    "\n",
+    "    for l in range(N_hidden):\n",
+    "        # From the list of parameters P; find the correct weigths and bias for this layer\n",
+    "        w_hidden = deep_params[l]\n",
+    "\n",
+    "        # Add a row of ones to include bias\n",
+    "        x_prev = np.concatenate((np.ones((1,num_values)), x_prev ), axis = 0)\n",
+    "\n",
+    "        z_hidden = np.matmul(w_hidden, x_prev)\n",
+    "        x_hidden = sigmoid(z_hidden)\n",
+    "\n",
+    "        # Update x_prev such that next layer can use the output from this layer\n",
+    "        x_prev = x_hidden\n",
+    "\n",
+    "    ## Output layer:\n",
+    "\n",
+    "    # Get the weights and bias for this layer\n",
+    "    w_output = deep_params[-1]\n",
+    "\n",
+    "    # Include bias:\n",
+    "    x_prev = np.concatenate((np.ones((1,num_values)), x_prev), axis = 0)\n",
+    "\n",
+    "    z_output = np.matmul(w_output, x_prev)\n",
+    "    x_output = z_output\n",
+    "\n",
+    "    return x_output\n",
+    "\n",
+    "def solve_ode_deep_neural_network(x, num_neurons, num_iter, lmb):\n",
+    "    # num_hidden_neurons is now a list of number of neurons within each hidden layer\n",
+    "\n",
+    "    # Find the number of hidden layers:\n",
+    "    N_hidden = np.size(num_neurons)\n",
+    "\n",
+    "    ## Set up initial weigths and biases\n",
+    "\n",
+    "    # Initialize the list of parameters:\n",
+    "    P = [None]*(N_hidden + 1) # + 1 to include the output layer\n",
+    "\n",
+    "    P[0] = npr.randn(num_neurons[0], 2 )\n",
+    "    for l in range(1,N_hidden):\n",
+    "        P[l] = npr.randn(num_neurons[l], num_neurons[l-1] + 1) # +1 to include bias\n",
+    "\n",
+    "    # For the output layer\n",
+    "    P[-1] = npr.randn(1, num_neurons[-1] + 1 ) # +1 since bias is included\n",
+    "\n",
+    "    print('Initial cost: %g'%cost_function_deep(P, x))\n",
+    "\n",
+    "    ## Start finding the optimal weigths using gradient descent\n",
+    "\n",
+    "    # Find the Python function that represents the gradient of the cost function\n",
+    "    # w.r.t the 0-th input argument -- that is the weights and biases in the hidden and output layer\n",
+    "    cost_function_deep_grad = grad(cost_function_deep,0)\n",
+    "\n",
+    "    # Let the update be done num_iter times\n",
+    "    for i in range(num_iter):\n",
+    "        # Evaluate the gradient at the current weights and biases in P.\n",
+    "        # The cost_grad consist now of N_hidden + 1 arrays; the gradient w.r.t the weights and biases\n",
+    "        # in the hidden layers and output layers evaluated at x.\n",
+    "        cost_deep_grad =  cost_function_deep_grad(P, x)\n",
+    "\n",
+    "        for l in range(N_hidden+1):\n",
+    "            P[l] = P[l] - lmb * cost_deep_grad[l]\n",
+    "\n",
+    "    print('Final cost: %g'%cost_function_deep(P, x))\n",
+    "\n",
+    "    return P\n",
+    "\n",
+    "## Set up the cost function specified for this Poisson equation:\n",
+    "\n",
+    "# The right side of the ODE\n",
+    "def f(x):\n",
+    "    return (3*x + x**2)*np.exp(x)\n",
+    "\n",
+    "def cost_function_deep(P, x):\n",
+    "\n",
+    "    # Evaluate the trial function with the current parameters P\n",
+    "    g_t = g_trial_deep(x,P)\n",
+    "\n",
+    "    # Find the derivative w.r.t x of the trial function\n",
+    "    d2_g_t = elementwise_grad(elementwise_grad(g_trial_deep,0))(x,P)\n",
+    "\n",
+    "    right_side = f(x)\n",
+    "\n",
+    "    err_sqr = (-d2_g_t - right_side)**2\n",
+    "    cost_sum = np.sum(err_sqr)\n",
+    "\n",
+    "    return cost_sum/np.size(err_sqr)\n",
+    "\n",
+    "# The trial solution:\n",
+    "def g_trial_deep(x,P):\n",
+    "    return x*(1-x)*deep_neural_network(P,x)\n",
+    "\n",
+    "# The analytic solution;\n",
+    "def g_analytic(x):\n",
+    "    return x*(1-x)*np.exp(x)\n",
+    "\n",
+    "if __name__ == '__main__':\n",
+    "    npr.seed(4155)\n",
+    "\n",
+    "    ## Decide the vales of arguments to the function to solve\n",
+    "    Nx = 10\n",
+    "    x = np.linspace(0,1, Nx)\n",
+    "\n",
+    "    ## Set up the initial parameters\n",
+    "    num_hidden_neurons = [200,100]\n",
+    "    num_iter = 1000\n",
+    "    lmb = 1e-3\n",
+    "\n",
+    "    P = solve_ode_deep_neural_network(x, num_hidden_neurons, num_iter, lmb)\n",
+    "\n",
+    "    g_dnn_ag = g_trial_deep(x,P)\n",
+    "    g_analytical = g_analytic(x)\n",
+    "\n",
+    "    # Find the maximum absolute difference between the solutons:\n",
+    "    max_diff = np.max(np.abs(g_dnn_ag - g_analytical))\n",
+    "    print(\"The max absolute difference between the solutions is: %g\"%max_diff)\n",
+    "\n",
+    "    plt.figure(figsize=(10,10))\n",
+    "\n",
+    "    plt.title('Performance of neural network solving an ODE compared to the analytical solution')\n",
+    "    plt.plot(x, g_analytical)\n",
+    "    plt.plot(x, g_dnn_ag[0,:])\n",
+    "    plt.legend(['analytical','nn'])\n",
+    "    plt.xlabel('x')\n",
+    "    plt.ylabel('g(x)')\n",
+    "    plt.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Comparing with a numerical scheme\n",
+    "\n",
+    "The Poisson equation is possible to solve using Taylor series to approximate the second derivative.\n",
+    "\n",
+    "Using Taylor series, the second derivative can be expressed as\n",
+    "\n",
+    "$$\n",
+    "g''(x) = \\frac{g(x + \\Delta x) - 2g(x) + g(x-\\Delta x)}{\\Delta x^2} + E_{\\Delta x}(x)\n",
+    "$$\n",
+    "\n",
+    "where $\\Delta x$ is a small step size and $E_{\\Delta x}(x)$ being the error term.\n",
+    "\n",
+    "Looking away from the error terms gives an approximation to the second derivative:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation} \\label{approx} \\tag{15}\n",
+    "g''(x) \\approx \\frac{g(x + \\Delta x) - 2g(x) + g(x-\\Delta x)}{\\Delta x^2}\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "If $x_i = i \\Delta x = x_{i-1} + \\Delta x$ and $g_i = g(x_i)$ for $i = 1,\\dots N_x - 2$ with $N_x$ being the number of values for $x$, ([15](#approx)) becomes"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\begin{aligned}\n",
+    "g''(x_i) &\\approx \\frac{g(x_i + \\Delta x) - 2g(x_i) + g(x_i -\\Delta x)}{\\Delta x^2} \\\\\n",
+    "&= \\frac{g_{i+1} - 2g_i + g_{i-1}}{\\Delta x^2}\n",
+    "\\end{aligned}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Since we know from our problem that"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\begin{aligned}\n",
+    "-g''(x) &= f(x) \\\\\n",
+    "&= (3x + x^2)\\exp(x)\n",
+    "\\end{aligned}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "along with the conditions $g(0) = g(1) = 0$,\n",
+    "the following scheme can be used to find an approximate solution for $g(x)$ numerically:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation}\n",
+    "  \\begin{aligned}\n",
+    "  -\\Big( \\frac{g_{i+1} - 2g_i + g_{i-1}}{\\Delta x^2} \\Big) &= f(x_i) \\\\\n",
+    "  -g_{i+1} + 2g_i - g_{i-1} &= \\Delta x^2 f(x_i)\n",
+    "  \\end{aligned}\n",
+    "\\end{equation} \\label{odesys} \\tag{16}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "for $i = 1, \\dots, N_x - 2$ where $g_0 = g_{N_x - 1} = 0$ and $f(x_i) = (3x_i + x_i^2)\\exp(x_i)$, which is given for our specific problem.\n",
+    "\n",
+    "The equation can be rewritten into a matrix equation:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\begin{aligned}\n",
+    "\\begin{pmatrix}\n",
+    "2 & -1 & 0 & \\dots & 0 \\\\\n",
+    "-1 & 2 & -1 & \\dots & 0 \\\\\n",
+    "\\vdots & & \\ddots & & \\vdots \\\\\n",
+    "0 & \\dots & -1 & 2 & -1  \\\\\n",
+    "0 & \\dots & 0 & -1 & 2\\\\\n",
+    "\\end{pmatrix}\n",
+    "\\begin{pmatrix}\n",
+    "g_1 \\\\\n",
+    "g_2 \\\\\n",
+    "\\vdots \\\\\n",
+    "g_{N_x - 3} \\\\\n",
+    "g_{N_x - 2}\n",
+    "\\end{pmatrix}\n",
+    "&=\n",
+    "\\Delta x^2\n",
+    "\\begin{pmatrix}\n",
+    "f(x_1) \\\\\n",
+    "f(x_2) \\\\\n",
+    "\\vdots \\\\\n",
+    "f(x_{N_x - 3}) \\\\\n",
+    "f(x_{N_x - 2})\n",
+    "\\end{pmatrix} \\\\\n",
+    "\\boldsymbol{A}\\boldsymbol{g} &= \\boldsymbol{f},\n",
+    "\\end{aligned}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "which makes it possible to solve for the vector $\\boldsymbol{g}$.\n",
+    "\n",
+    "\n",
+    "## Setting up the code\n",
+    "\n",
+    "We can then compare the result from this numerical scheme with the output from our network using Autograd:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "import autograd.numpy as np\n",
+    "from autograd import grad, elementwise_grad\n",
+    "import autograd.numpy.random as npr\n",
+    "from matplotlib import pyplot as plt\n",
+    "\n",
+    "def sigmoid(z):\n",
+    "    return 1/(1 + np.exp(-z))\n",
+    "\n",
+    "def deep_neural_network(deep_params, x):\n",
+    "    # N_hidden is the number of hidden layers\n",
+    "    N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n",
+    "\n",
+    "    # Assumes input x being an one-dimensional array\n",
+    "    num_values = np.size(x)\n",
+    "    x = x.reshape(-1, num_values)\n",
+    "\n",
+    "    # Assume that the input layer does nothing to the input x\n",
+    "    x_input = x\n",
+    "\n",
+    "    # Due to multiple hidden layers, define a variable referencing to the\n",
+    "    # output of the previous layer:\n",
+    "    x_prev = x_input\n",
+    "\n",
+    "    ## Hidden layers:\n",
+    "\n",
+    "    for l in range(N_hidden):\n",
+    "        # From the list of parameters P; find the correct weigths and bias for this layer\n",
+    "        w_hidden = deep_params[l]\n",
+    "\n",
+    "        # Add a row of ones to include bias\n",
+    "        x_prev = np.concatenate((np.ones((1,num_values)), x_prev ), axis = 0)\n",
+    "\n",
+    "        z_hidden = np.matmul(w_hidden, x_prev)\n",
+    "        x_hidden = sigmoid(z_hidden)\n",
+    "\n",
+    "        # Update x_prev such that next layer can use the output from this layer\n",
+    "        x_prev = x_hidden\n",
+    "\n",
+    "    ## Output layer:\n",
+    "\n",
+    "    # Get the weights and bias for this layer\n",
+    "    w_output = deep_params[-1]\n",
+    "\n",
+    "    # Include bias:\n",
+    "    x_prev = np.concatenate((np.ones((1,num_values)), x_prev), axis = 0)\n",
+    "\n",
+    "    z_output = np.matmul(w_output, x_prev)\n",
+    "    x_output = z_output\n",
+    "\n",
+    "    return x_output\n",
+    "\n",
+    "def solve_ode_deep_neural_network(x, num_neurons, num_iter, lmb):\n",
+    "    # num_hidden_neurons is now a list of number of neurons within each hidden layer\n",
+    "\n",
+    "    # Find the number of hidden layers:\n",
+    "    N_hidden = np.size(num_neurons)\n",
+    "\n",
+    "    ## Set up initial weigths and biases\n",
+    "\n",
+    "    # Initialize the list of parameters:\n",
+    "    P = [None]*(N_hidden + 1) # + 1 to include the output layer\n",
+    "\n",
+    "    P[0] = npr.randn(num_neurons[0], 2 )\n",
+    "    for l in range(1,N_hidden):\n",
+    "        P[l] = npr.randn(num_neurons[l], num_neurons[l-1] + 1) # +1 to include bias\n",
+    "\n",
+    "    # For the output layer\n",
+    "    P[-1] = npr.randn(1, num_neurons[-1] + 1 ) # +1 since bias is included\n",
+    "\n",
+    "    print('Initial cost: %g'%cost_function_deep(P, x))\n",
+    "\n",
+    "    ## Start finding the optimal weigths using gradient descent\n",
+    "\n",
+    "    # Find the Python function that represents the gradient of the cost function\n",
+    "    # w.r.t the 0-th input argument -- that is the weights and biases in the hidden and output layer\n",
+    "    cost_function_deep_grad = grad(cost_function_deep,0)\n",
+    "\n",
+    "    # Let the update be done num_iter times\n",
+    "    for i in range(num_iter):\n",
+    "        # Evaluate the gradient at the current weights and biases in P.\n",
+    "        # The cost_grad consist now of N_hidden + 1 arrays; the gradient w.r.t the weights and biases\n",
+    "        # in the hidden layers and output layers evaluated at x.\n",
+    "        cost_deep_grad =  cost_function_deep_grad(P, x)\n",
+    "\n",
+    "        for l in range(N_hidden+1):\n",
+    "            P[l] = P[l] - lmb * cost_deep_grad[l]\n",
+    "\n",
+    "    print('Final cost: %g'%cost_function_deep(P, x))\n",
+    "\n",
+    "    return P\n",
+    "\n",
+    "## Set up the cost function specified for this Poisson equation:\n",
+    "\n",
+    "# The right side of the ODE\n",
+    "def f(x):\n",
+    "    return (3*x + x**2)*np.exp(x)\n",
+    "\n",
+    "def cost_function_deep(P, x):\n",
+    "\n",
+    "    # Evaluate the trial function with the current parameters P\n",
+    "    g_t = g_trial_deep(x,P)\n",
+    "\n",
+    "    # Find the derivative w.r.t x of the trial function\n",
+    "    d2_g_t = elementwise_grad(elementwise_grad(g_trial_deep,0))(x,P)\n",
+    "\n",
+    "    right_side = f(x)\n",
+    "\n",
+    "    err_sqr = (-d2_g_t - right_side)**2\n",
+    "    cost_sum = np.sum(err_sqr)\n",
+    "\n",
+    "    return cost_sum/np.size(err_sqr)\n",
+    "\n",
+    "# The trial solution:\n",
+    "def g_trial_deep(x,P):\n",
+    "    return x*(1-x)*deep_neural_network(P,x)\n",
+    "\n",
+    "# The analytic solution;\n",
+    "def g_analytic(x):\n",
+    "    return x*(1-x)*np.exp(x)\n",
+    "\n",
+    "if __name__ == '__main__':\n",
+    "    npr.seed(4155)\n",
+    "\n",
+    "    ## Decide the vales of arguments to the function to solve\n",
+    "    Nx = 10\n",
+    "    x = np.linspace(0,1, Nx)\n",
+    "\n",
+    "    ## Set up the initial parameters\n",
+    "    num_hidden_neurons = [200,100]\n",
+    "    num_iter = 1000\n",
+    "    lmb = 1e-3\n",
+    "\n",
+    "    P = solve_ode_deep_neural_network(x, num_hidden_neurons, num_iter, lmb)\n",
+    "\n",
+    "    g_dnn_ag = g_trial_deep(x,P)\n",
+    "    g_analytical = g_analytic(x)\n",
+    "\n",
+    "    # Find the maximum absolute difference between the solutons:\n",
+    "\n",
+    "    plt.figure(figsize=(10,10))\n",
+    "\n",
+    "    plt.title('Performance of neural network solving an ODE compared to the analytical solution')\n",
+    "    plt.plot(x, g_analytical)\n",
+    "    plt.plot(x, g_dnn_ag[0,:])\n",
+    "    plt.legend(['analytical','nn'])\n",
+    "    plt.xlabel('x')\n",
+    "    plt.ylabel('g(x)')\n",
+    "\n",
+    "    ## Perform the computation using the numerical scheme\n",
+    "\n",
+    "    dx = 1/(Nx - 1)\n",
+    "\n",
+    "    # Set up the matrix A\n",
+    "    A = np.zeros((Nx-2,Nx-2))\n",
+    "\n",
+    "    A[0,0] = 2\n",
+    "    A[0,1] = -1\n",
+    "\n",
+    "    for i in range(1,Nx-3):\n",
+    "        A[i,i-1] = -1\n",
+    "        A[i,i] = 2\n",
+    "        A[i,i+1] = -1\n",
+    "\n",
+    "    A[Nx - 3, Nx - 4] = -1\n",
+    "    A[Nx - 3, Nx - 3] = 2\n",
+    "\n",
+    "    # Set up the vector f\n",
+    "    f_vec = dx**2 * f(x[1:-1])\n",
+    "\n",
+    "    # Solve the equation\n",
+    "    g_res = np.linalg.solve(A,f_vec)\n",
+    "\n",
+    "    g_vec = np.zeros(Nx)\n",
+    "    g_vec[1:-1] = g_res\n",
+    "\n",
+    "    # Print the differences between each method\n",
+    "    max_diff1 = np.max(np.abs(g_dnn_ag - g_analytical))\n",
+    "    max_diff2 = np.max(np.abs(g_vec - g_analytical))\n",
+    "    print(\"The max absolute difference between the analytical solution and DNN Autograd: %g\"%max_diff1)\n",
+    "    print(\"The max absolute difference between the analytical solution and numerical scheme: %g\"%max_diff2)\n",
+    "\n",
+    "    # Plot the results\n",
+    "    plt.figure(figsize=(10,10))\n",
+    "\n",
+    "    plt.plot(x,g_vec)\n",
+    "    plt.plot(x,g_analytical)\n",
+    "    plt.plot(x,g_dnn_ag[0,:])\n",
+    "\n",
+    "    plt.legend(['numerical scheme','analytical','dnn'])\n",
+    "    plt.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Partial Differential Equations\n",
+    "\n",
+    "A partial differential equation (PDE) has a solution here the function\n",
+    "is defined by multiple variables.  The equation may involve all kinds\n",
+    "of combinations of which variables the function is differentiated with\n",
+    "respect to.\n",
+    "\n",
+    "In general, a partial differential equation for a function $g(x_1,\\dots,x_N)$ with $N$ variables may be expressed as"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation} \\label{PDE} \\tag{17}\n",
+    "  f\\left(x_1, \\, \\dots \\, , x_N, \\frac{\\partial g(x_1,\\dots,x_N) }{\\partial x_1}, \\dots , \\frac{\\partial g(x_1,\\dots,x_N) }{\\partial x_N}, \\frac{\\partial g(x_1,\\dots,x_N) }{\\partial x_1\\partial x_2}, \\, \\dots \\, , \\frac{\\partial^n g(x_1,\\dots,x_N) }{\\partial x_N^n} \\right) = 0\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "where $f$ is an expression involving all kinds of possible mixed derivatives of $g(x_1,\\dots,x_N)$ up to an order $n$. In order for the solution to be unique, some additional conditions must also be given.\n",
+    "\n",
+    "\n",
+    "## Type of problem\n",
+    "\n",
+    "The problem our network must solve for, is similar to the ODE case.\n",
+    "We must have a trial solution $g_t$ at hand.\n",
+    "\n",
+    "For instance, the trial solution could be expressed as"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\begin{align*}\n",
+    "  g_t(x_1,\\dots,x_N) = h_1(x_1,\\dots,x_N) + h_2(x_1,\\dots,x_N,N(x_1,\\dots,x_N,P))\n",
+    "\\end{align*}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "where $h_1(x_1,\\dots,x_N)$ is a function that ensures $g_t(x_1,\\dots,x_N)$ satisfies some given conditions.\n",
+    "The neural network $N(x_1,\\dots,x_N,P)$ has weights and biases described by $P$ and $h_2(x_1,\\dots,x_N,N(x_1,\\dots,x_N,P))$ is an expression using the output from the neural network in some way.\n",
+    "\n",
+    "The role of the function $h_2(x_1,\\dots,x_N,N(x_1,\\dots,x_N,P))$, is to ensure that the output of $N(x_1,\\dots,x_N,P)$ is zero when $g_t(x_1,\\dots,x_N)$ is evaluated at the values of $x_1,\\dots,x_N$ where the given conditions must be satisfied. The function $h_1(x_1,\\dots,x_N)$ should alone make $g_t(x_1,\\dots,x_N)$ satisfy the conditions.\n",
+    "\n",
+    "\n",
+    "\n",
+    "## Network requirements\n",
+    "\n",
+    "The network tries then the minimize the cost function following the\n",
+    "same ideas as described for the ODE case, but now with more than one\n",
+    "variables to consider.  The concept still remains the same; find a set\n",
+    "of parameters $P$ such that the expression $f$ in ([17](#PDE)) is as\n",
+    "close to zero as possible.\n",
+    "\n",
+    "As for the ODE case, the cost function is the mean squared error that\n",
+    "the network must try to minimize. The cost function for the network to\n",
+    "minimize is"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "C\\left(x_1, \\dots, x_N, P\\right) = \\left(  f\\left(x_1, \\, \\dots \\, , x_N, \\frac{\\partial g(x_1,\\dots,x_N) }{\\partial x_1}, \\dots , \\frac{\\partial g(x_1,\\dots,x_N) }{\\partial x_N}, \\frac{\\partial g(x_1,\\dots,x_N) }{\\partial x_1\\partial x_2}, \\, \\dots \\, , \\frac{\\partial^n g(x_1,\\dots,x_N) }{\\partial x_N^n} \\right) \\right)^2\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## More details\n",
+    "\n",
+    "If we let $\\boldsymbol{x} = \\big( x_1, \\dots, x_N \\big)$ be an array containing the values for $x_1, \\dots, x_N$ respectively, the cost function can be reformulated into the following:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "C\\left(\\boldsymbol{x}, P\\right) = f\\left( \\left( \\boldsymbol{x}, \\frac{\\partial g(\\boldsymbol{x}) }{\\partial x_1}, \\dots , \\frac{\\partial g(\\boldsymbol{x}) }{\\partial x_N}, \\frac{\\partial g(\\boldsymbol{x}) }{\\partial x_1\\partial x_2}, \\, \\dots \\, , \\frac{\\partial^n g(\\boldsymbol{x}) }{\\partial x_N^n} \\right) \\right)^2\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "If we also have $M$ different sets of values for $x_1, \\dots, x_N$, that is $\\boldsymbol{x}_i = \\big(x_1^{(i)}, \\dots, x_N^{(i)}\\big)$ for $i = 1,\\dots,M$ being the rows in matrix $X$, the cost function can be generalized into"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "C\\left(X, P \\right) = \\sum_{i=1}^M f\\left( \\left( \\boldsymbol{x}_i, \\frac{\\partial g(\\boldsymbol{x}_i) }{\\partial x_1}, \\dots , \\frac{\\partial g(\\boldsymbol{x}_i) }{\\partial x_N}, \\frac{\\partial g(\\boldsymbol{x}_i) }{\\partial x_1\\partial x_2}, \\, \\dots \\, , \\frac{\\partial^n g(\\boldsymbol{x}_i) }{\\partial x_N^n} \\right) \\right)^2.\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Example: The diffusion equation\n",
+    "\n",
+    "In one spatial dimension, the equation reads"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\frac{\\partial g(x,t)}{\\partial t} = \\frac{\\partial^2 g(x,t)}{\\partial x^2}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "where a possible choice of conditions are"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\begin{align*}\n",
+    "g(0,t) &= 0 ,\\qquad t \\geq 0 \\\\\n",
+    "g(1,t) &= 0, \\qquad t \\geq 0 \\\\\n",
+    "g(x,0) &= u(x),\\qquad x\\in [0,1]\n",
+    "\\end{align*}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "with $u(x)$ being some given function.\n",
+    "\n",
+    "\n",
+    "## Defining the problem\n",
+    "\n",
+    "For this case, we want to find $g(x,t)$ such that"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation}\n",
+    "  \\frac{\\partial g(x,t)}{\\partial t} = \\frac{\\partial^2 g(x,t)}{\\partial x^2}\n",
+    "\\end{equation} \\label{diffonedim} \\tag{18}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "and"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\begin{align*}\n",
+    "g(0,t) &= 0 ,\\qquad t \\geq 0 \\\\\n",
+    "g(1,t) &= 0, \\qquad t \\geq 0 \\\\\n",
+    "g(x,0) &= u(x),\\qquad x\\in [0,1]\n",
+    "\\end{align*}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "with $u(x) = \\sin(\\pi x)$.\n",
+    "\n",
+    "First, let us set up the deep neural network.\n",
+    "The deep neural network will follow the same structure as discussed in the examples solving the ODEs.\n",
+    "First, we will look into how Autograd could be used in a network tailored to solve for bivariate functions.\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "## Setting up the network using Autograd\n",
+    "\n",
+    "The only change to do here, is to extend our network such that\n",
+    "functions of multiple parameters are correctly handled.  In this case\n",
+    "we have two variables in our function to solve for, that is time $t$\n",
+    "and position $x$.  The variables will be represented by a\n",
+    "one-dimensional array in the program.  The program will evaluate the\n",
+    "network at each possible pair $(x,t)$, given an array for the desired\n",
+    "$x$-values and $t$-values to approximate the solution at."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "def sigmoid(z):\n",
+    "    return 1/(1 + np.exp(-z))\n",
+    "\n",
+    "def deep_neural_network(deep_params, x):\n",
+    "    # x is now a point and a 1D numpy array; make it a column vector\n",
+    "    num_coordinates = np.size(x,0)\n",
+    "    x = x.reshape(num_coordinates,-1)\n",
+    "\n",
+    "    num_points = np.size(x,1)\n",
+    "\n",
+    "    # N_hidden is the number of hidden layers\n",
+    "    N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n",
+    "\n",
+    "    # Assume that the input layer does nothing to the input x\n",
+    "    x_input = x\n",
+    "    x_prev = x_input\n",
+    "\n",
+    "    ## Hidden layers:\n",
+    "\n",
+    "    for l in range(N_hidden):\n",
+    "        # From the list of parameters P; find the correct weigths and bias for this layer\n",
+    "        w_hidden = deep_params[l]\n",
+    "\n",
+    "        # Add a row of ones to include bias\n",
+    "        x_prev = np.concatenate((np.ones((1,num_points)), x_prev ), axis = 0)\n",
+    "\n",
+    "        z_hidden = np.matmul(w_hidden, x_prev)\n",
+    "        x_hidden = sigmoid(z_hidden)\n",
+    "\n",
+    "        # Update x_prev such that next layer can use the output from this layer\n",
+    "        x_prev = x_hidden\n",
+    "\n",
+    "    ## Output layer:\n",
+    "\n",
+    "    # Get the weights and bias for this layer\n",
+    "    w_output = deep_params[-1]\n",
+    "\n",
+    "    # Include bias:\n",
+    "    x_prev = np.concatenate((np.ones((1,num_points)), x_prev), axis = 0)\n",
+    "\n",
+    "    z_output = np.matmul(w_output, x_prev)\n",
+    "    x_output = z_output\n",
+    "\n",
+    "    return x_output[0][0]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Setting up the network using Autograd; The trial solution\n",
+    "\n",
+    "The cost function must then iterate through the given arrays\n",
+    "containing values for $x$ and $t$, defines a point $(x,t)$ the deep\n",
+    "neural network and the trial solution is evaluated at, and then finds\n",
+    "the Jacobian of the trial solution.\n",
+    "\n",
+    "A possible trial solution for this PDE is\n",
+    "\n",
+    "$$\n",
+    "g_t(x,t) = h_1(x,t) + x(1-x)tN(x,t,P)\n",
+    "$$\n",
+    "\n",
+    "with $A(x,t)$ being a function ensuring that $g_t(x,t)$ satisfies our given conditions, and $N(x,t,P)$ being the output from the deep neural network using weights and biases for each layer from $P$.\n",
+    "\n",
+    "To fulfill the conditions, $A(x,t)$ could be:\n",
+    "\n",
+    "$$\n",
+    "h_1(x,t) = (1-t)\\Big(u(x) - \\big((1-x)u(0) + x u(1)\\big)\\Big) = (1-t)u(x) = (1-t)\\sin(\\pi x)\n",
+    "$$\n",
+    "since $(0) = u(1) = 0$ and $u(x) = \\sin(\\pi x)$.\n",
+    "\n",
+    "\n",
+    "## Why the jacobian?\n",
+    "\n",
+    "The Jacobian is used because the program must find the derivative of\n",
+    "the trial solution with respect to $x$ and $t$.\n",
+    "\n",
+    "This gives the necessity of computing the Jacobian matrix, as we want\n",
+    "to evaluate the gradient with respect to $x$ and $t$ (note that the\n",
+    "Jacobian of a scalar-valued multivariate function is simply its\n",
+    "gradient).\n",
+    "\n",
+    "In Autograd, the differentiation is by default done with respect to\n",
+    "the first input argument of your Python function. Since the points is\n",
+    "an array representing $x$ and $t$, the Jacobian is calculated using\n",
+    "the values of $x$ and $t$.\n",
+    "\n",
+    "To find the second derivative with respect to $x$ and $t$, the\n",
+    "Jacobian can be found for the second time. The result is a Hessian\n",
+    "matrix, which is the matrix containing all the possible second order\n",
+    "mixed derivatives of $g(x,t)$."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "# Set up the trial function:\n",
+    "def u(x):\n",
+    "    return np.sin(np.pi*x)\n",
+    "\n",
+    "def g_trial(point,P):\n",
+    "    x,t = point\n",
+    "    return (1-t)*u(x) + x*(1-x)*t*deep_neural_network(P,point)\n",
+    "\n",
+    "# The right side of the ODE:\n",
+    "def f(point):\n",
+    "    return 0.\n",
+    "\n",
+    "# The cost function:\n",
+    "def cost_function(P, x, t):\n",
+    "    cost_sum = 0\n",
+    "\n",
+    "    g_t_jacobian_func = jacobian(g_trial)\n",
+    "    g_t_hessian_func = hessian(g_trial)\n",
+    "\n",
+    "    for x_ in x:\n",
+    "        for t_ in t:\n",
+    "            point = np.array([x_,t_])\n",
+    "\n",
+    "            g_t = g_trial(point,P)\n",
+    "            g_t_jacobian = g_t_jacobian_func(point,P)\n",
+    "            g_t_hessian = g_t_hessian_func(point,P)\n",
+    "\n",
+    "            g_t_dt = g_t_jacobian[1]\n",
+    "            g_t_d2x = g_t_hessian[0][0]\n",
+    "\n",
+    "            func = f(point)\n",
+    "\n",
+    "            err_sqr = ( (g_t_dt - g_t_d2x) - func)**2\n",
+    "            cost_sum += err_sqr\n",
+    "\n",
+    "    return cost_sum"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Setting up the network using Autograd; The full program\n",
+    "\n",
+    "Having set up the network, along with the trial solution and cost function, we can now see how the deep neural network performs by comparing the results to the analytical solution.\n",
+    "\n",
+    "The analytical solution of our problem is\n",
+    "\n",
+    "$$\n",
+    "g(x,t) = \\exp(-\\pi^2 t)\\sin(\\pi x)\n",
+    "$$\n",
+    "\n",
+    "A possible way to implement a neural network solving the PDE, is given below.\n",
+    "Be aware, though, that it is fairly slow for the parameters used.\n",
+    "A better result is possible, but requires more iterations, and thus longer time to complete.\n",
+    "\n",
+    "\n",
+    "Indeed, the program below is not optimal in its implementation, but rather serves as an example on how to implement and use a neural network to solve a PDE.\n",
+    "Using TensorFlow results in a much better execution time. Try it!"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "import autograd.numpy as np\n",
+    "from autograd import jacobian,hessian,grad\n",
+    "import autograd.numpy.random as npr\n",
+    "from matplotlib import cm\n",
+    "from matplotlib import pyplot as plt\n",
+    "from mpl_toolkits.mplot3d import axes3d\n",
+    "\n",
+    "## Set up the network\n",
+    "\n",
+    "def sigmoid(z):\n",
+    "    return 1/(1 + np.exp(-z))\n",
+    "\n",
+    "def deep_neural_network(deep_params, x):\n",
+    "    # x is now a point and a 1D numpy array; make it a column vector\n",
+    "    num_coordinates = np.size(x,0)\n",
+    "    x = x.reshape(num_coordinates,-1)\n",
+    "\n",
+    "    num_points = np.size(x,1)\n",
+    "\n",
+    "    # N_hidden is the number of hidden layers\n",
+    "    N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n",
+    "\n",
+    "    # Assume that the input layer does nothing to the input x\n",
+    "    x_input = x\n",
+    "    x_prev = x_input\n",
+    "\n",
+    "    ## Hidden layers:\n",
+    "\n",
+    "    for l in range(N_hidden):\n",
+    "        # From the list of parameters P; find the correct weigths and bias for this layer\n",
+    "        w_hidden = deep_params[l]\n",
+    "\n",
+    "        # Add a row of ones to include bias\n",
+    "        x_prev = np.concatenate((np.ones((1,num_points)), x_prev ), axis = 0)\n",
+    "\n",
+    "        z_hidden = np.matmul(w_hidden, x_prev)\n",
+    "        x_hidden = sigmoid(z_hidden)\n",
+    "\n",
+    "        # Update x_prev such that next layer can use the output from this layer\n",
+    "        x_prev = x_hidden\n",
+    "\n",
+    "    ## Output layer:\n",
+    "\n",
+    "    # Get the weights and bias for this layer\n",
+    "    w_output = deep_params[-1]\n",
+    "\n",
+    "    # Include bias:\n",
+    "    x_prev = np.concatenate((np.ones((1,num_points)), x_prev), axis = 0)\n",
+    "\n",
+    "    z_output = np.matmul(w_output, x_prev)\n",
+    "    x_output = z_output\n",
+    "\n",
+    "    return x_output[0][0]\n",
+    "\n",
+    "## Define the trial solution and cost function\n",
+    "def u(x):\n",
+    "    return np.sin(np.pi*x)\n",
+    "\n",
+    "def g_trial(point,P):\n",
+    "    x,t = point\n",
+    "    return (1-t)*u(x) + x*(1-x)*t*deep_neural_network(P,point)\n",
+    "\n",
+    "# The right side of the ODE:\n",
+    "def f(point):\n",
+    "    return 0.\n",
+    "\n",
+    "# The cost function:\n",
+    "def cost_function(P, x, t):\n",
+    "    cost_sum = 0\n",
+    "\n",
+    "    g_t_jacobian_func = jacobian(g_trial)\n",
+    "    g_t_hessian_func = hessian(g_trial)\n",
+    "\n",
+    "    for x_ in x:\n",
+    "        for t_ in t:\n",
+    "            point = np.array([x_,t_])\n",
+    "\n",
+    "            g_t = g_trial(point,P)\n",
+    "            g_t_jacobian = g_t_jacobian_func(point,P)\n",
+    "            g_t_hessian = g_t_hessian_func(point,P)\n",
+    "\n",
+    "            g_t_dt = g_t_jacobian[1]\n",
+    "            g_t_d2x = g_t_hessian[0][0]\n",
+    "\n",
+    "            func = f(point)\n",
+    "\n",
+    "            err_sqr = ( (g_t_dt - g_t_d2x) - func)**2\n",
+    "            cost_sum += err_sqr\n",
+    "\n",
+    "    return cost_sum /( np.size(x)*np.size(t) )\n",
+    "\n",
+    "## For comparison, define the analytical solution\n",
+    "def g_analytic(point):\n",
+    "    x,t = point\n",
+    "    return np.exp(-np.pi**2*t)*np.sin(np.pi*x)\n",
+    "\n",
+    "## Set up a function for training the network to solve for the equation\n",
+    "def solve_pde_deep_neural_network(x,t, num_neurons, num_iter, lmb):\n",
+    "    ## Set up initial weigths and biases\n",
+    "    N_hidden = np.size(num_neurons)\n",
+    "\n",
+    "    ## Set up initial weigths and biases\n",
+    "\n",
+    "    # Initialize the list of parameters:\n",
+    "    P = [None]*(N_hidden + 1) # + 1 to include the output layer\n",
+    "\n",
+    "    P[0] = npr.randn(num_neurons[0], 2 + 1 ) # 2 since we have two points, +1 to include bias\n",
+    "    for l in range(1,N_hidden):\n",
+    "        P[l] = npr.randn(num_neurons[l], num_neurons[l-1] + 1) # +1 to include bias\n",
+    "\n",
+    "    # For the output layer\n",
+    "    P[-1] = npr.randn(1, num_neurons[-1] + 1 ) # +1 since bias is included\n",
+    "\n",
+    "    print('Initial cost: ',cost_function(P, x, t))\n",
+    "\n",
+    "    cost_function_grad = grad(cost_function,0)\n",
+    "\n",
+    "    # Let the update be done num_iter times\n",
+    "    for i in range(num_iter):\n",
+    "        cost_grad =  cost_function_grad(P, x , t)\n",
+    "\n",
+    "        for l in range(N_hidden+1):\n",
+    "            P[l] = P[l] - lmb * cost_grad[l]\n",
+    "\n",
+    "    print('Final cost: ',cost_function(P, x, t))\n",
+    "\n",
+    "    return P\n",
+    "\n",
+    "if __name__ == '__main__':\n",
+    "    ### Use the neural network:\n",
+    "    npr.seed(15)\n",
+    "\n",
+    "    ## Decide the vales of arguments to the function to solve\n",
+    "    Nx = 10; Nt = 10\n",
+    "    x = np.linspace(0, 1, Nx)\n",
+    "    t = np.linspace(0,1,Nt)\n",
+    "\n",
+    "    ## Set up the parameters for the network\n",
+    "    num_hidden_neurons = [100, 25]\n",
+    "    num_iter = 250\n",
+    "    lmb = 0.01\n",
+    "\n",
+    "    P = solve_pde_deep_neural_network(x,t, num_hidden_neurons, num_iter, lmb)\n",
+    "\n",
+    "    ## Store the results\n",
+    "    g_dnn_ag = np.zeros((Nx, Nt))\n",
+    "    G_analytical = np.zeros((Nx, Nt))\n",
+    "    for i,x_ in enumerate(x):\n",
+    "        for j, t_ in enumerate(t):\n",
+    "            point = np.array([x_, t_])\n",
+    "            g_dnn_ag[i,j] = g_trial(point,P)\n",
+    "\n",
+    "            G_analytical[i,j] = g_analytic(point)\n",
+    "\n",
+    "    # Find the map difference between the analytical and the computed solution\n",
+    "    diff_ag = np.abs(g_dnn_ag - G_analytical)\n",
+    "    print('Max absolute difference between the analytical solution and the network: %g'%np.max(diff_ag))\n",
+    "\n",
+    "    ## Plot the solutions in two dimensions, that being in position and time\n",
+    "\n",
+    "    T,X = np.meshgrid(t,x)\n",
+    "\n",
+    "    fig = plt.figure(figsize=(10,10))\n",
+    "    ax = fig.gca(projection='3d')\n",
+    "    ax.set_title('Solution from the deep neural network w/ %d layer'%len(num_hidden_neurons))\n",
+    "    s = ax.plot_surface(T,X,g_dnn_ag,linewidth=0,antialiased=False,cmap=cm.viridis)\n",
+    "    ax.set_xlabel('Time $t$')\n",
+    "    ax.set_ylabel('Position $x$');\n",
+    "\n",
+    "\n",
+    "    fig = plt.figure(figsize=(10,10))\n",
+    "    ax = fig.gca(projection='3d')\n",
+    "    ax.set_title('Analytical solution')\n",
+    "    s = ax.plot_surface(T,X,G_analytical,linewidth=0,antialiased=False,cmap=cm.viridis)\n",
+    "    ax.set_xlabel('Time $t$')\n",
+    "    ax.set_ylabel('Position $x$');\n",
+    "\n",
+    "    fig = plt.figure(figsize=(10,10))\n",
+    "    ax = fig.gca(projection='3d')\n",
+    "    ax.set_title('Difference')\n",
+    "    s = ax.plot_surface(T,X,diff_ag,linewidth=0,antialiased=False,cmap=cm.viridis)\n",
+    "    ax.set_xlabel('Time $t$')\n",
+    "    ax.set_ylabel('Position $x$');\n",
+    "\n",
+    "    ## Take some slices of the 3D plots just to see the solutions at particular times\n",
+    "    indx1 = 0\n",
+    "    indx2 = int(Nt/2)\n",
+    "    indx3 = Nt-1\n",
+    "\n",
+    "    t1 = t[indx1]\n",
+    "    t2 = t[indx2]\n",
+    "    t3 = t[indx3]\n",
+    "\n",
+    "    # Slice the results from the DNN\n",
+    "    res1 = g_dnn_ag[:,indx1]\n",
+    "    res2 = g_dnn_ag[:,indx2]\n",
+    "    res3 = g_dnn_ag[:,indx3]\n",
+    "\n",
+    "    # Slice the analytical results\n",
+    "    res_analytical1 = G_analytical[:,indx1]\n",
+    "    res_analytical2 = G_analytical[:,indx2]\n",
+    "    res_analytical3 = G_analytical[:,indx3]\n",
+    "\n",
+    "    # Plot the slices\n",
+    "    plt.figure(figsize=(10,10))\n",
+    "    plt.title(\"Computed solutions at time = %g\"%t1)\n",
+    "    plt.plot(x, res1)\n",
+    "    plt.plot(x,res_analytical1)\n",
+    "    plt.legend(['dnn','analytical'])\n",
+    "\n",
+    "    plt.figure(figsize=(10,10))\n",
+    "    plt.title(\"Computed solutions at time = %g\"%t2)\n",
+    "    plt.plot(x, res2)\n",
+    "    plt.plot(x,res_analytical2)\n",
+    "    plt.legend(['dnn','analytical'])\n",
+    "\n",
+    "    plt.figure(figsize=(10,10))\n",
+    "    plt.title(\"Computed solutions at time = %g\"%t3)\n",
+    "    plt.plot(x, res3)\n",
+    "    plt.plot(x,res_analytical3)\n",
+    "    plt.legend(['dnn','analytical'])\n",
+    "\n",
+    "    plt.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Example: Solving the wave equation with Neural Networks\n",
+    "\n",
+    "The wave equation is"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\frac{\\partial^2 g(x,t)}{\\partial t^2} = c^2\\frac{\\partial^2 g(x,t)}{\\partial x^2}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "with $c$ being the specified wave speed.\n",
+    "\n",
+    "Here, the chosen conditions are"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "\\begin{align*}\n",
+    "\tg(0,t) &= 0 \\\\\n",
+    "\tg(1,t) &= 0 \\\\\n",
+    "\tg(x,0) &= u(x) \\\\\n",
+    "\t\\frac{\\partial g(x,t)}{\\partial t} \\Big |_{t = 0} &= v(x)\n",
+    "\\end{align*}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "where $\\frac{\\partial g(x,t)}{\\partial t} \\Big |_{t = 0}$ means the derivative of $g(x,t)$ with respect to $t$ is evaluated at $t = 0$, and $u(x)$ and $v(x)$ being given functions.\n",
+    "\n",
+    "\n",
+    "## The problem to solve for\n",
+    "\n",
+    "The wave equation to solve for, is"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation} \\label{wave} \\tag{19}\n",
+    "\\frac{\\partial^2 g(x,t)}{\\partial t^2} = c^2 \\frac{\\partial^2 g(x,t)}{\\partial x^2}\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "where $c$ is the given wave speed.\n",
+    "The chosen conditions for this equation are"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{aligned}\n",
+    "g(0,t) &= 0, &t \\geq 0 \\\\\n",
+    "g(1,t) &= 0, &t \\geq 0 \\\\\n",
+    "g(x,0) &= u(x), &x\\in[0,1] \\\\\n",
+    "\\frac{\\partial g(x,t)}{\\partial t}\\Big |_{t = 0} &= v(x), &x \\in [0,1]\n",
+    "\\end{aligned} \\label{condwave} \\tag{20}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "In this example, let $c = 1$ and $u(x) = \\sin(\\pi x)$ and $v(x) = -\\pi\\sin(\\pi x)$.\n",
+    "\n",
+    "\n",
+    "\n",
+    "## The trial solution\n",
+    "Setting up the network is done in similar matter as for the example of solving the diffusion equation.\n",
+    "The only things we have to change, is the trial solution such that it satisfies the conditions from ([20](#condwave)) and the cost function.\n",
+    "\n",
+    "The trial solution becomes slightly different since we have other conditions than in the example of solving the diffusion equation. Here, a possible trial solution $g_t(x,t)$ is\n",
+    "\n",
+    "$$\n",
+    "g_t(x,t) = h_1(x,t) + x(1-x)t^2N(x,t,P)\n",
+    "$$\n",
+    "\n",
+    "where\n",
+    "\n",
+    "$$\n",
+    "h_1(x,t) = (1-t^2)u(x) + tv(x)\n",
+    "$$\n",
+    "\n",
+    "Note that this trial solution satisfies the conditions only if $u(0) = v(0) = u(1) = v(1) = 0$, which is the case in this example.\n",
+    "\n",
+    "\n",
+    "## The analytical solution\n",
+    "\n",
+    "The analytical solution for our specific problem, is\n",
+    "\n",
+    "$$\n",
+    "g(x,t) = \\sin(\\pi x)\\cos(\\pi t) - \\sin(\\pi x)\\sin(\\pi t)\n",
+    "$$\n",
+    "\n",
+    "\n",
+    "## Solving the wave equation - the full program using Autograd"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 16,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "import autograd.numpy as np\n",
+    "from autograd import hessian,grad\n",
+    "import autograd.numpy.random as npr\n",
+    "from matplotlib import cm\n",
+    "from matplotlib import pyplot as plt\n",
+    "from mpl_toolkits.mplot3d import axes3d\n",
+    "\n",
+    "## Set up the trial function:\n",
+    "def u(x):\n",
+    "    return np.sin(np.pi*x)\n",
+    "\n",
+    "def v(x):\n",
+    "    return -np.pi*np.sin(np.pi*x)\n",
+    "\n",
+    "def h1(point):\n",
+    "    x,t = point\n",
+    "    return (1 - t**2)*u(x) + t*v(x)\n",
+    "\n",
+    "def g_trial(point,P):\n",
+    "    x,t = point\n",
+    "    return h1(point) + x*(1-x)*t**2*deep_neural_network(P,point)\n",
+    "\n",
+    "## Define the cost function\n",
+    "def cost_function(P, x, t):\n",
+    "    cost_sum = 0\n",
+    "\n",
+    "    g_t_hessian_func = hessian(g_trial)\n",
+    "\n",
+    "    for x_ in x:\n",
+    "        for t_ in t:\n",
+    "            point = np.array([x_,t_])\n",
+    "\n",
+    "            g_t_hessian = g_t_hessian_func(point,P)\n",
+    "\n",
+    "            g_t_d2x = g_t_hessian[0][0]\n",
+    "            g_t_d2t = g_t_hessian[1][1]\n",
+    "\n",
+    "            err_sqr = ( (g_t_d2t - g_t_d2x) )**2\n",
+    "            cost_sum += err_sqr\n",
+    "\n",
+    "    return cost_sum / (np.size(t) * np.size(x))\n",
+    "\n",
+    "## The neural network\n",
+    "def sigmoid(z):\n",
+    "    return 1/(1 + np.exp(-z))\n",
+    "\n",
+    "def deep_neural_network(deep_params, x):\n",
+    "    # x is now a point and a 1D numpy array; make it a column vector\n",
+    "    num_coordinates = np.size(x,0)\n",
+    "    x = x.reshape(num_coordinates,-1)\n",
+    "\n",
+    "    num_points = np.size(x,1)\n",
+    "\n",
+    "    # N_hidden is the number of hidden layers\n",
+    "    N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n",
+    "\n",
+    "    # Assume that the input layer does nothing to the input x\n",
+    "    x_input = x\n",
+    "    x_prev = x_input\n",
+    "\n",
+    "    ## Hidden layers:\n",
+    "\n",
+    "    for l in range(N_hidden):\n",
+    "        # From the list of parameters P; find the correct weigths and bias for this layer\n",
+    "        w_hidden = deep_params[l]\n",
+    "\n",
+    "        # Add a row of ones to include bias\n",
+    "        x_prev = np.concatenate((np.ones((1,num_points)), x_prev ), axis = 0)\n",
+    "\n",
+    "        z_hidden = np.matmul(w_hidden, x_prev)\n",
+    "        x_hidden = sigmoid(z_hidden)\n",
+    "\n",
+    "        # Update x_prev such that next layer can use the output from this layer\n",
+    "        x_prev = x_hidden\n",
+    "\n",
+    "    ## Output layer:\n",
+    "\n",
+    "    # Get the weights and bias for this layer\n",
+    "    w_output = deep_params[-1]\n",
+    "\n",
+    "    # Include bias:\n",
+    "    x_prev = np.concatenate((np.ones((1,num_points)), x_prev), axis = 0)\n",
+    "\n",
+    "    z_output = np.matmul(w_output, x_prev)\n",
+    "    x_output = z_output\n",
+    "\n",
+    "    return x_output[0][0]\n",
+    "\n",
+    "## The analytical solution\n",
+    "def g_analytic(point):\n",
+    "    x,t = point\n",
+    "    return np.sin(np.pi*x)*np.cos(np.pi*t) - np.sin(np.pi*x)*np.sin(np.pi*t)\n",
+    "\n",
+    "def solve_pde_deep_neural_network(x,t, num_neurons, num_iter, lmb):\n",
+    "    ## Set up initial weigths and biases\n",
+    "    N_hidden = np.size(num_neurons)\n",
+    "\n",
+    "    ## Set up initial weigths and biases\n",
+    "\n",
+    "    # Initialize the list of parameters:\n",
+    "    P = [None]*(N_hidden + 1) # + 1 to include the output layer\n",
+    "\n",
+    "    P[0] = npr.randn(num_neurons[0], 2 + 1 ) # 2 since we have two points, +1 to include bias\n",
+    "    for l in range(1,N_hidden):\n",
+    "        P[l] = npr.randn(num_neurons[l], num_neurons[l-1] + 1) # +1 to include bias\n",
+    "\n",
+    "    # For the output layer\n",
+    "    P[-1] = npr.randn(1, num_neurons[-1] + 1 ) # +1 since bias is included\n",
+    "\n",
+    "    print('Initial cost: ',cost_function(P, x, t))\n",
+    "\n",
+    "    cost_function_grad = grad(cost_function,0)\n",
+    "\n",
+    "    # Let the update be done num_iter times\n",
+    "    for i in range(num_iter):\n",
+    "        cost_grad =  cost_function_grad(P, x , t)\n",
+    "\n",
+    "        for l in range(N_hidden+1):\n",
+    "            P[l] = P[l] - lmb * cost_grad[l]\n",
+    "\n",
+    "\n",
+    "    print('Final cost: ',cost_function(P, x, t))\n",
+    "\n",
+    "    return P\n",
+    "\n",
+    "if __name__ == '__main__':\n",
+    "    ### Use the neural network:\n",
+    "    npr.seed(15)\n",
+    "\n",
+    "    ## Decide the vales of arguments to the function to solve\n",
+    "    Nx = 10; Nt = 10\n",
+    "    x = np.linspace(0, 1, Nx)\n",
+    "    t = np.linspace(0,1,Nt)\n",
+    "\n",
+    "    ## Set up the parameters for the network\n",
+    "    num_hidden_neurons = [50,20]\n",
+    "    num_iter = 1000\n",
+    "    lmb = 0.01\n",
+    "\n",
+    "    P = solve_pde_deep_neural_network(x,t, num_hidden_neurons, num_iter, lmb)\n",
+    "\n",
+    "    ## Store the results\n",
+    "    res = np.zeros((Nx, Nt))\n",
+    "    res_analytical = np.zeros((Nx, Nt))\n",
+    "    for i,x_ in enumerate(x):\n",
+    "        for j, t_ in enumerate(t):\n",
+    "            point = np.array([x_, t_])\n",
+    "            res[i,j] = g_trial(point,P)\n",
+    "\n",
+    "            res_analytical[i,j] = g_analytic(point)\n",
+    "\n",
+    "    diff = np.abs(res - res_analytical)\n",
+    "    print(\"Max difference between analytical and solution from nn: %g\"%np.max(diff))\n",
+    "\n",
+    "    ## Plot the solutions in two dimensions, that being in position and time\n",
+    "\n",
+    "    T,X = np.meshgrid(t,x)\n",
+    "\n",
+    "    fig = plt.figure(figsize=(10,10))\n",
+    "    ax = fig.gca(projection='3d')\n",
+    "    ax.set_title('Solution from the deep neural network w/ %d layer'%len(num_hidden_neurons))\n",
+    "    s = ax.plot_surface(T,X,res,linewidth=0,antialiased=False,cmap=cm.viridis)\n",
+    "    ax.set_xlabel('Time $t$')\n",
+    "    ax.set_ylabel('Position $x$');\n",
+    "\n",
+    "\n",
+    "    fig = plt.figure(figsize=(10,10))\n",
+    "    ax = fig.gca(projection='3d')\n",
+    "    ax.set_title('Analytical solution')\n",
+    "    s = ax.plot_surface(T,X,res_analytical,linewidth=0,antialiased=False,cmap=cm.viridis)\n",
+    "    ax.set_xlabel('Time $t$')\n",
+    "    ax.set_ylabel('Position $x$');\n",
+    "\n",
+    "\n",
+    "    fig = plt.figure(figsize=(10,10))\n",
+    "    ax = fig.gca(projection='3d')\n",
+    "    ax.set_title('Difference')\n",
+    "    s = ax.plot_surface(T,X,diff,linewidth=0,antialiased=False,cmap=cm.viridis)\n",
+    "    ax.set_xlabel('Time $t$')\n",
+    "    ax.set_ylabel('Position $x$');\n",
+    "\n",
+    "    ## Take some slices of the 3D plots just to see the solutions at particular times\n",
+    "    indx1 = 0\n",
+    "    indx2 = int(Nt/2)\n",
+    "    indx3 = Nt-1\n",
+    "\n",
+    "    t1 = t[indx1]\n",
+    "    t2 = t[indx2]\n",
+    "    t3 = t[indx3]\n",
+    "\n",
+    "    # Slice the results from the DNN\n",
+    "    res1 = res[:,indx1]\n",
+    "    res2 = res[:,indx2]\n",
+    "    res3 = res[:,indx3]\n",
+    "\n",
+    "    # Slice the analytical results\n",
+    "    res_analytical1 = res_analytical[:,indx1]\n",
+    "    res_analytical2 = res_analytical[:,indx2]\n",
+    "    res_analytical3 = res_analytical[:,indx3]\n",
+    "\n",
+    "    # Plot the slices\n",
+    "    plt.figure(figsize=(10,10))\n",
+    "    plt.title(\"Computed solutions at time = %g\"%t1)\n",
+    "    plt.plot(x, res1)\n",
+    "    plt.plot(x,res_analytical1)\n",
+    "    plt.legend(['dnn','analytical'])\n",
+    "\n",
+    "    plt.figure(figsize=(10,10))\n",
+    "    plt.title(\"Computed solutions at time = %g\"%t2)\n",
+    "    plt.plot(x, res2)\n",
+    "    plt.plot(x,res_analytical2)\n",
+    "    plt.legend(['dnn','analytical'])\n",
+    "\n",
+    "    plt.figure(figsize=(10,10))\n",
+    "    plt.title(\"Computed solutions at time = %g\"%t3)\n",
+    "    plt.plot(x, res3)\n",
+    "    plt.plot(x,res_analytical3)\n",
+    "    plt.legend(['dnn','analytical'])\n",
+    "\n",
+    "    plt.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Resources on differential equations and deep learning\n",
+    "\n",
+    "1. [Artificial neural networks for solving ordinary and partial differential equations by I.E. Lagaris et al](https://pdfs.semanticscholar.org/d061/df393e0e8fbfd0ea24976458b7d42419040d.pdf)\n",
+    "\n",
+    "2. [Neural networks for solving differential equations by A. Honchar](https://becominghuman.ai/neural-networks-for-solving-differential-equations-fa230ac5e04c)\n",
+    "\n",
+    "3. [Solving differential equations using neural networks by M.M Chiaramonte and M. Kiener](http://cs229.stanford.edu/proj2013/ChiaramonteKiener-SolvingDifferentialEquationsUsingNeuralNetworks.pdf)\n",
+    "\n",
+    "4. [Introduction to Partial Differential Equations by A. Tveito, R. Winther](https://www.springer.com/us/book/9783540225515)"
+   ]
+  }
+ ],
+ "metadata": {},
+ "nbformat": 4,
+ "nbformat_minor": 4
+}
diff --git a/doc/src/LectureNotes/chapter11.ipynb b/doc/src/LectureNotes/chapter11.ipynb
new file mode 100644
index 000000000..200cdd664
--- /dev/null
+++ b/doc/src/LectureNotes/chapter11.ipynb
@@ -0,0 +1,1030 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "# Data Analysis and Machine Learning: \n",
+    "\n",
+    "  \n",
+    "**Christian Forssén**, Department of Physics, Chalmers University of Technology, Sweden  \n",
+    "\n",
+    " **Morten Hjorth-Jensen**, Department of Physics, University of Oslo and Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University\n",
+    "\n",
+    "Date: **Dec 23, 2020**\n",
+    "\n",
+    "Copyright 1999-2020, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "# Elements of Bayesian theory and Bayesian Neural Networks\n",
+    "\n",
+    "\n",
+    "## Why Bayesian Statistics?\n",
+    "\n",
+    "We have already made ourselves familiar with elements of a statistical\n",
+    "data analysis via quantities like the bias-variance tradeoff as well\n",
+    "as some central distribution functions such as the Normal\n",
+    "distribution, the binomial distribution and other probability\n",
+    "distribution functions.  \n",
+    "\n",
+    "In essentially all the Machine Learning algorithms we have studied,\n",
+    "our focus has been on a so-called **frequentist approach**, where\n",
+    "knowledge of an underlying likelihood function has not been\n",
+    "emphasized. Our data, whether we had a classification or a regression\n",
+    "problem, have been our central points of departure.\n",
+    "\n",
+    "Here we wish to merge this approach with the derivation of a\n",
+    "likelihood function which can be used to make prediction on how our\n",
+    "system under study evolves.  We will venture into the realm of what is\n",
+    "called Bayesian Neural Networks. To get an overarching view on what\n",
+    "this entails, the following figure conveys the essential differences\n",
+    "between a standard Neural network that we have met earlier and a\n",
+    "Bayesian Neural Network. In order to get there, we need to present\n",
+    "some of the basic elements of Bayesian statistics, starting with the\n",
+    "product rule and Bayes' theorem.\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "## Inference\n",
+    "Inference:\n",
+    "  :    \n",
+    "  \"the act of passing from one proposition, statement or judgment considered as true to another whose truth is believed to follow from that of the former\" (Webster) \n",
+    "  Do premises $A, B, \\ldots \\to$ hypothesis, $H$? \n",
+    "\n",
+    "Deductive inference:\n",
+    "  :    \n",
+    "  Premises allow definite determination of truth/falsity of H (syllogisms, symbolic logic, Boolean algebra) \n",
+    "  $B(H|A,B,...) = 0$ or $1$\n",
+    "\n",
+    "Inductive inference:\n",
+    "  :    \n",
+    "  Premises bear on truth/falsity of H, but don’t allow its definite determination (weak syllogisms, analogies)\n",
+    "  $A, B, C, D$ share properties $x, y, z$; $E$ has properties $x, y$\n",
+    "  $\\to$ $E$ probably has property $z$.\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "## Statistical Inference\n",
+    "* Quantify the strength of inductive inferences from facts, in the form of data ($D$), and other premises, e.g. models, to hypotheses about the phenomena producing the data.\n",
+    "\n",
+    "* Quantify via probabilities, or averages calculated using probabilities. Frequentists ($\\mathcal{F}$) and Bayesians ($\\mathcal{B}$) use probabilities very differently for this.\n",
+    "\n",
+    "* To the pioneers such as Bernoulli, Bayes and Laplace, a probability represented a *degree-of-belief* or plausability: how much they thought that something as true based on the evidence at hand. This is the Bayesian approach.\n",
+    "\n",
+    "* To the 19th century scholars, this seemed too vague and subjective. They redefined probability as the *long run relative frequency* with which an event occurred, given (infinitely) many repeated (experimental) trials.\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "## Some history\n",
+    "Adapted from D.S. Sivia[^Sivia]:\n",
+    "\n",
+    "[^Sivia]: Sivia, Devinderjit, and John Skilling. Data Analysis : A Bayesian Tutorial, OUP Oxford, 2006\n",
+    "\n",
+    "> Although the frequency definition appears to be more objective, its range of validity is also far more limited. For example, Laplace used (his) probability theory to estimate the mass of Saturn, given orbital data that were available to him from various astronomical observatories. In essence, he computed the posterior pdf for the mass M , given the data and all the relevant background information I (such as a knowledge of the laws of classical mechanics): prob(M|{data},I); this is shown schematically in the figure [Fig. 1.2].\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "
    \n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "> To Laplace, the (shaded) area under the posterior pdf curve between $m_1$ and $m_2$ was a measure of how much he believed that the mass of Saturn lay in the range $m_1 \\le M \\le m_2$. As such, the position of the maximum of the posterior pdf represents a best estimate of the mass; its width, or spread, about this optimal value gives an indication of the uncertainty in the estimate. Laplace stated that: ‘ . . . it is a bet of 11,000 to 1 that the error of this result is not 1/100th of its value.’ He would have won the bet, as another 150 years’ accumulation of data has changed the estimate by only 0.63%!\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "> According to the frequency definition, however, we are not permitted to use probability theory to tackle this problem. This is because the mass of Saturn is a constant and not a random variable; therefore, it has no frequency distribution and so probability theory cannot be used.\n",
+    "> \n",
+    "> If the pdf [of Fig. 1.2] had to be interpreted in terms of the frequency definition, we would have to imagine a large ensemble of universes in which everything remains constant apart from the mass of Saturn.\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "> As this scenario appears quite far-fetched, we might be inclined to think of [Fig. 1.2] in terms of the distribution of the measurements of the mass in many repetitions of the experiment. Although we are at liberty to think about a problem in any way that facilitates its solution, or our understanding of it, having to seek a frequency interpretation for every data analysis problem seems rather perverse.\n",
+    "> For example, what do we mean by the ‘measurement of the mass’ when the data consist of orbital periods? Besides, why should we have to think about many repetitions of an experiment that never happened? What we really want to do is to make the best inference of the mass given the (few) data that we actually have; this is precisely the Bayes and Laplace view of probability.\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "> Faced with the realization that the frequency definition of probability theory did not permit most real-life scientific problems to be addressed, a new subject was invented — statistics! To estimate the mass of Saturn, for example, one has to relate the mass to the data through some function called the statistic; since the data are subject to ‘random’ noise, the statistic becomes the random variable to which the rules of probability the- ory can be applied. But now the question arises: How should we choose the statistic? The frequentist approach does not yield a natural way of doing this and has, therefore, led to the development of several alternative schools of orthodox or conventional statis- tics. The masters, such as Fisher, Neyman and Pearson, provided a variety of different principles, which has merely resulted in a plethora of tests and procedures without any clear underlying rationale. This lack of unifying principles is, perhaps, at the heart of the shortcomings of the cook-book approach to statistics that students are often taught even today.\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "## The Bayesian recipe\n",
+    "Assess hypotheses by calculating their probabilities $p(H_i | \\ldots)$ conditional on known and/or presumed information using the rules of probability theory.\n",
+    "\n",
+    "\n",
+    "Probability Theory Axioms:\n",
+    "Product (AND) rule :\n",
+    "  :    \n",
+    "  $p(A, B | I) = p(A|I) p(B|A, I) = p(B|I)p(A|B,I)$\n",
+    "  Should read $p(A,B|I)$ as the probability for propositions $A$ AND $B$ being true given that $I$ is true.\n",
+    "\n",
+    "Sum (OR) rule:\n",
+    "  :    \n",
+    "  $p(A + B | I) = p(A | I) + p(B | I) - p(A, B | I)$\n",
+    "  $p(A+B|I)$ is the probability that proposition $A$ OR $B$ is true given that $I$ is true.\n",
+    "\n",
+    "Normalization:\n",
+    "  :    \n",
+    "  $p(A|I) + p(\\bar{A}|I) = 1$\n",
+    "  $\\bar{A}$ denotes the proposition that $A$ is false.\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "## Bayes' theorem\n",
+    "Bayes' theorem follows directly from the product rule"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "$$\n",
+    "p(A|B,I) = \\frac{p(B|A,I) p(A|I)}{p(B|I)}.\n",
+    "$$\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The importance of this property to data analysis becomes apparent if we replace $A$ and $B$ by hypothesis($H$) and data($D$):"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation}\n",
+    "p(H|D,I) = \\frac{p(D|H,I) p(H|I)}{p(D|I)}.\n",
+    "\\label{eq:bayes} \\tag{1}\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The power of Bayes’ theorem lies in the fact that it relates the quantity of interest, the probability that the hypothesis is true given the data, to the term we have a better chance of being able to assign, the probability that we would have observed the measured data if the hypothesis was true.\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "The various terms in Bayes’ theorem have formal names. \n",
+    "* The quantity on the far right, $p(H|I)$, is called the *prior* probability; it represents our state of knowledge (or ignorance) about the truth of the hypothesis before we have analysed the current data. \n",
+    "\n",
+    "* This is modified by the experimental measurements through $p(D|H,I)$, the *likelihood* function, \n",
+    "\n",
+    "* The denominator $p(D|I)$ is called the *evidence*. It does not depend on the hypothesis and can be regarded as a normalization constant.\n",
+    "\n",
+    "* Together, these yield the *posterior* probability, $p(H|D, I )$, representing our state of knowledge about the truth of the hypothesis in the light of the data. \n",
+    "\n",
+    "In a sense, Bayes’ theorem encapsulates the process of learning.\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "## The friends of Bayes' theorem\n",
+    "Normalization:\n",
+    "  :    \n",
+    "  $\\sum_i p(H_i|\\ldots) = 1$.\n",
+    "\n",
+    "Marginalization:\n",
+    "  :    \n",
+    "  $\\sum_i p(A,H_i|I) = \\sum_i p(H_i|A,I) p(A|I) = p(A|I)$.\n",
+    "\n",
+    "Marginalization (continuum limit):\n",
+    "  :    \n",
+    "  $\\int dx p(A,H(x)|I) = p(A|I)$.\n",
+    "\n",
+    "In the above, $H_i$ is an exclusive and exhaustive list of hypotheses. For example,let’s imagine that there are five candidates in a presidential election; then $H_1$ could be the proposition that the first candidate will win, and so on. The probability that $A$ is true, for example that unemployment will be lower in a year’s time (given all relevant information $I$, but irrespective of whoever becomes president) is then given by $\\sum_i p(A,H_i|I)$.\n",
+    "\n",
+    "In the continuum limit of propositions we must understand $p(\\ldots)$ as a pdf (probability density function).\n",
+    "\n",
+    "Marginalization is a very powerful device in data analysis because it enables us to deal with nuisance parameters; that is, quantities which necessarily enter the analysis but are of no intrinsic interest. The unwanted background signal present in many experimental measurements are examples of nuisance parameters.\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "## Inference With Parametric Models\n",
+    "Inductive inference with parametric models is a very important tool in the natural sciences.\n",
+    "* Consider $N$ different models $M_i$ ($i = 1, \\ldots, N$), each with parameters $\\boldsymbol{\\alpha}_i$. Each of them implies a sampling distribution (conditional predictive distribution for possible data)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "$$\n",
+    "p(D|\\boldsymbol{\\alpha}_i, M_i)\n",
+    "$$\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "* The $\\boldsymbol{\\alpha}_i$ dependence when we fix attention on the actual, observed data ($D_\\mathrm{obs}$) is the likelihood function:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "$$\n",
+    "\\mathcal{L}_i (\\boldsymbol{\\alpha}_i) \\equiv p(D_\\mathrm{obs}|\\boldsymbol{\\alpha}_i, M_i)\n",
+    "$$\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "* We may be uncertain about $i$ (model uncertainty),\n",
+    "\n",
+    "* or uncertain about $\\boldsymbol{\\alpha}_i$ (parameter uncertainty).\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "Parameter Estimation:\n",
+    "  :    \n",
+    "  Premise = choice of model (pick specific $i$)\n",
+    "  $\\Rightarrow$ What can we say about $\\boldsymbol{\\alpha}_i$?\n",
+    "\n",
+    "Model comparison:\n",
+    "  :    \n",
+    "  Premise = $\\{M_i\\}$\n",
+    "  $\\Rightarrow$ What can we say about $i$?\n",
+    "\n",
+    "Model adequacy:\n",
+    "  :    \n",
+    "  Premise = $M_1$\n",
+    "  $\\Rightarrow$ Is $M_1$ adequate?\n",
+    "\n",
+    "Hybrid Uncertainty:\n",
+    "  :    \n",
+    "  Models share some common params: $\\boldsymbol{\\alpha}_1 = \\{ \\boldsymbol{\\varphi}, \\boldsymbol{\\eta}_i\\}$\n",
+    "  $\\Rightarrow$ What can we say about $\\boldsymbol{\\varphi}$? (Systematic error is an example)\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "## Illustrative examples with python code\n",
+    "* Is this a fair coin? (analytical)\n",
+    "\n",
+    "* Flux from a star (single parameter, MCMC)\n",
+    "\n",
+    "* The lighthouse problem (two parameters, MCMC)\n",
+    "\n",
+    "* Linear fit with outliers (nuisance parameters)\n",
+    "\n",
+    "* ...\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "## Example: Is this a fair coin?\n",
+    "Let us begin with the analysis of data from a simple coin-tossing experiment. \n",
+    "Given that we had observed 6 heads in 8 flips, would you think it was a fair coin? By fair, we mean that we would be prepared to lay an even 1 : 1 bet on the outcome of a flip being a head or a tail. If we decide that the coin was fair, the question which follows naturally is how sure are we that this was so; if it was not fair, how unfair do we think it was? Furthermore, if we were to continue collecting data for this particular coin, observing the outcomes of additional flips, how would we update our belief on the fairness of the coin?\n",
+    "\n",
+    "A sensible way of formulating this problem is to consider a large number of hypotheses about the range in which the bias-weighting of the coin might lie. If we denote the bias-weighting by $H$, then $H = 0$ and $H = 1$ can represent a coin which produces a tail or a head on every flip, respectively. There is a continuum of possibilities for the value of H between these limits, with $H = 0.5$ indicating a fair coin. Our state of knowledge about the fairness, or the degree of unfairness, of the coin is then completely summarized by specifying how much we believe these various propositions to be true. \n",
+    "\n",
+    "Let us perform a computer simulation of a coin-tossing experiment. This provides the data that we will be analysing."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "0\n",
+    " \n",
+    "<\n",
+    "<\n",
+    "<\n",
+    "!\n",
+    "!\n",
+    "C\n",
+    "O\n",
+    "D\n",
+    "E\n",
+    "_\n",
+    "B\n",
+    "L\n",
+    "O\n",
+    "C\n",
+    "K\n",
+    " \n",
+    " \n",
+    "p\n",
+    "y\n",
+    "c\n",
+    "o\n",
+    "d"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "np.random.seed(999)         # for reproducibility\n",
+    "a=0.6                       # biased coin\n",
+    "flips=np.random.rand(2**12) # simulates 4096 coin flips\n",
+    "heads=flips\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation}\n",
+    "\\int_0^1 p(H|D,I) dH = 1.\n",
+    "\\label{eq:coin_posterior_norm} \\tag{2}\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The prior pdf, $p(H|I)$, represents what we know about the coin given only the information $I$ that we are dealing with a ‘strange coin’. We could keep a very open mind about the nature of the coin; a simple probability assignment which reflects this is a uniform, or flat, prior"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation}\n",
+    "p(H|I) = \\left\\{ \\begin{array}{ll}\n",
+    "1 & 0 \\le H \\le 1, \\\\\n",
+    "0 & \\mathrm{otherwise}.\n",
+    "\\end{array} \\right.\n",
+    "\\label{eq:coin_prior_uniform} \\tag{3}\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "We will get back later to the choice of prior and its effect on the analysis.\n",
+    "\n",
+    "This prior state of knowledge, or ignorance, is modified by the data through the likelihood function $p(D|H,I)$. It is a measure of the chance that we would have obtained the data that we actually observed, if the value of the bias-weighting was given (as known). If, in the conditioning information $I$, we assume that the flips of the coin were independent events, so that the outcome of one did not influence that of another, then the probability of obtaining the data `R heads in N tosses' is given by the binomial distribution (we leave a formal definition of this to a statistics textbook)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation}\n",
+    "p(D|H,I) \\propto H^R (1-H)^{N-R}.\n",
+    "\\label{_auto1} \\tag{4}\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "It seems reasonable because $H$ is the chance of obtaining a head on any flip, and there were $R$ of them, and $1-H$ is the corresponding probability for a tail, of which there were $N-R$. We note that this binomial distribution also contains a normalization factor, but we will ignore it since it does not depend explicitly on $H$, the quantity of interest. It will be absorbed by the normalization condition ([2](#eq:coin_posterior_norm)).\n",
+    "\n",
+    "We perform the setup of this Bayesian framework on the computer."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "def prior(H):\n",
+    "    p=np.zeros_like(H)\n",
+    "    p[(0<=x)&(x<=1)]=1      # allowed range: 0<=H<=1\n",
+    "    return p                # uniform prior\n",
+    "def likelihood(H,data):\n",
+    "    N = len(data)\n",
+    "    no_of_heads = sum(data)\n",
+    "    no_of_tails = N - no_of_heads\n",
+    "    return H**no_of_heads * (1-H)**no_of_tails\n",
+    "def posterior(H,data):\n",
+    "    p=prior(H)*likelihood(H,data)\n",
+    "    norm=np.trapz(p,H)\n",
+    "    return p/norm"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The next step is to confront this setup with the simulated data. To get a feel for the result, it is instructive to see how the posterior pdf evolves as we obtain more and more data pertaining to the coin. The results of such an analyses is shown in Fig. [fig:coinflipping](#fig:coinflipping)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "x=np.linspace(0,1,100)\n",
+    "fig, axs = plt.subplots(nrows=4,ncols=3,sharex=True,sharey='row')\n",
+    "axs_vec=np.reshape(axs,-1)\n",
+    "axs_vec[0].plot(x,prior(x))\n",
+    "for ndouble in range(11):\n",
+    "    ax=axs_vec[1+ndouble]\n",
+    "    ax.plot(x,posterior(x,heads[:2**ndouble]))\n",
+    "    ax.text(0.1, 0.8, '$N={0}$'.format(2**ndouble), transform=ax.transAxes)\n",
+    "for row in range(4): axs[row,0].set_ylabel('$p(H|D_\\mathrm{obs},I)$')\n",
+    "for col in range(3): axs[-1,col].set_xlabel('$H$')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "\n",
+    "
    \n",
+    "\n",
+    "
    The evolution of the posterior pdf for the bias-weighting of a coin, as the number of data available increases. The figure on the top left-hand corner of each panel shows the number of data included in the analysis.
    \n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "The panel in the top left-hand corner shows the posterior pdf for $H$ given no data, i.e., it is the same as the prior pdf of Eq. ([3](#eq:coin_prior_uniform)). It indicates that we have no more reason to believe that the coin is fair than we have to think that it is double-headed, double-tailed, or of any other intermediate bias-weighting.\n",
+    "\n",
+    "The first flip is obviously tails. At this point we have no evidence that the coin has a side with heads, as indicated by the pdf going to zero as $H \\to 1$. The second flip is obviously heads and we have now excluded both extreme options $H=0$ (double-tailed) and $H=1$ (double-headed). We can note that the posterior at this point has the simple form $p(H|D,I) = H(1-H)$ for $0 \\le H \\le 1$.\n",
+    "\n",
+    "The remainder of Fig. [fig:coinflipping](#fig:coinflipping) shows how the posterior pdf evolves as the number of data analysed becomes larger and larger. We see that the position of the maximum moves around, but that the amount by which it does so decreases with the increasing number of observations. The width of the posterior pdf also becomes narrower with more data, indicating that we are becoming increasingly confident in our estimate of the bias-weighting. For the coin in this example, the best estimate of $H$ eventually converges to 0.6, which, of course, was the value chosen to simulate the flips.\n",
+    "\n",
+    "\n",
+    "## A few words on different priors\n",
+    "* uniform\n",
+    "\n",
+    "* Gaussian\n",
+    "\n",
+    "* Jeffrey's prior\n",
+    "\n",
+    "Repeat the coin flipping experiment with other priors.\n",
+    "\n",
+    "\n",
+    "## Bayesian parameter estimation (single parameter)\n",
+    "We will now consider the very important task of model parameter estimation using statistical inference. \n",
+    "[CF 1: maybe stress that model parameters are not random variables, and the meaning of parameter estimation is therefore very different between frequentist and bayesian approaches.]\n",
+    "\n",
+    "Throughout this section we will consider a specific example that involves a model with a single parameter: \"Measured flux from a star\".\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "### Example: Measured flux from a star\n",
+    "\n",
+    "Adapted from the blog [Pythonic Perambulations](http://jakevdp.github.io) by Jake VanderPlas.\n",
+    "\n",
+    "Imagine that we point our telescope to the sky, and observe the light coming from a single star. For the time being, we'll assume that the star's true flux is constant with time, i.e. that is it has a fixed value $F_\\mathrm{true}$ (we'll also ignore effects like sky noise and other sources of systematic error). We'll assume that we perform a series of $N$ measurements with our telescope, where the ith measurement reports the observed photon flux $F_i$ and error $e_i$[^errors].\n",
+    "The question is, given this set of measurements $D = \\{F_i, e_i\\}$, what is our best estimate of the true flux $F_\\mathrm{true}$?\n",
+    "\n",
+    "[^errors]: We'll make the reasonable assumption that errors are Gaussian. In a Frequentist perspective, $e_i$ is the standard deviation of the results of a single measurement event in the limit of repetitions of *that event*. In the Bayesian perspective, $e_i$ is the standard deviation of the (Gaussian) probability distribution describing our knowledge of that particular measurement given its observed value.\n",
+    "\n",
+    "Because the measurements are number counts, a Poisson distribution is a good approximation to the measurement process:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "np.random.seed(1)      # for repeatability\n",
+    "F_true = 1000          # true flux, say number of photons measured in 1 second\n",
+    "N = 50                 # number of measurements\n",
+    "F = stats.poisson(F_true).rvs(N)\n",
+    "                       # N measurements of the flux\n",
+    "e = np.sqrt(F)         # errors on Poisson counts estimated via square root"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Now let's make a simple visualization of the \"observed\" data, see Fig. [fig:flux](#fig:flux)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "fig, ax = plt.subplots()\n",
+    "ax.errorbar(F, np.arange(N), xerr=e, fmt='ok', ecolor='gray', alpha=0.5)\n",
+    "ax.vlines([F_true], 0, N, linewidth=5, alpha=0.2)\n",
+    "ax.set_xlabel(\"Flux\");ax.set_ylabel(\"measurement number\");"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "\n",
+    "
    \n",
+    "\n",
+    "
    Single photon counts (flux measurements).
    \n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "These measurements each have a different error $e_i$ which is estimated from Poisson statistics using the standard square-root rule. In this toy example we already know the true flux $F_\\mathrm{true}$, but the question is this: given our measurements and errors, what is our best estimate of the true flux?\n",
+    "\n",
+    "Let's take a look at the frequentist and Bayesian approaches to solving this.\n",
+    "\n",
+    "### Simple Photon Counts: Frequentist Approach\n",
+    "\n",
+    "We'll start with the classical frequentist maximum likelihood approach. Given a single observation $D_i = (F_i, e_i)$, we can compute the probability distribution of the measurement given the true flux Ftrue given our assumption of Gaussian errors"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation}\n",
+    "p(D_i | F_\\mathrm{true}, I) = \\frac{1}{\\sqrt{2\\pi e_i^2}} \\exp \\left( \\frac{-(F_i-F_\\mathrm{true})^2}{2e_i^2} \\right).\n",
+    "\\label{_auto2} \\tag{5}\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "This should be read \"the probability of $D_i$ given $F_\\mathrm{true}$\n",
+    "equals ...\". You should recognize this as a normal distribution with mean $F_\\mathrm{true}$ and standard deviation $e_i$.\n",
+    "\n",
+    "We construct the *likelihood function* by computing the product of the probabilities for each data point"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation}\n",
+    "\\mathcal{L}(D | F_\\mathrm{true}, I) = \\prod_{i=1}^N p(D_i | F_\\mathrm{true}, I),\n",
+    "\\label{_auto3} \\tag{6}\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "here $D = \\{D_i\\}$ represents the entire set of measurements. Because the value of the likelihood can become very small, it is often more convenient to instead compute the log-likelihood. Combining the previous two equations and computing the log, we have"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation}\n",
+    "\\log\\mathcal{L} = -\\frac{1}{2} \\sum_{i=1}^N \\left[ \\log(2\\pi e_i^2) +  \\frac{(F_i-F_\\mathrm{true})^2}{e_i^2} \\right].\n",
+    "\\label{_auto4} \\tag{7}\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "What we'd like to do is determine $F_\\mathrm{true}$ such that the likelihood is maximized. For this simple problem, the maximization can be computed analytically (i.e. by setting $d\\log\\mathcal{L}/d F_\\mathrm{true} = 0$). This results in the following observed estimate of $F_\\mathrm{true}$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation}\n",
+    "F_\\mathrm{est} = \\sum_{i=1}^N w_i F_i; \\quad w_i = 1/e_i^2.\n",
+    "\\label{_auto5} \\tag{8}\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Notice that in the special case of all errors $e_i$ being equal, this reduces to"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation}\n",
+    "F_\\mathrm{est} = \\frac{1}{N} \\sum_{i=1} F_i.\n",
+    "\\label{_auto6} \\tag{9}\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "That is, in agreement with intuition, $F_\\mathrm{est}$ is simply the mean of the observed data when errors are equal.\n",
+    "\n",
+    "We can go further and ask what the error of our estimate is. In the frequentist approach, this can be accomplished by fitting a Gaussian approximation to the likelihood curve at maximum; in this simple case this can also be solved analytically (the sum of Gaussians is also a Gaussian). It can be shown that the standard deviation of this Gaussian approximation is"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation}\n",
+    "\\sigma_\\mathrm{est} = \\sum_{i=1}^N w_i.\n",
+    "\\label{_auto7} \\tag{10}\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "These results are fairly simple calculations; let's evaluate them for our toy dataset:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "w=1./e**2\n",
+    "print(\"\"\"\n",
+    "F_true = {0}\n",
+    "F_est = {1:.0f} +/- {2:.0f} (based on {3} measurements) \"\"\"\\\n",
+    "          .format(F_true, (w * F).sum() / w.sum(), w.sum() ** -0.5, N))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "`F_true = 1000` \n",
+    "`F_est = 998 +/- 4 (based on 50 measurements)` \n",
+    "\n",
+    "We find that for 50 measurements of the flux, our estimate has an error of about 0.4% and is consistent with the input value.\n",
+    "\n",
+    "\n",
+    "### Simple Photon Counts: Bayesian Approach\n",
+    "\n",
+    "The Bayesian approach, as you might expect, begins and ends with probabilities. Our hypothesis is that the star has a constant flux $F_\\mathrm{true}$. It recognizes that what we fundamentally want to compute is our knowledge of the parameters in question given the data and other information (such as our knowledge of uncertainties for the observed values), i.e. in this case, $p(F_\\mathrm{true} | D,I)$.\n",
+    "Note that this formulation of the problem is fundamentally contrary to the frequentist philosophy, which says that probabilities have no meaning for model parameters like $F_\\mathrm{true}$. Nevertheless, within the Bayesian philosophy this is perfectly acceptable.\n",
+    "\n",
+    "To compute this result, Bayesians next apply Bayes' Theorem ([1](#eq:bayes)).\n",
+    "If we set the prior $p(F_\\mathrm{true}|I) \\propto 1$ (a flat prior), we find\n",
+    "$p(F_\\mathrm{true}|D,I) \\propto p(D | F_\\mathrm{true},I) \\equiv \\mathcal{L}(D | F_\\mathrm{true},I)$\n",
+    "and the Bayesian probability is maximized at precisely the same value as the frequentist result! So despite the philosophical differences, we see that (for this simple problem at least) the Bayesian and frequentist point estimates are equivalent.\n",
+    "\n",
+    "### A note about priors\n",
+    "\n",
+    "The prior allows inclusion of other information into the computation, which becomes very useful in cases where multiple measurement strategies are being combined to constrain a single model. The necessity to specify a prior, however, is one of the more controversial pieces of Bayesian analysis.\n",
+    "A frequentist will point out that the prior is problematic when no true prior information is available. Though it might seem straightforward to use a noninformative prior like the flat prior mentioned above, there are some [surprisingly subtleties](http://normaldeviate.wordpress.com/2013/07/13/lost-causes-in-statistics-ii-noninformative- priors/comment-page-1/) involved. It turns out that in many situations, a truly noninformative prior does not exist! Frequentists point out that the subjective choice of a prior which necessarily biases your result has no place in statistical data analysis.\n",
+    "A Bayesian would counter that frequentism doesn't solve this problem, but simply skirts the question. Frequentism can often be viewed as simply a special case of the Bayesian approach for some (implicit) choice of the prior: a Bayesian would say that it's better to make this implicit choice explicit, even if the choice might include some subjectivity.\n",
+    "\n",
+    "### Simple Photon Counts: Bayesian approach in practice\n",
+    "\n",
+    "Leaving these philosophical debates aside for the time being, let's address how Bayesian results are generally computed in practice. For a one parameter problem like the one considered here, it's as simple as computing the posterior probability $p(F_\\mathrm{true} | D,I)$ as a function of $F_\\mathrm{true}$: this is the distribution reflecting our knowledge of the parameter $F_\\mathrm{true}$.\n",
+    "But as the dimension of the model grows, this direct approach becomes increasingly intractable. For this reason, Bayesian calculations often depend on sampling methods such as Markov Chain Monte Carlo (MCMC). For this practical example, let us apply an MCMC approach using Dan Foreman-Mackey's [emcee](http://dan.iel.fm/emcee/current/) package. Keep in mind here that the goal is to generate a set of points drawn from the posterior probability distribution, and to use those points to determine the answer we seek.\n",
+    "To perform this MCMC, we start by defining Python functions for the prior $p(F_\\mathrm{true} | I)$, the likelihood $p(D | F_\\mathrm{true},I)$, and the posterior $p(F_\\mathrm{true} | D,I)$, noting that none of these need be properly normalized. Our model here is one-dimensional, but to handle multi-dimensional models we'll define the model in terms of an array of parameters $\\boldsymbol{\\alpha}$, which in this case is $\\boldsymbol{\\alpha} = [F_\\mathrm{true}]$"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "def log_prior(alpha):\n",
+    "    return 0 # flat prior\n",
+    "\n",
+    "def log_likelihood(alpha, F, e):\n",
+    "    return -0.5 * np.sum(np.log(2 * np.pi * e ** 2) \\\n",
+    "                             + (F - alpha[0]) ** 2 / e ** 2)\n",
+    "                             \n",
+    "def log_posterior(alpha, F, e):\n",
+    "    return log_prior(alpha) + log_likelihood(alpha, F, e)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Now we set up the problem, including generating some random starting guesses for the multiple chains of points."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "ndim = 1      # number of parameters in the model\n",
+    "nwalkers = 50 # number of MCMC walkers\n",
+    "nburn = 1000  # \"burn-in\" period to let chains stabilize\n",
+    "nsteps = 2000 # number of MCMC steps to take\n",
+    "# we'll start at random locations between 0 and 2000\n",
+    "starting_guesses = 2000 * np.random.rand(nwalkers, ndim)\n",
+    "sampler = emcee.EnsembleSampler(nwalkers, ndim, log_posterior, args=[F,e])\n",
+    "sampler.run_mcmc(starting_guesses, nsteps)\n",
+    "# Shape of sampler.chain  = (nwalkers, nsteps, ndim)\n",
+    "# Flatten the sampler chain and discard burn-in points:\n",
+    "samples = sampler.chain[:, nburn:, :].reshape((-1, ndim))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "If this all worked correctly, the array sample should contain a series of 50,000 points drawn from the posterior. Let's plot them and check. See results in Fig. [fig:flux-bayesian](#fig:flux-bayesian)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "fig, ax = plt.subplots()\n",
+    "ax.hist(samples, bins=50, histtype=\"stepfilled\", alpha=0.3, normed=True)\n",
+    "ax.set_xlabel(r'$F_\\mathrm{est}$')\n",
+    "ax.set_ylabel(r'$p(F_\\mathrm{est}|D,I)$')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "\n",
+    "
    \n",
+    "\n",
+    "
    Bayesian posterior pdf (represented by a histogram of MCMC samples) from flux measurements.
    \n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "### Best estimates and confidence intervals\n",
+    "\n",
+    "The posterior distribution from our Bayesian data analysis is the key quantity that encodes our inference about the values of the model parameters, given the data and the relevant background information. Often, however, we wish to summarize this result with just a few numbers: the best estimate and a measure of its reliability. \n",
+    "\n",
+    "There are a few different options for this. The choice of the most appropriate one depends mainly on the shape of the posterior distribution:\n",
+    "\n",
+    "*Symmetric posterior pdfs*: Since the probability (density) associated with any particular value of the parameter is a measure of how much we believe that it lies in the neighbourhood of that point, our best estimate is given by the maximum of the posterior pdf. If we denote the quantity of interest by $X$, with a posterior pdf $P =p(X|D,I)$, then the best estimate of its value $X_0$ is given by the condition $dP/dX|_{X=X_0}=0$. Strictly speaking, we should also check the sign of the second derivative to ensure that $X_0$ represents a maximum.\n",
+    "\n",
+    "To obtain a measure of the reliability of this best estimate, we need to look at the width or spread of the posterior pdf about $X_0$. When considering the behaviour of any function in the neighbourhood of a particular point, it is often helpful to carry out a Taylor series expansion; this is simply a standard tool for (locally) approximating a complicated function by a low-order polynomial. The linear term is zero at the maximum and the quadratic term is often the dominating one determining the width of the posterior pdf. Ignoring all the higher-order terms we arrive at the Gaussian approximation"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "
    \n",
+    "\n",
+    "$$\n",
+    "\\begin{equation}\n",
+    "p(X|D,I) \\approx \\frac{1}{\\sigma\\sqrt{2\\pi}} \\exp \\left[ -\\frac{(x-\\mu)^2}{2\\sigma^2} \\right],\n",
+    "\\label{_auto8} \\tag{11}\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "where the mean $\\mu = X_0$ and the variance $\\sigma = \\left( - \\left. \\frac{d^2L}{dX^2} \\right|_{X_0} \\right)^{-1/2}$, where $L$ is the logarithm of the posterior $P$. Our inference about the quantity of interest is conveyed very concisely, therefore, by the statement $X = X_0 \\pm \\sigma$, and"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "$$\n",
+    "p(X_0-\\sigma < X < X_0+\\sigma | D,I) = \\int_{X_0-\\sigma}^{X_0+\\sigma} p(X|D,I) dX \\approx 0.67.\n",
+    "$$\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "*Asymmetric posterior pdfs*: While the maximum of the posterior ($X_0$) can still be regarded as giving the best estimate, the true value is now more likely to be on one side of this rather than the other. Alternatively one can compute the mean value, $\\langle X \\rangle = \\int X p(X|D,I) dX$, although this tends to overemphasise very long tails. The best option is probably a compromise that can be employed when having access to a large sample from the posterior (as provided by an MCMC), namely to give the median of this ensamble.\n",
+    "\n",
+    "Furthermore, the concept of an error-bar does not seem appropriate in this case, as it implicitly entails the idea of symmetry. A good way of expressing the reliability with which a parameter can be inferred, for an asymmetric posterior pdf, is rather through a *confidence interval*. Since the area under the posterior pdf between $X_1$ and $X_2$ is proportional to how much we believe that $X$ lies in that range, the shortest interval that encloses 67% of the area represents a sensible measure of the uncertainty of the estimate. Obviously we can choose to provide some other degree-of-belief that we think is relevant for the case at hand. Assuming that the posterior pdf has been normalized, to have unit area, we need to find $X_1$ and $X_2$ such that:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "$$\n",
+    "$$\n",
+    "p(X_1 < X < X_2 | D,I) = \\int_{X_1}^{X_2} p(X|D,I) dX \\approx 0.67, \n",
+    "$$\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "where the difference $X_2 - X_1$ is as small as possible. The region $X_1 < X < X_2$ is then called the shortest 67% confidence interval. \n",
+    "\n",
+    "*Multimodal posterior pdfs*: We can sometimes obtain posteriors which are multimodal; i.e. contains several disconnected regions with large probabilities. There is no difficulty when one of the maxima is very much larger than the others: we can simply ignore the subsidiary solutions, to a good approximation, and concentrate on the global maximum. The problem arises when there are several maxima of comparable magnitude. What do we now mean by a best estimate, and how should we quantify its reliability? The idea of a best estimate and an error-bar, or even a confidence interval, is merely an attempt to summarize the posterior with just two or three numbers; sometimes this just can’t be done, and so these concepts are not valid. For the bimodal case we might be able to characterize the posterior in terms of a few numbers: two best estimates and their associated error-bars, or disjoint confidence intervals. For a general multimodal pdf, the most honest thing we can do is just display the posterior itself.\n",
+    "\n",
+    "### Simple Photon Counts: Best estimates and confidence intervals\n",
+    "\n",
+    "To compute these numbers for our example, you would run:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "sampper=np.percentile(samples, [2.5, 16.5, 50, 83.5, 97.5],axis=0).flatten()\n",
+    "print(\"\"\"\n",
+    "F_true = {0}\n",
+    "Based on {1} measurements the posterior point estimates are:\n",
+    "...F_est = {2:.0f} +/- {3:.0f}\n",
+    "or using credible intervals:\n",
+    "...F_est = {4:.0f}          (posterior median) \n",
+    "...F_est in [{5:.0f}, {6:.0f}] (67% credible interval) \n",
+    "...F_est in [{7:.0f}, {8:.0f}] (95% credible interval) \"\"\"\\\n",
+    "          .format(F_true, N, np.mean(samples), np.std(samples), \\\n",
+    "                      sampper[2], sampper[1], sampper[3], sampper[0], sampper[4]))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "`F_true = 1000` \n",
+    "`Based on 50 measurements the posterior point estimates are:` \n",
+    "`...F_est = 998 +/- 4` \n",
+    "`or using credible intervals:` \n",
+    "`...F_est = 998          (posterior median)`  \n",
+    "`...F_est in [993, 1002] (67% credible interval)`  \n",
+    "`...F_est in [989, 1006] (95% credible interval)`  \n",
+    "\n",
+    "In this particular example, the posterior pdf is actually a Gaussian (since it is constructed as a product of Gaussians), and the mean and variance from the quadratic approximation will agree exactly with the frequentist approach.\n",
+    "\n",
+    "From this final result you might come away with the impression that the Bayesian method is unnecessarily complicated, and in this case it certainly is. Using an MCMC sampler to characterize a one-dimensional normal distribution is a bit like using the Death Star to destroy a beach ball, but we did this here because it demonstrates an approach that can scale to complicated posteriors in many, many dimensions, and can provide nice results in more complicated situations where an analytic likelihood approach is not possible.\n",
+    "\n",
+    "Furthermore, as data and models grow in complexity, the two approaches can diverge greatly. \n",
+    "\n",
+    "\n",
+    "## Bayesian parameter estimation (multiple parameters, covariance)\n",
+    "* multidimensional posterior pdf:s\n",
+    "\n",
+    "* nuisance parameters (e.g. background subtraction?)\n",
+    "\n",
+    "* corner plots, covariance, correlations\n",
+    "\n",
+    "* best example?\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "## Bayesian model selection\n",
+    "* Bayesian evidence\n",
+    "\n",
+    "* Occam's razor\n",
+    "\n",
+    "* Best example? How many spectral lines are there?"
+   ]
+  }
+ ],
+ "metadata": {},
+ "nbformat": 4,
+ "nbformat_minor": 4
+}
diff --git a/doc/src/LectureNotes/gaussian.pdf b/doc/src/LectureNotes/gaussian.pdf
index 6726dec52..b58084dc5 100644
Binary files a/doc/src/LectureNotes/gaussian.pdf and b/doc/src/LectureNotes/gaussian.pdf differ