diff --git a/doc/LectureNotes/_build/.doctrees/environment.pickle b/doc/LectureNotes/_build/.doctrees/environment.pickle index 403cb743d..21be02a4c 100644 Binary files a/doc/LectureNotes/_build/.doctrees/environment.pickle and b/doc/LectureNotes/_build/.doctrees/environment.pickle differ diff --git a/doc/LectureNotes/_build/.doctrees/week43.doctree b/doc/LectureNotes/_build/.doctrees/week43.doctree index 3ebb010d5..5c22ee805 100644 Binary files a/doc/LectureNotes/_build/.doctrees/week43.doctree and b/doc/LectureNotes/_build/.doctrees/week43.doctree differ diff --git a/doc/LectureNotes/_build/html/_sources/week43.ipynb b/doc/LectureNotes/_build/html/_sources/week43.ipynb index 47032b0fe..202263be5 100644 --- a/doc/LectureNotes/_build/html/_sources/week43.ipynb +++ b/doc/LectureNotes/_build/html/_sources/week43.ipynb @@ -2,8 +2,10 @@ "cells": [ { "cell_type": "markdown", - "id": "cda62a86", - "metadata": {}, + "id": "677a195d", + "metadata": { + "editable": true + }, "source": [ "\n", @@ -12,8 +14,10 @@ }, { "cell_type": "markdown", - "id": "bdae1fa5", - "metadata": {}, + "id": "e943e8f0", + "metadata": { + "editable": true + }, "source": [ "# Week 43: Deep Learning: Constructing a Neural Network code and solving differential equations\n", "**Morten Hjorth-Jensen**, Department of Physics, University of Oslo and Department of Physics and Astronomy and Facility for Rare Isotope Beams, Michigan State University\n", @@ -23,8 +27,10 @@ }, { "cell_type": "markdown", - "id": "e2c09dd5", - "metadata": {}, + "id": "22cdced0", + "metadata": { + "editable": true + }, "source": [ "## Plans for week 43\n", "\n", @@ -39,8 +45,10 @@ }, { "cell_type": "markdown", - "id": "7de8bfe0", - "metadata": {}, + "id": "a57ef6df", + "metadata": { + "editable": true + }, "source": [ "## Exercises and lab session week 43\n", "**Lab sessions on Tuesday and Wednesday.**\n", @@ -54,8 +62,10 @@ }, { "cell_type": "markdown", - "id": "2664d901", - "metadata": {}, + "id": "c4a134ee", + "metadata": { + "editable": true + }, "source": [ "## Mathematics of deep learning\n", "\n", @@ -68,8 +78,10 @@ }, { "cell_type": "markdown", - "id": "46bb5558", - "metadata": {}, + "id": "dd73045e", + "metadata": { + "editable": true + }, "source": [ "## Reminder on books with hands-on material and codes\n", "* Sebastian Rashcka et al, Machine learning with Scikit-Learn and PyTorch at " @@ -77,8 +89,10 @@ }, { "cell_type": "markdown", - "id": "1c6ad86d", - "metadata": {}, + "id": "372f4aa4", + "metadata": { + "editable": true + }, "source": [ "## Reading recommendations\n", "\n", @@ -89,8 +103,10 @@ }, { "cell_type": "markdown", - "id": "9fbf4898", - "metadata": {}, + "id": "05efbc6f", + "metadata": { + "editable": true + }, "source": [ "## Using Automatic differentiation\n", "\n", @@ -100,8 +116,10 @@ }, { "cell_type": "markdown", - "id": "135a7122", - "metadata": {}, + "id": "d04b0948", + "metadata": { + "editable": true + }, "source": [ "## Back propagation and automatic differentiation\n", "\n", @@ -115,16 +133,20 @@ }, { "cell_type": "markdown", - "id": "45236eaf", - "metadata": {}, + "id": "b9d41f5c", + "metadata": { + "editable": true + }, "source": [ "## Lecture Monday October 21" ] }, { "cell_type": "markdown", - "id": "8997a10d", - "metadata": {}, + "id": "68606991", + "metadata": { + "editable": true + }, "source": [ "## Setting up the back propagation algorithm and algorithm for a feed forward NN, initalizations\n", "This is a reminder from where we ended last week.\n", @@ -148,8 +170,10 @@ }, { "cell_type": "markdown", - "id": "df88cb72", - "metadata": {}, + "id": "a564a394", + "metadata": { + "editable": true + }, "source": [ "## Setting up the back propagation algorithm, part 1\n", "\n", @@ -169,8 +193,10 @@ }, { "cell_type": "markdown", - "id": "46d7ebde", - "metadata": {}, + "id": "43ab4381", + "metadata": { + "editable": true + }, "source": [ "## Setting up the back propagation algorithm, part 2\n", "\n", @@ -179,8 +205,10 @@ }, { "cell_type": "markdown", - "id": "3ad237f5", - "metadata": {}, + "id": "f578cee4", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\delta_j^L = \\sigma'(z_j^L)\\frac{\\partial {\\cal C}}{\\partial (a_j^L)}.\n", @@ -189,16 +217,20 @@ }, { "cell_type": "markdown", - "id": "f6906530", - "metadata": {}, + "id": "fbc05943", + "metadata": { + "editable": true + }, "source": [ "Then we compute the back propagate error for each $l=L-1,L-2,\\dots,1$ as" ] }, { "cell_type": "markdown", - "id": "e9613a49", - "metadata": {}, + "id": "6ad3c28f", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\delta_j^l = \\sum_k \\delta_k^{l+1}w_{kj}^{l+1}\\sigma'(z_j^l).\n", @@ -207,8 +239,10 @@ }, { "cell_type": "markdown", - "id": "dba468b9", - "metadata": {}, + "id": "135513b5", + "metadata": { + "editable": true + }, "source": [ "## Setting up the Back propagation algorithm, part 3\n", "\n", @@ -219,8 +253,10 @@ }, { "cell_type": "markdown", - "id": "21c684a1", - "metadata": {}, + "id": "f643ecdf", + "metadata": { + "editable": true + }, "source": [ "$$\n", "w_{ij}^l\\leftarrow = w_{ij}^l- \\eta \\delta_j^la_i^{l-1},\n", @@ -229,8 +265,10 @@ }, { "cell_type": "markdown", - "id": "6acf04a3", - "metadata": {}, + "id": "077ef1c2", + "metadata": { + "editable": true + }, "source": [ "$$\n", "b_j^l \\leftarrow b_j^l-\\eta \\frac{\\partial {\\cal C}}{\\partial b_j^l}=b_j^l-\\eta \\delta_j^l,\n", @@ -239,16 +277,20 @@ }, { "cell_type": "markdown", - "id": "b0558fbe", - "metadata": {}, + "id": "50cf7472", + "metadata": { + "editable": true + }, "source": [ "with $\\eta$ being the learning rate." ] }, { "cell_type": "markdown", - "id": "6b424064", - "metadata": {}, + "id": "a10f406c", + "metadata": { + "editable": true + }, "source": [ "## Updating the gradients\n", "\n", @@ -257,8 +299,10 @@ }, { "cell_type": "markdown", - "id": "7bc86e31", - "metadata": {}, + "id": "626cba8f", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\delta_j^l = \\sum_k \\delta_k^{l+1}w_{kj}^{l+1}\\sigma'(z_j^l),\n", @@ -267,16 +311,20 @@ }, { "cell_type": "markdown", - "id": "f7a10087", - "metadata": {}, + "id": "2ebe05ad", + "metadata": { + "editable": true + }, "source": [ "we update the weights and the biases using gradient descent for each $l=L-1,L-2,\\dots,1$ and update the weights and biases according to the rules" ] }, { "cell_type": "markdown", - "id": "2bf12837", - "metadata": {}, + "id": "bfbbac35", + "metadata": { + "editable": true + }, "source": [ "$$\n", "w_{ij}^l\\leftarrow = w_{ij}^l- \\eta \\delta_j^la_i^{l-1},\n", @@ -285,8 +333,10 @@ }, { "cell_type": "markdown", - "id": "ad07731e", - "metadata": {}, + "id": "ac515399", + "metadata": { + "editable": true + }, "source": [ "$$\n", "b_j^l \\leftarrow b_j^l-\\eta \\frac{\\partial {\\cal C}}{\\partial b_j^l}=b_j^l-\\eta \\delta_j^l,\n", @@ -295,8 +345,10 @@ }, { "cell_type": "markdown", - "id": "30c6ae90", - "metadata": {}, + "id": "4703cc51", + "metadata": { + "editable": true + }, "source": [ "## Activation functions\n", "\n", @@ -316,8 +368,10 @@ }, { "cell_type": "markdown", - "id": "5dfa6d1c", - "metadata": {}, + "id": "cab6ad03", + "metadata": { + "editable": true + }, "source": [ "### Activation functions, examples\n", "\n", @@ -326,8 +380,10 @@ }, { "cell_type": "markdown", - "id": "e8d0be63", - "metadata": {}, + "id": "ad6042eb", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\sigma(x) = \\frac{1}{1 + e^{-x}},\n", @@ -336,16 +392,20 @@ }, { "cell_type": "markdown", - "id": "b3ec0efa", - "metadata": {}, + "id": "1e96b7e7", + "metadata": { + "editable": true + }, "source": [ "and the *hyperbolic tangent* function" ] }, { "cell_type": "markdown", - "id": "78c46915", - "metadata": {}, + "id": "c600792f", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\sigma(x) = \\tanh(x)\n", @@ -354,8 +414,10 @@ }, { "cell_type": "markdown", - "id": "87e6973b", - "metadata": {}, + "id": "76d1da8e", + "metadata": { + "editable": true + }, "source": [ "## The RELU function family\n", "\n", @@ -373,8 +435,10 @@ }, { "cell_type": "markdown", - "id": "9c990bc8", - "metadata": {}, + "id": "8e7e0e0c", + "metadata": { + "editable": true + }, "source": [ "## ELU function\n", "\n", @@ -385,8 +449,10 @@ }, { "cell_type": "markdown", - "id": "9b7760f9", - "metadata": {}, + "id": "4b1f77d6", + "metadata": { + "editable": true + }, "source": [ "$$\n", "ELU(z) = \\left\\{\\begin{array}{cc} \\alpha\\left( \\exp{(z)}-1\\right) & z < 0,\\\\ z & z \\ge 0.\\end{array}\\right.\n", @@ -395,8 +461,10 @@ }, { "cell_type": "markdown", - "id": "47927a82", - "metadata": {}, + "id": "10826ba9", + "metadata": { + "editable": true + }, "source": [ "## Which activation function should we use?\n", "\n", @@ -415,8 +483,10 @@ }, { "cell_type": "markdown", - "id": "52554330", - "metadata": {}, + "id": "79d307b6", + "metadata": { + "editable": true + }, "source": [ "## More on activation functions, output layers\n", "\n", @@ -435,8 +505,10 @@ }, { "cell_type": "markdown", - "id": "4b7d3292", - "metadata": {}, + "id": "429b8d4e", + "metadata": { + "editable": true + }, "source": [ "## Setting up a Multi-layer perceptron model for classification\n", "\n", @@ -461,8 +533,10 @@ }, { "cell_type": "markdown", - "id": "38715b52", - "metadata": {}, + "id": "90d9f195", + "metadata": { + "editable": true + }, "source": [ "$$\n", "P(y = 0 \\mid \\boldsymbol{x}, \\boldsymbol{\\theta}) = \\frac{1}{1 + \\exp{(- \\boldsymbol{x}})} ,\n", @@ -471,16 +545,20 @@ }, { "cell_type": "markdown", - "id": "dcc0cbfa", - "metadata": {}, + "id": "73d2763e", + "metadata": { + "editable": true + }, "source": [ "and" ] }, { "cell_type": "markdown", - "id": "c686ecf6", - "metadata": {}, + "id": "e2a70a54", + "metadata": { + "editable": true + }, "source": [ "$$\n", "P(y = 1 \\mid \\boldsymbol{x}, \\boldsymbol{\\theta}) = 1 - P(y = 0 \\mid \\boldsymbol{x}, \\boldsymbol{\\theta}) ,\n", @@ -489,8 +567,10 @@ }, { "cell_type": "markdown", - "id": "79a8b207", - "metadata": {}, + "id": "7ed6c0dc", + "metadata": { + "editable": true + }, "source": [ "where $y \\in \\{0, 1\\}$ and $\\boldsymbol{\\theta}$ represents the weights and biases\n", "of our network." @@ -498,8 +578,10 @@ }, { "cell_type": "markdown", - "id": "3218b074", - "metadata": {}, + "id": "a42826b9", + "metadata": { + "editable": true + }, "source": [ "## Defining the cost function\n", "\n", @@ -508,8 +590,10 @@ }, { "cell_type": "markdown", - "id": "c0e50336", - "metadata": {}, + "id": "d8e401c5", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\mathcal{C}(\\boldsymbol{\\theta}) = - \\ln P(\\mathcal{D} \\mid \\boldsymbol{\\theta}) = - \\sum_{i=1}^n\n", @@ -519,8 +603,10 @@ }, { "cell_type": "markdown", - "id": "79138403", - "metadata": {}, + "id": "17808105", + "metadata": { + "editable": true + }, "source": [ "This last equality means that we can interpret our *cost* function as a sum over the *loss* function\n", "for each point in the dataset $\\mathcal{L}_i(\\boldsymbol{\\theta})$. \n", @@ -542,8 +628,10 @@ }, { "cell_type": "markdown", - "id": "1b3db35e", - "metadata": {}, + "id": "5a9b90ce", + "metadata": { + "editable": true + }, "source": [ "$$\n", "P(y_{ic} = 1 \\mid \\boldsymbol{x}_i, \\boldsymbol{\\theta}) = \\frac{\\exp{((\\boldsymbol{a}_i^{hidden})^T \\boldsymbol{w}_c)}}\n", @@ -553,8 +641,10 @@ }, { "cell_type": "markdown", - "id": "90626f76", - "metadata": {}, + "id": "57f33029", + "metadata": { + "editable": true + }, "source": [ "which reduces to the logistic function in the binary case. \n", "The likelihood of this $C$-class classifier\n", @@ -563,8 +653,10 @@ }, { "cell_type": "markdown", - "id": "596eb444", - "metadata": {}, + "id": "422eb23f", + "metadata": { + "editable": true + }, "source": [ "$$\n", "P(\\mathcal{D} \\mid \\boldsymbol{\\theta}) = \\prod_{i=1}^n \\prod_{c=0}^{C-1} [P(y_{ic} = 1)]^{y_{ic}} .\n", @@ -573,16 +665,20 @@ }, { "cell_type": "markdown", - "id": "5529d979", - "metadata": {}, + "id": "0eb634c3", + "metadata": { + "editable": true + }, "source": [ "Again we take the negative log-likelihood to define our cost function:" ] }, { "cell_type": "markdown", - "id": "f7d2523f", - "metadata": {}, + "id": "b3b208fc", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\mathcal{C}(\\boldsymbol{\\theta}) = - \\log{P(\\mathcal{D} \\mid \\boldsymbol{\\theta})}.\n", @@ -591,8 +687,10 @@ }, { "cell_type": "markdown", - "id": "7d8d61f1", - "metadata": {}, + "id": "3788aba1", + "metadata": { + "editable": true + }, "source": [ "See the logistic regression lectures for a full definition of the cost function.\n", "\n", @@ -601,8 +699,10 @@ }, { "cell_type": "markdown", - "id": "9751fb82", - "metadata": {}, + "id": "4d38ceec", + "metadata": { + "editable": true + }, "source": [ "## Example: binary classification problem\n", "\n", @@ -611,8 +711,10 @@ }, { "cell_type": "markdown", - "id": "3e6587c7", - "metadata": {}, + "id": "cee503b9", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\mathcal{C}(\\boldsymbol{\\beta}) = - \\sum_{i=1}^n \\left(y_i\\log{p(y_i \\vert x_i,\\boldsymbol{\\beta})}+(1-y_i)\\log{1-p(y_i \\vert x_i,\\boldsymbol{\\beta})}\\right),\n", @@ -621,16 +723,20 @@ }, { "cell_type": "markdown", - "id": "39bb5ba4", - "metadata": {}, + "id": "19b29e02", + "metadata": { + "editable": true + }, "source": [ "where we had defined the logistic (sigmoid) function" ] }, { "cell_type": "markdown", - "id": "fcb1f3d9", - "metadata": {}, + "id": "19507159", + "metadata": { + "editable": true + }, "source": [ "$$\n", "p(y_i =1\\vert x_i,\\boldsymbol{\\beta})=\\frac{\\exp{(\\beta_0+\\beta_1 x_i)}}{1+\\exp{(\\beta_0+\\beta_1 x_i)}},\n", @@ -639,16 +745,20 @@ }, { "cell_type": "markdown", - "id": "20fc130a", - "metadata": {}, + "id": "a15191e4", + "metadata": { + "editable": true + }, "source": [ "and" ] }, { "cell_type": "markdown", - "id": "66c7c0e9", - "metadata": {}, + "id": "c9df7a22", + "metadata": { + "editable": true + }, "source": [ "$$\n", "p(y_i =0\\vert x_i,\\boldsymbol{\\beta})=1-p(y_i =1\\vert x_i,\\boldsymbol{\\beta}).\n", @@ -657,8 +767,10 @@ }, { "cell_type": "markdown", - "id": "e757c738", - "metadata": {}, + "id": "0cae75ec", + "metadata": { + "editable": true + }, "source": [ "The parameters $\\boldsymbol{\\beta}$ were defined using a minimization method like gradient descent or Newton-Raphson's method. \n", "\n", @@ -668,8 +780,10 @@ }, { "cell_type": "markdown", - "id": "d048e79e", - "metadata": {}, + "id": "36b0506b", + "metadata": { + "editable": true + }, "source": [ "$$\n", "a_i^l = y_i = \\frac{\\exp{(z_i^l)}}{1+\\exp{(z_i^l)}},\n", @@ -678,16 +792,20 @@ }, { "cell_type": "markdown", - "id": "b97827e7", - "metadata": {}, + "id": "0c729c48", + "metadata": { + "editable": true + }, "source": [ "with" ] }, { "cell_type": "markdown", - "id": "c11528eb", - "metadata": {}, + "id": "060813af", + "metadata": { + "editable": true + }, "source": [ "$$\n", "z_i^l = \\sum_{j}w_{ij}^l a_j^{l-1}+b_i^l,\n", @@ -696,8 +814,10 @@ }, { "cell_type": "markdown", - "id": "94839c96", - "metadata": {}, + "id": "f2a5059b", + "metadata": { + "editable": true + }, "source": [ "where the superscript $l-1$ indicates that these are the outputs from layer $l-1$.\n", "Our cost function at the final layer $l=L$ is now" @@ -705,8 +825,10 @@ }, { "cell_type": "markdown", - "id": "694596c9", - "metadata": {}, + "id": "6763074d", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\mathcal{C}(\\boldsymbol{W}) = - \\sum_{i=1}^n \\left(t_i\\log{a_i^L}+(1-t_i)\\log{(1-a_i^L)}\\right),\n", @@ -715,16 +837,20 @@ }, { "cell_type": "markdown", - "id": "b4c8232e", - "metadata": {}, + "id": "f498b6b5", + "metadata": { + "editable": true + }, "source": [ "where we have defined the targets $t_i$. The derivatives of the cost function with respect to the output $a_i^L$ are then easily calculated and we get" ] }, { "cell_type": "markdown", - "id": "9d611a08", - "metadata": {}, + "id": "fedce928", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\frac{\\partial \\mathcal{C}(\\boldsymbol{W})}{\\partial a_i^L} = \\frac{a_i^L-t_i}{a_i^L(1-a_i^L)}.\n", @@ -733,16 +859,20 @@ }, { "cell_type": "markdown", - "id": "cb7847ef", - "metadata": {}, + "id": "429c11ae", + "metadata": { + "editable": true + }, "source": [ "In case we use another activation function than the logistic one, we need to evaluate other derivatives." ] }, { "cell_type": "markdown", - "id": "cd5be179", - "metadata": {}, + "id": "ddde8370", + "metadata": { + "editable": true + }, "source": [ "## The Softmax function\n", "In case we employ the more general case given by the Softmax equation, we need to evaluate the derivative of the activation function with respect to the activation $z_i^l$, that is we need" @@ -750,8 +880,10 @@ }, { "cell_type": "markdown", - "id": "96b7fe79", - "metadata": {}, + "id": "637bd194", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\frac{\\partial f(z_i^l)}{\\partial w_{jk}^l} =\n", @@ -761,16 +893,20 @@ }, { "cell_type": "markdown", - "id": "1040c590", - "metadata": {}, + "id": "0792109c", + "metadata": { + "editable": true + }, "source": [ "For the Softmax function we have" ] }, { "cell_type": "markdown", - "id": "3162bd5e", - "metadata": {}, + "id": "10f14691", + "metadata": { + "editable": true + }, "source": [ "$$\n", "f(z_i^l) = \\frac{\\exp{(z_i^l)}}{\\sum_{m=1}^K\\exp{(z_m^l)}}.\n", @@ -779,16 +915,20 @@ }, { "cell_type": "markdown", - "id": "b7f2b7b8", - "metadata": {}, + "id": "186529b5", + "metadata": { + "editable": true + }, "source": [ "Its derivative with respect to $z_j^l$ gives" ] }, { "cell_type": "markdown", - "id": "735fd9c0", - "metadata": {}, + "id": "68afc0da", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\frac{\\partial f(z_i^l)}{\\partial z_j^l}= f(z_i^l)\\left(\\delta_{ij}-f(z_j^l)\\right),\n", @@ -797,16 +937,20 @@ }, { "cell_type": "markdown", - "id": "c3cbbb04", - "metadata": {}, + "id": "cf48ea60", + "metadata": { + "editable": true + }, "source": [ "which in case of the simply binary model reduces to having $i=j$." ] }, { "cell_type": "markdown", - "id": "6462edbb", - "metadata": {}, + "id": "861a3bbe", + "metadata": { + "editable": true + }, "source": [ "## Developing a code for doing neural networks with back propagation\n", "\n", @@ -827,8 +971,10 @@ }, { "cell_type": "markdown", - "id": "7915dc1d", - "metadata": {}, + "id": "c8007ce7", + "metadata": { + "editable": true + }, "source": [ "## Collect and pre-process data\n", "\n", @@ -875,8 +1021,11 @@ { "cell_type": "code", "execution_count": 1, - "id": "eb736d79", - "metadata": {}, + "id": "29678f97", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "%matplotlib inline\n", @@ -927,8 +1076,10 @@ }, { "cell_type": "markdown", - "id": "01fcb50b", - "metadata": {}, + "id": "3ba6d7d6", + "metadata": { + "editable": true + }, "source": [ "## Train and test datasets\n", "\n", @@ -946,8 +1097,11 @@ { "cell_type": "code", "execution_count": 2, - "id": "e6b7f2e3", - "metadata": {}, + "id": "7df51dd1", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "from sklearn.model_selection import train_test_split\n", @@ -981,8 +1135,10 @@ }, { "cell_type": "markdown", - "id": "1ca032f8", - "metadata": {}, + "id": "249898fd", + "metadata": { + "editable": true + }, "source": [ "## Define model and architecture\n", "\n", @@ -1023,8 +1179,10 @@ }, { "cell_type": "markdown", - "id": "6e4e1e18", - "metadata": {}, + "id": "2d30ddd3", + "metadata": { + "editable": true + }, "source": [ "## Layers\n", "\n", @@ -1061,8 +1219,10 @@ }, { "cell_type": "markdown", - "id": "ab0ee6d8", - "metadata": {}, + "id": "ce9f14e4", + "metadata": { + "editable": true + }, "source": [ "## Weights and biases\n", "\n", @@ -1080,8 +1240,11 @@ { "cell_type": "code", "execution_count": 3, - "id": "15ad1599", - "metadata": {}, + "id": "c3f75b32", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "# building our neural network\n", @@ -1103,8 +1266,10 @@ }, { "cell_type": "markdown", - "id": "3efd576a", - "metadata": {}, + "id": "4ebf4f68", + "metadata": { + "editable": true + }, "source": [ "## Feed-forward pass\n", "\n", @@ -1129,8 +1294,10 @@ }, { "cell_type": "markdown", - "id": "715c5d46", - "metadata": {}, + "id": "c6760d1f", + "metadata": { + "editable": true + }, "source": [ "## Matrix multiplications\n", "\n", @@ -1164,8 +1331,11 @@ { "cell_type": "code", "execution_count": 4, - "id": "ecaa833e", - "metadata": {}, + "id": "98f3a5d5", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "# setup the feed-forward pass, subscript h = hidden layer\n", @@ -1207,8 +1377,10 @@ }, { "cell_type": "markdown", - "id": "c0319e65", - "metadata": {}, + "id": "19cd49ec", + "metadata": { + "editable": true + }, "source": [ "## Choose cost function and optimizer\n", "\n", @@ -1236,8 +1408,10 @@ }, { "cell_type": "markdown", - "id": "98c6696e", - "metadata": {}, + "id": "b0342f54", + "metadata": { + "editable": true + }, "source": [ "## Optimizing the cost function\n", "\n", @@ -1272,8 +1446,10 @@ }, { "cell_type": "markdown", - "id": "b5d1145a", - "metadata": {}, + "id": "7219731b", + "metadata": { + "editable": true + }, "source": [ "## Regularization\n", "\n", @@ -1304,8 +1480,10 @@ }, { "cell_type": "markdown", - "id": "b73247fd", - "metadata": {}, + "id": "248590b9", + "metadata": { + "editable": true + }, "source": [ "## Matrix multiplication\n", "\n", @@ -1343,8 +1521,11 @@ { "cell_type": "code", "execution_count": 5, - "id": "3d61438f", - "metadata": {}, + "id": "1810a36c", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "# to categorical turns our integer vector into a onehot representation\n", @@ -1419,8 +1600,10 @@ }, { "cell_type": "markdown", - "id": "6a20a495", - "metadata": {}, + "id": "e094a404", + "metadata": { + "editable": true + }, "source": [ "## Improving performance\n", "\n", @@ -1438,8 +1621,10 @@ }, { "cell_type": "markdown", - "id": "717b63a1", - "metadata": {}, + "id": "1fb3aab5", + "metadata": { + "editable": true + }, "source": [ "## Full object-oriented implementation\n", "\n", @@ -1450,8 +1635,11 @@ { "cell_type": "code", "execution_count": 6, - "id": "9d6c9929", - "metadata": {}, + "id": "20b3c187", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "class NeuralNetwork:\n", @@ -1557,8 +1745,10 @@ }, { "cell_type": "markdown", - "id": "e70667c5", - "metadata": {}, + "id": "f85564ae", + "metadata": { + "editable": true + }, "source": [ "## Evaluate model performance on test data\n", "\n", @@ -1574,8 +1764,11 @@ { "cell_type": "code", "execution_count": 7, - "id": "083dfe41", - "metadata": {}, + "id": "9be7082e", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "epochs = 100\n", @@ -1598,8 +1791,10 @@ }, { "cell_type": "markdown", - "id": "bffeadd3", - "metadata": {}, + "id": "255b5b3d", + "metadata": { + "editable": true + }, "source": [ "## Adjust hyperparameters\n", "\n", @@ -1610,8 +1805,11 @@ { "cell_type": "code", "execution_count": 8, - "id": "6513ac74", - "metadata": {}, + "id": "fc014642", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "eta_vals = np.logspace(-5, 1, 7)\n", @@ -1638,8 +1836,10 @@ }, { "cell_type": "markdown", - "id": "a6e305f6", - "metadata": {}, + "id": "96089af8", + "metadata": { + "editable": true + }, "source": [ "## Visualization" ] @@ -1647,8 +1847,11 @@ { "cell_type": "code", "execution_count": 9, - "id": "d920e285", - "metadata": {}, + "id": "04c4d29a", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "# visual representation of grid search\n", @@ -1688,8 +1891,10 @@ }, { "cell_type": "markdown", - "id": "0f1aaadd", - "metadata": {}, + "id": "472a5060", + "metadata": { + "editable": true + }, "source": [ "## scikit-learn implementation\n", "\n", @@ -1709,8 +1914,11 @@ { "cell_type": "code", "execution_count": 10, - "id": "156764f3", - "metadata": {}, + "id": "42e40777", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "from sklearn.neural_network import MLPClassifier\n", @@ -1733,8 +1941,10 @@ }, { "cell_type": "markdown", - "id": "2e5fec48", - "metadata": {}, + "id": "57fc5d93", + "metadata": { + "editable": true + }, "source": [ "## Visualization" ] @@ -1742,8 +1952,11 @@ { "cell_type": "code", "execution_count": 11, - "id": "ace47e70", - "metadata": {}, + "id": "90477fe7", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "# optional\n", @@ -1784,8 +1997,10 @@ }, { "cell_type": "markdown", - "id": "9d327f9b", - "metadata": {}, + "id": "29af8aff", + "metadata": { + "editable": true + }, "source": [ "## Building neural networks in Tensorflow and Keras\n", "\n", @@ -1800,8 +2015,10 @@ }, { "cell_type": "markdown", - "id": "2280cbce", - "metadata": {}, + "id": "a70cf999", + "metadata": { + "editable": true + }, "source": [ "## Tensorflow\n", "\n", @@ -1833,8 +2050,11 @@ { "cell_type": "code", "execution_count": 12, - "id": "dff81198", - "metadata": {}, + "id": "812c98fe", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "pip3 install tensorflow" @@ -1842,8 +2062,10 @@ }, { "cell_type": "markdown", - "id": "e9bf9cac", - "metadata": {}, + "id": "694c0bd7", + "metadata": { + "editable": true + }, "source": [ "and/or if you use **anaconda**, just write (or install from the graphical user interface)\n", "(current release of CPU-only TensorFlow)" @@ -1852,8 +2074,11 @@ { "cell_type": "code", "execution_count": 13, - "id": "1ccb8dba", - "metadata": {}, + "id": "1ecd18cc", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "conda create -n tf tensorflow\n", @@ -1862,8 +2087,10 @@ }, { "cell_type": "markdown", - "id": "efc3f4e2", - "metadata": {}, + "id": "584ab0ab", + "metadata": { + "editable": true + }, "source": [ "To install the current release of GPU TensorFlow" ] @@ -1871,8 +2098,11 @@ { "cell_type": "code", "execution_count": 14, - "id": "01556b01", - "metadata": {}, + "id": "a811e23d", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "conda create -n tf-gpu tensorflow-gpu\n", @@ -1881,8 +2111,10 @@ }, { "cell_type": "markdown", - "id": "b900f015", - "metadata": {}, + "id": "ab99008f", + "metadata": { + "editable": true + }, "source": [ "## Using Keras\n", "\n", @@ -1894,8 +2126,11 @@ { "cell_type": "code", "execution_count": 15, - "id": "3fd1f12f", - "metadata": {}, + "id": "fe547415", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "conda install keras" @@ -1903,8 +2138,10 @@ }, { "cell_type": "markdown", - "id": "20ec35c5", - "metadata": {}, + "id": "ed1e0fc6", + "metadata": { + "editable": true + }, "source": [ "You can look up the [instructions here](https://keras.io/) for more information.\n", "\n", @@ -1913,8 +2150,10 @@ }, { "cell_type": "markdown", - "id": "249e56f7", - "metadata": {}, + "id": "6aebcb51", + "metadata": { + "editable": true + }, "source": [ "## Collect and pre-process data\n", "\n", @@ -1924,8 +2163,11 @@ { "cell_type": "code", "execution_count": 16, - "id": "0ff66efc", - "metadata": {}, + "id": "a4a7d68d", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "# import necessary packages\n", @@ -1976,8 +2218,11 @@ { "cell_type": "code", "execution_count": 17, - "id": "2806c62b", - "metadata": {}, + "id": "21335418", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "from tensorflow.keras.layers import Input\n", @@ -2002,8 +2247,11 @@ { "cell_type": "code", "execution_count": 18, - "id": "3bf4c3f6", - "metadata": {}, + "id": "e81d58de", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "\n", @@ -2020,7 +2268,7 @@ " model.add(Dense(n_neurons_layer2, activation='sigmoid', kernel_regularizer=regularizers.l2(lmbd)))\n", " model.add(Dense(n_categories, activation='softmax'))\n", " \n", - " sgd = optimizers.SGD(lr=eta)\n", + " sgd = optimizers.SGD(learning_rate=eta)\n", " model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])\n", " \n", " return model" @@ -2029,8 +2277,11 @@ { "cell_type": "code", "execution_count": 19, - "id": "81a46485", - "metadata": {}, + "id": "b91287cc", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "DNN_keras = np.zeros((len(eta_vals), len(lmbd_vals)), dtype=object)\n", @@ -2053,8 +2304,11 @@ { "cell_type": "code", "execution_count": 20, - "id": "6e619c14", - "metadata": {}, + "id": "d108674e", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "# optional\n", @@ -2092,8 +2346,10 @@ }, { "cell_type": "markdown", - "id": "c303d97b", - "metadata": {}, + "id": "1198d799", + "metadata": { + "editable": true + }, "source": [ "## The Breast Cancer Data, now with Keras" ] @@ -2101,8 +2357,11 @@ { "cell_type": "code", "execution_count": 21, - "id": "03a57bfd", - "metadata": {}, + "id": "eb1eb4fb", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "\n", @@ -2215,7 +2474,7 @@ " else: #Subsequent layers are capable of automatic shape inferencing\n", " model.add(Dense(n_neuron,activation='relu',kernel_regularizer=regularizers.l2(lamda)))\n", " model.add(Dense(2,activation='softmax')) #2 outputs - ordered and disordered (softmax for prob)\n", - " sgd=optimizers.SGD(lr=eta)\n", + " sgd=optimizers.SGD(learning_rate=eta)\n", " model.compile(loss='categorical_crossentropy',optimizer=sgd,metrics=['accuracy'])\n", " return model\n", "\n", @@ -2275,8 +2534,10 @@ }, { "cell_type": "markdown", - "id": "f9ad1085", - "metadata": {}, + "id": "92834502", + "metadata": { + "editable": true + }, "source": [ "## Building a neural network code\n", "\n", @@ -2292,8 +2553,10 @@ }, { "cell_type": "markdown", - "id": "cf4f3c14", - "metadata": {}, + "id": "deb24cc1", + "metadata": { + "editable": true + }, "source": [ "### Learning rate methods\n", "\n", @@ -2312,8 +2575,11 @@ { "cell_type": "code", "execution_count": 22, - "id": "467ea7a2", - "metadata": {}, + "id": "d3619281", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "import autograd.numpy as np\n", @@ -2450,8 +2716,10 @@ }, { "cell_type": "markdown", - "id": "6af64d5a", - "metadata": {}, + "id": "f37fd7bc", + "metadata": { + "editable": true + }, "source": [ "### Usage of the above learning rate schedulers\n", "\n", @@ -2464,8 +2732,11 @@ { "cell_type": "code", "execution_count": 23, - "id": "290e0d00", - "metadata": {}, + "id": "5db34b9b", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "momentum_scheduler = Momentum(eta=1e-3, momentum=0.9)\n", @@ -2474,8 +2745,10 @@ }, { "cell_type": "markdown", - "id": "d1f43cc5", - "metadata": {}, + "id": "8b25a24d", + "metadata": { + "editable": true + }, "source": [ "Here is a small example for how a segment of code using schedulers\n", "could look. Switching out the schedulers is simple." @@ -2484,8 +2757,11 @@ { "cell_type": "code", "execution_count": 24, - "id": "0def5cec", - "metadata": {}, + "id": "2d5f2887", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "weights = np.ones((3,3))\n", @@ -2503,8 +2779,10 @@ }, { "cell_type": "markdown", - "id": "9b27dcd7", - "metadata": {}, + "id": "9070c2d3", + "metadata": { + "editable": true + }, "source": [ "### Cost functions\n", "\n", @@ -2517,8 +2795,11 @@ { "cell_type": "code", "execution_count": 25, - "id": "4217839c", - "metadata": {}, + "id": "c360c736", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "import autograd.numpy as np\n", @@ -2552,8 +2833,10 @@ }, { "cell_type": "markdown", - "id": "ef5f9fe9", - "metadata": {}, + "id": "42c7ec61", + "metadata": { + "editable": true + }, "source": [ "Below we give a short example of how these cost function may be used\n", "to obtain results if you wish to test them out on your own using\n", @@ -2563,8 +2846,11 @@ { "cell_type": "code", "execution_count": 26, - "id": "305e1479", - "metadata": {}, + "id": "cd9cb3a5", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "from autograd import grad\n", @@ -2581,8 +2867,10 @@ }, { "cell_type": "markdown", - "id": "1499b193", - "metadata": {}, + "id": "796aab4a", + "metadata": { + "editable": true + }, "source": [ "### Activation functions\n", "\n", @@ -2595,8 +2883,11 @@ { "cell_type": "code", "execution_count": 27, - "id": "e7b59b25", - "metadata": {}, + "id": "5ea4891d", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "import autograd.numpy as np\n", @@ -2650,8 +2941,10 @@ }, { "cell_type": "markdown", - "id": "57599619", - "metadata": {}, + "id": "2ff3c57c", + "metadata": { + "editable": true + }, "source": [ "Below follows a short demonstration of how to use an activation\n", "function. The derivative of the activation function will be important\n", @@ -2663,8 +2956,11 @@ { "cell_type": "code", "execution_count": 28, - "id": "5ee68acd", - "metadata": {}, + "id": "34289210", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "z = np.array([[4, 5, 6]]).T\n", @@ -2681,8 +2977,10 @@ }, { "cell_type": "markdown", - "id": "2d3295d9", - "metadata": {}, + "id": "0839f98b", + "metadata": { + "editable": true + }, "source": [ "### The Neural Network\n", "\n", @@ -2703,8 +3001,11 @@ { "cell_type": "code", "execution_count": 29, - "id": "c4cfc50a", - "metadata": {}, + "id": "0fdc2707", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "import math\n", @@ -3172,8 +3473,10 @@ }, { "cell_type": "markdown", - "id": "2e6e81c1", - "metadata": {}, + "id": "17bc192e", + "metadata": { + "editable": true + }, "source": [ "Before we make a model, we will quickly generate a dataset we can use\n", "for our linear regression problem as shown below" @@ -3182,8 +3485,11 @@ { "cell_type": "code", "execution_count": 30, - "id": "0d4f1ff8", - "metadata": {}, + "id": "f3f5d088", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "import autograd.numpy as np\n", @@ -3223,8 +3529,10 @@ }, { "cell_type": "markdown", - "id": "a8f1940f", - "metadata": {}, + "id": "53c3b4c2", + "metadata": { + "editable": true + }, "source": [ "Now that we have our dataset ready for the regression, we can create\n", "our regressor. Note that with the seed parameter, we can make sure our\n", @@ -3237,8 +3545,11 @@ { "cell_type": "code", "execution_count": 31, - "id": "ff6bd29b", - "metadata": {}, + "id": "890d682f", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "input_nodes = X_train.shape[1]\n", @@ -3249,8 +3560,10 @@ }, { "cell_type": "markdown", - "id": "a7b906df", - "metadata": {}, + "id": "dc4da852", + "metadata": { + "editable": true + }, "source": [ "We then fit our model with our training data using the scheduler of our choice." ] @@ -3258,8 +3571,11 @@ { "cell_type": "code", "execution_count": 32, - "id": "db7a5bf2", - "metadata": {}, + "id": "1e98a3f8", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "linear_regression.reset_weights() # reset weights such that previous runs or reruns don't affect the weights\n", @@ -3270,8 +3586,10 @@ }, { "cell_type": "markdown", - "id": "508c351b", - "metadata": {}, + "id": "68c5d793", + "metadata": { + "editable": true + }, "source": [ "Due to the progress bar we can see the MSE (train_error) throughout\n", "the FFNN's training. Note that the fit() function has some optional\n", @@ -3284,8 +3602,11 @@ { "cell_type": "code", "execution_count": 33, - "id": "6b0562b2", - "metadata": {}, + "id": "e735086e", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "linear_regression.reset_weights() # reset weights such that previous runs or reruns don't affect the weights\n", @@ -3295,8 +3616,10 @@ }, { "cell_type": "markdown", - "id": "f03172d9", - "metadata": {}, + "id": "360468f6", + "metadata": { + "editable": true + }, "source": [ "We see that given more epochs to train on, the regressor reaches a lower MSE.\n", "\n", @@ -3308,8 +3631,11 @@ { "cell_type": "code", "execution_count": 34, - "id": "26d1d1c3", - "metadata": {}, + "id": "c5937c59", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "from sklearn.datasets import load_breast_cancer\n", @@ -3331,8 +3657,11 @@ { "cell_type": "code", "execution_count": 35, - "id": "6a78c633", - "metadata": {}, + "id": "2e06b929", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "input_nodes = X_train.shape[1]\n", @@ -3343,8 +3672,10 @@ }, { "cell_type": "markdown", - "id": "5feaced2", - "metadata": {}, + "id": "94140fb1", + "metadata": { + "editable": true + }, "source": [ "We will now make use of our validation data by passing it into our fit function as a keyword argument" ] @@ -3352,8 +3683,11 @@ { "cell_type": "code", "execution_count": 36, - "id": "6a9b1538", - "metadata": {}, + "id": "da82b266", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "logistic_regression.reset_weights() # reset weights such that previous runs or reruns don't affect the weights\n", @@ -3364,8 +3698,10 @@ }, { "cell_type": "markdown", - "id": "90c7f8f3", - "metadata": {}, + "id": "19e8d4c2", + "metadata": { + "editable": true + }, "source": [ "Finally, we will create a neural network with 2 hidden layers with activation functions." ] @@ -3373,8 +3709,11 @@ { "cell_type": "code", "execution_count": 37, - "id": "58954140", - "metadata": {}, + "id": "4744accd", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "input_nodes = X_train.shape[1]\n", @@ -3390,8 +3729,11 @@ { "cell_type": "code", "execution_count": 38, - "id": "89536ca9", - "metadata": {}, + "id": "f5b5b198", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "neural_network.reset_weights() # reset weights such that previous runs or reruns don't affect the weights\n", @@ -3402,8 +3744,10 @@ }, { "cell_type": "markdown", - "id": "803f4791", - "metadata": {}, + "id": "1f5fc0a0", + "metadata": { + "editable": true + }, "source": [ "### Multiclass classification\n", "\n", @@ -3415,8 +3759,11 @@ { "cell_type": "code", "execution_count": 39, - "id": "50405951", - "metadata": {}, + "id": "4308d82f", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "from sklearn.datasets import load_digits\n", @@ -3449,8 +3796,10 @@ }, { "cell_type": "markdown", - "id": "d906edf4", - "metadata": {}, + "id": "96fc3508", + "metadata": { + "editable": true + }, "source": [ "## Testing the XOR gate and other gates\n", "\n", @@ -3460,8 +3809,11 @@ { "cell_type": "code", "execution_count": 40, - "id": "414ee306", - "metadata": {}, + "id": "5aa9a2a4", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "X = np.array([ [0, 0], [0, 1], [1, 0],[1, 1]],dtype=np.float64)\n", @@ -3480,18 +3832,22 @@ }, { "cell_type": "markdown", - "id": "982d1747", - "metadata": {}, + "id": "d57346ae", + "metadata": { + "editable": true + }, "source": [ "Not bad, but the results depend strongly on the learning reate. Try different learning rates." ] }, { "cell_type": "markdown", - "id": "616eae3e", - "metadata": {}, + "id": "0219b252", + "metadata": { + "editable": true + }, "source": [ - "## Solving ODEs with Deep Learning\n", + "## Solving differential equations with Deep Learning\n", "\n", "The Universal Approximation Theorem states that a neural network can\n", "approximate any function at a single hidden layer along with one input\n", @@ -3513,10 +3869,12 @@ }, { "cell_type": "markdown", - "id": "43fb869c", - "metadata": {}, + "id": "80032cfc", + "metadata": { + "editable": true + }, "source": [ - "## Ordinary Differential Equations\n", + "## Ordinary Differential Equations first\n", "\n", "An ordinary differential equation (ODE) is an equation involving functions having one variable.\n", "\n", @@ -3525,8 +3883,10 @@ }, { "cell_type": "markdown", - "id": "4855aef1", - "metadata": {}, + "id": "3c5546df", + "metadata": { + "editable": true + }, "source": [ "\n", "
\n", @@ -3540,8 +3900,10 @@ }, { "cell_type": "markdown", - "id": "d31c26fc", - "metadata": {}, + "id": "e4f540a5", + "metadata": { + "editable": true + }, "source": [ "where $g(x)$ is the function to find, and $g^{(n)}(x)$ is the $n$-th derivative of $g(x)$.\n", "\n", @@ -3554,8 +3916,10 @@ }, { "cell_type": "markdown", - "id": "08d629ec", - "metadata": {}, + "id": "75ec2cfe", + "metadata": { + "editable": true + }, "source": [ "## The trial solution\n", "\n", @@ -3564,8 +3928,10 @@ }, { "cell_type": "markdown", - "id": "b2cfd7bd", - "metadata": {}, + "id": "a71e7eb8", + "metadata": { + "editable": true + }, "source": [ "\n", "
\n", @@ -3580,8 +3946,10 @@ }, { "cell_type": "markdown", - "id": "f4cd106f", - "metadata": {}, + "id": "9b06d2bf", + "metadata": { + "editable": true + }, "source": [ "where $h_1(x)$ is a function that makes $g_t(x)$ satisfy a given set\n", "of conditions, $N(x,P)$ a neural network with weights and biases\n", @@ -3599,8 +3967,10 @@ }, { "cell_type": "markdown", - "id": "74da655f", - "metadata": {}, + "id": "0aea2f3e", + "metadata": { + "editable": true + }, "source": [ "## Minimization process\n", "\n", @@ -3614,8 +3984,10 @@ }, { "cell_type": "markdown", - "id": "efe0ec18", - "metadata": {}, + "id": "a21c6a43", + "metadata": { + "editable": true + }, "source": [ "$$\n", "C\\left(x, P\\right) = \\big(f\\left(x, \\, g(x), \\, g'(x), \\, g''(x), \\, \\dots \\, , \\, g^{(n)}(x)\\right)\\big)^2\n", @@ -3624,8 +3996,10 @@ }, { "cell_type": "markdown", - "id": "f9c1643a", - "metadata": {}, + "id": "ff939582", + "metadata": { + "editable": true + }, "source": [ "If $N$ inputs are given as a vector $\\boldsymbol{x}$ with elements $x_i$ for $i = 1,\\dots,N$,\n", "the cost function becomes" @@ -3633,8 +4007,10 @@ }, { "cell_type": "markdown", - "id": "9b900e26", - "metadata": {}, + "id": "9f63085a", + "metadata": { + "editable": true + }, "source": [ "\n", "
\n", @@ -3648,8 +4024,10 @@ }, { "cell_type": "markdown", - "id": "82e8e341", - "metadata": {}, + "id": "2875658a", + "metadata": { + "editable": true + }, "source": [ "The neural net should then find the parameters $P$ that minimizes the cost function in\n", "([3](#cost)) for a set of $N$ training samples $x_i$." @@ -3657,8 +4035,10 @@ }, { "cell_type": "markdown", - "id": "b89f2f97", - "metadata": {}, + "id": "8a63e4c6", + "metadata": { + "editable": true + }, "source": [ "## Minimizing the cost function using gradient descent and automatic differentiation\n", "\n", @@ -3671,8 +4051,10 @@ }, { "cell_type": "markdown", - "id": "cc38c37f", - "metadata": {}, + "id": "2d5b4c88", + "metadata": { + "editable": true + }, "source": [ "## Example: Exponential decay\n", "\n", @@ -3681,8 +4063,10 @@ }, { "cell_type": "markdown", - "id": "3c5f0410", - "metadata": {}, + "id": "daab1d5e", + "metadata": { + "editable": true + }, "source": [ "\n", "
\n", @@ -3696,8 +4080,10 @@ }, { "cell_type": "markdown", - "id": "4531d4a4", - "metadata": {}, + "id": "fc2cedc3", + "metadata": { + "editable": true + }, "source": [ "with $g(0) = g_0$ for some chosen initial value $g_0$.\n", "\n", @@ -3706,8 +4092,10 @@ }, { "cell_type": "markdown", - "id": "d580caea", - "metadata": {}, + "id": "da284bc1", + "metadata": { + "editable": true + }, "source": [ "\n", "
\n", @@ -3722,16 +4110,20 @@ }, { "cell_type": "markdown", - "id": "6f384ec5", - "metadata": {}, + "id": "04c2531a", + "metadata": { + "editable": true + }, "source": [ "Having an analytical solution at hand, it is possible to use it to compare how well a neural network finds a solution of ([4](#solve_expdec))." ] }, { "cell_type": "markdown", - "id": "e650af49", - "metadata": {}, + "id": "62a110db", + "metadata": { + "editable": true + }, "source": [ "## The function to solve for\n", "\n", @@ -3740,8 +4132,10 @@ }, { "cell_type": "markdown", - "id": "68e3a73b", - "metadata": {}, + "id": "7b5b722c", + "metadata": { + "editable": true + }, "source": [ "\n", "
\n", @@ -3755,8 +4149,10 @@ }, { "cell_type": "markdown", - "id": "61f7dce9", - "metadata": {}, + "id": "02446b0e", + "metadata": { + "editable": true + }, "source": [ "where $g(0) = g_0$ with $\\gamma$ and $g_0$ being some chosen values.\n", "\n", @@ -3765,8 +4161,10 @@ }, { "cell_type": "markdown", - "id": "47e985ed", - "metadata": {}, + "id": "205c8e89", + "metadata": { + "editable": true + }, "source": [ "## The trial solution\n", "To begin with, a trial solution $g_t(t)$ must be chosen. A general trial solution for ordinary differential equations could be" @@ -3774,8 +4172,10 @@ }, { "cell_type": "markdown", - "id": "b5787755", - "metadata": {}, + "id": "9e46aee1", + "metadata": { + "editable": true + }, "source": [ "$$\n", "g_t(x, P) = h_1(x) + h_2(x, N(x, P))\n", @@ -3784,16 +4184,20 @@ }, { "cell_type": "markdown", - "id": "efd0d663", - "metadata": {}, + "id": "cb0d1f30", + "metadata": { + "editable": true + }, "source": [ "with $h_1(x)$ ensuring that $g_t(x)$ satisfies some conditions and $h_2(x,N(x, P))$ an expression involving $x$ and the output from the neural network $N(x,P)$ with $P $ being the collection of the weights and biases for each layer. For now, it is assumed that the network consists of one input layer, one hidden layer, and one output layer." ] }, { "cell_type": "markdown", - "id": "de4cfa12", - "metadata": {}, + "id": "8485bdd3", + "metadata": { + "editable": true + }, "source": [ "## Setup of Network\n", "\n", @@ -3810,8 +4214,10 @@ }, { "cell_type": "markdown", - "id": "4de8eeb1", - "metadata": {}, + "id": "730e057b", + "metadata": { + "editable": true + }, "source": [ "\n", "
\n", @@ -3825,8 +4231,10 @@ }, { "cell_type": "markdown", - "id": "c3076a5d", - "metadata": {}, + "id": "db8907ef", + "metadata": { + "editable": true + }, "source": [ "## Reformulating the problem\n", "\n", @@ -3842,8 +4250,10 @@ }, { "cell_type": "markdown", - "id": "00459420", - "metadata": {}, + "id": "ce4a7022", + "metadata": { + "editable": true + }, "source": [ "$$\n", "g_t(x, P) = g_0 + x \\cdot N(x, P)\n", @@ -3852,16 +4262,20 @@ }, { "cell_type": "markdown", - "id": "493318e4", - "metadata": {}, + "id": "b17ab00d", + "metadata": { + "editable": true + }, "source": [ "has been chosen such that it already solves the condition $g(0) = g_0$. What remains, is to find $P$ such that" ] }, { "cell_type": "markdown", - "id": "b1cd6594", - "metadata": {}, + "id": "f4af75e0", + "metadata": { + "editable": true + }, "source": [ "\n", "
\n", @@ -3875,16 +4289,20 @@ }, { "cell_type": "markdown", - "id": "bf2aa493", - "metadata": {}, + "id": "4ea72b19", + "metadata": { + "editable": true + }, "source": [ "is fulfilled as *best as possible*." ] }, { "cell_type": "markdown", - "id": "5eda172b", - "metadata": {}, + "id": "428cf327", + "metadata": { + "editable": true + }, "source": [ "## More technicalities\n", "\n", @@ -3897,8 +4315,10 @@ }, { "cell_type": "markdown", - "id": "2bfc8161", - "metadata": {}, + "id": "1b4c9872", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\min_{P}\\Big\\{ \\big(g_t'(x, P) - ( -\\gamma g_t(x, P) \\big)^2 \\Big\\}\n", @@ -3907,8 +4327,10 @@ }, { "cell_type": "markdown", - "id": "9af6a125", - "metadata": {}, + "id": "f12896dd", + "metadata": { + "editable": true + }, "source": [ "(the notation $\\min_{P}\\{ f(x, P) \\}$ means that we desire to find $P$ that yields the minimum of $f(x, P)$)\n", "\n", @@ -3917,8 +4339,10 @@ }, { "cell_type": "markdown", - "id": "bea03f10", - "metadata": {}, + "id": "70e2187e", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\min_{P_{\\text{hidden} }, \\ P_{\\text{output} }}\\Big\\{ \\big(g_t'(x, \\{ P_{\\text{hidden} }, P_{\\text{output} }\\}) - ( -\\gamma g_t(x, \\{ P_{\\text{hidden} }, P_{\\text{output} }\\}) \\big)^2 \\Big\\}\n", @@ -3927,16 +4351,20 @@ }, { "cell_type": "markdown", - "id": "352fb11e", - "metadata": {}, + "id": "e0136d5f", + "metadata": { + "editable": true + }, "source": [ "for an input value $x$." ] }, { "cell_type": "markdown", - "id": "98d5219e", - "metadata": {}, + "id": "ff915c59", + "metadata": { + "editable": true + }, "source": [ "## More details\n", "\n", @@ -3945,8 +4373,10 @@ }, { "cell_type": "markdown", - "id": "83d7b299", - "metadata": {}, + "id": "ca46667e", + "metadata": { + "editable": true + }, "source": [ "\n", "
\n", @@ -3960,16 +4390,20 @@ }, { "cell_type": "markdown", - "id": "1ba53d11", - "metadata": {}, + "id": "3efa1dc6", + "metadata": { + "editable": true + }, "source": [ "Letting $\\boldsymbol{x}$ be a vector with elements $x_i$ and $C(\\boldsymbol{x}, P) = \\frac{1}{N} \\sum_i \\big(g_t'(x_i, P) - ( -\\gamma g_t(x_i, P) \\big)^2$ denote the cost function, the minimization problem that our network must solve, becomes" ] }, { "cell_type": "markdown", - "id": "3953e778", - "metadata": {}, + "id": "a3d800e8", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\min_{P} C(\\boldsymbol{x}, P)\n", @@ -3978,8 +4412,10 @@ }, { "cell_type": "markdown", - "id": "706a1978", - "metadata": {}, + "id": "8fb679f5", + "metadata": { + "editable": true + }, "source": [ "In terms of $P_{\\text{hidden} }$ and $P_{\\text{output} }$, this could also be expressed as\n", "\n", @@ -3990,8 +4426,10 @@ }, { "cell_type": "markdown", - "id": "8ff57d1a", - "metadata": {}, + "id": "08b2a19b", + "metadata": { + "editable": true + }, "source": [ "## A possible implementation of a neural network\n", "\n", @@ -4004,8 +4442,10 @@ }, { "cell_type": "markdown", - "id": "fb487d3d", - "metadata": {}, + "id": "082074c8", + "metadata": { + "editable": true + }, "source": [ "## Technicalities\n", "\n", @@ -4014,8 +4454,10 @@ }, { "cell_type": "markdown", - "id": "1d6ac6a6", - "metadata": {}, + "id": "b69f9148", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{aligned}\n", @@ -4034,8 +4476,10 @@ }, { "cell_type": "markdown", - "id": "fee86c29", - "metadata": {}, + "id": "9feeefde", + "metadata": { + "editable": true + }, "source": [ "## Final technicalities I\n", "\n", @@ -4044,8 +4488,10 @@ }, { "cell_type": "markdown", - "id": "317dd762", - "metadata": {}, + "id": "cde763a7", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{aligned}\n", @@ -4065,8 +4511,10 @@ }, { "cell_type": "markdown", - "id": "4641c716", - "metadata": {}, + "id": "3c83e5dc", + "metadata": { + "editable": true + }, "source": [ "## Final technicalities II\n", "\n", @@ -4079,8 +4527,10 @@ }, { "cell_type": "markdown", - "id": "a28fd5d3", - "metadata": {}, + "id": "4566531c", + "metadata": { + "editable": true + }, "source": [ "$$\n", "f(z) = \\frac{1}{1 + \\exp{(-z)}}\n", @@ -4089,8 +4539,10 @@ }, { "cell_type": "markdown", - "id": "d2c6ce13", - "metadata": {}, + "id": "e316d1e7", + "metadata": { + "editable": true + }, "source": [ "It is possible to use other activations functions for the hidden layer also.\n", "\n", @@ -4111,8 +4563,10 @@ }, { "cell_type": "markdown", - "id": "5bf0304c", - "metadata": {}, + "id": "41ac9a1c", + "metadata": { + "editable": true + }, "source": [ "## Final technicalities III\n", "\n", @@ -4121,8 +4575,10 @@ }, { "cell_type": "markdown", - "id": "f4ee906f", - "metadata": {}, + "id": "b0a8a3c2", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{aligned}\n", @@ -4140,8 +4596,10 @@ }, { "cell_type": "markdown", - "id": "454cd32f", - "metadata": {}, + "id": "67447b23", + "metadata": { + "editable": true + }, "source": [ "## Final technicalities IV\n", "\n", @@ -4150,8 +4608,10 @@ }, { "cell_type": "markdown", - "id": "49b52638", - "metadata": {}, + "id": "12d3e1ee", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\boldsymbol{z}_{1}^{\\text{output}} =\n", @@ -4167,16 +4627,20 @@ }, { "cell_type": "markdown", - "id": "75d1e0dc", - "metadata": {}, + "id": "562a122c", + "metadata": { + "editable": true + }, "source": [ "In this case we seek a continuous range of values since we are approximating a function. This means that after computing $\\boldsymbol{z}_{1}^{\\text{output}}$ the neural network has finished its feed forward step, and $\\boldsymbol{z}_{1}^{\\text{output}}$ is the final output of the network." ] }, { "cell_type": "markdown", - "id": "243508b9", - "metadata": {}, + "id": "78789a96", + "metadata": { + "editable": true + }, "source": [ "## Back propagation\n", "\n", @@ -4187,8 +4651,10 @@ }, { "cell_type": "markdown", - "id": "cfaa5264", - "metadata": {}, + "id": "76b734fb", + "metadata": { + "editable": true + }, "source": [ "$$\n", "C(\\boldsymbol{x}, P) = \\frac{1}{N} \\sum_i \\big(g_t'(x_i, P) - ( -\\gamma g_t(x_i, P) \\big)^2\n", @@ -4197,8 +4663,10 @@ }, { "cell_type": "markdown", - "id": "188a58e4", - "metadata": {}, + "id": "e2911f97", + "metadata": { + "editable": true + }, "source": [ "In order to minimize the cost function, an optimization method must be chosen.\n", "\n", @@ -4207,8 +4675,10 @@ }, { "cell_type": "markdown", - "id": "6f3c7251", - "metadata": {}, + "id": "5e0c281c", + "metadata": { + "editable": true + }, "source": [ "## Gradient descent\n", "\n", @@ -4222,8 +4692,10 @@ }, { "cell_type": "markdown", - "id": "b48662af", - "metadata": {}, + "id": "077e3318", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\boldsymbol{\\omega}_{\\text{new} } = \\boldsymbol{\\omega} - \\lambda \\nabla_{\\boldsymbol{\\omega}} C(\\boldsymbol{x}, \\boldsymbol{\\omega})\n", @@ -4232,8 +4704,10 @@ }, { "cell_type": "markdown", - "id": "46cc1b9b", - "metadata": {}, + "id": "b6e9387d", + "metadata": { + "editable": true + }, "source": [ "for a number of iterations or until $ \\big|\\big| \\boldsymbol{\\omega}_{\\text{new} } - \\boldsymbol{\\omega} \\big|\\big|$ becomes smaller than some given tolerance.\n", "\n", @@ -4252,8 +4726,10 @@ }, { "cell_type": "markdown", - "id": "018877f6", - "metadata": {}, + "id": "3fa21e60", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{aligned}\n", @@ -4265,8 +4741,10 @@ }, { "cell_type": "markdown", - "id": "35e348bd", - "metadata": {}, + "id": "e05e6028", + "metadata": { + "editable": true + }, "source": [ "## The code for solving the ODE" ] @@ -4274,8 +4752,11 @@ { "cell_type": "code", "execution_count": 41, - "id": "dafe8d25", - "metadata": {}, + "id": "eb77d1cf", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "import autograd.numpy as np\n", @@ -4426,8 +4907,10 @@ }, { "cell_type": "markdown", - "id": "65e4f300", - "metadata": {}, + "id": "bbd6562b", + "metadata": { + "editable": true + }, "source": [ "## The network with one input layer, specified number of hidden layers, and one output layer\n", "\n", @@ -4439,8 +4922,11 @@ { "cell_type": "code", "execution_count": 42, - "id": "5ef5f766", - "metadata": {}, + "id": "24ba1afd", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "import autograd.numpy as np\n", @@ -4455,8 +4941,8 @@ "# but with number of hidden layers specified by the user.\n", "def deep_neural_network(deep_params, x):\n", " # N_hidden is the number of hidden layers\n", - "\n", - " N_hidden = np.size(deep_params) - 1 # -1 since params consists of\n", + " # deep_params is a list, len() should be used\n", + " N_hidden = len(deep_params) - 1 # -1 since params consists of\n", " # parameters to all the hidden\n", " # layers AND the output layer.\n", "\n", @@ -4605,8 +5091,10 @@ }, { "cell_type": "markdown", - "id": "c7ad45ef", - "metadata": {}, + "id": "3974040a", + "metadata": { + "editable": true + }, "source": [ "## Example: Population growth\n", "\n", @@ -4616,8 +5104,10 @@ }, { "cell_type": "markdown", - "id": "2d1376bc", - "metadata": {}, + "id": "a9d203ea", + "metadata": { + "editable": true + }, "source": [ "\n", "
\n", @@ -4631,8 +5121,10 @@ }, { "cell_type": "markdown", - "id": "1f039785", - "metadata": {}, + "id": "ede35f92", + "metadata": { + "editable": true + }, "source": [ "where $g(t)$ is the population density at time $t$, $\\alpha > 0$ the growth rate and $A > 0$ is the maximum population number in the environment.\n", "Also, at $t = 0$ the population has the size $g(0) = g_0$, where $g_0$ is some chosen constant.\n", @@ -4645,8 +5137,10 @@ }, { "cell_type": "markdown", - "id": "8dfcfb5e", - "metadata": {}, + "id": "a9720dfc", + "metadata": { + "editable": true + }, "source": [ "## Setting up the problem\n", "\n", @@ -4656,8 +5150,10 @@ }, { "cell_type": "markdown", - "id": "3cb5e674", - "metadata": {}, + "id": "5ef6b555", + "metadata": { + "editable": true + }, "source": [ "\n", "
\n", @@ -4671,8 +5167,10 @@ }, { "cell_type": "markdown", - "id": "56581c9f", - "metadata": {}, + "id": "bd5c646e", + "metadata": { + "editable": true + }, "source": [ "where $g(0) = g_0$.\n", "\n", @@ -4681,8 +5179,10 @@ }, { "cell_type": "markdown", - "id": "ebf7032a", - "metadata": {}, + "id": "cdc5561b", + "metadata": { + "editable": true + }, "source": [ "## The trial solution\n", "\n", @@ -4706,8 +5206,10 @@ }, { "cell_type": "markdown", - "id": "a943c5c6", - "metadata": {}, + "id": "cebde164", + "metadata": { + "editable": true + }, "source": [ "## The program using Autograd\n", "\n", @@ -4717,8 +5219,11 @@ { "cell_type": "code", "execution_count": 43, - "id": "a3620769", - "metadata": {}, + "id": "c71a9592", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "import autograd.numpy as np\n", @@ -4737,9 +5242,12 @@ " g0 = 1.2\n", " return alpha, A, g0\n", "\n", - "def deep_neural_network(P, x):\n", + "def deep_neural_network(deep_params, x):\n", " # N_hidden is the number of hidden layers\n", - " N_hidden = np.size(P) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n", + " # deep_params is a list, len() should be used\n", + " N_hidden = len(deep_params) - 1 # -1 since params consists of\n", + " # parameters to all the hidden\n", + " # layers AND the output layer.\n", "\n", " # Assumes input x being an one-dimensional array\n", " num_values = np.size(x)\n", @@ -4756,7 +5264,7 @@ "\n", " for l in range(N_hidden):\n", " # From the list of parameters P; find the correct weigths and bias for this layer\n", - " w_hidden = P[l]\n", + " w_hidden = deep_params[l]\n", "\n", " # Add a row of ones to include bias\n", " x_prev = np.concatenate((np.ones((1,num_values)), x_prev ), axis = 0)\n", @@ -4770,7 +5278,7 @@ " ## Output layer:\n", "\n", " # Get the weights and bias for this layer\n", - " w_output = P[-1]\n", + " w_output = deep_params[-1]\n", "\n", " # Include bias:\n", " x_prev = np.concatenate((np.ones((1,num_values)), x_prev), axis = 0)\n", @@ -4781,6 +5289,8 @@ " return x_output\n", "\n", "\n", + "\n", + "\n", "def cost_function_deep(P, x):\n", "\n", " # Evaluate the trial function with the current parameters P\n", @@ -4888,8 +5398,10 @@ }, { "cell_type": "markdown", - "id": "9a21afc2", - "metadata": {}, + "id": "bd70d6cc", + "metadata": { + "editable": true + }, "source": [ "## Using forward Euler to solve the ODE\n", "\n", @@ -4906,8 +5418,10 @@ }, { "cell_type": "markdown", - "id": "8a8ad46c", - "metadata": {}, + "id": "ee3bdd02", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{aligned}\n", @@ -4919,8 +5433,10 @@ }, { "cell_type": "markdown", - "id": "7b83336b", - "metadata": {}, + "id": "d81b6054", + "metadata": { + "editable": true + }, "source": [ "along with the condition that $g(0) = g_0$.\n", "\n", @@ -4931,8 +5447,10 @@ }, { "cell_type": "markdown", - "id": "ea68eaac", - "metadata": {}, + "id": "0d7a2272", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{aligned}\n", @@ -4945,16 +5463,20 @@ }, { "cell_type": "markdown", - "id": "c8a2f92c", - "metadata": {}, + "id": "ca29846f", + "metadata": { + "editable": true + }, "source": [ "Now, if $g_i = g(t_i)$ then" ] }, { "cell_type": "markdown", - "id": "4f1a2187", - "metadata": {}, + "id": "1a51d849", + "metadata": { + "editable": true + }, "source": [ "\n", "
\n", @@ -4973,8 +5495,10 @@ }, { "cell_type": "markdown", - "id": "cb87fc93", - "metadata": {}, + "id": "700a9bac", + "metadata": { + "editable": true + }, "source": [ "for $i \\geq 1$ and $g_0 = g(t_0) = g(0) = g_0$.\n", "\n", @@ -4985,8 +5509,11 @@ { "cell_type": "code", "execution_count": 44, - "id": "f8c976e7", - "metadata": {}, + "id": "482cf93c", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "# Assume that all function definitions from the example program using Autograd\n", @@ -5058,8 +5585,10 @@ }, { "cell_type": "markdown", - "id": "23756e15", - "metadata": {}, + "id": "4ed3cefd", + "metadata": { + "editable": true + }, "source": [ "## Example: Solving the one dimensional Poisson equation\n", "\n", @@ -5068,8 +5597,10 @@ }, { "cell_type": "markdown", - "id": "3a68185f", - "metadata": {}, + "id": "d0bdfcdb", + "metadata": { + "editable": true + }, "source": [ "\n", "
\n", @@ -5083,8 +5614,10 @@ }, { "cell_type": "markdown", - "id": "dc784286", - "metadata": {}, + "id": "656240e7", + "metadata": { + "editable": true + }, "source": [ "where $f(x)$ is a given function for $x \\in (0,1)$.\n", "\n", @@ -5093,8 +5626,10 @@ }, { "cell_type": "markdown", - "id": "e0f7d593", - "metadata": {}, + "id": "fb310d7c", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{align*}\n", @@ -5106,8 +5641,10 @@ }, { "cell_type": "markdown", - "id": "4cacfaeb", - "metadata": {}, + "id": "e6b102dc", + "metadata": { + "editable": true + }, "source": [ "This equation can be solved numerically using programs where e.g Autograd and TensorFlow are used.\n", "The results from the networks can then be compared to the analytical solution.\n", @@ -5116,8 +5653,10 @@ }, { "cell_type": "markdown", - "id": "69ba705a", - "metadata": {}, + "id": "797d4bb8", + "metadata": { + "editable": true + }, "source": [ "## The specific equation to solve for\n", "\n", @@ -5126,8 +5665,10 @@ }, { "cell_type": "markdown", - "id": "995cfbc9", - "metadata": {}, + "id": "da2e90b8", + "metadata": { + "editable": true + }, "source": [ "$$\n", "-g''(x) = f(x),\\qquad x \\in (0,1)\n", @@ -5136,16 +5677,20 @@ }, { "cell_type": "markdown", - "id": "769f9670", - "metadata": {}, + "id": "318f8f3a", + "metadata": { + "editable": true + }, "source": [ "where $f(x)$ is a given function, along with the chosen conditions" ] }, { "cell_type": "markdown", - "id": "855cddbb", - "metadata": {}, + "id": "c5e8ac9e", + "metadata": { + "editable": true + }, "source": [ "\n", "
\n", @@ -5159,8 +5704,10 @@ }, { "cell_type": "markdown", - "id": "7006cb54", - "metadata": {}, + "id": "220ceeb4", + "metadata": { + "editable": true + }, "source": [ "In this example, we consider the case when $f(x) = (3x + x^2)\\exp(x)$.\n", "\n", @@ -5169,8 +5716,10 @@ }, { "cell_type": "markdown", - "id": "5216ca22", - "metadata": {}, + "id": "ec8b38dc", + "metadata": { + "editable": true + }, "source": [ "$$\n", "g_t(x) = x \\cdot (1-x) \\cdot N(P,x)\n", @@ -5179,16 +5728,20 @@ }, { "cell_type": "markdown", - "id": "b77670be", - "metadata": {}, + "id": "0ed927a9", + "metadata": { + "editable": true + }, "source": [ "The analytical solution for this problem is" ] }, { "cell_type": "markdown", - "id": "bf2bed23", - "metadata": {}, + "id": "3a51ddcc", + "metadata": { + "editable": true + }, "source": [ "$$\n", "g(x) = x(1 - x)\\exp(x)\n", @@ -5197,8 +5750,10 @@ }, { "cell_type": "markdown", - "id": "a22ccb78", - "metadata": {}, + "id": "d5392b8d", + "metadata": { + "editable": true + }, "source": [ "## Solving the equation using Autograd" ] @@ -5206,8 +5761,11 @@ { "cell_type": "code", "execution_count": 45, - "id": "7b3b5f5a", - "metadata": {}, + "id": "95059f71", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "import autograd.numpy as np\n", @@ -5220,7 +5778,10 @@ "\n", "def deep_neural_network(deep_params, x):\n", " # N_hidden is the number of hidden layers\n", - " N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n", + " # deep_params is a list, len() should be used\n", + " N_hidden = len(deep_params) - 1 # -1 since params consists of\n", + " # parameters to all the hidden\n", + " # layers AND the output layer.\n", "\n", " # Assumes input x being an one-dimensional array\n", " num_values = np.size(x)\n", @@ -5261,6 +5822,7 @@ "\n", " return x_output\n", "\n", + "\n", "def solve_ode_deep_neural_network(x, num_neurons, num_iter, lmb):\n", " # num_hidden_neurons is now a list of number of neurons within each hidden layer\n", "\n", @@ -5364,8 +5926,10 @@ }, { "cell_type": "markdown", - "id": "6668b7ef", - "metadata": {}, + "id": "7c568bf3", + "metadata": { + "editable": true + }, "source": [ "## Comparing with a numerical scheme\n", "\n", @@ -5384,8 +5948,10 @@ }, { "cell_type": "markdown", - "id": "ec1adda2", - "metadata": {}, + "id": "eab0bb68", + "metadata": { + "editable": true + }, "source": [ "\n", "
\n", @@ -5399,16 +5965,20 @@ }, { "cell_type": "markdown", - "id": "33486af4", - "metadata": {}, + "id": "c0ada0c0", + "metadata": { + "editable": true + }, "source": [ "If $x_i = i \\Delta x = x_{i-1} + \\Delta x$ and $g_i = g(x_i)$ for $i = 1,\\dots N_x - 2$ with $N_x$ being the number of values for $x$, ([15](#approx)) becomes" ] }, { "cell_type": "markdown", - "id": "42c3ef84", - "metadata": {}, + "id": "b483eace", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{aligned}\n", @@ -5420,16 +5990,20 @@ }, { "cell_type": "markdown", - "id": "f454dbea", - "metadata": {}, + "id": "30d45122", + "metadata": { + "editable": true + }, "source": [ "Since we know from our problem that" ] }, { "cell_type": "markdown", - "id": "7411a707", - "metadata": {}, + "id": "52c68e05", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{aligned}\n", @@ -5441,8 +6015,10 @@ }, { "cell_type": "markdown", - "id": "c47c3822", - "metadata": {}, + "id": "1657934d", + "metadata": { + "editable": true + }, "source": [ "along with the conditions $g(0) = g(1) = 0$,\n", "the following scheme can be used to find an approximate solution for $g(x)$ numerically:" @@ -5450,8 +6026,10 @@ }, { "cell_type": "markdown", - "id": "1cca824c", - "metadata": {}, + "id": "9fc46a1b", + "metadata": { + "editable": true + }, "source": [ "\n", "
\n", @@ -5468,8 +6046,10 @@ }, { "cell_type": "markdown", - "id": "82ecd5a1", - "metadata": {}, + "id": "cd79f75c", + "metadata": { + "editable": true + }, "source": [ "for $i = 1, \\dots, N_x - 2$ where $g_0 = g_{N_x - 1} = 0$ and $f(x_i) = (3x_i + x_i^2)\\exp(x_i)$, which is given for our specific problem.\n", "\n", @@ -5478,8 +6058,10 @@ }, { "cell_type": "markdown", - "id": "39f5fe07", - "metadata": {}, + "id": "501ac880", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{aligned}\n", @@ -5513,16 +6095,20 @@ }, { "cell_type": "markdown", - "id": "5ac54c35", - "metadata": {}, + "id": "8551615b", + "metadata": { + "editable": true + }, "source": [ "which makes it possible to solve for the vector $\\boldsymbol{g}$." ] }, { "cell_type": "markdown", - "id": "79ad3ebc", - "metadata": {}, + "id": "dbd6d2a1", + "metadata": { + "editable": true + }, "source": [ "## Setting up the code\n", "\n", @@ -5532,8 +6118,11 @@ { "cell_type": "code", "execution_count": 46, - "id": "be03bf2d", - "metadata": {}, + "id": "c71a3063", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "import autograd.numpy as np\n", @@ -5546,7 +6135,10 @@ "\n", "def deep_neural_network(deep_params, x):\n", " # N_hidden is the number of hidden layers\n", - " N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n", + " # deep_params is a list, len() should be used\n", + " N_hidden = len(deep_params) - 1 # -1 since params consists of\n", + " # parameters to all the hidden\n", + " # layers AND the output layer.\n", "\n", " # Assumes input x being an one-dimensional array\n", " num_values = np.size(x)\n", @@ -5587,6 +6179,7 @@ "\n", " return x_output\n", "\n", + "\n", "def solve_ode_deep_neural_network(x, num_neurons, num_iter, lmb):\n", " # num_hidden_neurons is now a list of number of neurons within each hidden layer\n", "\n", @@ -5730,8 +6323,10 @@ }, { "cell_type": "markdown", - "id": "faa0daa4", - "metadata": {}, + "id": "da58cdec", + "metadata": { + "editable": true + }, "source": [ "## Partial Differential Equations\n", "\n", @@ -5745,8 +6340,10 @@ }, { "cell_type": "markdown", - "id": "fb7b7ff6", - "metadata": {}, + "id": "df991968", + "metadata": { + "editable": true + }, "source": [ "\n", "
\n", @@ -5760,16 +6357,20 @@ }, { "cell_type": "markdown", - "id": "71da1640", - "metadata": {}, + "id": "59ef3099", + "metadata": { + "editable": true + }, "source": [ "where $f$ is an expression involving all kinds of possible mixed derivatives of $g(x_1,\\dots,x_N)$ up to an order $n$. In order for the solution to be unique, some additional conditions must also be given." ] }, { "cell_type": "markdown", - "id": "c77e6225", - "metadata": {}, + "id": "a580ae0f", + "metadata": { + "editable": true + }, "source": [ "## Type of problem\n", "\n", @@ -5781,8 +6382,10 @@ }, { "cell_type": "markdown", - "id": "98f82ea2", - "metadata": {}, + "id": "a56f3fe1", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{align*}\n", @@ -5793,8 +6396,10 @@ }, { "cell_type": "markdown", - "id": "bc33501c", - "metadata": {}, + "id": "69586714", + "metadata": { + "editable": true + }, "source": [ "where $h_1(x_1,\\dots,x_N)$ is a function that ensures $g_t(x_1,\\dots,x_N)$ satisfies some given conditions.\n", "The neural network $N(x_1,\\dots,x_N,P)$ has weights and biases described by $P$ and $h_2(x_1,\\dots,x_N,N(x_1,\\dots,x_N,P))$ is an expression using the output from the neural network in some way.\n", @@ -5804,8 +6409,10 @@ }, { "cell_type": "markdown", - "id": "818e8900", - "metadata": {}, + "id": "d8374dce", + "metadata": { + "editable": true + }, "source": [ "## Network requirements\n", "\n", @@ -5822,8 +6429,10 @@ }, { "cell_type": "markdown", - "id": "526d5428", - "metadata": {}, + "id": "f3fe961e", + "metadata": { + "editable": true + }, "source": [ "$$\n", "C\\left(x_1, \\dots, x_N, P\\right) = \\left( f\\left(x_1, \\, \\dots \\, , x_N, \\frac{\\partial g(x_1,\\dots,x_N) }{\\partial x_1}, \\dots , \\frac{\\partial g(x_1,\\dots,x_N) }{\\partial x_N}, \\frac{\\partial g(x_1,\\dots,x_N) }{\\partial x_1\\partial x_2}, \\, \\dots \\, , \\frac{\\partial^n g(x_1,\\dots,x_N) }{\\partial x_N^n} \\right) \\right)^2\n", @@ -5832,8 +6441,10 @@ }, { "cell_type": "markdown", - "id": "279b9296", - "metadata": {}, + "id": "4e453aa1", + "metadata": { + "editable": true + }, "source": [ "## More details\n", "\n", @@ -5842,8 +6453,10 @@ }, { "cell_type": "markdown", - "id": "bcb8ce40", - "metadata": {}, + "id": "e1ebf5bd", + "metadata": { + "editable": true + }, "source": [ "$$\n", "C\\left(\\boldsymbol{x}, P\\right) = f\\left( \\left( \\boldsymbol{x}, \\frac{\\partial g(\\boldsymbol{x}) }{\\partial x_1}, \\dots , \\frac{\\partial g(\\boldsymbol{x}) }{\\partial x_N}, \\frac{\\partial g(\\boldsymbol{x}) }{\\partial x_1\\partial x_2}, \\, \\dots \\, , \\frac{\\partial^n g(\\boldsymbol{x}) }{\\partial x_N^n} \\right) \\right)^2\n", @@ -5852,16 +6465,20 @@ }, { "cell_type": "markdown", - "id": "6549ea7b", - "metadata": {}, + "id": "b8c35553", + "metadata": { + "editable": true + }, "source": [ "If we also have $M$ different sets of values for $x_1, \\dots, x_N$, that is $\\boldsymbol{x}_i = \\big(x_1^{(i)}, \\dots, x_N^{(i)}\\big)$ for $i = 1,\\dots,M$ being the rows in matrix $X$, the cost function can be generalized into" ] }, { "cell_type": "markdown", - "id": "0956062e", - "metadata": {}, + "id": "22a123d5", + "metadata": { + "editable": true + }, "source": [ "$$\n", "C\\left(X, P \\right) = \\sum_{i=1}^M f\\left( \\left( \\boldsymbol{x}_i, \\frac{\\partial g(\\boldsymbol{x}_i) }{\\partial x_1}, \\dots , \\frac{\\partial g(\\boldsymbol{x}_i) }{\\partial x_N}, \\frac{\\partial g(\\boldsymbol{x}_i) }{\\partial x_1\\partial x_2}, \\, \\dots \\, , \\frac{\\partial^n g(\\boldsymbol{x}_i) }{\\partial x_N^n} \\right) \\right)^2.\n", @@ -5870,8 +6487,10 @@ }, { "cell_type": "markdown", - "id": "8fca0166", - "metadata": {}, + "id": "b15eedf5", + "metadata": { + "editable": true + }, "source": [ "## Example: The diffusion equation\n", "\n", @@ -5880,8 +6499,10 @@ }, { "cell_type": "markdown", - "id": "58ad6f25", - "metadata": {}, + "id": "1a92cc8c", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\frac{\\partial g(x,t)}{\\partial t} = \\frac{\\partial^2 g(x,t)}{\\partial x^2}\n", @@ -5890,16 +6511,20 @@ }, { "cell_type": "markdown", - "id": "e6e5728e", - "metadata": {}, + "id": "e4862313", + "metadata": { + "editable": true + }, "source": [ "where a possible choice of conditions are" ] }, { "cell_type": "markdown", - "id": "6921faa1", - "metadata": {}, + "id": "5c186d68", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{align*}\n", @@ -5912,16 +6537,20 @@ }, { "cell_type": "markdown", - "id": "80e632ef", - "metadata": {}, + "id": "57d4f000", + "metadata": { + "editable": true + }, "source": [ "with $u(x)$ being some given function." ] }, { "cell_type": "markdown", - "id": "a2711421", - "metadata": {}, + "id": "7c2cdc6b", + "metadata": { + "editable": true + }, "source": [ "## Defining the problem\n", "\n", @@ -5930,8 +6559,10 @@ }, { "cell_type": "markdown", - "id": "981da4af", - "metadata": {}, + "id": "bde064ea", + "metadata": { + "editable": true + }, "source": [ "\n", "
\n", @@ -5945,16 +6576,20 @@ }, { "cell_type": "markdown", - "id": "c1b8c16d", - "metadata": {}, + "id": "7a558c97", + "metadata": { + "editable": true + }, "source": [ "and" ] }, { "cell_type": "markdown", - "id": "b8ed6509", - "metadata": {}, + "id": "534a4844", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{align*}\n", @@ -5967,8 +6602,10 @@ }, { "cell_type": "markdown", - "id": "200050ac", - "metadata": {}, + "id": "a07e0730", + "metadata": { + "editable": true + }, "source": [ "with $u(x) = \\sin(\\pi x)$.\n", "\n", @@ -5979,8 +6616,10 @@ }, { "cell_type": "markdown", - "id": "5dca99d2", - "metadata": {}, + "id": "74b72286", + "metadata": { + "editable": true + }, "source": [ "## Setting up the network using Autograd\n", "\n", @@ -5996,8 +6635,11 @@ { "cell_type": "code", "execution_count": 47, - "id": "85f4b843", - "metadata": {}, + "id": "4a0c57e9", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "def sigmoid(z):\n", @@ -6011,7 +6653,7 @@ " num_points = np.size(x,1)\n", "\n", " # N_hidden is the number of hidden layers\n", - " N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n", + " N_hidden = len(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n", "\n", " # Assume that the input layer does nothing to the input x\n", " x_input = x\n", @@ -6048,8 +6690,10 @@ }, { "cell_type": "markdown", - "id": "9dd68ff8", - "metadata": {}, + "id": "350b1d3d", + "metadata": { + "editable": true + }, "source": [ "## Setting up the network using Autograd; The trial solution\n", "\n", @@ -6076,8 +6720,10 @@ }, { "cell_type": "markdown", - "id": "65f68c5a", - "metadata": {}, + "id": "9fd438b5", + "metadata": { + "editable": true + }, "source": [ "## Why the jacobian?\n", "\n", @@ -6103,8 +6749,11 @@ { "cell_type": "code", "execution_count": 48, - "id": "91612f4f", - "metadata": {}, + "id": "d1c81690", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "# Set up the trial function:\n", @@ -6147,8 +6796,10 @@ }, { "cell_type": "markdown", - "id": "ff9df4ba", - "metadata": {}, + "id": "ca22d9ce", + "metadata": { + "editable": true + }, "source": [ "## Setting up the network using Autograd; The full program\n", "\n", @@ -6171,8 +6822,11 @@ { "cell_type": "code", "execution_count": 49, - "id": "e6451b9e", - "metadata": {}, + "id": "5a9146bc", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "import autograd.numpy as np\n", @@ -6195,7 +6849,7 @@ " num_points = np.size(x,1)\n", "\n", " # N_hidden is the number of hidden layers\n", - " N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n", + " N_hidden = len(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n", "\n", " # Assume that the input layer does nothing to the input x\n", " x_input = x\n", @@ -6338,7 +6992,7 @@ " T,X = np.meshgrid(t,x)\n", "\n", " fig = plt.figure(figsize=(10,10))\n", - " ax = fig.gca(projection='3d')\n", + " ax = fig.add_suplot(projection='3d')\n", " ax.set_title('Solution from the deep neural network w/ %d layer'%len(num_hidden_neurons))\n", " s = ax.plot_surface(T,X,g_dnn_ag,linewidth=0,antialiased=False,cmap=cm.viridis)\n", " ax.set_xlabel('Time $t$')\n", @@ -6346,14 +7000,14 @@ "\n", "\n", " fig = plt.figure(figsize=(10,10))\n", - " ax = fig.gca(projection='3d')\n", + " ax = fig.add_suplot(projection='3d')\n", " ax.set_title('Analytical solution')\n", " s = ax.plot_surface(T,X,G_analytical,linewidth=0,antialiased=False,cmap=cm.viridis)\n", " ax.set_xlabel('Time $t$')\n", " ax.set_ylabel('Position $x$');\n", "\n", " fig = plt.figure(figsize=(10,10))\n", - " ax = fig.gca(projection='3d')\n", + " ax = fig.add_suplot(projection='3d')\n", " ax.set_title('Difference')\n", " s = ax.plot_surface(T,X,diff_ag,linewidth=0,antialiased=False,cmap=cm.viridis)\n", " ax.set_xlabel('Time $t$')\n", @@ -6402,8 +7056,10 @@ }, { "cell_type": "markdown", - "id": "7f867cf2", - "metadata": {}, + "id": "e59d2116", + "metadata": { + "editable": true + }, "source": [ "## Example: Solving the wave equation with Neural Networks\n", "\n", @@ -6412,8 +7068,10 @@ }, { "cell_type": "markdown", - "id": "ded83be1", - "metadata": {}, + "id": "e2ffdbaf", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\frac{\\partial^2 g(x,t)}{\\partial t^2} = c^2\\frac{\\partial^2 g(x,t)}{\\partial x^2}\n", @@ -6422,8 +7080,10 @@ }, { "cell_type": "markdown", - "id": "f0aba0c9", - "metadata": {}, + "id": "37b2cc03", + "metadata": { + "editable": true + }, "source": [ "with $c$ being the specified wave speed.\n", "\n", @@ -6432,8 +7092,10 @@ }, { "cell_type": "markdown", - "id": "28a48fe7", - "metadata": {}, + "id": "524df5d3", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{align*}\n", @@ -6447,16 +7109,20 @@ }, { "cell_type": "markdown", - "id": "61ded78d", - "metadata": {}, + "id": "d175b25f", + "metadata": { + "editable": true + }, "source": [ "where $\\frac{\\partial g(x,t)}{\\partial t} \\Big |_{t = 0}$ means the derivative of $g(x,t)$ with respect to $t$ is evaluated at $t = 0$, and $u(x)$ and $v(x)$ being given functions." ] }, { "cell_type": "markdown", - "id": "990695ea", - "metadata": {}, + "id": "11b3289e", + "metadata": { + "editable": true + }, "source": [ "## The problem to solve for\n", "\n", @@ -6465,8 +7131,10 @@ }, { "cell_type": "markdown", - "id": "58841089", - "metadata": {}, + "id": "e469d65e", + "metadata": { + "editable": true + }, "source": [ "\n", "
\n", @@ -6480,8 +7148,10 @@ }, { "cell_type": "markdown", - "id": "303eb900", - "metadata": {}, + "id": "5f773fa8", + "metadata": { + "editable": true + }, "source": [ "where $c$ is the given wave speed.\n", "The chosen conditions for this equation are" @@ -6489,8 +7159,10 @@ }, { "cell_type": "markdown", - "id": "ce5c5167", - "metadata": {}, + "id": "2952f7b4", + "metadata": { + "editable": true + }, "source": [ "\n", "
\n", @@ -6507,16 +7179,20 @@ }, { "cell_type": "markdown", - "id": "609a7556", - "metadata": {}, + "id": "99057b48", + "metadata": { + "editable": true + }, "source": [ "In this example, let $c = 1$ and $u(x) = \\sin(\\pi x)$ and $v(x) = -\\pi\\sin(\\pi x)$." ] }, { "cell_type": "markdown", - "id": "5edb8797", - "metadata": {}, + "id": "7debdfea", + "metadata": { + "editable": true + }, "source": [ "## The trial solution\n", "Setting up the network is done in similar matter as for the example of solving the diffusion equation.\n", @@ -6539,8 +7215,10 @@ }, { "cell_type": "markdown", - "id": "c6be7518", - "metadata": {}, + "id": "2d5d5c44", + "metadata": { + "editable": true + }, "source": [ "## The analytical solution\n", "\n", @@ -6553,8 +7231,10 @@ }, { "cell_type": "markdown", - "id": "368bb540", - "metadata": {}, + "id": "f97806ce", + "metadata": { + "editable": true + }, "source": [ "## Solving the wave equation - the full program using Autograd" ] @@ -6562,8 +7242,11 @@ { "cell_type": "code", "execution_count": 50, - "id": "65085f11", - "metadata": {}, + "id": "9337b7c7", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "import autograd.numpy as np\n", @@ -6620,7 +7303,7 @@ " num_points = np.size(x,1)\n", "\n", " # N_hidden is the number of hidden layers\n", - " N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n", + " N_hidden = len(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n", "\n", " # Assume that the input layer does nothing to the input x\n", " x_input = x\n", @@ -6725,7 +7408,7 @@ " T,X = np.meshgrid(t,x)\n", "\n", " fig = plt.figure(figsize=(10,10))\n", - " ax = fig.gca(projection='3d')\n", + " ax = fig.add_suplot(projection='3d')\n", " ax.set_title('Solution from the deep neural network w/ %d layer'%len(num_hidden_neurons))\n", " s = ax.plot_surface(T,X,res,linewidth=0,antialiased=False,cmap=cm.viridis)\n", " ax.set_xlabel('Time $t$')\n", @@ -6733,7 +7416,7 @@ "\n", "\n", " fig = plt.figure(figsize=(10,10))\n", - " ax = fig.gca(projection='3d')\n", + " ax = fig.add_suplot(projection='3d')\n", " ax.set_title('Analytical solution')\n", " s = ax.plot_surface(T,X,res_analytical,linewidth=0,antialiased=False,cmap=cm.viridis)\n", " ax.set_xlabel('Time $t$')\n", @@ -6741,7 +7424,7 @@ "\n", "\n", " fig = plt.figure(figsize=(10,10))\n", - " ax = fig.gca(projection='3d')\n", + " ax = fig.add_suplot(projection='3d')\n", " ax.set_title('Difference')\n", " s = ax.plot_surface(T,X,diff,linewidth=0,antialiased=False,cmap=cm.viridis)\n", " ax.set_xlabel('Time $t$')\n", @@ -6790,8 +7473,10 @@ }, { "cell_type": "markdown", - "id": "41011c90", - "metadata": {}, + "id": "6071d006", + "metadata": { + "editable": true + }, "source": [ "## Resources on differential equations and deep learning\n", "\n", @@ -6805,25 +7490,7 @@ ] } ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.9.15" - } - }, + "metadata": {}, "nbformat": 4, "nbformat_minor": 5 } diff --git a/doc/LectureNotes/_build/html/searchindex.js b/doc/LectureNotes/_build/html/searchindex.js index d29a47560..bb364a7d0 100644 --- a/doc/LectureNotes/_build/html/searchindex.js +++ b/doc/LectureNotes/_build/html/searchindex.js @@ -1 +1 @@ -Search.setIndex({docnames:["additionweek42","chapter1","chapter10","chapter11","chapter12","chapter13","chapter2","chapter3","chapter4","chapter5","chapter6","chapter7","chapter8","chapter9","chapteroptimization","clustering","exercisesweek34","exercisesweek35","exercisesweek36","exercisesweek37","exercisesweek38","exercisesweek39","exercisesweek41","exercisesweek42","intro","linalg","project1","project2","schedule","statistics","teachers","textbooks","week34","week35","week36","week37","week38","week39","week40","week41","week42","week43"],envversion:{"sphinx.domains.c":2,"sphinx.domains.changeset":1,"sphinx.domains.citation":1,"sphinx.domains.cpp":4,"sphinx.domains.index":1,"sphinx.domains.javascript":2,"sphinx.domains.math":2,"sphinx.domains.python":3,"sphinx.domains.rst":2,"sphinx.domains.std":2,"sphinx.ext.intersphinx":1,sphinx:56},filenames:["additionweek42.ipynb","chapter1.ipynb","chapter10.ipynb","chapter11.ipynb","chapter12.ipynb","chapter13.ipynb","chapter2.ipynb","chapter3.ipynb","chapter4.ipynb","chapter5.ipynb","chapter6.ipynb","chapter7.ipynb","chapter8.ipynb","chapter9.ipynb","chapteroptimization.ipynb","clustering.ipynb","exercisesweek34.ipynb","exercisesweek35.ipynb","exercisesweek36.ipynb","exercisesweek37.ipynb","exercisesweek38.ipynb","exercisesweek39.ipynb","exercisesweek41.ipynb","exercisesweek42.ipynb","intro.md","linalg.ipynb","project1.ipynb","project2.ipynb","schedule.md","statistics.ipynb","teachers.md","textbooks.md","week34.ipynb","week35.ipynb","week36.ipynb","week37.ipynb","week38.ipynb","week39.ipynb","week40.ipynb","week41.ipynb","week42.ipynb","week43.ipynb"],objects:{},objnames:{},objtypes:{},terms:{"0":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,22,23,25,26,27,29,30,32,33,34,35,36,37,38,39,41],"00":[1,2,6,7,10,12,32,33,34,38,39,40,41],"000":[2,4,10,40,41],"0000":10,"00000":[10,37],"000000":[6,12,33],"00000000e":[6,33,38,39],"00000262":38,"00001":37,"00001365":[],"00001572":[],"00001716":[],"00001976":38,"00002":37,"00002192":38,"00002276":[],"00002585e":[],"00002833":[],"0000293":[],"00003":37,"00003022":22,"00003645":[],"00003828":[],"00004164":38,"00005":37,"00005688":[],"00005799":[],"00006114":[],"00007":37,"00007706":[],"00007738":[],"00007964":[],"00008335":[],"00009":37,"00009004":[],"0001":[2,18,40,41],"00010067":37,"00010366":[],"0001079":[],"00011562":[],"00011577":[],"00012":37,"00012565":[],"00013464":[],"000137":[],"00013938":[39,40],"00013955":[],"00014166":[],"00014631":[],"00015394":[],"00015678":[],"00016062":[],"00016449":[],"00016535":[],"00016719":37,"00017038":[],"00017733":[],"0001775":[],"00018":37,"00018053":[],"00018212":[],"00018256":[],"00018651":[],"00018876":[],"00019258":[],"0001984":[],"00019998":6,"0002065":[],"00021147":[],"00021259":[],"00021270681344726395":[],"00021771":[39,40],"00021916":[],"000225":37,"00022969":[],"00023":37,"00023102":[],"00023414":[],"00023813":[],"00023978":[],"00024087":6,"00024415":22,"00024755":[],"00024929":37,"00025662":37,"00026549":[],"00026784":[],"000269":[],"00027188":[],"0002762":37,"00027746":37,"00028":37,"00028167":37,"00028359":38,"00028552":[],"00028632":38,"00028825":[],"00029012":6,"0002926":[],"00030094":[],"00030102":[],"00030132":[],"00030601":37,"00030827":[],"00031208":37,"00031272":38,"00031321":[],"00031535148309577417":[7,34],"0003153514830958095":[7,34],"00032566":38,"00033415":[],"00033562":[],"00033679":[],"00033904":37,"00034006":[39,40],"00034132":[],"00034209":[],"00034576":37,"00034688149480437765":[],"00034944":6,"00034953":[],"00035214":[],"00035483":[],"00035708":[],"00035912":38,"00036464":[],"00036566":[],"00036797":[],"0003701342170783489":[],"00037398":38,"00037564":37,"00037737":[],"00038309":37,"000387":[],"00039072":[],"0003919793592273513":[],"00039632":[],"00039821":[],"00040218":[],"00040516":[],"00041241":38,"00041619":37,"000417932":[],"00042072":[],"00042089":6,"00042093":37,"00042436":[],"00042444":37,"00042721":[],"00042767357088773335":10,"00042947":38,"00043701":[],"0004377":22,"0004388":[],"0004394":[],"00044":37,"00044937":[],"00045472":[],"00045923":[],"00046006":37,"00046111":37,"000464088":[],"00046616":[],"00046631":[],"00047025":37,"0004736":38,"00048158":[],"00048407":[],"00048474":[],"00048499":[],"0004932":38,"00049344":[],"00049636":[],"00049754":[],"00050627":[],"00050694":6,"00050952":[],"00051088":37,"00051916":[],"00052051":[],"00052102":37,"00052874":[],"00053001":[],"00053118":[39,40],"00053122":[],"00053635":[],"00053775":22,"00053797":[],"00053824":[],"00054387":38,"00055112":[],"00055695":[],"00056337":[],"00056414":[],"00056603":37,"00056637":38,"00057256":[],"00057362":[],"00057726":37,"00057772":[],"00058016":[7,34],"00059105":[],"00059247":22,"00059681":33,"00059703":[],"00059734":[],"00059993":[],"00060":37,"00060704":7,"00061058":6,"00061352":[],"00061711":[],"00061724":[],"00062457":38,"00062592":7,"00062668":[],"000626681519836":[],"00062713":37,"00063362":35,"00063866":35,"00063957":37,"00063965":[],"00064081":[],"00065041":38,"00065276":22,"00065307":[],"00065505":[],"00066091":[],"00066259":[],"00066293":[],"00066395":[],"00066668":[7,34],"00066798":[],"00067008":37,"00067192":[],"00067567":33,"00067793":22,"00068091":35,"00068734":[7,34],"00069":37,"00069123":7,"00069482":37,"00070819":[],"0007086":37,"00071003":[],"00071092":[],"00071724":38,"00071918":22,"00072527":[],"00072563":38,"00073144":[],"00073534":[],"00073541":6,"00073571":[],"0007373":[],"00074274":[],"00074692":[22,38],"00074747":[],"00075296":[],"00075590":35,"00076202":37,"00076495":[7,34],"00076521":[],"00076908":7,"00076983":37,"0007698473260556325":[7,34],"0007698473260556343":[7,34],"00077369":[],"00077389":[],"000783":[],"00078324":[],"00078509":37,"00079125":7,"00079237":22,"00079623":[],"00079904":35,"00079968":6,"00080345":33,"00080497":[],"00080683":[],"00081607":[],"00081649":[],"00081693":[],"00082235":[],"00082292":22,"00082366":38,"00082916":[],"0008297":[39,40],"0008297120772365539":[],"00083355":35,"0008415":[],"00084215":[],"00084244":[],"00084707":7,"00084831":[],"00085292":37,"00085774":38,"00085892":7,"00086675":[],"00086984":37,"00087102":[],"000873":22,"00087697":[7,34],"00088573":6,"00088848":[],"00089171":[],"00089193":35,"00090567":[],"00090614":[],"00090666":22,"00091307":[],"00091707":[],"00092668":[],"00092678":7,"00092898":35,"00093243":[],"00093356":[],"00093864":[],"00093994":[],"00094163":[],"00094353":[],"00094499":37,"00094587":38,"00094737":[],"0009532583857747255":[],"00095488":[],"00095624":[],"00096183":22,"00096314":6,"00096373":37,"00096829":[],"00097416":[],"00097604":[],"00097839":[],"0009806":[],"00098376":[],"00098501":38,"00099778":[],"0009983":[],"00099892":22,"00099936":[],"001":[2,3,9,14,18,23,34,36,37,40,41],"00100081":[],"00100202":[],"00100423":[],"00100511":[],"00100519":[7,34],"00100546":[],"00100563":[],"00100913":[],"00101555":[],"00102711":[],"00103024":[],"00103819":[],"00103867":[],"001047":37,"0010479245693919886":[7,35],"00105081":[7,34],"00105724":[],"0010597":22,"00106654":[],"00106677":6,"00106776":37,"00107405":[7,34],"00107739":[],"00108":37,"00108146":[],"0010827":[],"00108471":[],"00108622":38,"00108673":[],"00109354":[],"00110057":22,"00110396":[],"00110643":[],"001106660107757":[],"00110721":[],"00110867":[],"00111547":[],"00111564":[],"00111605":[],"00111756":[7,34],"00111809":[],"001123":[],"00113116":38,"00114401":[],"00114451":[],"00114855":[],"0011526":[7,34],"00115455":[],"00115508":[],"00115631":37,"00115999":6,"00116001":37,"00116737":[],"00116754":22,"00118299":[],"00118302":37,"00118584":7,"00118642":[],"00119688":35,"00119865":[],"00119876":[],"00120071":[],"00121256":22,"00121712":[],"00121925":[],"00122762":[],"00123795":[],"00123858":[],"0012394":[],"00124283":[],"00124429":[],"00124738":38,"00125057":[],"00125802":[],"00126053":37,"001268544905013411":[],"00127606":[],"00127645":[],"00128394":[],"00128453":[],"00128479":6,"00128523":37,"00128634":22,"00128778":[],"00129209":[],"00129599":[39,40],"00129899":38,"0013052":[],"00130648":[],"00130885":[],"00131072":37,"001323":[7,34],"00132494":[],"00133595":22,"00133638":[],"00134264":[],"00134424":[],"00135176":[],"00135237":[],"00135922":[],"00136125":[],"00137387":[],"00137458":[],"00137814":7,"0013839":[],"00138835":[],"00139195":[],"00139705":6,"00139918":[],"00140184":[],"00140253":[],"0014028":[],"00140846":35,"00141724":22,"00141727":37,"00142146":[],"00142396":37,"00142784":[],"00142849":[],"00142986":[],"00143013":[],"00143164":[],"00143246":38,"00143869":[],"0014423":[],"00144539":37,"00144622":[],"0014522":37,"0014593":[],"001468":[],"00146807":[],"0014719":22,"0014785":[],"00148216":[],"00148663":[],"00149173":38,"00149311":[7,34],"00149679":[],"00149956":[7,34],"00151204":[],"00151416":[],"00151426":[],"00152117":[7,34],"00152182":[],"00152471":[],"0015255":[],"00152602":[],"00154115":[],"00154733":6,"00154737":[],"00154853":35,"0015552":[],"00155695":[],"00156145":22,"00156374":7,"00157005":[],"00157346":33,"0015752":[],"00157766":37,"00157826":[],"00158":37,"00158427":[],"00158786":[],"00158859":[],"0016017":[],"00160895":37,"00161463":[],"0016202":33,"00162167":22,"00162812":[],"00162921":[],"00163159":[],"00163548":[],"00164501":38,"0016472":[],"00166119":[],"00166889":[],"00167058":[],"00167329":37,"00167406":22,"00168251":6,"00168427":38,"00168745":[],"00169":37,"00169056":[],"00169212":[],"00169299":[],"00170579":[],"00170809":[],"00170913":[],"00171047":[],"00171171":[],"00171307":38,"00171326":[],"00172034":22,"00172865":[],"00173182":[],"001732":[],"0017421":[],"00174276":[7,34],"00174459":[],"00174738":[],"00174743":[],"00174795":37,"00174884":[],"00175331":[7,34],"00176582":[],"00176663":[],"00177336":[],"00177485":[],"00178263":37,"00178669":22,"00178952":[],"00179288":[],"00179419":[],"001795":[],"00180673":37,"00180834":[],"00181885":[],"00182423":[],"0018310257999764163":[],"00184571":[],"00185976":[],"00186052":[],"00186347":6,"00187032":[],"00187415":[],"00187791":[],"00187888":[],"0018798":[],"001880":6,"00188909":38,"00189016":[],"00189522":[],"0018954":22,"00191076":[],"00192114":[],"00192154":[],"00192165":[],"00192879":[],"00193326":[],"00193661":37,"00195147":[],"0019575":[],"00196372":[],"00196462":[],"00196725":38,"00196801":[],"0019685":22,"00196852":[],"00197431":[],"00197505":37,"00198129":[],"00199645":[],"00200":9,"00200982":[],"00202433":[39,40],"00202532":[],"00202624":6,"00202756":[7,34],"00202904":[],"00203961":[],"0020489":[],"00206234":[],"00208188":37,"0020841":[],"00208517":[],"00208828":22,"0020899":22,"00209403":[],"00210569":33,"00211332":[],"00211418":[],"00211555":[],"00211924":[],"00212766":[],"00213716":33,"00213888":[],"00214288":[],"0021452":33,"00214559":37,"00216162":[],"00216458":[],"00216881":22,"00216938":38,"00217143":[],"0021741":[],"00217499":[7,34],"00217968":[],"00218058":[],"00218381":[],"00218481":[],"002186":[],"00218823":37,"00219195":35,"00220013":[],"00220472":[],"00223857":[],"00224413":6,"00224491":[],"00224638":[],"0022476":[],"00225136":[],"00225842":37,"00225914":38,"0022626":[],"00226433":[],"00228042":[],"00228740":7,"00229111":[],"00230059":[],"00230078":22,"00230308":[],"00230805":[],"00231293":[],"00231412":[],"00232963":[],"0023327":[],"0023548":[7,34],"00236284":[],"00236918":[],"00237699":37,"002381316302584885":[7,34],"0023813163025848865":[7,34],"00238951":22,"0023941":[],"00240001":[],"00242441":37,"0024271":[],"00242856":[],"00242954":35,"00242967":[],"00242999":7,"00243186":[7,34],"00243835":[],"0024401":6,"00244459":[],"002446":[],"00244871":[],"00248232":[],"00248779":[],"00249126":38,"00249435":[7,34],"00250787":[],"0025089":[],"00251359":[],"00252192":[],"00253034":[],"0025349":22,"00253932":[],"00254671":[],"00254966":[],"00254973":[],"00256001":[],"00256537":[],"00256622":[],"00256688":[],"0025682":[],"00259434":38,"00259997":[],"00260527":[],"00260839":[],"0026126":[],"00263266":22,"00263296":37,"00263638":[],"00263755":[],"00263812":[],"00264":37,"00264181":[],"00264795":[],"00266858":[],"00267419":[],"00268602":37,"00270073":[],"00270197":[],"00270244":6,"00270262":[],"00272337":[],"00274212":[],"002743":[],"00274676":[],"00274989":[7,34],"00275212":[],"00275768":[],"00276424":[],"00279284":22,"00279722":[],"00282226":[],"00282302":37,"00282498":[],"00282501":[],"00283324":33,"00284402":[],"00284525":[],"00284866":[],"00284871":[],"00285017":[],"00285068":[],"00286091":38,"00286836":[],"00288198":33,"00288656":[],"00288666":[],"00289724":[7,34],"00290055":22,"00290316":[],"00290613":[],"00291535":37,"0029276":[],"00292882":[],"00293201":[],"00293758":[],"00293838":6,"00295372":[],"00295713":[],"00297571":37,"00297928":38,"00298783":[],"00299754":[],"00300342":[],"00300479":[],"00300908":[],"0030094":[],"00302509":[],"00302547":[],"00303686":33,"00307265":[],"00307704":22,"00309418":[],"00309594":[],"00309834":[],"00310113":[],"00311085":[],"00312361":[7,34],"00315593":7,"00316149":[],"003162":[39,40],"00316311":[],"00316452":[],"00318725":[],"00319571":22,"00319695":[],"00319887":[],"00320702":[],"0032153180657605116":[7,35],"00322456":37,"00323332":[7,34],"0032542":6,"00325450":35,"00325635":[],"00326986":[],"00327104":[],"00327555":[],"00327797":[],"00328539":38,"0032882":[],"00329616":37,"00329927":[],"00330046":[],"003301":[7,34],"00330245":[],"00331168":[],"00331927":[],"0033213":[],"00334002":[],"00334047":[],"00334155":[],"00334797":[],"00335101":[],"00335174":[],"00337016":[],"00339014":22,"00339369":[],"0033955154592040936":[7,35],"00340452":[],"00342132":38,"00342565":[],"0034318":37,"00344192":[],"00344887":[],"00347534":[],"00348246":[],"00348281":[],"00348553":[],"0034881":[],"0035086":[],"00351039":[],"00351649":[],"00352046":[],"00352089":22,"00352878":37,"00353823":6,"00353938":[],"00354483":[],"00355323":[],"00355614":37,"00356356":[],"00356549":[],"00359628":[],"00361046":[],"0036237":[7,34],"0036367":[7,34],"00363823":[],"00364159":[],"00364235":[],"00364967":37,"00365819":[],"00366866":[],"00368369":[],"00369201":[],"00369236":[],"00369758":[7,34],"00369852":[],"00370353":[],"00371435":[],"00371605":[],"00371656":[],"0037314":[],"00373511":22,"00377231":[],"00377286":38,"00379682":[],"0038022":[],"00380258":[],"00381349":[],"00382137":[],"0038287":[],"00383048":[],"0038335":[7,34],"00385557":[],"00387671":[],"00387917":22,"00388071":[],"00389038":37,"00389382":[],"00389577":[],"003909404072811226":[7,35],"00391839":6,"00392402":[],"0039259":[],"00392639":[],"00392691":[],"00392853":[],"00392895":38,"0039455":[],"00396815":[],"00397085":[],"00398429":[],"00398499":[],"00399354":[],"00399521":[],"0039987":[7,34],"00399972":[],"004":[6,34,35],"00403677":37,"00404166":[],"00405576":[],"00405581":[],"00406466":[],"00406952":[],"0040792":[],"00408":37,"004091940707753939":[7,35],"00410183":[],"00410387":[7,34],"00410478":7,"00410974":[],"004113634617443131":[7,33,34],"00411363461744314":[7,33,34],"004113634617443147":[7,33,34],"0041148":[],"00411514":22,"00412":37,"0041283":[],"00413064":[],"00413253":[],"00415405":[],"00415497":[],"00415598634586135":[7,35],"00416103":37,"00416704":[],"00418347":[],"00418664":[],"00420072":35,"00420536":[],"00422259":[],"00424491":[],"00424909":[],"00424967":[7,34],"00426027":6,"00426902":[],"0042739":22,"00429456":[],"0042992":37,"00431893":[],"00432129":[],"00432337":[],"00432864":[],"00433264":38,"00433417":12,"00434043":[],"00434704":[],"0043587":[],"00437846":[],"00438318":[],"00439674":[],"00440346":[7,34],"00441065":[],"00441097":37,"00442091":[],"004423":[],"00443629":33,"00443743":[7,34],"00444806":[],"0044505":37,"00445186":37,"0044547":[],"00445655":12,"00451186":38,"0045162":[],"00453377":22,"00455966":[],"00456562":[],"00457441":[],"00457599":[],"004578":33,"004579219539673834":[7,35],"00458083":33,"00458631":[],"00458878":[7,34],"0045909":[],"00459357":[],"00459568":[],"00460917":[],"004610275230656212":[7,35],"00461571":[],"00462287":[7,34],"00462662":[],"00463124":[],"00463246":[],"00464677":[],"00465099":35,"0046544":[],"0046741":[],"00468914":[],"00470102":[],"00470879":22,"00471617":[],"00471782":6,"00472199":7,"00472276":[],"00472512":[7,34],"00473136":[],"00473744":33,"00474898":[],"00476012":37,"00478417":[],"0047956":[],"00479764":[],"00480881":[],"00481334":[],"00483798":[],"00483858":[],"00485232":33,"00487029":37,"00493903":[39,40],"0049544":[7,34],"00497544":38,"00497706":[],"00497837":34,"00498004":[],"00498065":[],"00498243":[],"00498263":[],"00499474":22,"004999999999999987":[],"004999999999999989":1,"004999999999999991":[],"004999999999999996":[],"004999999999999998":[],"005":[],"005000000000000001":[],"00500716":[],"00502886":[],"00503732":[],"00503932":[],"0050554":[],"00505872":[],"00507014":[],"00507531":[],"00509128":37,"00510839":[],"00511073":[],"00512149":33,"00512599":[],"00512927":6,"00513051":[],"00514817":[],"00515165":[],"00515181":[],"00515889":[],"00516067":[],"00517114":[7,34],"00517673":[],"00518116":38,"0051879":22,"00519717":[],"00520618":[],"00520752":33,"00520911":37,"00521014":[],"00522055":[],"00523789":[],"00525279":[],"00525303":[],"00525734":[],"00526348":[7,34],"00527578":[],"00528252":[],"0053018":[7,34],"00531268":[],"00532167":[],"00532314":[],"00534541":[],"00536751":[],"00537955":[],"00539563":[],"00541941":[],"00544547":37,"00546503":[],"00546963":[],"00547113":[],"00547332":[],"00550187":22,"00551372":37,"00551695":[],"00551716":[],"00552462":[],"00554202":[],"00554552":[7,34],"00554707":[],"00556826":[7,34],"0055687":[],"00556946":[],"0055715":37,"00557893":[],"00558145":[],"00559722":[],"0055997":[],"00560098":[],"00560316":[],"00561212":[],"00561262":[],"00562142":[],"005643":[],"00564344":[],"00564365":37,"00564874":[],"00566707":[],"00567869":[],"0056799":6,"00568091":[],"00568352":[],"00569512":[],"00570059":[],"00570321":[],"0057065":[],"005708":[],"00570826":[],"00571352":38,"00571565":22,"00571608":[],"00571884":[],"00572613":[],"00573236":[],"00573661":[],"00573681":33,"00573727":[],"00574735":[],"00575681":[],"00576997":[],"00577229":[],"00578449":[],"00578809":[],"00579953":[7,34],"00579972":[],"00582034":[],"0058243":37,"00582949":[],"00585143":[],"00586355":[],"00588657":[7,34],"00588836":[],"00590379":[],"00590456":[],"0059267":[],"00593362":[],"00594949":38,"0059591":37,"00597698":[],"00598464":[],"00598510e":[],"00600791":[],"00602064":[],"0060439":[],"00605833":22,"00606112":[],"00606382":[],"00607072":[],"00607165":[],"00607783":7,"00609802848297877":[],"0061078":[],"0061084":[],"006150":12,"00615047":12,"006162":[7,34],"00617221":[],"00617261":[],"00617499":6,"0062202":[],"00622617":[],"00622949":37,"00623535":[],"00625051":[],"00625222":[],"00625572":[],"00625672":[],"00627115":[],"00627746":[],"0062968":22,"00630331":[7,34],"00630802":[],"00631759":[],"00632049":[],"00632251":[],"00633289":[],"00634299":[],"00634575":[],"00635016":[],"006352536528464608":[],"00635621":[],"00636951":[],"00637188":[],"00637367":37,"0063878":[],"00641423":[],"00642221":[7,34],"00642737":[],"0064349":[],"00644":37,"0064472":34,"00645121":[],"00647029":[],"00652931":[],"00653725":[],"00654058":[],"00654406":[],"00654835":[],"00654864":[],"00656079":38,"00657032":[],"00657601":[],"00659604":[],"00660427":[7,34],"00666287":37,"0066645":22,"00667611":[],"00669649":[],"00672471":[],"00672505":[],"00672607":[7,34],"00672654":[],"00673407":[7,34],"00673693":33,"00674158":[],"00676256":[],"00676387":[7,34],"00677215":[],"00679013":[],"00680029":[],"0068011":[7,34],"00681708":37,"00682686":[],"00683102":38,"00683748":6,"00683875021199284":[],"00683964":[7,34],"00685287":[],"00687309":[],"00688012":[],"00688182":[],"00689214":37,"00691012":[],"00691271":[],"00692345":[],"00693334":[],"00693613":22,"00694315":[],"0069435":[],"00695371":[],"00696127":[],"0069668":[],"00697033":[],"00698568":[],"00700361":[],"007024126888938232":[7,35],"00702562":[],"00702721":[],"00702885":[],"00703651":[],"00709577":[],"007096":[],"00710152":[],"0071129":[],"00711457":[],"00712219":[],"0071264":37,"00715512":[],"00717018":[],"00719176":[7,34],"00721287":[],"00722437":[],"00723172":[],"00724303":[],"00725498":[],"00727646693":[1,32],"00729134":37,"00731149":22,"00731392e":[],"007315":33,"00732483":[],"00732665":[],"00733216":32,"00735068":[],"0073544":[],"00736737":[],"007370297974992432":[],"00737863":[],"00738299":[],"00740723":[],"00740838":[],"0074331":6,"00745015":[],"00746317":[],"00747773":[],"00748862":[],"00752703":[],"00753078":[],"00753289":38,"00754823":[],"007557567939261264":32,"00755917":[],"00759119":7,"00759427":[],"00761224":[],"00761546":33,"00761755":[],"00762217":37,"0076268":23,"00763764":22,"00764333":[],"007658477904313023":[],"0076989":[],"00770419":[],"00771475":[39,40],"0077313":[],"00774376":[],"00774948":[],"00777619":33,"00778316":[],"00779859":37,"00781297":[],"00781659":[],"00781918":35,"00782481":[],"00784099":38,"00784393":[7,34],"00784531":[],"00786896":[],"00789262":[],"00791263":[],"00791445":[],"00792032":[],"00796116":22,"0079634":[],"00798909":[],"00799475":[],"00800011":[],"00801346":[],"00802302":[],"00802792":[],"00803064":[7,34],"008043811612683473":[],"00805074":35,"00806447":[],"00809364":[],"00810345":[],"00812487":[],"008128643056242772":33,"00813033":[],"00813803":7,"00815244":37,"0081568":[],"00816675":[],"00817631":[7,34],"00818251":[],"00819774":[],"00819875":[],"00821525":33,"00823002":6,"008230739628188457":[],"00827534":[],"00827728":[7,34],"00828121":[],"00829346e":[],"00831018":[7,34],"00834112":37,"00834567":[7,34],"00836649":[],"00838255":[],"00838555":[],"00840184":22,"00842216":[],"00842507":[],"00847367":[],"00848586":22,"00848739":[],"00848904":[7,34],"0085274606925055":[],"00853302":[],"0085392":[],"0085548":[],"008560581831215528":[],"0085639":[],"008574628106492125":[],"00857702":[],"00859693":[],"00860892":[],"00861518":37,"00864664":38,"00865335":[],"0086649156":[1,32],"00866766":[],"0086714":[],"00867406":[],"008675369724975977":6,"0086885":[],"00869259":[],"00870208":[],"00871267":[],"00871959":37,"00871967":[],"00872006":[],"00872869":[],"00873267":[],"00875286":[],"008753288788081405":[],"00877737":[],"00877816":[],"00877892":[],"00878815":[],"00881749":[],"00881939":[],"00885858":[],"00886253":[],"00887262":[],"00889027":[],"00889475":[],"00891351":[],"00891702":[],"0089204":[],"0089214":37,"00893112":[],"00893649":33,"00894639":6,"00896253":33,"00898223":[],"008994":38,"0089987":33,"00900007":[],"00901661":[],"00903778":[],"00904514":22,"00905423":[7,34],"0090724":[],"00908537":[],"009134699065945493":[],"00914784":33,"009163470508352218":6,"009164545680330616":[7,35],"00917248":[7,34],"00917694":[],"00917881":[],"00919984":[],"00920294":[],"009211602191395454":[],"00921748":22,"00922905":[],"00923082":[],"00923474":[],"00925406":37,"009277782622954527":33,"00930146":[],"00930457":33,"00931621":[],"00931882":[],"0093262":37,"009338082195270294":[],"00934327e":33,"00934499":[7,34],"00935742":[],"00938275":[],"00939407":[],"009396472959497925":[],"009397356386856284":[],"00939922":[],"00940611":[],"00942574":[],"00943026":[],"00943959":[],"00951885":[],"00952138":33,"00954206":37,"00955125":[],"0095585":[],"00956827":[],"00958615":[],"00959357":32,"0096208":[7,34],"00963683":[],"00964128":22,"00964129":[],"00965205":[],"00965904":[],"00966192":[],"00966668":[],"0096667":[],"00968268":[],"00969623":[],"00970059":[],"00970896":[],"00971344":[],"00972948":[],"00974758":[],"00978267":[],"00979031":[],"00979357":[],"00982052":[],"0098443":[],"009880124918446542":[],"00988081":[],"00988356":[],"00988992":[],"00989118":[],"00989834e":[],"00990475":6,"00991812":38,"00992331":[7,34],"00995699":32,"00996754":[7,34],"00996762":[],"00997215":[],"00997502":37,"00997697":[],"009998596494598343":[],"01":[0,1,2,3,6,7,10,12,14,18,22,31,32,33,34,37,38,39,40,41],"010018312644137363":[7,35],"01003519":[],"01003651":22,"01006":37,"01006224":[],"01006335":[],"01006984":[],"0100706":[7,34],"01008116":[],"01009321":[],"01009625":[],"01013106":[],"01013849":[],"01014331":[],"010148621093080332":[],"01016442":[],"01016493":[],"01016961":[],"010175219431920396":[],"01018089":[],"01018465":33,"01019212":[],"01020588":37,"01024875":[],"01027183":[],"01027671":22,"01029828":38,"01031":37,"01032298":[],"01032533":[],"010331721306655127":[7,35],"010348289064969316":[],"01035006":[],"01039355":[],"01039879":[],"01039977":[],"010405448150048726":[],"01044475":[],"01045576":[],"01048771":[],"010516485576645508":[7,35],"01055384":[],"01055538":[],"0105667":[],"0105836":[],"01059242":[],"01060492":[],"010612904886352397":[],"010638334964957053":[],"01066519":[7,34],"01066897":37,"01067432":[],"01069799":22,"01070103":[],"01070717":[],"01071876":[],"01074379":[],"01074493":33,"0107497":[],"01075018":[],"01076611":6,"01076897":37,"0107744":[],"01078008":[],"01080465e":[],"01084359":[],"01085261":[],"01085571":[],"01086401":[],"01086733":[],"01087815":[],"01089212":[],"01091589":37,"01093529":[],"01093553":[],"01094916":[],"01095403":22,"01097287":[],"01098247":[],"01099776":[],"0110":29,"01102311":[],"01102737":[],"01105668":[],"01105813":33,"0110602":33,"01107333":[],"011076219011379514":[7,35],"01109809":[],"01109955":[],"01112157":[],"0111346":[],"01115456":[],"011156605304609659":[],"01116732":[],"01118392":[],"01119357":[],"01119504":[],"01119738":[],"01119981":[],"011208613520466846":[],"011225":[],"01123792e":[],"01127138":[],"01127942":[],"01129731":[],"0113104":[7,34],"01131771":[],"01131825":[],"01133846":[],"01133903":[],"01135511":[],"01135642":38,"01140194":[],"01140307":22,"01141119":37,"01142354":[],"01146857":[],"01149099":[],"01149386":33,"01149735":[],"01150437":[],"01151556":[],"01153142":[],"01153294477941533":33,"0115357":[],"011594819155615":[],"01161645":[],"01163942":[],"0116753":37,"01167598":22,"01169016":[],"01169528":[],"01171276":[],"01172319":[],"01173863":38,"01174937":[],"01176967":[],"01178583":[],"01178857":[],"01179128":[],"01179415":[],"01179792":[7,34],"01180564":[],"01180692":[],"01181458":[],"01181473":[],"011818646101922625":[],"011886":[],"01191824":6,"01192135":[],"01193263":[],"01194492":[],"01195336":[],"01196506":[],"01199378":[],"01199434":[],"01199741":[],"01203439":[],"01205039":[39,40],"01206221":[],"012073649446193166":[7,35],"01211133":[],"0121289":[],"01214076":[],"01214862":[],"01215461":22,"01216075":[],"01216231":[],"01216264":[],"0121639":[],"0121766":[],"01218403":[],"01218736":[],"01219292":[7,34],"01220301":[],"01220505":37,"01222822":33,"01222848":[],"01222875":[],"01223198":[7,34],"01223754":[],"01229008":[],"01229102":[],"01229692":[],"01231917":[7,34],"01232366":[],"0123263":[],"01234":[],"01234261":[],"01235079":[],"01235257":[],"01241284":[],"012423940191689783":[],"01244297":[],"01244551":22,"01247909":[],"01248754":37,"01249445":[],"01249968":[],"01250964":[],"01257912":[],"01258423":[],"01260207":[],"01261048":[],"01263474":33,"01265755":33,"012658":33,"01266855":[],"01266999":[],"01267007":34,"0126711":[],"01268737":[],"012687664954397":29,"01272102":[],"01273648":[],"01276369":[],"01278044":[],"01280793":[],"01282563":[],"01286909":[],"01288787":[],"01289266":[],"01290947":[7,34],"0129191":[],"01294476":38,"01294799":[],"01294954":[],"01295356":6,"01295569":22,"01299711":[],"01300036":[],"01305415":37,"013121574015585152":[7,35],"013121574015585155":[7,35],"01312585":[],"01315875":[],"01316233":[],"01316638":[],"01317824":33,"01318643":[7,34],"0132081":[],"01321707":[],"01322347":[],"01322434":[],"01322589":38,"01323341":[],"01323523":[],"01326577":22,"01327148e":[],"01327475":[],"01328658":[],"01328972":[],"01329403":[],"01332832":[],"01333573":[],"01335628":37,"0134113":[],"01343464":[],"01343585":[],"01346122":37,"01346635":[],"01347916":[7,34],"01348565":[7,34],"01349244":[],"01353569":[],"01353638":[],"01355882":[],"01357366":[],"01359774":[],"01361517":[],"01366555":[],"01367553":[7,34],"01369614":[],"01370129":[],"0137297":37,"01373315":[],"01373484":[],"01376216":[],"01377529":[],"01377545":[],"01380833":[],"01380957":22,"01383689":[],"01384583":[],"013846":[],"01386236":[],"01393427":[],"01393496":[],"01395192":[],"01396231":37,"01397146":[7,34],"01399101":[],"01399457":[],"01405935":[7,34],"01406777":[],"01408975":[],"01409478":[],"01414008":22,"01415781":[],"01416528":[7,34],"01418766":[],"01419341":[],"01420773":[],"0142117":[],"01425384":[],"01427864":[],"01428546":37,"01429853":[],"01433809":6,"01438804":[],"014394967608286841":[],"0144045":[],"01440748":[],"01441661":[],"01442378":[],"01442817":33,"01443283":[],"014436800088904942":[7,35],"01444013":[],"01444966":38,"0144623":[],"01449782":[7,34],"01454469":[],"01455301":33,"01455304":[],"014565":[],"01456865":[],"01457889":[],"01458337":[7,34],"01458483":38,"014586":33,"01458611":33,"01459912":[],"0146081":[7,34],"01462254":[],"01463049":[7,34],"01463112":[],"01464686":[],"01471973":22,"01473588":[],"01473759":[],"01476927":[],"01478037":[],"01479672":[],"01480212":[],"01480256":[],"01480927":[],"01493365":37,"014934":37,"01494559":[],"01498697":[],"01499192":[],"01499389":[],"01499589":[],"01504759":[],"01506354":[],"01507202":22,"01507238889517706":[7,34],"01507238889517717":[7,34],"01509793":[],"015101938515449663":29,"01511937":37,"01512884":[],"01515818":[],"01520159":[],"01522108":[],"01523958":[],"01525387":33,"01527618":33,"01527928":37,"01530321":[],"01531845":[7,34],"01532072":33,"01532738":[],"01534161":[],"01543076":[],"01546126":[],"01548486":[],"01549377":[7,34],"01549917":[],"01551724":[],"01551775":[],"015518":[],"01551815":[],"01552039":[],"01552874":[],"01554658":[],"01558197":6,"01562424":25,"01563358":33,"01566026":[],"01566493":[],"01567258":[],"01568403":[],"01568987":22,"01572":37,"01573062":[],"01573768":[],"01574065":[],"01574456":[],"01575266":[],"01578667":38,"01584129":[],"01593089":[],"01593435":38,"01594452":35,"01597256":37,"01597958":[],"01600363":[],"01604235":[],"01604832":[],"01606538":22,"01606644":[],"01607157":[],"01612308":[],"01612637":[],"01613409":[],"01615836":[],"01616657":[],"01620955":[],"01622922":[],"01624447":[],"01625794":[],"01628578269596628":[7,35],"01630755":[],"01633913":[7,34],"01634224":37,"01635525":[],"01639344":[],"01639489":[],"01640891":7,"01641044":[],"01643795":[],"01643952":33,"01647074":[],"0164974":[],"01654574":[],"01655001":33,"01655318":[7,34],"01655384":[],"01656815":[],"016587414993043573":[7,35],"01671659":33,"01672395":22,"01674956":[],"01677178":[],"01678891":[],"01679297":[],"01681173":[],"01681612":[],"0168171":[],"01682317":[],"01682652":37,"01684463":[],"01685118":[],"01686439":[],"01691985":[7,34],"0169643":6,"01698477":[],"01699169":[],"01700746":[],"01700897":[],"0170097":[],"01701429":[],"01705695":32,"01705781":[],"01706166":[],"01708375":37,"01708781":[7,34],"01708852":[7,34],"01709886":33,"01712421":22,"01713187":[],"01713366":[7,34],"01717238":[],"01717549":[],"01717775":[],"01721277":[],"01722503":[],"01723872":[],"01724499":6,"01724739":38,"0173119":[],"017355848195593347":[7,35],"01735584819559336":[7,35],"01735999":[],"01736099":[],"01737595":[],"01737634":[],"01740381":[],"01740873":38,"01743124":[],"01746176":33,"01747915":37,"01748034":[],"01752868":[],"01756114":[],"01759442":[],"01759974":[],"01760144":[],"01761069":[],"017665":6,"01772174":[],"01772779":[],"0177579":[],"01779091e":[],"01782618":22,"01783096":[],"01783414e":33,"0178462":[],"01784845":[],"01787801":[],"01789589":[],"01793261":[],"01793972":[],"01795684":[],"01797196":[],"01797685":[],"0179826":[],"018":[],"01804672":[],"0180908":[],"01811279":[],"01811373":[],"01813363":[],"01814351":[],"01814683":[],"01817048":[],"01817739":[],"0182116":[],"018232":33,"01825282":22,"01826645":38,"01826811e":[],"01827225":37,"01827302":[],"01830039":[],"01831050e":[7,34],"01831190e":[7,34],"018365206985554446":[],"01838531":[],"01839352":[],"01844584":[],"01844822":[],"01846605":[],"01853796":[],"01854489":[],"0186037":[],"01863203":[],"01866537":[7,34],"01868119":[],"01869515":37,"01870931":[],"01871612":[],"01872002":[],"01873344":12,"01873823":33,"01873869":6,"01874605":[],"01877081":6,"018771":6,"01879191":[],"01881237":[],"01882218":[],"01882264":[39,40],"01884327":38,"01886197":[],"01893445":[],"01897462":[],"01898383":[],"01898855":[7,34],"01900106":22,"01901953":38,"01905384":[],"01905883":[7,34],"01908329":33,"01908936":[7,34],"01909296":[],"01909929":[],"01915337":[],"01916065":[],"01918426":[],"01919791":[],"01922269":[],"01922564":[],"01924105":[],"01925934":[],"01930676":[],"0193075":[],"0193386":[],"01944073":[],"01945582":22,"01947793":[],"01950331":33,"01952679":[],"0195302":[],"01954342":37,"01962001":[],"01963611":[7,34],"01966173":[],"01968447":[],"01969093":[],"01969145":[7,34],"01974315":[],"01975006":[],"01975416527185249":[7,35],"01975755":[],"01975848":[7,34],"01979808":[],"01981665":[],"01991068":[],"01991819":[],"01999575":37,"02":[0,1,5,7,8,13,32,34,36,38,39,40],"0200041":[],"02010178":[],"02014577":[],"02015755":33,"02016185":[],"02019986":[],"02021288":[],"02024962":[7,34],"02025337":22,"02025574":[],"02026756":[],"02028151":33,"02028936":[],"02031789":[],"02031987":[],"02032056":[],"02034593":[],"02034655":[],"02036067":33,"02037155":[],"02038063":33,"02044223":[],"02045223":[],"02045753":[],"02054837e":[7,34],"02056537":[],"02058408":[],"02058681":38,"0206033":[],"02060364":[],"02068067":[7,34],"02070909":[],"02072876":[],"02073509":6,"0207381":22,"02075746":[],"02077431":34,"02077938":38,"02079774":33,"02079913":[],"02083309":[],"02084474":[],"02084969":[],"02087411":[],"02088307":[],"02090304":37,"02091662":[],"02092464":[],"02098261":[7,34],"02101168":[],"02102403":[],"02103315":37,"02104639":[],"02108599":[],"02109876":33,"0211042":33,"0211429":[],"02117843":[],"02118344":[],"02123176":[7,34],"02124568":[],"02125608":[],"02126205":[],"02132557":[],"02134409":[],"02137093":[],"02137508":[],"02138683":37,"02141849":[],"02143946e":[],"02146636":[],"02148021":37,"02150453":[],"02151521":[],"021544346900318832":34,"02156196":[],"02156565":[],"02157768":38,"02158822":22,"021592704588025022":[7,35],"021592704588025025":[7,35],"0216304":[],"02163476":[],"02164667":[],"02169399":[],"02171224":[],"02175561":[],"02176052":[],"0217610577940561":[],"02177725":[],"02181163":[],"02183817":[],"0218856":[],"02189606":[],"02189863":[],"02189994":[],"02193078":[],"02196086":[],"02198702e":[7,34],"02198703e":[7,34],"02199232":[],"022":[],"02203851":[],"02204591":[],"02206393":33,"02207054":[],"02208512":[7,34],"0221049":22,"02214409916811925":[],"02218045":[],"02218076":[],"02225493":[],"022255":[],"02228115":[7,34],"02229529":[7,34],"02232526":[],"02234283":33,"02235723":37,"022379426843681074":[],"02239876":[],"02241952":[],"02249167":38,"022498":[],"02251592":[],"02252765":6,"02253659":[],"02257687":[],"02260949":[],"02260989":[],"02265005":[],"02265404":[],"02265998":[],"02270207":38,"02280634":[],"02281004":[],"0228324":[],"02286267":[],"02287468":37,"02288069":[],"02290895":[],"02296298":37,"022999498260366312":[7,35],"02301105":22,"02312079":[],"0231353":[],"02319624":[],"0232":[],"0232101":[],"02322297":[],"02322594":[],"02322786":[],"02323402":[],"02325785":29,"02327527":[],"023295599127611474":[],"02330799":[],"02330901":[],"02333688":[],"02334754":[],"02336409":33,"02338085":[],"02342235":[],"02345858":[],"02348765":[7,34],"02349274":[],"02352005":[],"02352663":[],"02352843":[],"02354386":[],"02355756":[],"02356004":[],"02356178":22,"02358547":[],"02359571":[],"02361665":[],"02363304":[],"02365049":[7,34],"02369044":[],"02374496":[],"02375066":33,"02386932":[],"02387182":[],"0239126":37,"02391906":[],"02392067":[],"0239521":[],"024":[],"024004":[],"024011020121022356":[],"02404706":[],"02409835":[],"0241039":[],"02413786":[],"02424794":35,"02429596":[],"02431532":[],"02432309":[],"02432632":[],"02432881":[],"02436346":[],"02439287":[],"02441422":[],"02444266":[],"02444569":[],"02446604":37,"02447063e":[],"02448011":33,"0244965108017645":[],"02452765":22,"02452792":[],"02454051":[],"0245528":[7,34],"02456":37,"024562":[],"02456235":[],"0245728":38,"02458099":[],"02459084":37,"02472936":[],"02474828":[],"02477293":[],"02478142":[],"024792624800382218":[],"02480266":38,"02483712":33,"02490769":[],"02492265":6,"02496085":37,"02498282":[],"02498832":[7,34],"02499437":[],"02500488":[],"02502325":[],"02503392":[],"02503753":[7,34],"02504521":[],"02506901":[],"02511468":22,"02511518":7,"02519009":[],"02521917":[],"02522069":7,"02526293":[],"02527621":[],"02529207":[],"02530745":[],"02531253":[],"0253208":[],"02533443":[],"02535112":[],"02535694":[],"02535973":[],"02540212":[],"02544154":[],"02545":37,"02546675":35,"0255141":[],"02552966":[],"02553865":[],"02553907":[],"02557617":37,"02561386":[],"02564828":[],"02566356":[],"025709":[12,33],"02572446":[],"02572685":[],"02576358":[],"02576685":[],"02580654":[],"02584233":[],"02585059":[],"02586427":[7,34],"02593707":[],"02594314":[],"02595095":[],"02597067":[],"02597861":[],"02599188":[],"02600605":[],"02601643":[],"02604362":[],"02606605":[],"0260728":33,"0260906":[7,34],"02612988e":[],"0261442":22,"02616812":37,"02617297":[],"02617887":[],"02622039":[],"02623761":[],"0262517":[],"02625193":9,"02625852":[],"02629144":37,"02629373":[],"0263106":[],"02631244":[],"02631942":[],"02632742":[],"02635861":[],"02636268":[],"02646648":[],"02649469":[],"026495":[],"02651857":[],"02654585":[],"02658761":[],"02660572763718093":[7,35],"026605727637180936":[7,35],"02661264":[],"02664408":[],"02667406":[],"02676992":22,"02677463":[],"02684648":38,"02687671":[],"02692965":[],"02693015":[],"02693858":34,"02694975":[],"02704796":[],"02705202":[],"02707227":6,"02709478":[],"02709761":38,"02710291":[],"0271259":33,"0271583":[],"02717039":[],"02723445":[7,34],"0272743":[],"02729375":[],"02731808":[],"02732402":[],"02735547":37,"02751373":[],"02754248":[],"02760064":[],"027609773491022525":[7,35],"02760977349102253":[7,35],"02761893":[],"02766946":[],"02769274":[],"02780107":[],"02781444":[],"02784163":32,"02784266":[],"0278673":22,"02787184":[],"02792633":[],"02793082":[],"02797154":[],"0279886":37,"028":[],"0280223":[],"02805988":[],"02806268":[],"02808229":[],"02809972":[],"02813369":[],"02813549":[],"02816961":[],"02818667":[],"028188673105960467":[],"02832687":[],"02833443":[],"02835701":[],"0284405":[],"02846506":[],"02850506e":[],"02853426":22,"02856881":[],"02857":5,"02859759":[],"028638258927275215":[],"02872216":[],"02876848":[],"02880987":[],"02884485":29,"02895708":[],"02895911":[],"02895918":34,"02899912":[],"02900153e":37,"029003":[],"02901273e":[],"02901441":[],"02909574":[],"02910539":22,"02910881":[],"02911348":[],"029114513319594925":29,"02920675":[],"02923006":[],"0292555":[],"02925855":37,"02928194":[],"02930753":[],"02933055":38,"02940083":[39,40],"02941655":[],"02943644":[],"02944152":[],"02944974":[],"02946852":[],"029483":6,"02953975":[],"02960445":[],"02960492":38,"02962583":[],"02962903":[],"02963147":[],"02963424":[],"02965159":[],"029664":[],"02966421":[],"02970225":[],"02970397":22,"029733":33,"02973733":[],"02976145":[7,34],"02976686":[],"02979087":[],"02980537":[],"02982009":[],"02987043":[],"02987496":[],"02988432":[],"02990404":33,"02992287":[],"02993573":37,"02994311":6,"02996909":[],"02999715":[],"02f":[7,26],"03":[2,7,33,34,40,41],"0300023":[],"03000689":[],"03003494":33,"03003891":[],"03006815":[],"03008641":[],"03012648":[],"03015288036252618":[],"03016341":[],"030164788184997":[],"03021588":[],"03031173":[],"03032441e":[7,34],"03032534":33,"03033791":[],"0303586629618948":[],"03041332":[],"03041488":22,"0304378":[],"0305221":[],"03057995":[],"03059882":[],"0306232":[],"030645975175292262":[],"03065856":33,"03068149":[],"03076019":[],"0307725":[],"03077640549":5,"03092729":[],"03099776":6,"03099794":[],"031":[6,34,35],"03100531":[],"03101165":[],"03112129":[],"03115122":[],"0311613":[],"03119062":[],"03120433":[],"03120942":[],"03143856":33,"03158013":[],"03159246":[],"0316146":[],"03163926":[],"03166151":[],"03166168":22,"03166546":[],"03170652":[],"0317606":[],"03177331":[],"03181997":[],"03188133":[],"03194307":[],"03196357":[7,34],"03196582":[],"03201179e":[],"03201217":[],"03201832":37,"03201922":[],"03204446":38,"03205795":[],"03206321":[],"03206757":[],"03207075e":[39,40],"0321222":[],"03212854":[],"032142":[],"03214247":[],"03214829":[],"0321956":[],"032196":[],"03221473":[],"03221782":[],"03230611":[],"03234422":38,"03235719":[],"03237424":[],"03239118":[],"03239393":[],"03241171":[],"03241936":25,"03241945":22,"03244944":[],"03246707":[],"03247316":[],"0324793":[],"03248618":33,"03251863":6,"03254179":[],"03254614":[],"03256632e":[2,40,41],"03261373":[],"0326167":[],"03266152":[],"03267527":[7,34],"03274273":[],"03279636":[7,34],"032825":6,"03282533":6,"03283413":[],"03285784":[],"0328643":37,"03291795":[],"0330308045183194":[7,34],"0330308045187757":[7,34],"03304417":[],"03306538":37,"0330705":[],"03308408":6,"03309191e":[],"03311432":[],"03313193":[],"03313259e":[],"03317599":[],"03318519":[],"03319608":[],"0332335":[],"033233504714921":[],"0334136":[],"033414":[],"03343379612645295":[],"0334505":[],"03345624":[],"03351257":33,"03354468":[],"03354844":[],"03357723":[],"03358833":[],"03361037":[],"03364384":[],"03365012":[],"03365665":[],"033657685071527665":[7,35],"03374843":22,"0337609":[],"03382942":[],"03396658":[],"034":[],"03400724e":[],"03404758":[],"034048":[],"03409194":[],"03412067":[],"03418643":[],"0342028":[],"0342458":37,"03426412":[],"03426934":[],"03430385":[],"03439189":[],"034404":[],"03440431":[],"03447512":[7,34],"034476":[],"03447649":[],"03454641":[],"03455614":22,"034557":33,"03455885":[],"03468878":[],"03472308":[],"03474443":[],"03476611e":[],"03479236":[],"03479936":[],"03484266":[],"03484946":33,"0348572":[],"03500569":[],"03500949":38,"03509683":[],"03510464":[],"03511387":[],"03514117":29,"03529083":[],"03529801":[],"03530044":[],"03532164":33,"03533698":38,"03538463":[],"03538673e":[],"0354495":[],"0354894":[],"03554165":[],"03558034":[],"03559176":[],"03562355":[7,34],"03563709":[],"03563906":[],"03568439":[7,34],"03568495":[],"03570093":[],"03571339":[],"03573214":[],"03578526":[],"035810431523915":12,"03582724":[],"03586482":[],"0359565":6,"03596557":[],"0359727":22,"03601781":33,"03609183":[],"03612962":33,"03613058":[],"03613164":6,"036132":6,"03624103":[],"03624411":[],"0362483569261072":[],"03625417":[],"03630548":[7,34],"03642173":[],"03647182":25,"03648711":[],"036527092051294":[],"03656288":[],"03662823":37,"03664879":[],"03680811":[],"03681113":[],"03681479262838276":[],"03682195":[],"03683365":22,"03692132794542241":[],"03692749":32,"03692828":[],"0369544130635358":[],"03696458":[],"03699304":[],"03701105":[],"03703391":[],"03707133":12,"0370757046153366":[],"03710736":33,"03722287":[],"03733016":[],"03734925":[],"03736678":[],"03742586":[],"0374535":[],"03745816":[],"03751833":[],"03753059":[],"03768291":[],"03770824":[],"03776519":[],"03778607":[],"037813671417389005":[7,35],"03781367141738902":[7,35],"03796359":[],"03796582":[],"03798785":[],"038":[],"03805295":[],"03814292":[7,34],"03815288":[7,34],"03820746":[],"03824887":38,"0382721":[],"03827385":[],"038300":[12,33],"03834357":22,"0383613556521865":[],"03837":37,"03843478":33,"03844073":[],"0384468":[],"03846169":34,"03853213":[],"03853268":[],"03853313":[],"03853677":[],"03856054":[],"03856748":[],"03858916":[],"03860666":38,"03868311":[],"03868372":[],"03870024":[],"03872795610436844":[],"03873507":[],"03877201":[],"03887801":[],"03887807":[],"03888998":[],"038890":[],"03896708":[],"03901061":[],"039039":6,"03906948":[],"0391259":[],"03916813":[],"03923714":[],"03926126":22,"0394226":[],"03942371":[],"03948301":[],"03960002":[],"03961179":[],"0396789":[],"0397137":[],"039732":[],"03973227":[],"03973359":[],"03974553487733608":[],"039764":[],"03976851":37,"03976976e":[],"03979291":[],"0398603":[],"03990261":[],"039967668952797":[7,34],"0399676689527975":[7,34],"04":[2,7,12,34,40,41],"04000457":[],"04001696":[],"04006242":33,"040102":6,"04010697":[7,34],"04015455":[],"04018043":[],"04018705":[],"04018985":[],"04021377":[],"04025059":[],"04026":41,"04026126":[],"04030508":[],"04033524":[],"04038595":[],"04040902":[],"04041644":32,"04046513":[],"04046984":[],"04051441":[],"040569":[],"04061345":[],"04063532":[],"04063602":[7,34],"04065686":[],"040657":[],"040684":[],"04072235":[],"04073097":[],"04074169":[],"04081099":[],"040810991569604":[],"0408223":[],"04085864":[],"04085976":[],"0408707":22,"0409646":[],"04097813":[],"04098313":[],"04098879":[],"041":10,"04101344":[],"04104182":[],"0410697":[],"04108038":37,"04110474":[],"0411487294305088":[7,34],"0411487294305226":[7,34],"04121466":[],"0412828":1,"0414517":33,"04150522":[],"04158557":[],"04161572":[],"04161649":[],"04164966":[],"0417409":[39,40],"04178558":12,"041786":12,"04178798":38,"04183444":[],"04184888":22,"04186033":[],"04191323":[],"04193203":33,"04198387":[],"042060":[],"04206048":[],"042136":[],"04216532":[],"04217103":[],"04217888":38,"0422":[],"04220758":[7,34],"04220931":[],"04224929":[],"04235439":[],"04238763":[],"04240593":[],"04245961":38,"04253288e":[],"04254787":[],"04262217":[],"04268493":[],"04277528":[],"04286013":[],"04293545":[],"04295726":[],"04295757":35,"043":10,"043072":[],"04307275":[],"04315108":6,"04315855":[],"04322019":[],"04323936e":[],"04323964":[],"04328852":[],"04329607":[],"04329799":[],"04330558":[],"04330907":[],"0433282860079154":[],"04346721":6,"04350821":[],"04353549":[],"04355062":[],"04355106":[],"04355837":[7,34],"04356439":22,"04362":10,"04362042":[],"043637":[],"04369752":[],"04372016":[],"04372067685794603":[],"0437499":3,"043839":[],"04389027":[7,34],"044014":[],"04401421":[],"04403475":[],"04404068":[],"04413243":[],"04418179":[],"04421197":[],"0442242":[],"04423486":[7,34],"04426744e":33,"04431557":[],"044334":33,"044338305845698256":[],"04435458":[],"04443985":[],"04452918":[],"04460703":22,"044613":[7,34],"04465557":[],"04487298":[],"0449":[],"04494234":[],"04495752":[],"04497158":[],"04497816":[],"04501146":[],"04501548":[],"04504728e":[],"04513394":[],"045161":[],"04516434":[],"04519174":[],"045258":[],"04532405":[],"04537385":[7,34],"04543942":[7,34],"04544327":[],"04545125":[],"04545296":[],"04562561":[],"045630":[],"04564589":[],"04565456":38,"04566964":[7,34],"04575775":[],"0458":10,"0458524213754298":[],"04585476":[],"04587765":[],"04588592":[],"04590873":[],"04592382":[39,40],"04596002":32,"045963":[],"04601419":[],"04608163":38,"04626298":[],"0462999":[],"04638043":[],"04640695":[],"04641529":[],"04643561":22,"04646338":[],"04648225":[],"04648335":6,"04648472":12,"046485":12,"04652794":[],"04655574":[],"04657426":[],"04658881":[],"04661875":[],"04664255":7,"046663":[],"046678":[],"04670255":[],"046764":[],"04683565":6,"04684509":[],"04686491":[],"04688012":[],"04702861":[],"04718566894028431":[],"04719868":[],"047299":[],"04729958":[],"0473":7,"04736252":[],"0473916":[],"04740084":[],"047447724189861":[],"0474753":37,"047489":[],"04754697":22,"04757138":[],"04759123":[],"0476266":[],"047760":[],"04776242":[],"04784395":[7,34],"04794409":[],"048020":[],"04808723":[],"04816191":[],"0481872773043029":[7,35],"048194":[],"04823243":[],"0482407":33,"04829439":[],"04832356":[],"04836325":33,"04838148":[],"04838489":[],"04843417":[],"04843794":[],"04852743":[],"04853398":[],"04857266":[],"04861316":[],"04863792":29,"04877931":[],"048781":[],"0488139":[],"04881585":[],"04892055":[7,34],"04899367":[],"049060":[],"04909093":[7,34],"04912436":7,"04915593":[],"049196":[],"04922273":[],"04926746":35,"049276":[],"04933083":[],"04937026":[],"049380":[],"04939861":[],"049479":[],"049481":[],"04949606":22,"049556996627824":[7,34],"0495569966278269":[7,34],"04955942":[],"04963309":[],"049677":[],"04984743":[],"04987891":38,"04999052":[],"05":[2,5,7,26,33,34,37,38,39,40,41],"05000245":[],"050015":[],"05002471":[],"05005634426334421":[],"05006037":[],"05006956":[],"05009826":[7,34],"05013368":[],"0501942":[],"050281":6,"05029018":[],"05029453":[],"050315327923114654":[],"05034088e":[],"0503455":38,"050357":6,"05040878":[],"0504379":[],"05043958":[],"05045":[],"05054319":[],"05055465":[],"05060379":[],"05068067":22,"05068486":[],"05072744":[],"05077224":[],"050796":[],"05080769":[],"05087018":[],"05097344":[],"05098714":[],"05100875":[7,34],"051021":[],"051042":[],"05105586":33,"05110835":[],"05124625":[],"05135047":37,"051368":[],"05141614":[],"051418":6,"05144177":[],"05148401":[],"05151178":[],"051554":[],"05159088":[],"051598":[],"051645":[],"051649":[12,33],"05169046":[],"0517473":6,"05176711":[],"051782":25,"051787":[],"05180367":[],"051916914594944144":25,"052010":[],"05201041":[],"05211919":[],"05224117":[],"05224627":[],"0522653":[],"05227563":[],"052279218012056844":[7,35],"05227921801205686":[7,35],"052305":33,"05232855":33,"05233008":[],"05233928":29,"052360":6,"052370":[],"05240397":[],"05241265":[],"052427":[],"0524497":[],"05245442":[],"05247827":[],"0525733":[],"05263":10,"05265465":[],"05267589":[],"052692":[],"05269476":[],"052738":[],"05275823":22,"052856":[],"05285677":[],"052918":6,"05293216":[],"05293756":[],"05296475":[],"0529838":[],"05302":10,"053036":[],"05321642":33,"05334469":33,"05336637":[],"053387":[],"053417":33,"053462":[],"05347795":[],"05353553":[],"053599":[],"05363722":[],"05364579":[],"05364854":9,"05376358":[],"053781":6,"053787":[],"05383795":[7,34],"05383947":[],"05384858":[],"053849":6,"053854":[],"05395761":[],"05401912":[],"05402091":22,"05404387":[],"05415049":[],"05415575":[],"05415968":[],"054197":[],"05424351":[],"0542797":[],"054286":[],"05430625":[],"05431945":[],"05442743":[],"054432":6,"054434":[],"05447415":[7,34],"05449413":38,"054500":6,"05458323":[],"054609":[],"05461692":[],"054617":33,"054655":33,"05476114":[],"05484773":[],"054932":[],"055000":[],"05500389":38,"05505310046363":[],"055091049054481":[],"05509105":[],"055105":[],"05519999":34,"05526935":[],"0552878":[],"055314":[],"0553198496333376":[],"05533":10,"055349":6,"05546402":[],"055477":[],"05548187":[],"05549529":[],"05549907":33,"05554795":[],"05563358":[],"055653":[],"055676":33,"055715":6,"055734":33,"055737":[],"05574159":[],"05577845931438246":[],"055866":[],"055876":[],"055910":33,"055925":[],"055987":33,"056026":[],"056041":[],"05605667":[],"05605892":[],"0560663":[],"05607583":[],"0560917":[],"05614483":6,"0562":[],"05623":10,"05623539":22,"056287":[],"05632246":[],"056328":[],"05636265":[],"05638246":[],"05641352":[],"056414":[],"05648":10,"05648118":[],"05648836":[],"056491":6,"05651951":[7,34],"05655534":33,"056561":6,"05660129":[],"05664639":[],"05665429e":[],"05665875086534638":[],"05667":10,"05668052":[],"056683":33,"056717":[],"056718":6,"056730":[],"056826":[],"056856":[],"05692207":[],"05696574":[],"05706322":25,"05716368155342608":[7,35],"057216":[],"057287":[],"057298":[],"057300":33,"057312":6,"057316":[],"05731824":[],"05737289":[],"05738485":[],"05738629":[],"057387":[],"057393":33,"057418":33,"057420":[],"057457":33,"05747369":[],"05749111":[],"05750493e":[],"057514":[],"057607":[],"057609":[],"05761244":[],"05761295":[],"05767":39,"057682":[],"057711":6,"057722":33,"0578214":[],"057835":33,"05783782":[],"05785343":[7,34],"05785974":[],"05788664":[],"05788748":[],"05789007":[7,34],"05791283":[],"057913350391706":[],"057929":[],"05792927":[],"05793564":33,"057951":[],"057961":[],"05796251":[7,34],"057964":[],"05804539":[],"05804744e":37,"0580672":[],"05806981":[],"05807125":[7,34],"058133":[],"0581406":33,"058152":[],"05818561278267464":[],"05818879":[],"058234":[],"058245":[],"058254":[],"05831017":[],"05834577":[],"058347":12,"058361":[],"058409":[],"058422":[],"058427":[],"0584713":[],"058503":[],"05850899":[],"05853984":[],"058552":33,"058582":[],"058593":[],"058641":[],"058645":33,"0586484":37,"058670":6,"05867956":[],"058721":[],"058726":6,"058770":[],"058774":6,"05879106":[],"05883":10,"05884":10,"058854":33,"058866":[],"05888195":[],"05893076":[],"058935":[],"058952":33,"058967":[],"058970":[],"058996":33,"058997":[],"059003":6,"059031":33,"059093":6,"05914563":[],"059181":[],"059201":[],"059233":6,"059243":[],"059340":[],"059348":6,"05938299":[],"059398":6,"05941988e":[],"059480":[],"0595364":38,"059624":[],"059669":[],"059761":[],"05978512":[],"059805":12,"059807":33,"059827":6,"05984726":[],"059864":6,"059876":[],"05992237":[],"05994172":22,"059951":33,"059956":[],"05996":37,"059977":[],"059979":[],"05998413e":[],"05999":10,"059992":[],"059993":33,"05999828":32,"06":[7,33,34,37,38,39,40],"060071":[],"060083":33,"060085":[],"060087":[],"06009333":38,"060149":[],"060160":[],"060163":6,"060171":[],"06019833":[],"06020264":[],"06020587":[7,34],"06021793":[],"060251":[],"060254":33,"060275":[],"060288":[],"060312":[],"06031928":[],"06034627":33,"06037137":33,"060384":[],"060435":6,"06043581":[7,34],"060453":[],"060470":[],"060471":[],"060475":[],"0604888":[],"060490":[],"060547":[],"06055507":[],"060565":[],"06057711":[],"060590":[],"06061276":[],"060629":[],"060631":[],"060652":[],"060678":[],"06068192":[],"06069047":33,"0606958":[],"060716":33,"06072":37,"060726":[],"060734":[],"060735":[],"060756":33,"06079614":[],"06080649":[],"060824":[],"06082419":[],"06082594":33,"06083072":[],"060870":[],"060872":33,"06088245":[],"060898":[],"060994":[],"061025":[],"061045":6,"06110658":[],"061118":[],"06112467":[],"061154":[],"061163":33,"06117338":[],"061178":12,"061187":[],"061284":[],"06128647":[],"061319":[],"061333":6,"06133572":[],"061359":33,"06136823":[],"061386":[],"0613875":[],"061388":[],"061394":[],"06139439":[],"061411":12,"061443":33,"061447":[],"061452":33,"06146873":[],"061482":6,"06151386":[],"06154484":[],"061545":6,"06155151":[],"061588":[],"06159762":[],"061614":33,"061637":[],"061656":[],"061706":[],"061707":[],"061724":[],"061772005077293704":[],"061775":[],"061779":6,"061793":[],"061795":[],"061848":[],"061862":[],"061898":[],"061907":6,"061915":6,"061921":[],"061956":[],"06199119":[],"06200174":6,"062004":[],"06200442":[],"06200455":[],"062016":33,"062053":[],"062058":[],"06208238634231949":[7,35],"06210358":[],"062161":[],"06216141":[],"062173":[],"0621855":[],"062199":[],"06220192":[],"062216":[],"062231":[],"062241":[],"06225028e":[],"062260":[],"062263":[],"062282":[],"06228382":[],"062292565":5,"062300":[],"062310":[],"062337":33,"062345":6,"062352":[],"062356":[],"062424":[],"062429":[],"062446":6,"062481":6,"062489":[],"062520":[],"0625462":[],"062568":[],"062599":33,"06260611":[],"062616":[],"062624":33,"06263176":[],"062637":[],"06265161":[],"06267057":[],"06267966":[],"062691":[],"062693":[],"062695":6,"062700":[],"062705":[],"062709":6,"062719":[],"06272254":32,"062733":12,"06276815e":[],"06277313":[],"062786":[],"062797":[],"062853":[],"062874":33,"06287939":[],"062932":[],"06295579":[],"06301273":[],"063017":[],"063019":33,"063048":[],"063061":33,"063064":[],"063080":33,"063082":[],"06310155":[],"063130":[],"063133":[],"063161":[],"063163":[],"063180":[],"06318287":[],"063197":[],"063206":[],"06320757":[],"063260":33,"063264":6,"063283":[],"063294":[],"063327":[],"063332":[],"063336":12,"063354":[],"06335853":[],"06336262":[],"063395":[],"063401":6,"063482":[],"063517":[],"063518":[],"063525":[],"063526":[],"063536":[],"063545":[],"063549":[],"063554":[],"063560":[],"06357887":[],"063621005600529":25,"063637":[],"063646":[],"063653":33,"063660":[],"063673":[],"063682":[],"063768":[],"0637712":[],"063781":[],"063786":[],"063791":[],"063802":[],"063809":[],"0638484":37,"06385189":[],"063862":[],"063883":6,"06389233":22,"063894":33,"063910":[],"06392846":[],"063933":[],"063954":[],"06395476":[],"063959":[],"063977":[],"063978":[],"063995":[],"064000":[],"064054":6,"064071":[],"064074":33,"064077":[],"06410638":[],"064107":[],"064164":[],"06416459":[],"064166":[],"064176":6,"064182":[],"064186":[],"06418809":37,"064189":[],"064202":[],"064218":[],"06423317":[],"064242":[],"064294":[],"06430789":[],"064311":6,"064314":6,"064322":[],"06432267":[],"064324":[],"06437266":[],"064373":[],"064382":[],"064427":[],"064428":[],"06443054":[],"06443215":[],"06444":10,"06444224":[],"064471":[],"064501":33,"064505":[],"064507":[],"064514":[],"06453579006728324":[7,35],"064540":[],"064547":[],"064573":[],"064604":[],"064606":33,"064609":33,"064615":[],"0646237":[],"064667":[],"064668":[],"064690":[],"064695":[],"064698":12,"064699":[6,33],"064703":[],"064728":[],"064739":[],"06474536":25,"064790":[],"064793":33,"0648":[],"064820":[],"064823":[],"064834":[],"06484181":[],"064845":[],"064873":[],"064878":[],"06491455":[],"06491736":[7,34],"064924":[],"064935":[],"06494075":[],"064954":[],"06496916e":[],"064978":[],"064980":[],"065001":[],"065007":[],"065028":[],"06504522":38,"06505375":[],"065062":[],"06506624":[],"065077":33,"06507762":22,"065096":[],"065108":[],"06512065":[],"065121":[],"065139":[],"065149":[],"065156":6,"065183":[],"065212":[],"065217":[],"065219":[],"065282":6,"065305":[],"065330":[],"065352":[],"065358":[],"06536971":[],"065375":[],"065378":33,"065387":12,"065390":33,"065410":33,"065469":6,"065472":[],"06547790180152355":[7,35],"06547790180152359":[7,35],"065482":[],"065508":[],"065556":[],"065565":12,"065591":[],"065616":[],"065619":[],"065650":[],"06565368":38,"065654":[],"065655":12,"065661":[],"065674":6,"065679":[],"065690":6,"065778":[],"065788":[],"0658141e":[],"065818":[],"065821":[],"065823":[],"065830":[],"065842":12,"065859":[],"065884":[],"06590458":[],"065949":[],"065960":[],"065963":[],"065989":[],"06600954":[],"066030":[],"06603447":[],"066044":[],"066054":[],"066059":[],"06606215":[],"066066":33,"066127":[],"066145":[],"066153":[],"066162":[],"066192":[],"066199":[],"066205":[],"066207":[],"06622369":[],"066241":[],"066256":[],"066286":[],"066330":[],"06634473":[],"066354":6,"066358":[],"0663596":[],"06636934":[],"06637":10,"066425":[],"066452":[],"066458":[],"066474":33,"066483":[],"066500":33,"0665111":[],"066520":6,"066524":[],"066545":[],"06659997":[],"066612":33,"06662069e":[],"06662675":[],"066633":[],"066647":33,"066653":[],"066680":6,"0666807":3,"066686":[],"06674611e":[],"066763":[],"066766":[],"06677444e":[],"066778":[],"066786":[],"066787":33,"066824":[],"066830":[],"066837":[12,33],"066838":[],"066841":[],"06684886":[],"06685171":[],"066854":[6,33],"066855":[],"066865":[],"066877":6,"06687898":[],"066896":6,"06690071":[],"066912":[],"066929":[],"066948":[],"06698472":[],"066988":[],"066997":[],"066999":33,"067000":[],"067003":[],"067032":[],"06703705":[],"06704902":[],"06706123":[],"067074":[],"067077":[],"067141":[],"067164":[],"067168":[],"067192":[],"067204":6,"067209":[],"067224":[],"06724062":6,"067249":6,"067260":[],"067270675787016":[],"0672706757870163":[37,38],"067273":[],"067288":[],"067313":[],"067314":12,"067367":[],"067374":[],"067380":[],"06739484":[],"067400":[],"067431":[],"067435":12,"0674567":[],"067492":[],"067507":12,"067511":[],"067515":[],"067523":[],"067539":[],"067597":6,"067623":[],"067637":33,"067662":[],"067671":[],"067707":33,"067719":[],"067731":[],"067758":[],"067764":[],"06777772":[],"067779":[],"067780":[],"06778421":[],"067793":[],"067809":[],"067819":[],"067843":[],"067851":12,"067883":[],"067917":[],"067922":[],"067932":[],"067935":6,"067940":12,"067943":[],"067948":[],"067967":[],"067974":[],"068008":[],"0680225":[],"068024":[],"06804775":[],"068057":12,"06807123":[],"06808421":[],"068088":[],"06810332":22,"068116":[],"068131":[],"068171":[],"06819354":29,"068206":[],"068215":[],"068222":[],"068226":12,"068229":[],"06826275":[],"068264":33,"068268":[],"068274":[],"068278":12,"06828347":[],"068289":[],"068307":33,"068324":[],"068341":[],"068353":[],"068360":[],"068364":[],"068369":[],"068394":[],"068406":[],"068408":[],"068429":[],"068440":[],"068441":33,"06844519414009445":[7,35],"06844519414009446":[7,35],"068446":[],"068479":[],"06849372":[],"068494":[],"068499":[],"0684990216069429":[],"068504":[],"068512":[],"06851496":[],"068533":[],"068539":[],"068587":[],"068592":[],"068597":[],"068607":[],"06862038":[],"068629":33,"068637":[],"068680":[],"068684":[],"068701":6,"068727":33,"068758":[],"068775":[],"06877599":[],"068797":6,"06881293":[],"068827":[],"068843":[],"068873":12,"068897":[],"068903":[],"068906":33,"068912":[],"06891861":[],"068924":[],"0689303":[],"068931":6,"068939":6,"068967":[],"068977":[],"068984":[],"068985":[],"06901081":[],"069028":33,"069045":[],"06908518":[],"069129":[],"06913112":[],"069132":[],"069136":12,"06915429":[],"069179":[],"06919343":33,"069203":[],"06926151969326948":[],"06926192":[],"069270":[],"069293":[],"06929519":[],"069317":[],"069320":[],"069322":[],"069388":[],"069391":[],"069399":[],"06942115e":[],"0694226":[],"069427":[],"069439":[],"069452":33,"069457":6,"069484":[],"069486":[],"069492":[],"069507":6,"069509":[],"069526":[],"06957317":[],"069584":33,"06958788":[],"069592":12,"069597":6,"069606":6,"069636":[],"069637":[],"06967433":[],"069675":[],"069676":[],"069680":[],"069686":[],"06970419":[],"069711":[],"06971862":[],"069754":[],"069761":[],"069784":6,"069785":6,"069802":[],"069811":[],"06984502":[],"069866":[],"069873":[],"069885":[],"069907":[],"069916":12,"0699274":[],"069948":6,"069964":[],"069970":[],"069978":[],"069979":[],"07":[7,33,34,37,39,40],"070010":[],"07001431":[],"070017":[],"070021":[],"070024":[],"070027":[],"070043":33,"070063":[],"07006502":[],"070087":6,"070095":[],"070098":[],"070135":[],"070146":33,"070150":[],"07016":10,"07017":10,"070172":12,"070178":[],"070181":[],"070183":[],"070190":[],"070213":33,"070248":12,"070258":[],"070273":6,"070274":[],"070275":[],"07027791":[],"070318":[],"07036197":33,"070367":12,"07037089":25,"070380":[],"07039":10,"070414":[],"07043187":[],"070432":[],"070441":[],"070457":33,"070460":[],"070462":[],"070480":[],"070485":[],"070514":[],"070527":[],"070529":12,"070531":[],"070547":[],"070552":[],"070556":[],"070558":[],"070581":6,"070582":33,"07058967":[],"070595":12,"070600":[],"070605":[],"070622":[],"07062318":[7,34],"070625":[],"070630":[],"070647":12,"070655":[],"070657":[],"070667":[],"070696":[],"070712":[],"070730":[],"070737":33,"070747":[],"070769":[],"070772":[],"070774":[],"070780":[],"070805":[],"07082844":33,"07084394":[],"070855":[],"070859":[],"070886":[],"070889":[12,33],"070891":[],"070898":[],"070916":[],"070918":[],"070924":[],"070950":[],"070975":[],"070980":[],"070986":33,"070991":[],"071001":[],"071012":[],"07101262":[],"071018":[],"071027":[],"071049":[],"071055":[],"071058":12,"07106376":38,"07106891":[],"071070":12,"071087":[],"071092":[],"071103":[],"071143":[],"07114326":[],"07115":10,"071173":[],"071186":[],"071191":[],"071197":[],"071208":[],"071217":[],"07123177":[],"071252":[],"071253":[],"071279":[],"0712848":[],"071291":[],"071294":[],"0713":[1,32],"071314":[],"071328":[],"07134161":[],"071356":[],"071379":[],"071406":[],"071427":[],"071433":[],"071436":[],"071437":[],"071449":[],"07145103":12,"071459":[],"071471":[],"07147868":[],"07147935":[],"071530":[],"07153889":[],"071560":[],"071572":[],"071575":[],"071578":[],"071580":[],"071587":12,"071588":[],"07160048164233102":[7,35],"07160048164233104":[7,35],"071608":[],"071638":[],"071643":[],"071646":[],"071655":[],"071656":[],"071666":[],"07166646":[],"071680":[],"071687":[],"071704":12,"07170448":[],"07172852":38,"07172932":[],"07173249":[39,40],"071758":[],"071759":[],"071774":6,"071789":[],"071803":[],"071831":[],"0718316":37,"071868":[],"07187601":[],"071919":6,"071921":[],"0719267":[],"071935":[],"071937":[],"071954":[],"07195418":[],"071962":[],"071969":[],"071976":[],"071978":[],"072000":33,"072006":12,"072011":6,"072013":12,"072015":[],"072028":[],"072031":[],"072037":[],"07204042":[],"072083":[],"072085":[],"072092":[],"072101":[],"072111":33,"072132":33,"072140":[],"072147":[],"072148":[],"072171":[],"072173":[],"07218473624441492":[],"072199":[],"072200":[],"072209":[],"072220":[],"072221":[],"072223":[],"072226":6,"072234":[],"072238":[],"072240":[],"072241":[],"072262":[],"072263":6,"072286":[],"072290639572326":[],"07229064":[],"072303":[],"072304":[],"072329":[],"072344":[],"07234492":[],"07235641":[],"072380":[],"072388":[],"072404":[],"072406":[],"07241855":[],"072476":33,"072500":[],"072518":[],"072533":12,"07253999":32,"072551":[],"072577":6,"0725872":[],"07258982":[],"072590":[],"072591":[],"07259548":33,"072615":12,"072621":33,"07263033":[],"072638":[],"072653":6,"072684":[],"072688":[],"072689":[],"072693":[],"072694":[],"072697":[],"072721":[],"072728":[],"072733":[],"072752":[],"072773":[],"072787":[],"072817":[],"072820":[],"072824":[],"072826":[],"072835":[],"072843":[],"072849":[],"07285":4,"072859":6,"072863":[],"07286621":[],"072885":[],"072888":[],"072902":[],"07291598":[],"07292363":[],"072932":[],"072946":[],"072947":[],"072961":[],"072967":6,"072973":[],"07298295":[],"072997":[],"073021":[],"073048":[],"073050":12,"073071":[],"073102":[],"073109":12,"073114":[],"073115":[],"073144":6,"073182":6,"073198":[],"073253":12,"073258":[],"073260":[],"073262":[],"073270":[],"073288":[],"073320":12,"07332248":[],"073339":[],"073340":12,"073344":[],"073349":[],"07335575":[],"073392":[],"073405":[],"073408":[],"073411":[],"073415":[],"073416":[],"07341744e":[],"073418":[],"073422":[],"073423":[],"073442":12,"073483":[],"073499":[],"073514":[],"073525":6,"073528":[],"073576975500551":[],"073598":[12,33],"073612":[],"073644":33,"073647":[],"073662":[],"073667":[],"073672":[],"073683":12,"073684":[],"073685":[],"07372439e":[],"07373479":[],"07376699":[],"07378525":[],"073791":[],"073794":[],"073800":[],"073841":[],"073858":[],"073863":6,"073866":[],"073882":[],"073883":[],"073899":[],"073911":[],"073923":[],"07397193":[],"073985":[],"073993":[],"074009":[],"074048":[],"07405556":[],"074067":[12,33],"074073":[],"074084":33,"074087":[],"074112":[],"074114":[],"074115":[],"07412144024494684":[],"074130":[],"074134":[],"07415258":[],"074193":[],"074210":33,"07421084":6,"074212":[],"074216":[],"074221":[],"074228":[],"07424587":[],"074265":[],"074283":[],"07429213":[],"074296":[],"07429886e":[],"074301":[],"074314":[],"074323":33,"074334":[],"074358":[],"07435812":[],"074360":[],"074368":[],"074372":[],"074374":[],"074417":[],"074454":[],"074460":[],"074482":[],"0744881":[],"07449122":[],"074506":[],"074524":[],"074531":[],"074561":[],"07456491":6,"07459764":[],"074622":[],"07468007":[],"074688":[],"074693":[],"07469645":[],"074713":[],"07471361":[],"074738":12,"074776":[],"074787":[],"074802":[],"074807":[],"074824":[],"074830":[],"074851":12,"074859":[],"074866":6,"07487128":[],"074882":[],"074892":[],"07489511":[],"07490892":[7,34],"074911":6,"074928":[],"074935":[],"074971":[],"074976":[],"075021":[],"075028":[],"075048":[],"075052":[],"075058":6,"075070":[],"075083":12,"075086":[],"07510564":[],"07510754":[],"07510785":25,"075109":[],"075114":[],"075129":[],"075136":[],"075155":[],"075157":[],"07517385":[],"075176":[],"075177":[],"075198":[],"075238":6,"075239":[],"075241":[],"075274":[],"075281":[],"075284":12,"075289":[],"075293":[],"075298":[],"075302":[],"075310":12,"07531349":[],"075320":[],"075334":[],"075356":[],"075363":[],"07537801":[],"075385":[],"075392":[],"075396":[],"075409":[],"075416":[],"075423":[],"075429":[],"075446":[],"075463":[],"075472":[],"075478":[],"07549007":[],"075496":[],"075498":[],"075501":[],"075513":[],"075522":[],"075525":[],"075542":[],"075544":[],"07555537":[],"075557":[],"075562":[],"075563":[],"075576":[],"075595":[],"075602":[],"075609":[],"075613":12,"075615":[],"075634":[],"075639":[],"075658":[],"075671":[],"075677":[],"075689":[],"075711":[],"075712":[],"075753":[],"075761":6,"075771":[],"075794":[],"075809":[],"075816":33,"075822":[],"075823":[],"075830":[],"075834":[],"075838":[],"075842":[],"075864":[],"075868":[],"075872":[],"075876":[],"075878":[],"075880":[],"075883":[],"075885":[],"075889":33,"075897":[],"075909":[],"0759216":[],"075956":[],"07597049":[],"075974":[],"075978":[],"075988":[],"07600369":[],"0760318":[],"076057":12,"076065":[],"076066":[],"076071":[],"076079":12,"076082":[],"076091":[],"076101":[],"076108":[],"07612762":[],"076138":[],"076150":33,"076154":12,"076157":[],"076161":[],"076163":6,"076167":[],"07617734331359052":[],"076189":[],"076203":[],"076212":12,"076225":[],"076232":[],"076244":[],"076248":6,"076259":[],"07630326327869198":[],"076306":12,"076321":[],"076333":[],"076334":[],"076340":[],"076351":[],"076361":[],"076378":[],"076389":[],"076393":[],"076396":[],"076411":[],"076418":[],"07641937":35,"076434":[],"076435":[],"076436":[],"076448":[],"076459":[],"076473":[],"076479":[],"0764924":[7,34],"076508":[],"076542":[],"076543":[],"076547":12,"076555":[],"0765856870687196":[],"07658569":[],"076586":[],"076616":[],"076618":[],"076638":[],"076647":[],"076648":[],"076654":12,"07665857":[],"076665":6,"076679":[],"07668538":[],"076693":[],"076776":[],"076776262573027":[],"076777":[],"07678":10,"076791":12,"076803":[],"076829":[],"076833":[],"076857":[],"076859":[],"076876":[],"0768805855280187":[],"076894":12,"07692307692307693":10,"076934":[],"076938":[12,33],"076947":[],"076951":[],"076968":[],"07698164":[],"077035":[],"07704298":[],"07705051":[],"077053":[],"077060":12,"077061":[],"077091":[],"07712197e":[],"07712506":[],"07712724":[],"077131":[],"077139":6,"077141":[],"077168":33,"077180":[],"077184":12,"077226":[],"077248":[],"077254":[],"07726368":[],"077280":[],"077292":[],"077301":[],"077304":[],"077310":[],"077329":[],"077337":[],"077341":[],"0773554":[],"077358":[],"077382":[],"077391":[],"07740964":[],"077411":[],"077414":[],"077431":[],"07745579":[],"077460":33,"077476":[],"0774777":[],"077485":[],"077486":[],"077488":[],"077490":[],"077510":[],"077516":[],"077523":[],"077529":[],"077534":[],"077539":[],"077541":[],"077550":[],"077553":[],"077566":[],"077567":[],"077601":[],"077608":[],"077613":[],"07763918":38,"077641":[],"077643":[],"077653":[],"0776532":[],"077672":[],"077683":[],"07768993":[],"077711":[],"077736":[],"077745":[],"07776381":[],"077764":[],"07777777777777778":[2,40,41],"077782":[],"077813":[],"077837":6,"077839":[],"077857":[],"077899":6,"077906":[],"077941":[],"077953":[],"077965":[],"077975":[],"07797763":[],"078000":[],"078035":[],"078039":12,"078044":[],"078051":12,"078067":[],"078071":[],"07808426989543932":[],"078110":33,"078118":[],"078123":[],"078131":[],"078146":[],"078170":[],"07820":10,"07820074":[],"078210":[],"078238":[],"078267":12,"078268":[],"078282":[],"078284":[],"078313":[],"078324":[],"078336":33,"078359":[],"078363":[],"07836545":38,"07836997022107646":[],"078386":[],"078409":[],"078447":[],"078454":[],"078474":6,"078540":33,"078541":[],"078544":[],"078548":12,"078557":[],"078573":[],"07858042":[],"078588":[],"078589":6,"078636":[],"07864":10,"07865450504129884":12,"078661":[],"078684":[],"078688":[],"078708":[],"07871":10,"078711":[],"078731":[],"078741":[],"078751":[],"078753":[],"078782":12,"078825":12,"078855":[],"078867":12,"078889":[],"078898":[],"07890044":[],"078945":[],"078953":[],"078958":[],"078974":33,"078981":[],"078986":33,"079000":12,"079002":[],"079013":[],"079017":[],"079027":[],"079030":[],"079074":[],"079080":[],"079089":[],"079107":[],"07911268":[],"079114":[],"079138":12,"079161":[],"079174":[],"079184":12,"07919041":[],"079202":[],"079218":[],"079240":[],"079334":[],"079347":[],"079369":12,"079391":[],"07939331":[],"079420":[],"079429":12,"07944154":[25,32],"079443":6,"079444":[],"079487":6,"07950356694388497":[],"079510":[],"079518":[],"079535":[],"079562":[],"079566":[],"079638":[],"079670":[],"0796891867672603":[7,35],"07969087":[],"07971187802560528":[],"079739":[],"079745":[],"079747":[],"07974741":[],"079783":[],"07978850553011713":[],"079855":[],"079874":[],"079887":[],"079903":[],"079946":33,"079947":[],"079953":[],"079959":[],"079967":[],"07996807":[],"079976":[],"079977":[],"079982":[],"079987":12,"07998756":[],"079989":[],"079998":[],"07it":7,"08":[10,29,33,37,38,39,40],"080032":[],"080082":[],"080094":[],"08009874":[],"080103":[],"08010965":[],"080110":[],"080139":[],"080150":[],"080171":[],"08019747":[],"080199":[],"080205":[],"080220":[],"080222":[],"08023512":37,"080237":[],"080258":[],"080268":[],"080271":[],"080286":[],"080287":[],"080301":[],"080316":[],"08032505":[],"080330":[],"080345":[],"080353":[],"08036056":6,"080363":12,"080380":12,"080403":[],"08043851":6,"080448":[],"080453":[],"080474":[],"080493":[],"080535":12,"080549":[],"080555":[],"080559":[],"080576":[],"080585":[],"080605":[],"080608":[],"080616":33,"080630":[],"080631":[],"080651":[],"080657":[],"080658":[],"08068207":32,"080702":[],"080704":[],"080706":[],"080735":12,"080747":[],"080764":33,"080766":[],"080793":[],"080832":[],"080849":[],"080875":[],"080903":[],"08090942":25,"080947":[],"080950":[],"080987":[],"080997":[],"081031":[],"081038":[],"08109405":[],"081103":[],"081107":12,"081151":[],"081194":[],"081195":[],"081217":[],"081221":[],"08122943":[],"081232":6,"081235":[],"081252":[],"081276":33,"081283":12,"081296":[],"08131003":7,"0813156":[],"081324":[],"081342":[],"081405":[],"081410":[],"081411":[],"081431":[],"081450":[],"081457":[],"081494":[],"081523":12,"081527":[],"081530":[],"08153923":[],"08154221680664263":6,"081544":[],"08156108":[7,34],"08159479":[],"081637":[],"08165104":33,"081652":[],"081655":33,"081681":[],"081686":[],"081702":12,"081728":[],"081730":[],"081796":[],"08179906":[],"081811":[],"081823":[],"081835":[],"081836":[],"081845":[],"08187062":[],"081871":[],"08189696":[],"08191754":[],"081926":[],"081931":[],"081955":[],"081978":[],"082002":[],"082009":[],"0820185":[],"082043":[],"082057":[],"082085":[],"082115":[],"082121":[],"082126":[],"082149":[],"082151":12,"08216308":[],"082170":[],"082173":[],"082174":[],"082180":[],"082230":[],"082240":[],"082249":[],"082256":[],"082264":[],"08227057":[],"082274":12,"08227668":[],"082303":[],"082307":[],"082316":[],"082318":[],"08233627":[],"082342":[],"082355":12,"082361":[],"082412":12,"082414":12,"08244283":[],"082464":[],"082478":[],"082482":[],"08251519":[7,34],"082518":[],"082519":[],"082533":[],"082537":[],"082553":[],"082596":[],"08259753":[],"082598":12,"082653":33,"08266769":[],"082670":[],"082742":[],"082764":[],"082784":[],"082785":[],"08280012":[],"082803":[],"082857":[],"082878":[],"082940":[],"082957":[],"082971":[],"08299273e":[7,34],"08300526":32,"083016":[],"083034":[],"083061":[],"083092":[],"083119":[],"08312573":29,"083128":[],"08313374":[],"0831367":[],"083138":[],"083179":[],"083182":[],"08318298e":[2,40,41],"08319383":[],"0832":[],"083203":[],"083248":[],"083257":[],"083259":[],"083260":[],"083268":[],"083288":[],"08332537":[],"083331":6,"08333333333333333":[2,10,40,41],"083342":[],"08336233266":5,"083405":[],"083446":[],"083459":[],"083465":[],"083471":[],"083479":[],"083499":[],"083501":[],"083527":33,"083547":[],"083548":[],"083564":[],"083596":[],"083645":[],"08370977":[],"083714":[],"083734":[],"083765":[],"08376632":[7,33,34],"083766322923899":[7,33,34],"0837663229239043":[7,33,34],"083793":[],"083801":[],"083864":[],"083900":[],"083930":[],"083968":12,"083970":[],"083982":12,"08398399377155011":[],"084018":[],"084042":[],"084076":33,"084091":[],"084112":[],"084122":[],"084127":[],"08418548":[],"08418956":[],"084195":[],"08419839":[],"084209":[],"084218":[],"08421808":[],"084222":[],"084255":[],"08426840630693411":[7,35],"08426840630693413":[7,35],"084289":[],"084312":[],"084319":[],"084340":[],"084346":12,"084358":[],"084376":[],"084406":[],"084423":[],"084434":[],"084440":[],"084486":12,"084514":[],"084531":[],"084535":12,"084548":[],"08455":10,"084550":[],"084568":[],"08458591":[],"08460718":[],"084619":[],"084637":[],"0846409":[],"084674":[],"084687":12,"084694":[],"084695":[],"084702":[],"084712":[],"08472879":[],"08474":10,"084801":[],"084809":33,"08482302":38,"084847":[],"084858":[],"08487999":[],"084910":[],"084915":[],"084934":[],"084963":[],"084967":[],"084976":[],"084987":[],"084996":[],"085079":[],"085099":[],"085120":[],"085139":[],"085141":12,"085145":[],"085147":[],"085150":[],"085155":[],"08516604":33,"085197":[],"085201":[],"085212":[],"085222":[],"085268":[],"085294":[],"085295":[],"085353":[],"08536248571780691":[],"085369":[],"085378":[],"085390":[],"08539155":37,"085392":[],"08539583":[],"08540984":[],"08543112":[],"085436":[],"085455":[],"08546261e":[],"085463":[],"085486":[],"08551306":[7,34],"085518":[],"085548":[],"085573":[],"085592":[],"08559230156092426":[],"08561649":38,"085655":[],"085659":[],"085690":[],"08569593":[],"085712":[],"085716":[],"085722":[],"08572513":[],"085728":[],"085747":[],"085762":12,"085764":[],"08576932":7,"085790":[],"085807":[],"085819":[],"085820":[],"085833":[],"085853":[],"08587209":[],"085902":[],"085918":[],"08593216":[7,34],"085935":[],"085937":[],"085970":[],"085973":[],"085980":[],"085988":[],"085996":[],"08601328":[],"086021":[],"086026":[],"086058":[],"086068":[],"08607283":[],"08609047":[],"086092":[],"086095":[],"086096":[],"086101":[],"086107":[],"08611111111111111":[2,40,41],"086125":[],"086130":[],"086212":[],"086218":[],"08622478":[],"086227":[],"086230":[],"086239":[],"08624193":[],"086250":[],"086256":[],"086286":[],"086287":[],"086289":[],"086306":[],"086318":[],"086326":12,"086338":[],"086339":[],"086346":[],"086364":[],"0864086":[],"086410":12,"086428":[],"08642966":[],"086471":[],"086483":[],"086489":[],"086508":[],"086548":12,"086576":[],"08660891":[],"086611":[],"086653":[],"086655":[],"08667591":[],"086694":[],"086712":[],"086730":[],"08675226":[],"086797":[],"086799":[],"086808":[],"086809":[],"08683446":[],"08687303":37,"08690":10,"08690184845323":[],"08691139":37,"086913":[],"086970":[],"08698097":32,"08699295e":[],"08700526":[],"087020":[],"08702578":[],"08705631913312815":[],"087102":[],"087135":[],"087210":[],"087255":[],"08728977":[],"087297":[],"087327":[],"087340":12,"08734023":[],"087369":[],"087424":[],"08742496":[],"087429":[],"08747795":[],"0875061":[],"087560":[],"08758":10,"08758013":[],"087585":[],"087621":12,"087675":[],"0877492":[],"087772":[],"08777858":[],"087789":[],"08779325":[],"087828":[],"087864":[],"087871":[],"087876":[],"087909":[],"08793554992813543":[],"087946":[],"087988":[],"08799729":[],"088023":[],"088029":[],"088030":[],"088038":[],"088058":[],"08809115":[],"088102":[],"088127":[],"08816816":[],"0881981":6,"088199":[],"088242":[],"08824304e":[],"08826182458028335":[],"08826256":[],"08833634":[],"088375":[],"088390":[],"08839544":[],"088425":[],"088441":[],"088455":[],"088481":[],"08854388":[],"088574":[],"08858739":[],"088599":[],"08860502":[],"088609":[],"088636":[],"088642":[],"088650":[],"088651":[],"088680":[],"088694":[],"08871404":6,"088726":[],"08881497884574564":33,"08888888888888889":[2,40,41],"088926":[],"08896995":6,"088970":6,"089":[],"089007":[],"08902":10,"089052":[],"089065":[],"089086":[],"089288":12,"089311":[],"08931169286872433":[],"08933713":[],"089338":[],"08935132374099551":[],"089354":[],"089376":[],"089377":[],"089386":[],"0894015":[],"089404":[],"089408":[],"089417":[],"089421":[],"08946029":[],"08947292":[],"08949703e":[],"0895043":[],"0895387":33,"089539":33,"089548":[],"08956994":[],"089576":[],"089578":[],"089600":[],"08961438":[],"089619":[],"089626":[],"089639":[],"089653":[],"089668":[],"089690":[],"089704":[],"089708":[],"089714":[],"089718":[],"089726":[],"089730":[],"089735":[],"089738":[],"089748":[],"089755":[],"089760":[],"089761":[],"089763":[],"089769":[],"08977751":[],"089783":[],"089785":[],"089786":[],"089790":[],"089795":[],"089800":[],"089826":[],"08984905":[],"089858":[],"089859":[],"089872":[],"089940":[],"089949":[],"08996":10,"089996":[],"09":[2,33,40,41],"090002":[],"09000319":[],"090033":[],"090038":[],"090075":[],"09007789":[],"090081":[],"090114":[],"090118":[],"09014451":[],"090154":[],"09016979":[],"090174":[],"090204":[],"09024373":[],"09028702":[],"090302":[],"090319":[],"09033649":[],"090387":[],"090429":[],"090437":12,"090439":[],"090440":[],"090468":[],"090475":12,"090476":[],"09048717285916912":[],"090492":[],"090561":[],"090565":[],"090624":[],"090626":[],"09065603":[],"09070064":[],"090717":[],"090721":[],"09078672":[],"0907958":[],"090804":[],"090935":[],"090985":[],"091":[],"091015":[],"091034":[],"091068":[],"09109617":[],"091159":[],"09120503":[],"09129027":33,"09135059e":[],"091362":[],"091365":[],"091388":[],"09156055":[],"09163947":7,"09165112":[],"091653":[],"09166666666666666":[2,40,41],"091674":[],"0917":10,"09171886":[],"091731":[],"091732":[],"091871":[],"09189665":[],"09192497":[],"092013":[],"092062":[],"092064":[],"09206825":[],"09209032":[],"09211239":[],"09219464":[],"09224244":[],"092252":[],"09225273":[],"092347":[],"092374":[],"092401":[],"092421":[],"09242193":[],"09246293455466892":[],"092486":[],"09251":10,"092530":[],"092533":[],"092607":[],"092608577084595":[],"09267158":38,"092689":[],"092703":[],"092746":[],"09280524":[],"09291556244521161":[],"09291806":[],"092956":[],"09296935":[],"092982":[],"09299349":[],"09300398":[],"093004":[],"093011":[],"09303501":32,"0930438":[],"093045":[],"093094":[],"09310479":[],"093115":[],"093226":[],"093288":[],"09331672":[],"09346106":[],"09347829":[],"09353846":38,"093554":[],"093572":[],"09360113":[],"09368":37,"093680":[],"09371427":[],"09374315e":22,"0938492":[],"09397626":[],"09398153":[],"094027":[],"094044":[],"0940821989627646e":[7,34],"0940821989652176e":[7,34],"094099":[],"094189":[],"094243":[],"094254":[],"094304":12,"094362":[],"09440493433371173":[],"094439":[],"09444444444444444":[2,40,41],"09452695":[],"094527":[],"094630":[],"09469095":[],"094697":[],"094797":[],"094849":[],"094857":[],"094862":[],"09487358":[],"094877":[],"09491833":[],"094941":[],"09497303":[],"095037":[],"095090":[],"095106":[],"095125":[],"09512539":[],"09515455":[],"095178":[],"095278":[],"09529067":[],"095358":[],"095395":[],"095425":[],"09543871010617433":[],"095465":[],"095626":[],"095710":[],"09572047":[],"095803":[],"095805":[],"09584387":[],"09590269e":[],"095915":[],"09604976":[],"096077":[],"096078":[],"09609807":6,"09610694":[],"096203":[],"09621323":[],"09622593":[],"096270":[],"09627032":[],"096287":[],"096365":[],"09637807":[],"09649893":38,"096511363983559":[],"09651792":[],"096570":[],"096630":[],"09664324":[],"09666342":[],"09667969":[],"096690":[],"096700":[],"096731":[],"096811":[],"09695222":[],"097017":[],"09702948":[],"097060":[],"097115":[],"097365":[],"097380":[],"0973900819831327":[],"09739637":[],"09740532e":[],"0974061":[],"0974119":[],"09744":10,"09755017":[],"09760893":[],"0976598":[],"097667":[],"09780":10,"097873":[],"09790216282081063":[],"09791":10,"09808149":[],"09808284":[],"09808622":[],"09812323e":[],"098144":[],"09821823":[],"09829902":[],"0983565":[],"098371":[],"098414":[],"098431":[],"09846933":[],"09848937":[],"09851247":[],"09861229":[25,32],"09862097":[],"09880256":[],"098803":[],"09895512":[],"09900258":[],"09903804":9,"09905657":[],"09913654":[],"09919198949275869":[7,35],"09919725":25,"09927808":[],"09928714":[],"09930135":[],"09931721":[],"09941275e":[],"099494":[],"09951287404314545":[2,40,41],"099588":[],"099622":[],"09964573":[],"0n":[1,32],"0s":[40,41],"0x0006":[4,5,15],"0x10746eeb0":22,"0x10b911550":[],"0x10b9828b0":[],"0x10cc52cd0":[],"0x10fcfaeb0":[],"0x1174af160":[],"0x11753f700":[],"0x117715eb0":[],"0x117893340":[],"0x117c17be0":[],"0x117edb760":[],"0x11892e7f0":[],"0x118995f70":[],"0x11975e550":[],"0x11a8ace50":[],"0x11af6aa90":[],"0x11b160a60":[],"0x11ba4a5b0":38,"0x11baaec10":38,"0x11bc3a1f0":[],"0x11bcee130":[],"0x11bdd4640":[],"0x11c3bd700":[],"0x11f13fb20":[],"0x11fa3db20":22,"0x11fc31790":[],"0x11fd7c160":[],"0x11ffa0ee0":[],"0x120cc1b20":[],"0x121cf68e0":[],"0x124252b80":[],"0x124b30190":[],"0x124b96d60":[],"0x125dbb640":[],"0x126431130":[],"0x1273ffac0":[],"0x1279bd460":[],"0x127d8ceb0":[],"0x12aaa0b20":[],"0x12b14fa90":[],"0x12ccea880":[],"0x12e13a850":[],"0x130ca6a60":37,"0x13134a100":37,"0x1322a9a00":[],"0x135f8b1f0":[],"0x1360ef5e0":[],"0x145270160":[],"0x1452d6eb0":[],"0x1522ef130":[],"1":[0,2,3,4,5,6,7,8,9,10,11,12,13,14,15,20,22,25,27,28,29,30,31,34,35,36,37,38],"10":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,22,23,25,26,28,29,30,32,33,34,35,36,37,38,39,40,41],"100":[1,2,3,4,5,6,7,9,10,11,12,14,15,16,17,18,22,23,25,29,30,32,33,34,35,36,37,38,39,40,41],"1000":[0,1,2,3,5,6,9,12,14,15,22,23,24,29,32,33,36,37,38,40,41],"10000":[3,6,7,11,12,14,29,34,35,41],"100000":9,"10001":11,"100083":[],"10008326":[],"1000911":[],"1001":[10,29],"1001408041761458":[],"1002":29,"100209":[],"10029389":37,"1003":29,"10030":10,"10036694":[],"10041812":34,"1004727":[],"10047943":[],"1005":29,"100523":[],"10068448":[],"1007":35,"1007711427354898":[7,35],"100816":[],"1009":29,"10100211":[],"101008878523571":[],"10107914":[],"1011":29,"101150":[],"10115017":[],"10116149":[],"10124634":38,"101263":[],"101288":[],"101292":[],"1013":29,"1013904243":29,"10141413e":[7,34],"10145028":[],"1015":29,"1015491":37,"10158754":[],"1015am":32,"10175169":[],"10179509":25,"10202674":29,"102147":[],"10219344":38,"102209":[],"1023":29,"10230":10,"1024":4,"10240571":[],"10242458":[],"1026":29,"10261093":[],"1027":29,"10274274e":[],"10276112":[],"103":[2,40,41],"1030":29,"10321406":[],"10325557":[],"10331028":[],"1033455":[],"10340":10,"10340733":[],"10348862":[],"10365224":[],"10369090e":[],"1037":29,"10371767":32,"10378326e":[2,40,41],"1038":29,"10387439":[],"10391807":[7,34],"10398646080125032":[7,35],"10398646080125035":[7,35],"1040":29,"10402095":[],"10405456":12,"10407019":[],"10425587":[],"10426556":[],"10426802":[],"10430":10,"104411":33,"10453899":[],"10454793727627841":[],"10465415":[],"1047":29,"10470031":[],"10478375":[],"10499357":37,"105":[3,34],"1050951":[],"10517643":[],"10520":10,"10520393":32,"10537416":[],"10539984":[],"105400":[],"10541723644166373":[],"10542736e":38,"10555555555555556":[2,40,41],"10556487":[],"10560458":[],"10560535":[],"10572":35,"10587277":[],"10589577":6,"10595991":[],"10598715e":[39,40],"106":3,"10604579e":[],"106095":[12,33],"10637914":[],"1064601390492845":[],"10646014":[],"10687432e":[],"10688434":[],"10695771":[],"10719554":[],"10719859":[],"10738656":[],"107387":[],"10747579":[],"107476":[],"1074792":[],"10754764":[],"10786423":[],"1079":35,"10790125813226321":33,"108":[3,7,10],"10801706":[],"1080176":[],"10837392":[],"10851177":[],"10854118":[],"10888197":[],"10898643":[],"10907699":[],"10908858":[],"10909182e":[],"10913":7,"10923006e":[],"10931453":[7,34],"10936832":[],"10950876":[],"10960":10,"10987614":[],"10991429":[],"10994958":[],"10995472e":[39,40],"10e":[40,41],"10th":10,"10x":[1,32],"11":[0,1,3,6,7,8,9,10,11,12,13,14,16,17,22,25,26,27,28,29,30,31,32,33,34,35,36,37,38,40,41],"110":3,"1100":29,"11005335":[],"11006373":[],"1101":29,"11022302e":[6,38,39],"11042239":[],"11048466":[],"110485":[],"11052859":[],"11061452":38,"11080997912276":[],"11081083":[],"1108407":[],"111":[0,2,3,8,13,36,38,39,40,41],"11100":10,"11113486":[],"111135":[],"1112416":[],"11129947":[],"11131389":[],"11147544":[],"11163239":[],"11164003":[],"11164926":38,"11171148":[],"1118933":[],"112":3,"11202337":35,"11204514":[39,40],"11214365":[],"11232457":[],"112383":[12,33],"1124":10,"11245282":[],"112619":[],"11261935":[],"11265643e":[],"11277166":[],"112772":[],"113":3,"1130676":[],"11315542":[],"11315771":[],"11320323":[],"11320726":[],"11331834":[],"11361003":[],"1137048":[],"11371176e":[],"11382896":34,"11388017":[],"11388888888888889":[2,40,41],"11390":10,"114":3,"11406297261341168":[],"11416086":[],"11433693":[],"114384":[],"11438414":[],"11439407":[],"11449851":[],"11449867":[],"114499":[],"114550":35,"11457699":[],"11462415":6,"114795":6,"11479519":6,"11482289e":[7,34],"1148541":[],"11487977":[],"11490318":[],"11498192":[],"11501291":[],"11507992e":[2,40,41],"11526665":[],"115356":[],"11535632":[],"11543883":[],"11547777218872497":[7,35],"11548661":[],"11556912":22,"11570114e":[],"11571507":[],"115822":7,"1158848":[],"11588954":[],"11590":10,"11596287":25,"11596834":[],"11597515":[],"11620886":[],"11634286":29,"11634286035477":29,"11660":10,"11666666666666667":[2,40,41],"11675128":[],"11683558":[],"1169204":[],"1169242":[],"11692966":[],"11694":26,"11694197":[],"117":9,"11715109e":[],"11725761":[],"11744554e":[7,34],"11747908":[],"11762444":[],"11780":10,"11797382":[],"118":[1,8,36],"11822173":[],"11826404":[],"11830757":34,"118318":33,"1184":35,"11840":10,"11844773":[],"11861803":[],"11868033":[],"11877604":[],"11890":10,"11891833e":[],"119":[1,8,36],"11922031":[],"11928633e":38,"11930916":[],"11935351":[],"11969984":[],"1198891":[],"11991488":32,"11992194":[],"119922":[],"119936":[],"12":[1,2,3,4,5,6,7,8,9,10,12,13,22,23,25,27,29,31,32,33,35,36,37,38,40,41],"120":[1,4,8,36],"1203":10,"12031117":[],"1203284":9,"12040799":[],"120435":[],"12043521":[],"12044974":33,"120450":33,"12068251":[],"12079371":[],"12081773":37,"12084953":38,"12085937e":[],"12099567":[],"121":[9,10,11],"12123488":[],"12124104":[],"121292":[],"12135727":[],"12143852":[],"12152902":[],"12155483":[],"1215pm":[30,32],"12166797":[],"12182967":[7,34],"12183393":[],"12183785":[],"12190193":[],"121902":[],"12198":38,"122":[9,10,11],"12204312":[],"12206013":[],"12206812":[],"12208052":[],"12210357":[],"12220088":[],"12220276":[],"12222222222222222":[2,40,41],"12235593":[],"12242545":[],"12279959":[],"12282958127703":6,"123":[],"12318726e":[7,34],"12329646":[],"12330033":33,"12333649":[7,34],"12363332":[],"123711":7,"12380":10,"12382443":[],"123928000581825":[],"12393064":[],"12394921":32,"124":[1,32],"12400":10,"1242":9,"124228866018077":[],"12425815":[],"124325":[],"12443871":[],"12461758":[],"12462963":22,"12466453":[],"12498355":[],"1251681":[39,40],"12534765":[],"12536737":37,"12544009":[],"12544297":[],"12552073e":[7,34],"125757":[],"12591227":33,"126":10,"12600095":[],"1261":10,"12618549":6,"1265":10,"12667402":[],"12675974":25,"12686507":[],"127":5,"12702643":[],"1271":7,"12733488":[],"12733892":[],"12777777777777777":[2,40,41],"12790":10,"128":[4,5,14,37,38],"1280526":[],"12829994e":[],"12831838":[],"128393685824704":[],"12840176":[],"1284100258440377":29,"12846660e":[],"12871842":33,"12873424":[],"12893778":[],"129":[],"129012":7,"12903093":[],"12910258":[],"12922837":[],"12936913":29,"12952339":34,"12957454":[],"1297":10,"1298":10,"129963":35,"12it":[],"12pm":[30,32],"13":[1,3,6,7,8,10,12,13,19,22,25,27,29,32,33,35,36,37,38,41],"130":10,"13003291":[7,34],"13009402":[],"13012167":[],"13024288":[],"13035147135400782":[],"13037395":[],"13040489":[],"13055555555555556":[2,40,41],"13055762":[],"13059483":[],"130694":33,"13069442":33,"13086825":[],"13092444":[],"131":10,"13107995":[],"13117805":[],"13134148011170077":[],"13137881":[],"131379":[],"13152828":[],"13162821e":[],"13164176":[],"13181493":[],"13186167":[],"13196676":[],"132":10,"13203156":38,"13206207781887236":29,"13217667":[],"13220608e":[7,34],"132441":[],"13244119":[],"1325":7,"1326":10,"13260573":[],"1326197715":32,"132756":7,"13280":10,"13286177":[],"13296813":[],"13298019":[],"133":[8,36],"1332":35,"13324158":[],"13326664":38,"13337992":[],"13355309":[],"13373501":[],"134":34,"13437168":[],"134534":[],"13453411":[],"13455019":[],"13457622":[],"135":10,"1350":35,"13518746":[],"13531906":[],"13535942":[7,34],"13564988":[],"1356995":[],"13590426":[],"136":[1,8,36],"13600596":[],"136066":[],"13607502":[],"13625288":32,"13646574":6,"13659941":[],"13661243e":[7,34],"13663125":[],"13675593":[],"136756":[],"13679863":[7,34],"13680048":[],"13684661":[],"13699574":[],"136996":[],"137":[1,8,36],"13706862":38,"13728124":25,"13729154":[],"13738633":[],"13740":10,"137652":[12,33],"1378267322316808":[],"13788033":[],"13791264":[],"13799037e":[],"138":[1,8,36],"13819592e":[],"13820":37,"13825388":[],"13845051":[],"13847167":[],"1385":7,"1385510301651882":[],"13858163":[],"13865173":6,"13866942":[],"138775":[12,33],"13878649":[],"1388888888888889":[2,40,41],"13890":10,"13896095":[],"139":[1,8,36],"13911934":[],"13913033":[],"139224":[],"1392559585020426e":[7,34],"139255958997547e":[7,34],"1393548":[],"13945974":[],"13956654":[],"13976073":[],"1399386":[],"13it":[],"14":[0,1,3,5,6,7,8,9,10,11,12,13,22,23,25,29,31,33,35,36,37,38,41],"140":[1,8,10,36],"14002608":[],"14021063":[7,34],"14029264":[],"14038959":12,"140390":12,"14043884":[],"14045682":[],"14046224e":[],"14084366":[],"141":[1,8,36],"14100":10,"14104212":[],"14112437":[],"14114294":[],"141182":[],"14118216":[],"1412":[22,37,38],"14122256":[],"14123881":[],"1416398":[7,34],"14174745":[7,34],"1418":10,"14187173":[],"14192454":[],"142":[1,8,10,36],"14213876":[],"142188":[],"14218838":[],"14239961":[],"1424112":[],"14249052":[],"14250":10,"14251309":[],"14260706":[],"14277718e":33,"14285244":[],"143":[0,1,3,8,10,36],"14309433":[],"14335140e":[],"14369927":[],"1437":[2,40,41],"14373934e":[39,40],"14383145":[],"144":[1,3,8,36],"14400":10,"14404536":[],"1440501043841336":[2,40,41],"1441031":[],"14424825":38,"14433067":[],"14437776":[],"14440":10,"14452065":[],"14452678":[],"14465374":6,"144654":6,"1446729567":5,"144848":[],"14484831":[],"14495737":[],"145":[],"14523337e":[],"14550854":[],"145509":[],"14559761":38,"14568914":34,"145753":12,"1457534":12,"14580867":33,"14599499":[],"146":3,"14605929":[],"14610161":[],"146141":[],"14629156":35,"14638":37,"14643345e":[],"14690374":6,"146904":6,"14697222":[],"14697641":[],"147":[],"14708":[],"14710":10,"14716":7,"14721582":[],"14722222222222223":[2,40,41],"147400":[12,33],"147420":[12,33],"14751649":[],"147545":[],"1477190177681485":[],"1477805":22,"147836":[],"14783612":[],"1479":10,"148":[3,4,5,15],"14812206":7,"14815202":[],"14824254":[],"14825560e":[],"14828178":[],"14829228":[],"14841322":[],"148477":[],"1486419":[],"148776":[],"14880915":[],"1488598":[],"14890012":[],"148913":[],"148932":6,"14893216":6,"149":3,"149003":7,"149063":[],"149111":[],"14913555":[],"149266":[],"14934":[],"14934258650797513":[],"149353":[],"149379":7,"149388":[],"149416":[],"149433":[],"149466":[],"149483":[],"14949972":[],"149524":[],"149904":[],"149921":[],"14g":[7,35],"14it":[],"15":[0,1,3,5,7,8,9,10,13,14,16,17,18,22,29,32,35,36,37,38,39,40,41],"150":[4,5,9,10,15,23],"15001903":[],"15005476":6,"15008902218011533":10,"1502":39,"150301":[],"150337":[],"15042293":[],"15052":[],"150527":[],"15058132343282243":[],"150735":[],"15079545":[],"15098090e":[7,34],"151":[4,5,15],"15126318":[],"15130074e":[7,34],"15143483e":[],"151522":[],"15173932":[],"151892":35,"15199":[],"152":10,"15200":10,"15202654":[],"15207119":[],"152149":[],"15215415":[],"15235148":[],"15242624e":[],"152494":[],"1527777777777778":[2,40,41],"15280951":[],"15283463":[],"1529":[],"15294186382924843":[],"153":[4,5,15],"153036":33,"15313054":35,"15338896e":[],"15351626":[],"15360063":[],"15361856":[],"15367403":[],"1537797":[],"15384615384615385":10,"15388662":[],"15390225":[],"154":[4,5,15],"15403275":[],"15422927":[],"15424679":32,"15435127":[],"15435937":[],"15451852":[],"154751":[],"1547556":[],"1548131":[],"15489318":[],"15493099":[],"155":[4,5,10,15],"15527976":[],"15546887":[],"15563816":[],"15572395":[],"155724":[],"15586219":[],"15593478":[],"155994":[],"15599422":[],"156":[4,5,15],"15605625":[],"15654145":[],"15670465":25,"15670921":37,"1567286":[],"15690073":[],"156956":6,"157":[4,5,15],"15709522":[],"157128":[],"1571284":[],"1575":10,"1575046":[],"15751952":[],"15759533":38,"1576057":[],"1579":35,"158":10,"15830902":[],"15833751":[],"15847501":[],"1587":10,"1590":10,"15905706e":[],"15906954":38,"15924557e":[],"15951871":37,"15970814":[],"1598058":[],"15990":10,"15991124e":[],"15am":32,"15g":[7,35],"15it":[],"16":[2,3,4,5,6,7,9,10,11,20,22,23,29,32,33,34,35,37,38,39,40,41],"16013842":[],"1602625":[],"1603":4,"16039162":[],"16043208":29,"16050357":[],"16063661":[],"16072855":32,"16111111111111112":[2,40,41],"16118615":[],"16119967":38,"16124569":[],"161246":[],"16131839":[],"16146139":25,"16147054":38,"16158074":[],"161581":[],"16172219e":[],"16180248":[],"16181877":32,"16188604":[],"16201555":[],"16209264e":[],"16211139":6,"16211781":[],"1621766238509487":12,"16220":10,"162246":6,"16226023":[],"16231451":5,"1625":10,"16257339":[],"16257872":[],"16271817":[],"16275926":[],"1628":10,"16292908":[],"16304863":33,"163049":33,"1630775253":[2,40,41],"16312434e":[],"16328315":[],"16342407":6,"16343471":[7,34],"1636508":[],"16373437":[],"16384":4,"16388664":[],"163887":[],"16404402":[],"16405708":[],"16407038":[],"16411163":[],"164112":[],"16413938":[],"16431956":[],"16436977e":[],"16437294":[],"16445858":[],"16452931":[],"16469725":[],"16486151":[],"16500":10,"165033577601387":[],"16513872":[],"165272":[],"16532977":[],"16535252":[],"16543524":[],"16552497":32,"16567361":[],"166":10,"1660497e":[],"16606924":[],"16617445":[],"16620159":[],"16622624":[],"1664372":[],"166540":[],"16654048":[],"16674251":32,"16702443":[],"16710567":[],"167116584998668":[],"16744759":[],"16777789":[],"167787":6,"16791183":[],"168":10,"16805821e":[7,34],"16879134":[],"16881787":[],"16913489":[],"16925937":[],"169299":[],"16949234":29,"16it":7,"17":[2,3,6,7,8,9,10,14,22,29,33,35,36,37,38,39,40,41],"17021424":[],"17026621":[],"17069366":[],"17080691":[],"170807":[],"17089674":[],"1709":10,"17093974":[],"171":34,"17101958":32,"171525":33,"1715252":33,"17158357":[],"171611":[],"17168028":[],"17174962e":[2,40,41],"17214545":[],"17214797":[],"1721774":38,"17222222222222222":[2,40,41],"17222494":[],"17239551e":[],"17246397":[],"17249725":[],"17253354":[],"1726":10,"17260271":[],"17272182":[],"17285183":25,"1729993":[],"173":[22,38],"17300":10,"1731":10,"1732863504708044":[],"17359583e":[],"17367225":[],"17377547":[],"17378802":38,"17382534":[],"17405911e":[],"17418673":[],"17446471":7,"17483760":35,"175":35,"17501258":[39,40],"1752":10,"175300":33,"17547294e":[],"17563274":[],"17567503":[],"17568084":[],"17569961":[],"176077701534979":[],"17629205":[],"17641709":[7,34],"17643975e":[],"17669259":[],"176762":[],"17676247":[],"1767857":[],"17714797":[],"17728993":[],"17739525":[],"17748881e":37,"17757295":[],"17777777777777778":[2,40,41],"177786562639708":[],"17780089":[],"1778106":[],"17801022":6,"17801827":[],"17806281e":[],"1780797":[],"17827738":[],"17834506e":[],"17840283":29,"17849836":[],"17856374":[],"17861098":7,"17861414":[],"17865868":[],"17869861e":[],"17882416":[],"17886464e":[],"179":[],"17913653":[],"17917768":6,"1792":[],"17934093":[],"1793417":[],"17934801":[],"1793636":[],"17941158":[],"17944696":[],"17948385":[],"17948638":[],"17949575":6,"17953942":12,"17968231":[],"1797":[2,4,40,41],"18":[0,3,7,8,9,10,11,14,22,23,25,29,35,36,37,38,40,41],"180":[],"18013978":32,"1801709":34,"18019623e":[],"18029127":6,"18038547":[],"18043393":[],"18052195":[],"1807":5,"1808228":[],"18085537":[],"1809":10,"18093549":[],"18094575":[],"181":10,"1810415856976313":[],"18106943":[],"18114057":[],"181141":[],"18118338":[],"1812":10,"18120973":[],"181210":[],"18146425":32,"18147009":[],"181766185664273":[],"18180717":25,"182086698929565":[],"1821":10,"18234308":[],"18234499":[],"18251883":32,"18274735":[],"18277213":[],"18285689e":[],"18307821":34,"18333333333333332":[2,40,41],"18383522":33,"18388":7,"184":10,"1840":35,"18409254":[],"184119":[],"18415031":[],"18418938":[],"18418988":[],"184519":33,"18464035":[],"18474924":[],"18485685":[],"184866696796263":[],"18497496":[],"184975":[],"185":34,"18501900e":[],"18510775":[],"18522369":[],"18551655":[],"185710":12,"18571046":12,"18572822":[],"1860":10,"18603548":37,"18604416":[],"18611111111111112":[2,40,41],"18613217e":[7,34],"1863889":[],"18646225":[],"18660":10,"18670072e":33,"18673098":12,"18702469":[],"18711407":[],"18712313":[],"18715465":[],"18725816e":[],"18742521":[],"18750351":37,"18757012":22,"18781225":[],"1880258":22,"18810873":38,"18816387":[],"18824338":[],"18840034e":[],"18850776":[],"1885906477601598":[],"18859065":[],"18864495":38,"18872206":[],"18925625":[],"189496":33,"18956157":[],"189622":33,"18981286":[],"18986837":38,"18it":[],"19":[3,7,10,16,22,29,32,35,37,38,41],"19001247":[],"19009085621304586":[],"19036055":12,"19052323e":[],"190529":[],"19052935":[],"19069576e":37,"19073442":[],"19126179":[],"19166666666666668":[2,40,41],"19180036e":[],"191833":[],"192":35,"19207979":6,"19220":10,"19299283":[],"19336703":[],"19350397125914334":[],"19364388":[],"19367324":[],"19371605e":[39,40],"19375828":[],"1940":1,"194042826649355e":[7,34],"194042826815211e":[7,34],"1943":[13,38,39],"19433318":[],"1944":7,"19447592":[],"194836":6,"19483625":6,"194889":[],"19488947":[],"19517243":[],"19517373":12,"195174":12,"19526866":22,"19532208":[],"19547923":[],"19559737e":[],"1956":10,"19569961":[7,33,34],"19583547":[],"19588651":37,"19601208":[],"19661197":37,"19677112":[],"1970":[25,32],"19725211":[],"1973":10,"197370":[12,33],"19740":10,"1974189":[],"1979":[7,35],"19791352":[],"197914":[],"19798258":[],"19800":10,"19800999":25,"19808709":[],"1982973034395375":6,"19848482":[],"19849513":[],"19852045":[],"19857342950283463":25,"19861706":[],"19884659":[],"1989":39,"1991":39,"19920353":[],"199324":[],"19932442":[],"19945294":[],"19951849":[],"19973189":[],"199824":[],"19982428":[],"1999":27,"1_1":[13,38],"1_2":[13,38],"1_3":[13,38],"1cm":[1,9,11,29,32,39,40],"1d":[2,3,4,40,41],"1e":[2,3,5,14,15,22,37,38,40,41],"1e10":15,"1e4":7,"1f":[2,40,41],"1k":25,"1n":[1,32],"1ocqm1tltwb6hzsf9i7ggugw9m8sbvequ":[],"1s":[40,41],"1x":[1,32],"1zkibvqf":23,"2":[0,2,3,4,5,6,7,8,9,10,11,12,13,14,15,20,22,24,25,26,28,29,31,35,36,37,38],"20":[0,1,2,3,7,8,9,10,14,20,22,29,30,32,34,35,36,37,38,39,40,41],"200":[1,3,4,5,9,10,11,41],"2000":[1,33],"20005403":32,"20015436":7,"20020163":[39,40],"2004":[14,36],"20040481":[],"2004542":[],"20051449":[],"2006":31,"2008":32,"2010":[2,39,40],"2011":[2,40,41],"20135691":[],"2014":5,"2015":[2,40,41],"20158083":[],"201581":[],"2016":[1,32],"2017":41,"20173175":[],"2018":[1,7,33,35],"2019":[9,10],"20194579":[],"2021":[7,15,33,34,36],"20218457":[],"2022":[27,32,39,40,41],"20224027":[],"20228273":[],"2023":[26,40,41],"2024":[0,16,17,18,19,20,21,22,23,27,32,33,34,35,36,41],"20263701":32,"202698":[],"2026985":[],"2027":10,"20277777777777778":[2,40,41],"20288581":[],"20290903":[],"20294638":[],"203":[],"20300793":[],"2032806":[],"20336784":[],"20347912":[],"20353821":32,"20357736":[],"20360":41,"20365164":32,"20367355":[],"20375464":[],"20393163":[],"20393534":[],"203963":[],"20396326":[],"20402019":[],"204049":[],"20404927":[],"20437065":[],"204488":[],"2044881":[],"20480187":[],"204802":[],"20500":10,"205116":[],"20511614":[],"2051784":[],"20519013":[],"20519189":[],"20520318":32,"20527785":[],"20536289":[],"20536579":6,"205366":6,"20539421":[],"205757":[],"20575734":[],"2060":10,"20601797e":[],"20607165":[],"20608548e":[],"20621146":[],"20640822":32,"20651434":[],"20656353":32,"206664":[],"20666447":[],"20680982":[],"2069":10,"20716644":[],"2072382":[],"20728198":[],"20742956":[],"20743662":38,"207545":[12,33],"20755569":[],"2077068":[],"20775115":[],"20793362":[],"20833333333333334":[2,40,41],"20834188":[],"2084894":[],"20867052175034223":[7,35],"20867716":[],"20898872":[],"20912557":[],"20915094":[],"20916295":33,"20980":10,"20986506":[],"21":[0,1,2,6,7,8,10,13,14,22,25,32,33,35,36,37,38,39,40],"210340":[12,33],"21047414e":[],"2105":41,"21058097":6,"21061589":[],"21070843":[],"2107546970521115":[],"21093775":7,"211":[],"21130":10,"2116753732":5,"21169159e":[7,34],"21181534":[],"21192602e":[],"212":[],"2127032":[],"21277816e":[],"213":[],"213103":[12,33],"2131067":[],"2133127":[],"21332621":[],"213336":[],"21333606":[],"21333766":[],"21344077":[],"213743":[12,33],"21390371e":[],"21392756":[],"21394125":[],"21407599":[],"2142385":[],"21430643e":37,"21432413e":[],"21444055":[],"21457724":[],"21501369":[],"21506004":[],"21517262":7,"21531156":[],"21565007e":[],"2157318":[],"21596432":7,"2160073466353432":[],"216171":[],"21617146":[],"216290":[12,33],"21631786":[],"21635399":[],"21641173":[],"21655059e":[],"216683":[12,33],"21680864":[],"21690241":[],"21699817":[],"2170":23,"2171":23,"2172":23,"21724894e":[],"2173":23,"2174":23,"21742484":[],"21752693":[],"21757908":[],"21776599":[],"21784616":[],"21786054e":[],"2180":[],"21805426":[],"2181":[],"21813899e":[],"21817966":[],"2182":[],"2183":[],"21833102":[],"21838482":[],"2184":[],"21846767e":[],"21894241":32,"21905519":[],"21914111":[],"21919019":29,"21941484e":[],"21949761":[],"21998973":[],"22":[1,2,6,7,10,13,14,22,25,32,33,35,37,38,40,41],"22004411":[],"22004496":[],"22013655":[],"22044605e":[6,33,38,39],"22087319":[],"221":9,"2211133":[],"221180":33,"2212296":[],"22128142":[],"22158725223474995":[],"2216":10,"221666864828611":[],"22169087":[],"2218":10,"221805":[],"221921":6,"22194497":[],"221945":[],"22217237e":[],"22226881":[],"222400":33,"22370461988004753":[],"22377412":[],"22388052":[],"2239":23,"2240":23,"2241":23,"2242":23,"22426265":32,"22462544":[],"22467274":35,"22480853":[],"224809":[],"22481166":[],"2249":[],"225":5,"2250":[],"2251":[],"2252":[],"22523018":[],"22528536":[],"22545286":[],"225453":[],"22569872":[],"2257449":[],"22592946e":[],"226014":[],"22601408":[],"22623421":[],"22625657":32,"22667819":[],"22688898":[],"22690428":6,"22706777":[],"227072949395066":[],"22707295":[],"22715479":[],"22718624":29,"22755839e":[],"22756518":[],"22773898":[],"227738984674172":[],"22783539":[],"22809974":[],"22810365":[],"228104":[],"228216":[],"22821607":[],"22830771":[],"22835717e":[],"22842468702219465":[7,35],"2284246870221947":[7,35],"228467858509994":[],"22846786":[],"22847924":6,"22849742":[],"22851508":[],"22871":37,"22896144":[],"22897303":[],"22919456":[],"22929165":25,"2293":[],"22936132":[],"2294":[],"22946401":[],"22959895":[],"229598952352728":[],"2296":[],"2297":[],"22974904":[],"22977659964799":[],"2297766":[],"22985907723809532":[],"22it":7,"23":[2,7,10,13,16,21,22,25,32,35,38,39,40,41],"23002365e":[7,34],"23006632e":[],"23007231e":[39,40],"230154":[],"23015412":[],"23038906":[],"23040825":[],"230760":12,"23076025":12,"23076923076923078":10,"23077255":[],"23093598e":[],"2310":41,"2311959":[],"23139242":[],"23142852":[],"23164401":[],"23167717":6,"23238973":[],"232435":33,"23262877":[],"23263884":[],"23303727":[],"23326473":[],"23333333333333334":[2,40,41],"23344210e":[],"233481":[],"23348144":[],"2338675":33,"233868":33,"234":7,"23407025":[],"23423987":[],"234332597247826e":[],"23449217":[],"234496":[],"23449607":[],"2345":23,"2346":23,"23469347":[],"23472994":[],"2348":23,"23481471":[],"2349":23,"23499549":[],"23499891":[],"2350":23,"2351":7,"23526911":[],"23528337":35,"2355":[],"2356":[],"23565337":6,"23570126":[],"2358":[],"2359":[],"235969":[],"23596934":[],"2360":[],"23616967":[],"2364":10,"23663482":[],"23692142":[],"2373286546935638":[],"23770713":38,"23794815":[],"23807169":[],"2383703":[],"23847441":[],"239":35,"23950884":[],"2397":10,"23971032":33,"23994188e":[],"23it":[],"24":[1,2,7,10,22,25,32,35,37,38,40,41],"24000075":[],"2401":26,"2401586":[],"24032019":[],"24033384":[],"24054861":[],"24071032":[],"24088952":[],"24106503":37,"24121476598003452":[],"24140":10,"24149532":[],"24157456":[],"24159785":33,"2416":10,"24164234":[],"24175744e":[7,34],"24185931":[],"24188439":25,"2419":10,"24200844":25,"2420287e":[],"24224776e":[],"24225043":[],"24254071":[],"24284399":[],"24299211":[],"2430":10,"24335376":[],"24344979e":[],"24346138":[],"24369936":[],"24377554":[],"243776":[],"24382159e":[],"24390":10,"24398826":[],"24412628":[],"24427624":[],"244293":[],"24429334":[],"24430166":[],"24433117":[],"24444444444444444":[2,40,41],"24454968":[],"24485814":37,"24493666":35,"24508783":[],"24512233":[],"24524426":[],"245781":[],"24578123":[],"24582593":[],"2458435":[],"246":[],"24602146":[],"24613581":[],"24629299":[],"24640494":37,"2468382":[],"24688853":[],"24692014":[],"24719968":[],"2475908482537097":[],"24759085":[],"24768421e":37,"24777879":[],"24780663":[],"24816468":[],"2481902":[],"24829629":25,"24829908":6,"24848631e":[],"24856992e":[],"24874378":[],"24906604e":[7,34],"24911588e":[],"24999088":[],"24it":7,"25":[3,4,5,6,7,9,10,12,22,26,33,34,35,37,38,39,41],"250":[3,5,8,10,36,41],"25000":1,"25003261":[],"25007671":[],"250077":[],"25018094":[],"25034533":25,"25068519":[],"25095067":25,"25106936":[],"25111821":[],"25145374e":[],"25185716":34,"251879":33,"25197648":[],"252008":[],"25200841":[],"25206089":[],"25206371":[],"25233253":[],"252333":[],"252436":33,"2527027":[],"252703":[],"25294371":[],"25299716":[],"25337053":[],"25343155":35,"25345918":[],"25354173":[],"25360039":[],"25386687":[],"253891":[],"25389111":[],"25444295":[],"25445":6,"254450":6,"25458506e":[],"2546186":[],"254619":[],"2546817701709956":[],"255":4,"255001":33,"25509222":[],"255107903576545":[],"2551096333979939":[],"25512598":[],"25555619":[],"25568695":[],"255888":6,"25588843":6,"256":5,"2561719":22,"25617655e":[7,34],"25617658e":[7,34],"25646067":[],"25659561":[],"25673276":[],"256733":[],"25680702":[],"25695398":35,"257":[],"25705685":[],"2572":10,"25729558":[],"2575":10,"25767462":22,"25816519":[],"25820894":[],"25849623":[],"25868261":[],"25879315330607":[],"25904489":[],"259153":[12,33],"25915746":38,"25934872":[],"2597":10,"25970297":[],"25982474":[],"259825":[],"25986026":[],"25989189":[],"26":[7,10,17,22,35,37,38],"26001134":[],"26024513":[],"26028795":[],"26048873":[],"26083712":[],"26090013":[],"26113912":[],"26120167":[],"26124692":7,"26128189":[],"26152075":[],"26188646":[],"26191905":[],"26196737":[],"26240613":[],"26254957e":[],"262648":[],"26264849":[],"26280614":[],"26281735":37,"26292364":35,"26292935":32,"26297455":33,"26301436":6,"26307235":[],"2630723514841553":[],"26310762":29,"26321660e":[],"26378176e":[],"26402837":[],"26409315307910025":[7,34],"2640931530791004":[7,34],"26421616":[],"26443391":[],"26464303e":[],"26479234":[],"2650":10,"265109911":5,"2652667":[],"26527089":[],"26536626e":[],"2654":10,"26571335":[],"265868":[],"26586817":[],"26590299":25,"26604611":[],"2662452":[],"26638638":[],"26639089e":[],"26646852":[],"2666905":[],"26674792":[],"26675647":[],"26680052":[],"266801":[],"26692247":[],"26710969":6,"26715191":[],"26767021":[],"26775644":[],"26780278":6,"26781531":38,"2679757":[],"268":10,"268142":[],"26814205":[],"26814548":7,"26817703":[],"26821199":[],"26835242":25,"26897875e":[],"2694078":[],"26953857":[],"26956051":25,"26969649":[],"26995402":33,"27":[1,2,7,21,22,25,35,38,40,41],"27037635":[],"27065633":34,"27097785":[],"270978":[],"27108866":[],"27124938":[],"271260217275557":[],"27126022":[],"27182417":[],"27186049":[],"27223854":22,"27227895":[],"27231287":[],"2729933":[],"27336592":[39,40],"27352535e":[],"27403356":[],"27410687":[],"27425372":[],"274394546800107":[],"27439455":[],"27458376":[],"27465664":[],"27470622":37,"27482829e":[],"2750":10,"2751616":[],"27530242":[],"27597777e":[],"27600259":[],"27602210e":[],"27623637":[],"276263":[12,33],"27633474":[],"2765068":[],"2766892":[],"27677742":[],"27693602e":[38,39],"27698328":[],"27700":10,"27750534":[],"27760":10,"27788872":38,"2781456":[],"27861634e":[],"27874136":[],"27906869":[],"27916593":6,"279166":6,"27924636":6,"27933134":[],"2793874":[],"27952294":[],"27968813e":38,"27993904e":[],"27it":7,"27n_":29,"28":[2,4,5,7,10,22,35,37,38,40,41],"28000355":[],"280004":[],"28021598":[],"2802234e":[],"280464e":[],"280573":6,"280647":[12,33],"2806901924685401":[],"2807716628772665":[],"28078417":32,"2809076":[],"28093636":[],"28098147":25,"28109198":[],"2813829":[],"28158348":[],"281627":6,"28162714":6,"28164443e":[],"28183460e":[],"28192769":[],"28205578e":33,"28210156":[],"28225179":[],"2822642":[],"28244905":[],"28253033":[],"282727":[12,33],"28276275":[],"28292756":[],"28306304e":[],"2830637392":5,"28316949":[],"28336218e":[7,34],"28341218":38,"28341932":25,"28347758":6,"283478":6,"28358004":[],"28364423":[],"28365323e":[],"28366671":[],"2837242622462304":[],"28377068":38,"28390":10,"28391978":33,"28399125":[],"28415767":[],"28443039":35,"28457291":[],"28472162":[],"284722":[],"28475098":9,"28496213":[],"285056":[],"28505619":[],"28519494":[],"28542801":[],"286038":[],"28603828":[],"2861":29,"28614301":[],"2863391":[],"28641033":[],"28645331e":[],"28709718":[],"2871":10,"28726748":[],"2873":10,"28754232":[],"2876045":[],"28766899":[],"28776411":[],"28793787":[],"2882":29,"28833782":[],"28836035":[],"28844641e":[],"2885519":[],"2886":29,"288851":[],"28885103":[],"28886314":[],"2890":[1,32],"28902948":[],"2890577":[],"2892":29,"28923":[],"28924386":[],"28956161":[],"2895786617507":[],"28963909":[],"28973035":[],"28it":7,"29":[7,10,22,35,37,38],"29015871":[],"29030256":[],"29033075e":[],"29078236":[],"29087396":[],"290874":[],"29089154":[],"29091211":[],"29100624e":[],"291056":[],"29107892":[],"29134342":25,"29142853":[],"29147052":[],"2915":29,"29155617":22,"29162184":[],"29167186":6,"29201128":[],"29223521":[],"29234681":[],"29236762":[],"29238145":[],"29253213e":[],"29306584e":[],"2930953":[],"2931":32,"29350":37,"2936657":[],"293665700387852":[],"2938162":32,"29390668":[],"29398019":[],"294":[],"29411652":[],"29414225":[],"29419924":[],"29474356":[],"29491144e":[],"295":[],"29509520e":[],"29541992e":[],"29560433":[],"29561686":[],"29593305":[],"29620357":[],"29627774":[],"29633608":[],"2964378":[],"29645662":[],"29653038":37,"2965315":[],"2965479984453907":[],"29669411":[],"29678339":[],"2968":32,"29687889":32,"297":[],"29716773":37,"29738252":[],"2974512":38,"297476846305301":[],"29747685":[],"297581":[],"29758118":[],"297648":[],"29764815":[],"2976637":[],"29777705":38,"298":[],"2980":[23,32],"298037":[],"29803738":[],"29822833":[7,34],"298499":[],"29849933":[],"29860173":37,"2986082":[],"29861395":[],"29866395":[],"29881308":[],"299":7,"2990":32,"29902518":[],"29950088":[],"299748":33,"29995377":[],"299954":[],"29it":7,"2_":[13,38],"2_1":[13,38],"2_2":[13,38],"2_3":[13,38],"2_i":[13,38],"2_m":[7,29,35],"2_t":[14,37,38],"2_x":29,"2b":29,"2c8f433990d1":[37,38],"2cm":9,"2d":[2,4,12,13,24,32,38,39,40,41],"2e":[7,35],"2f":[0,1,8,10,11,12,13,32,33,36,38,39],"2g":[3,41],"2g_i":[3,41],"2k":4,"2m":[7,35],"2n":[1,3,4,32,33,41],"2nd":[10,38],"2p":[29,39],"2pm":30,"2pt":5,"2x":[1,4,9,14,32,37,38,39],"2x_ix_jy_iy_j":9,"2x_j":9,"2xb":39,"2y_i":11,"2y_j":9,"3":[2,3,4,5,6,7,8,9,10,11,12,13,14,15,18,19,22,24,25,26,27,28,29,30,34,35,36,37,38],"30":[0,1,2,5,7,8,10,11,14,17,22,30,33,35,36,37,40,41],"300":[22,38,39],"30000":[1,32],"30000613":[],"3000498":32,"30005983":[],"30010":10,"30013135":[],"300231":[],"30023144":[],"30043041":[],"30054056":[],"30069427":[],"3008836":[],"30117681":[],"30119421":9,"30124108":[],"30134885":[],"30136959":[],"30141906":[],"30190897":[],"301909":[],"30191327e":[],"30196557e":[],"302":35,"30258509":[25,32],"30269539":[],"30283188":34,"303":7,"30306759":37,"30322722":[],"30332201":[],"30334592e":[],"30387411e":[],"30400113":[],"30442002e":[],"30442964":33,"30450049":[],"30458085":[],"30466214e":[7,34],"30493151":[],"30506075e":[],"30593976":25,"30607268":37,"30616802":[],"30633304":[],"30637636":[],"30655251e":[],"30675215e":[],"3068659":[],"306866":[],"307":[],"3072":4,"3075371":[],"3076923076923077":10,"30782691":[],"30787294":7,"30887484":[],"30894285":[],"30897153":[],"308972":[],"309":[],"30915293":[],"309153":[],"30916402":[],"30916988":[],"309170":[],"30917499e":[],"30940":10,"30959659":[],"3096154":[],"30982436":[],"3099":35,"30it":7,"31":[13,25,29,38],"31008026":[],"310579":33,"3105791":33,"31068981":25,"310957":[],"31095745":[],"31096156":[],"31159615":32,"31160886":[],"31162122":[],"31175179":[],"31186409":25,"312":[],"31210861":[],"31216736":25,"31227657e":[],"3123314713548606":[7,35],"31248425":37,"31276579e":[7,34],"31286143":[],"31294131":[],"31299668e":[],"313":[],"313155":[],"31315504":[],"31318084":6,"31330137":[],"31335664":32,"313568":[],"31356849":[],"31362428":[],"31363366":[],"31367041":[],"31370028":[],"31381731":[],"31436277":[],"31453949":[],"31457796":6,"314869":[],"31486917":[],"315":[7,34],"31541884":[],"3155":[1,6,7,34,35,36],"31555734196707536":[],"31562008":[],"31568288":[],"315683":[],"31571995":[],"316":[],"31601423":[],"31607059":[],"31639624":[],"31664443e":37,"31702609":[],"31711533":[],"31718909":12,"31727523":[],"317367":12,"31739522":25,"31743443":32,"31749096":29,"31758645":[],"31764285":38,"317799":[],"31779928":[],"31787864e":[],"31799354":[],"31812561":[],"31849289":[],"31876431e":[],"31888254":32,"31896852":9,"31897935":[],"31903447":[],"31925135":[],"31936237":[],"31938579":[],"3194467":[],"31950286":[],"31971976e":22,"31973052":[],"31983943":[],"31it":[],"32":[4,5,7,9,13,14,25,29,37,38],"3200":[2,40,41],"32003025":[],"32007041":[],"32052031":[],"320656":[],"32065614":[],"32068736":[],"3207":35,"32084559e":[],"32089655":[],"32114535":[],"32128064":[],"32149601703519115":[7,35],"3214960170351912":[7,35],"3215":10,"3218791":[],"32232922":32,"32262934":37,"32265127e":[],"323059":[],"32305911":[],"323066643161066":[],"323184775146844":[],"32318478":[],"32320616":[],"32329897":[],"32336220689648504":[],"32354671":22,"3236445":[],"32393078":[],"324":[],"32411274":[],"32419549":[],"3241988":[],"32422234e":[],"32447198":29,"3245":[],"32450054":33,"32466925":[],"32493877":[],"3250":[2,40,41],"32513272":[],"325228546249804":[],"32522855":[],"3254":3,"3255":3,"3256":3,"32567311":[],"3257":3,"3258":3,"32588151":22,"3259":3,"3260513":[],"32606139":[],"32606365":[],"32616922":[],"3262":7,"326238":33,"32641434":[],"32641558":[],"326430":[],"32643011":[],"3265097":[],"32651488":[],"32670322":[],"32683063":[],"32684035":[],"32701708":[],"32717946":[],"32733949":[],"32738537":[],"32786329":[],"32798457e":[],"32799141":[],"3280928":[],"328093":[],"32825214":[],"328462040651484":6,"3284789":[],"32906581":[],"32932635":[],"32941912":[],"32962444":[],"32972613":[],"32it":[],"33":[10,13,25,30,37,38],"330":34,"33000573":[],"33006628":[],"33025801":[],"33027933":[],"33045013":[],"33046180e":[],"33049619":[],"33062878":[],"330629":[],"330660516571777":[],"33066907e":[6,33],"33081484":29,"33095972":[],"33096549":[],"331":34,"33107417":[],"331075":[],"33145794":[],"33149531":[],"33163086":[],"331631":[],"33165194":38,"33166055e":6,"33180288":[],"331939":33,"33212971":[],"33217312":[],"33220123":[],"33235647":[],"33241952":[],"33245403":[],"33266422":[],"33282365":22,"333":[8,34,36],"33300231":[],"33329216":[],"33357783":37,"334":34,"33433563":[],"33435637":[],"33477019":[],"33492417e":[],"335144":12,"3351443":12,"33519245":38,"33528448":[],"33532886":[],"335708":[],"33570831":[],"33596404":[],"33604208":[],"33615742":[],"33675308":[],"33700495":[],"33718234":37,"33734605":[],"33742926":[],"33745439":[],"33826500e":[],"33871003":[],"3388112":[],"3391965479837245":[],"33978889":37,"33it":7,"34":[10,25],"340071371496255":33,"34023682":[],"340237":[],"34026086":[],"3403":10,"34031551":38,"3404509":[],"34061974":[],"340620":[],"340782":[12,33],"34098895":[],"34112554":[],"34114547":6,"34121963e":[],"34131583":[],"34134773":[],"34139775e":[39,40],"34182884":[],"34205649":[],"34225662":[],"34249217":[],"342680":33,"34271514":[],"34332677":[],"34346829":[],"3436":[1,32],"34367970":7,"3437":[1,32],"3440539":[],"34412923":33,"344176915013831":[],"34427423":[],"34478594":[],"34483937":[],"34483955":[],"34488863":25,"34549184e":[],"34554342":[],"34568909":[],"34569596":6,"34569632":[],"34577375":[],"34582863":[],"34598722":[],"34599467e":[],"34599598":[],"346000154268618":[],"34601775":[],"34603176":[],"34620097":[],"34654596":[],"346546":[],"34661376":[],"346614":[],"3466389":32,"34704885":[],"34726166":[],"347262":[],"34732758":[],"347328":[],"34759233":[],"3478095":[],"3480527":[],"34832197":25,"34851141":[],"34862407":[],"34871778":[],"34904819":[],"349312":[],"34931214":[],"34951108":[],"3499634874668227":[],"34it":[],"35":[1,7,18,26,30,32],"3500312e":[],"35018806":[],"35098705":[],"35140":10,"3514835":[],"351636":[12,33],"35182854":6,"35195241":[],"35196964":[],"35243372":[],"35268457":12,"352685":12,"35275513":[],"35282003":[],"35313779":[],"35320952":[],"35354073":[],"35388766e":[],"35432563":[],"354360":12,"35436034":12,"3544313922":7,"35468713e":[],"35470445e":[6,33],"35506347e":[],"35520169":[],"35533773":7,"35537114e":[],"355734685106391":[],"35587321":[],"35595337":[],"35613536e":[],"356399":33,"356673":[],"35667342":[],"35689303":[],"35713539":[],"35714377":[],"35731585":[],"35736":37,"357508":33,"3577615":[],"35785072":[],"35798999":[],"357990":[],"35806319":[],"3581341341":5,"35823372e":[],"35826030e":[],"3583246":[],"35870701":[],"3587467":[],"359":[6,34],"35914932":[],"35921835":[],"35956676":[],"35957915":[],"35966688":[],"35980374":[],"35986305":[],"35it":[],"36":[1,6,7,26,29,39,40],"360":[2,40,41],"36014743":[],"36031586":[],"36091273":[],"36116396":[],"3613":10,"3615115":32,"361541":[],"36154104":[],"361556":33,"36161658":[],"36185989":[],"3619775819765686":[],"36209091":[],"3621311":6,"36219301":[],"36235052e":[],"36255393":[],"36278226":[],"36303682e":[],"3632959111950474e":[7,34],"363295916366933e":[7,34],"36339156":[],"36353321":[],"36388542":[],"36401981":[],"36403018e":[],"36439409e":[],"36482524":[],"36490166":22,"36545697e":[],"3655222":6,"36579128":[],"36623476":[],"366375489616074":[],"36658646":[],"36694417":25,"367":[3,10],"36754671":[],"36790471":[],"36841864":[],"36845893":32,"36846593":29,"36880327":[],"368818":[],"36881837":[],"36889994":22,"36895644":35,"369139":[12,33],"36919318":[],"36it":[],"37":[10,26,36,37],"37007195":[],"37008469":[],"37013105":[],"37023353":29,"37023448":22,"37024276":[],"370243":[],"37055062":[],"37057274":[],"370782966":5,"37081457":[],"37093787":[],"37132543":[],"37135473":[],"37137157":[],"37153115":[],"37175575":37,"37175588":[],"37180834e":[],"37215356":[],"37239927e":33,"37255873":[],"3727791":[],"37293361":[],"372934":[],"37343957":[],"37354121":[],"37363888":6,"373639":6,"37396662":[7,34],"37415625e":[],"37449168":[],"37454707":[],"374675":[],"37467515":[],"37468967":25,"37507799e":[],"37532078":[],"37564555":[],"37582891":[],"375829":[],"37600844":[],"37600869":[],"37685349":[],"37700967e":[],"37705573":25,"37717637":[],"37730061":[],"377372":33,"37737221":33,"3777801602":7,"378271":[],"37827145":[],"37837864":[],"37857846":[],"37900111":[7,34],"37919989":[],"38":[10,26,29],"380":10,"3807176":[],"380718":[],"38102552":[],"38123077":[],"38135733e":[7,34],"38155181":29,"3817475779":[7,35],"38198866":[],"38203725":[],"38240257":[],"3824865":[],"38259084":37,"38298234e":[],"38309751":[],"38337498":[],"38381781":[],"38399622":[],"38422061":[],"38440469":[],"38445171":[],"38447892":[],"38460453":[],"38461538461538464":10,"38461539":37,"38506982":38,"38535182":[],"38571859":[],"38592549":[],"386":10,"38612299e":[],"386161":[],"38616125":[],"38618024":[],"38618158":[],"38629436":[25,32],"38675231":[],"38687011":[],"387":35,"38713573":[],"387620":[],"38762019":[],"387672":[],"3881343":[],"3882273":[],"38827994":7,"38853437":[],"38865681e":[],"38916861e":[7,34],"3892":35,"38934047":[],"38943":[],"389430":[],"38949027":[],"38962192e":[7,34],"38992371":[],"38it":[],"39":[1,10,27,30,32,38,39,41],"390334838933238":[],"39033484":[],"39039235":[],"3908470004999506":[],"39090680e":[],"3910042":[],"39184027":32,"39191126":[],"39197698":33,"391977":33,"39214397":33,"392144":33,"39223923":[],"39225935":[],"3924501":[],"39279291":[],"39292847":[],"39376038":[],"39386718":[],"3939184e":[],"39398622":[],"394":10,"39401392":[],"394505933":37,"39503373":25,"39516594":[],"39517073":[],"39525059":[],"39529587":[],"39540235":[],"39579407":6,"39590571":[],"395980256477532":[],"39598026":[],"39614949":[],"39621036e":[],"39646454e":[],"39692815":[],"39697001":[],"39706038":6,"39730902":25,"39741875":32,"397700":[12,33],"39789527":[25,32],"39799981":[],"3983":7,"39855569":[],"39865452":[],"39914635":[],"39919362":25,"39922082":[],"39929428":[],"39931051e":33,"39941008":[],"39952356":[],"399533503407795":[],"399836":33,"39991048":[],"39it":[],"3d":[3,4,5,7,14,26,35,37,41],"3f":[2,4,10,40,41],"3n":25,"3x":[3,9,41],"3x_0x_1":39,"3x_i":[3,41],"3y":9,"3yk470mj5p931p9dtkk0y6jw0000gn":[2,7,32,35,37,40,41],"4":[2,3,4,5,6,7,8,9,10,11,12,13,14,15,18,22,25,26,29,32,34,35,36,37,38,39,41],"40":[2,7,10,26,27,30,32,35,39,40,41],"400":5,"4000":[10,32],"40012736e":[],"40030606e":[],"40037367e":[],"40058341":[],"40072098":[],"4008393":[],"40093324":[],"40101943":[],"40140374":[],"401842":[12,33],"40184681":[],"40209956":[],"40266454":[],"40310129":[],"40354798e":[],"40362686":[],"40415979e":[],"40418675":[],"40431935":[],"40434293":[],"40447213":[],"404783":[],"40478305":[],"40491159":[],"405":10,"4050":[31,32],"405163":[],"4051634":[],"40558184":[],"405890":[12,33],"405964":[],"40596423":[],"40605118":[],"40615633":[],"40620066":33,"406201":33,"40632663":[],"40640797":[],"4069157":[],"407":10,"408":10,"40800396":[],"40833375":[],"40848657":[],"4087":10,"4087793":6,"40903389":[],"40906565":[],"40913138":29,"409131380658218":29,"40927184e":[7,34],"40961688":[],"4096949":[],"40it":[],"41":[1,10,25,27],"41062015":[],"4107":10,"411":[],"41146753":[],"412":[],"4121966338442804":[],"41229845":[],"41259788":[],"41298822":[],"413":[],"41310208":[],"41366128e":[],"41377873":[],"41381309":[],"41388745":[],"41396339":[],"41408221":[],"41416175":[],"41433969":6,"4143542":[],"41452482":[],"41487642":[],"4150026":[],"41511965e":[2,40,41],"41524442":[],"41532521":[],"4155":[3,16,41],"415597":[],"4155974":[],"4162706317":7,"41636775":[],"41664613":[],"4166666666666667":10,"41675053":[],"41705228":[],"41744203":[],"41764479":[],"417644792969273":[],"41788797":6,"417888":6,"41794469":[],"41820116":[],"41842093e":[],"41847686e":[],"418506":[12,33],"4185532":[],"418596857594715":[],"418656":[],"41866748":[],"41869784":[],"41882037e":[7,34],"41925189":[],"419718":[],"41971814":[],"41987877":[],"4198932":[],"42":[2,5,9,10,11,25,27,41],"42013616e":[],"42045168":29,"42083463e":[],"42095725":[],"421":[],"42124182":[],"4214275":[],"4215795":[],"42197297e":[],"422":[],"423":[],"42305151":[],"4230769230769231":10,"423102":[],"42310878":[],"42338272":29,"423646":[],"42364634":[],"42392359":[],"42394588":[],"424274":[],"4242743":[],"42439452":25,"42441033":6,"42450":10,"42497724":[],"42501752":[],"4252885":[],"42553096":[],"42575338":[],"42590512":25,"42592679":[],"426":[0,7,10],"42644801":[],"42670187":[],"42679392":[],"42699034":[39,40],"42772604":[],"42774546":[],"42793344e":[],"42813351":[],"42839645e":[],"42841423":[],"42873361":[],"429":35,"42914251":[],"42945319":[],"43":[0,1,2,8,10,25,36,40],"43007284":[],"4303589":22,"43054282":6,"4310":32,"4313428":32,"43167881e":[],"43208935911369845":32,"43258818":[],"43285487":[],"433":35,"43330971e":[7,34],"43373107":[],"43418708":38,"43492736":[],"43506123":25,"435163":33,"4353359":[],"43563433":[],"43579948e":[7,34],"43639284e":33,"436462435":5,"43703545":[],"43713984":32,"43724406":[],"43727575":[],"43754843":[],"43766686":12,"43779698e":[],"438":[4,5,15],"438060758":7,"438136":33,"43836612":25,"439":[4,5,15],"4390092":[],"439230":7,"43951798e":[],"43962226":[],"43976975777097144":[],"43984553e":[],"43it":[],"44":[1,2,25,40,41],"440":[4,5,15],"44007188":[],"44089210e":[6,33],"4408921e":[],"441":[4,5,15],"44108308":[],"44129766":[],"44137667":[],"442":[4,5,15],"44203729":[],"44239442":[],"44255455":[],"442600":[12,33],"44286735":[],"443":[],"443217":33,"44384079":38,"44395301":34,"444":10,"44418124":37,"44440947":[],"44454123":[],"44456367":[],"44476964":[],"444770":[],"44498054":[],"445":[],"44512218":[],"445128":[],"44536331":[],"44563226":[],"446033":35,"44624525e":33,"44658658e":[],"44756744":29,"44767398":[],"4477214":[],"44830934":[],"44832989":25,"44838546":[],"44845593":[],"44867801":[],"4488194":25,"44895662":[],"44970586e":[2,40,41],"44995924":[],"44it":[],"45":[10,30,32],"450":10,"45007578":[],"450257":[12,33],"4504":10,"45052717":25,"45053025":[],"45054585":[],"45104086":[],"45170202":[],"451702023099923":[],"451834":[],"45183405":[],"45193407":[],"45273617":[],"45397592":[],"45403192":[],"45408759":[],"454126":[],"45412633":[],"45430236":[],"45457427":[],"455173":35,"45538713":25,"45560415":[],"45568883":[],"45572817":[],"45576546":[],"4557763":12,"455947":33,"456":10,"45609936e":[],"45620975":[],"45647803":[],"4565147":[],"4565685":[],"457":[],"45709724":[],"45730151":[],"457311":[],"45731144":[],"45731379":[],"4576811":[],"45781905e":37,"457962":12,"45796224":12,"458027":33,"458078":[12,33],"4580854":[],"45840653":[],"45847011":[],"4591127":[],"45915671e":33,"45921927":[],"45960079":6,"45993741":[],"45it":[],"46":[3,10,23,30,32],"460":[0,10,12],"4601":10,"46049816":[],"46053947":25,"46065811":[],"460688958585373":[],"46068896":[],"46069873e":[],"46086448":[],"46107913":[],"46134248e":[],"46153846153846156":10,"46166231":[],"46170878e":37,"4619526":[],"462":[8,36],"462295":[],"46229544":[],"46242424":[],"46318204":[],"46342934":[],"4635149":[],"4636751":[],"46383925e":[7,34],"46383926e":[7,34],"46387965":[],"46415522":[],"46415888336127775":34,"4644459":29,"46544987":[],"46593404":[],"4659454":[],"46660941e":[],"46713267":34,"46736215":35,"46736822":[],"4676059":33,"467606":33,"46796873":25,"468":41,"46891823":[],"46893501":32,"46911594e":[],"46954702e":[],"46963249":[],"46984559":[],"46984697e":[7,34],"47":[3,10,23,30,32],"47018223":[],"47042744":6,"47051432":35,"47052809":[],"470714":33,"47075725":7,"47116868e":[7,34],"47123617":[],"47125748":6,"47130422":25,"47132891":6,"47187403":32,"47193243":[],"4725246793626128":[],"4729218":[],"47297587e":[],"47308061":[],"47313680":35,"47334547e":[],"4735968":[],"473730000968826":[],"47382705":[],"4742433":32,"47433659":[],"4745474":[],"47478922e":[],"47493254":[],"47544463":[],"47609206":[],"47610036":7,"4761747":[],"476849792":37,"47729832":[],"477405":12,"47740516":12,"47757553":[],"47767335":[],"47815203":12,"47823856":37,"47830372":32,"47851966":[],"47865692":[],"47892661":[],"47900239":32,"47905601e":[],"47910545":[],"479465113":5,"47989224":[],"47991892":[],"47it":[],"48":[3,10,23,35],"48004784":[],"48017159":[],"48062913":[],"48121459":[],"481215":[],"48138036":[],"48151166":[],"48170743":[],"481979":7,"48200792":[],"48233952":[],"482495":[],"48249527":[],"4825169":[],"48257387":[30,32],"48268005e":[],"48306473":[],"48307405":[],"4831139":[],"48326853":[],"48333419":[],"48364163":[],"483642":[],"48399942":[],"48411213":[],"48420061":[],"4843772":[],"48441798":[],"48442601":[],"48455788":[],"48476997":12,"4848097":[],"484810":[],"484905":[],"48490503":[],"48494803":34,"48511333":[],"48521055":[],"48549903":[],"48550":41,"4859188":[],"48597324":22,"48697869":[],"486979":[],"48705697":[],"4870934":[],"48717555":38,"48786826e":[],"48788378":[],"48805429e":[],"48826894e":[],"48827791":[],"48843174e":[],"48853875e":[],"48867802":[],"48875568e":[],"48891141":29,"48900413":[],"48904404":[],"48931464":[],"489315":[],"48944878":25,"48994188":6,"48it":[],"49":[6,7,10,12,26,33,37,38],"49008744":[],"49018385":[],"49035757":[],"49074536":[],"49077038e":[],"49123502e":[],"49152":4,"49156266e":[],"49211524":[],"49224801":[],"49285045e":[],"49312535":[],"49342785":[],"493428":[],"49361065":[],"4937346":[],"49381549":[],"49386718":[],"49392067e":[],"4940954":[1,32],"49470005":[],"49503632":25,"49513019":[],"4951301940723654":[],"49529361":[],"49557088":[],"49584063e":[],"4959161509357395e":[7,34],"4959161509366685e":[7,34],"49608088":[],"49611541":[],"49742477":[],"49761526":[],"49768808":25,"49849206e":[],"49888671":[],"498887":[],"49897504e":[39,40],"4990":29,"49910481e":[],"4992":29,"49939493":[],"499394932852416":[],"4997":29,"49it":[],"4c4c7f":[10,11],"4d":4,"4f":7,"4pm":[30,32],"4y":9,"4y_i":11,"5":[0,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,22,25,26,29,32,33,34,35,36,37,38,39,40,41],"50":[0,2,3,4,5,7,8,9,10,11,14,22,32,33,35,36,37,38,39,40,41],"500":[2,4,5,7,10,11,14,35,36,37,38,40,41],"50000000e":[38,39],"50000455":6,"50000553":6,"50000718":6,"50000855":6,"50000969":6,"50001063":6,"50001142":6,"50001207":6,"50001261":6,"50001306":6,"50001343":6,"50001374":6,"500014":6,"50001414":6,"50001422":6,"50001439":6,"50001454":6,"50001466":6,"50001476":6,"50001485":6,"50001492":6,"50001498":6,"50001502":6,"50001506":6,"5000151":6,"50001512":6,"50001515":6,"50001517":6,"50001518":6,"50001519":6,"50001521":6,"50001522":6,"50001523":6,"50001524":6,"50001525":6,"50029371":[],"50100007":[],"50152186":[],"5018":29,"5020612":[],"50227564e":[7,34],"50233673":[],"50258788e":[],"502698":[],"50269821":[],"50306335":[],"50321091":6,"50327198":[],"50352245":[],"50414881":12,"50427787":35,"50429483":[],"504295":[],"50482139":[],"50505108":25,"50516278":[],"50530832":35,"50558976":[],"5057642":29,"506":1,"50600514e":[],"50610071e":37,"50638863":[],"506389":[],"506553":33,"50655336":33,"50655379":[],"50667812":[],"50714096e":[],"50744967":32,"5076963":[],"507d50":[10,11],"50846111e":[7,34],"50846112e":[7,34],"50850373":[],"50861923":[],"5086728":[],"50887105":[],"5088893":25,"50895251":[],"50911289":[],"50937003":[],"5095016":[],"50it":[],"50j":[14,37],"50x10":[2,40,41],"51":11,"510":[2,40,41],"51019911":32,"51045268":25,"5105332296929628":[],"51069279":[],"51069748":[],"510697484548":[],"51104913":[],"51136852":[],"51154388":[],"5116375":[],"51164347":[],"511888":6,"51262599":[],"51279028":[],"51279273":[],"5128288":[],"51302994":[],"51317506":[],"51364117":[],"51374050":35,"514219":33,"5142856":[],"51437058e":[],"51447004":[],"51470773":[],"514708":[],"51495078":12,"514951":12,"51512235":[],"51532469":[],"51546825":[],"51553626":[],"51561341":[],"51564682":[],"51611732":[],"51615779":[],"51643579":[],"51659846":37,"51690815":[],"51698843":[],"51709529":[],"51713064":22,"51728978":[],"5177783846":5,"51808934e":[39,40],"51857188":[],"51858116":[],"51860019":25,"5187643":[],"51914183e":[],"51928402":[],"51936974":[],"519370":[],"51937727":[],"51it":[],"52":[36,37,38],"52012715":[],"52031552e":[],"52038652":25,"52074297":[],"52087499":[],"52121734e":[],"521422502348998":[],"52154966":32,"521558":[],"52155823":[],"52173381":[],"52200578":[],"52211567":[],"522116":[],"52212442":32,"5222222222222223":[2,40,41],"52228066e":[],"52229337":[],"5223261":[],"52284329e":[],"52301447e":[],"52317963":[],"52359414e":[],"52362157e":33,"5238548":[],"52397611":[],"52407775":[],"52446101":[],"52480767":[],"52488688":[],"52488859":[],"52568928":[],"5257011":[],"52573336":[],"52606333":[],"5260801":[],"52621829":[],"52634209":12,"52635761":[],"52637646":[],"52654886":[],"52657934":[],"5267331":29,"526744":[12,33],"52694246e":[],"52698542":[],"527024":[],"52702419":[],"52706754":[],"527068":[],"52721412":[],"52729975":[],"52732966":[],"52757272":[],"52761872":[],"52780600":35,"52789388":[],"52806096":[],"52829142":29,"52874252":6,"52876551":[],"52888979":[],"5291335":[],"52927353":[],"52989287e":[],"52998985":[],"52it":[],"53":[10,36,37],"5303329":12,"53052247":[],"5305555555555556":[2,40,41],"53105728":[],"53125507":[],"531280":33,"53145384":[],"53149078":[],"532":[],"5321613":[],"53228519":[],"53235952":[],"53269627e":[],"53285379":[],"532854":[],"533":[],"53312817":[],"53332849":[],"53361935":32,"53397188":[],"534":[],"53480404":[],"53488344":[],"535":[],"53512750e":[],"53534008":[],"53585173":[],"535852":[],"53629564":[],"53640693":[],"53684024":[],"537":[],"53703498":7,"53723188":[],"53729411e":[],"53732988":[],"53738247":35,"5378811":12,"538":[],"5384615384615384":10,"53851084":[],"538511":[],"53869015":[],"538811":[],"5388115":[],"539":[],"53911457":[],"539261":[12,33],"53942728":[],"53996947":[],"5399913":[],"53it":[],"54":[7,10,29],"540":10,"54011398":[],"54039921":6,"54041041e":6,"54057582":[],"54150298":[39,40],"541605":33,"54175461":37,"54193001":[],"54214936":32,"54222783":[],"54237093":[],"5426270409877345":[],"54295853":[],"543939":33,"54393936":33,"54406885":[],"5443909":[],"544421":[],"54442191":[],"544422":[],"544439":33,"54453037":[],"54490887":[],"54593165":[],"54637219":35,"54725976":[],"54760497":[],"54804438":[],"54811237":[],"548263635652985":[],"54843215":[],"548432150659993":[],"54856099":[],"54867722":[],"54873252":[],"54874017":25,"54880148":[],"54910928":[],"54921401":[],"54928947e":[39,40],"54935794":[],"54977724":[],"54it":[],"55":[2,10,40,41],"55020405":[],"55020649":[],"55042779":[],"55084297":[],"55094784":[],"55095316":[],"551023":[],"55102329":[],"55148543":25,"55170845":[],"55179339":[],"55196383":[],"55208611e":[],"55225408":[],"55261137":[],"55386681":[],"55440601":[],"5544062":[],"55451651":[],"55460327e":[],"55542568":[],"5555555555555556":[2,40,41],"555617937832463":[],"55565884":[],"55582327":[],"55582742":[],"55643340e":[],"5567":7,"55678525":25,"55688514":[],"557795":[12,33],"55838":37,"55854694":12,"55862544":[],"55900177":[],"55916434":[],"55929651":38,"559791959661649":[],"55988542":[],"55996153":[],"55it":[],"56":[2,10,40,41],"560275":[],"56027537":[],"56033697":6,"560379":[],"56037913":[],"56042437":[],"56072658":[],"56102529":[],"56108455":[],"561410":[],"56141009":[],"56141925":[],"56198284":6,"5620646":[],"56237849":25,"56291214":[],"56342158":[],"564":10,"56408898":[],"56423957":[],"56423957069263":[],"564374":[12,33],"56453869":[],"56469864":23,"564759":[],"56475921":[],"56476512e":[],"565":10,"5653087":[],"56536":[1,32],"56541622e":[],"56576662":29,"56589683":35,"566":10,"56636616e":[7,34],"56665044e":[],"56685379":[],"56690562":32,"567":10,"56763673":[],"5676397":[],"56766321":[],"567753":[],"56775345":[],"56778296":[],"567783":[],"56783103":[],"56790195e":[],"568":[10,35],"56804783":[],"56832496e":[],"56850995":[],"56857739e":[],"5686930390258937":29,"56869304":29,"569":[2,10,40,41],"56912044e":[7,34],"56913182":32,"56922102":[],"56939714":6,"56947078":[],"56965674":35,"56967917":[],"56977352":[],"56990537":[],"56it":[],"57":[1,9,10,30,32],"570":10,"57000682":32,"57016227":[],"57080764":[],"571":[6,34],"571105947979344e":[7,34],"571105947979412e":[7,34],"57120988":29,"5715150087179857":[],"57192197":[],"57201944e":[7,34],"57212292":[],"57231587":22,"57252451e":[],"57261202":[],"57293306":[],"57324367":[],"57331229":[],"57334868":[],"5736973":[],"574465":[12,33],"57467546":[],"57468142":[],"57475388e":[],"57492326":[],"57502449":[],"57521514":[],"57584612":[],"575870409023631":[],"57599494":[],"576":35,"57604071":32,"57610367":[],"5769230769230769":10,"57735536":[],"57803254":[],"57819238e":[],"57833531":[],"57847325":[],"57852915":[],"57854115":[],"57881465":[],"57886743":[],"57914918":[],"57933547":[],"57966619":29,"58":[3,10,11,30,32],"58013829":[],"58084123":[],"58161787":[],"58202531":[],"58218587e":[],"5823186":32,"58257727":[],"58284836":[],"582987":[],"58298728":[],"58304607":[],"5833333333333334":10,"583595":33,"58379362":[],"58408765":[],"58452753":[],"5847256":[],"58492636e":33,"58493493":[],"58493681":[],"58556424":[],"58570955":25,"58583003":[],"58589719":[],"585982":12,"58598227":12,"58641507967355":[],"58735893":[],"58799372":[],"58825417":[],"58827236":7,"5888888888888889":[2,40,41],"58890784":[],"58914105":[],"58933143":[],"589841":[],"58it":[],"59":[3,10],"59001966":[],"59009792":[],"59030403":[],"59078383":[],"590784":[],"59088350e":[],"591317992":5,"59184268":[],"59188296":[],"59206051":32,"59215613":[],"59244881":[],"592766658048914":[],"59348521":[],"59354079":7,"59355752e":[],"59372982":[],"59388777":[],"59391196":[],"593912":[],"59411814":[],"594241":[],"59424119":[],"5944444444444444":[2,40,41],"59465457e":[],"59491404e":[],"59560497":[],"59630232e":[],"5965161e":[39,40],"59683811":[],"59698459":[],"59730078":[],"59741748e":[],"59770229":[],"59779819":37,"59833837":[],"598549916355722":[],"59854992":[],"59875207":[],"59879372":[],"59884169":[],"59898921":[],"59908591":[],"59908884":[],"59958693":25,"59963873":[],"59971727":[],"59it":[],"5cm":29,"5f":[9,37],"5x":9,"5y":9,"6":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,18,22,25,27,29,30,32,33,34,35,36,37,38,39,40,41],"60":[2,4,7,10,40,41],"60000":5,"60001225":[],"60026673":[],"60041459e":[],"600529":[],"60052901":[],"60084189":25,"60116388":[],"601164":[],"60116948":[],"60151468":[],"60160003e":[],"60181469":[],"60184333":[],"6019067271":5,"60193996":[],"6025456":[],"602546":[],"60271731":[],"6029291":[],"60293962":6,"60321647":[],"60342978":35,"60357007":29,"60375434":[],"6038642":[],"60418171":[],"60420593":6,"604213":[],"60421343":[],"60451189e":[],"604731":[],"60480445":25,"60489634":[],"60534421":[],"60543038":33,"60552651":[],"60568686":32,"60570744":[],"60575887":[],"60586595":[],"60588173":[],"60605083e":[],"60610368":23,"606251":[],"60625144":[],"60664448":[],"60672354":29,"60673226":12,"606760":6,"60720029":[],"60737006e":[39,40],"60740312":[],"60753606":38,"60762863":[],"60794187":[],"607942":[],"60837213":32,"6087346":[],"60900861":[],"60907486":[],"60915019":[],"60921896":[],"60929685":35,"60940356":[],"60943791":[25,32],"60999067":[],"61":[],"61069681":[],"61076053":32,"61083596e":[],"61106937e":[],"6111111111111112":[2,40,41],"61113781":[],"61130099":[],"611301":[],"61192203":[],"61192712":[],"61209378e":[39,40],"612939":33,"61321722e":[],"61325173":[],"613579":33,"61366611":[],"61382924":[],"613893902124586":[],"61395701":[],"61402242":[],"61414743":[],"614161296466206":[],"61468323":[],"614808":[12,33],"61501312":[],"6153846153846154":10,"61564092":[],"6156409232714415":[],"61565184":38,"61602014e":[],"6161104":[],"61621902":[],"6177137":6,"61774362":[],"61775176":35,"61860028":[],"618746":[],"61883807":[],"618982":33,"61974366":[],"61979664e":[],"62":[],"6200":32,"6201":32,"6202":32,"6203":32,"62033782":[],"6204":32,"62068649":[],"62113349e":[],"62149436e":[],"621584":[],"62158409":[],"621787":[],"62178739":[],"62198131":[],"62242138e":[],"62246687":25,"62275788":37,"622777986360822":29,"62277799":29,"62286539":[],"62288579":[],"62299126":38,"62299866":[],"622998661867777":[],"62323175":[39,40],"62351789":[],"62373464":12,"6243301":[],"62475001":[],"625":[8,36],"62551462":[],"62571719":25,"62573269":[],"62617788":[],"62620041e":[],"626635268":[7,35],"626678":[],"6266783":[],"6267112":[],"62671667":[],"626889e":22,"62714703":[39,40],"62716271":[],"62725226":34,"628":7,"62816974":[],"62817267":[],"62846663":[],"62863106e":[],"62863887":[],"62886648":[],"62894215":6,"62905281e":[],"62908496":[],"62918713":[],"62927207":33,"62954328e":[],"62977556":29,"62it":[],"63":[1,2,7,10,35,40,41],"63025821e":[7,34],"63030526":[],"63031478":[],"630315":[],"63037846":7,"63042739":[],"63046054":[],"63071772":[],"63116786":32,"63142399":[],"63142731":[],"63144217":[],"631471":6,"63162342":33,"631986":[],"63198618":[],"63204599":[],"63207408":[],"63209435":[],"63249532e":[7,34],"63307537":[],"63309898":[],"63322884":[],"6333349473431832":[],"63347711":[],"63353014":[],"63371136e":[],"63443371":[],"63461575":[],"63483713":[],"63490568":[],"63498144":6,"63500255":[],"63520537":25,"63542565":32,"63546236":[],"63546905":[],"63555267e":[],"63615006":[],"63619709":[],"63693246":[],"6373":[],"63731768":12,"637318":12,"6373454":[],"63760101e":[],"63797452":12,"63801791":[],"6380763200092493":[],"63833441e":[],"6383903":[],"63896182":[],"638962":[],"6389681":[],"63898112e":[],"63919355":[],"639194":[],"63928106":[],"63980097":[],"63980912":[],"63993855":[],"64":[2,4,5,14,25,32,37,38,40,41],"64009263":[],"64012627":6,"6401576":[],"640567":[],"6405671":[],"6407249":[],"64082047":[],"640x480":[7,14],"64192473":[],"64291046":[],"64297976":[],"64316192":35,"64322974":[],"64344541":32,"64361375":[],"64362404":[],"64373221":[],"64373635e":37,"64399916":[],"64408368":[],"64411666":[],"64412879":[],"6441397":[],"64425744":[],"64493447e":[],"64505146":[],"64516328":[],"64564368e":[],"646283":[12,33],"64630711":[],"6463440100796987":12,"64638876":[],"64642277":33,"64672839":[],"64689773":[],"64690121":[],"64713979":[],"64721597":[],"64731484":[],"64742912e":[7,34],"647473":[12,33],"6477208":29,"64796751":[],"64797382":[],"648131":[],"64813145":[],"64836333":[],"64857542":[],"64885265":[],"649":[],"64909048":[],"64911288":[],"649382":[12,33],"64952115":[],"64968814e":[],"64976691":[],"64it":[],"64x50":[2,40,41],"65":[2,9,10,40,41],"650":[],"65003213":[],"65082441":[],"651":[],"65147818":[],"65183066":[],"651831":[],"652":[],"65243388e":[],"65259487":[],"65293772":[],"65295017":25,"653":[],"6533977":[],"653398":[],"65354422":32,"6536392":33,"65396469":29,"654":[],"65410649":[],"65442354":33,"654424":33,"65475653":[],"65476419":[],"6548596":[],"655":[],"65525178":[],"65532045":[],"65543115":[],"65554958":[],"65571174e":33,"65591748":[],"65593984":[],"65599456":33,"656":[],"6561141":25,"65632442":22,"65641173":[],"6564482":32,"65651288":[],"6565507":37,"65672295":[],"65681217":[],"6568154596723088":[],"6568686180915178":[],"65686862":[],"65720152e":[],"6572612e":[39,40],"65738717e":[],"65743689":33,"65744898":[],"65751456":[],"65759127":22,"65759901":[],"6576130374029565":[],"65765268":25,"658":[],"6582728":[],"65833132":33,"65870313":[],"65885453":6,"65889641e":[],"65891389":33,"658914":33,"659":[],"65900052e":[],"65917558":[],"65944342":37,"65971184e":[],"65976951":25,"65it":[],"66":3,"660":[],"66003032":[],"66018725e":[],"66047885e":[],"66063786":[],"66064822":33,"66080313":33,"66130478":[],"66155431":29,"66204648":7,"66215333":[],"662153330760193":[],"66272614":[],"662836197064178":[],"66289235":[],"66294408":33,"6629598":[],"662960":[],"66339452":29,"66340579":32,"66342634e":[],"66363129":[],"66365453e":[],"6638":10,"664314":[],"66431407":[],"66456459":[],"66487438":[],"66498292":[],"6652177":33,"66533454e":[],"66560":10,"66575697":[],"66603461":[],"6663569":[],"66694492":[39,40],"667":10,"66707072e":[],"6671767":[],"66729685":[],"66734487":38,"66757105e":[],"66763324e":[],"66774691":[],"66809369":[],"668172":33,"66841959":[],"6690108":[],"66951679":[],"6696885":[],"66990322":[],"669908227073521":[],"66it":[],"67":[3,10],"67015946e":[],"6702":[],"67036507":[],"67047975e":[7,34],"67088558":[],"67100236":[],"67199457":[],"67231298":[],"67238764e":[],"67249437":32,"672721":33,"672759":[],"67275915":[],"67313212":[],"67314874e":6,"67364995":[],"67374815":[],"674":35,"67435043":[],"67464946":[],"67477123e":[],"67490857":37,"67495943":[],"67509321":[],"67511113":[],"67524588":[],"67527216":[],"67555087":37,"67561021":[],"67747547":[],"67819487e":[],"67834752":[],"67849271":32,"67875366":[],"678754":[],"67925887":[],"67955629":[],"6795562921258607":[],"67982841":[],"67it":[],"68":3,"68008931":32,"6801":[],"68012971":32,"68029423":[],"68043896":[],"6805151":[],"68072161":[],"68074588":37,"6809195":[],"680920":[],"68092237e":[],"68110474":[],"68129036":[],"68163369e":[],"6816947":[],"68192193":6,"68257097":[],"68258595":[],"68281739":[],"68286725":33,"68291639":32,"6831505":[],"68352799":[],"68402334":[39,40],"68532803":[],"68534263e":[7,34],"68542204":6,"68591105":[],"68609856":[],"68637642":[],"68640031":[],"68655456":[],"6869":10,"68735473":[],"68748147":[],"68749642":[],"68753434":[],"6876042":[],"68763036":[],"68822559":[],"688226":[],"68824222":[],"68826552":[],"68827947":[],"68830164":[],"68857788":[],"6887363571":5,"68879472":[],"6888043337746685":[],"68891428":[],"68929213e":[7,34],"6893421391051477":[],"689519":[12,33],"68953877":[],"689539":[],"68962656":[],"68989446":[],"69":[3,10,29],"690":10,"6901769":[],"69030225":[],"690617":33,"69069n_":29,"69089532":[],"691":[2,40,41],"691364":[],"69136414":[],"69164986":32,"69169665":[],"69182892e":[],"69203997":[],"69227923":[],"69230769":37,"6923076923076923":10,"69243548":[],"69247718":[],"6927554":[],"69303453":[],"693035":[],"69306119":[],"69311221":[],"6932681":[],"69391797":[],"69417174":[],"69432059e":[],"69435454":[],"69438869":[],"69463406":[],"69465386":[],"69484813e":6,"69484973":[],"69496248":29,"69504801":7,"69515531":[],"69547403":29,"6955259":[],"69566966":[],"6957182489166325":[],"69634577e":[7,34],"69678073":[],"696781":[],"69695259":6,"697346":[],"69734609":[],"69734853":12,"697349":12,"69749114":[],"69756791":[],"69768266":[],"6980":[22,37,38],"69806017e":[],"69811070e":[],"69824402":[],"6988905":[],"69892546":[],"698996":[],"69899609":[],"69903406":[],"69908819":[],"69925648":[],"69978197":[],"699782":[],"69983227":[],"6999536":12,"69it":[],"6n_":29,"6pm":32,"7":[1,2,3,4,5,6,7,8,9,10,12,13,14,15,16,22,25,27,28,29,31,32,33,34,35,36,37,38,40,41],"70":[2,7,8,10,36,40,41],"70015185":[],"7002164":[],"70037324e":33,"70039784":[],"70040367e":[],"70090777":[],"700908":[],"70135856":[],"701370":6,"70159137":[],"70175778":[],"70241548":[],"7024611":25,"7026722043092946":[],"7026908764394864":[],"70277293":25,"7029975":[],"70301068":[],"70302827":[],"7033198":[],"70334935":[],"70362677":[],"70367841":32,"70408202":[],"70469324":[],"70474145":[],"70485195":[],"7049679":[],"70499955":[],"70514808":[],"70530438":[],"70587399":[],"705946":[],"70594608":[],"70630967":[],"70653767":5,"70665448":[],"70704566":[],"7070883":[],"70710678":[6,33],"70711022":[],"70711514":32,"7074705":[],"70751625":[],"7080469":[],"70811405e":[39,40],"70832814":6,"70857635":[],"7086067479626619":33,"70885855":[],"70928399":[],"709284":[],"70944365":[],"70981984":29,"71":[2,3,40,41],"71007076":[],"7100707615589594":[],"71059619":[],"7112242518460867":10,"71129677":[],"71142161":33,"711422":33,"71143744":25,"7119":10,"712018":[12,33],"71261809":[],"71263085":[],"71263851e":[39,40],"7127128":[],"7129466":[],"712947":[],"71314288":[],"713143":[],"71332117":[],"713763":[],"71376303":[],"71380438":[],"71384267e":37,"71409384":[],"71441086":[],"71460518":[],"71469351":[],"7150195":[],"71505132e":[],"71507714e":[],"71524686":[],"71527295":22,"71582784":[],"71633658":[],"7163835161938888":[],"71640303e":[],"71664506":[],"7167077":[],"71672742e":[],"71672791":[],"71737253":33,"71761101":33,"717794":[],"71779401":[],"718165":6,"71838601":[],"71860732":[],"71957514":[],"7198234":[],"72":3,"72033728":[],"7204561":[],"72047148":[],"72055288":[],"72066289":[],"720889407643293":[],"72174172":12,"72183526":[],"72184754":25,"72219757":[],"72236654":[],"72271878e":[7,34],"7228852":[],"72288611":[],"72309979e":[],"72315862":[],"72364915":[],"72365764":[],"7236674":6,"724":4,"72407267e":[],"72427854":[],"72444668":[],"7249499":[],"72521196":[],"725212":[],"725256":[],"7252563":[],"72577807":[],"72589774":[],"72598313":[],"726017":[],"72601722":[],"72602818":[],"72690169":[],"72710494":[],"727132":[],"7271324":[],"72744372":[],"72754455":25,"72758060e":[],"72770687":[],"727707":[],"72771857":[],"72848736":[],"72859758":6,"72863433":[],"72876728":[],"72882234":[],"72952595":[],"72962885":[],"72964493e":[],"72965547e":22,"72966563":[],"729666":[],"72981762":9,"72992196":[],"72993254":[],"73":[7,10,35],"73018893":[],"73031999":[],"730364":[],"73036439":[],"73062648":[],"73063053":[],"73068535":[],"73071831":[],"731000":33,"73122034":[],"73176319":[],"73203674":[],"73204357":[],"73252708":[],"73265885":[],"73299464":[],"73305640e":[],"733096":33,"73337355":[],"7333854378748542":[],"73374853e":[39,40],"7337962":25,"73380463":[],"73390423":32,"73397147":[],"73423383":[],"73425394":[],"735490390687699":12,"73550536":[],"73565966":[],"73578772":[],"73594414e":[],"73595198":[],"73614157":[],"736282":[],"73628249":[],"73646130e":[],"73690387":37,"737184":[],"73718419":[],"73741131e":[],"73785069":[],"737851":[],"73853838":[],"73854926":[],"73876416607927":[],"73888203":[],"73958682":[],"739705":[],"73970542":[],"73986200e":[],"73994248":[],"74":[7,35,37,38],"740":10,"74028787e":[],"7403629637766833":[],"74038232":[],"74038873":[],"74075864":[],"74081822":9,"740859":12,"7408594":12,"74100159":[],"74101067e":[],"74138002e":[],"74171049":[],"74180248":[],"74208907":[],"74259663":[],"742597":[],"7426664":29,"74272285e":[],"74291603e":[],"74336924":12,"74384858":[],"74384949":33,"74407544":[],"74412221":32,"74463241":[],"74471533":[],"74499034":[],"74506271":[],"74523085":29,"74533876":[],"7458714725617228":12,"74607851":33,"74613334":[],"7461895":[],"74632311":[],"74692641":[],"74724414e":[],"74724774":37,"74761815":[],"74803743":38,"74829661":33,"74840212":6,"7487814387207385":[],"74878144":[],"74883972":[],"74900488":[],"74957448":[],"749765":33,"7499639":[],"75":[6,7,9,10,12,26,33,35,38,39],"75015599e":[],"75018686":[],"750187":[],"75025744":[],"750445":33,"75050135":33,"75080943":[],"7509571":[],"7510197":[],"7511273":[],"75131415":[],"75138064":[],"75146945":[],"7515117":[],"751699":[12,33],"75170092":6,"75216828e":37,"75217508":[],"75240139":[],"75269037":33,"75300068":[],"75345893":[],"7539148284817965":[],"75406226e":[],"75409398":37,"754130":[],"75413023":[],"75424998":[],"75433448":[],"7546022135629743":[],"7546682196464342":[],"75488337e":[],"75532041":[],"75573622e":38,"75576555":33,"75592582":37,"75600095":[],"756352":33,"75654448":[],"75669703":[],"75674251":[],"75677341":[],"756828":[],"75682834":[],"75733813e":[],"7573694380469245":[],"75737136":[],"75760403":[],"7577":35,"75770568":33,"7577113351177733":[],"75956076":[],"75956617":[],"75978215":[],"76":[10,30,32,36,37],"76030841":[],"76065256":[],"76120718":[],"76134837":[],"76140219":[],"761574":[],"76157429":[],"76159011":[],"76183628":32,"76190591":34,"76262775":[],"76277254":[],"762891":[],"76289128":[],"76329289":[],"7642448":[],"764245":[],"76426396":[],"765":[8,36],"76527062":[],"76533281":[],"765333":[],"76542156e":[],"76561858":25,"7658599":[],"76587916":25,"76601702":29,"76612827":[],"766163196245293":[],"76632434":[],"7663967":[],"76686148":[],"76688358":[],"76690383":35,"76692036e":[],"767285":[],"76728511":[],"767367275553824":[],"7674217625964539":[],"76751116":38,"76790092":[],"76807288":[],"76852722":[],"76855921":[],"76880575":[],"76887316e":[],"7692307692307693":10,"76936315":6,"76940759":[],"76942272":[],"7694444444444445":[2,40,41],"76996262e":[],"77":[10,30,32,37,38],"77014839":[],"77030637":[],"77037707":[],"77039278":[],"77106886":[],"77112374":25,"771128":[],"77132083":[],"77152076":6,"77152924":[],"7718":10,"77199379e":[],"7721643":25,"772588722239781":[37,38],"77306239":[],"77313451":25,"77352152e":[],"77354301":[],"774498":[],"77449804":[],"77465911":[],"77482712":[],"77497266":[],"77507653":[],"775204":[],"77551387":[],"7755649":[],"775565":[],"77559615":[],"77579275":[],"77594586":[],"77596142":[],"77597019":[],"77602426e":[],"77618181":[],"77626954":[],"776322":[],"77632217":[],"776352510835556":[],"77635684e":[],"77636e":[37,38],"77647539":[],"77652198e":[],"77659961":[],"7766443":[],"77699722":[],"777112373675558":[],"77714169":9,"77725472":[],"777255":[],"77803692":[],"7782028952":5,"77829399":[],"77845284":[],"77873845e":[],"77878621":25,"77959897":[],"77982629e":[],"77994763":32,"78":[],"78009660e":33,"78010668":[],"78025829":[],"78144936":29,"78156479e":6,"78157908":[],"78184120e":[7,34],"78186404":[],"78198192":[],"78243922":[],"78251088":[],"78254537":[],"78261152":29,"78278086":[],"78326199e":[],"78338459":[],"78340902e":[],"78380113":25,"78385675":[],"78459458":[],"78477759e":[],"78575679":29,"78596158":[],"78617455":[],"78630865":[],"78646096":[],"78671736":[],"78714005":[],"7873282":[],"78745927":[],"78758569":[],"78762558":[],"78810857":38,"78850066":29,"78876155e":[],"78888045":[],"78892891":[],"7889793":32,"78904605":[],"78941903":6,"78945959":[],"78949699e":[],"79":[],"79010601":[],"79041505":[],"79086674":[],"79102612":[],"79111643":6,"79135536e":[],"79171202":[],"79247398":[],"79251831":[],"79280475":[],"793167":33,"79401569":[],"794282":[12,33],"79443618":[],"79469098":[],"794749":12,"7947491":12,"7949388":38,"79503363":[],"79522963":[],"7953443":34,"79540402":[],"79558414":[],"795584143123602":[],"79558423":[],"79619113":[],"79622905":[],"79623487":[],"796235":[],"79628257":[],"79633065":35,"7963438":[],"79673108":[],"79718864":[],"79778008":7,"79787194":[],"797872":[],"797e":7,"7981908":[],"79849321":[],"79866484":[],"79870689e":[],"79873884e":[],"7989356":[],"79898903":[],"79942527e":[],"7d7d58":[10,11],"8":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,19,22,23,25,26,27,29,32,33,34,35,36,37,38,39,40,41],"80":[1,2,6,7,9,10,33,34,35,40,41],"800":[5,8,36],"80011603e":[],"80033616":[],"80056922":[],"80066012":[],"80072356":[],"800724":[],"800782161095708":[],"80118954e":[],"801356":[],"80147746":32,"8030191":[],"80311453":[],"80354994":7,"80369722e":[],"80373997":[],"80374623":[],"80410179":[],"80413696":[],"80421021":25,"80453713":[],"80460179":35,"80469739":6,"80499859":[],"8055555555555556":[2,40,41],"8056560957923695":[],"80576215":[],"80609615e":[7,34],"80609616e":[7,34],"80633871":[],"80637385":[],"8063779":[],"80668002":[],"8073322":[],"80740939":[],"80750113":[],"80835002":32,"80845707":6,"80846898":[],"80847477e":[7,34],"80848093e":[],"8084934":[],"80853029":[],"80855691":[],"808557":[],"80873785":[],"80907609":[],"80979265":[],"80980728":[],"81":[2,40,41],"81006889":[],"81007381":[],"81012733":25,"810150":[],"81015037":[],"81035098":[],"81040897":[],"810409":[],"81042619e":[],"81048318e":[7,34],"81091776":[],"8110093e":[],"81160425":6,"81208109":[],"81218742e":[],"81246697":[],"81283649":[],"81318307":[],"81329201":32,"8133":[],"81333808":7,"81337164":[],"81385326":[],"813957":[],"81395716":[],"81425129e":[],"81478187":[],"81479491":[],"81506255":[],"81522211":[],"81559586":[],"81592028":[],"81598832":[],"815am":[30,32],"81622636":[],"81633628":12,"81642349":[],"816454":33,"816847":33,"81684805":[],"81707832":[],"81737781":[],"81818338":[],"81825652":25,"81826799":[],"81839897":[],"8184887":[],"81889188":[],"81919129":[],"81924504":[],"819253":[],"81925302":[],"81931348":[],"81947005":[],"81979166":[],"819805":[],"81980509":[],"81984817":32,"81987609":[],"82":[],"82004436":[],"82060913e":[],"8207274870895702":[],"82079885":[],"8211787":[],"82121282":[],"82130663":[],"82133401":25,"82146046":[],"82161167":[],"82198978":6,"82204612":25,"82212861e":[],"8221979401496963":[],"82242798":[],"82247747":[],"82259916":[],"8227":[],"8232102961414203":[],"823242":[],"82324216":[],"823417":[],"82341746":[],"82360724":29,"82376174e":[],"82418008":[],"82418352":[],"82435177e":[],"8244":[],"8248473":[],"8250103":[],"82528238":[],"82533209":[],"82551764":[],"82559478":[],"82561703":[],"82620196":[],"82623709e":[],"8265786":6,"82677051":38,"82688363":[],"82749474":29,"82757017":[],"82802004":[],"82832579":[],"82844656":[],"82864173":[],"82891669":38,"82914126":[],"829252":[],"82925222":[],"82934918":25,"83":[],"83020679":[],"8305555555555556":[2,40,41],"83146596":35,"83164611e":[],"83203197":[],"83220985":[],"83230774":[],"83231208e":[],"83234002":[],"83294718":[],"83317854e":[],"8332972":[],"83363025":[],"83369175":[],"83377498e":[],"83408369":32,"83429536":[],"83463634e":[],"83466807":[],"83512277":6,"83519517":[],"8352834":[],"83641435":[],"83655756":[],"83671148":[],"83681263":[],"83693088":32,"83718206":[],"83741078":[],"83759075e":[],"83785062":32,"83816147":[],"83823317e":[],"83824223":[],"83870189":7,"8387133":[],"83884553e":[],"83915415":25,"839957":[],"83995705":[],"84":[],"84019294":[],"84047912":[],"8406741e":[],"84078641":[],"84155431":[],"84159992":[],"842":[],"8423144":[],"842436":33,"84254917":[],"8426271":[],"84269323":32,"843":[],"84338322":[],"84351715":[],"84355903e":[2,40,41],"844":[],"84401886":[],"84416298":[],"8443626e":[],"84443254e":[2,40,41],"84478178":37,"84514616":[],"84581375":[],"846":[1,8,36],"84605507":[],"8461538461538461":10,"84629286":[],"84634327e":[],"84659427":[],"84661230e":[],"84668873":[],"847":[1,8,36],"84707251e":[],"84711297":[],"84726382":[],"8473429":12,"847343":12,"84745063e":[],"84745525":32,"8475889":[],"84767088":[],"8476874":[],"84780262":7,"84789579e":[],"848":[1,8,36],"8485":[],"84858":35,"84862355":[],"84869686":25,"8488":[],"84907109":[],"84908687":[],"84923989e":[7,34],"8496945":38,"84972787":[],"84973247":[],"84984612":[],"84985833":[],"84992628":38,"84993546e":[],"84994524":6,"85":[2,10,40,41],"85005989":[],"85014487":[],"850164":6,"85018556e":[],"85025162":[],"8502729":12,"85035562":[],"85043503":[],"85049285":32,"85050649":[],"8505072899794293":[],"85064942":[],"85068028":[],"85079059":[],"850859":6,"85085928":6,"8509859":32,"85108001":[],"85112594":[],"85142265":32,"8517571":[],"85189083":25,"85222243":[],"85259666e":[],"85261484":[],"85263220":7,"85276538":[],"85278920e":6,"8529175":[],"85293303e":[],"85294194":[],"85298956":[],"853":[],"85378708":[],"85408468e":37,"85422602":[],"85427641":[],"85437905":[],"85551266e":[],"85601992":6,"85613835":[],"85625134":[],"85703369":[],"85711016":[],"85714286":0,"85724692":25,"85777073":[],"857771":[],"85786373":[],"858":10,"8583333333333333":[2,40,41],"85859459":[],"85866294e":[],"85871255":[],"85897711":[],"85945208e":[],"85954931":[],"8595836":[],"85975973e":[],"86":[],"86009093":[],"86013055e":[],"86016288":[],"86030593e":[],"8605188":[],"86062977":7,"86066395":[],"86067818":[],"86070736":[],"86084064":[],"86117291":6,"86123808e":[],"86134827":6,"86138568":22,"86143379":[],"86145244":12,"8614682":[],"86174943":[],"86192682e":[],"86197958":[],"86215487e":[],"86240935":[],"86247117":[],"86267497":[],"862675":[],"86273371":34,"86274590e":[],"86284277":[],"8631855":32,"86346614":[],"8638888888888889":[2,40,41],"86416941":[],"86434639":[],"8644215":[],"86445334":[],"86458368e":[39,40],"86459987":[],"86493273":25,"865":[],"86506494":[],"86519139":[],"86567539":[],"866":[],"86602331":[],"86611894":[],"8662":[],"86629161":[],"86630":10,"86632028":[],"86639832e":[],"86643649":[],"86660944":[],"86664528":[],"8666666666666667":[2,40,41],"86666667":0,"86668761":[],"86696026":[],"866960260598739":[],"867":[],"86737671":[],"86757762":[],"86799623":[],"86810":10,"86833907e":[],"86861591e":[],"86867512":[],"86868711":[],"86885547":32,"86891741":[],"869":[1,8,36],"8691787":[],"86943859":[],"86978877":[],"87":10,"870":[1,8,36],"87001185":[],"87020155":[],"8702784034":5,"870337":[],"87068171":34,"87070464":[],"870704644539707":[],"87072815e":33,"87093752":[],"871":[1,8,36],"87148597":[],"87169607e":[],"87171492":[],"87186187":[],"8720648":[],"8722222222222222":[2,40,41],"87228174":32,"87231849":[],"87238538":[],"87323528":[],"87340054":[],"87349626":[],"87381451":6,"87399194":[],"8745559":38,"8746702":[],"87473452":[],"87478387":25,"87494456e":[],"875":[2,40,41],"87584434":[],"8758851":[],"8759":[37,38],"87616945":[],"8762":[],"87627342":35,"87633005":[],"87637891":[],"87661362":[],"87696461":[],"877":7,"87700882":[],"87732748":[],"87736602":25,"87740526":[],"87788573":[],"87818331":[],"87830277":[],"87878356":[],"87884833":[],"87894459":[],"878945":[],"87981676":[],"88":[32,37,38],"8800906083513416":29,"88009061":29,"88046261":6,"88055343":[],"8805555555555555":[2,40,41],"88064505":[],"8809429":[],"88135577e":[],"88147037":[],"88157423":[],"88168312e":[7,34],"8817842e":[],"88182591":33,"88208835":[],"88236598e":[],"8825782":[],"88289263":[],"882972378790114":[],"88297238":[],"88323026":33,"88333467":[],"88336879":6,"8834":[],"88355913":[],"88377332":[],"88415819":[],"88445111":32,"884669":33,"88477745e":[],"88501323":[],"88514106":[],"88518":[],"88529063e":[7,34],"88551766e":[],"88569607":[],"88594444":[],"8859754":25,"88613493":33,"88614317":[],"8861811":[],"88632765":[],"88639567":[],"88653291":[],"88685415":[],"88691234":[],"88726714":[],"88738394e":[],"88809826":[],"88860399":[],"8888888888888888":[2,40,41],"888908":6,"8889085":6,"88930323":[],"88960947":[],"8896094720230128":[],"89":[1,10],"890033":[],"89003324":[],"8901199":[],"89017198e":[],"89044267":[],"89054207":[],"89058005":[],"8905826":[],"89069029":[],"8908983":[],"89100924":[],"89126914e":33,"89155606":[],"891559":[],"89155934":[],"8917148":[],"891715":[],"89184351":[],"89211256":22,"89219980e":[],"89222305":[],"89229722":[],"89288636":12,"8931686":37,"8932188":[],"89378344":[],"89408972":[],"89410423":6,"8944444444444445":[2,40,41],"89457785":[],"89463761e":[],"89527291":[],"89533642":[],"89540936":[],"89543945":[],"89563074":[39,40],"89576555":29,"8962":[],"89654756":[],"89676583":[],"89692576":[],"89708415":[],"89710385e":[],"89714357":[],"89724282":[],"89730533":[],"89755794":[],"89766337e":[],"89770892":[],"8978":[],"89790867":[],"89845198":[],"89876098":[],"898761":[],"89961861":[],"89989636":[],"8f":[7,35],"8g":[7,35],"8n":25,"8x8":[2,40,41],"9":[0,1,2,3,4,5,6,7,8,9,10,12,13,14,15,19,22,23,25,26,28,29,30,32,33,34,36,37,38,39,40,41],"90":[2,7,10,40,41],"90075537":6,"90105594":29,"9011":7,"90114639":[],"90135464e":[],"90137221":[],"90137794":[],"90140097":[],"90162133":32,"90168941":[],"90182442":[],"90210098":32,"90220243":6,"90232941":25,"90266948":6,"90273253":22,"9027777777777778":[2,40,41],"90281987":[],"90300219":[],"90300704":[],"90304177e":[],"9031162":22,"9040":10,"90407685":[],"90475506e":[7,34],"90536337":[],"9054":35,"9055555555555556":[2,40,41],"9057119872872414":32,"906589":[],"9065894":[],"906747":6,"90717653":[],"907177":[],"9074157":[],"90751311":[],"90800497":[],"90842732":[],"90860768":[],"90871271":[],"90885733e":[],"90894281":[],"90903158":[],"90909173":[],"909092":[],"9092793":[],"91":[30,32],"910":10,"910294":[],"91029443":[],"91029782":[],"910297822789896":[],"91099021":[],"91100141e":37,"9111111111111111":[2,40,41],"91114704":[],"91128596":6,"91152267":[],"91193672":[],"911937":[],"91247848e":[39,40],"91264011":[],"91270333":[],"91312563":[],"91322403":[],"91365263":[],"91410559e":[],"91416944e":[],"91417133":[],"91433102":[],"91434750e":[],"914672":[],"91467223":[],"91483251":[],"91492986e":[7,34],"91496317":25,"91507022":25,"91520631":[],"91527102":[],"91546928":[],"91568958e":[],"91624469":[],"916245":[],"91648657":[],"91650453":[],"9166666666666666":10,"91669165":[],"916692":[],"91672129":[],"91692081":[],"91722512e":[],"91744988":[],"91752854":[],"91760278":6,"91811422":[],"91812702":6,"91854134":[],"91864509":[],"9187554":[],"918785":[],"918992":33,"91933003":[],"919503":[],"9195033":[],"91961463":[],"919895":12,"91989533":12,"92":[7,10,30,32],"92003207":[],"92004534":25,"92021197":[],"9202867":[],"9206617":[],"92080432":[],"92159388":[],"92169638":[],"92192012":[],"9225384545636204":[],"922622":[],"92262217":[],"92292668":[],"92301743":32,"9230769230769231":10,"92308585":[],"92308867":[],"92313548e":[39,40],"92319842":[],"92326255":[],"92341148":[],"92353194":[],"92370959":[],"92376655":[],"924011":[],"92452609":[],"92458892":[],"92466997":[],"92474572":[],"924746":[],"924e":7,"925":[2,40,41],"92507116e":[2,40,41],"92512799":[],"92527681e":[],"92552059":[],"92578916":6,"92599642":[],"92604454":[],"92622842":[],"92656908":29,"92661467e":37,"92663007":[],"92665802":[],"92678221":[],"92713642":[],"9271707090281667":[],"92719687":[],"92756256":34,"92761972":25,"92768548":[],"9277777777777778":[2,40,41],"92779167e":[],"92808577":[],"92857143":0,"928769588424018e":[],"9288904":[],"92891949":[],"92892022":[],"92901115":[],"92905422":[],"92910808":[],"92921498":[],"92929225e":[],"9294527":[],"92991181":[],"93001752":25,"93019783":[],"930536":[],"93053605":[],"9305555555555556":[2,40,41],"93059014":[],"9306628":[],"93074023":[],"9308197":25,"931":[1,32],"93148781":[],"93155188":6,"93158979":6,"93159699":[],"93164276":[],"93164299":25,"93237862":[],"932382":[],"93272428":[],"93277949":[],"93289669":[],"93293067":[],"933":[6,34],"93325371e":[39,40],"93366198":[],"933662":[],"93397254e":[],"93416236e":[],"934339":12,"93460701":[],"9349":[],"93492130e":[7,34],"93518383":[],"93519647":[],"93597168":[],"935972":[],"93601008e":33,"93602214":[],"93619369":[],"9366":[],"93668828":[],"936877889972962":[],"937":29,"937057":[],"937082":33,"93708805":[],"93709777e":[],"93787605":[],"93799826":6,"938":29,"9387":10,"9388939e":[],"93891335e":[],"939":[1,29,32],"93900569e":[],"93909866":[],"93937564":[],"93942872":[],"93967766":33,"93968601":[],"93969971":[],"93970238":[],"93987367":[],"93989887":[],"94":[0,10,29],"94008244":[],"94027059":[],"94033034":[],"94040981":37,"9405":[],"94082113e":[],"94091671":[],"94092702":[],"94096731":[],"94099173":[],"94114646":[],"94121002":[],"9412311":[],"94141801":[],"94217756":[],"942178":[],"94221383":[],"94226022e":[7,34],"94255277":[],"94272259":[],"94273542":33,"942739":[],"94284104":6,"94285762":[],"94291287":[],"94320205":6,"94335":[],"9434":[],"94388948":[],"94419718":[],"9444444444444444":[2,40,41],"944557":[],"94455739":[],"94463114":[],"94466211":[],"94484047e":[38,39],"94485679":[],"94493877":[],"94495682":[],"94499279":37,"9450635":[],"94533329":[],"9458396":[],"94591015":[25,32],"94607554":25,"946167":[],"94616709":[],"94623184":[],"94625725":[],"946276":[],"94627617":[],"946372":[],"94637206":[],"94639099":12,"94652":[],"94662523":[],"946957":6,"94704941e":[],"9472222222222222":[2,40,41],"9473":7,"94735263":[],"947607":[],"94766419e":[],"94781064":34,"94802663":[],"94805555":[],"9482527":6,"9485531":[],"948638":6,"94863824":6,"94897376":22,"9495484":[],"94963563":[],"94973606":[],"95":[2,10,12,35,40,41],"95000259":[],"95001004":[],"95008046":7,"95033155e":[],"95039348e":[],"950423":[],"95053694e":[],"950873":[],"95115528":[],"95132723":25,"95152814":[],"95162248e":[],"95166920e":[],"9518022":[],"952063":[],"95206335":[],"95226607":[],"952268":12,"95226818":12,"95231424":6,"952387":33,"9525328":[],"9527777777777777":[2,40,41],"95284275":6,"95289375":22,"953065564":[2,40,41],"95317772":38,"95338273":[],"95343966":[],"95351665":6,"95354276":[],"95365435":[],"95395895":[],"953959":[],"954":29,"95400323":[],"954148":12,"95462534":[],"95482298":[],"954823":[],"95497813":[],"95511792":33,"955118":33,"95518112":[],"9555555555555556":[2,40,41],"95563211":[],"95568161":22,"95578775":[],"955820c21e8b":5,"95611042":37,"95647867":[],"956563":[12,33],"95684892":6,"95703":[37,38],"95712462":[],"95715036":[],"957171":32,"95719454":[],"9573":7,"95743466e":[],"95744254":[],"95745476":[],"95745651":[],"957549":[],"95766504e":[],"95770298":[],"95776346":[],"95793988e":[],"9579870417283":23,"958228616652075":6,"95842106":[],"95845431":[],"95872127":[],"95881375":[],"95882223":[],"95882586":25,"95889379":[],"95890587":[],"95906059":37,"959076":[],"95935439":[],"9594":[],"96":[7,10,12,35],"960":29,"9601304850035702e":[7,34],"9601304850073734e":[7,34],"96015381":[],"96024953":6,"96029461":32,"96037553":22,"96054705e":[],"96057795e":[],"9606":[],"96067118":[],"96075051":37,"96080358":[],"96084123":25,"96084663":6,"961":29,"961042":[],"961423":[],"96142343":[],"96154578":[],"96157921":33,"96164155":[],"96175251":[],"962":29,"9623":[],"96231225":[],"96241501":[],"96251859":[],"96282505e":[],"96301764":[],"96305058":[],"963187":[],"96330728":22,"96332346":37,"9637117593816477":[7,34],"96379806e":[],"9638104":22,"96385782":[],"96385798":[],"963858":[],"96391888":[],"9639885":[],"9640435":6,"96414331":[],"96417888":[],"96432396":[],"96461948":[],"9647801":[],"9649652536":5,"96504394":[],"96544479":[],"96547946":[],"96548194":[],"965548":33,"9656484356890704":[],"96564844":[],"96581896":[],"96585824":[],"96631321":33,"96660465":[],"96688672":6,"967":[],"96734276":25,"967388":[],"96738845":[],"96745096":[],"9674916":6,"96762953e":[],"96777468":[],"967809":[12,33],"96799937":[],"96801916":22,"96826039":[],"96850191":[],"96853248":[],"968650":[],"96865004":[],"96866031":[],"968704":[],"96870431":[],"96894842":[],"96897719":[],"96899218":29,"96907179":32,"96916206":[],"96917669":[],"969190":[],"96919021":[],"96919836":[],"9692":[],"9693":[],"969413":[],"96961052":[],"96963458":[39,40],"96991192":[],"97005689":6,"97023479":[],"97065359":35,"97068015":[],"970827":[],"97082748":[],"970965":[],"97108e":[37,38],"97120565":[],"9712983":[],"9716":35,"97165491":[],"97169918":[],"97183244":[],"97190351":[],"97202371":[],"972024":[],"97206657":[],"972067":[],"972149":[],"97218946e":[],"9722222222222222":[2,40,41],"9722424":38,"97228653":[],"9722865317690382":[],"97235642":[],"97243128":6,"97243946":[],"97252343":[],"972533":[],"97253307":[],"97272223":[],"97286326":[],"97300836":6,"973118":[],"97311824":[],"97321835":32,"9734":35,"973663":[],"97366347":[],"97383031":[],"97435908":[],"97477277":29,"97480604":[],"97482498":[],"974825":[],"97486336":[],"97497404e":[7,34],"97498674":[],"975":[2,40,41],"97507735":6,"97512176":[],"975484":[],"97548406":[],"97570392":[],"97622642":[],"97626519":[],"97628064":[],"97630233":35,"97664837":[],"9767506308987675":[],"97680846":[],"97683575":[],"97696399":[],"977":[],"97705827":6,"97707044":[],"97716508":[],"97758848":6,"9777777777777777":[2,40,41],"978":35,"97802245":[],"9780387310732":31,"9780387848570":31,"9781098134174":32,"9781492032632":31,"9781801819312":32,"97836358":[],"9784055938098051":29,"978553":6,"97898392":[7,33,34],"97906022e":33,"9792":[],"97926491":6,"97935648":[],"9794096":[],"9796":[],"979656":[],"97965627":[],"97979891":7,"98":[1,2,10,39,40,41],"980":10,"9805555555555555":[2,40,41],"98091621":6,"98101481":25,"98108401":[],"981321":33,"98139097":6,"98140006e":[],"98155457":29,"981554571728081":29,"98190064":[],"98232596":[],"98244726":32,"98246764":[],"98261559e":[],"98271562":[],"98275501":6,"98281205":37,"98292239":[],"983310":33,"9833205":[],"98338288":[],"983383":[],"983450":[],"9834505":[],"9838929":[],"98413059":6,"98413606":[],"9841870203276504":[],"98454786":6,"98473694":[],"98474769":[],"984748":[],"98481245":[],"985":29,"98533636":[],"98556236":[],"98556486e":[],"98566191":6,"985725":[],"985725003240627":[],"986":29,"9861111111111112":[2,40,41],"98619989":[],"98621772e":[],"98636199":[],"986362":[],"98650457":[],"98654811":[],"986699":6,"98680716":6,"98716878":6,"98723376":[],"98740373":[],"98751068":[],"98770503":[],"9878":[],"98806419":[],"98808176":6,"98822371":7,"9888888888888889":[2,40,41],"98893579":[],"988996":[],"9889962":[],"989":29,"9890348":6,"9893447":6,"98938878":[],"98947674e":[],"98947894":33,"98955596":[],"9896":[],"98966074":[],"9898ff":[10,11],"99":[7,10,12,14,22,35,37,38],"990":10,"99003341":[],"99009525":6,"9904314810719695":[],"99057275":[],"99071523":[],"99072374":[],"99088801":6,"991":29,"991047":[],"99104747":[],"99114739e":[],"99115119":6,"9912346":[],"99166852":[],"99176998":6,"99194201":[],"992":29,"99201684e":[],"992232777849821":[],"99232771":38,"99242921":6,"99246366":[],"99265097":6,"99284588":[],"9929":[],"99294056":[],"9929666457835595":[],"993":29,"9930344":[],"9930742":[],"99316252":6,"99318183":[],"99348646":[],"99360578e":[],"99362554":[],"99368326":[],"99370678":[],"99371056":6,"99389612":6,"99415296":[],"99418621":[],"99427512":[],"9943201":6,"99434472":[],"994345":[],"994461":[],"99446145":[],"994474":[],"99447418":[],"99449129":1,"99456598":37,"99457257":22,"99458998":[],"99475306":37,"9947756":6,"9948998579029953":[],"99492986":6,"99498272":[],"9950019477819025":[],"9950865473984227":[],"9951059931014423":[],"99512017":[],"9952183728736417":[],"9952505213910134":[],"9952537939995855":[],"99528218":6,"9953466931203151":[],"99539415":6,"995522":[],"9955318336036834":[],"99556258":[],"99562995":[],"99563484":[],"99566069":6,"995702810640425":[],"9957325252769553":[],"9957680433056965":33,"99578809":6,"9958244721767004":[],"9958589197366403":[],"99588057":22,"9958983289118531":[],"9959177108554428":33,"9959308805732706":[],"9959739":[],"9959836634296064":[],"996":[6,34,35],"9960490331171794":33,"99607299":[],"99607686":22,"99608161":6,"9960887274532307":[],"99619572":[],"99624324":[],"9963961":6,"99647992e":[],"99650061":6,"9966129":29,"996626":[],"99662602":[],"99663433":[],"9967181696265849":[],"9967458":6,"9969461173312454":[],"9969513794144311":[],"99700706":6,"99709215":6,"99723951":[],"997252717901263":[],"9972554229935837":[],"99729015":[],"99729756":6,"99751458":6,"99753321e":[],"99755766":[],"99757634":[],"99758326":6,"99759503":[],"99762142":34,"9976783":[],"997746":[],"99774602":[],"99774972":[],"99775587":6,"99782503":22,"99782627":[],"99783284":[],"99788953":22,"99792291":[],"99793613":6,"99799099":6,"99802696":37,"99812668e":[],"99813653":6,"99817613":[],"99828624":6,"9983289":[],"9983295":6,"99833653":38,"99835783":37,"99835872":[],"99839581":[],"99845267":6,"99848215":[],"99848815":37,"99849105":[],"99852187":[],"99854557":33,"9985605":[],"998577":6,"99861053":6,"99861193":[],"99862019":33,"99864751":[],"99869482":33,"99870462":[],"99871521":6,"99876945":33,"99881845":6,"99883628":33,"99884384":6,"99884409":33,"99884986":34,"99887767":[],"99891093":33,"99891873":33,"99893323":6,"99895448":[],"99898558":33,"999":[10,22,29,37,38,40,41],"99901896":6,"99902531":[],"99903755":6,"99906023":33,"9990927157696581":34,"99911427":6,"99912709":33,"99913489":33,"99916114":37,"99918546":6,"99919837":6,"99919981":38,"99920175":33,"99926459":6,"99927642":33,"99929677":[],"9993237":6,"99933188":6,"9993511":33,"99938942":6,"99941797":33,"99942911":[],"99943837":[],"9994385":6,"99944272":6,"9994574":[],"99949266":33,"99949306":6,"99950283":[],"99952851":[],"99953381":6,"99953475":6,"99954278":[],"99954434":[],"99954925":37,"99956033":[],"99956181":[],"9995659":[],"99956735":33,"99957911":6,"99959609":[],"99959681":[],"99961294":6,"99963172":[],"99965056":6,"99966798":[],"99967865":6,"99968451":[],"99968483":[],"99969576":[],"99969642":[],"99969895":[],"99970894":33,"99970988":6,"99973141":[],"9997332":6,"99974365":[],"99975636":[],"99975913":6,"99976375":22,"99977375":[],"99977849":6,"99977853":[],"99978365":33,"99978784":[],"99979896":[],"9997992":[],"99980002":6,"9998161":6,"99983332":[],"99984732":6,"99984836":[],"99985591":[],"99985884":[],"99987324":6,"99989103":37,"99989296":[],"99989476":6,"99989951":29,"99990413":[],"99990706":[],"99991245":[],"99991263":6,"99991295":[],"99991971":[],"99992107":[],"99992477":[],"99992746":6,"99993596":[],"99993978":6,"99994537":[],"99995":6,"99995454":38,"999955585168597":[7,34],"99996471":[],"99996755":[],"99996897":[],"99997244":[],"99997356":[],"99997644":[],"99998727":[],"9999923":38,"99999776":[],"9x":[7,26],"9y":[7,26],"\u00f8yvind":[7,33,34],"\u03b4":[40,41],"abstract":[2,21,37,40,41],"boolean":5,"break":[1,5,7,12,15,32],"byte":[25,32],"case":[0,1,2,3,4,5,6,7,8,12,13,14,15,17,22,24,25,26,27,32,35,38,39,40,41],"catch":1,"class":[1,2,4,5,7,8,9,10,12,13,14,23,29,32,35,37,38,40,41],"default":[1,2,3,5,7,8,9,17,25,26,27,32,33,34,36,39,40,41],"do":[1,3,4,5,6,7,9,10,11,12,13,14,15,16,17,22,23,25,26,27,32,33,36],"ekstr\u00f8m":5,"export":10,"f\u00f8470":[30,32],"final":[0,1,2,3,4,5,6,7,8,9,10,11,12,14,15,17,19,20,21,23,26,28,29,30,32,35,36],"float":[1,4,5,6,10,12,14,15,25,32,33,37,38,40,41],"function":[3,4,5,6,10,15,16,17,18,19,20,23,24,25],"import":[0,1,2,3,4,5,7,8,9,10,11,12,13,14,15,16,17,18,23,26,29,35,36,37,41],"int":[1,2,3,4,5,6,7,12,14,15,22,25,29,33,35,37,38,40,41],"long":[1,2,4,5,13,14,23,32,36,37,38,39,40],"m\u00f8svatn":[7,26],"new":[1,2,3,4,6,7,8,9,10,11,12,14,15,18,22,25,32,33,36,37,38,40,41],"public":[1,24,32],"return":[0,1,2,3,4,5,6,7,8,9,10,12,14,15,22,23,25,26,29,32,33,34,35,36,37,38,39,40,41],"sch\u00f8yen":[7,33,34],"short":[5,6,21,26,27,34,35,40,41],"super":[6,33,34,40,41],"switch":[1,40,41],"throw":[4,7,29,35],"true":[0,1,2,3,4,5,6,7,8,9,10,11,13,14,15,16,17,20,23,25,26,27,29,32,33,34,35,36,37,38,39,40,41],"try":[1,2,3,5,6,7,8,9,10,11,12,14,15,16,17,23,24,25,26,27,29,32,33,34,35,36,37,38,39,40,41],"var":[2,6,7,11,12,19,20,26,29,32,33,34,35,37,40,41],"while":[0,1,2,4,5,6,7,8,9,10,12,13,14,22,23,29,32,33,34,35,36,37,38,39,40,41],A:[3,4,6,7,8,11,12,13,14,17,19,21,24,25,26,27,28,29,30,31,33,37],AND:[0,3,41],And:[1,4,5,6,7,10,14,21,24,26,27,29],As:[1,2,3,4,5,6,7,9,11,13,14,17,25,26,27,29,32,33,34,35,36,37,38,39,40,41],At:[1,5,7,14,26,32,37],BE:[1,32],Be:[3,24,32,41],Being:[14,37],But:[1,2,3,4,6,7,10,11,23,27,29,33,34,35,39,40,41],By:[1,4,6,7,13,14,18,25,32,33,34,35,36,37,38],For:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,22,23,24,25,26,27,28,29,31,32,34,35,36,37,38,39,40,41],IF:[7,34],IN:31,If:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,17,22,23,24,25,26,27,29,32,33,34,35,36,37,38,39,40,41],In:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,20,22,23,24,25,26,27,29,31,32,33,34,35,36,37,38,40,41],Is:12,Ising:[6,13,33,38,39],It:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,22,23,24,25,26,27,29,32,33,34,35,36,37,38,39,40,41],Its:[2,3,5,12,40,41],NO:[0,10,12],No:[1,7,8,10,32,34,36,38,40,41],Not:[1,2,6,7,33,34,35,38,39,40,41],OR:29,Of:29,On:[1,4,10,16,26,28,29,30,31,32],One:[1,2,4,5,6,7,8,9,12,13,14,18,22,26,29,33,35,36,37,38,39,40,41],Or:[1,2,7,26,32,36,39,40],Such:[1,7,13,17,29,35,36,37,38,39,40],TO:[40,41],That:[1,6,8,11,12,13,15,20,26,29,32,34,35,36,39,40],The:[5,11,14,15,16,17,19,20,21,22,23,25,26,27,28,29,30,31],Their:[],Then:[1,2,7,9,10,11,12,13,14,15,23,25,26,32,35,36,37,38,39,40,41],There:[1,4,5,6,7,9,10,12,13,15,25,26,28,29,30,32,33,34,36,37,38,39],These:[1,4,5,6,9,10,11,12,13,14,15,16,17,25,26,27,29,30,32,33,34,37,38,39,40,41],To:[1,2,3,4,5,6,7,8,9,10,11,12,13,14,22,23,25,27,29,34,35,36,37,38,39,40,41],With:[1,6,7,9,10,11,12,13,15,19,23,25,26,27,29,32,33,35,38,39,40,41],_0:[6,9,11,12,14,33,36,37],_1:[3,6,7,9,11,12,13,14,15,25,33,34,36,37,38,39,40,41],_2:[3,6,9,12,13,14,25,33,37,38,41],_3:25,_4:25,_9:[14,37,38],_:[0,1,2,3,5,6,7,8,9,10,11,12,13,14,18,19,20,22,23,25,26,32,33,34,35,36,37,38,40,41],__array_function__:[],__class__:[11,40,41],__doc__:[7,35],__enter__:[1,8,36],__future__:[9,10,39],__getattr__:34,__getattribute__:32,__getitem__:[1,8,36],__init__:[0,1,2,4,5,8,15,34,36,38,40,41],__main__:[0,3,38,41],__mul__:[],__name__:[0,3,11,34,38,40,41],__rsub__:[],__rtruediv__:[],_accessor:32,_accuraci:[40,41],_allclose_dispatch:23,_auto10:[7,13,38],_auto11:[7,38],_auto12:[7,38],_auto1:[3,4,5,6,7,8,13,14,22,25,29,33,36,37,38,39,40,41],_auto2:[3,4,5,6,7,13,14,25,29,37,38,39,40,41],_auto3:[4,5,6,7,13,14,25,37,38,39,40],_auto4:[5,7,13,14,25,37,38],_auto5:[5,7,13,14,25,37,38],_auto6:[5,7,13,25,38],_auto7:[5,7,13,25,38],_auto8:[7,13,38],_auto9:[7,13,38],_backpropag:[40,41],_base:9,_build:[1,18,20,24,26,31,32],_c:[2,40,41],_can_hold_identifiers_and_holds_nam:32,_check_optimize_result:[0,10,12],_class:9,_compon:12,_coordinate_desc:7,_depth:10,_feedforward:[40,41],_format:[40,41],_fraction:10,_generatorcontextmanag:[1,8,36],_i:[1,2,3,6,7,8,9,12,13,14,16,17,18,20,26,32,33,34,35,36,37,38,39,40,41],_info_axi:32,_j:[1,2,3,4,6,7,9,14,18,20,26,33,34,35,37,40,41],_k:[14,36,37,38,40,41],_l:[13,38,39,40,41],_lambda:[7,32],_leaf:10,_ll:[4,5,15],_logist:[0,10,12],_m:11,_make_vjp:[],_multilayer_perceptron:[2,40,41],_n:[3,6,9,12,14,33,36,37,41],_node:10,_np:[],_o:[4,5,15],_open_file_or_url:[1,8,36],_p:[6,9,33],_plugin_dir:[4,5,15],_progress_bar:[40,41],_ratio:12,_rc_params_in_fil:[1,8,36],_s:[4,5,15],_sampl:10,_set_classif:[40,41],_split:[7,10,26],_src:[22,38],_t:[14,22,37,38],_test:[7,26],_tf_getinputpropertieslist:[4,5,15],_trace:[],_valu:[],_varianc:12,_weight:10,_wrapreduct:[],a0:4,a0faa0:[10,11],a1:[1,23,32],a2:[1,23,32],a3:[1,32],a4:[1,32],a_0:[1,22,27,32,39,40],a_1:[39,40],a_1a:[1,32],a_1x:[22,27],a_2:[39,40],a_2a:[1,32],a_2x:[22,27],a_3:[1,32],a_3a:[1,32],a_4:[1,32],a_4a:[1,32],a_:[1,2,17,25,32,33,39,40,41],a_h:[2,40,41],a_i:[1,2,3,13,32,39,40,41],a_j:[2,13,39,40,41],a_k:[1,2,13,39,40,41],a_matric:[40,41],a_ndim:[],aaron:31,ab:[1,3,6,14,15,22,26,32,33,34,37,38,39,41],ab_channel:[24,38,39,40,41],abandon:[2,39,40],abbrevi:28,abid:29,abil:[1,11],abl:[1,2,5,6,7,8,11,13,14,17,23,26,33,36,37,38,39,40],about:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,24,25,26,27,30,34,35,36,37,38,39,40,41],abov:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,22,23,25,26,27,29,31,32,33,35,36,37,38],abovement:[7,32,35],abscissa:[14,36,37],absolut:[1,3,6,7,14,32,33,34,35,37,38,41],absorb:[33,34],acceler:[14,22,37,38],accept:[1,4,7,10,23,26,32,33],access:[1,4,12,29,32],accid:[5,7,35,36],accompani:[1,32,33],accomplish:[9,10,14,37,38],accord:[1,2,3,6,7,10,13,14,15,16,17,23,29,32,34,35,36,37,38,39,40,41],accordingli:12,account:[1,4,6,14,16,29,32,34,35,37,38],accumul:[13,14,22,29,37,38,39,40,41],accur:[1,4,5,7,11,14,35,37,38],accuraci:[0,1,2,4,5,6,7,8,10,11,12,13,23,27,32,33,36,38,39,40,41],accuracy_scor:[1,2,11,23,32,40,41],accuracy_score_numpi:[2,40,41],acheiv:23,achiev:[1,2,6,7,9,13,25,32,34,35,38,39,40],aco:29,acquaint:24,acquir:[2,24,32,39,40],acr:1,across:[2,4,7,10,24,32,35,40,41],act:[2,4,25,40,41],act_func:[40,41],act_func_deriv:[40,41],actic:23,action:29,activ:[1,3,4,5,10,28,30,32,36,37],activation_func:23,actual:[1,2,5,6,7,9,12,17,23,25,29,32,33,34,35,39,40],ad:[0,2,4,5,6,9,14,16,17,25,34,35,36,37,41],ada_clf:11,adaboostclassifi:11,adadelta:[14,37,38],adagrad:[27,39,40,41],adagradmomentum:[40,41],adam:[2,4,5,23,27,30,32,39,40,41],adam_schedul:[40,41],adap:39,adapt:[5,7,14,31,35,36,39],add:[1,2,3,4,5,6,7,9,11,12,13,16,17,18,22,23,26,27,29,32,33,34,35,37,39,40,41],add_outgrad:[],add_subplot:[0,2,8,13,15,26,36,37,38,39,40,41],addendum:6,addit:[0,1,3,4,6,7,8,9,10,11,13,14,16,18,22,23,24,25,26,29,30,31,32,33,34,35,36,37,38,39,40,41],addition:[13,14,36,37,38,39],address:[2,10,12,14,32,37,38,39,40],adjac:[4,13,38,39],adjoint:[6,33,34],adjust:[1,6,13,14,36,37,38],admir:[1,32],advanc:[5,7,13,31,32,35,39],advantag:[2,4,6,7,11,14,25,34,35,36,37,38,40,41],adversari:32,afecionado:32,affect:[4,40,41],affin:[1,4,9,12,33,39],afford:4,aficionado:32,aforement:15,african:1,after:[1,2,3,5,6,7,10,12,13,14,16,22,23,24,25,26,27,29,32,33,34,35,37,38,39,40,41],afterward:[1,32],ag:[1,8,28,32,33,36],ag_0:[3,41],again:[1,2,5,6,7,8,9,11,12,13,14,16,17,22,26,27,29,32,33,34,35,37,38,39,40,41],against:[2,5,8,11,22,27,36,40,41],agegroup:[8,36],agegroupmean:[8,36],aggreg:[10,11],agorithm:11,agre:[6,7,29,34,35],agreement:[14,22,37,38],ahead:10,ai:[1,27,31],aid:[12,21],aim:[1,2,5,7,8,12,15,16,17,24,25,26,27,33,35,36,39,40],ainv:6,airplan:4,aka:[6,34,35],al:[1,3,5,16,17,18,26,27,31,32,33,34,35,36,37,39,40,41],alarm:[6,8,34,35],aldo:33,alexsmola:41,algebra:[1,4,6,14,22,24,33,34,35,37,38],algorithm:[0,1,2,3,5,6,7,8,9,14,15,24,25,26,29,31,32,34,35,36],align:[0,1,3,6,7,8,9,14,26,29,32,33,34,35,36,37,41],all:[1,2,3,4,5,6,7,8,10,11,12,13,14,15,16,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,41],allclos:23,allevi:[2,14,36,37,39,40],alloc:[4,25],allow:[1,2,3,4,6,7,9,11,14,16,17,24,25,26,32,33,34,35,36,37,38,39,40,41],almost:[1,2,7,9,12,14,22,29,35,36,37,38,39,40],alon:[3,10,41],along:[3,4,5,6,7,10,11,12,23,24,25,32,33,34,35,36,40,41],alpha:[0,1,2,3,4,5,7,8,9,10,11,14,15,29,32,33,35,36,37,39,40,41],alpha_0:4,alpha_1:4,alpha_2:4,alpha_:11,alpha_i:[4,14,37],alpha_k:[14,37],alpha_m:11,alpha_n:4,alpha_opt:[14,37],alreadi:[3,4,5,6,7,11,13,24,25,29,32,33,34,35,38,39,41],also:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,22,23,24,25,26,27,28,29,32,33,34,35,36,37,38,39,40,41],alter:[2,40,41],altern:[0,1,2,5,6,7,9,10,12,14,25,26,32,33,34,35,36,37,40,41],although:[1,2,6,7,9,11,14,17,22,32,34,35,37,38,39,40,41],alwai:[1,4,6,7,13,14,17,22,23,26,29,32,33,34,35,36,37,38,39],am:5,ambit:[39,40],ame2016:[1,32],american:1,among:[1,4,6,10,11,13,25,32,33,34,38,39],amongst:[6,34,35],amount:[1,2,4,5,7,9,11,15,24,35,39,40,41],amp:[38,39,40,41],an:[0,2,3,4,6,7,8,9,10,12,13,14,15,16,17,18,20,21,22,23,24,25,26,27,29,30,31,33,34,35,36,37,38,40,41],an_:29,anaconda:[1,2,16,24,26,32,40,41],analog:[14,37,38],analys:[7,34,35],analysi:[0,2,4,5,8,15,16,17,18,20,22,25,31,36,40,41],analyt:[3,4,6,7,8,13,14,16,24,26,27,32,33,34,35,36,37,38,39],analytical_gradi:[22,38],analyz:[1,2,4,5,6,7,17,18,26,27,29,33,34,41],andrew:[2,39,40,41],angl:[1,4,10,33],anharmon:4,ani:[0,1,2,3,4,5,6,7,8,9,10,11,13,15,19,23,29,32,33,34,35,38,39,40,41],anim:[5,13,38,39],ann:[13,38,39],annot:[0,1,2,4,8,9,32,36,40,41],announc:32,anoth:[1,2,3,4,5,6,7,8,9,11,12,13,14,25,26,27,29,32,33,36,37,38,39,40,41],anp:[],ans_vspac:[],ansatz:[1,32],answer:[1,2,4,6,7,25,26,27,30,32,34,35,40,41],antialias:[3,7,26,41],anticip:5,anymor:[2,9,39,40],anyon:[5,9],anyth:[2,23,29,39,40,41],anytim:[30,32],apach:[2,40,41],apart:[12,14,36,37],api:[2,24,32,40,41],appar:[3,41],appear:[1,2,4,14,17,25,29,37,38,39,40,41],append:[2,4,5,9,10,14,22,23,32,37,38,40,41],appendix:26,appli:[1,2,4,5,7,8,9,10,11,12,13,14,16,17,26,27,29,31,32,33,34,35,36,37,38,39,40,41],applic:[0,1,2,4,5,6,7,8,10,13,14,22,25,26,29,31,32,33,35,36,37,38,39,40,41],apply_gradi:5,approach:[0,2,3,5,6,7,10,11,12,13,14,18,22,23,24,29,31,33,36,39,40,41],appropri:[3,7,10,13,14,24,29,35,37,38,41],approv:32,approx:[1,3,4,7,11,12,14,29,32,35,36,37,38,41],approxim:[1,2,3,4,5,6,7,8,11,12,14,19,20,26,29,32,33,34,35,36,37,38,40,41],apt:[1,16,24,26,32],aq:29,ar:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,20,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41],aragorn:32,arang:[0,2,4,5,7,8,10,11,13,14,26,32,36,37,38,39,40,41],arbitrari:[2,5,7,9,13,14,29,35,36,37,38,39,40,41],arbitrarili:[1,2,12,32,40,41],arc:[7,26],architectur:[4,5,13,39],archiv:27,area:[1,4,7,10,26,31,32],arg:[3,23],argmax:[2,12,23,40,41],argmin:[5,11,15],argnum:[],argnum_0:[],argnum_1:[],argsort:12,argu:[2,14,37,38,39,40],argument:[1,3,4,6,7,12,13,14,22,23,32,33,34,35,39,40,41],argval:[],aris:[1,7,13,14,29,32,35,36,37],arithmet:[1,14,25,32,37,38],arm:[7,34],armadillo:25,arnulf:[39,40,41],around:[1,2,5,6,7,12,23,29,32,34,35,39,40,41],arr:[],arr_list:[],arrai:[0,1,2,3,4,5,6,7,8,9,10,12,13,14,15,22,23,24,26,29,33,34,35,36,37,38,39,40,41],arrang:[4,32],array_function_dispatch:23,arraybox:[14,37],arriv:[1,7,10,12,25,29,32,35],arrow:[13,38,39,40,41],arrowprop:9,art3d:37,art:[1,2,16,24,32,39,40],articl:[1,4,5,7,11,19,26,27,32,33,34,35],artifici:[0,1,3,8,13,31,32,36,41],artificialneuron:[13,38,39],arug:[14,37,38],arxiv:[4,5,22,26,37,38,39,41],asanyarrai:23,asarrai:[1,3,7,10,22,33,34,37,38],asid:33,ask:[6,7,12,13,26,34,35,39,40],aspect:[1,7,24,26,32,33,34,39,40,41],assembl:[1,4,32],assert:[5,40,41],assess:[1,7,26,32,33,34,35],assici:5,assign:[0,1,8,9,10,13,14,15,28,30,31,32,36,37,38,39,40],associ:[1,7,10,13,15,29,32,35,38,39],assum:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,20,25,26,29,32,33,34,35,36,37,38,39,40,41],assumpt:[1,4,6,7,10,12,19,26,29,32,33],ast:[1,6,7,19,26,32,34,35,36],astronomi:[0,32,34,36,37,38,40,41],astyp:[5,10,11],asymmetri:[1,32],asymptot:[5,7,35],atleast_2d:[],atol:23,atom:[1,32],attempt:[1,5,7,8,9,11,32,34,36,39,40],attend:28,attent:[1,25,32],attr:34,attract:[1,11,32],attribut:[1,3,10,27,32,34,40,41],attributeerror:[3,32,34],audi:[1,32],audio:[4,5],august:[16,17,32],aurelien:[1,16,28,31,32],austfjel:[7,26],author:[1,2,11,29,39,40],authour:32,auto:[9,10,11,29,40,41],auto_exampl:[23,26,34],autocor:29,autocorrelation_tim:29,autocorrelform:29,autocovari:29,autoencod:[5,24,32],autoencond:24,autograd:[23,24,27,32,39,40],autom:[1,24,31,32],automac:25,automag:32,automat:[1,2,3,4,5,12,17,23,24,25,32,40],automobil:4,autonom:5,avail:[1,2,5,7,8,11,12,16,21,22,24,25,26,27,28,30,31,32,35,36,38,40,41],averag:[1,2,4,7,10,11,14,15,22,29,30,32,33,34,35,36,37,38,40,41],avoid:[1,5,6,7,10,12,14,22,23,25,32,33,35,36,37,38,40,41],awai:[3,4,7,33,34,39,41],awar:[3,11,41],award:[30,32],ax:[0,1,2,3,4,5,7,8,9,10,11,12,13,14,15,23,25,26,27,32,35,36,37,38,39,40,41],axes3d:[3,7,14,26,36,37,41],axes_grid1:7,axhlin:9,axi:[1,2,3,4,5,7,8,9,10,11,12,13,14,15,22,23,25,26,29,32,33,34,35,36,37,38,39,40,41],axiom:[6,34,35],axis_arg:[],axlabel:[1,32],axvlin:[5,9],axvspan:5,b1:[9,23],b2:[9,23],b3:9,b:[1,2,4,5,6,7,9,10,11,13,14,15,16,17,22,23,29,30,32,33,34,35,36,38,39,40,41],b_0:[1,39],b_1:[1,3,13,14,37,38,39,40,41],b_2:[1,14,37,39,40],b_5:[14,37,38],b_:[1,2,25,39,40,41],b_g:23,b_group:10,b_i:[1,2,3,13,32,38,39,40,41],b_ia_:[1,32],b_ia_i:1,b_index:10,b_is_vec:[],b_j:[2,13,38,39,40,41],b_k:[1,2,13,14,37,38,39,40,41],b_m:[13,38],b_meta:[],b_score:10,b_valu:10,babcock:32,bachelor:[28,30],back:[1,4,5,6,7,9,10,11,16,23,25,27,29,32,33,34,36,37,38],backbon:25,backend:[1,2,5,8,22,36,38,40,41],background:[31,32,33,39,40,41],backpropag:[2,23,39,40,41],backtrack:10,backup:25,backward:[2,3,5,13,25,39,40,41],backward_pass:[],bad:[7,33,34,40,41],badli:29,bag:[10,24,32],bag_clf:11,baggin:32,baggingboot:11,baggingclassifi:11,baggingtre:11,balanc:[7,35,36],balanced_eq:[],baluka:41,band:25,bandwidth:25,bar:[1,7,12,16,17,26,32,40,41],barber:31,bare:[5,11],base:[1,2,4,5,6,8,9,10,11,15,24,29,30,31,32,33,34,36,39,40,41],basi:[6,8,9,11,12,13,14,25,33,34,36,37,38,39],basic:[7,9,13,14,15,24,26,29,32,37,38,40],batch:[4,5,12,13,14,22,23,27,36,41],batch_shap:5,batch_siz:[2,4,5,40,41],batchnorm:5,bay:[0,8,36],baydin:39,bayesian:[6,24,31,32,34,35],beam:[0,32,36,37,38,40,41],becattini:30,becaus:[1,2,3,4,5,6,7,9,10,13,14,15,32,34,35,36,37,38,39,40,41],becom:[0,1,2,3,6,7,8,10,13,14,22,29,32,33,34,35,36,37,38,39,40,41],been:[1,2,3,4,5,6,7,12,13,14,24,25,26,27,32,33,34,35,37,38,39,40,41],befor:[1,2,3,4,5,6,7,8,9,13,14,15,21,23,25,26,27,29,32,33,34,35,36,37,38,39,40,41],beforehand:[1,29,32],begin:[0,1,2,3,4,5,6,7,8,9,10,12,13,14,15,22,25,26,29,30,32,33,34,35,36,37,38,39,40,41],behav:[2,7,14,35,36,37,39,40],behavior:[1,2,14,32,36,37,38,39,40],behaviour:[13,38,39,40],behind:[1,2,7,9,14,32,36,37,40,41],being:[0,1,2,3,4,5,6,8,9,11,12,13,14,18,29,32,33,34,35,36,37,38,39,40,41],believ:[10,25],belong:[0,8,9,10,14,15,36,37,40,41],below:[1,2,3,4,5,6,7,8,9,10,11,12,13,14,23,25,26,27,29,32,33,34,35,36,37,38,39,40,41],benchmark:11,benefici:[2,14,37,38,40,41],benefit:[1,2,5,12,14,16,24,32,36,37,38,40,41],bengio:[2,22,27,28,31,32,33,37,38],benign:[0,2,8,10,36,40,41],benno:[39,40,41],berner:[39,40,41],besid:[5,6,34],bessel:[6,33,34,35],best:[0,1,2,3,4,5,6,7,8,9,10,11,13,14,23,26,27,30,32,33,35,36,37,39,40,41],best_estimator_:36,beta1:[22,37,38],beta2:[22,37,38],beta:[0,1,2,4,6,7,8,11,12,14,17,18,19,20,22,26,32,33,36,37,38,40,41],beta_0:[0,1,2,4,6,7,8,14,32,33,34,36,37,40,41],beta_0x_:[1,32,33],beta_1:[0,1,2,4,6,7,8,11,14,32,33,34,36,37,38,40,41],beta_1x_0:[1,32],beta_1x_1:[0,1,8,32,36],beta_1x_2:[1,32],beta_1x_:[1,32,33],beta_1x_i:[0,8,14,33,36,37],beta_2:[1,4,14,32,33,37,38],beta_2x_0:[1,32],beta_2x_1:[1,32],beta_2x_2:[0,1,8,32,36],beta_2x_:[1,32,33],beta_2x_i:33,beta_3:4,beta_3x_i:33,beta_4x_i:33,beta_:[0,1,4,7,8,14,32,33,34,36,37,38],beta_i:[1,4,6,32,33,34],beta_j:[1,6,7,14,19,26,32,33,34,35,37,38],beta_k:[14,36,37],beta_lasso:38,beta_linreg:[14,22,36,37,38],beta_logreg:[0,38],beta_m:11,beta_mg_m:11,beta_n:4,beta_ol:38,beta_p:[0,8,36],beta_px_p:[0,8,36],beta_ridg:38,betavalu:6,better:[1,2,3,4,5,7,10,11,12,13,14,22,32,33,34,35,37,40,41],between:[0,1,2,3,4,5,6,7,8,9,10,12,13,14,15,18,22,26,27,29,32,33,34,35,36,37,38,39,40,41],beyond:[1,2,6,7,9,14,22,26,32,33,36,37,38,39,40],bf:[14,15,25,29,36,37],bg:32,bgd:[14,37,38],bia:[1,2,3,4,6,9,10,11,13,14,20,23,32,33,34,36,37,38,39,40,41],bias:[2,3,4,6,7,10,13,23,27,34,38],bibliographi:26,big:[1,2,3,6,7,15,19,34,35,40,41],bigger:[2,7,33,34,39,40],bigr:[13,38],bike:10,bilbo:32,billion:[4,13,24,38,39],bin:[0,1,8,29,36],binari:[0,1,4,6,8,10,11,13,27,32,34,35,36],binarycrossentropi:5,bind:[1,33],binomi:[24,29,32],binsboot:[7,35],bioinformat:1,biolog:[2,13,38,39,40],bios1100:[24,32],bird:[1,4],birth:32,bishop:[28,31,32],bit:[2,5,23,25,29,32,39,40,41],bitwis:29,bivari:[3,41],bk:[1,14,37,38],bla:[25,32],black:[9,10,15,22,38],blob:[17,21,26,36,37,38],block:[7,11,24,25,29,32,35],blog:[27,32,41],blogpost:5,blue:[1,4],bmatrix:[1,2,4,6,8,9,12,14,25,32,33,34,36,37,39,40,41],bmi:[2,40,41],bn:[],bodi:[1,2,5,13,32,38,39,40,41],bold:[2,40,41],boldfac:[1,6,17,33,34],boldsymbol:[0,1,2,3,4,6,7,8,9,11,12,14,15,16,17,18,19,20,22,26,32,36,37,38,39,40,41],boltzmann:[13,24,32,38,39],book1:31,book:[20,26,27,31,32,33,35],bool:23,boost:[2,10,24,32,39,40],boostrap:11,bootstrap:[2,14,20,24,26,27,32,36,37,38,39,40,41],borrow:32,boston_dataset:1,bot:9,both:[1,2,5,6,7,9,10,11,14,15,16,17,24,25,26,27,29,30,32,33,34,35,36,37,38,39,40,41],bottl:[8,36],bound:[1,9,13,37,38,39,40,41],boundari:[3,5,9,12,13,41],box:[5,10,23],boxed_arg:[],boyd:[9,14,36,37],bracket:[5,29],brain:[2,8,13,36,38,39,40,41],branch:10,breast:[0,6,8,12,27,34,35,36],breat:27,breviti:[14,22,37,38],brew:[1,16,24,26,32],brg:9,brief:[26,27,33],briefli:[1,32],bring:[1,6,7,11,27,33,34],broad:1,broadcast:23,broadcast_ax:[],broadcast_idx:[],broadcast_to:[],brought:[14,22,24,32,37,38],brownle:5,browser:32,brute:[4,6,12,33,34,36,39],bs:[9,10,11],buffer_s:5,bui:5,build:[1,5,6,7,11,25,29,32,34,35,36,37,38,39],built:[1,2,4,5,7,35,40,41],bunch:12,busi:1,bxe2t:[38,39,40],c1:[9,12],c2:[9,12],c:[0,1,2,3,5,6,7,8,9,10,11,12,13,14,15,16,17,20,23,24,25,29,30,31,33,34,35,36,37,38,39,40,41],c_0:29,c_1:[13,38],c_2:[13,38],c_3:[13,38],c_4:[13,38],c_:[1,9,10,11,14,22,29,36,37,38],c_i:[13,14,37,38],c_k:29,ca:[2,32,39],cach:11,cal:[1,9,11,13,14,16,17,32,36,37,39,40,41],calcul:[0,1,2,3,5,6,7,9,10,11,12,13,14,15,16,17,19,22,25,27,29,32,34,35,36,37,38,39,40,41],california:27,call:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,17,19,20,22,23,24,25,26,27,29,30,32,33,34,35,36,37,38,39,40,41],callabl:[40,41],calor:[1,33],cambridg:[14,31,36,37,39,40,41],can:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,23,24,25,26,27,29,30,31,33,34,36,40,41],cancel:[1,14,32,33,37,38],cancer:[6,11,27,34,35],cancerpd:[0,8,36],candid:[9,10,11,37],cannot:[0,1,2,5,6,7,8,9,10,28,29,33,34,36,40,41],canopi:[1,16,24,26,32],canva:[26,27,32],cap:[6,34,35],capabl:[1,2,9,14,24,32,37,38,40,41],capac:[3,30,41],capita:1,caption:[26,27],captur:[5,12,13,38,39],car:[4,5],card:[1,8,32,36],cardin:[2,39,40],care:[12,36],carefulli:[14,37],carlo:[1,7,24,29,31,32,35],carri:[3,7,8,26,35,36,41],cart:11,casella:31,cast:[2,23,39,40],cat:[4,5],categor:[1,2,4,10,12,32,40,41],categori:[0,1,2,4,8,11,13,15,32,36,38,39,40,41],categorical_crossentropi:[2,4,40,41],caus:[1,6,7,8,29,32,33,34,35,36],causal:1,causat:[1,32],cax:[2,40,41],cb:[7,32],cbar:[2,40,41],cc:[1,2,6,14,27,32,33,34,36,37,39,40,41],ccc:[6,13,34,38],cd_fast:7,cdf:29,cdot:[1,3,7,13,14,15,25,29,32,35,36,37,38,41],celebr:[14,36,37],cell:[1,2,3,4,5,7,8,9,10,11,14,15,23,26,34,36,40,41],center:[0,1,2,7,8,9,10,12,15,26,29,32,34,35,36,39,40,41],central:[1,4,6,7,9,17,25,27,32,33,34,39,40],centroid:[15,29],centroid_differ:15,centuri:4,certain:[1,4,7,8,10,23,29,32,33,35,36],certainti:35,cg:[14,37],cha:1,chain:[1,2,14,24,29,32,37,38,41],challeng:[37,38,39],chanc:[2,6,14,29,34,35,37,38,40,41],chang:[1,2,3,4,5,6,7,9,10,12,13,14,15,22,23,25,26,27,29,32,33,34,35,36,37,38,39,40,41],channel:4,chap4:[39,40],chapter3:[1,20,26],chapter:[1,7,11,12,17,19,22,25,26,27,31,32,33,34,35,36,37,38,39,40,41],charact:[1,4,6,9,32,33],character:[9,10,11,13,29,38,39,40,41],characterist:[1,2,4,11,14,22,32,37,38,40,41],charg:[1,32],charl:1,chase:5,chatgpt:26,chd:[8,36],chddata:[8,36],cheap:[6,33],cheaper:[2,14,37,38,39,40],check:[1,2,4,5,6,7,12,14,23,25,32,37,38,39,40,41],checkmark:4,checkpoint:5,checkpoint_dir:5,checkpoint_prefix:5,chen:11,cheng:33,chiaramont:[3,41],choic:[1,2,3,4,5,7,10,13,14,15,20,25,27,32,33,35,36,37,38,41],choleski:[6,25,33],choos:[3,4,7,10,11,12,14,15,23,26,27,35,36,37,38],chosen:[1,2,3,7,9,10,11,14,17,22,27,29,32,35,36,37,38,39,40,41],chosen_datapoint:[2,40,41],christian:31,christoph:[28,31,32],cifar10:4,cifar:4,circ:[2,13,39,40,41],circl:[1,9,13,33,38,39],circuit:4,circumfer:10,circumv:[2,6,14,33,37,38,39,40],cite:26,ckpt:5,clarifi:23,clariti:29,class_nam:[4,10],class_val:10,class_valu:10,classic:[0,8,10,14,36,37,38,39],classif:[1,4,6,7,8,9,12,13,23,24,26,31,32,33,34,35,37,39],classifi:[0,1,2,5,8,10,11,12,27,32,40,41],classificaton:[2,40,41],classifii:11,clean:[2,40,41],clear:[2,6,11,13,14,22,37,38,40,41],clearli:[0,1,4,6,7,8,9,19,29,33,34,35,36],clever:[2,11,40,41],clf3:1,clf:[1,7,9,10,11,32,33],clf_lasso:7,clf_ridg:[7,32],clip:[4,29],clone:30,close:[1,2,3,5,7,9,10,12,13,14,15,26,29,31,32,35,36,37,38,39,40,41],closer:[4,6,14,33,37,38],closest:[9,12,14,15,37],closur:[24,32],cloud:[24,32],cluster:[1,2,5,7,12,24,32,35,40,41],cluster_label:15,cm:[2,3,4,7,9,10,14,26,36,37,40,41],cmap:[1,2,3,4,5,7,9,10,11,26,32,40,41],cmap_arg:7,cmb:28,cmd:10,cn_:29,cnn:[13,38,39],cnn_kera:4,cntk:[24,32],co:[1,3,4,7,10,14,22,32,35,37,38,41],code:[0,4,5,7,8,9,16,17,18,20,23,24,25,26,29,31],codebas:[40,41],coef0:9,coef:[1,32],coef_:[1,6,7,9,10,14,32,33,34,36,37],coeff:6,coeffici:[1,4,6,7,8,9,10,14,16,17,25,32,33,34,35,36,37,38],coerc:[1,7,23,32,35],coin:[11,29],coin_toss:11,col:[1,12,32,33],colab:[23,24,32],cold:10,colinear:1,collabor:[26,27],collaps:9,collect:[1,3,7,11,12,22,24,29,31,32,35,38,39],collinear:[6,33,34],color:[1,4,5,7,9,10,11,22,26,29,37,38],color_channel:4,color_cod:7,colorbar:[2,7,26,40,41],coloumn:[40,41],colsample_bytre:11,colsaobject:11,column:[0,1,2,3,6,7,8,9,10,12,13,18,25,32,33,34,35,36,38,39,40,41],columntransform:10,com:[5,7,17,21,22,23,24,26,27,31,32,35,36,37,38,39,40,41],combin:[2,3,6,7,8,11,29,34,35,36,37,38,39,40,41],come:[1,2,4,5,6,13,14,15,16,27,32,33,34,35,37,38,39,40,41],command:[1,2,32,40,41],comment:[1,5,6,7,16,17,26,27,32,33],commerci:[1,16,24,26,32],commod:[1,32],common:[0,1,2,4,6,7,8,10,12,14,15,26,27,29,32,33,35,36,37,38,39,40,41],commonli:[1,2,5,7,8,10,14,15,33,35,36,37,38,40,41],commun:[1,13,26,38,39],commut:4,commutatitav:4,compact:[1,2,4,6,7,8,10,12,13,14,15,23,32,33,34,35,37,38,41],compair:1,compar:[1,4,5,6,7,12,14,16,17,22,25,26,27,32,33,34,35,36,37,38,39],comparison:[3,5,14,27,37,38,41],compat:[8,36],compet:1,competit:11,compil:[1,2,4,5,14,16,22,24,25,32,37,38,40,41],compl:23,complet:[0,1,3,4,5,10,13,23,32,38,41],completenn:[13,38],complex:[2,6,9,10,12,13,14,16,17,20,27,32,35,37,38,39,40],complic:[1,2,10,14,32,35,36,40,41],compoment:33,compon:[0,1,2,4,5,6,7,8,10,15,17,24,32,33,34,35,36,39,40,41],components_:12,compos:[10,13,14,15,22,24,32,37,38,39],compphys:[1,7,17,18,20,21,24,26,28,30,31,32,36,37,38],compress:[1,32,33],compris:[7,36],compromis:[6,33],compulsori:[24,32],comput:[0,1,2,3,4,5,6,7,8,9,11,12,13,14,16,17,18,22,23,24,25,26,27,28,29,31,32,33,34,35,36,39,40,41],computation:[1,4,7,10,14,29,32,36,37,38,39],computationalscienceuio:32,computerlab:[26,27],con:27,concaten:[3,5,7,15,41],concatenate_arg:[],concav:[2,10,14,33,36,37,40,41],concentr:[1,11],concept:[1,3,24,32,33,41],conceptu:[13,14,36,37,38,39],concern:[1,2,5,8,32,36,40,41],concic:32,conclud:[1,6,14,22,34,35,37,38],conclus:[2,40,41],cond:[3,41],conda:[1,2,16,24,26,32,40,41],condis:33,condit:[1,3,5,6,7,9,10,12,14,29,32,33,38,41],conduct:24,condwav:[3,41],confid:[1,6,7,8,9,19,26,32,33,34,36],configur:[4,22,38],confirm:[6,13,23,34,35,38],confus:[6,7,8,11,25,33,34,35],confusion_matrix:10,congruenti:29,conjug:[5,9],conjugaci:[14,37],conjunct:4,connect:[1,2,4,5,10,12,13,14,25,32,33,36,37,38,39,40,41],consequ:[6,7,9,11,13,14,33,34,35,36,37,38],conserv:[6,15,33],consid:[1,2,3,4,6,7,8,9,10,11,13,14,17,20,22,25,26,27,29,32,33,34,35,36,37,38,39,40,41],consider:[1,2,6,14,32,33,34,35,36,37,41],consist:[1,2,3,4,5,7,13,14,20,22,26,27,29,33,35,36,37,38,39,40,41],constant:[1,3,5,6,7,9,13,14,29,32,33,34,35,36,37,38,39,40,41],constitu:[1,32],constitut:[3,7,35,36,41],constrain:[2,4,6,8,12,34,36,40,41],constraint:[6,7,9,14,33,34,35,37,38],construct:[0,1,2,3,4,6,7,8,9,10,11,12,25,29,32,33,34,35,36],consult:27,contact:[1,32],contain:[0,1,3,4,5,6,7,8,9,10,12,13,14,18,20,22,23,25,26,27,29,31,32,33,34,35,36,37,38,39,40,41],contemporari:32,content:[2,24,25,32,40,41],context:[7,11,14,26,35,36,37,38,39],contextlib:[1,8,36],contigu:25,continu:[1,2,3,4,5,6,7,8,9,10,11,13,14,18,19,22,25,26,27,29,32,33,34,35,36,37,39,40,41],contour:[10,11,14,37],contourf:[9,10,11],contrast:[2,5,10,11,13,38,39,40],contribut:[1,4,6,14,22,29,32,33,34,37,38],contributor:[1,26],control:[1,2,4,10,14,16,24,32,37,38,40,41],conv2d:[4,5],conv2dtranspos:5,conv:[4,5],convei:32,conveni:[1,6,7,13,14,25,26,27,32,34,35,36,37,38],convent:[13,33],converg:[0,2,3,5,6,7,9,10,12,14,15,22,27,33,34,36,37,38,39,40,41],convergencewarn:[0,2,7,9,10,12,40,41],convers:37,convert:[1,2,5,6,10,12,14,25,32,33,37,38,40,41],converttomatrix:5,convex:[0,5,6,8,33],convinc:[14,36,37],convolut:[2,5,24,32,40,41],cook:26,cool:[5,10],coolwarm:[7,26],coordin:[6,13,15,33,34,38],coorel:1,copi:[1,2,15,33,40,41],copyright:27,core:[1,3,8,11,23,32,36],corel:32,coronari:[8,36],corr:[0,1,6,8,12,33,36],correalt:[12,24],correct:[1,2,3,4,5,6,8,14,23,25,29,32,33,34,35,37,38,40,41],correctli:[2,3,7,8,11,23,26,27,40,41],correl:[1,2,4,6,7,8,11,13,14,22,24,29,32,34,35,37,38,39,40,41],correlation_matrix:[0,1,6,8,12,33,36],correspond:[1,4,6,7,9,10,12,13,16,17,18,24,25,26,29,32,33,34,35,38,39],cortex:[13,38,39],cosin:[4,7,35],cost:[1,3,4,6,7,8,9,10,13,14,17,18,20,22,23,26,27,32,38],cost_deep_grad:[3,41],cost_func:[40,41],cost_func_deriv:[40,41],cost_funct:[3,41],cost_function_deep:[3,41],cost_function_deep_grad:[3,41],cost_function_grad:[3,41],cost_function_train:[40,41],cost_function_v:[40,41],cost_grad:[3,41],cost_sum:[3,41],costcrossentropi:[40,41],costlogreg:[40,41],costol:[14,22,37,38,40,41],could:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,18,23,25,26,27,29,32,33,34,35,36,37,38,39,40,41],coulomb:[1,32],count:[1,10,28,29,30,32],countor:[14,37],coupl:[5,6,7,23,34,35],cours:[1,2,4,6,12,23,26,27,28,30,33,35,40,41],courvil:[22,27,28,31,32,33,37,38],cov:[6,7,12,25,29,32,33,34,35],cov_xi:[6,12,33],cov_xx:[6,12,33],cov_yi:[6,12,33],covari:[0,1,8,24,25,32,34,36],covariance_matrix:[6,12,15,33],cover:[1,6,18,24,30,31,33,34],covert:[1,32],covxi:29,covxx:29,covxz:29,covyi:29,covyz:29,covzz:29,cpu:[2,40,41],cqofi41lfdw:[39,40],craft:4,creat:[2,4,5,6,10,11,12,13,23,24,32,34,36,37,38,39,40,41],create_biases_and_weight:[2,40,41],create_convolutional_neural_network_kera:4,create_lay:23,create_layers_4:[],create_layers_batch:23,create_neural_network_kera:[2,40,41],create_x:[6,12,33,40,41],credit:[1,8,30,32,36],crim:1,crime:1,criteria:[1,5,10,11,15,29,32],criterion:[10,11,14,36,37,39],critic:[7,26,33,34],critiqu:[26,27],cross:[1,2,4,8,10,11,14,23,24,27,29,32,33,34,37,38,39,40,41],cross_entropi:[5,23],cross_val_scor:[7,35,36],cross_valid:[0,8,11,36],crossvalid:[7,35],crucial:[2,29,39,40],cs231:4,cs231n:41,cs231n_2017_lecture4:41,cs:[28,30],csr_matrix:[25,32],csv:[1,5,7,8,10,32,35,36],ctnk:[2,40,41],cube:39,cubic:1,cumbersom:[6,34,35],cumsum:[11,12,32],cumul:[8,11,29],cumulative_heads_ratio:11,cup:[6,34,35],current:[2,3,4,5,14,15,22,31,36,37,38,39,40,41],curs:[1,33],curv:[0,7,8,11,13,26,36,38,39,40],curvatur:[14,36,37,38],custom:[7,15,26],custom_cmap2:[10,11],custom_cmap:[10,11],cutpoint:10,cv:[0,7,8,11,35,36],cvxbook:[14,36],cvxopt:[6,9,33],cybenko:39,cycl:[2,13,38,39,40,41],cyclotron:34,d2_g_t:[3,41],d:[2,3,4,5,6,7,8,9,10,11,12,14,15,23,25,29,30,32,33,34,35,36,37,38,39,40,41],d_f:[14,36,37],d_g_t:[3,41],d_net_out:[3,41],da:[4,39],dagger:[6,25,33],dai:[2,10,24,40,41],damp:4,daniel:30,darget:10,darkr:29,dat:[1,32],dat_id:[1,7,8,10,32,33,35,36],data1:15,data2:15,data3:15,data4:15,data:[3,5,6,9,11,13,14,15,18,19,20,22,25,27,31,35,37,38],data_id:[1,7,8,10,32,33,35,36],data_indic:[2,40,41],data_panda:32,data_path:[1,7,8,10,32,33,35,36],databas:[2,40,41],datafil:[1,7,8,10,26,32,33,35,36],datafram:[0,1,5,6,8,10,12,32,33,36],datapoint:[0,2,6,7,8,12,14,33,35,36,37,38,40,41],dataset:[0,1,5,7,8,9,10,11,12,14,15,16,17,20,23,26,27,32,35,36,37,38,39],datatyp:5,date:[0,16,17,18,19,20,21,22,23,26,27,32,33,34,35,36,37,38,39,40,41],daughter:11,david:[31,32],davison:35,db:39,dbh:[2,40,41],dbo:[2,40,41],dc5e85cd93c3:27,dcomposit:25,ddot:[3,41],dead:[2,39,40,41],deadlin:[17,18,19,20,21,22,23,28],deal:[1,2,4,6,7,9,12,14,15,18,19,25,26,29,32,33,34,36,37,38,39,40,41],dealt:1,debt:[8,36],debug:[1,6,7,34,35,36,40,41],decad:[1,4,38],decai:[1,14,29,32],decemb:[28,30,32],decent:11,decid:[1,3,4,6,7,10,26,34,35,36,40,41],decim:[1,32,40,41],decis:[1,2,9,12,24,31,32,40,41],decision_funct:9,decision_tre:10,decisiontreeclassifi:[10,11],decisiontreeregressor:[1,10,11],decisiontreeregressordecisiontreeregressor:10,declar:[1,5,25,32],decompos:[6,7,25,33,39],decomposit:[1,7,13,26,32,34,38,39],decompost:[6,33],deconvolut:4,decorrel:[11,14,22,37,38],decreas:[2,3,5,6,7,11,12,14,34,35,36,37,38,40,41],deduc:[1,32],deep:[4,8,13,14,22,24,27,28,31,33,37,38],deep_neural_network:[3,41],deep_param:[3,41],deep_tree_clf1:10,deep_tree_clf2:10,deep_tree_clf:[10,11],deepcopi:[40,41],deepen:[6,24,32],deeper:[1,4,5,32],deepimag:41,deeplearningbook:[27,31,32,34],deer:4,def:[0,1,2,3,4,5,6,7,8,9,10,11,12,14,15,22,23,26,29,32,33,34,35,36,37,38,39,40,41],def_covari:29,default_tim:5,defect:[6,33],defici:[6,33],defin:[0,1,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,22,23,25,26,27,29,33,34,35,36,37,38],definit:[2,3,6,7,8,9,11,12,13,14,18,25,29,33,34,35,36,37,41],defint:29,defualt:[40,41],defvjp:[],defvjp_argnum:[],degre:[4,6,7,9,10,11,12,16,17,18,20,26,29,32,34,35,36],deisenroth:33,del:[2,40,41],delet:7,delimit:5,deliv:[28,32],delta:[1,3,4,7,9,13,14,15,22,32,37,38,39,40,41],delta_0:[4,39],delta_1:[4,39,40],delta_2:[4,39,40],delta_2a_1:[39,40],delta_3:4,delta_4:4,delta_5:4,delta_:[2,25,39,40,41],delta_h:[1,2,32,40,41],delta_i:[39,40],delta_j:[4,13,39,40,41],delta_k:[13,39,40,41],delta_l:[2,4,40,41],delta_matrix:[40,41],delta_momentum:[14,22,37,38],delta_n:[1,4,32],delug:24,delv:1,demand:[14,36,37],demonstr:[1,4,6,7,8,12,13,24,32,33,34,35,36,40,41],demystifi:[38,39,40],den:5,denomin:[2,6,34,35,40,41],denot:[2,3,7,8,14,29,36,37,38,40,41],dens:[2,4,5,40,41],densiti:[1,3,7,29,32,35,41],depart:[0,27,30,32,33,34,35,36,37,38,39,40,41],depend:[1,2,3,5,6,7,8,9,12,13,14,16,22,24,25,26,29,32,33,34,35,36,37,38,39,40,41],depict:29,deploy:[1,16,24,26,32],deprec:37,deprecationwarn:37,depth:[1,4,10,11,25,35,40],deriv:[0,1,2,3,7,8,9,11,12,14,17,18,20,22,24,26,32,38,41],derivati:[14,37,38],derivative_fn:[14,37],derivb1:[39,40],derivb2:[39,40],derivw1:[39,40],derivw2:[39,40],descend:[6,10,12,33,34],descent:[1,2,4,8,9,13,32,33,39,40],descent_i:[22,38],descent_x:[22,38],describ:[1,3,5,6,7,9,11,12,13,14,19,22,25,26,27,32,34,35,37,38,39,41],descript:[1,9,10,27,32,40,41],design:[1,2,4,5,6,7,8,11,12,13,14,16,17,18,19,20,26,27,32,34,35,36,37,38,39,40,41],designmatrix:[1,32],desir:[1,3,5,6,14,15,32,33,36,37,40,41],despit:[2,13,38,39,40],destroi:25,det:[6,25,33],detail:[1,7,12,14,15,23,25,26,27,33,36,37,40],detect:[3,4,9,13,38,39],determin:[1,3,4,5,6,7,9,10,11,12,13,14,17,25,29,32,33,34,35,36,37,38,39,40,41],determinist:[0,8,14,29,36,37,38,39],deternin:39,dev:[2,26,39,40],develop:[1,4,6,9,11,12,13,22,24,25,26,27,32,33,34,35,38,39],deviat:[1,2,3,5,6,7,20,26,29,32,33,34,35,39,40,41],devic:[4,5,15],devis:[13,38,39],df1:32,df:[5,9,12,14,32,37,39],di:1,diag:[6,9,33,34],diagnost:[2,11,39,40],diagon:[0,1,6,8,14,19,22,25,26,29,32,33,34,36,37,38,40,41],diagonaliz:[6,33],diagram:11,diagsvd:7,dice:[7,29,35],dict:[7,9,36,40,41],dictionari:[1,40,41],did:[0,1,2,6,7,8,11,12,15,26,27,32,34,35,36,37,40,41],die:[2,39,40,41],diff1:[3,41],diff2:[3,41],diff:[3,39,41],diff_ag:[3,41],diffeent:9,differ:[1,2,3,4,5,6,7,8,10,11,12,13,14,15,16,18,22,23,24,25,26,29,31,32,33,34,35,36,39,40,41],different:[39,40,41],differenti:[1,4,17,22,23,24,25,32,33,36,40],differential_oper:[],difficult:[1,2,7,11,14,22,26,29,32,35,37,38,39,40],difficulti:[1,2,14,32,36,37,38,39,40],diffonedim:[3,41],digit:[1,2,4,5,7,26,27,28,30,32,40,41],digress:39,dilemma:[14,37,38],dilut:[2,39,40],dim:[5,12,15,25,40,41],dimens:[1,2,3,4,5,6,9,10,12,15,17,18,25,32,33,34,39,40,41],dimension:[1,5,6,7,10,12,14,15,20,22,24,25,27,32,34,35,36,37,38],dimensionless:[1,4,32],diment:25,dimnsion:5,diod:4,direct:[1,2,3,5,8,12,13,14,15,22,32,33,36,37,38,39,40,41],directli:[2,5,6,7,29,33,40,41],directori:[1,8,36],disadvantag:[1,32],disappear:[4,7,35],disc_loss:5,disc_tap:5,discard:[7,12,35,36],disciplin:[1,4,13,38,39],disclaim:29,discontinu:39,discord:[23,32],discourag:[14,36,37],discov:[1,32],discover:[6,34],discret:[2,4,6,8,14,34,35,36,37,38,39,40],discrimin:[0,5,8,11,12,36],discriminator_loss:5,discriminator_loss_list:5,discriminator_model:5,discriminator_optim:5,discuss:[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,24,25,26,27,29,31,32,33,34,35,37,38,39,40,41],diseas:[8,36],disguis:[7,33,34],disord:[2,8,36,40,41],dispai:[38,39],displai:[0,1,2,4,5,6,7,8,9,10,11,12,13,15,26,29,32,33,35,36,38,39,40,41],displaystyl:[1,6,18,32,33,34],disregard:[1,32],dissimilar:[12,15],dist:15,distanc:[1,9,10,12,15,29],distance_list:10,distinct:[0,4,8,9,10,11,15,36],distinctli:9,distinguish:[0,1,5,8,9,29,32,36],distplot:1,distribut:[1,2,5,7,8,11,12,14,15,16,17,19,20,23,24,25,26,27,32,33,36,37,39,40,41],distrubut:[1,16,24,26,32],dive:[1,9,25,32],diverg:[2,14,36,37,38,39,40],divid:[1,2,4,6,7,8,9,10,12,13,27,29,32,33,34,35,36,38,39,40,41],divis:[7,9,10,14,22,25,29,35,36,37,38,39,40,41],dlopen:[4,5,15],dna:[8,36],dnn1:5,dnn2_gru2:5,dnn:[1,2,3,5,13,32,38,39,40,41],dnn_kera:[2,40,41],dnn_model:[2,40,41],dnn_numpi:[2,40,41],dnn_scikit:[1,2,32,40,41],doamin:[34,35],doc:[1,7,17,18,20,21,24,26,27,28,30,31,32,36,37,38],document:[0,5,10,12,14,36,41],doe:[1,2,3,4,5,6,7,9,11,12,13,14,15,17,22,23,25,26,29,32,35,36,37,40,41],doesn:[4,10,13,39,40],dog:[2,4,5,40,41],doi:41,domain:[6,9,14,26,27,34,35,36,37],domin:[1,32],don:[1,2,4,6,7,8,9,12,14,16,22,23,24,26,27,32,33,36,37,38,39,40,41],done:[1,3,4,5,6,7,10,11,12,14,18,25,26,32,33,34,35,36,37,39,40,41],dot:[0,1,3,4,6,7,8,9,10,11,12,13,14,22,25,26,29,32,33,34,35,36,38,39,40,41],doubl:[4,5,25,32],doubli:[2,39,40],doubt:26,down:[1,4,7,10,12,13,14,22,36,37,38,41],download:[1,2,4,6,7,25,26,31,32,40,41],downsampl:4,dozen:[2,39,40],dq:[7,35],drag:[14,37,38],dramat:12,drastic:5,draw:[5,7,11,14,35,36,37],drawback:[1,2,4,14,33,36,37,38,39,40],drawn:[2,5,7,8,12,29,32,35,36,39,40,41],drive:[4,5,23],driven:4,drop:[1,2,6,7,12,14,22,29,32,33,34,35,37,38,39,40],dropna:[1,7,32,35],dropout:5,dt:[3,4,14,23,29,37,38,39,41],dtype:[1,2,4,5,15,22,23,25,32,33,38,39,40,41],dual:9,dualiti:7,dub:[1,32],due:[2,3,6,7,9,11,13,14,30,32,33,35,36,37,38,39,40,41],dummi:1,dure:[1,2,3,4,5,9,10,12,18,21,24,26,32,35,37,38,39,40,41],dwell:1,dwh:[2,40,41],dwo:[2,40,41],dx:[3,4,9,29,39,41],dx_1:29,dx_1p:[7,35],dx_2p:[7,35],dx_mp:[7,35],dx_n:29,dxp:[7,35],dy:[2,9,29,39,40,41],dylib:[4,5,15],dynam:5,dz:9,e:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,19,20,22,23,29,30,32,33,34,35,36,37,38,39,40,41],e_:[1,3,32,41],e_z:23,each:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,18,22,24,25,27,28,29,30,32,33,34,35,37,38,39,40,41],eager:35,eapprox:[1,32],earli:[2,14,22,37,38,40,41],earlier:[1,6,8,9,10,12,13,14,21,22,23,32,33,36,37,38,39,40],earthexplor:[7,26],eas:[7,10,15,35],easi:[0,1,6,7,8,9,10,11,12,13,14,19,22,23,24,25,27,32,33,34,35,36,37,38,39,40,41],easier:[6,7,9,10,14,23,27,29,32,33,35,37,38],easiest:[14,36,37],easili:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,25,26,32,33,34,35,36,37,39,40,41],eastern:[30,32],ebind:[1,32],eblock:10,econometr:32,economi:6,ecosystem:[24,32],ect:28,edg:4,edgecolor:[7,35],edit:23,edu:[14,27,36,41],educ:[0,1,26,27,32,36],eff:29,effect:[2,5,11,14,29,37,38,39,40,41],effic:[2,40,41],effici:[1,4,11,14,23,24,25,29,32,36,37,38,39],efron:[7,35],egrad:[14,37,38],eig:[6,12,14,22,25,29,32,33,36,37,38],eigen:29,eigenpair:[6,12,33],eigenvalu:[1,6,9,12,14,22,25,32,33,34,36,37,38],eigenvector:[6,12,14,18,33,34,37],eight:[25,32],eigval:[25,29,32],eigvalu:[12,14,22,36,37,38],eigvec:[25,29,32],eigvector:[12,14,22,36,37,38],eispack:[25,32],either:[0,2,6,7,8,9,10,11,12,14,16,17,26,27,29,32,33,34,35,36,37,39,40,41],elabor:29,elarn:4,electr:[1,4,13,32,38,39],electron:32,eleg:12,element:[0,2,3,4,5,6,7,8,9,12,13,14,18,19,21,22,23,24,25,26,27,28,31,33,34,35,36,39,40,41],elementari:[11,14,25,37,38,39],elementwis:[4,14,37,38],elementwise_grad:[3,14,37,38,40,41],elessar:32,elif:[15,22,38,40,41],elim:25,elimin:[4,9],els:[0,2,3,4,5,8,10,13,14,15,23,25,36,37,38,39,40,41],elu:2,elus:[1,32],email:[23,28,30,32],embark:39,embed:[1,12,33],embodi:[7,20,26,35],emit:29,emner:31,emphas:[1,11,24,32],emphasi:[1,24,31,32],empir:[2,12,29,39,40],emploi:[1,2,6,7,12,14,26,27,29,32,33,34,35,36,37,40,41],employ:1,empti:[7,11,35,40,41],emul:[13,38,39],en:[24,31],enabl:[12,22,38,40,41],enbodi:[7,35],encod:[1,4,6,8,10,12,15,18,32,33,34,36],encompass:[1,26,29],encount:[0,1,2,6,7,8,14,22,23,29,32,33,35,36,37,38,39,40,41],encourag:[26,27],end:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,17,22,25,26,29,30,32,33,34,35,36,37,38,39,40,41],end_box:[],end_nod:[],end_valu:[],endpoint:[4,7],energi:[1,5,7,33,35],enet_coordinate_desc:7,enforc:[13,38,39],eng:31,engin:[1,2,4,5,24,32,39,40],english:[26,27],enocurag:26,enoent:[4,5,15],enorm:4,enough:[1,7,14,32,35,36,37],ensembl:[2,10,39,40],ensur:[1,2,3,4,6,7,12,14,22,29,33,34,35,36,37,38,39,40,41],entail:32,enter:[6,7,18,33,34],enthought:[1,16,24,26,32],entir:[2,4,8,10,23,24,29,32,36,37,38,39,40,41],entiti:[10,13,25,32],entri:[1,6,9,12,13,25,32,33,34,35],entropi:[2,4,8,11,14,23,32,37,38,40,41],enumer:[1,2,3,4,5,7,9,23,32,33,34,40,41],env:[0,1,2,3,4,5,7,8,9,10,12,15,22,23,29,32,34,36,38,40,41],environ:[3,10,22,23,24,26,38,41],eo:[1,7,32,35],eol:[1,32],eosfit:[1,32],epoch:[1,2,4,5,13,14,22,23,27,32,37,38,40,41],epsilon:[1,6,7,8,14,20,26,32,33,34,35,36,37,38,39],epsilon_0:[1,32],epsilon_1:[1,32],epsilon_2:[1,32],epsilon_:[1,32],epsilon_i:[1,32,33],eq:[4,14,15,25,29,36,37],eqnarrai:[4,6,7,34,35],equal:[1,2,3,4,5,6,7,9,10,12,13,14,15,16,23,25,26,27,29,32,33,34,35,36,37,38,39,40,41],equal_nan:23,equat:[0,2,4,5,6,7,8,9,10,11,12,14,15,17,18,19,20,22,25,26,29,35,38],equilibrium:[3,13,38,39,41],equiv:[4,14,25,29,36,37],equival:[1,2,6,8,9,12,14,22,24,25,32,33,34,35,37,38,40,41],equivel:23,erf:29,eriador:32,eric:[40,41],eridg:32,err:[1,8,11,36],err_:[7,35],err_sqr:[3,41],errat:[14,36,37],errno:[1,4,5,8,15,36],erron:[3,41],error:[0,2,3,5,6,7,8,10,12,13,14,16,17,18,19,20,22,23,24,25,26,27,29,34,36,37,38,39,40,41],error_estimate_corr_tim:29,error_hidden:[2,40,41],error_output:[2,40,41],escap:[14,36,37,38],especi:[2,4,10,13,14,26,27,37,38,39,40,41],essenti:[1,6,7,10,11,13,15,26,27,29,33,38,39,40],establish:[1,7,11,12,16,26,27,32],estim:[0,1,2,6,7,8,11,12,14,24,29,32,33,36,37,38,39,40,41],estimated_mse_fold:[7,35,36],estimated_mse_kfold:[7,35,36],estimated_mse_sklearn:[7,35,36],et:[1,3,5,16,17,18,26,27,31,32,33,34,35,36,37,39,40,41],eta0:[9,14,36,37],eta:[1,2,4,9,13,14,22,27,32,36,37,38,39,40,41],eta_:[14,22,37,38],eta_t:[14,37,38],eta_v:[1,2,4,32,40,41],etc:[1,2,4,6,8,9,10,12,13,14,15,16,22,24,25,26,27,29,33,36,37,38,40,41],ethic:24,etsim:35,euclidean:[1,15,33],evalu:[1,3,4,5,6,7,10,14,23,26,29,32,33,34,35,36,37,38],evaluationform:[21,26,37,38],evaluationgrad:[21,26,37,38],evalut:[14,37,38],even:[1,2,4,5,6,7,9,10,11,12,13,14,15,24,25,29,32,33,34,35,36,37,38,39,40],evenli:5,event:[0,6,8,11,29,34,35,36],eventu:[1,6,7,12,13,14,26,27,30,33,34,35,36,37,38,39],everi:[1,2,3,4,5,6,7,10,11,12,13,14,15,23,24,29,30,32,33,34,35,36,37,38,39,40,41],everyth:[5,13,22,23,39,40],everywher:[5,14,36,37],evolv:1,exact:[1,6,12,13,14,25,29,32,33,37,39,40],exactli:[1,4,5,7,13,24,26,33,35,38,39,40],exam:32,examin:[7,35],exampl:[6,12,13,14,16,17,20,21,24,25,26,27,29,31],exce:[2,13,14,22,37,38,39,40],excel:[1,2,5,6,11,27,32,33,39,40],except:[1,3,4,5,7,8,9,10,25,36,40,41],excess:[1,32],excit:1,exclud:[2,7,13,26,33,34,35,36,38,39,40],exclus:[1,2,4,7,29,32,35,36,39,40,41],execut:[3,6,14,33,37,38,41],exemplifi:[14,37,38],exercic:[30,32],exercis:[6,24,26,27,28,30,34,35,36,37,38,40],exercisesweek35:18,exhaust:[7,35,36],exhibit:[1,6,7,9,32,33,35],exist:[1,2,3,4,5,6,7,8,9,10,14,15,19,25,26,27,32,33,34,35,36,37,39,40,41],exit:[6,25,33],exp:[0,1,2,3,6,7,8,9,11,12,13,14,16,17,22,23,26,29,32,33,34,35,36,37,38,39,40,41],exp_term:[2,40,41],expand:[0,6,8,12,14,33,36,37],expandus:[1,8,36],expans:[1,4,6,9,11,13,14,32,33,36,37,39],expect:[1,2,6,7,8,12,13,14,16,17,20,22,24,26,27,32,33,36,37,38,39,40,41],expectation_value_of_h_wrt_p:29,expens:[7,11,14,36,37,38],experi:[1,2,7,9,14,24,26,32,33,35,36,37,38,40,41],experiment:[1,5,7,10,29,32,35],expert:[2,10,39,40],explain:[1,7,10,11,12,14,20,26,32,36,37],explained_variance_ratio_:12,explan:27,explanatori:[1,32],explicit:[1,4,7,14,22,25,26,32,33,34,37,38],explicitli:[1,5,9,22,23,38],explod:2,exploit:[1,4,13,14,32,37,38,39],explor:[2,5,7,9,14,24,26,27,36,37,38,39,40],expon:[2,40,41],exponenti:[1,2,6,7,11,14,26,29,32,34,35,36,37,38,39,40],export_graphviz:10,export_text:10,exporttext:10,expos:24,expr:39,express:[1,3,4,6,7,8,11,13,14,17,22,25,26,27,29,32,34,35,38,41],exptmean:29,exptvari:29,extend:[1,3,8,12,14,18,24,32,41],extens:[1,13,16,24,32,38,39],extent:[1,2,7,31,35,40,41],extern:[4,7,10],extra:[2,4,6,30,32,33,40,41],extract:[0,1,4,6,7,8,9,12,14,25,32,33,36,37,39],extrapol:[1,32],extrem:[1,2,5,6,7,8,9,10,14,17,25,33,34,35,36,37,38,39,40],extremum:[14,36,37],extrins:12,ey:[1,6,7,14,15,25,32,34,36,37,38],f11:[1,32],f12:[1,32],f13:[1,32],f1:[14,37,38],f1_grad:[14,37,38],f1d:[14,37],f2:[14,37,38],f2_grad_x1:[14,37,38],f2_grad_x1_analyt:[14,37,38],f2_grad_x2:[14,37,38],f2_grad_x2_analyt:[14,37,38],f3:[14,37,38],f3_grad:[14,37,38],f3_grad_analyt:[14,37,38],f4:[14,37,38],f4_grad:[14,37,38],f4_grad_analyt:[14,37,38],f5:[14,37,38],f5_grad:[14,37,38],f6:[14,37,38],f6_for:[14,37,38],f6_for_grad:[14,37,38],f6_grad_analyt:[14,37,38],f6_while:[14,37,38],f6_while_grad:[14,37,38],f7:[14,37,38],f7_grad:[14,37,38],f7_grad_analyt:[14,37,38],f8:[14,37],f8_grad:[14,37],f9:[1,14,32,37],f9_altern:[14,37],f9_alternative_grad:[14,37],f9_grad:[14,37],f:[1,2,3,4,5,6,7,8,9,11,13,14,15,17,19,20,22,25,29,30,32,33,34,35,36,37,38,39,40,41],f_0:[4,11],f_1:[11,14,36,37],f_2:[13,14,36,37,38],f_3:[13,38],f_:11,f_d:29,f_grad:[14,37,38],f_grad_analyt:[14,37,38],f_i:[1,7,13,17,35,38],f_m:[4,11],f_n:4,f_raw:[3,23],f_vec:[3,41],f_wrap:[3,23],face:[14,36,37,38],facecolor:[7,9,29,35],facil:[0,1,16,24,32,36,37,38,40,41],facilit:[13,38,39],fact:[1,2,4,6,10,12,13,14,32,33,34,35,36,37,38,39,40],factor:[1,2,4,6,7,10,11,12,14,25,29,32,33,34,35,37,38,40,41],factori:[14,37,38],fade:7,fafab0:[10,11],fahimeh:[30,32],fail:[0,1,7,9,10,12,14,30,32,35,36,37,39],fail_on_error:[1,8,36],failur:[8,36],fairli:[2,3,29,39,40,41],faisal:[17,33],fake:5,fake_loss:5,fake_output:5,fale:27,fall:[9,10,28],fals:[1,2,3,4,5,6,7,8,10,11,15,23,25,26,32,33,34,35,36,37,38,40,41],famili:[1,8,9,29,36],familiar:[1,4,6,7,9,16,24,25,26,29,32,34,35,39],famou:[7,13,40,41],far:[1,4,5,6,7,9,12,13,14,15,23,32,33,36,37,38,39],fashion:[1,10,11,32],fast:[2,4,7,11,13,14,24,29,32,35,36,37,39,40],faster:[2,12,14,23,37,38,39,40,41],fastest:[14,25,36,37],favor:[0,8,36],favorit:29,fc:4,fd:[1,8,36],featur:[0,1,2,4,6,7,8,9,11,12,13,14,19,23,24,26,27,29,32,34,35,36,37,38,39,40,41],feature_nam:[0,1,2,8,10,23,36,40,41],feautur:10,fed:[2,39,40,41],feed:[1,3,4,12,23,24,27,32],feed_forward:[2,23,40,41],feed_forward_4:[],feed_forward_all_relu:23,feed_forward_batch:23,feed_forward_out:[2,40,41],feed_forward_train:[2,40,41],feedback:[5,21,32],feeddorward:5,feedforward:[2,5,13,39,40,41],feel:[1,6,7,12,14,16,17,22,23,24,26,27,30,32,34,37,38,39],feet:1,felt:[26,27],fetch:[7,26],few:[2,4,5,6,10,29,32,37,39,40,41],fewer:[1,10,12,32],ffnn:[2,13,27,38,39,40,41],field:[1,4,7,13,24,38,39],fifteen:39,fifth:[1,7,16,17,26,32],fig:[0,1,2,3,4,5,7,8,13,14,15,26,32,36,37,38,39,40,41],fig_id:[1,7,8,10,32,33,35,36],figaxi:29,figsiz:[0,1,2,3,4,5,7,8,9,10,11,32,35,36,40,41],figur:[0,1,2,3,4,5,6,7,8,9,10,11,13,14,15,16,17,22,24,26,27,32,33,34,35,36,37,38,39,40,41],figure_id:[1,7,8,10,32,33,35,36],figurefil:[1,7,8,10,32,33,35,36],file:[1,3,4,5,6,7,8,10,15,23,26,27,32,33,34,35,36],file_prefix:5,filenam:32,filenotfounderror:[1,8,36],fill:[6,10,33,40,41],filter:[1,4,5,8,36],financ:1,find:[1,2,3,4,6,7,8,9,10,11,12,13,14,15,16,17,22,23,24,26,27,29,32,33,34,36,38,39,40,41],fine:[1,15,16,32],finicki:[],finish:[3,23,40,41],finit:[4,6,7,13,14,18,29,33,34,35,37,38,39],first:[0,1,2,3,4,6,7,8,9,10,11,12,14,15,16,17,18,22,23,25,26,29,30,31,35,38,41],first_moment:[22,37,38],first_term:[22,37,38],firsteigvector:12,fit:[0,2,4,5,6,7,8,9,10,12,13,14,16,17,18,22,26,27,29,33,35,36,37,38,39,40,41],fit_beta:[7,33,34],fit_intercept:[1,6,7,33,34,35],fit_mod:10,fit_transform:[1,7,9,10,12,35,36],fiti:[1,32],five:[1,10,18,26,32,33,36,39],fix:[1,4,5,7,11,12,13,14,22,26,27,32,35,36,37,38,40,41],fixedloc:7,flag:5,flat:[4,5,13,14,15,22,36,37,38],flatten:[2,4,5,6,25,40,41],flexibl:[2,7,9,11,13,27,32,35,38,39,40,41],flip:[23,30,32],float32:[5,10,22,38,40,41],float64:[5,22,25,32,33,38,39,40,41],floatingpointerror:[40,41],floor:[40,41],flop:[6,25,33],flow:[2,5,13,38,39,40,41],flower:23,fluctuat:[6,34],flush:[40,41],fly:12,fm:[1,32],fmax:4,fmesh:[14,37],fn:8,fname:[1,8,36],focu:[1,4,5,6,7,16,24,27,31,32,33,34,35],focus:[0,2,7,8,25,33,34,36,40,41],fold:[7,10,26],folder:[1,2,4,5,7,15,26,27,32,35,37,40,41],follow:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41],font:[1,8,29,32,36],fontdict:29,fontsiz:[2,7,9,10,11,29,40,41],fontweight:[2,40,41],footprint:4,foral:[9,33,39],forc:[1,6,7,11,12,32,33,34,36,39],forcast:5,forecast:[5,13,38,39],forest:[1,2,10,24,32,39,40],forget:12,form:[0,1,4,5,6,7,8,9,10,12,13,14,18,22,24,25,26,27,29,32,33,34,35,36,37,38,39,40,41],formal:[4,5,15,29,39],format:[0,1,2,4,5,7,8,9,10,11,12,24,29,31,33,34,35,36,38,39,40,41],format_data:5,formatstrformatt:[7,14,26,36,37],formul:[5,7,12,15],formula:[4,14,29,36,37,38,39],forth:[5,13,38],fortran2003:[24,32],fortran2008:[26,27],fortran90:29,fortran:[1,16,24,25,32],fortun:[1,12,33],forward:[1,4,7,23,24,25,27,32,35],forwardpropag:[39,40],foster:32,found:[2,3,4,5,6,7,13,14,15,20,21,23,26,27,32,33,34,35,37,38,39,40,41],foundat:[24,32],four:[5,6,7,9,13,23,25,26,28,30,32,34,38,39,40],fourier:[1,32,39],fourierdef1:4,fourierdef2:4,fourierseriessign:4,fourth:[13,32,33],fp:8,frac:[0,1,2,3,4,6,7,8,9,10,11,12,13,14,15,16,17,18,20,23,25,26,27,29,32,33,34,35,36,37,38,39,40,41],fractal:10,fraction:10,frame:[0,8,36],framework:[2,4,5,9,11,15,29,40,41],frank:[6,12,22,27],frankefunct:[6,7,12,26,33],fredli:[23,30,32],free:[1,7,12,14,16,17,22,23,24,25,26,27,29,30,31,32,37,38,39],freecodecamp:24,freedom:[6,34],freeli:[1,26],frequenc:[0,4,7,8,29,35,36],frequent:[1,9,10,14,36,37,38],frequentist:24,fresh:11,frf4l5qax1m:41,fridai:[17,18,19,20,21,22,23,30,32],friedman:[7,19,26,28,31,32],friendli:5,frodo:32,frog:4,from:[1,2,3,4,5,7,8,9,10,12,14,15,17,18,19,20,23,24,25,26,29,30,31,32,41],from_cod:10,from_logit:[4,5],from_tensor_slic:5,fromnumer:3,front:[1,5,6,16,17,32,33,34],fulfil:[3,6,13,33,38,39,40,41],full:[1,2,4,6,8,10,11,14,23,29,32,33,36,37,38],full_matric:[6,33],fulli:[4,7,13,28,29,35,36,38,39],fun:[24,32],fun_nam:[],func:[3,23,40,41],functionali:12,fundament:[1,7,24,32,35],funtion:[3,41],furthemor:39,further:[3,8,10,32,39,41],furthermor:[0,1,4,6,7,8,12,13,14,18,24,26,32,33,34,35,36,37,38,39,40],futur:[1,5,9,10,32,37],futurewarn:9,fy:[16,23,26,27,28,30,31,32],fys4155:[26,27],fys5419:[31,32],fys5429:[31,32],g0:[3,41],g:[1,2,3,4,5,7,9,10,11,12,14,29,32,34,35,36,37,38,39,40,41],g_0:[3,41],g_1:[3,11,41],g_2:[3,11,41],g_3:39,g_:[3,10,11,41],g_analyt:[3,41],g_dnn_ag:[3,41],g_euler:[3,41],g_i:[3,39,41],g_j:39,g_m:[4,11],g_n:4,g_re:[3,41],g_t:[3,40,41],g_t_d2t:[3,41],g_t_d2x:[3,41],g_t_dt:[3,41],g_t_hessian:[3,41],g_t_hessian_func:[3,41],g_t_invers:[40,41],g_t_jacobian:[3,41],g_t_jacobian_func:[3,41],g_trial:[3,41],g_trial_deep:[3,41],g_vec:[3,41],gain:[2,6,8,10,11,14,33,34,35,37,38,40,41],galleri:[1,32],game:5,gamge:32,gamma1:9,gamma2:9,gamma:[1,3,9,10,11,12,14,22,32,36,37,38,41],gamma_0:11,gamma_1:11,gamma_1x:11,gamma_:[1,32],gamma_i:[1,9,29,32],gamma_j:[14,37,38],gamma_k:[14,36,37],gamma_m:11,gamma_x:[1,32],gap:[7,9],gate:[0,5,13],gather:[1,2,13,33,38,39,40,41],gaug:[13,38,39],gaussbacksub:25,gaussian:[5,6,7,9,15,29,34,35],gaussian_point:15,gaussian_rbf:9,gave:[14,37,38],gavra:32,gbc:[28,32],gca:[3,7,9,14,41],gd:[2,27,36,39,40],gd_clf:11,gdclassiffiercgain:11,gdclassiffierconfus:11,gdclassiffierroc:11,gdfit:[0,38],gdm:[14,22,37,38],gdregress:11,ge:[2,6,8,29,33,36,39,40,41],gen:[1,8,36],gen_loss:5,gen_tap:5,gender:[1,32],genener:5,gener:[0,1,2,3,4,6,7,9,11,12,13,14,15,16,17,18,20,21,22,23,25,26,29,31,33,34,35,36,37,38,40,41],generaliz:[40,41],generallay:[13,38],generate_and_save_imag:5,generate_imag:5,generate_latent_point:5,generate_simple_clustering_dataset:15,generated_imag:5,generator_loss:5,generator_loss_list:5,generator_model:5,generator_optim:5,genom:24,geodes:12,geometr:[1,14,32],geometri:[6,34,35],georg:31,geotif:[7,26],geq:[3,6,9,10,14,33,34,36,37,41],geron:[1,16,22,28,31,32,37,38],get:[1,2,3,4,5,6,7,8,10,11,12,14,16,21,23,24,25,26,27,29,30,32,33,34,35,36,37,40,41],get_dummi:10,get_paramet:[3,41],get_split:10,get_yaxi:9,get_yticklabel:7,getattr:[],getval:[],gg:32,gibb:[24,32],gif:5,gini:11,gini_index:10,ginvers:14,git:[1,16,24,32],giter:[14,22,37,38],github:[1,7,10,16,17,18,20,21,22,24,26,27,28,30,31,32,33,36,37,38,39,40,41],gitlab:[1,16,24,26,27,32],gitta:[39,40,41],give:[1,2,3,4,6,7,8,9,10,11,13,14,15,18,24,26,27,29,32,33,34,35,36,37,38,39,40,41],given:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,22,23,25,26,27,29,32,33,34,35,36,37,38,39,40,41],glkfgrjhtlnplbx4:23,global:[0,7,8,14,26,36,37,38],gloriou:27,glorot:2,gnew:14,go:[1,2,4,6,7,9,10,12,13,14,23,26,32,33,34,35,36,37,39,40,41],goal:[1,8,10,32,36],goe:[1,2,3,6,7,14,15,19,25,32,33,34,35,36,37,38,40,41],golden:[14,37],gone:[6,33,34],gong:[2,40,41],good:[2,4,5,6,7,10,11,12,14,18,22,23,24,29,31,33,34,35,36,37,38,39,40,41],goodfellow:[5,27,28,31,32,33,34,36,37,39,40,41],googl:[2,5,22,23,24,32,38,40,41],got:[2,7,14,23,26,40,41],gotcha:[22,38],gotten:[32,40,41],gov:[7,26],gp:31,gpu:[2,14,22,24,32,37,38,40,41],grad:[3,14,22,23,37,38,40,41],grad_analyt:[14,37,38],grad_broadcast_to:[],grad_concatenate_arg:[],grad_np_sum:[],grade:[26,28],gradient:[1,4,5,8,9,10,13,23,24,32,33],gradient_bia:[40,41],gradient_desc:[22,37,38],gradient_func:23,gradient_weight:[40,41],gradientboostingclassifi:11,gradientboostingregressor:11,gradients_of_discrimin:5,gradients_of_gener:5,gradienttap:5,gradual:[2,15,39,40],grai:[5,7,26],graph:[2,10,12,13,14,36,37,38,39,40,41],graph_from_dot_data:10,graphic:[1,2,10,32,40,41],grasp:1,gray_r:[2,4,40,41],grayscal:4,great:[6,14,23,36,37,39,41],greater:[0,2,8,29,36,39,40,41],greatli:[14,37,38],greedi:[10,36],green:[1,4,10,29],gregor:[40,41],grei:5,grid:[0,2,4,7,8,9,13,29,33,34,35,38,39,40,41],gridsearch:36,gridsearchcv:36,groh:[39,40,41],grossli:[14,36,37],ground:[1,32],group:[1,7,8,10,15,24,26,27,28,30,32,35],groupbi:[1,32],grow:[2,4,10,11,39,40,41],growth:[1,32],gru:5,guarante:[1,5,14,29,32,33,36,37],guess:[2,5,11,14,15,22,27,36,37,38,40,41],guestrin:11,guid:[2,23,40,41],guidanc:26,guidelin:[21,37,38],h1:[3,41],h:[1,2,6,7,9,14,17,19,22,23,26,29,30,31,32,33,36,37,38,40,41],h_1:[3,14,36,37,41],h_2:[3,14,36,37,41],h_:[1,14,32,36,37],h_m:11,ha:[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,17,22,23,25,26,27,29,32,33,34,35,36,37,38,39,40,41],haa:30,habit:[1,33],had:[1,2,7,8,14,32,35,36,37,39,40,41],hadamard:[2,13,14,22,37,38,39,40,41],half:[2,9,10,39,40,41],halv:11,hand:[1,2,3,4,6,12,13,14,16,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38],handi:[4,26,27],handl:[1,2,3,6,10,12,24,34,39,40,41],handle_unknown:10,handsid:[13,39,40],handwrit:[13,38,39],handwritten:[2,6,40,41],handwrittennot:[36,38],happen:[2,3,4,5,6,7,11,14,29,33,37,38,39,40,41],hard:[0,2,8,9,11,14,23,36,37,39,40,41],hardcopi:[24,32],harder:[1,2,23,33,39,40],harmon:4,hasn:[2,40,41],hassl:[1,16,24,32],hast:[24,32],hasti:[1,7,16,17,18,19,26,28,31,32,33,35,36],hat:[1,2,6,7,8,10,11,12,13,14,18,19,25,26,33,34,35,37,38,39],have:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,22,23,24,25,26,27,29,30,32,33,34,35,36,37,38,39,40,41],haven:[2,40,41],he:[8,36],head:[1,5,11,29],header:[1,32],heads_proba:11,health:[1,33],hear:[1,14,32,37,38],heart:[1,8,32,36],heatmap:[0,1,2,4,8,32,36,40,41],heavili:1,heavisid:[2,40,41],height:[2,4,7,33,34,40,41],hein:41,held:[14,22,37,38],help:[1,2,5,13,14,17,22,26,27,32,37,38,39,40],helper:[5,15],henc:[1,6,7,9,10,11,13,14,19,26,32,33,34,35,36,37,38],henrik:[30,32],her:[8,36],here:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,22,23,24,25,26,27,29,32,33,34,35,36,37,38,39,40,41],hereaft:[1,9,13,32],hermitian:25,hessenberg:25,hessian:[1,3,6,14,17,22,34,38,41],heterogen:[10,11],hi:[8,36],hidden:[2,4,5,13,23,27,38],hidden_bia:[2,40,41],hidden_bias_gradi:[2,39,40,41],hidden_deriv:[40,41],hidden_func:[40,41],hidden_layer_s:[1,2,32,40,41],hidden_neuron:5,hidden_nodes1:[40,41],hidden_nodes2:[40,41],hidden_weight:[2,40,41],hidden_weights_gradi:[2,39,40,41],hierarch:[6,33,34],high:[1,2,3,4,5,6,7,10,11,12,14,15,22,23,24,25,26,32,33,35,36,37,38,39,40,41],higher:[1,2,4,6,7,9,14,22,26,27,32,33,34,35,36,37,38,40,41],highest:[2,3,40,41],highli:[1,4,5,11,19,24,25,27,31,32,33,34,35],highwai:1,hing:9,hint:[14,17,23,27,33,34,36,37],hip:[24,41],hire:1,hist:[0,5,7,8,29,35,36],histogram:[0,1,7,8,29,36],histor:[8,12,36],histori:[4,5,13,38,39],hitherto:[6,34],hjorth:[0,30,32,33,34,35,36,37,38,39,40,41],hobbi:29,hoc:[6,33],hoff:31,hold:[2,4,7,14,15,22,35,36,37,38,40,41],holder:[1,32],holm:41,holomorphic_grad:[],home:1,homepag:[26,27,32],homework:[7,14],homogen:[2,4,10,11,14,22,37,38],honchar:[3,41],hopefulli:[1,12,29,32,38],horizont:12,hornik:39,hors:[4,8,36],hot:[2,10,40,41],hour:[2,24,28,29,30,32,40,41],how:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,22,23,24,25,26,27,29,32,33,34,35,37,38,39,40,41],howev:[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,22,23,24,25,26,27,29,32,33,34,35,36,37,38,39,40,41],hspace:[1,5,9,11,29,32,39,40],hstack:[2,40,41],htf:[28,32],html:[0,1,10,12,18,20,23,24,26,27,28,30,31,32,34,39,40,41],http:[0,1,4,5,7,10,12,14,17,18,20,21,22,23,24,25,26,27,28,30,31,32,33,34,35,36,37,38,39,40,41],huang:[1,32],huber:[1,32],huge:[2,4,5,24,39,40,41],human:[1,2,4,7,10,13,33,34,38,39,40,41],humid:10,hundr:[2,39,40],hungri:[2,39,40],huse:32,hvae:38,hybrid:28,hydrogen:[1,32],hyper:[22,27],hyperbol:[2,5,13,41],hyperparam:9,hyperparamat:39,hyperparamet:[4,5,6,7,10,14,18,32,33,34,36,37,38],hyperplan:12,i0:[1,32],i1:[1,7,9,13,32,33,34,38],i2:[1,9,13,32,38],i3:[1,13,32,38],i5:[1,32],i:[0,1,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,22,23,25,26,27,29,30,32,33,34,35,36,37,38,39,40],i_1:[6,7,34,35],i_2:[6,7,34,35],i_:[14,36,37],i_siz:23,ian:31,iayaan2:23,ic:[2,27,40,41],id:[8,14,36,37,38],ida:[30,32],idea:[1,2,3,4,5,7,10,11,13,14,25,26,27,33,34,35,36,37,38,39,40,41],ideal:[1,3,7,9,14,29,32,35,38,40,41],idem:[7,35],ident:[6,7,13,14,18,25,26,33,37,38,40,41],identical:[34,35],identifi:[1,2,8,10,12,13,14,15,32,33,36,37,38,40,41],ieor:29,ifi:[31,41],ifs:[24,32],ignor:[1,2,4,10,33,39,40,41],ii:[25,29,40],iii:[25,32,40],ij:[1,2,4,7,9,13,15,17,19,25,26,29,32,33,34,38,39,40,41],ik:[1,25,32,33],ilg3ggewq5u:[39,40],illustr:[6,8,11,13,14,15,24,32,35,36,37,40],ilsvrc:[37,38],im:7,imag:[2,4,5,7,10,12,13,15,27,31,32,38,39,40,41],image_at_epoch_:5,image_batch:5,image_height:4,image_path:[1,7,8,10,32,33,35,36],image_width:4,imageio:[7,26],images_from_seed_imag:5,imagin:[2,39,40],immedi:[1,4,5,7,16,24,32],implement:[1,3,4,5,6,7,9,10,11,12,13,14,15,18,22,23,26,27,29,32,33,34,36,37,39],impli:[4,6,7,8,14,25,33,34,35,36,37],implicit:4,implicitli:[12,29],importantli:4,impos:[1,7,12,13,32,38,39,40,41],imposs:[1,6,32,33],impress:[1,13,32,38,39],improv:[1,5,6,10,11,12,14,22,23,26,27,33,38],impur:10,imread:[7,26],imshow:[2,4,5,7,26,40,41],in1:[],in2:[],in3050:[31,32],in3310:32,in4080:[31,32],in4300:[31,32],in4310:31,in5400:4,in5550:31,in_out_neuron:5,inaccur:[14,36,37],inact:[13,38,39,40],inadequ:[1,32],inch:[7,33,34],includ:[1,2,3,4,5,6,7,8,12,13,16,17,18,21,23,24,26,27,29,30,31,32,33,34,35,40,41],include_bia:[7,10,35],inclus:[18,38],incom:[13,38,39],incorrect:[2,40,41],incoveni:9,increas:[0,1,2,4,5,6,7,9,10,12,13,14,22,26,29,32,33,34,35,37,38,39,40,41],increasingli:29,increment:37,ind:7,inde:[1,3,5,6,7,14,32,33,39,41],indefinit:5,independ:[1,6,7,8,9,13,14,29,32,33,36,37,38],index:[1,2,4,5,11,15,24,25,27,29,31,32,40,41],index_col:[1,32],indic:[1,2,4,5,6,7,10,11,12,14,17,22,26,27,32,33,37,38,39,40,41],indispens:[7,35],individu:[2,7,8,11,13,29,32,33,35,36,38,39,40,41],indu:1,indx1:[3,41],indx2:[3,41],indx3:[3,41],indx:25,ineffici:[4,14,37,38],inequ:[9,14],inequaltii:[36,37],inertia:[14,22,37,38],inf1000:[24,32],inf1100:[24,32],inf1100l:[24,32],inf1110:[24,32],inf3000:32,infeas:10,infer:[1,2,5,7,31,32,35,40,41],inferenc:[2,40,41],infil:[1,7,8,10,32,35,36],infin:[6,7,8,12,19,33,34,35,36,39,40],infinit:4,infinitesim:29,influenc:[7,11,35,36],influenti:[2,40,41],info:32,inform:[1,2,4,5,7,10,12,13,14,15,22,25,26,27,31,32,35,36,37,38,39,40,41],infti:[4,7,14,29,35,36,37,39],ingeni:[14,36,37,38],ingrad:[],ingredi:[1,10,32],inher:[7,35],inherit:[25,32],inhomogen:3,init:[],initi:[1,2,3,7,11,14,15,25,27,29,32,35,36,37,38,39,40,41],inititi:[40,41],inject:15,inlin:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,22,25,26,29,32,33,34,35,36,37,38,39,40,41],inner:[1,14,33,37],inp:5,inplac:14,inpput:39,input:[0,1,2,4,5,6,7,8,9,13,14,15,22,26,27,29,32,33,34,35,36,37,38],input_dim:[2,40,41],input_nod:[40,41],input_s:23,input_shap:[4,5],inputs:[2,40,41],inputs_shuffl:[1,2,33,40,41],insert:[4,6,7,9,11,29,33,34,35],insid:[0,1,5,8,23,36],insight:[1,2,6,22,24,27,32,33,34,35],insist:[7,14,33,34,37],inspect:36,inspir:[1,2,13,27,32,38,39,40,41],instabl:[3,41],instal:[1,2,6,7,10,16,40,41],instanc:[1,2,3,5,7,10,12,14,32,33,35,36,37,40,41],instanti:11,instead:[1,2,3,4,5,6,7,9,10,12,14,15,22,23,25,26,29,32,33,35,36,37,38,40,41],institut:[2,40,41],instruct:[1,2,16,17,32,40,41],int32:11,int_0:29,int_:[4,7,29,35,39],int_a:29,intak:[1,33],integ:[2,3,14,15,25,29,32,37,38,40,41],integer_vector:[2,40,41],integr:[4,7,29,32,35],intellig:[1,15,31,32],intend:11,intens:[2,39,40],intention:15,interact:[1,7,10,13,24,26,27,32,38,39],intercept:[1,7,9,12,14,17,26,32,34,35,36,37,38],intercept_:[1,7,9,10,14,32,33,34,36,37],interchang:[6,13,25,34,35,38,39],interconnect:[2,39,40],interest:[0,1,2,3,4,5,6,7,8,9,10,13,22,24,26,27,29,32,33,34,35,36,37,38,39,40,41],interfac:[1,2,25,33,40,41],interior:[1,10,32],intermedi:[25,33],intermediari:23,intern:[2,11,13,38,39,40,41],interpol:[2,4,5,7,13,26,38,39,40,41],interpr:[6,33,34],interpret:[1,2,7,10,11,13,14,18,20,23,25,26,27,29,39,40,41],interv:[1,4,6,7,8,14,19,26,29,32,33,34,36,37],intial:[14,36,37],intract:[1,5,33],intrins:[4,12,25,29,32],intro:[24,31,32,41],introduc:[1,2,6,7,9,11,13,18,25,26,29,32,35,36,39,40,41],introduct:[2,3,5,14,18,21,31,33,36,37,40,41],introductori:[1,5,25,31,32,33],intuit:[1,6,7,9,13,14,22,26,32,34,35,37,38,39,40],inv:[1,6,14,22,32,33,34,36,37,38],invalid:[2,9,40,41],invalu:[1,14,16,24,32,36,37],invari:[2,40,41],invd:[6,34],inver:[9,38,39],invers:[1,4,7,14,16,17,18,22,26,27,32,33,36,37,38],inverse_transform:9,invert:[1,6,8,11,14,22,32,34,36,37,38],invh:[14,22,37,38],invok:[1,9],involv:[0,1,3,7,8,12,13,33,35,36,37,38,39,40,41],io:[1,18,20,24,26,28,30,31,32,33,41],ion:38,ip:[1,9,29,32],ipca:12,ipykernel_12550:[],ipykernel_12588:[],ipykernel_1323:[],ipykernel_1390:[],ipykernel_1539:[],ipykernel_1579:[],ipykernel_22411:[],ipykernel_22458:[],ipykernel_22556:[],ipykernel_22575:[],ipykernel_47048:[],ipykernel_55841:[],ipykernel_57122:[],ipykernel_57183:[],ipykernel_57294:[],ipykernel_57311:[],ipykernel_57345:[],ipykernel_58656:[],ipykernel_58739:[],ipykernel_58840:[],ipykernel_58862:[],ipykernel_59478:[],ipykernel_59625:[],ipykernel_59734:[],ipykernel_59752:[],ipykernel_59845:[],ipykernel_6316:[],ipykernel_6403:[],ipykernel_64157:[],ipykernel_64203:[],ipykernel_64318:[],ipykernel_64336:[],ipykernel_64372:[],ipykernel_6690:[],ipykernel_6729:[],ipykernel_87117:[],ipykernel_87155:[],ipykernel_87222:2,ipykernel_87367:7,ipykernel_87525:32,ipykernel_87545:35,ipykernel_87566:37,ipykernel_87597:40,ipykernel_87605:41,ipykernel_95354:[],ipykernel_95419:[],ipykernel_95519:[],ipykernel_95542:[],ipynb:[24,26,32],ipython:[1,6,8,10,12,15,16,24,26,27,32,33,36],iq:[7,35],iri:[9,10,23],irreduc:[7,35],irrelev:[6,33],irrespect:[1,32],irvin:27,isbox:[],isclos:23,iscomplexobj:[],iseffici:[37,38],isfinit:23,isn:[6,34,35],isnan:[40,41],isnul:1,isomap:12,isotop:[0,32,36,37,40,41],issu:[2,10,25,34,39,40,41],it_arrai:[14,37],item:[1,14,32,37],items:[25,32],iter:[0,2,3,5,7,9,10,12,14,15,22,29,35,36,38,39,40,41],its:[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,19,23,24,25,26,27,29,32,34,35,36,37,38,39,40,41],itself:[6,7,13,20,26,27,29,33,34,35,39],iv:40,ix:[40,41],j1:25,j:[1,2,3,4,5,6,7,9,10,12,13,14,15,17,18,19,20,25,26,29,31,32,33,34,35,36,37,38,39,40,41],j_:7,j_lasso_sk:7,j_ridge_sk:7,j_sk:7,jackknif:[7,24,32,35],jacobian:[3,14,36,37],jacobian_shap:[],jakobsen:30,jason:5,jax:[24,27,32,39],jax_descend_i:[22,38],jax_descend_x:[22,38],jax_enable_x64:[22,38],jax_grad:[22,38],jensen:[0,30,32,33,34,35,36,37,38,39,40,41],jentzen:[39,40,41],jerom:[19,26,31],ji:[13,25,39,40],jit:[14,37],jj:[1,6,7,32,34,35],jk:[1,2,7,13,25,32,38,39,40,41],jl:[1,32],jm:25,jmlr:41,jnp:[14,22,37,38],job:[3,9,11,41],johansen:32,join:[1,4,5,7,8,10,15,32,33,35,36],joint:[5,6,34,35],judg:[14,36,37],judgement:[7,26],julia:[24,25,26],juliu:[39,40,41],jump:[26,29],junk:5,jupit:32,jupyt:[1,10,16,20,24,26,31,32,39,40,41],just:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,22,23,24,26,27,29,32,33,34,35,36,37,38,39,40,41],justif:1,justifi:[4,11],k0:[0,8,36],k1:[0,8,36],k:[0,1,2,4,6,7,8,9,10,11,12,13,14,15,23,24,25,26,29,30,32,33,34,37,38,41],kaggl:[7,26,27],kajda:[40,41],kappa_d:29,karl:[30,32],karush:9,katrin:[30,32],keep:[1,2,5,6,7,12,14,15,23,25,26,27,32,33,34,35,36,37,38,39,40],keepdim:[2,7,11,25,35,40,41],kei:[1,2,4,7,13,33,38,40,41],kept:[5,7,15,35],kera:[1,5,24,26,27,32],kernel:[1,2,4,5,15,24,32,33,40,41],kernel_librari:[4,5,15],kernel_regular:[2,4,40,41],kernel_s:5,kernelpca:12,kev:[1,32],kevin:[31,32],keyboardinterrupt:[40,41],keyerror:[],keyword:[7,14,25,32,40,41],kfold:[7,35,36],kg:[2,40,41],ki:25,kick:[2,14,37,38,39,40],kiener:[3,41],kilomet:[7,33,34],kind:[1,3,4,5,9,13,14,15,32,33,37,38,39,40,41],kj:[7,13,25,33,34,39,40,41],kjm:[24,32],kkt:9,kl:29,km:[13,32,38],kmean:15,kmeanspoint:15,kn_k:15,know:[1,2,3,6,7,9,14,24,32,33,34,36,37,38,39,40,41],knowledg:[1,24,32],known:[0,2,4,5,6,7,8,9,10,13,25,26,27,29,31,33,34,35,36,38,39,40,41],kondev:[1,32],kp:29,kpca:12,kristin:41,kroneck:15,kuckuck:[39,40,41],kuhn:9,kumar:41,kutyniok:[39,40,41],kvalsund:[30,32],kwarg:[3,23,40,41],kwown:[1,32],l0:[0,8,36],l1:[0,1,2,4,8,32,36,40,41],l1_l2:[2,4,40,41],l1regl:6,l2:[2,4,40,41],l:[0,1,2,3,4,6,7,8,9,11,12,13,14,20,25,26,29,32,33,36,37,38,41],l_1:[0,8,36,39],l_2:[0,8,14,27,36,37,39],l_:25,l_j:[13,39,40],l_ja:[40,41],la:[14,37],la_i:[13,39,40,41],la_k:[13,39],lab:[21,24,26,32],label:[1,2,3,4,5,6,7,8,9,10,11,13,14,15,22,24,25,26,27,29,32,33,34,35,36,37,38,39,40,41],label_prob:[37,38],labelencod:[0,8,11,36],labels:[7,9,10],labels_shuffl:[1,2,33,40,41],laboratori:[28,34],lack:[1,32],lagari:[3,41],lagrang:[9,12],lam:[40,41],lambda:[0,1,2,3,4,6,7,8,9,11,13,14,18,19,22,26,27,29,32,33,34,35,37,38,39,40,41],lambda_0:12,lambda_1:[6,9,12,33],lambda_2:[9,12],lambda_:12,lambda_i:[9,12],lambda_iy_i:9,lambda_jy_iy_j:9,lambda_k:9,lambda_n:[6,9,33],lamda:[2,40,41],land:[1,9],landmark:9,landscap:[14,22,36,37,38],langl:[1,7,12,29,32,33],languag:[1,2,5,9,16,24,25,26,27,31,32,39,40],lapack:[25,32],laplac:[6,34,35],laptop:24,larg:[1,2,3,5,6,7,9,10,11,12,14,16,24,25,26,29,31,32,33,35,36,37,38,39,40,41],larger:[1,4,6,7,9,11,12,14,18,29,32,33,34,35,36,37,38],largest:[5,9,12],lass:27,lasso:[0,1,8,24,27,32],lasso_sk:7,last:[1,2,3,4,5,6,7,8,9,11,13,14,15,18,19,23,25,26,29,30,32,34,35,37,41],latent:5,latent_dim:5,latent_point:5,latent_space_value_rang:5,later:[0,1,2,5,8,9,13,14,15,16,23,24,26,27,32,36,37,38,39,41],latest:[5,24],latest_checkpoint:5,latex:32,latter:[1,4,7,8,9,12,14,17,25,26,29,32,33,34,35,36,37,38,39],lattic:[13,38,39],law:1,lax_numpi:[22,38],layer:[1,5,14,22,27,32,37,38],layer_output_s:23,layers_grad:23,lbfg:[0,8,10,11,12,36],lcc:[6,7,34,35],lda:12,ldot:[1,7,12,20,26,32,35,36],le:[6,8,11,14,18,22,29,33,34,36,37,38],lead:[0,1,2,4,6,7,8,9,10,11,12,13,14,17,18,23,25,29,32,33,34,35,36,37,38,39,40,41],leaf:10,leaki:[2,27,39,40,41],leakyrelu:5,lear:[14,36,37],learn:[0,4,5,6,7,8,9,10,11,13,19,23,25,28,30,31],learnabl:4,learner:11,learnig:32,learning_r:[0,9,11,22,23,38],learning_rate_init:[1,2,32,40,41],learning_schedul:[14,22,37,38],learnt:[26,27],least:[1,8,9,11,12,16,17,18,20,22,24,25,27,29,35,36],leat:[14,22,37,38],leav:[1,2,4,6,7,10,12,23,32,34,35,36,39,40],lectur:[1,2,6,11,12,13,14,16,18,22,24,25,26,27,28,30,31],lecture_11_backpropag:41,lecturenot:[1,18,20,24,26,31,32],left:[0,1,2,3,4,6,7,8,9,10,11,12,13,14,15,17,18,20,25,26,29,32,33,34,35,36,37,38,39,40,41],leftarrow:[9,13,39,40,41],legend:[0,1,3,4,5,6,7,8,9,10,11,14,23,32,33,34,35,36,37,38,41],legend_el:23,len:[1,2,3,4,5,6,7,9,10,11,12,13,23,25,32,33,34,35,37,38,39,40,41],length:[1,2,4,5,9,10,14,17,23,24,32,33,36,37,38,40,41],length_of_sequ:5,leq:[1,6,8,9,14,15,29,32,33,34,36,37],less:[1,2,4,5,6,7,9,10,14,24,29,32,33,34,35,36,37,38,40,41],lessen:[2,39,40],let:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,25,29,32,33,34,35,36,37,38,39,40,41],letter:[1,17,25,29,32,33],level:[1,2,6,7,10,24,25,26,27,28,30,32,35,39,40,41],li:[9,12],lib:[0,1,2,3,4,5,7,8,9,10,12,15,22,23,32,34,36,38,40,41],liblinear:[9,11],libmetal_plugin:[4,5,15],librari:[1,2,3,4,5,6,7,8,10,11,12,15,16,22,25,26,27,29,31,33,34,36,40,41],library_loc:[4,5,15],licens:[1,2,16,24,26,27,32,40,41],lie:[1,7,12,29,32,33,35],life:[1,2,9,13,32,38,39,40,41],lifetim:[14,37,38],like:[1,2,3,4,5,6,7,8,10,11,12,13,14,16,17,22,23,24,25,26,27,29,32,33,34,35,36,37,38,39,40,41],likelihood:[1,2,6,10,32,33,37,38,40,41],lim_:29,lima:30,limit:[0,1,6,7,9,10,12,13,22,25,26,27,32,33],lin_clf:9,lin_model:1,lin_reg:10,linalg:[1,3,6,7,9,12,14,22,25,29,32,33,34,36,37,38,39,41],line1:9,line2:9,line2d:[22,37,38],line3:9,line:[1,2,3,4,5,7,8,9,11,12,14,15,22,23,26,32,33,34,35,36,37,38,39,40,41],linear:[0,2,4,6,7,8,10,11,12,13,18,19,22,23,24,26,29,35,37,38,39,40,41],linear_model:[0,1,6,7,8,9,10,11,12,14,32,33,34,35,36,37,38,39],linear_regress:[7,35,40,41],linearli:[6,33],linearloc:[7,14,26,36,37],linearregress:[1,7,8,10,32,33,34,35,36],linearsvc:9,lineat:34,liner:[2,4,40,41],linerar:11,linewidth:[1,3,5,7,9,10,11,26,35,41],link:[1,5,10,13,23,24,26,27,30,32,39],linlag:[6,34],linpack:[25,32],linreg:[1,32],linspac:[1,3,4,5,7,9,10,11,14,16,17,22,25,29,32,33,34,35,37,38,41],linu:5,linux:[1,2,16,24,26,32,40,41],liquid:[1,32],list:[1,2,3,4,5,8,10,23,24,26,27,32,33,36,37,38,41],listcomp:[],listedcolormap:[10,11],literatur:[2,8,15,31,35,36,39,40],littl:[2,4,10,13,39,40],live:9,ll:[1,29,32,33],lle:[1,33],lloyd:[5,15],lmb:[1,3,6,7,34,35,36,41],lmbd:[1,2,4,32,40,41],lmbd_val:[1,2,4,32,40,41],lmbda:[14,36,37],ln:[2,14,36,37,40,41],load:[0,1,2,4,5,7,8,10,11,15,26,36,40,41],load_boston:1,load_breast_canc:[0,2,8,10,11,12,36,40,41],load_data:[4,5],load_digit:[2,4,40,41],load_iri:[9,10,23],load_librari:[4,5,15],load_pluggable_device_librari:[4,5,15],loc:[0,1,4,7,8,9,10,11,23,32,35,36],local:[0,1,2,3,4,8,13,14,23,33,36,37,38,39,40,41],locat:[3,4,9,41],log10:[1,6,7,34,35,36,40,41],log:[0,1,2,3,5,6,7,8,10,11,12,14,23,25,26,27,32,33,34,35,36,37,38,40,41],log_:[1,32],log_clf:11,logaddexp2:[],logaddexp:[],logarithm:[0,1,6,8,25,32,34,35,36],logbook:[26,27],logic:[1,2,10,32,39,40],logist:[1,2,3,9,10,11,12,13,14,22,24,33,41],logistic_predict:[37,38],logistic_regress:[40,41],logisticregress:[0,8,10,11,12,36,38,39],logit:[0,8,36],logreg:[0,8,10,11,12,36,38,39],logspac:[1,2,4,6,7,32,34,35,36,40,41],longer:[3,4,9,11,15,25,27,29,32,41],loocv:[7,35,36],look:[1,2,3,4,5,6,7,8,9,10,11,12,14,17,19,22,25,26,27,29,32,33,34,35,36,37,38,39,40,41],loop:[2,5,7,11,13,15,24,25,32,35,40,41],lose:[2,40,41],loss:[0,1,2,4,5,6,7,8,9,11,12,14,23,25,26,27,32,34,35,38,39,40,41],loss_fil:5,lossfil:5,lost:5,lot:[1,2,5,7,35,37,38,39,40,41],low:[1,7,10,11,12,26,32,33,35],lower:[1,2,4,7,10,11,17,23,25,33,39,40,41],lowercas:[25,32],lowest:[10,14,29,37,38],lr:[2,4,5,11,40,41],lrelu:[40,41],lstat:1,lstm:5,lstm_2layer:5,lstsq:[1,32,33],lt:[7,35],lu:[1,6,32,33],lubksb:25,luckili:[3,41],ludcmp:25,lux:25,lvert:[2,40,41],lw:[1,32],m:[1,2,3,4,6,7,9,10,11,12,13,14,22,25,28,29,30,31,32,33,34,35,36,37,38,39,40,41],m_1:15,m_:[10,13,39,40],m_h:[1,32],m_k:15,m_l:[13,39,40],m_n:[1,32],m_p:[1,32],m_t:[14,37,38],ma:12,machin:[2,4,5,6,7,8,10,11,12,13,16,22,25,28,31,33,34,35,38,39,40,41],machinelearn:[1,7,17,18,20,21,24,26,28,30,31,32,36,37,38],mackai:31,made:[1,2,4,5,6,7,8,10,12,13,19,26,27,32,33,36,38,39,40],mae:[1,32],magic:5,magnitud:[0,2,7,8,14,23,33,34,36,37,38,39,40,41],mai:[0,1,2,3,4,6,7,8,9,10,12,13,14,19,22,24,25,26,27,29,32,33,34,35,36,37,38,39,40,41],mail:[28,30],main:[1,2,4,5,6,7,8,10,25,26,27,31,33,36,39,40],mainli:[0,1,6,7,8,10,32,33,34,35,36],maintain:[7,35],major:[2,7,10,11,14,25,32,35,36,37,38,40,41],make:[0,2,3,4,5,6,7,8,9,12,13,14,22,23,24,25,26,27,29,31,34,35,36,37,38,39,40,41],make_axes_locat:7,make_classif:[38,39],make_moon:[9,10,11],make_pipelin:[1,7,11,33,35],make_vjp:[],makedir:[1,7,8,10,32,33,35,36],makeplot:[1,32],malcondit:25,malign:[0,2,8,10,36,40,41],mammographi:[6,34,35],manag:[1,3,4,16,24,26,32,41],mandatori:[30,32],mani:[0,1,2,4,5,6,7,8,9,10,12,14,15,22,23,24,25,26,27,29,31,32,33,34,35,36,37,38,39,40,41],manifold:12,manner:4,manual:[7,23,33,34,36],map:[1,2,3,7,8,9,12,13,15,26,29,32,36,38,40,41],marc:33,margin:[1,6,9],marit:[1,32],mariu:41,mark:32,marker:[1,8,22,25,32,36,38],markov:[24,32],marsaglia:29,mass:[1,2,6,14,33,37,38,40,41],massag:[1,32],masses2016:[1,32],masses2016ol:[1,32],masses2016tre:1,masseval2016:[1,32],master:[7,17,21,26,28,30,36,37,38,41],mat1100:[24,32],mat1110:[24,32],mat1120:[24,32],mat:[24,32],match:[2,5,6,14,15,33,36,37,38,39,40],match_complex:[],materi:[5,6,8,14,18,25,30,37,38],math:[0,4,8,13,14,22,25,29,31,32,36,37,38,39,40,41],mathbb:[1,5,6,7,8,9,12,13,14,15,18,19,20,25,26,29,32,33,34,35,36,37,38,39],mathbf:[0,1,6,7,8,9,14,19,20,22,25,26,32,33,34,35,36,37,38,39],mathcal:[0,2,6,7,8,14,20,26,34,35,36,37,40,41],matheemat:4,mathemat:[1,7,12,13,14,18,23,24,25,29,31,32,34,36],mathemati:32,mathmlbook:17,mathrm:[0,1,2,4,5,6,7,8,9,10,11,12,13,14,15,18,19,20,26,29,32,33,34,35,36,37,38,39,40,41],matmul:[2,3,6,34,39,40,41],matmul_adjoint_1:[],matmul_vjp_1:[],matnat:31,matplotlib:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,22,23,24,25,26,29,32,33,34,35,36,37,38,39,40,41],matric:[1,2,4,5,7,8,9,12,14,17,18,24,33,36,37,39,40,41],matrix:[1,3,4,5,7,8,9,11,14,16,17,18,19,20,22,23,26,27,29,35,39],matshow:[2,40,41],matter:[3,4,14,33,36,37,38,39,41],max:[1,2,3,4,5,10,11,13,14,22,23,30,32,36,37,38,39,40,41],max_depth:[1,10,11],max_diff1:[3,41],max_diff2:[3,41],max_diff:[3,41],max_it:[0,1,2,9,10,12,14,32,36,37,38,39,40,41],max_iter:15,max_leaf_nod:11,max_sampl:11,maxdegre:[1,7,11,33,35],maxdepth:11,maxim:[0,2,5,6,8,9,12,34,35,36,40,41],maximum:[0,1,2,3,4,6,8,9,10,11,14,15,32,33,37,38,40,41],maxpolydegre:[6,7,34,35,36],maxpooling2d:4,mbox:[6,7,19,26,33,34,35],mcculloch:[13,38,39],md:[12,21,26,37,38],mdoel:5,mean:[2,3,4,5,6,7,8,10,11,12,13,14,15,16,17,18,19,20,22,24,25,26,27,29,32,35,37,38,39,40,41],mean_absolute_error:[1,32],mean_divisor:15,mean_i:29,mean_matrix:15,mean_squared_error:[1,5,7,8,11,32,33,35,36],mean_squared_log_error:[1,32],mean_vector:15,mean_x:29,meaning:[1,5,8,32,36],meansquarederror:[1,32],meant:[0,4,8,11,14,36,37,38,39],measur:[1,2,3,6,7,10,12,13,15,20,26,27,29,32,33,34,35,39,40,41],mechan:[1,5,29,32,41],median:[1,32,33],medicin:[13,38,39],medium:[5,9,14,27,37,38],medv:1,meet:[1,30],mehta:[1,32,33,34],memori:[4,5,12,13,14,22,25,37,38,39],mention:[1,13,14,26,27,29,32,36,37,38,39],mere:[1,16,27,32],meshgrid:[3,6,7,9,10,11,12,26,33,40,41],mess:[],messag:[6,14,37,38],messi:[3,41],met:[1,4,9,33],metadata:[],meteorolog:10,meter:[7,33,34],method:[0,1,2,3,4,5,6,8,9,12,13,15,17,18,19,20,23,24,25,27,29,31,33,34,39],metion:[7,26],metric:[0,1,2,4,7,8,10,11,15,16,17,23,32,33,35,36,40,41],metropoli:[24,32],mev:[1,29,32],mgd:[14,37,38],mglearn:[24,32],mgrid:[14,37],mhjensen:[0,2,4,5,7,9,10,12,15,22,38,40,41],mi:11,mia:[30,32],michael:[27,39,40],michigan:[0,32,34,36,37,38,40,41],microsoft:31,mid:[2,40,41],midel:5,midnight:[17,18,19,20,21,22,23],midpoint:10,might:[1,2,3,5,7,10,14,33,34,36,37,38,39,40,41],mild:10,millimet:[7,33,34],million:[1,32,33,37,38],mimic:[13,38,39],min:[1,3,6,9,10,41],min_:[1,3,6,15,18,32,33,34,41],min_samples_leaf:10,mind:[1,7,14,23,26,32,33,34,35,36,37],mindboard:5,mine:[24,32],mini:[2,12,13,14,22,27,36,39,40],minibatch:[2,12,14,40,41],minibathc:[14,37,38],miniforge3:[0,1,2,3,4,5,7,8,9,10,12,15,22,23,32,34,36,38,40,41],minim:[1,2,3,4,6,7,8,9,10,11,12,13,14,15,18,19,22,26,33,34,35,37,38,39,40],minima:[1,2,8,14,32,36,37,38,40,41],minimum:[1,2,3,7,9,10,12,14,33,35,36,37,38,40,41],minmaxscal:[1,33,40,41],minor:29,minst:[2,40,41],minu:[8,36],mirjalili:32,mirror:10,misc:[7,26],misclassif:[9,10,11],misclassifi:[9,11],miser:1,mismatch:[2,39,40],miss:[1,8,11],mistak:5,mit:31,mix:[2,3,32,39,40,41],mixtur:[14,22,37,38],mk:[10,25],mkdir:[1,7,8,10,32,33,35,36],ml:[1,2,11,14,25,26,27,33,36,37,38,39,40,41],mlab:29,mlbook:32,mle:[6,8,36],mlp:[2,38,39,40,41],mlpclassifi:[2,38,39,40,41],mlpregressor:[1,32],mm:25,mml:33,mn:[13,29,38],mnist:[2,12,27,39,40,41],mod:29,mode:[28,30,32,40,41],model:[0,3,4,6,8,9,10,11,12,14,15,16,17,19,20,22,23,24,26,29,31,33,34,35,36,37],model_select:[0,1,2,4,6,7,8,10,11,12,32,33,34,35,36,40,41],moder:11,modern:[1,7,8,24,32,35,36,39,40,41],modif:[3,13,14,37,38,41],modifi:[0,1,2,4,6,8,9,11,13,14,32,33,34,36,37,38,39,40,41],modul:[0,1,10,12,25,32,34],modular:29,modulo:29,moe:[12,33],moment:[6,7,14,22,29,34,35,40,41],moment_correct:[40,41],momentum:[27,39,40,41],momentum_schedul:[40,41],mondai:32,monitor:[14,22,37,38],monoton:[6,13,29,34,35,38,39,40,41],mont:[1,7,24,29,31,32,35],moor:[6,7,34],more:[1,2,3,5,6,8,9,10,11,12,13,14,15,19,23,24,27,29],moreov:[1,4],morten:[0,30,32,33,34,35,36,37,38,39,40,41],most:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,22,23,24,26,29,32,33,34,35,36,37,38,39,40,41],mostli:[2,12,39,40],motion:[1,14,37,38],motiv:[2,5,39,40],move:[1,5,6,7,8,10,13,14,15,22,23,26,29,33,34,35,36,37,38,39,40,41],mpl:[1,8,32,36],mpl_toolkit:[3,7,14,26,36,37,41],mplot3d:[3,7,14,26,36,37,41],mplregressor:[2,40,41],mse:[1,5,6,7,10,11,16,17,18,20,26,27,32,33,34,35,36,40,41],mse_simpletre:11,mselassopredict:[6,34],mselassotrain:[6,34],mseownridgepredict:[7,34],msepredict:[6,34],mseridgepredict:[1,6,7,34,36],msetrain:[6,34],msle:[1,32],mt:[0,8,13,36,38,39,40],mu0:29,mu1:29,mu2:29,mu:[1,7,12,14,29,32,35,37,38],mu_1:33,mu_:[7,29,33,34,35],mu_i:[7,33,34],mu_n:12,mu_x:29,much:[1,2,3,4,5,6,7,9,10,11,12,13,14,23,25,26,29,32,33,34,35,37,38,39,40,41],multi:[1,2,4,8,24,32,36],multiarrai:23,multiclass:[0,2,8,36],multidimension:[12,13,32,38,39],multilay:[2,40,41],multinomi:[0,8,36],multipl:[0,3,5,6,7,8,13,14,29,33,35,36,37,38,39],multipli:[4,6,7,12,14,25,29,33,34,37],multiplum:9,multivari:[1,3,11,12,24,29,32,41],multivariate_norm:[12,15],murphi:[12,31,32],must:[2,3,6,7,9,11,13,14,15,29,33,35,36,37,38,39,40,41],mutat:[8,36],mutual:[2,4,7,14,35,36,37,39,40,41],mx_:29,myenv:[0,1,2,3,4,5,7,8,9,10,12,15,22,23,32,34,36,38,40,41],myriad:[1,16,24,32],mz1:29,mz2:29,n1:25,n2:25,n:[0,1,2,3,4,5,6,7,8,9,11,12,13,14,15,16,17,18,19,20,22,25,26,27,29,32,33,34,35,36,37,38,39,40,41],n_0:[13,29,38],n_:[2,3,4,9,13,29,38,40,41],n_boostrap:[7,11,35],n_bootstrap:[7,35],n_categori:[2,4,40,41],n_cluster:15,n_compon:12,n_data:[0,38],n_epoch:[14,22,37,38,40,41],n_estim:11,n_examples_to_gener:5,n_featur:[2,39,40,41],n_filter:4,n_hidden:[3,41],n_hidden_neuron:[1,2,32,39,40,41],n_i:29,n_input:[1,2,4,33,39,40,41],n_instanc:10,n_iter:[36,37],n_iter_i:[0,10,12],n_job:11,n_k:15,n_l:[13,29,38],n_layer:[2,40,41],n_m:10,n_neuron:[2,40,41],n_neurons_connect:4,n_neurons_layer1:[2,40,41],n_neurons_layer2:[2,40,41],n_output:[39,40],n_point:15,n_sampl:[7,9,10,11,15,35,38,39],n_split:[7,35,36],n_step:5,n_t:[3,41],n_x:[3,41],nabla:[2,14,36,37,40,41],nabla_:[3,14,22,36,37,38,41],nabla_w:[14,37,38],nafter:[40,41],nag:[14,37,38],naimi:[1,32],naiv:[0,8,36],naive_kmean:15,najafi:[30,32],name:[1,2,4,5,6,7,8,9,10,11,13,14,15,16,23,24,25,26,29,30,32,33,34,35,36,37,38,39,40,41],nameerror:[11,26],namespac:[4,5,15],nan:[40,41],narrow:[14,37,38],nary_f:[],nary_op_arg:[],nary_op_kwarg:[],nary_oper:[],nation:[2,6,34,35,40,41],nativ:[24,32],natur:[1,2,5,9,10,13,14,26,27,29,31,32,36,37,38,39,40,41],navier:[13,38,39],nb:29,nb_:25,nbconvert:32,nbviewer:10,nd:15,ndarrai:[7,40,41],nderiv:[40,41],ndim:37,ne:[10,11,25,29,33],nearest:[2,4,7,12,40,41],nearli:[14,36,37],neat:32,neatli:[],neccesari:[7,35],necess:[3,41],necessari:[1,2,4,5,9,15,32,39,40,41],necessarili:[1,5,12,29,32],necesserali:[6,34,35],neck:[8,36],need:[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,18,22,23,25,27,29,33,34,35,36,37,38,39,40,41],neg:[0,1,2,4,6,7,8,11,14,25,29,32,34,35,36,37,38,39,40,41],neg_mean_squared_error:[7,35,36],neglect:29,neglig:29,neighbor:[4,7,12],neither:[5,14,37,38],neq:[14,15,29,36,37],nerual:[40,41],nervou:[13,38,39],nest:[10,13,38],nesterov:[14,37,38],net:[3,5,13,38,39,41],netlib:[25,32],network:[1,10,14,22,23,24,31,33,37],network_input_s:23,neural:[0,1,14,22,23,24,31,33,36,37],neural_network:[1,2,3,32,38,39,40,41],neuralnetwork:[2,40,41],neuralnetworksanddeeplearn:[27,39,40],neuron:[2,3,4,5,13,40,41],neutal:[],neutral:[1,32],neutron:[1,32],never:[2,5,7,10,29,35,39,40,41],new_box:[3,23],new_chang:[14,22,37,38],new_hobbit:32,new_root:[],new_trac:[],newaxi:[1,4,7,10,23,35,36],newli:[1,32],newton:[2,8,9,14,29,39,40,41],next:[1,2,3,4,5,6,7,8,9,10,14,15,22,23,32,33,34,36,37,38,39,40,41],next_guess:[14,37],next_input:5,ng:[2,39,40,41],ngini:10,nhow:[40,41],ni:[15,22],nice:[1,2,6,12,32,33,34,35,40,41],nielsen:[27,39,40],nine:[39,40],niter:[14,22,36,37,38],nitric:1,nlambda:[1,6,7,34,35,36],nlp:31,nm:29,nm_n:[1,32],nmse:[7,35],nn:[3,6,7,13,25,34,35,38],nn_model:[2,40,41],nnmin:[3,41],node:[2,3,4,10,11,13,23,27,38,41],node_constructor:[],nois:[1,5,6,7,9,10,11,14,16,17,20,26,32,33,34,35,36,37],noise_dimens:5,noisi:[2,7,20,26,35,40,41],non:[1,2,4,6,7,8,10,11,12,13,14,15,22,23,25,29,32,33,34,35,36,37,38,39,40,41],noncommerci:27,none:[0,1,2,3,5,6,10,11,14,29,32,33,34,35,36,39,40,41],nonlinear:[4,7,9,10,12,13,35,38,39],nonneg:[7,10,14,35,36,37],nonparametr:7,nonsens:29,nonsingular:25,nonumb:[0,4,8,9,14,22,25,36,37,38],nor:[2,5,14,37,38,39,40],norm:[1,2,6,7,9,12,14,18,32,33,34,35,36,37,38,39,40,41],normal:[0,4,5,6,7,8,9,10,11,12,13,14,16,17,19,20,22,23,24,25,26,27,29,32,33,34,36,37,38,41],normali:[25,32],norwai:[7,26,27,32,35,38],notat:[1,3,6,7,14,15,22,29,32,33,34,35,37,39,40,41],note:[0,1,2,3,4,5,6,7,8,9,12,13,14,15,16,17,24,25,27,29,31,32,35,36,37,39,40,41],notebook:[1,2,4,10,16,23,24,26,27,32,39,40,41],notesseptember16:36,notesseptember30:38,notfounderror:[4,5,15],noth:[2,3,6,9,13,15,29,33,34,38,39,40,41],notic:[5,6,13,14,25,29,32,34,37,38,39,40],notimplementederror:[40,41],notion:4,notrace_primit:[],noutput:[40,41],novel:[4,7,11,36],novemb:[2,28,30,32,40,41],now:[0,1,3,5,6,7,8,9,11,12,13,15,16,17,18,23,24,25,26,27,29,32,33,36,39],nowadai:[1,2,4,10,24,32,39,40,41],nox:1,np:[0,1,2,3,4,5,6,7,8,9,10,11,12,14,15,16,17,22,23,25,26,29,32,33,34,35,36,37,38,39,40,41],npr:[3,41],nsampl:[7,10,35,36],nt:[3,41],ntrained_model:[7,34],nu:29,nuclear:[6,33],nuclei:[1,29,32],nucleon:[1,32],nucleu:[1,32],num:5,num_coordin:[3,41],num_equ:[40,41],num_featur:[0,38],num_hidden_neuron:[3,41],num_it:[3,41],num_iter:[0,22,38],num_neuron:[3,41],num_neurons_hidden:[3,41],num_not:[40,41],num_point:[3,41],num_tre:11,num_valu:[3,41],number:[0,2,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,20,23,25,26,27,28,30,32,34,35,36,40],numberid:[8,36],numberparamet:4,numer:[1,6,7,10,11,12,13,14,18,22,23,24,25,31,32,33,34,35,36,37,38,39],numpi:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,23,24,26,29,33,34,35,36,39,40,41],numpy_box:[],numpy_vjp:[],numpy_wrapp:[],nunmpi:[6,33],nvalu:10,nx:[3,41],nx_test:[7,34],nx_train:[7,34],nx_train_mean:[7,34],ny:[29,40,41],ny_pr:[7,34],ny_train:[7,34],ny_train_mean:[7,34],o:[1,7,8,9,10,12,16,22,25,30,31,32,36,38],obei:[7,12,14,33,34,37],obj:[],object:[1,2,3,5,7,9,11,25,32,37,39],obliqu:[6,33],observ:[1,2,4,6,7,8,9,10,11,12,13,14,15,29,34,35,36,37,38],obtain:[0,1,2,6,7,8,9,10,11,13,14,15,18,22,25,26,27,29,32,33,34,35,36,37,38,39,40,41],obviou:[6,7,12,29,33],obviouli:32,obvious:[1,5,6,7,22,25,27,32,34,35],oc:33,occupi:1,occur:[1,3,7,9,10,25,29,32],oct:27,octob:[0,22,23,28,30,32,38],od:1,odd:[0,1,4,8,32,33,36],odenum:[3,41],odesi:[3,41],odin:32,oen:1,off:[2,4,5,6,10,14,20,29,34,35,37,38,40,41],offer:[7,12,24,25,28,30,32,35],offic:[30,32],offici:[28,32],offlin:23,often:[0,1,2,4,5,6,7,8,9,10,11,12,13,14,15,17,23,24,25,26,29,32,33,35,36,37,38,39,40,41],ofter:[25,32],og:[40,41],ol:[1,14,16,17,18,19,27,32,33],old:[2,6,11,14,34,35,36,37,40,41],ols_sk:7,ols_svd:7,olsbeta:[1,6,34],omega:[3,4,7,41],omega_0:4,omit:[1,6,32,33,34,35],omlmp_kkie0:36,onc:[2,7,10,12,14,35,36,37,38,39,40,41],one:[0,1,2,4,5,6,7,8,9,10,11,12,14,15,20,22,23,24,25,26,27,29,30,32,33,34,35,38],one_hot_predict:23,onehot:[2,40,41],onehot_vector:[2,40,41],onehotencod:10,ones:[1,3,6,7,9,10,11,12,14,22,23,25,26,32,33,34,35,36,37,41],ones_lik:5,ong:33,onl:4,onli:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,18,20,22,23,25,26,27,29,32,33,34,35,36,37,38,39,40,41],onlin:[12,28,39,40,41],onto:[6,12,33],open:[1,2,5,7,8,10,16,24,26,28,30,32,35,36,40,41],oper:[1,2,4,6,7,11,12,13,14,16,23,24,29,32,33,34,35,37,38,40,41],operation:29,oplu:29,opmiz:[14,37,38],opportun:1,oppos:[7,14,37,38],opposit:[2,6,9,33,39,40],opt:[2,6,27,32,34,39,40,41],optim:[1,3,4,5,6,7,8,10,11,12,15,16,17,18,19,20,22,23,26,27,34,35],optimis:[2,4,40,41],option:[0,1,2,4,6,7,9,10,12,22,25,26,27,33,34,35,38,40,41],optmiz:[2,9,14,22,33,37,38,39,40,41],oral:32,orang:1,order:[1,2,3,4,6,7,8,9,10,11,12,13,16,17,23,25,26,27,29,32,33,34,35,36,39,40,41],ordinari:[1,3,4,8,12,14,16,17,18,20,22,24,27,35,36,37,38],oreilli:[31,32],org:[0,1,4,5,10,12,22,23,24,25,26,27,31,32,34,37,38,39,41],organ:[7,8,11,25,35,36],orgin:39,orient:[2,6,29,33,34],origin:[1,4,6,7,9,12,13,14,25,32,33,35,37,38],orthogn:[6,33],orthogon:[1,6,7,9,12,14,18,25,32,33,34,37],orthonorm:[6,33,34],os:[1,2,5,6,7,8,9,10,30,32,33,34,35,36,40,41],oscar:[2,39,40],oscil:[4,14,37,38],oserror:[1,4,5,8,15,36],oslo:[0,1,16,24,26,27,28,30,32,33,34,35,36,37,38,39,40,41],osx:[1,16,24,26,32],other:[1,2,3,4,6,7,8,9,11,14,15,16,20,23,24,27,28,29,30,31,34,35,37],otherwis:[1,2,5,8,14,22,25,27,32,36,37,38,40,41],ouput:[6,8,13,34,35,36,41],our:[0,2,3,4,7,8,9,10,11,13,15,16,17,18,19,20,22,23,24,25,26,27,29,34,35,38,41],ourmodel:1,ourselv:[1,6,7,9,12,14,32,33,34,35,36,37],out1:[],out2:[],out:[1,2,3,5,6,7,8,9,10,11,12,13,14,17,22,23,24,25,26,27,29,32,33,34,35,36,37,38,39,40,41],out_deriv:[40,41],out_fil:10,outcom:[1,8,10,11,13,29,33,36],outdoor:10,outer:[7,13,14],outfil:5,outgrad:[],outlier:[1,9,32,33],outlin:[7,11,12,35],outlook:10,outperform:11,output:[0,1,2,4,5,6,7,8,9,10,11,13,14,17,23,25,26,27,29,32,33,34,35,36,37,38],output_bia:[2,40,41],output_bias_gradi:[2,39,40,41],output_func:[40,41],output_nod:[40,41],output_s:[],output_shap:5,output_weight:[2,40,41],output_weights_gradi:[2,39,40,41],outputlayer1:[13,38],outputlayer2:[13,38],outsid:5,over1:[14,37,38],over:[1,2,4,5,6,7,10,11,13,14,22,25,26,32,33,34,35,36,37,38,41],overal:[2,11,40,41],overcast:10,overcom:[13,14,37,38,39],overdetermin:[1,32],overfit:[1,2,4,7,10,11,14,22,35,37,38,39,40,41],overflow:[2,6,34,35,40,41],overflowerror:[40,41],overhead:[13,39,40],overlap:[0,4,8,9,10,36],overlin:[1,6,7,10,11,12,15,25,32,33,34],overst:1,overtrain:5,overview:[4,36],overwritten:[40,41],own:[5,6,7,9,13,14,22,24,25,26,34,37,38,39,40,41],owner:1,ownmsepredict:1,ownmsetrain:1,ownridgebeta:[1,7,34],ownypredictridg:1,ownytilderidg:1,oxid:1,p0:[3,41],p1:[3,41],p:[0,1,2,3,4,5,6,7,8,9,10,11,12,14,15,18,25,29,32,33,34,35,36,37,39,40,41],p_:[3,5,9,10,41],p_hidden:[3,41],p_i:[6,29,34,35],p_j:29,p_n:29,p_output:[3,41],p_x:29,pa:39,pack:[1,32],packag:[0,1,2,3,4,5,6,7,8,9,10,12,14,15,16,22,23,24,26,27,29,33,34,36,37,38,40,41],packtpub:32,packtpublish:32,pad:[4,5],page:[1,10,24,32,34,35,36,37],pai:[1,2,10,14,37,38,39,40],pair:[1,3,4,10,24,29,32,41],panda:[0,1,5,6,7,8,10,12,16,24,26,34,35,36],panel:32,paper:[2,41],paradigm:[1,32],parallel:[11,14,22,24,25,32,37,38],param:[3,41],param_distribut:36,param_grid:36,paramat:[3,41],paramet:[1,2,3,4,5,6,7,8,9,10,11,13,14,17,18,19,20,22,23,26,27,29,34,35,41],parameter:[1,7,11,26,32,33],parametr:[1,7,16,17,32,33,35],paramt:[4,6,34,35,39],parent:39,parent_argnum:[],part:[1,2,3,4,6,7,11,18,19,22,23,25,28,29,30,32,34,35,38],partial:[0,1,2,6,7,8,9,11,12,13,14,17,23,29,32,33,34,36,37,39,40],particip:[24,28,30,32],particl:[1,5,14,29,32,37,38],particular:[1,2,3,4,6,7,10,11,12,13,14,17,26,29,31,32,33,34,35,36,37,38,39,40,41],particularli:[6,7,9,12,14,22,29,33,35,36,37,38],partit:[2,5,10,40,41],partli:[7,32,35],pass:[3,4,13,15,23,36,39],passkwarg:[],password:[26,27],past:[11,29],patch:[7,29,35],path:[1,4,5,7,8,10,15,16,24,32,33,35,36],pathcollect:[22,38],patient:[8,36],patter:5,pattern:[1,4,5,13,28,31,32,38,39],pauli:[1,32],pc:[12,24],pca:[0,1,8,24,32,33,36],pd:[0,1,5,6,7,8,10,12,32,33,34,35,36],pde:[3,41],pdf:[1,4,5,6,7,10,17,21,26,27,31,32,34,35,36,38,41],pedagog:[1,32,33],penal:[7,33,34],penalti:[7,14,26,33,34,36,37],penros:[6,7,34],pentagon:[14,36,37],peopl:[1,2,10,14,24,37,38,39,40,41],per:[1,2,7,23,28,30,32,35,39,40,41],perc_print:[40,41],percentag:[1,11,12,30,40,41],perceptron:[0,1,2,8,32,36],peregrin:32,perfect:[1,2,14,22,32,37,38,40,41],perfectli:[5,7,35],perform:[1,3,4,5,6,7,9,11,12,13,14,15,16,17,18,20,22,23,24,25,26,27,29,32,33,34,35,36,37,38,39],performac:5,perhap:[1,6,14,32,33,34,36,37],perimet:[2,10,40,41],period:[2,5,29,40,41],permut:12,persist:[14,37,38],person:[6,7,8,28,30,32,33,34,35,36],perspect:31,pertin:[13,27,32,39,40,41],petal:[9,10],peter:[31,33],petersen:[39,40,41],phantom:29,phase:[7,13,38,39],phd:41,phenomena:29,phi:9,phi_k:9,philipp:[39,40,41],philosophi:[14,37],phone:[30,32],photo:5,php:27,phrase:[1,32],physic:[0,1,2,5,8,13,14,27,29,30,31,32,33,34,35,36,37,38,39,40,41],pi:[0,3,4,6,7,8,10,13,14,29,34,35,36,37,38,39,40,41],pick:[2,10,11,12,14,15,37,38,40,41],pickl:[2,40,41],pictur:[1,32],pie:[24,32],piec:[12,15,23],pillow:[1,16,24,26,32],pinv:[6,7,14,22,33,34,37,38,39],pip3:[1,2,16,26,32,40,41],pip:[1,2,16,24,26,32,40,41],pipelin:[1,7,9,11,33,35],pippin:32,pit:5,pitfal:[7,33,34],pitt:[13,38,39],pixel:[2,4,5,40,41],pixel_height:[2,4,40,41],pixel_width:[2,4,40,41],place:[1,5,7,9,14,25,26,32,35,36,37],plai:[1,4,5,6,7,9,12,24,32,33,34,35,36,39,40],plain:[9,11,13,14,15,22,27,36,37,39,40],plan:[7,10,30,31,32,40],plane:[9,10],plateau:[6,34],platform:[24,32],plausibl:[13,38,39,40],pleas:[0,10,12,14,26,27,30,32,37,38],plenti:[2,39,40],plethora:[4,13,38,39],pliahhy2ibx9hdharr6b7xevztgzra1p:[38,39,40],plot:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,22,23,24,25,26,27,29,32,33,34,37,38,39,40,41],plot_all_sc:[26,34],plot_confusion_matrix:[0,8,11,36],plot_count:7,plot_cumulative_gain:[0,8,11,36],plot_data:[2,40,41],plot_dataset:9,plot_decision_boundari:[10,11],plot_import:11,plot_iris_dataset:23,plot_max:5,plot_min:5,plot_model:5,plot_numb:5,plot_predict:9,plot_regression_predict:10,plot_result:5,plot_roc:[0,8,11,36],plot_surfac:[3,7,14,26,37,41],plot_train:10,plot_tre:[10,11],plqvvvaa0qudcjd5baw2dxe6of2tius3v3:[38,39,40],plt:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,22,23,25,26,29,32,33,34,35,36,37,38,39,40,41],plu:[1,4,6,8,32,33,34,36],pluggabl:[4,5,15],plugin:[4,5,15],pm:[9,35],pmatrix:[3,41],pml:31,pn:4,png:[1,5,7,8,10,32,33,35,36],point:[0,1,2,3,4,6,7,8,9,10,11,12,14,15,16,17,18,20,22,25,26,29,30,32,33,35,36,37,38,40,41],point_1:5,point_2:5,poisson:[24,29,32],poli:[7,9,35,36],poly100_kernel_svm_clf:9,poly3:1,poly3_plot:1,poly3dcollect:37,poly_degre:[40,41],poly_featur:[9,10],poly_features10:10,poly_fit10:10,poly_fit:10,poly_kernel_svm_clf:9,polydegre:[1,6,7,11,33,34,35],polygon:[14,36,37],polym:[13,38,39],polymi:26,polynomi:[1,6,7,8,9,10,11,12,16,17,18,20,22,26,27,32,33,35,36,38,39],polynomial_featur:[7,35],polynomial_svm_clf:9,polynomialfeatur:[1,7,9,10,33,35,36],polytrop:[1,7,32,35],pool:4,pool_siz:4,poor:[2,14,22,36,37,38,39,40],poorli:[1,33],pop:[],popul:[1,6,32,33,34,35],popular:[1,2,4,7,8,9,10,12,13,16,24,25,26,29,33,36,38,39,40],popularli:[1,32],portabl:11,portion:[12,14,22,37,38],pose:[1,5,6,7,12,29,32,35],posit:[1,2,3,4,6,8,9,11,12,14,15,23,25,29,32,33,34,35,36,37,38,39,40,41],possibl:[1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,20,22,23,24,25,26,27,29,30,32,33,34,35,36,37,38,39,40],posterior:[6,34,35],postpon:[1,32,33],postscript:[26,27],postul:[6,34,35],potenti:[1,4,6,7,13,14,32,33,34,35,37,38,39],pott:[13,38,39],power:[1,2,6,7,9,10,13,14,32,33,35,37,38,39,40,41],pp:[6,7,19,34,35,36,39,40,41],practic:[0,1,6,7,8,9,23,26,27,29,33,34,35,36],practition:[1,2,4,32,39,40,41],pre:32,precalcul:39,preced:[2,12,13,29,38,40,41],preceed:[5,40,41],preceq:9,precis:[1,3,6,12,14,25,26,27,29,32,33,34,35,37,38,39,41],pred:[7,35,37,38],pred_train:[40,41],pred_val:[40,41],predicit:1,prediciton:[40,41],predict:[0,1,2,6,7,8,9,10,11,16,17,24,26,31,32,33,34,35,36,38,39,40,41],predict_prob:[2,40,41],predict_proba:[0,8,11,36,38,39],predictor:[1,6,7,8,10,11,12,32,33,34],prefer:[1,2,7,9,10,12,14,16,24,26,27,32,39,40],prepar:[1,7,25,26,27,32,33],preprocess:[0,1,5,7,8,9,10,11,12,26,34,35,36,40,41],prerequisit:1,prescript:[26,27],presenc:[14,37,38],present:[1,6,7,8,10,13,14,22,25,26,27,29,32,33,34,37,38,39,40,41],preserv:[4,12,25],press:[14,31,36,37,39,40,41],pretrain:[2,5,39,40],pretti:[1,5,9,10,16,23,24,26,32],prev_centroid:15,prevent:[14,29,37,38],previou:[1,2,3,4,5,6,7,9,11,12,13,14,22,23,25,26,27,29,33,36,37,38,39,40,41],previous:[3,4,10,11,29,41],price:[1,5,10,14,37,38],primal:9,primari:[1,8,32,36],prime:29,primit:[3,23],princip:[0,1,6,8,24,32,33,36],principl:[1,7,8,9,15,32,35,36],print:[0,1,2,3,4,5,6,7,8,9,10,11,12,14,15,22,23,25,29,32,33,34,35,36,37,38,39,40,41],print_funct:[9,10],print_length:[40,41],printout:[1,32],prior:[1,6,7,32,34,35],privat:1,pro:27,prob:[2,29,40,41],probabilist:[1,31,32,33],probabl:[0,1,2,4,5,7,8,11,14,23,24,32,33,36,37,38,39,40,41],problem:[0,1,4,5,6,7,8,9,10,11,12,13,17,18,24,25,26,27,29,35],probml:31,proce:[1,6,7,8,9,10,11,12,14,25,32,33,34,35,37,39],procedur:[3,5,6,7,9,11,12,14,22,26,33,34,35,36,37,38,41],proceed:25,process:[1,3,5,7,10,11,13,14,16,22,24,25,26,29,31,32,35,36,37,38,39],prod:31,prod_:[2,6,8,34,35,36,40,41],produc:[1,4,5,6,7,10,11,12,13,14,24,25,26,29,32,33,34,35,37,38,39],product:[1,2,4,6,7,8,9,13,14,17,22,24,25,32,33,34,35,36,38,39,40,41],profess:[1,32],progag:27,program:[1,2,5,6,7,9,13,15,16,17,22,24,25,28,29,30,32,33,38,40],programm:25,progress:[2,5,15,37,39,40,41],prohibit:[7,35],project1:[7,26],project:[1,2,3,4,6,12,14,18,19,20,21,22,24,28,30,33,34,35,37,38,39,40,41],project_root_dir:[1,7,8,10,32,33,35,36],projectwrit:26,promin:[13,38,39],promis:9,promot:[30,32],prone:[10,23,39],pronounc:[14,24,32,37,38],proof:[1,12,13,14,32,35,36,37,39],prop:[40,41],propag:[3,4,14,23,27,37,38],proper:[1,3,7,8,26,35,41],properli:[2,7,9,11,14,22,26,27,37,38,39,40],properti:[1,2,4,13,14,17,18,25,32,35,37,38,39,40,41],propgag:39,proport:[1,2,6,10,12,14,17,29,32,33,37,38,40,41],propos:[2,5,7,11,26,27,32,39,40],propto:[6,14,34,35,36,37,38],proton:[1,32],prove:[4,14,36,37,38],provid:[1,2,4,5,6,7,9,10,11,13,14,16,22,23,24,25,26,27,29,32,33,34,35,36,37,38,39,40,41],proxi:[2,14,22,37,38,39,40],prune:10,pseudo:[25,29,37],pseudocod:[26,27],pseudoinv:[6,34],pseudoinvers:[6,7,34],pseudorandom:[7,29,35],psycholog:[1,32],pt:[14,37],publish:[39,40,41],punish:[1,2,32,40,41],pure:[4,10,29],purest:10,puriti:10,purpos:[1,4,11,13,15,23,32,38,39],put:[2,40,41],py:[0,1,2,3,4,5,6,7,8,9,10,12,15,22,23,32,34,35,36,37,38,40,41],py_tf:[4,5,15],pycod:32,pydata:24,pydot:10,pyhton2:32,pylab:[1,8,32,36],pypi:24,pyplot:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,22,23,25,26,29,32,33,34,35,36,37,38,39,40,41],pythagora:[6,34,35],python2:[1,16,26,32],python3:[0,1,2,3,4,5,7,8,9,10,12,15,16,22,23,24,26,32,34,36,38,40,41],python:[2,3,4,5,6,7,9,12,13,14,15,17,22,23,26,27,29,33,34,37,38,39,40,41],pytorch:[1,24,26,27,32,39,40,41],q:[6,7,9,12,29,33,35,40,41],qp:9,qquad:[3,12,14,25,37,38,41],qr:[6,7,25,33],quad:[2,14,25,37,40,41],quadrat:[1,9,10,14,16,17,32,37],qualit:[5,10,26,27,29],qualiti:[1,10,16,17,24,32,33,39],quantifi:[2,39,40],quantil:11,quantit:[1,7,10,26,27,32,35],quantiti:[1,3,6,7,8,10,11,12,13,15,17,25,29,32,33,34,35,36,39,40,41],quantum:[5,13,31,32,38,39],quartil:[1,33],quasi:39,quench:6,queri:10,question:[1,6,7,10,12,13,14,26,30,32,33,34,35,37,38,39,40],qugan:5,quick:[5,29],quickli:[2,4,10,12,14,36,37,39,40,41],quit:[2,6,7,10,11,13,33,35,38,39,40,41],quot:5,r2:[1,6,7,27,32,33,34,36],r2_score:[1,32],r2score:[1,32],r:[1,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,22,24,25,26,29,33,34,35,36,37,38,39,40,41],r_1:10,r_2:10,r_j:10,r_m:10,rad:1,radial:[1,9,13,38,39],radioact:29,radiu:[1,2,10,33,40,41],rag:[],rain:10,rais:[1,4,5,8,15,34,36,40,41],ramp:[2,39,40],ran0:29,ran1:29,ran2:29,ran3:29,rand:[1,5,6,7,10,11,14,16,17,22,23,25,32,33,34,35,36,37,38,40,41],randint:[7,10,14,22,35,37,38],randn:[1,2,3,6,7,10,12,14,16,17,22,23,32,33,34,35,36,37,38,39,40,41],random:[1,2,3,4,5,6,7,9,10,14,15,16,17,22,23,24,25,26,32,33,34,35,37,38,39,40,41],random_forest_model:11,random_index:[14,22,37,38],random_indic:[2,4,40,41],random_st:[0,1,8,9,10,11,12,36,38,39],randomforestclassifi:11,randomizedsearchcv:36,randomli:[2,7,10,14,15,22,35,36,37,38,40,41],randuniform:36,rang:[0,1,2,3,4,5,6,7,8,10,11,12,13,14,15,22,23,25,29,32,33,34,35,36,37,38,39,40,41],rangl:[1,7,12,29,32,33],rangle_x:29,rank:[6,33],rankdir:5,raphson:[2,9,14,40,41],rapidli:1,rare:[0,2,14,32,36,37,38,39,40,41],rasbt:41,raschka:[27,32,33,34,35,36,37,38],rasckha:32,rashcka:[39,40,41],rashkca:[39,40,41],rate:[1,2,3,4,5,9,10,11,13,14,27,36,39],rather:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,25,29,32,33,34,35,36,37,38,39,40,41],ratio:[0,5,8,10,11,12,36],rational:[1,32],ravel:[0,6,7,8,9,10,11,12,14,25,33,35,36,37,40,41],raw:4,rbf:[9,12,13,38,39],rbf_kernel_svm_clf:9,rbf_pca:12,rc:[1,29],rc_temp:[1,8,36],rcond:[1,32,33],rcparam:[1,2,4,8,9,10,11,29,32,36,40,41],re:[3,5,14,23,36,37,41],reach:[0,2,5,6,7,10,11,12,13,14,15,34,35,36,37,38,39,40,41],read:[1,3,4,5,6,7,8,9,12,13,17,22,25,26,27,28,29,31,34,37],read_csv:[1,7,8,10,32,35,36],read_fwf:[1,32],reader:[1,7,25,29,32,33,34],readi:[1,2,6,7,9,11,12,13,25,26,32,34,35,39,40,41],readili:[2,40,41],readthedoc:24,real:[0,1,2,5,8,11,12,13,25,33,35,36,38,40,41],real_loss:5,real_output:5,realist:9,realiti:29,realiz:[2,13,38,40,41],realli:[1,2,32,39,40,41],rearrang:[14,37,38],reason:[1,2,4,5,11,14,31,32,36,37,38,39,40],reassign:[2,40,41],reat:[40,41],reber:[40,41],recal:[6,7,10,11,12,13,25,29,32,33,34,35,36,37,39,40],recast:4,receiv:[2,4,11,13,29,38,39,40,41],recent:[1,3,4,5,7,8,11,14,15,22,23,26,31,32,34,35,36,37,38,39,40,41],recept:[4,13,38,39],receptive_field:4,recip:[1,7,8,25,26,27,32,33,36],reciproc:[6,34],recogn:[1,5,6,11,32,34,35],recognit:[1,2,4,13,28,31,32,38,39,40],recommen:32,recommend:[1,3,4,5,6,7,9,14,16,19,22,23,24,25,26,27,31,34,35,37,38],reconsid:10,reconstruct:12,record:[11,26,27,28,30,32,37,38],recreat:23,rectangl:[10,14,36,37],rectangular:[6,33],rectifi:[2,4,13,38,39,40,41],recur:[1,24,32],recurr:[1,2,24,32,40],recurs:[10,24,25,32],red:[1,4,5,7,9,10,22,35,37,38],redefin:[1,11,32,33],redefinit:34,reduc:[2,4,6,7,10,11,12,14,22,32,34,35,36,37,38,40,41],reduct:[1,11,12,24,29,32,33],refer:[0,1,2,3,4,6,7,10,12,13,14,15,21,25,27,31,32,33,35,36,37,38,39,40,41],referenc:[3,39,40,41],refin:[13,38,39],refit:[7,35],reflect:[1,2,5,6,26,27,29,32,40,41],refresh:[24,32],refreshprogrammingskil:32,reg:[11,12],regard:[2,10,14,37,39,40],regardless:[13,38,39,40],region:[4,5,7,10,13,26,38,39],regist:[7,26,29],reglasso:[6,34],regr_1:[1,10],regr_2:[1,10],regr_3:[1,10],regress:[2,9,12,13,16,17,22,24,25,40,41],regressor:[1,8,11,32,36,40,41],regridg:[1,6,7,34,36],regular:[0,1,4,5,6,7,8,10,14,18,27,30,32,33,34,35,37,38,39],regularis:7,reilli:[1,16,31,32],reinforc:[1,9,24,32,41],reiniti:[40,41],reiter:[2,40,41],reject:8,rel:[0,1,5,7,8,10,13,14,23,29,32,33,35,36,37,38,39,40],relat:[1,2,4,5,6,12,14,15,25,29,32,34,35,37,38,39,40],relationship:[1,5,10,32],relativeerror:[1,32,33],releas:[2,24,27,32,40,41],relev:[0,1,2,6,8,12,16,22,24,26,27,29,32,34,41],reli:[1,7,9],reliabilti:[26,27],reliabl:[8,29,36],relu:[4,5,23,27,32],remain:[2,3,5,7,13,25,29,33,34,35,36,38,39,40,41],remaind:29,reman:[3,41],remark:[2,39,40],rememb:[1,9,14,23,25,26,27,32,37,38],remind:[1,6,12,14,25,29,33,34,35],remov:[1,5,6,7,33,34],render:[1,10,32,33],reorder:[0,6,8,33,34,36],reorgan:[1,32],repeat:[1,2,4,5,6,7,10,11,12,14,15,22,25,26,27,29,32,34,35,36,37,38,39,40,41],repeat_to_match_shap:[],repeated:32,repeatedli:[1,7,11,14,35,37,38],repet:4,repetit:[7,32,33,35,36,37],rephras:[14,36,37],replac:[1,2,4,5,6,7,11,13,15,16,17,22,24,26,27,32,33,34,35,36,39,40,41],replica:[7,35],repo:[26,27],report:[21,32,37,38],reportexampl:[21,26],reportsampl:21,repositori:[1,5,26,27,32],repres:[0,1,2,3,4,5,6,7,8,9,10,11,13,14,27,29,32,33,34,35,36,37,38,39,40,41],represent:[1,2,4,7,10,29,32,35,36,40,41],representd:4,reproduc:[1,6,7,10,13,16,17,24,26,29,32,33,39,40,41],repuls:[1,32],request:[1,3,14,22,37,38],requir:[1,2,4,5,6,7,9,10,12,13,14,18,25,32,33,34,35,36,37,38,39,40],rerun:[10,40,41],res1:[3,41],res2:[3,41],res3:[3,41],res_analyt:[3,41],res_analytical1:[3,41],res_analytical2:[3,41],res_analytical3:[3,41],resaml:[7,26],resampl:[1,8,11,20,24,32,33,36,37,40,41],rescal:[1,12,13,33,38],rescu:[6,34,35],reseach:[7,26],research:[1,5,14,22,23,24,31,32,37,38],resembl:[7,29,35],reserv:[2,6,7,29,34,35,40,41],reset:[40,41],reset_weight:[40,41],reshap:[1,2,3,4,5,7,9,10,11,16,17,25,32,33,35,40,41],residenti:1,residu:[1,6,14,32,37],resiz:[6,33],resourc:32,respect:[0,1,2,3,4,6,7,8,9,11,12,13,14,15,17,18,22,23,26,29,32,33,34,35,36,37,38,39,40,41],respond:[13,38,39],respons:[1,8,10,13,32,33,36,38,39],rest:[1,6,23,33],restat:[1,13,32],restor:5,restored_discrimin:5,restored_gener:5,restrict:[1,4,10,13,32,38,39,40,41],result:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,22,23,24,25,26,27,29,32,35,36,37,38,41],result_typ:23,retail:1,retain:[6,7,33,35,36],rethink:35,retriev:32,return_data:15,return_sequ:5,return_x_i:10,reus:[2,4,7,19,20,21,26,27,39,40],reveal:[1,13,32,38,39],revers:[2,25,40,41],review:[24,25],revisit:[15,39],revolut:32,reward:[1,5,32],rewrit:[0,1,4,6,7,8,9,11,12,13,14,20,25,26,29,34,36,37,38,39,40],rewritten:[3,7,9,11,29,35,41],rewrot:[14,36,37],rf:11,rgb:4,rgoj5yh7evk:24,rh:[7,35],rho2:[40,41],rho:[1,11,14,22,32,37,38,40,41],rho_1:11,rho_2:11,rho_m:11,rich:[1,32],ride:10,rideclass:10,ridedata:10,ridg:[0,8,12,14,22,24,27,32],ridge_sk:7,ridgebeta:[6,34],ridgecv:36,right:[0,1,2,3,4,6,7,8,9,10,11,13,14,15,17,18,20,23,25,26,29,32,33,34,35,36,37,38,39,40,41],right_sid:[3,41],rightarrow:[1,2,6,7,9,12,13,14,29,32,33,35,36,37,38,39,40,41],rigor:[1,32],ring:7,rise:[1,32],risk:[1,14,16,17,32,36,37,38],rival:5,river:1,rlm:32,rm:[1,29,40,41],rms_prop:[40,41],rmse:1,rmsporp:[14,22,37,38],rmsprop:[2,4,5,14,27,39,40,41],rnd_clf:11,rng:29,rnn1:5,rnn2:5,rnn:[5,13,38,39],rnn_2layer:5,rnn_input:5,rnn_output:5,rnn_train:5,rntrick1:29,rntrick2:29,rntrick3:29,rntrick4:29,ro:[1,14,22,32,36,37,38],robert:[19,26,31],robust:[1,32],robustscal:[1,33],roc:[8,11],role:[1,3,6,7,9,24,32,33,34,35,36,39,40,41],roll:7,room:[1,30,32],root:[1,6,10,14,29,33,34,36,37,38],rot:32,rotat:[2,9,10,11,40,41],rotation_matrix:10,roughli:[2,4,39,40],round:[0,1,8,10,14,36,37,38,40,41],routin:[14,25,32,36,37],row:[1,2,3,6,7,10,12,23,25,32,33,34,35,40,41],row_stack:[],rr:[6,33],rrr:[6,33],rtol:23,rug:[14,36,37,38],rule:[1,2,6,7,14,23,26,32,33,34,35,37,38,41],run:[1,2,3,5,6,7,9,10,12,14,16,22,23,24,26,27,32,33,34,35,36,37,38,40,41],runtim:[2,7,15,39,40,41],runtimeerror:34,runtimewarn:[2,7,35,40,41],rust:[1,16,24,25,32],rvert:[2,40,41],rvert_2:[2,40,41],s:[1,2,3,4,5,6,7,8,10,12,13,14,17,18,19,23,24,25,26,27,29,32,33,34,39,40,41],s_1:7,s_:[4,7],s_i:[7,8,36],s_j:7,s_k:7,saddl:[14,36,37,38],safe:23,sai:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,25,26,29,32,33,34,35,36,39,40,41],said:[7,10,14,36,37],sake:[0,1,6,8,12,32,33,34,36,39,40],sale:[1,32],sam:32,same:[1,2,3,4,5,6,7,9,10,12,13,15,18,23,25,26,29,32,33,34,36,39,40,41],samm:11,sampl:[0,1,2,3,4,5,6,7,8,9,10,11,14,15,16,17,20,22,24,25,26,29,32,33,34,35,36,37,38,39,40,41],sample_vari:15,sampleexptvari:29,samwis:32,sandboxmod:23,sastri:12,satisfactori:[1,32],satisfi:[2,3,4,7,9,14,25,29,35,36,37,40,41],satur:[2,7,35,39,40],save:[1,5,7,8,10,14,22,32,33,35,36,37,38],save_fig:[1,7,8,10,11,32,33,35,36],savefig:[1,5,7,8,10,29,32,33,35,36],savetxt:5,saw:[6,33],scalabl:11,scalar:[3,6,7,11,33,34,35,37,39,40,41],scale:[0,1,2,4,6,7,8,9,10,11,12,13,14,16,17,18,22,24,25,26,27,30,32,34,36,37,38,39,40,41],scale_mean:5,scale_std:5,scaler:[1,8,9,10,11,12,26,33,34,40,41],scan:[6,8,34,35,36],scari:[6,34,35],scatter:[1,2,7,8,9,10,15,22,23,32,33,34,35,36,38,40,41],scenario:[7,14,36,37,38],schedul:[14,30,37,38],scheduler_arg:[40,41],schedulers_bia:[40,41],schedulers_weight:[40,41],scheme:[2,14,36,37,38,39,40],schrage:29,scienc:[0,1,2,11,13,14,24,28,29,30,31,36,37,38,39,40],scientif:[1,16,24,26,27,32,37,38],scientist:[1,32],scikit:[0,4,6,7,9,10,11,14,22,23,24,25,26,27,28,31,38,39],scikit_learn:[1,17],scikitlearn:32,scikitplot:[0,8,11,36],scipi:[1,4,6,7,14,16,24,25,26,32,33,34,35,36,37],scl:7,score:[0,1,2,4,7,8,10,11,12,16,17,23,26,27,30,32,33,35,36,37,38,39,40,41],scores_kfold:[7,35,36],scratch:[2,14,38,39,40],sdg:[14,22,37,38],sdv4f4s2sb8:38,seaborn:[0,1,2,4,7,8,22,27,32,36,40,41],seamless:[1,16,24,26,32],seamlessli:[40,41],search:[1,2,4,6,10,14,32,34,37,38,39,40,41],sebastian:[32,39,40,41],sebastianraschka:[27,32,41],sec:7,second:[0,1,3,4,5,6,7,8,9,10,12,13,15,16,17,23,24,25,26,27,29,30,32,33,35,36,39,40,41],second_correct:[40,41],second_mo:[22,37,38],second_term:[22,37,38],secondeigvector:12,secondli:[13,39,40,41],section:[5,12,17,18,25,26,29,33,34,36,37,38],sector:1,see:[0,1,2,3,4,5,6,7,8,9,11,12,13,14,16,17,18,19,22,23,24,25,26,27,29,32,33,34,35,36,37,38,39,40,41],seed:[1,2,3,4,5,6,7,9,10,12,14,15,16,17,22,23,26,29,32,33,34,35,36,37,38,39,40,41],seed_imag:5,seek:[2,3,9,39,40,41],seem:[2,4,5,37,38,39,40,41],seemingli:[1,32],seen:[1,2,4,6,11,13,18,29,32,40,41],segment:[14,36,37,40,41],seismic:7,seldomli:[1,32],select:[2,6,7,9,10,11,12,18,26,27,28,29,30,31,32,33,34,35,40,41],self:[0,1,2,6,8,22,32,33,34,36,38,40,41],sell:5,semest:[0,8,28,36],semi:[9,14,36,37],semilogx:7,send:[6,13,14,23,30,32,37,38,39],senior:[28,30],sens:[1,5,7,9,23,26,32,35],sensibl:[4,23],sensit:[1,6,7,10,14,32,33,34,35,38],sent:[3,23,39,40,41],sentdex:[38,39,40],sentenc:[5,13,38,39],separ:[1,2,3,5,7,9,10,13,15,16,23,24,26,29,32,35,38,39,40,41],septemb:[18,19,20,21,26,32],sequenc:[3,4,5,8,10,11,13,14,24,25,29,32,36,37,38,39],sequenti:[2,4,5,11,13,29,38,39,40,41],seri:[1,2,3,4,5,6,7,11,12,13,14,25,32,33,34,35,36,37,38,39,40,41],serif:[1,8,29,32,36],serv:[1,2,3,4,6,8,14,22,26,31,32,33,34,35,36,37,38,39,40,41],servic:[26,27],session:[2,21,26,28,30,32],set:[0,2,5,6,7,8,9,11,12,14,15,17,18,22,23,24,25,26,27,29,30,34,35,37,38],set_major_formatt:[7,26],set_major_loc:[7,26],set_tick:[2,7,9,40,41],set_ticklabel:[2,7,40,41],set_titl:[0,1,2,3,4,8,13,15,32,36,38,39,40,41],set_xlabel:[0,1,2,3,4,8,13,32,36,38,39,40,41],set_xlim:[0,8,13,36,38,39,40],set_xticklabel:[2,40,41],set_ylabel:[0,1,2,3,4,8,32,36,40,41],set_ylim:[0,8,13,36,38,39,40],set_ytick:[0,8,36],set_yticklabel:[2,7,40,41],set_zlim:[7,26],seth:5,setminu:[7,36],setosa:[9,10],setosa_or_versicolor:9,setp:[7,35],setup:[2,5,7,9,22,24,27,32,33,39,40],sever:[1,4,6,7,8,9,10,12,13,14,22,24,25,27,29,32,33,34,35,36,37,38,39],sgd:[2,4,22,27,36,40,41],sgd_clf:9,sgdclassifi:9,sgdreg:[14,36,37],sgdregressor:[14,36,37],sgn:[6,33,34],shallow:[14,37,38],shape:[0,1,2,3,4,5,6,7,8,9,10,11,12,14,15,16,17,23,25,32,33,34,35,36,37,38,39,40,41],shape_bas:[],share:[2,4,32,39,40,41],she:[8,36],shell:[22,38],shift:[2,7,13,29,34,38,39,40],ship:4,shire:32,shortcom:[14,36,37,38],shorten:5,shorter:29,shorthand:[32,35],shortli:[25,32],should:[1,3,4,6,7,9,10,12,13,16,17,22,23,25,26,27,29,32,33,34,35],show:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,17,18,19,20,22,25,26,29,32,33,34,35,36,37,38,39,40,41],show_shap:5,shown:[0,1,5,6,9,10,12,13,14,22,25,33,36,37,38,39,40,41],shrink:[4,6,7,9,12,26,33,34],shrinkag:[6,7,33,34],shrunk:12,shuffl:[1,2,5,7,14,22,33,35,36,37,38,40,41],sickit:[39,40],side:[1,3,6,9,13,14,25,27,32,34,35,36,37,38,39,40,41],sigh:[24,32],sigma0:29,sigma1:29,sigma2:29,sigma:[0,1,2,6,7,8,11,12,13,14,18,19,20,25,26,29,32,33,34,35,36,37,38,39,40,41],sigma_0:[6,33,34],sigma_1:[6,33,34,39,40],sigma_2:[6,33,34,39,40],sigma_:[6,25,32,33,34,35],sigma_fn:[0,8,13,36,38,39,40],sigma_i:[1,6,32,33,34],sigma_j:[6,18,33,34],sigma_m:[7,29,35],sigma_n:[12,29],sigma_t:[14,37,38],sigma_x:29,sigmoid:[0,2,3,5,8,9,11,13,23,27,36,37,38,41],sigmundson:[7,33,34],sign:[2,3,8,9,11,29,30,36,38,40,41],signal:[2,4,11,13,38,39,40],signifi:5,signific:[2,39,40],significantli:[2,14,22,29,36,37,38,39,40,41],sigurd:32,sim:[5,6,7,14,19,26,29,34,35,37,38],similar:[1,2,3,4,5,6,7,8,9,10,11,12,15,18,24,25,26,27,32,34,35,36,39,40,41],similarli:[1,2,4,6,9,11,14,21,29,32,33,34,39,40],similiar:[40,41],simpl:[0,2,3,4,6,7,8,9,11,12,13,15,16,17,18,20,22,24,25,26,27,29,35,41],simplefilt:[40,41],simplepredict:11,simpler:[1,2,6,7,8,14,17,20,22,24,26,27,32,34,37,38,40,41],simplernn:5,simplest:[1,2,4,5,10,11,13,15,32,38,39,40,41],simpletre:11,simpli:[1,2,3,5,6,7,9,10,11,12,13,16,24,25,26,27,29,32,33,34,35,38,39,40,41],simplic:[0,3,6,7,8,9,10,11,12,13,15,33,34,36,38,39,40,41],simplicti:[6,33],simplif:39,simplifi:[1,7,10,16,24,26,32,33,34,35,39],simplist:[4,7,29,35],simul:[7,35],simultan:[7,35],sin:[1,2,3,4,5,10,13,14,22,25,32,37,38,39,40,41],sinc:[0,1,2,3,4,6,7,8,9,10,11,12,14,22,23,25,27,29,31,32,33,34,35,36,37,38,39,40,41],sine:[4,13,38,39,40],singl:[0,1,2,3,4,6,7,8,9,10,13,14,23,25,27,29,32,33,34,35,36,37,40,41],singular:[1,7,14,25,26,32,34,35,36,37],sinusoid:4,site:[0,1,2,3,4,5,7,8,9,10,12,15,22,23,26,27,28,32,34,36,38,40,41],situat:[1,5,6,8,14,22,29,32,33,36,37,38],six:[4,29,39],size:[1,2,3,4,5,6,7,9,10,11,12,14,22,23,25,26,27,29,32,34,35,36,37,38,39,40,41],sketch:11,ski:10,skill:1,skip:12,skl:[1,7,32,33,34],sklearn:[0,1,2,4,6,7,8,9,10,11,12,14,15,23,32,33,34,35,36,37,38,39,40,41],skplt:[0,8,11,36],skrankefunct:[40,41],sl:[7,34,36],slack:9,slice:[3,25,32,41],slide:[1,4,16,17,26,27,29,32,33,35,37,39,40,41],slight:[7,14,35,37,38],slightli:[0,2,3,4,6,7,8,11,29,33,34,35,36,39,40,41],slope:[9,12,13,38],slow:[1,3,9,14,33,36,37,38,41],slower:[6,25,32,33,34],slowest:25,slowli:13,slp:[2,40,41],small:[1,2,3,4,6,7,9,10,11,12,13,14,22,23,24,25,26,29,32,33,34,35,36,37,38,39,40,41],smaller:[1,2,3,6,7,9,10,12,14,23,29,32,33,34,35,36,37,38,39,40,41],smallest:[1,5,15,16,17,32,36],smallest_row_index:15,smooth:[1,4,7,10,14,26,32,36,37],sn:[0,1,2,4,7,8,32,36,40,41],sne:12,sneak:32,so:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,17,19,20,22,23,24,25,26,27,29,30,32,33,34,35,36,37,38,39,40,41],soar:7,social:1,soft:[0,2,8,11,13,36,38,39,40,41],soften:9,softmax:[0,4,8,23,36,39],softmax_vec:23,softwar:[1,9,16,24,25,39],sol1:23,sol:9,sole:[1,7,32],solid:[1,8,36],solut:[1,2,3,4,6,7,9,11,12,14,18,19,23,25,26,27,29,32,33,34,35,36,37,38,39,40],solution_ev:37,soluton:[3,41],solv:[1,2,4,6,7,9,11,12,13,14,25,26,27,32,33,38,39,40],solve_expdec:[3,41],solve_ode_deep_neural_network:[3,41],solve_ode_neural_network:[3,41],solve_pde_deep_neural_network:[3,41],solveod:[3,41],solveode_popul:[3,41],solver:[0,3,8,9,10,11,12,22,25,32,36,38,41],some:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,15,23,26,29,32,34,35,38,40,41],some_model:[7,33,34],somehow:5,someth:[0,1,2,4,5,8,10,12,26,27,29,32,33,36,39,40,41],sometim:[1,2,12,13,14,15,32,33,37,38,39,40],somewhat:[38,39],soon:[25,30,33],sophist:[1,32],sopt:[14,37],sort:[6,7,10,12,29,33,35],sound:[4,6,34,35],sourc:[1,2,4,7,16,24,25,26,27,29,32,35,40,41],space:[1,2,5,6,9,10,12,13,14,15,22,29,33,34,35,36,37,38,39,40],span:[1,4,6,10,12,25,32,33],spare:[2,39,40,41],spars:[4,7,25,32],sparse_mtx:[25,32],sparsecategoricalcrossentropi:4,sparsiti:11,spatial:[2,3,4,13,38,39,40,41],speak:29,special:[7,8,11,13,14,25,29,32,33,34,36,37,39,40],specif:[0,1,2,3,4,5,6,7,8,9,10,12,13,17,24,25,26,27,29,31,32,33,34,35,36,38,39,40],specifi:[0,1,4,6,7,8,10,12,14,15,22,26,29,32,34,35,36,37,38,40],specifici:[1,11,32],spectacular:4,spectral:[2,40,41],speech:[1,2,4,5,13,38,39,40],speed:[2,3,5,14,37,38,39,40,41],spend:29,spent:[26,27],sphere:[1,33],spin:7,spite:1,spline:9,split:[2,4,5,6,7,9,10,11,12,15,18,23,26,29,34,35,36,40,41],splite:1,splitter:[2,11,40,41],spontan:29,spot:4,spread:[1,12,29,32,33],spring:[40,41],springer:[19,26,31,32,35],spuriou:[14,22,37,38],sqquar:34,sqrsignal:4,sqrt:[1,4,5,6,7,9,11,12,14,18,22,29,33,34,35,37,38,39,40,41],squar:[0,2,3,4,5,8,9,10,12,14,15,16,17,18,20,22,24,25,27,29,35,36,37,38,39,40,41],squarederror:11,squaredeuclidean:15,squash:[13,38,39,40],squeez:[],srtm:[7,26],srtm_data_norway_1:[7,26],stabil:[6,26,27],stabl:[0,1,5,6,7,10,12,24,26,32,33,34],stack:[4,5],stage:[6,14,22,26,27,37,38,39,40,41],stai:[1,3,5,6,12,18,32,33,40,41],stand:[1,6,10,13,32,33,34,38],standadscal:33,standard:[1,2,5,6,7,8,9,11,13,16,17,18,20,22,25,26,27,29,32,34,36,38,39,40,41],standard_basi:[],standardscal:[1,7,8,9,10,11,12,33,34],stanford:[14,36,41],start:[1,2,3,4,5,6,7,9,10,11,12,13,14,15,21,23,25,27,29,30,32,33,35,36,37,39,40,41],start_box:[],start_nod:[],start_tim:15,starting_point:[22,38],stat:[7,35,36],state:[0,2,3,5,6,7,8,9,11,12,13,14,24,27,29,33,34,35,36,37,38,39,40,41],statement:[0,1,8,25,32,36],stationari:[36,37],statist:[0,1,2,4,5,8,10,11,12,13,14,15,19,25,26,28,31,33,36,37,38,39,40],statu:[0,1,8,10,12,32,36],stavang:[7,26],std:[1,5,7,32,33,35],stdout:[40,41],steep:[14,22,36,37,38],steepest:38,step:[0,1,2,3,5,7,8,10,11,12,13,14,15,22,25,26,32,39,40,41],step_fn:[0,8,13,36,38,39,40],step_length:[14,37,38],step_siz:37,steps_list:10,stereo:4,still:[1,3,4,6,7,12,14,18,23,29,35,36,37,38,39,41],stimuli:[13,38,39],stk2100:[31,32],stk3155:[16,26,27,28,30],stk4021:[31,32],stk4051:[31,32],stk4155:[28,30],stk5000:31,stk:[31,32],stochast:[1,2,6,7,9,12,13,16,17,26,32,34,35,36,39,40,41],stock:5,stoke:[13,38,39],stone:[1,8,26,36,39],stop:[0,2,5,10,12,14,15,22,36,39,40,41],stopiter:[1,8,36],storag:[6,33],store:[1,2,3,4,7,12,14,26,29,32,37,38,40,41],storehaug:[30,32],str:[2,4,5,40,41],straight:[1,7,9,14,32,35,36,37],straightforward:[1,3,4,6,7,9,10,11,14,25,32,33,34,35,36,37,41],strategi:[1,2,10,32,40,41],stratifi:[7,35],strength:[1,6,15,33],stretch:12,strict:[9,14,36,37],strictli:[9,14,36,37],stride:[5,25],strike:7,string:[2,40,41],stroke:[8,36],strong:[4,7,10,11,13,25,29,35,38,39],strongli:[1,9,24,25,27,40,41],stronli:1,structur:[1,2,3,4,7,10,11,13,24,32,35,36,37,38,40,41],stuck:[2,14,36,37,38,39,40,41],student:[1,26,27,28,30,31,32,41],studi:[1,4,5,6,7,8,9,12,13,14,18,20,22,24,26,27,31,32,33,34,37,38,39,40,41],studier:31,stuff:23,style:[1,8,10,25,32,36],sub:[10,13,38,39],subarg:[],subdivid:[1,25,32],subfield:1,subject:[7,9,29],submit:32,subplot:[0,1,2,4,5,7,8,9,10,11,15,23,32,35,36,40,41],subplots_adjust:[9,29],subprogram:[25,32],subract:[1,33],subroutin:[1,32],subscript:[2,40,41],subsequ:[2,5,6,7,13,25,29,33,35,38,39,40,41],subset:[2,7,10,13,14,24,32,35,36,37,38,39,40,41],subspac:[1,9,12,33],substanti:[10,11],substep:12,substitut:[4,7,13,25,35,38],subsubset:10,subtask:7,subtl:[2,39,40],subtract:[1,5,6,7,12,14,18,22,25,26,29,34,35,37,38,40,41],subtre:10,subval:[],succeed:[1,5,32],success:[4,8,10,14,29,36,37],successfulli:[5,10],sudo:[1,16,24,26,32],suffer:[1,2,3,6,11,32,33,34,39,40,41],suffici:[2,7,9,12,14,35,36,37,40,41],suggest:[2,14,26,27,31,37,39,40],suit:[9,13,38,39],suitabl:[1,29,33],sum:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,22,23,25,26,29,32,33,34,36,37,38,41],sum_:[0,1,2,3,4,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,25,26,27,29,32,33,34,35,36,37,38,39,40,41],sum_i:[1,3,6,7,9,14,18,20,26,33,34,35,37,38,41],sum_j:7,sum_ja_:1,sum_k:[7,9,13,25,39,40,41],sum_logist:[14,37],sum_m:4,sum_n:4,sum_nx_:4,summar:[6,7,10,27,34,35,36,38],summari:[2,4,5,11,22,27,28,34,40,41],summat:[1,4,17,33],sunni:10,superconduct:34,superfici:4,superscript:[2,13,38,39,40,41],supervis:[1,6,7,8,10,13,24,32,33,35,36,38,41],supplement:[0,8,26,36],support:[1,2,10,11,12,14,23,24,32,33,37,39,40,41],suppos:[0,1,6,7,8,9,11,12,13,14,25,32,33,34,35,36,37,38,39],suppress:[6,9,14,34,37,38],sure:[1,2,5,7,23,26,39,40,41],surf:[7,26],surfac:[1,7,26,32],surpass:7,surpris:[1,32],surround:[4,24],survei:[1,6,7,32,33,34,35],svc:[9,10,11],svd:[1,7,12,18,32,35],svdinv:[6,34],svm:[9,10,11,12],svm_clf:[9,11],swap:23,swath:[6,33],sy:[14,36,37,40,41],symbol:[2,4,5,6,12,14,15,24,29,32,33,34,37,38,39,40,41],symmeteri:[2,40,41],symmetr:[1,6,9,12,13,14,25,32,33,37,38,39],symmetri:[7,10],sympi:[1,16,24,26,32,39],synonim:29,syntax:[2,14,40,41],syntaxerror:[2,9,40,41],system:[0,1,2,4,5,7,8,10,11,13,14,16,24,25,26,32,36,37,38,39,40,41],systemat:[5,7,35],t0:[4,7,14,22,37,38],t1:[3,14,22,37,38,41],t2:[3,41],t3:[3,41],t:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,22,23,24,25,26,27,29,30,32,35,36,37,38,39,40,41],t_0:[3,10,14,37,38,41],t_1:[14,37,38],t_:[3,41],t_b:11,t_batch:[40,41],t_i:[2,3,6,13,27,33,40,41],t_j:13,t_k:10,t_test:[40,41],t_train:[40,41],t_val:[40,41],tabl:[10,26,27,29,30,32,38,39],tabul:[1,32],tabular:32,tackl:5,tag:[3,4,5,6,7,8,13,14,15,22,25,29,33,36,37,38,39,40,41],tagrget:39,taht:[1,32],tail:29,tailor:[3,9,12,32,39,41],taiwan:[1,32],take:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,17,22,23,24,25,29,32,33,34,35,36,37,38,39,40,41],taken:[1,2,4,7,11,14,23,25,32,35,37,40,41],tan:4,tangent:[2,5,13,14,36,37,38,39,40,41],tanh:[0,2,5,8,9,13,36,37,38,39,40,41],target:[0,1,2,4,5,6,7,8,9,10,11,12,13,23,27,32,33,34,35,36,37,38,39,40,41],target_iscomplex:[],target_meta:[],target_nam:[10,23],target_ndim:[],target_shap:[],task:[1,2,4,7,10,12,13,15,21,23,26,32,35,38,39,40,41],tau:[4,6,29,34,35],taught:32,tax:1,taylor:[3,14,36,37,39,41],taylornr:[14,36,37],tba:[],tc:9,td:[2,7,32,35,37,40,41],teach:32,team:[2,40,41],teaser:1,technic:[1,6,7,14,16,26,32,33,37,38],techniqu:[1,2,9,11,14,24,29,31,32,33,35,36,37,38,39,40],technolog:[1,2,40,41],tell:[1,5,7,11,12,14,29,35,37,38],temp1:[2,40,41],temp2:[2,40,41],temp:[2,40,41],temperatur:[1,10,32],temporarili:[2,39,40],ten:[4,18,32,39],tend:[4,6,7,9,10,11,13,14,15,22,33,34,35,37,38],tendenc:[1,32],tension:[7,35],tensor:4,tensordot:[],tensordot_adjoint_0:[],tensordot_adjoint_1:[],tensorflow:[1,3,5,9,15,16,24,25,26,27,28,31,32,33],term1:[6,7,12,26,33],term2:[6,7,12,26,33],term3:[6,7,12,26,33],term4:[6,7,12,26,33],term:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,20,22,26,29,32,33,34,36,37,38,41],termin:[1,5,6,10,11,14,16,22,32,33,37,38],terrain1:[7,26],terrain:[7,22,26,27],test:[0,4,5,6,7,8,9,10,11,14,18,20,22,23,25,26,29,35,36,37,38,39],test_acc:4,test_accuraci:[2,4,40,41],test_error:7,test_imag:[4,5],test_ind:[7,35,36],test_input:5,test_label:[4,5],test_loss:4,test_pr:[2,40,41],test_predict:[2,40,41],test_rnn:5,test_scor:[0,8,11,36],test_siz:[1,2,4,6,7,11,32,33,34,35,36,40,41],test_split:10,tester:34,testerror:[1,7,33,35],testi:5,testpredict:5,testx:5,text:[1,2,3,5,6,9,10,12,14,16,22,25,27,29,31,33,35,36,37,38,40,41],textbook:[17,18,22,26,27,33,35],textual:10,textur:[2,10,40,41],tf:[2,4,5,14,15,36,37,40,41],tf_loadlibrari:[4,5,15],th:[0,1,2,3,6,7,8,10,13,14,15,16,17,25,26,29,32,33,34,35,36,37,38,39,40,41],than:[0,1,2,3,4,5,6,7,8,10,11,12,13,14,18,23,24,27,29,32,33,34,35,38,39,40,41],thank:[5,7,33,34,41],thats:[40,41],theano:[2,24,32,40,41],thei:[0,1,2,3,4,5,6,7,8,9,10,12,13,14,25,26,29,32,33,34,35,36,37,38,39,40,41],them:[1,2,4,5,7,9,10,11,12,13,14,23,25,26,27,32,33,34,37,38,39,40,41],theme:[1,32,37],themselv:[1,29,32],thenc:[7,35],theorem:[0,3,7,8,33,36,38,40,41],theoret:[1,5,11,32],theori:[1,2,4,9,10,13,14,19,24,26,31,32,34,37,38,39,40,41],thereaft:[1,6,7,12,13,16,17,18,25,26,27,32,35,39,40,41],therebi:[1,6,8,12,32,33,34,36,39],therefor:[0,1,2,3,4,5,7,8,9,12,14,29,32,33,35,36,37,38,40,41],therein:12,thereof:[1,7,14,32,35,37,38],thesi:41,theta:[2,5,14,22,29,37,38,39,40,41],theta_:[2,14,37,38,40,41],theta_i:[2,40,41],theta_linreg:[14,22,37,38],theta_t:[14,37,38],thetaand:[38,39],thetaor:[38,39],thetaxor:[38,39],thi:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,24,25,26,27,28,29,31,33,34,35,36,37,38,40,41],thing:[0,1,2,3,5,6,8,10,23,29,32,34,35,36,39,40,41],think:[1,2,4,5,7,10,13,14,15,29,32,35,36,37,38,39,40,41],third:[1,4,7,14,22,30,32,36,37,38],thirti:[0,8,36],thorughout:32,those:[1,4,6,7,9,10,11,12,22,25,26,27,28,32,33,35,39],though:[2,3,4,5,14,23,25,29,37,38,40,41],thought:[7,15,26,27,29,35],thousand:[1,2,26,33,37,38,39,40],three:[1,2,4,6,7,9,10,13,16,23,25,26,28,29,30,32,33,34,35,36,38],threshold:[2,4,10,11,12,13,14,37,38,39,40,41],through:[1,2,3,4,5,6,7,9,12,13,14,15,23,24,25,29,32,33,34,35,36,37,38,40,41],throughout:[1,5,6,15,24,25,29,32,40,41],thu:[1,2,3,6,7,8,9,11,12,13,14,22,26,30,32,33,34,35,36,37,38,39,40,41],thumb:[1,7,26,32,33],thursdai:[30,33],tibshirani:[7,19,26,28,31,32,35],tick:7,tick_param:7,ticker:[7,14,26,29,36,37],tif:[7,26],tight_layout:[0,2,8,36,40,41],tightli:12,tild:[1,6,7,8,12,16,17,18,19,20,26,29,32,33,34,35,39,40,41],till:[0,1,5,8,9,10,11,13,25,32,33,34,36,39,40,41],time:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,17,18,22,23,24,25,26,27,29,32,33,34,35,36,39,40,41],timeit:5,timer:5,tini:[2,39,40],tip:4,titl:[1,2,3,4,5,7,8,9,10,11,14,22,23,26,29,32,35,36,37,38,39,40,41],tmp:[14,37,38],tn:[3,4,8,41],to_categor:[2,4,5,40,41],to_categorical_numpi:[2,40,41],to_numer:[1,7,32,35],todai:4,togeth:[1,4,7,9,12,14,22,24,32,37,38,41],toi:[15,37,38],told:[14,37],toler:[3,7,15,23,41],tolist:5,tomographi:[13,38,39],too:[1,3,5,6,7,10,12,14,16,29,31,32,33,35,36,37,38,41],took:[9,32],tool:[1,2,4,7,14,24,33,35,37,38,39,40],toolbox:9,top:[1,4,6,7,10,11,19,24,32,34,35],topic:[1,6,7,8,9,24,26,27,33,34,36],topograph:26,topolog:[2,4,13,38,39,40],toposort:[],torkjellsdatt:[30,32],toss:[11,29],total:[0,1,2,3,4,5,7,8,9,10,11,12,13,14,15,16,17,25,27,29,30,32,33,35,36,37,39,40,41],total_loss:5,totalclustervari:15,totalscatt:15,toward:[2,3,8,13,14,26,36,37,40,41],towardsdatasci:[37,38],town:1,tp:[5,8],tpng:10,tpu:[14,22,24,32,37,38],tqdm:7,tr:33,trace:[3,23],trace_stack:[],traceback:[1,3,4,5,7,8,11,14,15,23,26,32,34,36],tracer:[3,23],track:[4,14,15,25,33,36,37,38],tract:1,tractabl:[1,32,33],trade:[6,10,20,34,35],tradeoff:[1,6,20,26,32,33,34,36],tradit:[1,2,5,7,32,35,40,41],train:[3,4,6,7,9,10,11,12,13,14,18,20,22,26,27,34,35,36,37,38],train_acc:[40,41],train_accuraci:[1,2,4,32,40,41],train_dataset:5,train_end:[1,2,33,40,41],train_error:[7,40,41],train_imag:[4,5],train_ind:[7,35,36],train_label:[4,5],train_network:23,train_pr:[2,40,41],train_siz:[1,2,4,33,40,41],train_step:5,train_test_split:[0,1,2,4,6,7,8,10,11,12,32,33,34,35,36,40,41],train_test_split_numpi:[1,2,33,40,41],trainable_vari:5,trained_model:[7,33,34],trainerror:[1,33],traini:5,training_checkpoint:5,training_dataset:5,training_gradi:[14,22,37,38],training_gradient_fun:[37,38],training_loss:[37,38],trainingerror:[7,35],trainpredict:5,trainscor:5,trainx:5,trait:[1,32],trajectori:5,trajectory_i:[22,38],trajectory_x:[22,38],transfer:10,transform:[0,1,6,7,8,9,10,11,12,13,14,22,23,24,25,32,33,34,35,36,37,38,39,40,41],transit:[7,13,38,39],translat:[2,5,7,11,32,33,34,40,41],transpos:[2,6,12,23,25,33,34,40,41],travers:[1,6],treat:[1,2,4,7,13,14,23,29,32,33,34,35,36,37,38,39,40,41],tree:[1,2,7,24,26,32,39,40,41],tree_clf:[10,11],tree_clf_:10,tree_clf_sr:10,tree_reg1:10,tree_reg2:10,tree_reg:10,trend:29,treue:8,trevor:[19,26,31],tri:[3,4,5,10,14,36,37,41],triain:1,trial:[1,3,5,7,14,29,32,35,36,37],triangl:[14,36,37],triangular:25,trick:[4,5,9,12,14,22,29,37,38],trickier:29,tridiagon:25,trigger:[1,8,36],trillion:24,trivial:[1,2,6,12,29,32,34,40,41],troubl:[1,9,13,23,33,39,40],truck:4,true_beta:[7,33,34],true_divid:[],true_fun:[7,35],truncat:[22,38,39],trust:10,tucker:9,tuesdai:[30,32,35,36,37,41],tumor:[0,8,10,27,36],tumour:[0,8,36],tunabl:[2,22,27,40,41],tune:[5,10,14,22,25,27,32,36,37,38],tup:[],tupl:[23,40,41],turn:[0,1,2,6,7,8,9,10,11,12,13,14,20,25,26,29,32,33,35,36,37,38,39,40,41],tutori:[2,5,40,41],tv:[3,41],tveito:[3,41],tvw1zdmznwm:38,tweak:[2,5,11,29,39,40,41],twice:[14,36,37],twist:12,two:[1,2,3,5,6,7,8,10,11,12,13,14,16,17,23,25,28,29,31,32,34,35,37,38,41],tx:[14,36,37,38,39],tx_1:[14,36,37],txt:5,ty:[14,36,37],type:[1,2,4,7,9,11,14,17,23,25,26,29,33,34,35,36,37,40],typeerror:[7,14,23],typic:[1,2,3,4,5,6,8,10,11,13,14,22,27,29,32,33,34,35,36,37,38,39,40,41],typo:[26,27],u:[1,3,6,7,9,11,12,13,18,25,32,33,34,38,39,41],u_:25,u_i:[13,38],u_m:11,ua:[1,32],ubuntu:[1,16,24,26,32],uci:[1,27],ufunc:23,uio:[23,26,27,30,31],un:15,unabl:[10,23],unari:[25,32],unary_f:[],unary_oper:[],unary_to_nari:[],unbalanc:[7,10,35,36],unbias:[1,6,7,32,34,35],unbroadcast:[],unbroadcast_f:[],uncent:[7,34],uncertainti:[1,6,32,34,35],uncertitud:29,unchang:[2,4,39,40],uncorrel:[11,29],undefin:[6,33],under:[1,2,6,7,11,14,16,17,24,26,27,32,33,34,35,36,37,40,41],underdetermin:[1,32],underfit:[2,7,35,39,40],underflowproblem:[6,34,35],undergo:[6,23,34,35],undergradu:[28,30],underli:[1,2,10,14,29,32,37,38,39,40],underset:[5,15],understand:[1,2,4,6,7,11,14,15,22,23,24,32,33,34,36,37,38,39,40,41],understood:[9,14,37,38],undesir:9,undetermin:[6,9,34,35],undo:5,unexpect:[7,14,35],unexpected:29,unfair:[7,33,34],unfortun:[2,9,10,11,39,40],unicode_liter:[9,10],uniform:[1,2,6,7,12,14,16,17,26,29,32,33,36,37,40,41],uniformli:[14,29,36,37,38],unifrompdf:29,unimport:[14,36,37],union:[6,7,34,35,36],uniqu:[1,3,7,14,15,25,32,35,36,37,41],unique_cluster_label:15,unit:[1,2,4,5,6,11,13,29,32,33,34,38,39,40,41],unitari:[6,7,25,33],unitarili:[25,32],uniti:29,univari:29,univers:[0,1,2,3,14,16,24,26,27,28,30,32,33,34,35,36,37,38,40,41],unix:[2,40,41],unknow:[1,25,32],unknown:[1,2,4,5,6,7,9,11,14,17,22,25,32,33,34,35,37,38,39,40,41],unknowwn:13,unlabel:[2,39,40],unless:[1,4,7,12,14,26,27,32,35,36,37,39],unlik:[2,4,9,14,36,37,38,39,40,41],unnecessarili:10,unord:4,unravel:[2,40,41],unrol:[4,12],unseen:[1,8,10,36],unstabl:[2,39,40],unsupervis:[1,2,5,13,24,32,38,39,40],unsymmetr:[25,32],until:[2,3,5,10,13,14,15,23,36,37,38,39,40,41],untouch:1,unusu:[13,38,39],up:[2,4,5,6,7,9,11,12,14,15,17,19,23,24,25,26,27,29,30,34,37],updat:[0,2,3,11,13,14,15,22,23,37,38],update_chang:[40,41],update_matrix:[40,41],uploa:32,upload:[24,26,27,31],upon:[1,2,7,8,12,25,26,39,40,41],upper:[1,9,10,17,25,33],uppercas:[25,32],upsampl:5,upscal:5,uptad:39,ur:40,url:[1,8,32,33,36,38],us:[5,6,7,9,10,11,12,13,15,16,17,18,19,20,23,25,28,29,31,34,35],usa:[32,38],usag:[0,1,9,24,32,33,38,39],usd10000:1,usd:1,use_bia:5,usecol:[1,32],useless:[2,40,41],user:[0,1,2,3,4,5,7,8,9,10,12,15,16,22,24,25,26,32,33,34,36,38,40,41],usernam:[26,27],userwarn:[7,22,38],usetex:29,usg:[7,26],usr:29,usual:[0,1,4,5,8,13,14,15,32,36,37,38,39],ut:[6,34],utf:[1,8,36],util:[2,4,5,7,8,11,15,27,32,35,36,39,40,41],ux:25,v0:29,v1:29,v2:29,v8xr:[38,39,40],v:[3,5,6,7,12,14,18,22,24,33,34,37,38,39,40,41],v_0:12,va:[2,40,41],vahid:32,val:[14,37,38],val_acc:[40,41],val_accuraci:4,val_error:[40,41],val_loss:5,val_set:[40,41],vale:[3,41],valid:[0,1,2,5,8,10,11,14,22,24,27,29,32,33,37,38,39,40,41],validation_data:4,validation_split:5,valu:[0,1,2,3,4,5,7,8,9,10,11,13,14,15,16,17,20,22,23,24,25,26,32,37,38,39,40,41],valuat:10,valued_at_a:[40,41],valued_at_z:[40,41],valueerror:3,valy:5,van:[1,19,26,32,33,34,35],vandenbergh:[9,14,36,37],vandermond:[1,32],vanilla:[1,7,12,15,33,34],vanish:[2,5,14,29,36,37],var_x:29,varabl:9,varepsilon:[6,7,19,26,34,35],varepsilon_:[6,7,34,35],varepsilon_i:[6,7,34,35],vari:[1,2,4,6,7,11,16,17,23,32,35,39,40,41],variabl:[1,2,3,6,7,8,9,11,12,13,14,15,22,23,25,32,33,34,35,38,39,40,41],varianc:[1,2,6,8,10,11,12,14,15,19,20,22,24,25,29,32,33,36,37,38,39,40],variance_i:[6,12,33],variance_x:[6,12,33],variant:[1,2,7,9,13,14,32,33,36,37,38,39,40,41],variat:[4,5,12,32],varieti:[1,4,13,16,24,26,32,38,39],variou:[0,2,4,6,7,8,9,10,12,13,14,17,19,22,24,25,26,29,33,34,37,38,39,40,41],varydimens:5,vastli:4,vaue:[2,40,41],vault:1,vdot:[3,14,36,37,41],ve:[26,27],vec:[7,35],vector:[0,1,2,3,4,5,6,7,8,10,11,12,14,15,17,18,22,23,24,34,35,36,37,39,40,41],vector_mean:15,ventur:[1,9,16,24,32],verbos:[2,4,5,40,41],veri:[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,23,26,27,29,31,32,33,34,35,36,37,38,39,40,41],verifi:[4,12,25,32],versatil:[9,32],versicolor:[9,10],version:[1,4,11,14,15,16,23,24,25,26,27,29,32,37],versu:[2,39,40],vert:[0,1,2,6,7,8,9,10,12,14,18,32,33,34,35,36,37,39,40,41],vert_1:[6,7,33,34,35],vert_2:[6,7,12,18,33,34,35],vi:[40,41],via:[0,1,6,7,8,9,10,11,12,13,16,20,24,25,26,28,29,30,32,33,34,35,36,38,39,40,41],vidal:12,video:[1,2,13,24,28,30,32,33,34,35,39,40,41],view:[2,4,6,7,13,14,22,29,31,32,34,35,37,38,40],vii:[40,41],viii:[40,41],violat:9,virginica:10,viridi:[1,2,3,4,32,40,41],virtual:[2,39,40],viscos:[14,37,38],viscou:[14,37,38],visibledeprecationwarn:[],vision:[1,4],visual:[1,4,12,13,16,24,32,33,38,39],visualis:[2,40,41],viz:[7,9,29],vjp:[],vjp_0:[],vjp_0_fun:[],vjp_1:[],vjp_1_fun:[],vjp_argnum:[],vjpfun:[],vjpmaker:[],vjpnode:[],vjps_dict:[],vmap:[14,37],vmax:[2,7,40,41],vmh0zpt0tli:38,vmin:[2,7,40,41],voic:4,volum:[1,4,32],volume18:41,von:[39,40,41],vote:11,voting_clf:11,votingclassifi:11,votingsimpl:11,vs:[1,5,7,37],vscode:23,vspace:[],vstack:[6,12,25,29,32,33,40,41],vt:[6,33,34],w1:[9,23],w2:[9,12,23],w3:9,w:[0,1,2,3,4,5,6,7,8,9,11,12,13,14,15,23,25,29,32,33,35,36,37,38,39,40,41],w_0:39,w_1:[9,25,39,40],w_1a_0:[39,40],w_1x:[39,40],w_1x_1:9,w_1x_:9,w_2:[9,25,39,40],w_2a_1:[39,40],w_2x_2:9,w_2x_:9,w_3:25,w_4:25,w_:[2,13,38,39,40,41],w_g:23,w_hidden:[3,41],w_i:[2,3,11,39,40,41],w_ix_i:[13,38,39],w_j:25,w_m:25,w_output:[3,41],w_px_:9,w_px_p:9,wa:[1,2,3,4,5,6,7,8,11,12,13,15,18,23,25,26,32,33,34,35,36,38,39,40,41],wai:[1,2,3,4,5,6,7,8,9,11,12,13,14,15,18,23,25,27,29,32,33,34,36,37,40,41],walk:10,walker:29,wang:[1,32],want:[1,2,3,4,5,6,7,9,10,11,12,13,14,15,16,17,18,23,24,26,27,29,32,33,34,35,36,37,38,39,40,41],warn:[2,5,9,40,41],warrant:[7,35,36],wast:[4,37,38],watch:[24,38,39,40,41],wave:4,wavelet:9,we:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,23,24,25,26,27,28,29,30,31,33,34,35,36],weak:[10,11,15],weather:[2,13,38,39,40,41],web:[24,28,30,32],weblink:27,webpag:32,websit:[7,25,26,28,32],wedg:[9,29,39,40],wednesdai:[30,32,35,36,37,41],wee:12,week:[1,6,7,8,26,27,28,30],weekli:[17,24,26,30,31,32,38],weierstrass:39,weight:[0,1,2,3,4,7,8,10,11,13,14,23,26,27,29,36,37,38,39],weight_arrai:[40,41],weigth:[3,41],welchlab:[38,39,40],welcom:[9,24],well:[0,1,2,3,4,5,6,7,8,9,10,11,13,14,16,17,20,21,22,23,24,25,26,27,29,31,32,33,34,35,36,37,38,39,40,41],went:9,were:[1,2,4,5,6,7,8,9,11,12,13,15,29,32,33,35,36,38,39,40,41],wessel:[1,19,26,32,33,34,35],what:[2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,20,23,24,25,26,27,29,36,37,38,39,40,41],whatev:[4,23],when:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,17,18,19,22,23,25,26,27,29,32,33,34,35,36,39,40,41],whenev:[14,22,29,37,38,39],where:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,22,23,24,25,26,27,29,30,32,33,34,35,36,37,38,39,40,41],wherea:[7,29,35],wherefrom:26,wherein:[2,13,38,39,40,41],whether:[1,4,6,8,10,26,27,29,32,34,36],which:[0,1,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,23,24,25,26,27,28,29,30,32,33,34,35,36],whichev:[2,4,40,41],white:10,whiteboard:[34,35,36,37,38,39,40],who:[1,28],whole:[2,4,5,6,10,12,14,23,34,35,37,38,40,41],whose:[1,7,11,29,33,35],whow:[12,33],why:[1,2,4,7,14,22,23,26,34,36,37,40],wide:[0,1,2,4,7,8,13,16,24,25,26,32,35,36,38,39,40,41],widehat:[7,35],width:[1,4,9,10,23,32],wieringen:[1,19,26,32,33,34,35],win:11,wind:10,wing:[30,32],winther:[3,41],wiothout:[7,33],wiscons:[0,8,36],wisconsin:[11,27,40,41],wisdom:[7,34],wise:[1,2,6,13,14,23,33,37,38,40,41],wish:[0,1,3,6,8,9,12,14,15,25,26,27,32,33,36,37,39,40,41],with_std:[1,33],wither:7,within:[0,1,3,4,5,8,10,13,14,15,23,29,31,32,36,37,41],withinclust:15,without:[1,2,6,7,9,10,12,13,14,22,27,32,33,34,35,37,38,39,40,41],wo5dmep_bbi:[38,39,40],won:[1,32,39],wonder:9,word:[1,2,4,5,6,7,8,15,29,32,33,34,35,40,41],work:[1,2,5,7,8,9,10,14,16,22,23,24,26,27,28,29,30,32,33,35,36,37,38,39,40,41],workshop:32,world:[1,9,33],worldwid:[1,32],wors:[1,2,4,5,7,32,35,39,40],worst:10,worth:[10,23],worthi:[26,27],would:[1,2,4,6,7,8,9,10,11,12,13,14,25,26,27,29,32,33,34,35,36,37,38,39,40,41],wrap:[7,25,32],wrap_util:[],wrapper:23,write:[1,2,3,4,6,7,8,9,13,14,16,17,18,21,22,23,25,32,33,35,36,37,38,39,41],written:[1,3,4,6,12,13,14,17,24,25,26,27,29,32,33,34,35,36,37,38,39,40,41],wrong:[2,9,39,40,41],wrongli:11,wrote:[6,12,33],wrt:[11,14,22,23,37,38,39,40],wth:[11,14,22,37,38],wurstemberg:[39,40,41],www:[24,25,26,27,31,32,34,38,39,40,41],wx_1:9,x0:9,x1:[5,9,10,11,14,37,38],x1_exampl:9,x1d:9,x2:[9,10,11,14,37,38],x2d:[9,12],x2d_train:12,x2dsl:12,x3:[9,10],x4:10,x:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,22,23,25,26,27,29,32,35,36,37,38,39,40,41],x_0:[1,6,12,25,32,33,34,35,39],x_1:[0,1,3,6,7,8,9,10,11,12,14,25,29,32,33,34,35,36,37,38,39,40,41],x_2:[0,1,3,6,7,8,9,10,11,12,14,25,29,32,33,35,36,37,38,39,40,41],x_3:[9,25,29,39],x_4:[25,39],x_5:39,x_:[1,3,4,6,7,9,11,12,14,15,19,25,26,29,32,33,34,35,36,37,39,41],x_batch:[40,41],x_center:12,x_data:[2,40,41],x_data_ful:[2,40,41],x_hidden:[3,41],x_i:[0,1,2,3,6,7,8,9,10,11,12,13,14,15,18,25,29,32,33,34,35,36,37,38,39,40,41],x_input:[3,41],x_ix_:[1,32],x_iy_i:9,x_j:[1,3,9,10,13,17,29,33,38,39,41],x_jy_j:9,x_k:[13,15,25,29,33,38],x_l:[29,39],x_m:[7,13,25,29,35,38],x_n:[1,3,4,7,9,12,13,14,25,29,32,35,36,37,38,39,41],x_new:[10,11],x_offset:[7,33,34],x_output:[3,41],x_p:[0,4,8,10,36],x_poli:10,x_poly10:10,x_pred:5,x_prev:[3,41],x_reduc:12,x_scale:9,x_small:[14,37],x_test:[0,1,2,4,6,7,8,10,11,12,32,33,34,35,36,40,41],x_test_own:7,x_test_scal:[1,7,8,10,11,12,33,34],x_tot:5,x_train:[0,1,2,4,5,6,7,8,10,11,12,32,33,34,35,36,40,41],x_train_mean:[7,33,34],x_train_own:7,x_train_scal:[1,7,8,10,11,12,33,34],x_val:[2,40,41],xarrai:[24,32],xavier:[2,39,40],xbkjd4ccgq:32,xbnew:[14,36,37],xcode:[1,16,24,26,32],xdclassiffierconfus:11,xdclassiffierroc:11,xfin:23,xg_clf:11,xgb:11,xgbclassifi:11,xgboost:10,xgboot:11,xgbregressor:11,xgparam:11,xgtree:11,xi:[9,14,22,37,38],xi_1:9,xi_:9,xi_i:9,xinv:[34,38,39],xk:9,xla:[14,22,24,32,37,38],xlabel:[1,2,3,4,5,6,7,8,9,10,11,14,22,23,26,29,32,34,35,36,37,38,40,41],xlim:[7,11,35],xm:10,xmesh:[14,37],xnew:[1,14,22,32,36,37,38],xp:29,xpanda:[1,33],xpd:[6,12,33],xplot:1,xs:10,xscale:[1,33],xsr:10,xt_x:[14,22,36,37,38],xtest:[7,35,36],xtick:[4,7,9,10,35],xtrain:[7,35,36],xu:[1,32],xx:[1,25,32],xy:[1,7,9,25,26,32],xytext:9,xz:[25,32],y1:5,y2:5,y3:5,y:[0,1,2,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,22,23,25,26,29,32,33,34,35,36,37,38,39,40,41],y_0:[1,6,12,25,32,33,34,35],y_1:[1,6,9,10,12,14,25,32,33,34,35,36,37],y_1y_1:9,y_1y_1k:9,y_1y_2:9,y_1y_2k:9,y_1y_n:9,y_1y_nk:9,y_2:[1,6,9,10,12,25,32,33],y_2y_1:9,y_2y_1k:9,y_2y_2:9,y_2y_2k:9,y_3:[1,10,25],y_4:25,y_:[1,2,6,7,11,12,25,32,33,34,35,40,41],y_data:[1,2,6,7,32,33,34,36,40,41],y_data_ful:[2,40,41],y_decis:9,y_fit:[1,33],y_i:[0,1,2,6,7,8,9,10,11,12,13,14,16,17,18,19,20,25,26,27,32,33,34,35,36,37,38,39,40,41],y_if_:11,y_ix_:[1,32],y_ix_i:[0,8,9,14,33,36,37],y_iy_jk:9,y_j:[7,9,13,20,26,35,38,39,40],y_k:[13,38],y_m:25,y_model:[1,5,6,7,32,33,34,36],y_n:[9,14,36,37],y_ny_1:9,y_ny_1k:9,y_ny_2:9,y_ny_2k:9,y_ny_n:9,y_ny_nk:9,y_offset:[7,33,34],y_plot:10,y_pred1:10,y_pred2:10,y_pred:[0,1,2,5,7,8,9,10,11,33,34,35,36,40,41],y_pred_rf:11,y_pred_tre:11,y_predict:[0,38],y_proba:[0,8,11,36],y_scaler:[7,34],y_test:[0,1,2,4,5,6,7,8,10,11,12,32,33,34,35,36,40,41],y_test_onehot:[2,40,41],y_test_predict:1,y_tot:5,y_train:[0,1,2,4,5,6,7,8,10,11,12,32,33,34,35,36,40,41],y_train_mean:[7,33,34],y_train_onehot:[2,40,41],y_train_predict:1,y_train_scal:[7,34],y_val:[2,40,41],yadav:41,yand:[38,39],ye:[4,7,8,35,36],year:[1,24,32],yet:[1,2,7,9,12,14,23,32,36,37,39,40,41],yfin:23,yi:[14,22,37,38],yield:[1,3,6,7,8,9,11,13,14,15,25,29,32,34,35,36,37,38,39,40,41],yk:9,ylabel:[1,2,3,4,5,6,7,8,9,10,11,14,22,23,26,29,32,34,35,36,37,38,40,41],ylim:[4,7,35],ym:10,ymesh:[14,37],yn:1,yo:[9,10,11],yor:[38,39],yoshua:[2,31,39,40],you:[1,2,4,5,6,7,9,10,11,12,14,16,17,18,20,21,22,23,24,25,26,27,29,30,31,32,33,34,35,36,37,38,39,40,41],young:1,your:[2,3,5,6,7,9,12,14,18,20,21,22,23,24,25,26,34,35,36,37,38,39,40,41],yourself:[12,14,32,37],youtu:[36,38],youtub:[24,38,39,40,41],ypred:[7,35,36],ypredict2:[14,22,36,37,38],ypredict:[1,14,22,32,33,36,37,38],ypredictlasso:[6,34],ypredictol:[1,6,34],ypredictown:[7,33,34],ypredictownridg:[7,34],ypredictridg:[1,6,7,34,36],ypredictskl:[7,33,34],yridg:32,ys:10,ytest:[7,35,36],ytick:[4,7,9,10,35],ytild:[1,7,32,33,35],ytildelasso:[6,34],ytildenp:[1,32,33],ytildeol:[1,6,34],ytildeownridg:[7,34],ytilderidg:[6,7,34],ytrain:[7,35,36],yuxi:32,yx:[25,32],yxor:[38,39,40,41],yy:[25,32],yz:[25,32],z1:23,z2:23,z:[0,1,2,3,4,5,6,7,8,9,10,12,13,14,23,25,26,29,32,33,35,36,37,38,39,40,41],z_0:[25,32,39],z_1:[25,32,39,40],z_2:[25,32,39,40],z_:[2,3,13,25,32,39,40,41],z_c:[2,40,41],z_h:[2,40,41],z_hidden:[3,41],z_i:[2,13,38,40,41],z_j:[2,13,41],z_k:[13,33,39,40],z_m:[2,40,41],z_matric:[40,41],z_mod:10,z_o:[2,40,41],z_output:[3,41],zaman:29,zaxi:[7,26],zero:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,18,19,20,22,23,25,26,29,32,33,34,35,36,37,38,39,40,41],zeros_lik:5,zeroth:33,zfill:5,zip:[5,7,23],zm_h:[1,32],zn:1,zone:1,zoom:32,zx:[25,32],zy:[25,32],zz:[25,32]},titles:["Exercises Week 42: Logistic Regression and Optimization, reminders from week 38 and week 40","3. Linear Regression","14. Building a Feed Forward Neural Network","15. Solving Differential Equations with Deep Learning","16. Convolutional Neural Networks","17. Recurrent neural networks: Overarching view","4. Ridge and Lasso Regression","5. Resampling Methods","6. Logistic Regression","8. Support Vector Machines, overarching aims","9. Decision trees, overarching aims","10. Ensemble Methods: From a Single Tree to Many Trees and Extreme Boosting, Meet the Jungle of Methods","11. Basic ideas of the Principal Component Analysis (PCA)","13. Neural networks","7. Optimization, the central part of any Machine Learning algortithm","12. Clustering and Unsupervised Learning","Exercises week 34","Exercises week 35","Exercises week 36","Exercises week 37","Exercises week 38","Exercises week 39","Exercises week 41","Exercises week 42","Applied Data Analysis and Machine Learning","2. Linear Algebra, Handling of Arrays and more Python Features","Project 1 on Machine Learning, deadline October 7 (midnight), 2024","Project 2 on Machine Learning, deadline November 4 (Midnight)","Teaching schedule with links to material","1. Elements of Probability Theory and Statistical Data Analysis","Teachers and Grading","Textbooks","Week 34: Introduction to the course, Logistics and Practicalities","Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression","Week 36: Linear Regression and Statistical interpretations","Week 37: Statistical interpretations and Resampling Methods","Week 38: Logistic Regression and Optimization","Week 39: Optimization and Gradient Methods","Week 40: Gradient descent methods (continued) and start Neural networks","Week 41 Neural networks and constructing a neural network code","Week 42 Constructing a Neural Network code with examples","Week 43: Deep Learning: Constructing a Neural Network code and solving differential equations"],titleterms:{"0":40,"1":[1,16,17,18,19,23,26,32,33,39,40,41],"11":39,"14":40,"16":36,"2":[1,16,17,18,19,23,27,32,33,34,39,40,41],"2023":30,"2024":[26,37,38,39,40],"21":41,"23":37,"26":33,"27":37,"3":[1,16,17,23,32,33,39,40,41],"30":38,"34":[16,32],"35":[17,33],"36":[18,34],"37":[19,35],"38":[0,20,36],"39":[21,22,37],"4":[1,23,27,33,40],"40":[0,22,38],"41":[22,39],"42":[0,23,40],"43":41,"5":[1,23],"6":23,"7":[23,26,39],"9":35,"case":[9,11,29,33,34,36,37],"class":[0,36,39],"do":[2,34,35,37,38,39,40,41],"final":[13,22,27,33,34,37,38,39,40,41],"float":39,"function":[0,1,2,7,8,9,11,12,13,14,22,26,27,29,32,33,34,35,36,37,38,39,40,41],"import":[6,22,25,32,33,34,38,39,40],"new":[5,34,35,39],A:[0,1,2,5,9,10,22,32,34,35,36,38,39,40,41],AND:[38,39],And:[22,32,33,34,36,37,38],But:[22,37,38],For:33,In:[30,39],Is:[39,40],Ising:7,OR:[38,39],The:[0,1,2,3,4,6,7,8,9,10,12,13,18,24,32,33,34,35,36,37,38,39,40,41],To:[32,33],With:[5,34],about:[32,33],abov:[34,39,40,41],activ:[2,13,23,27,34,38,39,40,41],ad:[1,7,18,26,32,33,38,39,40],adaboost:11,adagrad:[14,22,37,38],adam:[14,22,37,38],adapt:[11,22,37,38],adjust:[2,40,41],advanc:[22,38],adversari:5,again:[0,4,10,36],ai:[26,32],aim:[9,10,18,19,20,21,22,23,32],aka:[32,33],al:[22,38],algebra:[25,32],algorithm:[10,11,12,13,22,27,33,37,38,39,40,41],algortithm:[14,36,37],all:[9,39,40],an:[1,5,11,32,39],analys:[6,33],analysi:[1,6,7,12,24,26,27,29,32,33,34,35,39],analyt:[1,17,18,22,41],analyz:[39,40],ani:[14,36,37],anoth:[10,34,35],appli:24,approach:[1,9,15,32,35,37,38],approxim:[13,39],architectur:[2,40,41],argument:[37,38],arrai:[25,32],artifici:[38,39],assist:30,assumpt:[34,35],august:33,autocorrel:29,autograd:[3,14,22,37,38,41],automat:[14,22,37,38,39,41],avoid:[],b:[18,26,27,37],back:[2,12,13,39,40,41],background:[24,26,27,35],bag:11,base:[14,22,35,37,38],basic:[1,6,8,10,11,12,25,33,34,35,36,39],batch:[2,37,38,39,40],bay:[6,34,35],befor:12,bengio:[39,40],beta:[34,35],better:[9,38,39],bia:[7,26,35],bias:[39,40,41],binari:[2,40,41],bind:32,bird:11,boldsymbol:[33,34,35],book:[39,40,41],boost:11,bootstrap:[7,11,35],boston:1,breast:[2,40,41],brief:[32,35,36,37],bring:[13,39,40],build:[2,4,10,40,41],c:[26,27,32],calcul:33,can:[22,32,35,37,38,39],cancer:[0,2,8,10,12,36,40,41],cart:10,center:33,central:[14,24,29,35,36,37],chain:[13,39,40],challeng:36,chang:11,channel:32,chi:[1,32],choic:[39,40],choos:[2,40,41],cifar01:4,classic:12,classif:[0,2,10,11,27,36,40,41],classifi:[9,36],clip:[2,39,40],cluster:15,cnn:4,code:[1,2,3,6,10,12,13,14,15,22,27,32,33,34,35,36,37,38,39,40,41],collect:[2,4,40,41],come:36,commun:32,compact:[0,36,39,40],compar:[3,11,41],comparison:34,compet:[22,37,38],complet:[33,39,40],complex:[1,7,26,33],complic:[7,37,38,39],compon:12,comput:[10,37,38],computation:35,computerlab:32,con:10,concept:29,condit:[34,35,36,37],confid:35,conjug:[14,37],consider:[39,40],construct:[39,40,41],continu:38,contn:32,convex:[9,14,36,37],convolut:[4,13,38,39],correctli:[34,35],correl:[0,12,33,36],correspond:[36,37],cost:[0,2,11,33,34,35,36,37,39,40,41],count:39,cours:[24,31,32],covari:[6,12,29,33],cover:32,critic:27,cross:[0,7,26,35,36],custom:23,cython:32,d:[26,27],data:[0,1,2,4,7,8,10,12,16,17,23,24,26,29,32,33,34,36,39,40,41],dataset:[2,4,40,41],deadlin:[26,27,32],decai:[3,37,38,41],decis:[10,11],decomposit:[6,12,18,25,33],deep:[2,3,32,36,39,40,41],defin:[2,32,39,40,41],definit:[39,40],degre:[1,33],deliveri:[26,27],delta:35,dens:[1,32],deriv:[6,13,33,34,35,36,37,39,40],descent:[0,3,11,14,22,27,36,37,38,41],descript:26,design:33,detail:[4,32,41],develop:[2,40,41],diagon:12,differ:[9,27,37,38],different:22,differenti:[3,14,37,38,39,41],diffus:[3,41],dimension:[3,4,9,26,33,41],directli:[37,38],disadvantag:10,discret:29,discuss:[0,36],distribut:[6,29,34,35],distrubut:35,doe:[33,34,38,39],domain:29,dot:37,down:[2,39,40],dropout:[2,39,40],e:[26,27],each:[23,36],economi:33,electron:[26,27],element:[1,29,32,37,38],elimin:25,elu:[39,40,41],energi:32,ensembl:11,entri:[39,40],entropi:[0,10,36],environ:[1,16,32],equat:[1,3,13,32,33,34,36,37,39,40,41],error:[1,11,32,33,35],essenti:32,estim:[34,35],et:[22,38],etc:32,euler:[3,41],evalu:[2,27,39,40,41],exampl:[0,1,2,3,4,5,7,8,9,10,11,22,32,33,34,35,36,37,38,39,40,41],exercis:[0,1,7,16,17,18,19,20,21,22,23,32,33,39,41],expect:[19,29,34,35],expens:35,experi:29,explicit:[39,40],explod:[39,40],explor:[1,16,17,32],exponenti:[3,41],express:[0,18,19,33,36,37,39,40],extend:[0,36,37,39],extrapol:5,extrem:[11,32],ey:11,f:[26,27],fall:30,famili:[2,32,33,39,40,41],famou:25,fantast:33,featur:[10,25,33],feed:[2,13,38,39,40,41],find:[35,37],fine:[2,39,40],first:[5,13,27,32,33,34,36,37,39,40],fit:[1,11,32,34],fix:33,fold:[35,36],forc:4,forest:11,format:[26,27,32],forward:[2,3,13,38,39,40,41],fourier:4,frank:[7,26,33],freedom:[1,33],frequent:33,frequentist:[1,32],fridai:[],from:[0,6,11,13,22,27,33,34,35,36,37,38,39,40],full:[3,40,41],funtion:[39,40],further:[4,6,33],g:26,gan:5,gate:[38,39,40,41],gaussian:25,gd:[14,22,37,38],gener:[5,10,32,39],geometr:[12,36,37],get:[22,38,39],gini:10,glorot:[39,40],good:[1,32],goodfellow:[22,38],grade:[30,32],gradient:[0,2,3,11,14,22,27,36,37,38,39,40,41],grid:36,group:36,growth:[3,41],ha:24,hand:[39,40,41],handl:[25,32,33],happen:[34,35],hard:[],hessian:[33,36,37],hidden:[3,39,40,41],histogram:35,homework:[36,37],homogen:[39,40],hous:1,how:36,hyperbol:[38,39,40],hyperparamet:[2,39,40,41],hyperplan:9,i:[2,41],id3:10,idea:12,ideal:[36,37],ident:[34,35],identifi:35,ii:[32,41],iid:[34,35],iii:41,illustr:[34,38,39],implement:[2,38,40,41],implic:[6,33],improv:[2,37,40,41],includ:[0,14,22,36,37,38,39],increment:12,independ:[34,35],index:10,inform:30,ingredi:39,init:[40,41],input:[3,23,39,40,41],insight:[39,40],instal:[24,26,32],instructor:30,intercept:33,intermedi:39,interpret:[6,12,32,33,34,35,36,37],interv:35,introduc:[12,14,22,33,37,38],introduct:[1,7,24,25,26,27,32,38,39],invers:[6,25,34],invert:33,iter:[11,37],its:33,iv:41,jacobian:[33,41],jax:[14,22,37,38],job:[38,39],julia:32,jungl:11,k:[35,36,39,40],kera:[2,4,40,41],kernel:[9,12],l:[39,40],lab:[35,36,37,38,41],lagrangian:9,lambda:36,lasso:[6,7,26,33,34,35,36,38],last:[33,36,38,39,40],later:[6,33,40],layer:[2,3,4,13,23,39,40,41],layout:[39,40],learn:[1,2,3,12,14,15,16,17,22,24,26,27,32,33,34,35,36,37,38,39,40,41],least:[6,7,19,26,32,33,34],lectur:[32,33,34,35,36,37,38,39,40,41],level:11,librari:[24,32],likelihood:[0,8,34,35,36],limit:[2,14,29,35,36,37,38,39,40],linear:[1,9,14,25,27,32,33,34,36],link:[6,12,28,31,33,34,35],list:[39,40],literatur:[26,27],logist:[0,8,27,32,36,37,38,39,40],loop:[37,38],loss:[33,36,37],lu:25,machin:[1,9,14,24,26,27,32,36,37],made:[34,35],main:[29,32],make:[1,10,11,16,17,32,33],mani:[11,13],manipul:33,margin:[34,35],mass:32,materi:[26,28,32,33,34,35,36,39,40,41],math:[6,33],mathemat:[4,6,9,33,37,38,39,40,41],matric:[6,25,32,34],matrix:[0,2,6,12,13,25,32,33,34,36,37,38,40,41],matter:[1,32],max:33,maximum:[34,35,36],mean:[1,33,34,36],measur:[0,36],meet:[6,11,29,32,33],mercer:9,method:[7,10,11,14,22,26,32,35,36,37,38,40,41],midnight:[26,27],min:33,mini:[37,38],minibatch:[22,37,38],minim:[0,32,36,41],ml:32,mle:[34,35],mlp:13,mnist:[4,5],mode:39,model:[1,2,5,7,13,32,38,39,40,41],moment:[37,38],momentum:[14,22,37,38],mondai:[33,34,35,36,37,38,39,41],moon:[9,10],more:[0,4,7,22,25,26,32,33,34,35,36,37,38,39,40,41],multi:[38,39,40,41],multiclass:[40,41],multilay:[13,38,39],multipl:[2,4,23,40,41],multipli:9,multivari:39,need:[26,32],network:[0,2,3,4,5,8,13,27,32,36,38,39,40,41],neural:[2,3,4,5,8,13,27,32,38,39,40,41],neuron:[38,39],newton:[22,36,37,38],nn:[39,40,41],node:[39,40],noeds:40,noen:22,non:9,none:[37,38],normal:[1,2,35,39,40],notat:[13,38],note:[22,26,33,34,38],novemb:27,now:[2,10,14,22,34,35,37,38,40,41],nuclear:[1,32],nueral:[0,36],numba:32,number:[1,3,22,29,33,37,38,39,41],numer:[3,26,27,29,41],numpi:[22,25,32,37,38],object:[4,40,41],observ:[39,40],obtain:12,octob:[26,39,40,41],od:[3,41],off:[7,26],ol:[6,7,22,26,34,35,37,38],onc:23,one:[3,13,36,37,39,40,41],ones:[38,39,40],oper:[25,39],optim:[0,2,9,14,24,32,33,36,37,38,39,40,41],option:23,order:[14,22,37,38],ordinari:[6,7,19,26,32,33,34,41],organ:[1,32],orient:[40,41],oslo:31,other:[0,5,10,12,13,25,26,32,33,36,38,39,40,41],ouput:[39,40],our:[1,5,6,12,14,32,33,36,37,39,40],outcom:[24,32],output:[3,39,40,41],over:[39,40],overarch:[1,5,9,10,18,19,20,21,22,23,32,33,39],overview:[11,32,37,38],own:[1,11,12,16,17,27,32,33],packag:[25,32],panda:[32,33],paper:[26,39,40],parallel:39,paramet:[0,32,33,36,37,38,39,40],part:[14,24,26,27,33,36,37,39,40,41],partial:[3,41],pass:[2,40,41],pca:12,pdf:29,pencil:26,percepetron:[39,40],perceptron:[13,38,39,40,41],perform:[2,10,40,41],period:4,perspect:[2,39,40],plan:[33,34,35,36,37,38,39,41],plot:[35,36],point:[5,39],poisson:[3,41],polynomi:[4,34],popul:[3,41],popular:32,possibl:41,practic:[14,22,30,32,37,38],pre:[2,4,40,41],preambl:26,predict:[5,23],predictor:[0,36],preprocess:33,prerequisit:[4,24,32],princip:12,principl:4,pro:10,probabl:[6,29,34,35],problem:[2,3,14,22,32,33,34,36,37,38,39,40,41],procedur:[10,32],process:[2,4,23,40,41],product:37,program:[3,14,26,27,36,37,39,41],project:[7,26,27,32,36],prop:[14,37,38],propag:[2,13,39,40,41],properti:[6,29,33,36],python:[1,10,16,24,25,32],quick:9,r:32,random:[11,12,29,36],raphson:[36,37],rate:[22,37,38,40,41],read:[10,32,33,35,36,38,39,40,41],real:[7,23,26,32,39],recommend:[32,33,39,40,41],rectangular:34,recurr:[5,13,38,39],recurs:[37,38],reduc:[1,33,39],reduct:4,refer:26,reformul:[3,41],regress:[0,1,6,7,8,10,11,14,18,19,26,27,32,33,34,35,36,37,38,39],regular:[2,36,40,41],relat:33,relev:[31,33,36,38,39,40],relu:[2,39,40,41],remark:4,remind:[0,7,9,32,36,37,39,40,41],repeat:33,replac:[14,37,38],report:[26,27],repositori:35,repres:[],requir:[3,24,41],resampl:[7,26,35],rescal:[7,34],residu:33,resourc:[3,41],result:[33,34,39,40],revers:39,revisit:[14,36,37],rewrit:[32,33,35],rewritten:36,ridg:[1,6,7,18,19,26,33,34,35,36,37,38],rm:[14,37,38],rmsprop:[22,37,38],root:[39,40],rule:[13,39,40],s:[9,11,22,35,36,37,38],same:[14,22,35,37,38],sampl:12,scale:33,schedul:[28,32,40,41],schemat:10,scheme:[3,41],scienc:32,scikit:[1,2,12,16,17,32,33,34,35,36,37,40,41],search:36,second:[14,22,37,38],select:36,semest:30,sensit:[36,37],septemb:[34,35,36,37,38],seriou:39,session:[33,34,35,36,37,38,39,40,41],set:[1,3,4,10,13,16,32,33,36,39,40,41],setup:41,sgd:[14,37,38],should:[2,39,40,41],sigmoid:[39,40],similar:[14,22,37,38],simpl:[1,5,10,14,32,33,34,36,37,38,39,40],simpler:39,singl:[11,38,39],singular:[6,12,18,33],size:33,slightli:[37,38],smarter:39,soft:9,softmax:[2,40,41],softwar:[26,32],solut:41,solv:[3,34,36,37,41],solver:14,some:[14,25,33,36,37,39],specif:41,specifi:[3,41],split:[1,16,17,32,33],squar:[1,6,7,11,19,26,32,33,34],standard:[14,33,35,37],start:[22,38],state:[1,32],statist:[6,7,24,29,32,34,35],steepest:[11,14,36,37],step:[27,35,36,37,38],still:33,stochast:[14,22,27,29,37,38],stop:[37,38],strongli:32,studi:[0,36],subtract:33,suggest:[32,36,38],sum:[35,39,40],summari:[30,32,38],superposit:4,supervis:[2,39,40],support:9,svd:[6,33,34],syntax:37,systemat:4,t:[33,34],taken:[22,38],teach:[28,30],teacher:[30,32],technic:[34,41],techniqu:[7,12,26],technolog:24,tensorflow:[2,4,40,41],tent:32,term:[35,39,40],test:[1,2,16,17,27,32,33,34,40,41],text:32,textbook:[31,32],than:[36,37],theorem:[6,9,12,13,29,34,35,39],theori:29,thi:[18,19,20,21,22,23,32,39],think:33,three:[39,40],through:39,thursdai:[],time:[37,38],tip:[14,22,37,38],togeth:[13,39,40],tool:[26,32],top:[2,39,40],topic:32,toward:12,trade:[7,26],tradeoff:[7,35],train:[1,2,5,16,17,23,32,33,39,40,41],transform:4,tree:[10,11],trial:41,tuesdai:[34,38,39,40],tune:[2,39,40],two:[0,4,9,24,26,33,36,39,40],type:[3,5,13,32,38,39,41],uio:32,understand:35,univers:[13,31,39],unsupervis:15,unsupport:37,up:[1,3,10,13,16,22,32,33,35,36,38,39,40,41],updat:[39,40,41],us:[0,1,2,3,4,8,14,22,24,26,27,32,33,36,37,38,39,40,41],usag:[34,35,40,41],valid:[7,26,35,36],valu:[6,12,18,19,29,33,34,35,36],vanish:[39,40],vari:[22,37,38],variabl:[29,36,37],varianc:[7,26,34,35],variou:[1,16,27,32,35,36],vector:[9,13,25,32,33,38],veri:[],video:[36,37,38],view:[1,5,11,33,39],visual:[2,10,40,41],vs:4,wai:[10,22,35,38,39],warm:[22,38],wave:[3,41],we:[22,32,37,38,39,40,41],wednesdai:[34,38,39,40],week:[0,16,17,18,19,20,21,22,23,32,33,34,35,36,37,38,39,40,41],weekend:36,weekli:28,weight:[40,41],what:[1,32,33,34,35],when:[37,38],which:[2,22,37,38,39,40,41],why:[32,33,35,38,39,41],wisconsin:[0,8,36],word:39,wrap:[33,35],write:[5,12,26,27,34,40],x:[33,34],xgboost:11,xor:[38,39,40,41],yet:34,your:[1,11,16,17,27,32,33],yourself:36,z_j:[39,40]}}) \ No newline at end of file +Search.setIndex({docnames:["additionweek42","chapter1","chapter10","chapter11","chapter12","chapter13","chapter2","chapter3","chapter4","chapter5","chapter6","chapter7","chapter8","chapter9","chapteroptimization","clustering","exercisesweek34","exercisesweek35","exercisesweek36","exercisesweek37","exercisesweek38","exercisesweek39","exercisesweek41","exercisesweek42","intro","linalg","project1","project2","schedule","statistics","teachers","textbooks","week34","week35","week36","week37","week38","week39","week40","week41","week42","week43"],envversion:{"sphinx.domains.c":2,"sphinx.domains.changeset":1,"sphinx.domains.citation":1,"sphinx.domains.cpp":4,"sphinx.domains.index":1,"sphinx.domains.javascript":2,"sphinx.domains.math":2,"sphinx.domains.python":3,"sphinx.domains.rst":2,"sphinx.domains.std":2,"sphinx.ext.intersphinx":1,sphinx:56},filenames:["additionweek42.ipynb","chapter1.ipynb","chapter10.ipynb","chapter11.ipynb","chapter12.ipynb","chapter13.ipynb","chapter2.ipynb","chapter3.ipynb","chapter4.ipynb","chapter5.ipynb","chapter6.ipynb","chapter7.ipynb","chapter8.ipynb","chapter9.ipynb","chapteroptimization.ipynb","clustering.ipynb","exercisesweek34.ipynb","exercisesweek35.ipynb","exercisesweek36.ipynb","exercisesweek37.ipynb","exercisesweek38.ipynb","exercisesweek39.ipynb","exercisesweek41.ipynb","exercisesweek42.ipynb","intro.md","linalg.ipynb","project1.ipynb","project2.ipynb","schedule.md","statistics.ipynb","teachers.md","textbooks.md","week34.ipynb","week35.ipynb","week36.ipynb","week37.ipynb","week38.ipynb","week39.ipynb","week40.ipynb","week41.ipynb","week42.ipynb","week43.ipynb"],objects:{},objnames:{},objtypes:{},terms:{"0":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,22,23,25,26,27,29,30,32,33,34,35,36,37,38,39,41],"00":[1,2,6,7,10,12,32,33,34,38,39,40,41],"000":[2,4,10,40,41],"0000":10,"00000":[10,37],"000000":[6,12,33],"00000000e":[6,33,38,39],"00000262":38,"00001":37,"00001365":[],"00001572":[],"00001716":[],"00001976":38,"00002":37,"00002192":38,"00002276":[],"00002585e":[],"00002833":[],"0000293":[],"00003":37,"00003022":22,"00003645":[],"00003828":[],"00004164":38,"00005":37,"00005688":[],"00005799":[],"00006114":[],"00007":37,"00007706":[],"00007738":[],"00007964":[],"00008335":[],"00009":37,"00009004":[],"0001":[2,18,40,41],"00010067":37,"00010366":[],"0001079":[],"00011562":[],"00011577":[],"00012":37,"00012565":[],"00013464":[],"000137":[],"00013938":[39,40],"00013955":[],"00014166":[],"00014631":[],"00015394":[],"00015678":[],"00016062":[],"00016449":[],"00016535":[],"00016719":37,"00017038":[],"00017733":[],"0001775":[],"00018":37,"00018053":[],"00018212":[],"00018256":[],"00018651":[],"00018876":[],"00019258":[],"0001984":[],"00019998":6,"0002065":[],"00021147":[],"00021259":[],"00021270681344726395":[],"00021771":[39,40],"00021916":[],"000225":37,"00022969":[],"00023":37,"00023102":[],"00023414":[],"00023813":[],"00023978":[],"00024087":6,"00024415":22,"00024755":[],"00024929":37,"00025662":37,"00026549":[],"00026784":[],"000269":[],"00027188":[],"0002762":37,"00027746":37,"00028":37,"00028167":37,"00028359":38,"00028552":[],"00028632":38,"00028825":[],"00029012":6,"0002926":[],"00030094":[],"00030102":[],"00030132":[],"00030601":37,"00030827":[],"00031208":37,"00031272":38,"00031321":[],"00031535148309577417":[7,34],"0003153514830958095":[7,34],"00032566":38,"00033415":[],"00033562":[],"00033679":[],"00033904":37,"00034006":[39,40],"00034132":[],"00034209":[],"00034576":37,"00034688149480437765":[],"00034944":6,"00034953":[],"00035214":[],"00035483":[],"00035708":[],"00035912":38,"00036464":[],"00036566":[],"00036797":[],"0003701342170783489":[],"00037398":38,"00037564":37,"00037737":[],"00038309":37,"000387":[],"00039072":[],"0003919793592273513":[],"00039632":[],"00039821":[],"00040218":[],"00040516":[],"00041241":38,"00041619":37,"000417932":[],"00042072":[],"00042089":6,"00042093":37,"00042436":[],"00042444":37,"00042721":[],"00042767357088773335":10,"00042947":38,"00043701":[],"0004377":22,"0004388":[],"0004394":[],"00044":37,"00044937":[],"00045472":[],"00045923":[],"00046006":37,"00046111":37,"000464088":[],"00046616":[],"00046631":[],"00047025":37,"0004736":38,"00048158":[],"00048407":[],"00048474":[],"00048499":[],"0004932":38,"00049344":[],"00049636":[],"00049754":[],"00050627":[],"00050694":6,"00050952":[],"00051088":37,"00051916":[],"00052051":[],"00052102":37,"00052874":[],"00053001":[],"00053118":[39,40],"00053122":[],"00053635":[],"00053775":22,"00053797":[],"00053824":[],"00054387":38,"00055112":[],"00055695":[],"00056337":[],"00056414":[],"00056603":37,"00056637":38,"00057256":[],"00057362":[],"00057726":37,"00057772":[],"00058016":[7,34],"00059105":[],"00059247":22,"00059681":33,"00059703":[],"00059734":[],"00059993":[],"00060":37,"00060704":7,"00061058":6,"00061352":[],"00061711":[],"00061724":[],"00062457":38,"00062592":7,"00062668":[],"000626681519836":[],"00062713":37,"00063362":35,"00063866":35,"00063957":37,"00063965":[],"00064081":[],"00065041":38,"00065276":22,"00065307":[],"00065505":[],"00066091":[],"00066259":[],"00066293":[],"00066395":[],"00066668":[7,34],"00066798":[],"00067008":37,"00067192":[],"00067567":33,"00067793":22,"00068091":35,"00068734":[7,34],"00069":37,"00069123":7,"00069482":37,"00070819":[],"0007086":37,"00071003":[],"00071092":[],"00071724":38,"00071918":22,"00072527":[],"00072563":38,"00073144":[],"00073534":[],"00073541":6,"00073571":[],"0007373":[],"00074274":[],"00074692":[22,38],"00074747":[],"00075296":[],"00075590":35,"00076202":37,"00076495":[7,34],"00076521":[],"00076908":7,"00076983":37,"0007698473260556325":[7,34],"0007698473260556343":[7,34],"00077369":[],"00077389":[],"000783":[],"00078324":[],"00078509":37,"00079125":7,"00079237":22,"00079623":[],"00079904":35,"00079968":6,"00080345":33,"00080497":[],"00080683":[],"00081607":[],"00081649":[],"00081693":[],"00082235":[],"00082292":22,"00082366":38,"00082916":[],"0008297":[39,40],"0008297120772365539":[],"00083355":35,"0008415":[],"00084215":[],"00084244":[],"00084707":7,"00084831":[],"00085292":37,"00085774":38,"00085892":7,"00086675":[],"00086984":37,"00087102":[],"000873":22,"00087697":[7,34],"00088573":6,"00088848":[],"00089171":[],"00089193":35,"00090567":[],"00090614":[],"00090666":22,"00091307":[],"00091707":[],"00092668":[],"00092678":7,"00092898":35,"00093243":[],"00093356":[],"00093864":[],"00093994":[],"00094163":[],"00094353":[],"00094499":37,"00094587":38,"00094737":[],"0009532583857747255":[],"00095488":[],"00095624":[],"00096183":22,"00096314":6,"00096373":37,"00096829":[],"00097416":[],"00097604":[],"00097839":[],"0009806":[],"00098376":[],"00098501":38,"00099778":[],"0009983":[],"00099892":22,"00099936":[],"001":[2,3,9,14,18,23,34,36,37,40,41],"00100081":[],"00100202":[],"00100423":[],"00100511":[],"00100519":[7,34],"00100546":[],"00100563":[],"00100913":[],"00101555":[],"00102711":[],"00103024":[],"00103819":[],"00103867":[],"001047":37,"0010479245693919886":[7,35],"00105081":[7,34],"00105724":[],"0010597":22,"00106654":[],"00106677":6,"00106776":37,"00107405":[7,34],"00107739":[],"00108":37,"00108146":[],"0010827":[],"00108471":[],"00108622":38,"00108673":[],"00109354":[],"00110057":22,"00110396":[],"00110643":[],"001106660107757":[],"00110721":[],"00110867":[],"00111547":[],"00111564":[],"00111605":[],"00111756":[7,34],"00111809":[],"001123":[],"00113116":38,"00114401":[],"00114451":[],"00114855":[],"0011526":[7,34],"00115455":[],"00115508":[],"00115631":37,"00115999":6,"00116001":37,"00116737":[],"00116754":22,"00118299":[],"00118302":37,"00118584":7,"00118642":[],"00119688":35,"00119865":[],"00119876":[],"00120071":[],"00121256":22,"00121712":[],"00121925":[],"00122762":[],"00123795":[],"00123858":[],"0012394":[],"00124283":[],"00124429":[],"00124738":38,"00125057":[],"00125802":[],"00126053":37,"001268544905013411":[],"00127606":[],"00127645":[],"00128394":[],"00128453":[],"00128479":6,"00128523":37,"00128634":22,"00128778":[],"00129209":[],"00129599":[39,40],"00129899":38,"0013052":[],"00130648":[],"00130885":[],"00131072":37,"001323":[7,34],"00132494":[],"00133595":22,"00133638":[],"00134264":[],"00134424":[],"00135176":[],"00135237":[],"00135922":[],"00136125":[],"00137387":[],"00137458":[],"00137814":7,"0013839":[],"00138835":[],"00139195":[],"00139705":6,"00139918":[],"00140184":[],"00140253":[],"0014028":[],"00140846":35,"00141724":22,"00141727":37,"00142146":[],"00142396":37,"00142784":[],"00142849":[],"00142986":[],"00143013":[],"00143164":[],"00143246":38,"00143869":[],"0014423":[],"00144539":37,"00144622":[],"0014522":37,"0014593":[],"001468":[],"00146807":[],"0014719":22,"0014785":[],"00148216":[],"00148663":[],"00149173":38,"00149311":[7,34],"00149679":[],"00149956":[7,34],"00151204":[],"00151416":[],"00151426":[],"00152117":[7,34],"00152182":[],"00152471":[],"0015255":[],"00152602":[],"00154115":[],"00154733":6,"00154737":[],"00154853":35,"0015552":[],"00155695":[],"00156145":22,"00156374":7,"00157005":[],"00157346":33,"0015752":[],"00157766":37,"00157826":[],"00158":37,"00158427":[],"00158786":[],"00158859":[],"0016017":[],"00160895":37,"00161463":[],"0016202":33,"00162167":22,"00162812":[],"00162921":[],"00163159":[],"00163548":[],"00164501":38,"0016472":[],"00166119":[],"00166889":[],"00167058":[],"00167329":37,"00167406":22,"00168251":6,"00168427":38,"00168745":[],"00169":37,"00169056":[],"00169212":[],"00169299":[],"00170579":[],"00170809":[],"00170913":[],"00171047":[],"00171171":[],"00171307":38,"00171326":[],"00172034":22,"00172865":[],"00173182":[],"001732":[],"0017421":[],"00174276":[7,34],"00174459":[],"00174738":[],"00174743":[],"00174795":37,"00174884":[],"00175331":[7,34],"00176582":[],"00176663":[],"00177336":[],"00177485":[],"00178263":37,"00178669":22,"00178952":[],"00179288":[],"00179419":[],"001795":[],"00180673":37,"00180834":[],"00181885":[],"00182423":[],"0018310257999764163":[],"00184571":[],"00185976":[],"00186052":[],"00186347":6,"00187032":[],"00187415":[],"00187791":[],"00187888":[],"0018798":[],"001880":6,"00188909":38,"00189016":[],"00189522":[],"0018954":22,"00191076":[],"00192114":[],"00192154":[],"00192165":[],"00192879":[],"00193326":[],"00193661":37,"00195147":[],"0019575":[],"00196372":[],"00196462":[],"00196725":38,"00196801":[],"0019685":22,"00196852":[],"00197431":[],"00197505":37,"00198129":[],"00199645":[],"00200":9,"00200982":[],"00202433":[39,40],"00202532":[],"00202624":6,"00202756":[7,34],"00202904":[],"00203961":[],"0020489":[],"00206234":[],"00208188":37,"0020841":[],"00208517":[],"00208828":22,"0020899":22,"00209403":[],"00210569":33,"00211332":[],"00211418":[],"00211555":[],"00211924":[],"00212766":[],"00213716":33,"00213888":[],"00214288":[],"0021452":33,"00214559":37,"00216162":[],"00216458":[],"00216881":22,"00216938":38,"00217143":[],"0021741":[],"00217499":[7,34],"00217968":[],"00218058":[],"00218381":[],"00218481":[],"002186":[],"00218823":37,"00219195":35,"00220013":[],"00220472":[],"00223857":[],"00224413":6,"00224491":[],"00224638":[],"0022476":[],"00225136":[],"00225842":37,"00225914":38,"0022626":[],"00226433":[],"00228042":[],"00228740":7,"00229111":[],"00230059":[],"00230078":22,"00230308":[],"00230805":[],"00231293":[],"00231412":[],"00232963":[],"0023327":[],"0023548":[7,34],"00236284":[],"00236918":[],"00237699":37,"002381316302584885":[7,34],"0023813163025848865":[7,34],"00238951":22,"0023941":[],"00240001":[],"00242441":37,"0024271":[],"00242856":[],"00242954":35,"00242967":[],"00242999":7,"00243186":[7,34],"00243835":[],"0024401":6,"00244459":[],"002446":[],"00244871":[],"00248232":[],"00248779":[],"00249126":38,"00249435":[7,34],"00250787":[],"0025089":[],"00251359":[],"00252192":[],"00253034":[],"0025349":22,"00253932":[],"00254671":[],"00254966":[],"00254973":[],"00256001":[],"00256537":[],"00256622":[],"00256688":[],"0025682":[],"00259434":38,"00259997":[],"00260527":[],"00260839":[],"0026126":[],"00263266":22,"00263296":37,"00263638":[],"00263755":[],"00263812":[],"00264":37,"00264181":[],"00264795":[],"00266858":[],"00267419":[],"00268602":37,"00270073":[],"00270197":[],"00270244":6,"00270262":[],"00272337":[],"00274212":[],"002743":[],"00274676":[],"00274989":[7,34],"00275212":[],"00275768":[],"00276424":[],"00279284":22,"00279722":[],"00282226":[],"00282302":37,"00282498":[],"00282501":[],"00283324":33,"00284402":[],"00284525":[],"00284866":[],"00284871":[],"00285017":[],"00285068":[],"00286091":38,"00286836":[],"00288198":33,"00288656":[],"00288666":[],"00289724":[7,34],"00290055":22,"00290316":[],"00290613":[],"00291535":37,"0029276":[],"00292882":[],"00293201":[],"00293758":[],"00293838":6,"00295372":[],"00295713":[],"00297571":37,"00297928":38,"00298783":[],"00299754":[],"00300342":[],"00300479":[],"00300908":[],"0030094":[],"00302509":[],"00302547":[],"00303686":33,"00307265":[],"00307704":22,"00309418":[],"00309594":[],"00309834":[],"00310113":[],"00311085":[],"00312361":[7,34],"00315593":7,"00316149":[],"003162":[39,40],"00316311":[],"00316452":[],"00318725":[],"00319571":22,"00319695":[],"00319887":[],"00320702":[],"0032153180657605116":[7,35],"00322456":37,"00323332":[7,34],"0032542":6,"00325450":35,"00325635":[],"00326986":[],"00327104":[],"00327555":[],"00327797":[],"00328539":38,"0032882":[],"00329616":37,"00329927":[],"00330046":[],"003301":[7,34],"00330245":[],"00331168":[],"00331927":[],"0033213":[],"00334002":[],"00334047":[],"00334155":[],"00334797":[],"00335101":[],"00335174":[],"00337016":[],"00339014":22,"00339369":[],"0033955154592040936":[7,35],"00340452":[],"00342132":38,"00342565":[],"0034318":37,"00344192":[],"00344887":[],"00347534":[],"00348246":[],"00348281":[],"00348553":[],"0034881":[],"0035086":[],"00351039":[],"00351649":[],"00352046":[],"00352089":22,"00352878":37,"00353823":6,"00353938":[],"00354483":[],"00355323":[],"00355614":37,"00356356":[],"00356549":[],"00359628":[],"00361046":[],"0036237":[7,34],"0036367":[7,34],"00363823":[],"00364159":[],"00364235":[],"00364967":37,"00365819":[],"00366866":[],"00368369":[],"00369201":[],"00369236":[],"00369758":[7,34],"00369852":[],"00370353":[],"00371435":[],"00371605":[],"00371656":[],"0037314":[],"00373511":22,"00377231":[],"00377286":38,"00379682":[],"0038022":[],"00380258":[],"00381349":[],"00382137":[],"0038287":[],"00383048":[],"0038335":[7,34],"00385557":[],"00387671":[],"00387917":22,"00388071":[],"00389038":37,"00389382":[],"00389577":[],"003909404072811226":[7,35],"00391839":6,"00392402":[],"0039259":[],"00392639":[],"00392691":[],"00392853":[],"00392895":38,"0039455":[],"00396815":[],"00397085":[],"00398429":[],"00398499":[],"00399354":[],"00399521":[],"0039987":[7,34],"00399972":[],"004":[6,34,35],"00403677":37,"00404166":[],"00405576":[],"00405581":[],"00406466":[],"00406952":[],"0040792":[],"00408":37,"004091940707753939":[7,35],"00410183":[],"00410387":[7,34],"00410478":7,"00410974":[],"004113634617443131":[7,33,34],"00411363461744314":[7,33,34],"004113634617443147":[7,33,34],"0041148":[],"00411514":22,"00412":37,"0041283":[],"00413064":[],"00413253":[],"00415405":[],"00415497":[],"00415598634586135":[7,35],"00416103":37,"00416704":[],"00418347":[],"00418664":[],"00420072":35,"00420536":[],"00422259":[],"00424491":[],"00424909":[],"00424967":[7,34],"00426027":6,"00426902":[],"0042739":22,"00429456":[],"0042992":37,"00431893":[],"00432129":[],"00432337":[],"00432864":[],"00433264":38,"00433417":12,"00434043":[],"00434704":[],"0043587":[],"00437846":[],"00438318":[],"00439674":[],"00440346":[7,34],"00441065":[],"00441097":37,"00442091":[],"004423":[],"00443629":33,"00443743":[7,34],"00444806":[],"0044505":37,"00445186":37,"0044547":[],"00445655":12,"00451186":38,"0045162":[],"00453377":22,"00455966":[],"00456562":[],"00457441":[],"00457599":[],"004578":33,"004579219539673834":[7,35],"00458083":33,"00458631":[],"00458878":[7,34],"0045909":[],"00459357":[],"00459568":[],"00460917":[],"004610275230656212":[7,35],"00461571":[],"00462287":[7,34],"00462662":[],"00463124":[],"00463246":[],"00464677":[],"00465099":35,"0046544":[],"0046741":[],"00468914":[],"00470102":[],"00470879":22,"00471617":[],"00471782":6,"00472199":7,"00472276":[],"00472512":[7,34],"00473136":[],"00473744":33,"00474898":[],"00476012":37,"00478417":[],"0047956":[],"00479764":[],"00480881":[],"00481334":[],"00483798":[],"00483858":[],"00485232":33,"00487029":37,"00493903":[39,40],"0049544":[7,34],"00497544":38,"00497706":[],"00497837":34,"00498004":[],"00498065":[],"00498243":[],"00498263":[],"00499474":22,"004999999999999987":[],"004999999999999989":1,"004999999999999991":[],"004999999999999996":[],"004999999999999998":[],"005":[],"005000000000000001":[],"00500716":[],"00502886":[],"00503732":[],"00503932":[],"0050554":[],"00505872":[],"00507014":[],"00507531":[],"00509128":37,"00510839":[],"00511073":[],"00512149":33,"00512599":[],"00512927":6,"00513051":[],"00514817":[],"00515165":[],"00515181":[],"00515889":[],"00516067":[],"00517114":[7,34],"00517673":[],"00518116":38,"0051879":22,"00519717":[],"00520618":[],"00520752":33,"00520911":37,"00521014":[],"00522055":[],"00523789":[],"00525279":[],"00525303":[],"00525734":[],"00526348":[7,34],"00527578":[],"00528252":[],"0053018":[7,34],"00531268":[],"00532167":[],"00532314":[],"00534541":[],"00536751":[],"00537955":[],"00539563":[],"00541941":[],"00544547":37,"00546503":[],"00546963":[],"00547113":[],"00547332":[],"00550187":22,"00551372":37,"00551695":[],"00551716":[],"00552462":[],"00554202":[],"00554552":[7,34],"00554707":[],"00556826":[7,34],"0055687":[],"00556946":[],"0055715":37,"00557893":[],"00558145":[],"00559722":[],"0055997":[],"00560098":[],"00560316":[],"00561212":[],"00561262":[],"00562142":[],"005643":[],"00564344":[],"00564365":37,"00564874":[],"00566707":[],"00567869":[],"0056799":6,"00568091":[],"00568352":[],"00569512":[],"00570059":[],"00570321":[],"0057065":[],"005708":[],"00570826":[],"00571352":38,"00571565":22,"00571608":[],"00571884":[],"00572613":[],"00573236":[],"00573661":[],"00573681":33,"00573727":[],"00574735":[],"00575681":[],"00576997":[],"00577229":[],"00578449":[],"00578809":[],"00579953":[7,34],"00579972":[],"00582034":[],"0058243":37,"00582949":[],"00585143":[],"00586355":[],"00588657":[7,34],"00588836":[],"00590379":[],"00590456":[],"0059267":[],"00593362":[],"00594949":38,"0059591":37,"00597698":[],"00598464":[],"00598510e":[],"00600791":[],"00602064":[],"0060439":[],"00605833":22,"00606112":[],"00606382":[],"00607072":[],"00607165":[],"00607783":7,"00609802848297877":[],"0061078":[],"0061084":[],"006150":12,"00615047":12,"006162":[7,34],"00617221":[],"00617261":[],"00617499":6,"0062202":[],"00622617":[],"00622949":37,"00623535":[],"00625051":[],"00625222":[],"00625572":[],"00625672":[],"00627115":[],"00627746":[],"0062968":22,"00630331":[7,34],"00630802":[],"00631759":[],"00632049":[],"00632251":[],"00633289":[],"00634299":[],"00634575":[],"00635016":[],"006352536528464608":[],"00635621":[],"00636951":[],"00637188":[],"00637367":37,"0063878":[],"00641423":[],"00642221":[7,34],"00642737":[],"0064349":[],"00644":37,"0064472":34,"00645121":[],"00647029":[],"00652931":[],"00653725":[],"00654058":[],"00654406":[],"00654835":[],"00654864":[],"00656079":38,"00657032":[],"00657601":[],"00659604":[],"00660427":[7,34],"00666287":37,"0066645":22,"00667611":[],"00669649":[],"00672471":[],"00672505":[],"00672607":[7,34],"00672654":[],"00673407":[7,34],"00673693":33,"00674158":[],"00676256":[],"00676387":[7,34],"00677215":[],"00679013":[],"00680029":[],"0068011":[7,34],"00681708":37,"00682686":[],"00683102":38,"00683748":6,"00683875021199284":[],"00683964":[7,34],"00685287":[],"00687309":[],"00688012":[],"00688182":[],"00689214":37,"00691012":[],"00691271":[],"00692345":[],"00693334":[],"00693613":22,"00694315":[],"0069435":[],"00695371":[],"00696127":[],"0069668":[],"00697033":[],"00698568":[],"00700361":[],"007024126888938232":[7,35],"00702562":[],"00702721":[],"00702885":[],"00703651":[],"00709577":[],"007096":[],"00710152":[],"0071129":[],"00711457":[],"00712219":[],"0071264":37,"00715512":[],"00717018":[],"00719176":[7,34],"00721287":[],"00722437":[],"00723172":[],"00724303":[],"00725498":[],"00727646693":[1,32],"00729134":37,"00731149":22,"00731392e":[],"007315":33,"00732483":[],"00732665":[],"00733216":32,"00735068":[],"0073544":[],"00736737":[],"007370297974992432":[],"00737863":[],"00738299":[],"00740723":[],"00740838":[],"0074331":6,"00745015":[],"00746317":[],"00747773":[],"00748862":[],"00752703":[],"00753078":[],"00753289":38,"00754823":[],"007557567939261264":32,"00755917":[],"00759119":7,"00759427":[],"00761224":[],"00761546":33,"00761755":[],"00762217":37,"0076268":23,"00763764":22,"00764333":[],"007658477904313023":[],"0076989":[],"00770419":[],"00771475":[39,40],"0077313":[],"00774376":[],"00774948":[],"00777619":33,"00778316":[],"00779859":37,"00781297":[],"00781659":[],"00781918":35,"00782481":[],"00784099":38,"00784393":[7,34],"00784531":[],"00786896":[],"00789262":[],"00791263":[],"00791445":[],"00792032":[],"00796116":22,"0079634":[],"00798909":[],"00799475":[],"00800011":[],"00801346":[],"00802302":[],"00802792":[],"00803064":[7,34],"008043811612683473":[],"00805074":35,"00806447":[],"00809364":[],"00810345":[],"00812487":[],"008128643056242772":33,"00813033":[],"00813803":7,"00815244":37,"0081568":[],"00816675":[],"00817631":[7,34],"00818251":[],"00819774":[],"00819875":[],"00821525":33,"00823002":6,"008230739628188457":[],"00827534":[],"00827728":[7,34],"00828121":[],"00829346e":[],"00831018":[7,34],"00834112":37,"00834567":[7,34],"00836649":[],"00838255":[],"00838555":[],"00840184":22,"00842216":[],"00842507":[],"00847367":[],"00848586":22,"00848739":[],"00848904":[7,34],"0085274606925055":[],"00853302":[],"0085392":[],"0085548":[],"008560581831215528":[],"0085639":[],"008574628106492125":[],"00857702":[],"00859693":[],"00860892":[],"00861518":37,"00864664":38,"00865335":[],"0086649156":[1,32],"00866766":[],"0086714":[],"00867406":[],"008675369724975977":6,"0086885":[],"00869259":[],"00870208":[],"00871267":[],"00871959":37,"00871967":[],"00872006":[],"00872869":[],"00873267":[],"00875286":[],"008753288788081405":[],"00877737":[],"00877816":[],"00877892":[],"00878815":[],"00881749":[],"00881939":[],"00885858":[],"00886253":[],"00887262":[],"00889027":[],"00889475":[],"00891351":[],"00891702":[],"0089204":[],"0089214":37,"00893112":[],"00893649":33,"00894639":6,"00896253":33,"00898223":[],"008994":38,"0089987":33,"00900007":[],"00901661":[],"00903778":[],"00904514":22,"00905423":[7,34],"0090724":[],"00908537":[],"009134699065945493":[],"00914784":33,"009163470508352218":6,"009164545680330616":[7,35],"00917248":[7,34],"00917694":[],"00917881":[],"00919984":[],"00920294":[],"009211602191395454":[],"00921748":22,"00922905":[],"00923082":[],"00923474":[],"00925406":37,"009277782622954527":33,"00930146":[],"00930457":33,"00931621":[],"00931882":[],"0093262":37,"009338082195270294":[],"00934327e":33,"00934499":[7,34],"00935742":[],"00938275":[],"00939407":[],"009396472959497925":[],"009397356386856284":[],"00939922":[],"00940611":[],"00942574":[],"00943026":[],"00943959":[],"00951885":[],"00952138":33,"00954206":37,"00955125":[],"0095585":[],"00956827":[],"00958615":[],"00959357":32,"0096208":[7,34],"00963683":[],"00964128":22,"00964129":[],"00965205":[],"00965904":[],"00966192":[],"00966668":[],"0096667":[],"00968268":[],"00969623":[],"00970059":[],"00970896":[],"00971344":[],"00972948":[],"00974758":[],"00978267":[],"00979031":[],"00979357":[],"00982052":[],"0098443":[],"009880124918446542":[],"00988081":[],"00988356":[],"00988992":[],"00989118":[],"00989834e":[],"00990475":6,"00991812":38,"00992331":[7,34],"00995699":32,"00996754":[7,34],"00996762":[],"00997215":[],"00997502":37,"00997697":[],"009998596494598343":[],"01":[0,1,2,3,6,7,10,12,14,18,22,31,32,33,34,37,38,39,40,41],"010018312644137363":[7,35],"01003519":[],"01003651":22,"01006":37,"01006224":[],"01006335":[],"01006984":[],"0100706":[7,34],"01008116":[],"01009321":[],"01009625":[],"01013106":[],"01013849":[],"01014331":[],"010148621093080332":[],"01016442":[],"01016493":[],"01016961":[],"010175219431920396":[],"01018089":[],"01018465":33,"01019212":[],"01020588":37,"01024875":[],"01027183":[],"01027671":22,"01029828":38,"01031":37,"01032298":[],"01032533":[],"010331721306655127":[7,35],"010348289064969316":[],"01035006":[],"01039355":[],"01039879":[],"01039977":[],"010405448150048726":[],"01044475":[],"01045576":[],"01048771":[],"010516485576645508":[7,35],"01055384":[],"01055538":[],"0105667":[],"0105836":[],"01059242":[],"01060492":[],"010612904886352397":[],"010638334964957053":[],"01066519":[7,34],"01066897":37,"01067432":[],"01069799":22,"01070103":[],"01070717":[],"01071876":[],"01074379":[],"01074493":33,"0107497":[],"01075018":[],"01076611":6,"01076897":37,"0107744":[],"01078008":[],"01080465e":[],"01084359":[],"01085261":[],"01085571":[],"01086401":[],"01086733":[],"01087815":[],"01089212":[],"01091589":37,"01093529":[],"01093553":[],"01094916":[],"01095403":22,"01097287":[],"01098247":[],"01099776":[],"0110":29,"01102311":[],"01102737":[],"01105668":[],"01105813":33,"0110602":33,"01107333":[],"011076219011379514":[7,35],"01109809":[],"01109955":[],"01112157":[],"0111346":[],"01115456":[],"011156605304609659":[],"01116732":[],"01118392":[],"01119357":[],"01119504":[],"01119738":[],"01119981":[],"011208613520466846":[],"011225":[],"01123792e":[],"01127138":[],"01127942":[],"01129731":[],"0113104":[7,34],"01131771":[],"01131825":[],"01133846":[],"01133903":[],"01135511":[],"01135642":38,"01140194":[],"01140307":22,"01141119":37,"01142354":[],"01146857":[],"01149099":[],"01149386":33,"01149735":[],"01150437":[],"01151556":[],"01153142":[],"01153294477941533":33,"0115357":[],"011594819155615":[],"01161645":[],"01163942":[],"0116753":37,"01167598":22,"01169016":[],"01169528":[],"01171276":[],"01172319":[],"01173863":38,"01174937":[],"01176967":[],"01178583":[],"01178857":[],"01179128":[],"01179415":[],"01179792":[7,34],"01180564":[],"01180692":[],"01181458":[],"01181473":[],"011818646101922625":[],"011886":[],"01191824":6,"01192135":[],"01193263":[],"01194492":[],"01195336":[],"01196506":[],"01199378":[],"01199434":[],"01199741":[],"01203439":[],"01205039":[39,40],"01206221":[],"012073649446193166":[7,35],"01211133":[],"0121289":[],"01214076":[],"01214862":[],"01215461":22,"01216075":[],"01216231":[],"01216264":[],"0121639":[],"0121766":[],"01218403":[],"01218736":[],"01219292":[7,34],"01220301":[],"01220505":37,"01222822":33,"01222848":[],"01222875":[],"01223198":[7,34],"01223754":[],"01229008":[],"01229102":[],"01229692":[],"01231917":[7,34],"01232366":[],"0123263":[],"01234":[],"01234261":[],"01235079":[],"01235257":[],"01241284":[],"012423940191689783":[],"01244297":[],"01244551":22,"01247909":[],"01248754":37,"01249445":[],"01249968":[],"01250964":[],"01257912":[],"01258423":[],"01260207":[],"01261048":[],"01263474":33,"01265755":33,"012658":33,"01266855":[],"01266999":[],"01267007":34,"0126711":[],"01268737":[],"012687664954397":29,"01272102":[],"01273648":[],"01276369":[],"01278044":[],"01280793":[],"01282563":[],"01286909":[],"01288787":[],"01289266":[],"01290947":[7,34],"0129191":[],"01294476":38,"01294799":[],"01294954":[],"01295356":6,"01295569":22,"01299711":[],"01300036":[],"01305415":37,"013121574015585152":[7,35],"013121574015585155":[7,35],"01312585":[],"01315875":[],"01316233":[],"01316638":[],"01317824":33,"01318643":[7,34],"0132081":[],"01321707":[],"01322347":[],"01322434":[],"01322589":38,"01323341":[],"01323523":[],"01326577":22,"01327148e":[],"01327475":[],"01328658":[],"01328972":[],"01329403":[],"01332832":[],"01333573":[],"01335628":37,"0134113":[],"01343464":[],"01343585":[],"01346122":37,"01346635":[],"01347916":[7,34],"01348565":[7,34],"01349244":[],"01353569":[],"01353638":[],"01355882":[],"01357366":[],"01359774":[],"01361517":[],"01366555":[],"01367553":[7,34],"01369614":[],"01370129":[],"0137297":37,"01373315":[],"01373484":[],"01376216":[],"01377529":[],"01377545":[],"01380833":[],"01380957":22,"01383689":[],"01384583":[],"013846":[],"01386236":[],"01393427":[],"01393496":[],"01395192":[],"01396231":37,"01397146":[7,34],"01399101":[],"01399457":[],"01405935":[7,34],"01406777":[],"01408975":[],"01409478":[],"01414008":22,"01415781":[],"01416528":[7,34],"01418766":[],"01419341":[],"01420773":[],"0142117":[],"01425384":[],"01427864":[],"01428546":37,"01429853":[],"01433809":6,"01438804":[],"014394967608286841":[],"0144045":[],"01440748":[],"01441661":[],"01442378":[],"01442817":33,"01443283":[],"014436800088904942":[7,35],"01444013":[],"01444966":38,"0144623":[],"01449782":[7,34],"01454469":[],"01455301":33,"01455304":[],"014565":[],"01456865":[],"01457889":[],"01458337":[7,34],"01458483":38,"014586":33,"01458611":33,"01459912":[],"0146081":[7,34],"01462254":[],"01463049":[7,34],"01463112":[],"01464686":[],"01471973":22,"01473588":[],"01473759":[],"01476927":[],"01478037":[],"01479672":[],"01480212":[],"01480256":[],"01480927":[],"01493365":37,"014934":37,"01494559":[],"01498697":[],"01499192":[],"01499389":[],"01499589":[],"01504759":[],"01506354":[],"01507202":22,"01507238889517706":[7,34],"01507238889517717":[7,34],"01509793":[],"015101938515449663":29,"01511937":37,"01512884":[],"01515818":[],"01520159":[],"01522108":[],"01523958":[],"01525387":33,"01527618":33,"01527928":37,"01530321":[],"01531845":[7,34],"01532072":33,"01532738":[],"01534161":[],"01543076":[],"01546126":[],"01548486":[],"01549377":[7,34],"01549917":[],"01551724":[],"01551775":[],"015518":[],"01551815":[],"01552039":[],"01552874":[],"01554658":[],"01558197":6,"01562424":25,"01563358":33,"01566026":[],"01566493":[],"01567258":[],"01568403":[],"01568987":22,"01572":37,"01573062":[],"01573768":[],"01574065":[],"01574456":[],"01575266":[],"01578667":38,"01584129":[],"01593089":[],"01593435":38,"01594452":35,"01597256":37,"01597958":[],"01600363":[],"01604235":[],"01604832":[],"01606538":22,"01606644":[],"01607157":[],"01612308":[],"01612637":[],"01613409":[],"01615836":[],"01616657":[],"01620955":[],"01622922":[],"01624447":[],"01625794":[],"01628578269596628":[7,35],"01630755":[],"01633913":[7,34],"01634224":37,"01635525":[],"01639344":[],"01639489":[],"01640891":7,"01641044":[],"01643795":[],"01643952":33,"01647074":[],"0164974":[],"01654574":[],"01655001":33,"01655318":[7,34],"01655384":[],"01656815":[],"016587414993043573":[7,35],"01671659":33,"01672395":22,"01674956":[],"01677178":[],"01678891":[],"01679297":[],"01681173":[],"01681612":[],"0168171":[],"01682317":[],"01682652":37,"01684463":[],"01685118":[],"01686439":[],"01691985":[7,34],"0169643":6,"01698477":[],"01699169":[],"01700746":[],"01700897":[],"0170097":[],"01701429":[],"01705695":32,"01705781":[],"01706166":[],"01708375":37,"01708781":[7,34],"01708852":[7,34],"01709886":33,"01712421":22,"01713187":[],"01713366":[7,34],"01717238":[],"01717549":[],"01717775":[],"01721277":[],"01722503":[],"01723872":[],"01724499":6,"01724739":38,"0173119":[],"017355848195593347":[7,35],"01735584819559336":[7,35],"01735999":[],"01736099":[],"01737595":[],"01737634":[],"01740381":[],"01740873":38,"01743124":[],"01746176":33,"01747915":37,"01748034":[],"01752868":[],"01756114":[],"01759442":[],"01759974":[],"01760144":[],"01761069":[],"017665":6,"01772174":[],"01772779":[],"0177579":[],"01779091e":[],"01782618":22,"01783096":[],"01783414e":33,"0178462":[],"01784845":[],"01787801":[],"01789589":[],"01793261":[],"01793972":[],"01795684":[],"01797196":[],"01797685":[],"0179826":[],"018":[],"01804672":[],"0180908":[],"01811279":[],"01811373":[],"01813363":[],"01814351":[],"01814683":[],"01817048":[],"01817739":[],"0182116":[],"018232":33,"01825282":22,"01826645":38,"01826811e":[],"01827225":37,"01827302":[],"01830039":[],"01831050e":[7,34],"01831190e":[7,34],"018365206985554446":[],"01838531":[],"01839352":[],"01844584":[],"01844822":[],"01846605":[],"01853796":[],"01854489":[],"0186037":[],"01863203":[],"01866537":[7,34],"01868119":[],"01869515":37,"01870931":[],"01871612":[],"01872002":[],"01873344":12,"01873823":33,"01873869":6,"01874605":[],"01877081":6,"018771":6,"01879191":[],"01881237":[],"01882218":[],"01882264":[39,40],"01884327":38,"01886197":[],"01893445":[],"01897462":[],"01898383":[],"01898855":[7,34],"01900106":22,"01901953":38,"01905384":[],"01905883":[7,34],"01908329":33,"01908936":[7,34],"01909296":[],"01909929":[],"01915337":[],"01916065":[],"01918426":[],"01919791":[],"01922269":[],"01922564":[],"01924105":[],"01925934":[],"01930676":[],"0193075":[],"0193386":[],"01944073":[],"01945582":22,"01947793":[],"01950331":33,"01952679":[],"0195302":[],"01954342":37,"01962001":[],"01963611":[7,34],"01966173":[],"01968447":[],"01969093":[],"01969145":[7,34],"01974315":[],"01975006":[],"01975416527185249":[7,35],"01975755":[],"01975848":[7,34],"01979808":[],"01981665":[],"01991068":[],"01991819":[],"01999575":37,"02":[0,1,5,7,8,13,32,34,36,38,39,40],"0200041":[],"02010178":[],"02014577":[],"02015755":33,"02016185":[],"02019986":[],"02021288":[],"02024962":[7,34],"02025337":22,"02025574":[],"02026756":[],"02028151":33,"02028936":[],"02031789":[],"02031987":[],"02032056":[],"02034593":[],"02034655":[],"02036067":33,"02037155":[],"02038063":33,"02044223":[],"02045223":[],"02045753":[],"02054837e":[7,34],"02056537":[],"02058408":[],"02058681":38,"0206033":[],"02060364":[],"02068067":[7,34],"02070909":[],"02072876":[],"02073509":6,"0207381":22,"02075746":[],"02077431":34,"02077938":38,"02079774":33,"02079913":[],"02083309":[],"02084474":[],"02084969":[],"02087411":[],"02088307":[],"02090304":37,"02091662":[],"02092464":[],"02098261":[7,34],"02101168":[],"02102403":[],"02103315":37,"02104639":[],"02108599":[],"02109876":33,"0211042":33,"0211429":[],"02117843":[],"02118344":[],"02123176":[7,34],"02124568":[],"02125608":[],"02126205":[],"02132557":[],"02134409":[],"02137093":[],"02137508":[],"02138683":37,"02141849":[],"02143946e":[],"02146636":[],"02148021":37,"02150453":[],"02151521":[],"021544346900318832":34,"02156196":[],"02156565":[],"02157768":38,"02158822":22,"021592704588025022":[7,35],"021592704588025025":[7,35],"0216304":[],"02163476":[],"02164667":[],"02169399":[],"02171224":[],"02175561":[],"02176052":[],"0217610577940561":[],"02177725":[],"02181163":[],"02183817":[],"0218856":[],"02189606":[],"02189863":[],"02189994":[],"02193078":[],"02196086":[],"02198702e":[7,34],"02198703e":[7,34],"02199232":[],"022":[],"02203851":[],"02204591":[],"02206393":33,"02207054":[],"02208512":[7,34],"0221049":22,"02214409916811925":[],"02218045":[],"02218076":[],"02225493":[],"022255":[],"02228115":[7,34],"02229529":[7,34],"02232526":[],"02234283":33,"02235723":37,"022379426843681074":[],"02239876":[],"02241952":[],"02249167":38,"022498":[],"02251592":[],"02252765":6,"02253659":[],"02257687":[],"02260949":[],"02260989":[],"02265005":[],"02265404":[],"02265998":[],"02270207":38,"02280634":[],"02281004":[],"0228324":[],"02286267":[],"02287468":37,"02288069":[],"02290895":[],"02296298":37,"022999498260366312":[7,35],"02301105":22,"02312079":[],"0231353":[],"02319624":[],"0232":[],"0232101":[],"02322297":[],"02322594":[],"02322786":[],"02323402":[],"02325785":29,"02327527":[],"023295599127611474":[],"02330799":[],"02330901":[],"02333688":[],"02334754":[],"02336409":33,"02338085":[],"02342235":[],"02345858":[],"02348765":[7,34],"02349274":[],"02352005":[],"02352663":[],"02352843":[],"02354386":[],"02355756":[],"02356004":[],"02356178":22,"02358547":[],"02359571":[],"02361665":[],"02363304":[],"02365049":[7,34],"02369044":[],"02374496":[],"02375066":33,"02386932":[],"02387182":[],"0239126":37,"02391906":[],"02392067":[],"0239521":[],"024":[],"024004":[],"024011020121022356":[],"02404706":[],"02409835":[],"0241039":[],"02413786":[],"02424794":35,"02429596":[],"02431532":[],"02432309":[],"02432632":[],"02432881":[],"02436346":[],"02439287":[],"02441422":[],"02444266":[],"02444569":[],"02446604":37,"02447063e":[],"02448011":33,"0244965108017645":[],"02452765":22,"02452792":[],"02454051":[],"0245528":[7,34],"02456":37,"024562":[],"02456235":[],"0245728":38,"02458099":[],"02459084":37,"02472936":[],"02474828":[],"02477293":[],"02478142":[],"024792624800382218":[],"02480266":38,"02483712":33,"02490769":[],"02492265":6,"02496085":37,"02498282":[],"02498832":[7,34],"02499437":[],"02500488":[],"02502325":[],"02503392":[],"02503753":[7,34],"02504521":[],"02506901":[],"02511468":22,"02511518":7,"02519009":[],"02521917":[],"02522069":7,"02526293":[],"02527621":[],"02529207":[],"02530745":[],"02531253":[],"0253208":[],"02533443":[],"02535112":[],"02535694":[],"02535973":[],"02540212":[],"02544154":[],"02545":37,"02546675":35,"0255141":[],"02552966":[],"02553865":[],"02553907":[],"02557617":37,"02561386":[],"02564828":[],"02566356":[],"025709":[12,33],"02572446":[],"02572685":[],"02576358":[],"02576685":[],"02580654":[],"02584233":[],"02585059":[],"02586427":[7,34],"02593707":[],"02594314":[],"02595095":[],"02597067":[],"02597861":[],"02599188":[],"02600605":[],"02601643":[],"02604362":[],"02606605":[],"0260728":33,"0260906":[7,34],"02612988e":[],"0261442":22,"02616812":37,"02617297":[],"02617887":[],"02622039":[],"02623761":[],"0262517":[],"02625193":9,"02625852":[],"02629144":37,"02629373":[],"0263106":[],"02631244":[],"02631942":[],"02632742":[],"02635861":[],"02636268":[],"02646648":[],"02649469":[],"026495":[],"02651857":[],"02654585":[],"02658761":[],"02660572763718093":[7,35],"026605727637180936":[7,35],"02661264":[],"02664408":[],"02667406":[],"02676992":22,"02677463":[],"02684648":38,"02687671":[],"02692965":[],"02693015":[],"02693858":34,"02694975":[],"02704796":[],"02705202":[],"02707227":6,"02709478":[],"02709761":38,"02710291":[],"0271259":33,"0271583":[],"02717039":[],"02723445":[7,34],"0272743":[],"02729375":[],"02731808":[],"02732402":[],"02735547":37,"02751373":[],"02754248":[],"02760064":[],"027609773491022525":[7,35],"02760977349102253":[7,35],"02761893":[],"02766946":[],"02769274":[],"02780107":[],"02781444":[],"02784163":32,"02784266":[],"0278673":22,"02787184":[],"02792633":[],"02793082":[],"02797154":[],"0279886":37,"028":[],"0280223":[],"02805988":[],"02806268":[],"02808229":[],"02809972":[],"02813369":[],"02813549":[],"02816961":[],"02818667":[],"028188673105960467":[],"02832687":[],"02833443":[],"02835701":[],"0284405":[],"02846506":[],"02850506e":[],"02853426":22,"02856881":[],"02857":5,"02859759":[],"028638258927275215":[],"02872216":[],"02876848":[],"02880987":[],"02884485":29,"02895708":[],"02895911":[],"02895918":34,"02899912":[],"02900153e":37,"029003":[],"02901273e":[],"02901441":[],"02909574":[],"02910539":22,"02910881":[],"02911348":[],"029114513319594925":29,"02920675":[],"02923006":[],"0292555":[],"02925855":37,"02928194":[],"02930753":[],"02933055":38,"02940083":[39,40],"02941655":[],"02943644":[],"02944152":[],"02944974":[],"02946852":[],"029483":6,"02953975":[],"02960445":[],"02960492":38,"02962583":[],"02962903":[],"02963147":[],"02963424":[],"02965159":[],"029664":[],"02966421":[],"02970225":[],"02970397":22,"029733":33,"02973733":[],"02976145":[7,34],"02976686":[],"02979087":[],"02980537":[],"02982009":[],"02987043":[],"02987496":[],"02988432":[],"02990404":33,"02992287":[],"02993573":37,"02994311":6,"02996909":[],"02999715":[],"02f":[7,26],"03":[2,7,33,34,40,41],"0300023":[],"03000689":[],"03003494":33,"03003891":[],"03006815":[],"03008641":[],"03012648":[],"03015288036252618":[],"03016341":[],"030164788184997":[],"03021588":[],"03031173":[],"03032441e":[7,34],"03032534":33,"03033791":[],"0303586629618948":[],"03041332":[],"03041488":22,"0304378":[],"0305221":[],"03057995":[],"03059882":[],"0306232":[],"030645975175292262":[],"03065856":33,"03068149":[],"03076019":[],"0307725":[],"03077640549":5,"03092729":[],"03099776":6,"03099794":[],"031":[6,34,35],"03100531":[],"03101165":[],"03112129":[],"03115122":[],"0311613":[],"03119062":[],"03120433":[],"03120942":[],"03143856":33,"03158013":[],"03159246":[],"0316146":[],"03163926":[],"03166151":[],"03166168":22,"03166546":[],"03170652":[],"0317606":[],"03177331":[],"03181997":[],"03188133":[],"03194307":[],"03196357":[7,34],"03196582":[],"03201179e":[],"03201217":[],"03201832":37,"03201922":[],"03204446":38,"03205795":[],"03206321":[],"03206757":[],"03207075e":[39,40],"0321222":[],"03212854":[],"032142":[],"03214247":[],"03214829":[],"0321956":[],"032196":[],"03221473":[],"03221782":[],"03230611":[],"03234422":38,"03235719":[],"03237424":[],"03239118":[],"03239393":[],"03241171":[],"03241936":25,"03241945":22,"03244944":[],"03246707":[],"03247316":[],"0324793":[],"03248618":33,"03251863":6,"03254179":[],"03254614":[],"03256632e":[2,40,41],"03261373":[],"0326167":[],"03266152":[],"03267527":[7,34],"03274273":[],"03279636":[7,34],"032825":6,"03282533":6,"03283413":[],"03285784":[],"0328643":37,"03291795":[],"0330308045183194":[7,34],"0330308045187757":[7,34],"03304417":[],"03306538":37,"0330705":[],"03308408":6,"03309191e":[],"03311432":[],"03313193":[],"03313259e":[],"03317599":[],"03318519":[],"03319608":[],"0332335":[],"033233504714921":[],"0334136":[],"033414":[],"03343379612645295":[],"0334505":[],"03345624":[],"03351257":33,"03354468":[],"03354844":[],"03357723":[],"03358833":[],"03361037":[],"03364384":[],"03365012":[],"03365665":[],"033657685071527665":[7,35],"03374843":22,"0337609":[],"03382942":[],"03396658":[],"034":[],"03400724e":[],"03404758":[],"034048":[],"03409194":[],"03412067":[],"03418643":[],"0342028":[],"0342458":37,"03426412":[],"03426934":[],"03430385":[],"03439189":[],"034404":[],"03440431":[],"03447512":[7,34],"034476":[],"03447649":[],"03454641":[],"03455614":22,"034557":33,"03455885":[],"03468878":[],"03472308":[],"03474443":[],"03476611e":[],"03479236":[],"03479936":[],"03484266":[],"03484946":33,"0348572":[],"03500569":[],"03500949":38,"03509683":[],"03510464":[],"03511387":[],"03514117":29,"03529083":[],"03529801":[],"03530044":[],"03532164":33,"03533698":38,"03538463":[],"03538673e":[],"0354495":[],"0354894":[],"03554165":[],"03558034":[],"03559176":[],"03562355":[7,34],"03563709":[],"03563906":[],"03568439":[7,34],"03568495":[],"03570093":[],"03571339":[],"03573214":[],"03578526":[],"035810431523915":12,"03582724":[],"03586482":[],"0359565":6,"03596557":[],"0359727":22,"03601781":33,"03609183":[],"03612962":33,"03613058":[],"03613164":6,"036132":6,"03624103":[],"03624411":[],"0362483569261072":[],"03625417":[],"03630548":[7,34],"03642173":[],"03647182":25,"03648711":[],"036527092051294":[],"03656288":[],"03662823":37,"03664879":[],"03680811":[],"03681113":[],"03681479262838276":[],"03682195":[],"03683365":22,"03692132794542241":[],"03692749":32,"03692828":[],"0369544130635358":[],"03696458":[],"03699304":[],"03701105":[],"03703391":[],"03707133":12,"0370757046153366":[],"03710736":33,"03722287":[],"03733016":[],"03734925":[],"03736678":[],"03742586":[],"0374535":[],"03745816":[],"03751833":[],"03753059":[],"03768291":[],"03770824":[],"03776519":[],"03778607":[],"037813671417389005":[7,35],"03781367141738902":[7,35],"03796359":[],"03796582":[],"03798785":[],"038":[],"03805295":[],"03814292":[7,34],"03815288":[7,34],"03820746":[],"03824887":38,"0382721":[],"03827385":[],"038300":[12,33],"03834357":22,"0383613556521865":[],"03837":37,"03843478":33,"03844073":[],"0384468":[],"03846169":34,"03853213":[],"03853268":[],"03853313":[],"03853677":[],"03856054":[],"03856748":[],"03858916":[],"03860666":38,"03868311":[],"03868372":[],"03870024":[],"03872795610436844":[],"03873507":[],"03877201":[],"03887801":[],"03887807":[],"03888998":[],"038890":[],"03896708":[],"03901061":[],"039039":6,"03906948":[],"0391259":[],"03916813":[],"03923714":[],"03926126":22,"0394226":[],"03942371":[],"03948301":[],"03960002":[],"03961179":[],"0396789":[],"0397137":[],"039732":[],"03973227":[],"03973359":[],"03974553487733608":[],"039764":[],"03976851":37,"03976976e":[],"03979291":[],"0398603":[],"03990261":[],"039967668952797":[7,34],"0399676689527975":[7,34],"04":[2,7,12,34,40,41],"04000457":[],"04001696":[],"04006242":33,"040102":6,"04010697":[7,34],"04015455":[],"04018043":[],"04018705":[],"04018985":[],"04021377":[],"04025059":[],"04026":41,"04026126":[],"04030508":[],"04033524":[],"04038595":[],"04040902":[],"04041644":32,"04046513":[],"04046984":[],"04051441":[],"040569":[],"04061345":[],"04063532":[],"04063602":[7,34],"04065686":[],"040657":[],"040684":[],"04072235":[],"04073097":[],"04074169":[],"04081099":[],"040810991569604":[],"0408223":[],"04085864":[],"04085976":[],"0408707":22,"0409646":[],"04097813":[],"04098313":[],"04098879":[],"041":10,"04101344":[],"04104182":[],"0410697":[],"04108038":37,"04110474":[],"0411487294305088":[7,34],"0411487294305226":[7,34],"04121466":[],"0412828":1,"0414517":33,"04150522":[],"04158557":[],"04161572":[],"04161649":[],"04164966":[],"0417409":[39,40],"04178558":12,"041786":12,"04178798":38,"04183444":[],"04184888":22,"04186033":[],"04191323":[],"04193203":33,"04198387":[],"042060":[],"04206048":[],"042136":[],"04216532":[],"04217103":[],"04217888":38,"0422":[],"04220758":[7,34],"04220931":[],"04224929":[],"04235439":[],"04238763":[],"04240593":[],"04245961":38,"04253288e":[],"04254787":[],"04262217":[],"04268493":[],"04277528":[],"04286013":[],"04293545":[],"04295726":[],"04295757":35,"043":10,"043072":[],"04307275":[],"04315108":6,"04315855":[],"04322019":[],"04323936e":[],"04323964":[],"04328852":[],"04329607":[],"04329799":[],"04330558":[],"04330907":[],"0433282860079154":[],"04346721":6,"04350821":[],"04353549":[],"04355062":[],"04355106":[],"04355837":[7,34],"04356439":22,"04362":10,"04362042":[],"043637":[],"04369752":[],"04372016":[],"04372067685794603":[],"0437499":3,"043839":[],"04389027":[7,34],"044014":[],"04401421":[],"04403475":[],"04404068":[],"04413243":[],"04418179":[],"04421197":[],"0442242":[],"04423486":[7,34],"04426744e":33,"04431557":[],"044334":33,"044338305845698256":[],"04435458":[],"04443985":[],"04452918":[],"04460703":22,"044613":[7,34],"04465557":[],"04487298":[],"0449":[],"04494234":[],"04495752":[],"04497158":[],"04497816":[],"04501146":[],"04501548":[],"04504728e":[],"04513394":[],"045161":[],"04516434":[],"04519174":[],"045258":[],"04532405":[],"04537385":[7,34],"04543942":[7,34],"04544327":[],"04545125":[],"04545296":[],"04562561":[],"045630":[],"04564589":[],"04565456":38,"04566964":[7,34],"04575775":[],"0458":10,"0458524213754298":[],"04585476":[],"04587765":[],"04588592":[],"04590873":[],"04592382":[39,40],"04596002":32,"045963":[],"04601419":[],"04608163":38,"04626298":[],"0462999":[],"04638043":[],"04640695":[],"04641529":[],"04643561":22,"04646338":[],"04648225":[],"04648335":6,"04648472":12,"046485":12,"04652794":[],"04655574":[],"04657426":[],"04658881":[],"04661875":[],"04664255":7,"046663":[],"046678":[],"04670255":[],"046764":[],"04683565":6,"04684509":[],"04686491":[],"04688012":[],"04702861":[],"04718566894028431":[],"04719868":[],"047299":[],"04729958":[],"0473":7,"04736252":[],"0473916":[],"04740084":[],"047447724189861":[],"0474753":37,"047489":[],"04754697":22,"04757138":[],"04759123":[],"0476266":[],"047760":[],"04776242":[],"04784395":[7,34],"04794409":[],"048020":[],"04808723":[],"04816191":[],"0481872773043029":[7,35],"048194":[],"04823243":[],"0482407":33,"04829439":[],"04832356":[],"04836325":33,"04838148":[],"04838489":[],"04843417":[],"04843794":[],"04852743":[],"04853398":[],"04857266":[],"04861316":[],"04863792":29,"04877931":[],"048781":[],"0488139":[],"04881585":[],"04892055":[7,34],"04899367":[],"049060":[],"04909093":[7,34],"04912436":7,"04915593":[],"049196":[],"04922273":[],"04926746":35,"049276":[],"04933083":[],"04937026":[],"049380":[],"04939861":[],"049479":[],"049481":[],"04949606":22,"049556996627824":[7,34],"0495569966278269":[7,34],"04955942":[],"04963309":[],"049677":[],"04984743":[],"04987891":38,"04999052":[],"05":[2,5,7,26,33,34,37,38,39,40,41],"05000245":[],"050015":[],"05002471":[],"05005634426334421":[],"05006037":[],"05006956":[],"05009826":[7,34],"05013368":[],"0501942":[],"050281":6,"05029018":[],"05029453":[],"050315327923114654":[],"05034088e":[],"0503455":38,"050357":6,"05040878":[],"0504379":[],"05043958":[],"05045":[],"05054319":[],"05055465":[],"05060379":[],"05068067":22,"05068486":[],"05072744":[],"05077224":[],"050796":[],"05080769":[],"05087018":[],"05097344":[],"05098714":[],"05100875":[7,34],"051021":[],"051042":[],"05105586":33,"05110835":[],"05124625":[],"05135047":37,"051368":[],"05141614":[],"051418":6,"05144177":[],"05148401":[],"05151178":[],"051554":[],"05159088":[],"051598":[],"051645":[],"051649":[12,33],"05169046":[],"0517473":6,"05176711":[],"051782":25,"051787":[],"05180367":[],"051916914594944144":25,"052010":[],"05201041":[],"05211919":[],"05224117":[],"05224627":[],"0522653":[],"05227563":[],"052279218012056844":[7,35],"05227921801205686":[7,35],"052305":33,"05232855":33,"05233008":[],"05233928":29,"052360":6,"052370":[],"05240397":[],"05241265":[],"052427":[],"0524497":[],"05245442":[],"05247827":[],"0525733":[],"05263":10,"05265465":[],"05267589":[],"052692":[],"05269476":[],"052738":[],"05275823":22,"052856":[],"05285677":[],"052918":6,"05293216":[],"05293756":[],"05296475":[],"0529838":[],"05302":10,"053036":[],"05321642":33,"05334469":33,"05336637":[],"053387":[],"053417":33,"053462":[],"05347795":[],"05353553":[],"053599":[],"05363722":[],"05364579":[],"05364854":9,"05376358":[],"053781":6,"053787":[],"05383795":[7,34],"05383947":[],"05384858":[],"053849":6,"053854":[],"05395761":[],"05401912":[],"05402091":22,"05404387":[],"05415049":[],"05415575":[],"05415968":[],"054197":[],"05424351":[],"0542797":[],"054286":[],"05430625":[],"05431945":[],"05442743":[],"054432":6,"054434":[],"05447415":[7,34],"05449413":38,"054500":6,"05458323":[],"054609":[],"05461692":[],"054617":33,"054655":33,"05476114":[],"05484773":[],"054932":[],"055000":[],"05500389":38,"05505310046363":[],"055091049054481":[],"05509105":[],"055105":[],"05519999":34,"05526935":[],"0552878":[],"055314":[],"0553198496333376":[],"05533":10,"055349":6,"05546402":[],"055477":[],"05548187":[],"05549529":[],"05549907":33,"05554795":[],"05563358":[],"055653":[],"055676":33,"055715":6,"055734":33,"055737":[],"05574159":[],"05577845931438246":[],"055866":[],"055876":[],"055910":33,"055925":[],"055987":33,"056026":[],"056041":[],"05605667":[],"05605892":[],"0560663":[],"05607583":[],"0560917":[],"05614483":6,"0562":[],"05623":10,"05623539":22,"056287":[],"05632246":[],"056328":[],"05636265":[],"05638246":[],"05641352":[],"056414":[],"05648":10,"05648118":[],"05648836":[],"056491":6,"05651951":[7,34],"05655534":33,"056561":6,"05660129":[],"05664639":[],"05665429e":[],"05665875086534638":[],"05667":10,"05668052":[],"056683":33,"056717":[],"056718":6,"056730":[],"056826":[],"056856":[],"05692207":[],"05696574":[],"05706322":25,"05716368155342608":[7,35],"057216":[],"057287":[],"057298":[],"057300":33,"057312":6,"057316":[],"05731824":[],"05737289":[],"05738485":[],"05738629":[],"057387":[],"057393":33,"057418":33,"057420":[],"057457":33,"05747369":[],"05749111":[],"05750493e":[],"057514":[],"057607":[],"057609":[],"05761244":[],"05761295":[],"05767":39,"057682":[],"057711":6,"057722":33,"0578214":[],"057835":33,"05783782":[],"05785343":[7,34],"05785974":[],"05788664":[],"05788748":[],"05789007":[7,34],"05791283":[],"057913350391706":[],"057929":[],"05792927":[],"05793564":33,"057951":[],"057961":[],"05796251":[7,34],"057964":[],"05804539":[],"05804744e":37,"0580672":[],"05806981":[],"05807125":[7,34],"058133":[],"0581406":33,"058152":[],"05818561278267464":[],"05818879":[],"058234":[],"058245":[],"058254":[],"05831017":[],"05834577":[],"058347":12,"058361":[],"058409":[],"058422":[],"058427":[],"0584713":[],"058503":[],"05850899":[],"05853984":[],"058552":33,"058582":[],"058593":[],"058641":[],"058645":33,"0586484":37,"058670":6,"05867956":[],"058721":[],"058726":6,"058770":[],"058774":6,"05879106":[],"05883":10,"05884":10,"058854":33,"058866":[],"05888195":[],"05893076":[],"058935":[],"058952":33,"058967":[],"058970":[],"058996":33,"058997":[],"059003":6,"059031":33,"059093":6,"05914563":[],"059181":[],"059201":[],"059233":6,"059243":[],"059340":[],"059348":6,"05938299":[],"059398":6,"05941988e":[],"059480":[],"0595364":38,"059624":[],"059669":[],"059761":[],"05978512":[],"059805":12,"059807":33,"059827":6,"05984726":[],"059864":6,"059876":[],"05992237":[],"05994172":22,"059951":33,"059956":[],"05996":37,"059977":[],"059979":[],"05998413e":[],"05999":10,"059992":[],"059993":33,"05999828":32,"06":[7,33,34,37,38,39,40],"060071":[],"060083":33,"060085":[],"060087":[],"06009333":38,"060149":[],"060160":[],"060163":6,"060171":[],"06019833":[],"06020264":[],"06020587":[7,34],"06021793":[],"060251":[],"060254":33,"060275":[],"060288":[],"060312":[],"06031928":[],"06034627":33,"06037137":33,"060384":[],"060435":6,"06043581":[7,34],"060453":[],"060470":[],"060471":[],"060475":[],"0604888":[],"060490":[],"060547":[],"06055507":[],"060565":[],"06057711":[],"060590":[],"06061276":[],"060629":[],"060631":[],"060652":[],"060678":[],"06068192":[],"06069047":33,"0606958":[],"060716":33,"06072":37,"060726":[],"060734":[],"060735":[],"060756":33,"06079614":[],"06080649":[],"060824":[],"06082419":[],"06082594":33,"06083072":[],"060870":[],"060872":33,"06088245":[],"060898":[],"060994":[],"061025":[],"061045":6,"06110658":[],"061118":[],"06112467":[],"061154":[],"061163":33,"06117338":[],"061178":12,"061187":[],"061284":[],"06128647":[],"061319":[],"061333":6,"06133572":[],"061359":33,"06136823":[],"061386":[],"0613875":[],"061388":[],"061394":[],"06139439":[],"061411":12,"061443":33,"061447":[],"061452":33,"06146873":[],"061482":6,"06151386":[],"06154484":[],"061545":6,"06155151":[],"061588":[],"06159762":[],"061614":33,"061637":[],"061656":[],"061706":[],"061707":[],"061724":[],"061772005077293704":[],"061775":[],"061779":6,"061793":[],"061795":[],"061848":[],"061862":[],"061898":[],"061907":6,"061915":6,"061921":[],"061956":[],"06199119":[],"06200174":6,"062004":[],"06200442":[],"06200455":[],"062016":33,"062053":[],"062058":[],"06208238634231949":[7,35],"06210358":[],"062161":[],"06216141":[],"062173":[],"0621855":[],"062199":[],"06220192":[],"062216":[],"062231":[],"062241":[],"06225028e":[],"062260":[],"062263":[],"062282":[],"06228382":[],"062292565":5,"062300":[],"062310":[],"062337":33,"062345":6,"062352":[],"062356":[],"062424":[],"062429":[],"062446":6,"062481":6,"062489":[],"062520":[],"0625462":[],"062568":[],"062599":33,"06260611":[],"062616":[],"062624":33,"06263176":[],"062637":[],"06265161":[],"06267057":[],"06267966":[],"062691":[],"062693":[],"062695":6,"062700":[],"062705":[],"062709":6,"062719":[],"06272254":32,"062733":12,"06276815e":[],"06277313":[],"062786":[],"062797":[],"062853":[],"062874":33,"06287939":[],"062932":[],"06295579":[],"06301273":[],"063017":[],"063019":33,"063048":[],"063061":33,"063064":[],"063080":33,"063082":[],"06310155":[],"063130":[],"063133":[],"063161":[],"063163":[],"063180":[],"06318287":[],"063197":[],"063206":[],"06320757":[],"063260":33,"063264":6,"063283":[],"063294":[],"063327":[],"063332":[],"063336":12,"063354":[],"06335853":[],"06336262":[],"063395":[],"063401":6,"063482":[],"063517":[],"063518":[],"063525":[],"063526":[],"063536":[],"063545":[],"063549":[],"063554":[],"063560":[],"06357887":[],"063621005600529":25,"063637":[],"063646":[],"063653":33,"063660":[],"063673":[],"063682":[],"063768":[],"0637712":[],"063781":[],"063786":[],"063791":[],"063802":[],"063809":[],"0638484":37,"06385189":[],"063862":[],"063883":6,"06389233":22,"063894":33,"063910":[],"06392846":[],"063933":[],"063954":[],"06395476":[],"063959":[],"063977":[],"063978":[],"063995":[],"064000":[],"064054":6,"064071":[],"064074":33,"064077":[],"06410638":[],"064107":[],"064164":[],"06416459":[],"064166":[],"064176":6,"064182":[],"064186":[],"06418809":37,"064189":[],"064202":[],"064218":[],"06423317":[],"064242":[],"064294":[],"06430789":[],"064311":6,"064314":6,"064322":[],"06432267":[],"064324":[],"06437266":[],"064373":[],"064382":[],"064427":[],"064428":[],"06443054":[],"06443215":[],"06444":10,"06444224":[],"064471":[],"064501":33,"064505":[],"064507":[],"064514":[],"06453579006728324":[7,35],"064540":[],"064547":[],"064573":[],"064604":[],"064606":33,"064609":33,"064615":[],"0646237":[],"064667":[],"064668":[],"064690":[],"064695":[],"064698":12,"064699":[6,33],"064703":[],"064728":[],"064739":[],"06474536":25,"064790":[],"064793":33,"0648":[],"064820":[],"064823":[],"064834":[],"06484181":[],"064845":[],"064873":[],"064878":[],"06491455":[],"06491736":[7,34],"064924":[],"064935":[],"06494075":[],"064954":[],"06496916e":[],"064978":[],"064980":[],"065001":[],"065007":[],"065028":[],"06504522":38,"06505375":[],"065062":[],"06506624":[],"065077":33,"06507762":22,"065096":[],"065108":[],"06512065":[],"065121":[],"065139":[],"065149":[],"065156":6,"065183":[],"065212":[],"065217":[],"065219":[],"065282":6,"065305":[],"065330":[],"065352":[],"065358":[],"06536971":[],"065375":[],"065378":33,"065387":12,"065390":33,"065410":33,"065469":6,"065472":[],"06547790180152355":[7,35],"06547790180152359":[7,35],"065482":[],"065508":[],"065556":[],"065565":12,"065591":[],"065616":[],"065619":[],"065650":[],"06565368":38,"065654":[],"065655":12,"065661":[],"065674":6,"065679":[],"065690":6,"065778":[],"065788":[],"0658141e":[],"065818":[],"065821":[],"065823":[],"065830":[],"065842":12,"065859":[],"065884":[],"06590458":[],"065949":[],"065960":[],"065963":[],"065989":[],"06600954":[],"066030":[],"06603447":[],"066044":[],"066054":[],"066059":[],"06606215":[],"066066":33,"066127":[],"066145":[],"066153":[],"066162":[],"066192":[],"066199":[],"066205":[],"066207":[],"06622369":[],"066241":[],"066256":[],"066286":[],"066330":[],"06634473":[],"066354":6,"066358":[],"0663596":[],"06636934":[],"06637":10,"066425":[],"066452":[],"066458":[],"066474":33,"066483":[],"066500":33,"0665111":[],"066520":6,"066524":[],"066545":[],"06659997":[],"066612":33,"06662069e":[],"06662675":[],"066633":[],"066647":33,"066653":[],"066680":6,"0666807":3,"066686":[],"06674611e":[],"066763":[],"066766":[],"06677444e":[],"066778":[],"066786":[],"066787":33,"066824":[],"066830":[],"066837":[12,33],"066838":[],"066841":[],"06684886":[],"06685171":[],"066854":[6,33],"066855":[],"066865":[],"066877":6,"06687898":[],"066896":6,"06690071":[],"066912":[],"066929":[],"066948":[],"06698472":[],"066988":[],"066997":[],"066999":33,"067000":[],"067003":[],"067032":[],"06703705":[],"06704902":[],"06706123":[],"067074":[],"067077":[],"067141":[],"067164":[],"067168":[],"067192":[],"067204":6,"067209":[],"067224":[],"06724062":6,"067249":6,"067260":[],"067270675787016":[],"0672706757870163":[37,38],"067273":[],"067288":[],"067313":[],"067314":12,"067367":[],"067374":[],"067380":[],"06739484":[],"067400":[],"067431":[],"067435":12,"0674567":[],"067492":[],"067507":12,"067511":[],"067515":[],"067523":[],"067539":[],"067597":6,"067623":[],"067637":33,"067662":[],"067671":[],"067707":33,"067719":[],"067731":[],"067758":[],"067764":[],"06777772":[],"067779":[],"067780":[],"06778421":[],"067793":[],"067809":[],"067819":[],"067843":[],"067851":12,"067883":[],"067917":[],"067922":[],"067932":[],"067935":6,"067940":12,"067943":[],"067948":[],"067967":[],"067974":[],"068008":[],"0680225":[],"068024":[],"06804775":[],"068057":12,"06807123":[],"06808421":[],"068088":[],"06810332":22,"068116":[],"068131":[],"068171":[],"06819354":29,"068206":[],"068215":[],"068222":[],"068226":12,"068229":[],"06826275":[],"068264":33,"068268":[],"068274":[],"068278":12,"06828347":[],"068289":[],"068307":33,"068324":[],"068341":[],"068353":[],"068360":[],"068364":[],"068369":[],"068394":[],"068406":[],"068408":[],"068429":[],"068440":[],"068441":33,"06844519414009445":[7,35],"06844519414009446":[7,35],"068446":[],"068479":[],"06849372":[],"068494":[],"068499":[],"0684990216069429":[],"068504":[],"068512":[],"06851496":[],"068533":[],"068539":[],"068587":[],"068592":[],"068597":[],"068607":[],"06862038":[],"068629":33,"068637":[],"068680":[],"068684":[],"068701":6,"068727":33,"068758":[],"068775":[],"06877599":[],"068797":6,"06881293":[],"068827":[],"068843":[],"068873":12,"068897":[],"068903":[],"068906":33,"068912":[],"06891861":[],"068924":[],"0689303":[],"068931":6,"068939":6,"068967":[],"068977":[],"068984":[],"068985":[],"06901081":[],"069028":33,"069045":[],"06908518":[],"069129":[],"06913112":[],"069132":[],"069136":12,"06915429":[],"069179":[],"06919343":33,"069203":[],"06926151969326948":[],"06926192":[],"069270":[],"069293":[],"06929519":[],"069317":[],"069320":[],"069322":[],"069388":[],"069391":[],"069399":[],"06942115e":[],"0694226":[],"069427":[],"069439":[],"069452":33,"069457":6,"069484":[],"069486":[],"069492":[],"069507":6,"069509":[],"069526":[],"06957317":[],"069584":33,"06958788":[],"069592":12,"069597":6,"069606":6,"069636":[],"069637":[],"06967433":[],"069675":[],"069676":[],"069680":[],"069686":[],"06970419":[],"069711":[],"06971862":[],"069754":[],"069761":[],"069784":6,"069785":6,"069802":[],"069811":[],"06984502":[],"069866":[],"069873":[],"069885":[],"069907":[],"069916":12,"0699274":[],"069948":6,"069964":[],"069970":[],"069978":[],"069979":[],"07":[7,33,34,37,39,40],"070010":[],"07001431":[],"070017":[],"070021":[],"070024":[],"070027":[],"070043":33,"070063":[],"07006502":[],"070087":6,"070095":[],"070098":[],"070135":[],"070146":33,"070150":[],"07016":10,"07017":10,"070172":12,"070178":[],"070181":[],"070183":[],"070190":[],"070213":33,"070248":12,"070258":[],"070273":6,"070274":[],"070275":[],"07027791":[],"070318":[],"07036197":33,"070367":12,"07037089":25,"070380":[],"07039":10,"070414":[],"07043187":[],"070432":[],"070441":[],"070457":33,"070460":[],"070462":[],"070480":[],"070485":[],"070514":[],"070527":[],"070529":12,"070531":[],"070547":[],"070552":[],"070556":[],"070558":[],"070581":6,"070582":33,"07058967":[],"070595":12,"070600":[],"070605":[],"070622":[],"07062318":[7,34],"070625":[],"070630":[],"070647":12,"070655":[],"070657":[],"070667":[],"070696":[],"070712":[],"070730":[],"070737":33,"070747":[],"070769":[],"070772":[],"070774":[],"070780":[],"070805":[],"07082844":33,"07084394":[],"070855":[],"070859":[],"070886":[],"070889":[12,33],"070891":[],"070898":[],"070916":[],"070918":[],"070924":[],"070950":[],"070975":[],"070980":[],"070986":33,"070991":[],"071001":[],"071012":[],"07101262":[],"071018":[],"071027":[],"071049":[],"071055":[],"071058":12,"07106376":38,"07106891":[],"071070":12,"071087":[],"071092":[],"071103":[],"071143":[],"07114326":[],"07115":10,"071173":[],"071186":[],"071191":[],"071197":[],"071208":[],"071217":[],"07123177":[],"071252":[],"071253":[],"071279":[],"0712848":[],"071291":[],"071294":[],"0713":[1,32],"071314":[],"071328":[],"07134161":[],"071356":[],"071379":[],"071406":[],"071427":[],"071433":[],"071436":[],"071437":[],"071449":[],"07145103":12,"071459":[],"071471":[],"07147868":[],"07147935":[],"071530":[],"07153889":[],"071560":[],"071572":[],"071575":[],"071578":[],"071580":[],"071587":12,"071588":[],"07160048164233102":[7,35],"07160048164233104":[7,35],"071608":[],"071638":[],"071643":[],"071646":[],"071655":[],"071656":[],"071666":[],"07166646":[],"071680":[],"071687":[],"071704":12,"07170448":[],"07172852":38,"07172932":[],"07173249":[39,40],"071758":[],"071759":[],"071774":6,"071789":[],"071803":[],"071831":[],"0718316":37,"071868":[],"07187601":[],"071919":6,"071921":[],"0719267":[],"071935":[],"071937":[],"071954":[],"07195418":[],"071962":[],"071969":[],"071976":[],"071978":[],"072000":33,"072006":12,"072011":6,"072013":12,"072015":[],"072028":[],"072031":[],"072037":[],"07204042":[],"072083":[],"072085":[],"072092":[],"072101":[],"072111":33,"072132":33,"072140":[],"072147":[],"072148":[],"072171":[],"072173":[],"07218473624441492":[],"072199":[],"072200":[],"072209":[],"072220":[],"072221":[],"072223":[],"072226":6,"072234":[],"072238":[],"072240":[],"072241":[],"072262":[],"072263":6,"072286":[],"072290639572326":[],"07229064":[],"072303":[],"072304":[],"072329":[],"072344":[],"07234492":[],"07235641":[],"072380":[],"072388":[],"072404":[],"072406":[],"07241855":[],"072476":33,"072500":[],"072518":[],"072533":12,"07253999":32,"072551":[],"072577":6,"0725872":[],"07258982":[],"072590":[],"072591":[],"07259548":33,"072615":12,"072621":33,"07263033":[],"072638":[],"072653":6,"072684":[],"072688":[],"072689":[],"072693":[],"072694":[],"072697":[],"072721":[],"072728":[],"072733":[],"072752":[],"072773":[],"072787":[],"072817":[],"072820":[],"072824":[],"072826":[],"072835":[],"072843":[],"072849":[],"07285":4,"072859":6,"072863":[],"07286621":[],"072885":[],"072888":[],"072902":[],"07291598":[],"07292363":[],"072932":[],"072946":[],"072947":[],"072961":[],"072967":6,"072973":[],"07298295":[],"072997":[],"073021":[],"073048":[],"073050":12,"073071":[],"073102":[],"073109":12,"073114":[],"073115":[],"073144":6,"073182":6,"073198":[],"073253":12,"073258":[],"073260":[],"073262":[],"073270":[],"073288":[],"073320":12,"07332248":[],"073339":[],"073340":12,"073344":[],"073349":[],"07335575":[],"073392":[],"073405":[],"073408":[],"073411":[],"073415":[],"073416":[],"07341744e":[],"073418":[],"073422":[],"073423":[],"073442":12,"073483":[],"073499":[],"073514":[],"073525":6,"073528":[],"073576975500551":[],"073598":[12,33],"073612":[],"073644":33,"073647":[],"073662":[],"073667":[],"073672":[],"073683":12,"073684":[],"073685":[],"07372439e":[],"07373479":[],"07376699":[],"07378525":[],"073791":[],"073794":[],"073800":[],"073841":[],"073858":[],"073863":6,"073866":[],"073882":[],"073883":[],"073899":[],"073911":[],"073923":[],"07397193":[],"073985":[],"073993":[],"074009":[],"074048":[],"07405556":[],"074067":[12,33],"074073":[],"074084":33,"074087":[],"074112":[],"074114":[],"074115":[],"07412144024494684":[],"074130":[],"074134":[],"07415258":[],"074193":[],"074210":33,"07421084":6,"074212":[],"074216":[],"074221":[],"074228":[],"07424587":[],"074265":[],"074283":[],"07429213":[],"074296":[],"07429886e":[],"074301":[],"074314":[],"074323":33,"074334":[],"074358":[],"07435812":[],"074360":[],"074368":[],"074372":[],"074374":[],"074417":[],"074454":[],"074460":[],"074482":[],"0744881":[],"07449122":[],"074506":[],"074524":[],"074531":[],"074561":[],"07456491":6,"07459764":[],"074622":[],"07468007":[],"074688":[],"074693":[],"07469645":[],"074713":[],"07471361":[],"074738":12,"074776":[],"074787":[],"074802":[],"074807":[],"074824":[],"074830":[],"074851":12,"074859":[],"074866":6,"07487128":[],"074882":[],"074892":[],"07489511":[],"07490892":[7,34],"074911":6,"074928":[],"074935":[],"074971":[],"074976":[],"075021":[],"075028":[],"075048":[],"075052":[],"075058":6,"075070":[],"075083":12,"075086":[],"07510564":[],"07510754":[],"07510785":25,"075109":[],"075114":[],"075129":[],"075136":[],"075155":[],"075157":[],"07517385":[],"075176":[],"075177":[],"075198":[],"075238":6,"075239":[],"075241":[],"075274":[],"075281":[],"075284":12,"075289":[],"075293":[],"075298":[],"075302":[],"075310":12,"07531349":[],"075320":[],"075334":[],"075356":[],"075363":[],"07537801":[],"075385":[],"075392":[],"075396":[],"075409":[],"075416":[],"075423":[],"075429":[],"075446":[],"075463":[],"075472":[],"075478":[],"07549007":[],"075496":[],"075498":[],"075501":[],"075513":[],"075522":[],"075525":[],"075542":[],"075544":[],"07555537":[],"075557":[],"075562":[],"075563":[],"075576":[],"075595":[],"075602":[],"075609":[],"075613":12,"075615":[],"075634":[],"075639":[],"075658":[],"075671":[],"075677":[],"075689":[],"075711":[],"075712":[],"075753":[],"075761":6,"075771":[],"075794":[],"075809":[],"075816":33,"075822":[],"075823":[],"075830":[],"075834":[],"075838":[],"075842":[],"075864":[],"075868":[],"075872":[],"075876":[],"075878":[],"075880":[],"075883":[],"075885":[],"075889":33,"075897":[],"075909":[],"0759216":[],"075956":[],"07597049":[],"075974":[],"075978":[],"075988":[],"07600369":[],"0760318":[],"076057":12,"076065":[],"076066":[],"076071":[],"076079":12,"076082":[],"076091":[],"076101":[],"076108":[],"07612762":[],"076138":[],"076150":33,"076154":12,"076157":[],"076161":[],"076163":6,"076167":[],"07617734331359052":[],"076189":[],"076203":[],"076212":12,"076225":[],"076232":[],"076244":[],"076248":6,"076259":[],"07630326327869198":[],"076306":12,"076321":[],"076333":[],"076334":[],"076340":[],"076351":[],"076361":[],"076378":[],"076389":[],"076393":[],"076396":[],"076411":[],"076418":[],"07641937":35,"076434":[],"076435":[],"076436":[],"076448":[],"076459":[],"076473":[],"076479":[],"0764924":[7,34],"076508":[],"076542":[],"076543":[],"076547":12,"076555":[],"0765856870687196":[],"07658569":[],"076586":[],"076616":[],"076618":[],"076638":[],"076647":[],"076648":[],"076654":12,"07665857":[],"076665":6,"076679":[],"07668538":[],"076693":[],"076776":[],"076776262573027":[],"076777":[],"07678":10,"076791":12,"076803":[],"076829":[],"076833":[],"076857":[],"076859":[],"076876":[],"0768805855280187":[],"076894":12,"07692307692307693":10,"076934":[],"076938":[12,33],"076947":[],"076951":[],"076968":[],"07698164":[],"077035":[],"07704298":[],"07705051":[],"077053":[],"077060":12,"077061":[],"077091":[],"07712197e":[],"07712506":[],"07712724":[],"077131":[],"077139":6,"077141":[],"077168":33,"077180":[],"077184":12,"077226":[],"077248":[],"077254":[],"07726368":[],"077280":[],"077292":[],"077301":[],"077304":[],"077310":[],"077329":[],"077337":[],"077341":[],"0773554":[],"077358":[],"077382":[],"077391":[],"07740964":[],"077411":[],"077414":[],"077431":[],"07745579":[],"077460":33,"077476":[],"0774777":[],"077485":[],"077486":[],"077488":[],"077490":[],"077510":[],"077516":[],"077523":[],"077529":[],"077534":[],"077539":[],"077541":[],"077550":[],"077553":[],"077566":[],"077567":[],"077601":[],"077608":[],"077613":[],"07763918":38,"077641":[],"077643":[],"077653":[],"0776532":[],"077672":[],"077683":[],"07768993":[],"077711":[],"077736":[],"077745":[],"07776381":[],"077764":[],"07777777777777778":[2,40,41],"077782":[],"077813":[],"077837":6,"077839":[],"077857":[],"077899":6,"077906":[],"077941":[],"077953":[],"077965":[],"077975":[],"07797763":[],"078000":[],"078035":[],"078039":12,"078044":[],"078051":12,"078067":[],"078071":[],"07808426989543932":[],"078110":33,"078118":[],"078123":[],"078131":[],"078146":[],"078170":[],"07820":10,"07820074":[],"078210":[],"078238":[],"078267":12,"078268":[],"078282":[],"078284":[],"078313":[],"078324":[],"078336":33,"078359":[],"078363":[],"07836545":38,"07836997022107646":[],"078386":[],"078409":[],"078447":[],"078454":[],"078474":6,"078540":33,"078541":[],"078544":[],"078548":12,"078557":[],"078573":[],"07858042":[],"078588":[],"078589":6,"078636":[],"07864":10,"07865450504129884":12,"078661":[],"078684":[],"078688":[],"078708":[],"07871":10,"078711":[],"078731":[],"078741":[],"078751":[],"078753":[],"078782":12,"078825":12,"078855":[],"078867":12,"078889":[],"078898":[],"07890044":[],"078945":[],"078953":[],"078958":[],"078974":33,"078981":[],"078986":33,"079000":12,"079002":[],"079013":[],"079017":[],"079027":[],"079030":[],"079074":[],"079080":[],"079089":[],"079107":[],"07911268":[],"079114":[],"079138":12,"079161":[],"079174":[],"079184":12,"07919041":[],"079202":[],"079218":[],"079240":[],"079334":[],"079347":[],"079369":12,"079391":[],"07939331":[],"079420":[],"079429":12,"07944154":[25,32],"079443":6,"079444":[],"079487":6,"07950356694388497":[],"079510":[],"079518":[],"079535":[],"079562":[],"079566":[],"079638":[],"079670":[],"0796891867672603":[7,35],"07969087":[],"07971187802560528":[],"079739":[],"079745":[],"079747":[],"07974741":[],"079783":[],"07978850553011713":[],"079855":[],"079874":[],"079887":[],"079903":[],"079946":33,"079947":[],"079953":[],"079959":[],"079967":[],"07996807":[],"079976":[],"079977":[],"079982":[],"079987":12,"07998756":[],"079989":[],"079998":[],"07it":7,"08":[10,29,33,37,38,39,40],"080032":[],"080082":[],"080094":[],"08009874":[],"080103":[],"08010965":[],"080110":[],"080139":[],"080150":[],"080171":[],"08019747":[],"080199":[],"080205":[],"080220":[],"080222":[],"08023512":37,"080237":[],"080258":[],"080268":[],"080271":[],"080286":[],"080287":[],"080301":[],"080316":[],"08032505":[],"080330":[],"080345":[],"080353":[],"08036056":6,"080363":12,"080380":12,"080403":[],"08043851":6,"080448":[],"080453":[],"080474":[],"080493":[],"080535":12,"080549":[],"080555":[],"080559":[],"080576":[],"080585":[],"080605":[],"080608":[],"080616":33,"080630":[],"080631":[],"080651":[],"080657":[],"080658":[],"08068207":32,"080702":[],"080704":[],"080706":[],"080735":12,"080747":[],"080764":33,"080766":[],"080793":[],"080832":[],"080849":[],"080875":[],"080903":[],"08090942":25,"080947":[],"080950":[],"080987":[],"080997":[],"081031":[],"081038":[],"08109405":[],"081103":[],"081107":12,"081151":[],"081194":[],"081195":[],"081217":[],"081221":[],"08122943":[],"081232":6,"081235":[],"081252":[],"081276":33,"081283":12,"081296":[],"08131003":7,"0813156":[],"081324":[],"081342":[],"081405":[],"081410":[],"081411":[],"081431":[],"081450":[],"081457":[],"081494":[],"081523":12,"081527":[],"081530":[],"08153923":[],"08154221680664263":6,"081544":[],"08156108":[7,34],"08159479":[],"081637":[],"08165104":33,"081652":[],"081655":33,"081681":[],"081686":[],"081702":12,"081728":[],"081730":[],"081796":[],"08179906":[],"081811":[],"081823":[],"081835":[],"081836":[],"081845":[],"08187062":[],"081871":[],"08189696":[],"08191754":[],"081926":[],"081931":[],"081955":[],"081978":[],"082002":[],"082009":[],"0820185":[],"082043":[],"082057":[],"082085":[],"082115":[],"082121":[],"082126":[],"082149":[],"082151":12,"08216308":[],"082170":[],"082173":[],"082174":[],"082180":[],"082230":[],"082240":[],"082249":[],"082256":[],"082264":[],"08227057":[],"082274":12,"08227668":[],"082303":[],"082307":[],"082316":[],"082318":[],"08233627":[],"082342":[],"082355":12,"082361":[],"082412":12,"082414":12,"08244283":[],"082464":[],"082478":[],"082482":[],"08251519":[7,34],"082518":[],"082519":[],"082533":[],"082537":[],"082553":[],"082596":[],"08259753":[],"082598":12,"082653":33,"08266769":[],"082670":[],"082742":[],"082764":[],"082784":[],"082785":[],"08280012":[],"082803":[],"082857":[],"082878":[],"082940":[],"082957":[],"082971":[],"08299273e":[7,34],"08300526":32,"083016":[],"083034":[],"083061":[],"083092":[],"083119":[],"08312573":29,"083128":[],"08313374":[],"0831367":[],"083138":[],"083179":[],"083182":[],"08318298e":[2,40,41],"08319383":[],"0832":[],"083203":[],"083248":[],"083257":[],"083259":[],"083260":[],"083268":[],"083288":[],"08332537":[],"083331":6,"08333333333333333":[2,10,40],"083342":[],"08336233266":5,"083405":[],"083446":[],"083459":[],"083465":[],"083471":[],"083479":[],"083499":[],"083501":[],"083527":33,"083547":[],"083548":[],"083564":[],"083596":[],"083645":[],"08370977":[],"083714":[],"083734":[],"083765":[],"08376632":[7,33,34],"083766322923899":[7,33,34],"0837663229239043":[7,33,34],"083793":[],"083801":[],"083864":[],"083900":[],"083930":[],"083968":12,"083970":[],"083982":12,"08398399377155011":[],"084018":[],"084042":[],"084076":33,"084091":[],"084112":[],"084122":[],"084127":[],"08418548":[],"08418956":[],"084195":[],"08419839":[],"084209":[],"084218":[],"08421808":[],"084222":[],"084255":[],"08426840630693411":[7,35],"08426840630693413":[7,35],"084289":[],"084312":[],"084319":[],"084340":[],"084346":12,"084358":[],"084376":[],"084406":[],"084423":[],"084434":[],"084440":[],"084486":12,"084514":[],"084531":[],"084535":12,"084548":[],"08455":10,"084550":[],"084568":[],"08458591":[],"08460718":[],"084619":[],"084637":[],"0846409":[],"084674":[],"084687":12,"084694":[],"084695":[],"084702":[],"084712":[],"08472879":[],"08474":10,"084801":[],"084809":33,"08482302":38,"084847":[],"084858":[],"08487999":[],"084910":[],"084915":[],"084934":[],"084963":[],"084967":[],"084976":[],"084987":[],"084996":[],"085079":[],"085099":[],"085120":[],"085139":[],"085141":12,"085145":[],"085147":[],"085150":[],"085155":[],"08516604":33,"085197":[],"085201":[],"085212":[],"085222":[],"085268":[],"085294":[],"085295":[],"085353":[],"08536248571780691":[],"085369":[],"085378":[],"085390":[],"08539155":37,"085392":[],"08539583":[],"08540984":[],"08543112":[],"085436":[],"085455":[],"08546261e":[],"085463":[],"085486":[],"08551306":[7,34],"085518":[],"085548":[],"085573":[],"085592":[],"08559230156092426":[],"08561649":38,"085655":[],"085659":[],"085690":[],"08569593":[],"085712":[],"085716":[],"085722":[],"08572513":[],"085728":[],"085747":[],"085762":12,"085764":[],"08576932":7,"085790":[],"085807":[],"085819":[],"085820":[],"085833":[],"085853":[],"08587209":[],"085902":[],"085918":[],"08593216":[7,34],"085935":[],"085937":[],"085970":[],"085973":[],"085980":[],"085988":[],"085996":[],"08601328":[],"086021":[],"086026":[],"086058":[],"086068":[],"08607283":[],"08609047":[],"086092":[],"086095":[],"086096":[],"086101":[],"086107":[],"08611111111111111":[2,40,41],"086125":[],"086130":[],"086212":[],"086218":[],"08622478":[],"086227":[],"086230":[],"086239":[],"08624193":[],"086250":[],"086256":[],"086286":[],"086287":[],"086289":[],"086306":[],"086318":[],"086326":12,"086338":[],"086339":[],"086346":[],"086364":[],"0864086":[],"086410":12,"086428":[],"08642966":[],"086471":[],"086483":[],"086489":[],"086508":[],"086548":12,"086576":[],"08660891":[],"086611":[],"086653":[],"086655":[],"08667591":[],"086694":[],"086712":[],"086730":[],"08675226":[],"086797":[],"086799":[],"086808":[],"086809":[],"08683446":[],"08687303":37,"08690":10,"08690184845323":[],"08691139":37,"086913":[],"086970":[],"08698097":32,"08699295e":[],"08700526":[],"087020":[],"08702578":[],"08705631913312815":[],"087102":[],"087135":[],"087210":[],"087255":[],"08728977":[],"087297":[],"087327":[],"087340":12,"08734023":[],"087369":[],"087424":[],"08742496":[],"087429":[],"08747795":[],"0875061":[],"087560":[],"08758":10,"08758013":[],"087585":[],"087621":12,"087675":[],"0877492":[],"087772":[],"08777858":[],"087789":[],"08779325":[],"087828":[],"087864":[],"087871":[],"087876":[],"087909":[],"08793554992813543":[],"087946":[],"087988":[],"08799729":[],"088023":[],"088029":[],"088030":[],"088038":[],"088058":[],"08809115":[],"088102":[],"088127":[],"08816816":[],"0881981":6,"088199":[],"088242":[],"08824304e":[],"08826182458028335":[],"08826256":[],"08833634":[],"088375":[],"088390":[],"08839544":[],"088425":[],"088441":[],"088455":[],"088481":[],"08854388":[],"088574":[],"08858739":[],"088599":[],"08860502":[],"088609":[],"088636":[],"088642":[],"088650":[],"088651":[],"088680":[],"088694":[],"08871404":6,"088726":[],"08881497884574564":33,"08888888888888889":[2,40,41],"088926":[],"08896995":6,"088970":6,"089":[],"089007":[],"08902":10,"089052":[],"089065":[],"089086":[],"089288":12,"089311":[],"08931169286872433":[],"08933713":[],"089338":[],"08935132374099551":[],"089354":[],"089376":[],"089377":[],"089386":[],"0894015":[],"089404":[],"089408":[],"089417":[],"089421":[],"08946029":[],"08947292":[],"08949703e":[],"0895043":[],"0895387":33,"089539":33,"089548":[],"08956994":[],"089576":[],"089578":[],"089600":[],"08961438":[],"089619":[],"089626":[],"089639":[],"089653":[],"089668":[],"089690":[],"089704":[],"089708":[],"089714":[],"089718":[],"089726":[],"089730":[],"089735":[],"089738":[],"089748":[],"089755":[],"089760":[],"089761":[],"089763":[],"089769":[],"08977751":[],"089783":[],"089785":[],"089786":[],"089790":[],"089795":[],"089800":[],"089826":[],"08984905":[],"089858":[],"089859":[],"089872":[],"089940":[],"089949":[],"08996":10,"089996":[],"09":[2,33,40,41],"090002":[],"09000319":[],"090033":[],"090038":[],"090075":[],"09007789":[],"090081":[],"090114":[],"090118":[],"09014451":[],"090154":[],"09016979":[],"090174":[],"090204":[],"09024373":[],"09028702":[],"090302":[],"090319":[],"09033649":[],"090387":[],"090429":[],"090437":12,"090439":[],"090440":[],"090468":[],"090475":12,"090476":[],"09048717285916912":[],"090492":[],"090561":[],"090565":[],"090624":[],"090626":[],"09065603":[],"09070064":[],"090717":[],"090721":[],"09078672":[],"0907958":[],"090804":[],"090935":[],"090985":[],"091":[],"091015":[],"091034":[],"091068":[],"09109617":[],"091159":[],"09120503":[],"09129027":33,"09135059e":[],"091362":[],"091365":[],"091388":[],"09156055":[],"09163947":7,"09165112":[],"091653":[],"09166666666666666":[2,40,41],"091674":[],"0917":10,"09171886":[],"091731":[],"091732":[],"091871":[],"09189665":[],"09192497":[],"092013":[],"092062":[],"092064":[],"09206825":[],"09209032":[],"09211239":[],"09219464":[],"09224244":[],"092252":[],"09225273":[],"092347":[],"092374":[],"092401":[],"092421":[],"09242193":[],"09246293455466892":[],"092486":[],"09251":10,"092530":[],"092533":[],"092607":[],"092608577084595":[],"09267158":38,"092689":[],"092703":[],"092746":[],"09280524":[],"09291556244521161":[],"09291806":[],"092956":[],"09296935":[],"092982":[],"09299349":[],"09300398":[],"093004":[],"093011":[],"09303501":32,"0930438":[],"093045":[],"093094":[],"09310479":[],"093115":[],"093226":[],"093288":[],"09331672":[],"09346106":[],"09347829":[],"09353846":38,"093554":[],"093572":[],"09360113":[],"09368":37,"093680":[],"09371427":[],"09374315e":22,"0938492":[],"09397626":[],"09398153":[],"094027":[],"094044":[],"0940821989627646e":[7,34],"0940821989652176e":[7,34],"094099":[],"094189":[],"094243":[],"094254":[],"094304":12,"094362":[],"09440493433371173":[],"094439":[],"09444444444444444":[2,40],"09452695":[],"094527":[],"094630":[],"09469095":[],"094697":[],"094797":[],"094849":[],"094857":[],"094862":[],"09487358":[],"094877":[],"09491833":[],"094941":[],"09497303":[],"095037":[],"095090":[],"095106":[],"095125":[],"09512539":[],"09515455":[],"095178":[],"095278":[],"09529067":[],"095358":[],"095395":[],"095425":[],"09543871010617433":[],"095465":[],"095626":[],"095710":[],"09572047":[],"095803":[],"095805":[],"09584387":[],"09590269e":[],"095915":[],"09604976":[],"096077":[],"096078":[],"09609807":6,"09610694":[],"096203":[],"09621323":[],"09622593":[],"096270":[],"09627032":[],"096287":[],"096365":[],"09637807":[],"09649893":38,"096511363983559":[],"09651792":[],"096570":[],"096630":[],"09664324":[],"09666342":[],"09667969":[],"096690":[],"096700":[],"096731":[],"096811":[],"09695222":[],"097017":[],"09702948":[],"097060":[],"097115":[],"097365":[],"097380":[],"0973900819831327":[],"09739637":[],"09740532e":[],"0974061":[],"0974119":[],"09744":10,"09755017":[],"09760893":[],"0976598":[],"097667":[],"09780":10,"097873":[],"09790216282081063":[],"09791":10,"09808149":[],"09808284":[],"09808622":[],"09812323e":[],"098144":[],"09821823":[],"09829902":[],"0983565":[],"098371":[],"098414":[],"098431":[],"09846933":[],"09848937":[],"09851247":[],"09861229":[25,32],"09862097":[],"09880256":[],"098803":[],"09895512":[],"09900258":[],"09903804":9,"09905657":[],"09913654":[],"09919198949275869":[7,35],"09919725":25,"09927808":[],"09928714":[],"09930135":[],"09931721":[],"09941275e":[],"099494":[],"09951287404314545":[2,40,41],"099588":[],"099622":[],"09964573":[],"0n":[1,32],"0s":[40,41],"0x0006":[4,5,15],"0x10746eeb0":22,"0x10b911550":[],"0x10b9828b0":[],"0x10cc52cd0":[],"0x10fcfaeb0":[],"0x1174af160":[],"0x11753f700":[],"0x117715eb0":[],"0x117893340":[],"0x117c17be0":[],"0x117edb760":[],"0x11892e7f0":[],"0x118995f70":[],"0x11975e550":[],"0x11a8ace50":[],"0x11af6aa90":[],"0x11b160a60":[],"0x11ba4a5b0":38,"0x11baaec10":38,"0x11bc3a1f0":[],"0x11bcee130":[],"0x11bdd4640":[],"0x11c3bd700":[],"0x11f13fb20":[],"0x11fa3db20":22,"0x11fc31790":[],"0x11fd7c160":[],"0x11ffa0ee0":[],"0x120cc1b20":[],"0x121cf68e0":[],"0x124252b80":[],"0x124b30190":[],"0x124b96d60":[],"0x125dbb640":[],"0x126431130":[],"0x1273ffac0":[],"0x1279bd460":[],"0x127d8ceb0":[],"0x12aaa0b20":[],"0x12b14fa90":[],"0x12ccea880":[],"0x12e13a850":[],"0x130ca6a60":37,"0x13134a100":37,"0x1322a9a00":[],"0x135f8b1f0":[],"0x1360ef5e0":[],"0x145270160":[],"0x1452d6eb0":[],"0x1522ef130":[],"1":[0,2,3,4,5,6,7,8,9,10,11,12,13,14,15,20,22,25,27,28,29,30,31,34,35,36,37,38],"10":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,22,23,25,26,28,29,30,32,33,34,35,36,37,38,39,40,41],"100":[1,2,3,4,5,6,7,9,10,11,12,14,15,16,17,18,22,23,25,29,30,32,33,34,35,36,37,38,39,40,41],"1000":[0,1,2,3,5,6,9,12,14,15,22,23,24,29,32,33,36,37,38,40,41],"10000":[3,6,7,11,12,14,29,34,35,41],"100000":9,"10001":11,"100083":[],"10008326":[],"1000911":[],"1001":[10,29],"1001408041761458":[],"1002":29,"100209":[],"10029389":37,"1003":29,"10030":10,"10036694":[],"10041812":34,"1004727":[],"10047943":[],"1005":29,"100523":[],"10068448":[],"1007":35,"1007711427354898":[7,35],"100816":[],"1009":29,"10100211":[],"101008878523571":[],"10107914":[],"1011":29,"101150":[],"10115017":[],"10116149":[],"10124634":38,"101263":[],"101288":[],"101292":[],"1013":29,"1013904243":29,"10141413e":[7,34],"10145028":[],"1015":29,"1015491":37,"10158754":[],"1015am":32,"10175169":[],"10179509":25,"10202674":29,"102147":[],"10219344":38,"102209":[],"1023":29,"10230":10,"1024":4,"10240571":[],"10242458":[],"1026":29,"10261093":[],"1027":29,"10274274e":[],"10276112":[],"103":[2,40,41],"1030":29,"10321406":[],"10325557":[],"10331028":[],"1033455":[],"10340":10,"10340733":[],"10348862":[],"10365224":[],"10369090e":[],"1037":29,"10371767":32,"10378326e":[2,40,41],"1038":29,"10387439":[],"10391807":[7,34],"10398646080125032":[7,35],"10398646080125035":[7,35],"1040":29,"10402095":[],"10405456":12,"10407019":[],"10425587":[],"10426556":[],"10426802":[],"10430":10,"104411":33,"10453899":[],"10454793727627841":[],"10465415":[],"1047":29,"10470031":[],"10478375":[],"10499357":37,"105":[3,34],"1050951":[],"10517643":[],"10520":10,"10520393":32,"10537416":[],"10539984":[],"105400":[],"10541723644166373":[],"10542736e":38,"10555555555555556":[2,40,41],"10556487":[],"10560458":[],"10560535":[],"10572":35,"10587277":[],"10589577":6,"10595991":[],"10598715e":[39,40],"106":3,"10604579e":[],"106095":[12,33],"10637914":[],"1064601390492845":[],"10646014":[],"10687432e":[],"10688434":[],"10695771":[],"10719554":[],"10719859":[],"10738656":[],"107387":[],"10747579":[],"107476":[],"1074792":[],"10754764":[],"10786423":[],"1079":35,"10790125813226321":33,"108":[3,7,10],"10801706":[],"1080176":[],"10837392":[],"10851177":[],"10854118":[],"10888197":[],"10898643":[],"10907699":[],"10908858":[],"10909182e":[],"10913":7,"10923006e":[],"10931453":[7,34],"10936832":[],"10950876":[],"10960":10,"10987614":[],"10991429":[],"10994958":[],"10995472e":[39,40],"10e":[40,41],"10th":10,"10x":[1,32],"11":[0,1,3,6,7,8,9,10,11,12,13,14,16,17,22,25,26,27,28,29,30,31,32,33,34,35,36,37,38,40,41],"110":3,"1100":29,"11005335":[],"11006373":[],"1101":29,"11022302e":[6,38,39],"11042239":[],"11048466":[],"110485":[],"11052859":[],"11061452":38,"11080997912276":[],"11081083":[],"1108407":[],"111":[0,2,3,8,13,36,38,39,40,41],"11100":10,"11113486":[],"111135":[],"1112416":[],"11129947":[],"11131389":[],"11147544":[],"11163239":[],"11164003":[],"11164926":38,"11171148":[],"1118933":[],"112":3,"11202337":35,"11204514":[39,40],"11214365":[],"11232457":[],"112383":[12,33],"1124":10,"11245282":[],"112619":[],"11261935":[],"11265643e":[],"11277166":[],"112772":[],"113":3,"1130676":[],"11315542":[],"11315771":[],"11320323":[],"11320726":[],"11331834":[],"11361003":[],"1137048":[],"11371176e":[],"11382896":34,"11388017":[],"11388888888888889":[2,40],"11390":10,"114":3,"11406297261341168":[],"11416086":[],"11433693":[],"114384":[],"11438414":[],"11439407":[],"11449851":[],"11449867":[],"114499":[],"114550":35,"11457699":[],"11462415":6,"114795":6,"11479519":6,"11482289e":[7,34],"1148541":[],"11487977":[],"11490318":[],"11498192":[],"11501291":[],"11507992e":[2,40,41],"11526665":[],"115356":[],"11535632":[],"11543883":[],"11547777218872497":[7,35],"11548661":[],"11556912":22,"11570114e":[],"11571507":[],"115822":7,"1158848":[],"11588954":[],"11590":10,"11596287":25,"11596834":[],"11597515":[],"11620886":[],"11634286":29,"11634286035477":29,"11660":10,"11666666666666667":[2,40,41],"11675128":[],"11683558":[],"1169204":[],"1169242":[],"11692966":[],"11694":26,"11694197":[],"117":9,"11715109e":[],"11725761":[],"11744554e":[7,34],"11747908":[],"11762444":[],"11780":10,"11797382":[],"118":[1,8,36],"11822173":[],"11826404":[],"11830757":34,"118318":33,"1184":35,"11840":10,"11844773":[],"11861803":[],"11868033":[],"11877604":[],"11890":10,"11891833e":[],"119":[1,8,36],"11922031":[],"11928633e":38,"11930916":[],"11935351":[],"11969984":[],"1198891":[],"11991488":32,"11992194":[],"119922":[],"119936":[],"12":[1,2,3,4,5,6,7,8,9,10,12,13,22,23,25,27,29,31,32,33,35,36,37,38,40,41],"120":[1,4,8,36],"1203":10,"12031117":[],"1203284":9,"12040799":[],"120435":[],"12043521":[],"12044974":33,"120450":33,"12068251":[],"12079371":[],"12081773":37,"12084953":38,"12085937e":[],"12099567":[],"121":[9,10,11],"12123488":[],"12124104":[],"121292":[],"12135727":[],"12143852":[],"12152902":[],"12155483":[],"1215pm":[30,32],"12166797":[],"12182967":[7,34],"12183393":[],"12183785":[],"12190193":[],"121902":[],"12198":38,"122":[9,10,11],"12204312":[],"12206013":[],"12206812":[],"12208052":[],"12210357":[],"12220088":[],"12220276":[],"12222222222222222":[2,40,41],"12235593":[],"12242545":[],"12279959":[],"12282958127703":6,"123":[],"12318726e":[7,34],"12329646":[],"12330033":33,"12333649":[7,34],"12363332":[],"123711":7,"12380":10,"12382443":[],"123928000581825":[],"12393064":[],"12394921":32,"124":[1,32],"12400":10,"1242":9,"124228866018077":[],"12425815":[],"124325":[],"12443871":[],"12461758":[],"12462963":22,"12466453":[],"12498355":[],"1251681":[39,40],"12534765":[],"12536737":37,"12544009":[],"12544297":[],"12552073e":[7,34],"125757":[],"12591227":33,"126":10,"12600095":[],"1261":10,"12618549":6,"1265":10,"12667402":[],"12675974":25,"12686507":[],"127":5,"12702643":[],"1271":7,"12733488":[],"12733892":[],"12777777777777777":[2,40],"12790":10,"128":[4,5,14,37,38],"1280526":[],"12829994e":[],"12831838":[],"128393685824704":[],"12840176":[],"1284100258440377":29,"12846660e":[],"12871842":33,"12873424":[],"12893778":[],"129":[],"129012":7,"12903093":[],"12910258":[],"12922837":[],"12936913":29,"12952339":34,"12957454":[],"1297":10,"1298":10,"129963":35,"12it":[],"12pm":[30,32],"13":[1,3,6,7,8,10,12,13,19,22,25,27,29,32,33,35,36,37,38,41],"130":10,"13003291":[7,34],"13009402":[],"13012167":[],"13024288":[],"13035147135400782":[],"13037395":[],"13040489":[],"13055555555555556":[2,40],"13055762":[],"13059483":[],"130694":33,"13069442":33,"13086825":[],"13092444":[],"131":10,"13107995":[],"13117805":[],"13134148011170077":[],"13137881":[],"131379":[],"13152828":[],"13162821e":[],"13164176":[],"13181493":[],"13186167":[],"13196676":[],"132":10,"13203156":38,"13206207781887236":29,"13217667":[],"13220608e":[7,34],"132441":[],"13244119":[],"1325":7,"1326":10,"13260573":[],"1326197715":32,"132756":7,"13280":10,"13286177":[],"13296813":[],"13298019":[],"133":[8,36],"1332":35,"13324158":[],"13326664":38,"13337992":[],"13355309":[],"13373501":[],"134":34,"13437168":[],"134534":[],"13453411":[],"13455019":[],"13457622":[],"135":10,"1350":35,"13518746":[],"13531906":[],"13535942":[7,34],"13564988":[],"1356995":[],"13590426":[],"136":[1,8,36],"13600596":[],"136066":[],"13607502":[],"13625288":32,"13646574":6,"13659941":[],"13661243e":[7,34],"13663125":[],"13675593":[],"136756":[],"13679863":[7,34],"13680048":[],"13684661":[],"13699574":[],"136996":[],"137":[1,8,36],"13706862":38,"13728124":25,"13729154":[],"13738633":[],"13740":10,"137652":[12,33],"1378267322316808":[],"13788033":[],"13791264":[],"13799037e":[],"138":[1,8,36],"13819592e":[],"13820":37,"13825388":[],"13845051":[],"13847167":[],"1385":7,"1385510301651882":[],"13858163":[],"13865173":6,"13866942":[],"138775":[12,33],"13878649":[],"1388888888888889":[2,40],"13890":10,"13896095":[],"139":[1,8,36],"13911934":[],"13913033":[],"139224":[],"1392559585020426e":[7,34],"139255958997547e":[7,34],"1393548":[],"13945974":[],"13956654":[],"13976073":[],"1399386":[],"13it":[],"14":[0,1,3,5,6,7,8,9,10,11,12,13,22,23,25,29,31,33,35,36,37,38,41],"140":[1,8,10,36],"14002608":[],"14021063":[7,34],"14029264":[],"14038959":12,"140390":12,"14043884":[],"14045682":[],"14046224e":[],"14084366":[],"141":[1,8,36],"14100":10,"14104212":[],"14112437":[],"14114294":[],"141182":[],"14118216":[],"1412":[22,37,38],"14122256":[],"14123881":[],"1416398":[7,34],"14174745":[7,34],"1418":10,"14187173":[],"14192454":[],"142":[1,8,10,36],"14213876":[],"142188":[],"14218838":[],"14239961":[],"1424112":[],"14249052":[],"14250":10,"14251309":[],"14260706":[],"14277718e":33,"14285244":[],"143":[0,1,3,8,10,36],"14309433":[],"14335140e":[],"14369927":[],"1437":[2,40,41],"14373934e":[39,40],"14383145":[],"144":[1,3,8,36],"14400":10,"14404536":[],"1440501043841336":[2,40,41],"1441031":[],"14424825":38,"14433067":[],"14437776":[],"14440":10,"14452065":[],"14452678":[],"14465374":6,"144654":6,"1446729567":5,"144848":[],"14484831":[],"14495737":[],"145":[],"14523337e":[],"14550854":[],"145509":[],"14559761":38,"14568914":34,"145753":12,"1457534":12,"14580867":33,"14599499":[],"146":3,"14605929":[],"14610161":[],"146141":[],"14629156":35,"14638":37,"14643345e":[],"14690374":6,"146904":6,"14697222":[],"14697641":[],"147":[],"14708":[],"14710":10,"14716":7,"14721582":[],"14722222222222223":[2,40,41],"147400":[12,33],"147420":[12,33],"14751649":[],"147545":[],"1477190177681485":[],"1477805":22,"147836":[],"14783612":[],"1479":10,"148":[3,4,5,15],"14812206":7,"14815202":[],"14824254":[],"14825560e":[],"14828178":[],"14829228":[],"14841322":[],"148477":[],"1486419":[],"148776":[],"14880915":[],"1488598":[],"14890012":[],"148913":[],"148932":6,"14893216":6,"149":3,"149003":7,"149063":[],"149111":[],"14913555":[],"149266":[],"14934":[],"14934258650797513":[],"149353":[],"149379":7,"149388":[],"149416":[],"149433":[],"149466":[],"149483":[],"14949972":[],"149524":[],"149904":[],"149921":[],"14g":[7,35],"14it":[],"15":[0,1,3,5,7,8,9,10,13,14,16,17,18,22,29,32,35,36,37,38,39,40,41],"150":[4,5,9,10,15,23],"15001903":[],"15005476":6,"15008902218011533":10,"1502":39,"150301":[],"150337":[],"15042293":[],"15052":[],"150527":[],"15058132343282243":[],"150735":[],"15079545":[],"15098090e":[7,34],"151":[4,5,15],"15126318":[],"15130074e":[7,34],"15143483e":[],"151522":[],"15173932":[],"151892":35,"15199":[],"152":10,"15200":10,"15202654":[],"15207119":[],"152149":[],"15215415":[],"15235148":[],"15242624e":[],"152494":[],"1527777777777778":[2,40],"15280951":[],"15283463":[],"1529":[],"15294186382924843":[],"153":[4,5,15],"153036":33,"15313054":35,"15338896e":[],"15351626":[],"15360063":[],"15361856":[],"15367403":[],"1537797":[],"15384615384615385":10,"15388662":[],"15390225":[],"154":[4,5,15],"15403275":[],"15422927":[],"15424679":32,"15435127":[],"15435937":[],"15451852":[],"154751":[],"1547556":[],"1548131":[],"15489318":[],"15493099":[],"155":[4,5,10,15],"15527976":[],"15546887":[],"15563816":[],"15572395":[],"155724":[],"15586219":[],"15593478":[],"155994":[],"15599422":[],"156":[4,5,15],"15605625":[],"15654145":[],"15670465":25,"15670921":37,"1567286":[],"15690073":[],"156956":6,"157":[4,5,15],"15709522":[],"157128":[],"1571284":[],"1575":10,"1575046":[],"15751952":[],"15759533":38,"1576057":[],"1579":35,"158":10,"15830902":[],"15833751":[],"15847501":[],"1587":10,"1590":10,"15905706e":[],"15906954":38,"15924557e":[],"15951871":37,"15970814":[],"1598058":[],"15990":10,"15991124e":[],"15am":32,"15g":[7,35],"15it":[],"16":[2,3,4,5,6,7,9,10,11,20,22,23,29,32,33,34,35,37,38,39,40,41],"16013842":[],"1602625":[],"1603":4,"16039162":[],"16043208":29,"16050357":[],"16063661":[],"16072855":32,"16111111111111112":[2,40,41],"16118615":[],"16119967":38,"16124569":[],"161246":[],"16131839":[],"16146139":25,"16147054":38,"16158074":[],"161581":[],"16172219e":[],"16180248":[],"16181877":32,"16188604":[],"16201555":[],"16209264e":[],"16211139":6,"16211781":[],"1621766238509487":12,"16220":10,"162246":6,"16226023":[],"16231451":5,"1625":10,"16257339":[],"16257872":[],"16271817":[],"16275926":[],"1628":10,"16292908":[],"16304863":33,"163049":33,"1630775253":[2,40,41],"16312434e":[],"16328315":[],"16342407":6,"16343471":[7,34],"1636508":[],"16373437":[],"16384":4,"16388664":[],"163887":[],"16404402":[],"16405708":[],"16407038":[],"16411163":[],"164112":[],"16413938":[],"16431956":[],"16436977e":[],"16437294":[],"16445858":[],"16452931":[],"16469725":[],"16486151":[],"16500":10,"165033577601387":[],"16513872":[],"165272":[],"16532977":[],"16535252":[],"16543524":[],"16552497":32,"16567361":[],"166":10,"1660497e":[],"16606924":[],"16617445":[],"16620159":[],"16622624":[],"1664372":[],"166540":[],"16654048":[],"16674251":32,"16702443":[],"16710567":[],"167116584998668":[],"16744759":[],"16777789":[],"167787":6,"16791183":[],"168":10,"16805821e":[7,34],"16879134":[],"16881787":[],"16913489":[],"16925937":[],"169299":[],"16949234":29,"16it":7,"17":[2,3,6,7,8,9,10,14,22,29,33,35,36,37,38,39,40,41],"17021424":[],"17026621":[],"17069366":[],"17080691":[],"170807":[],"17089674":[],"1709":10,"17093974":[],"171":34,"17101958":32,"171525":33,"1715252":33,"17158357":[],"171611":[],"17168028":[],"17174962e":[2,40,41],"17214545":[],"17214797":[],"1721774":38,"17222222222222222":[2,40],"17222494":[],"17239551e":[],"17246397":[],"17249725":[],"17253354":[],"1726":10,"17260271":[],"17272182":[],"17285183":25,"1729993":[],"173":[22,38],"17300":10,"1731":10,"1732863504708044":[],"17359583e":[],"17367225":[],"17377547":[],"17378802":38,"17382534":[],"17405911e":[],"17418673":[],"17446471":7,"17483760":35,"175":35,"17501258":[39,40],"1752":10,"175300":33,"17547294e":[],"17563274":[],"17567503":[],"17568084":[],"17569961":[],"176077701534979":[],"17629205":[],"17641709":[7,34],"17643975e":[],"17669259":[],"176762":[],"17676247":[],"1767857":[],"17714797":[],"17728993":[],"17739525":[],"17748881e":37,"17757295":[],"17777777777777778":[2,40,41],"177786562639708":[],"17780089":[],"1778106":[],"17801022":6,"17801827":[],"17806281e":[],"1780797":[],"17827738":[],"17834506e":[],"17840283":29,"17849836":[],"17856374":[],"17861098":7,"17861414":[],"17865868":[],"17869861e":[],"17882416":[],"17886464e":[],"179":[],"17913653":[],"17917768":6,"1792":[],"17934093":[],"1793417":[],"17934801":[],"1793636":[],"17941158":[],"17944696":[],"17948385":[],"17948638":[],"17949575":6,"17953942":12,"17968231":[],"1797":[2,4,40,41],"18":[0,3,7,8,9,10,11,14,22,23,25,29,35,36,37,38,40,41],"180":[],"18013978":32,"1801709":34,"18019623e":[],"18029127":6,"18038547":[],"18043393":[],"18052195":[],"1807":5,"1808228":[],"18085537":[],"1809":10,"18093549":[],"18094575":[],"181":10,"1810415856976313":[],"18106943":[],"18114057":[],"181141":[],"18118338":[],"1812":10,"18120973":[],"181210":[],"18146425":32,"18147009":[],"181766185664273":[],"18180717":25,"182086698929565":[],"1821":10,"18234308":[],"18234499":[],"18251883":32,"18274735":[],"18277213":[],"18285689e":[],"18307821":34,"18333333333333332":[2,40],"18383522":33,"18388":7,"184":10,"1840":35,"18409254":[],"184119":[],"18415031":[],"18418938":[],"18418988":[],"184519":33,"18464035":[],"18474924":[],"18485685":[],"184866696796263":[],"18497496":[],"184975":[],"185":34,"18501900e":[],"18510775":[],"18522369":[],"18551655":[],"185710":12,"18571046":12,"18572822":[],"1860":10,"18603548":37,"18604416":[],"18611111111111112":[2,40],"18613217e":[7,34],"1863889":[],"18646225":[],"18660":10,"18670072e":33,"18673098":12,"18702469":[],"18711407":[],"18712313":[],"18715465":[],"18725816e":[],"18742521":[],"18750351":37,"18757012":22,"18781225":[],"1880258":22,"18810873":38,"18816387":[],"18824338":[],"18840034e":[],"18850776":[],"1885906477601598":[],"18859065":[],"18864495":38,"18872206":[],"18925625":[],"189496":33,"18956157":[],"189622":33,"18981286":[],"18986837":38,"18it":[],"19":[3,7,10,16,22,29,32,35,37,38,41],"19001247":[],"19009085621304586":[],"19036055":12,"19052323e":[],"190529":[],"19052935":[],"19069576e":37,"19073442":[],"19126179":[],"19166666666666668":[2,40,41],"19180036e":[],"191833":[],"192":35,"19207979":6,"19220":10,"19299283":[],"19336703":[],"19350397125914334":[],"19364388":[],"19367324":[],"19371605e":[39,40],"19375828":[],"1940":1,"194042826649355e":[7,34],"194042826815211e":[7,34],"1943":[13,38,39],"19433318":[],"1944":7,"19447592":[],"194836":6,"19483625":6,"194889":[],"19488947":[],"19517243":[],"19517373":12,"195174":12,"19526866":22,"19532208":[],"19547923":[],"19559737e":[],"1956":10,"19569961":[7,33,34],"19583547":[],"19588651":37,"19601208":[],"19661197":37,"19677112":[],"1970":[25,32],"19725211":[],"1973":10,"197370":[12,33],"19740":10,"1974189":[],"1979":[7,35],"19791352":[],"197914":[],"19798258":[],"19800":10,"19800999":25,"19808709":[],"1982973034395375":6,"19848482":[],"19849513":[],"19852045":[],"19857342950283463":25,"19861706":[],"19884659":[],"1989":39,"1991":39,"19920353":[],"199324":[],"19932442":[],"19945294":[],"19951849":[],"19973189":[],"199824":[],"19982428":[],"1999":27,"1_1":[13,38],"1_2":[13,38],"1_3":[13,38],"1cm":[1,9,11,29,32,39,40],"1d":[2,3,4,40,41],"1e":[2,3,5,14,15,22,37,38,40,41],"1e10":15,"1e4":7,"1f":[2,40,41],"1k":25,"1n":[1,32],"1ocqm1tltwb6hzsf9i7ggugw9m8sbvequ":[],"1s":[40,41],"1x":[1,32],"1zkibvqf":23,"2":[0,2,3,4,5,6,7,8,9,10,11,12,13,14,15,20,22,24,25,26,28,29,31,35,36,37,38],"20":[0,1,2,3,7,8,9,10,14,20,22,29,30,32,34,35,36,37,38,39,40,41],"200":[1,3,4,5,9,10,11,41],"2000":[1,33],"20005403":32,"20015436":7,"20020163":[39,40],"2004":[14,36],"20040481":[],"2004542":[],"20051449":[],"2006":31,"2008":32,"2010":[2,39,40],"2011":[2,40,41],"20135691":[],"2014":5,"2015":[2,40,41],"20158083":[],"201581":[],"2016":[1,32],"2017":41,"20173175":[],"2018":[1,7,33,35],"2019":[9,10],"20194579":[],"2021":[7,15,33,34,36],"20218457":[],"2022":[27,32,39,40,41],"20224027":[],"20228273":[],"2023":[26,40,41],"2024":[0,16,17,18,19,20,21,22,23,27,32,33,34,35,36,41],"20263701":32,"202698":[],"2026985":[],"2027":10,"20277777777777778":[2,40,41],"20288581":[],"20290903":[],"20294638":[],"203":[],"20300793":[],"2032806":[],"20336784":[],"20347912":[],"20353821":32,"20357736":[],"20360":41,"20365164":32,"20367355":[],"20375464":[],"20393163":[],"20393534":[],"203963":[],"20396326":[],"20402019":[],"204049":[],"20404927":[],"20437065":[],"204488":[],"2044881":[],"20480187":[],"204802":[],"20500":10,"205116":[],"20511614":[],"2051784":[],"20519013":[],"20519189":[],"20520318":32,"20527785":[],"20536289":[],"20536579":6,"205366":6,"20539421":[],"205757":[],"20575734":[],"2060":10,"20601797e":[],"20607165":[],"20608548e":[],"20621146":[],"20640822":32,"20651434":[],"20656353":32,"206664":[],"20666447":[],"20680982":[],"2069":10,"20716644":[],"2072382":[],"20728198":[],"20742956":[],"20743662":38,"207545":[12,33],"20755569":[],"2077068":[],"20775115":[],"20793362":[],"20833333333333334":[2,40,41],"20834188":[],"2084894":[],"20867052175034223":[7,35],"20867716":[],"20898872":[],"20912557":[],"20915094":[],"20916295":33,"20980":10,"20986506":[],"21":[0,1,2,6,7,8,10,13,14,22,25,32,33,35,36,37,38,39,40],"210340":[12,33],"21047414e":[],"2105":41,"21058097":6,"21061589":[],"21070843":[],"2107546970521115":[],"21093775":7,"211":[],"21130":10,"2116753732":5,"21169159e":[7,34],"21181534":[],"21192602e":[],"212":[],"2127032":[],"21277816e":[],"213":[],"213103":[12,33],"2131067":[],"2133127":[],"21332621":[],"213336":[],"21333606":[],"21333766":[],"21344077":[],"213743":[12,33],"21390371e":[],"21392756":[],"21394125":[],"21407599":[],"2142385":[],"21430643e":37,"21432413e":[],"21444055":[],"21457724":[],"21501369":[],"21506004":[],"21517262":7,"21531156":[],"21565007e":[],"2157318":[],"21596432":7,"2160073466353432":[],"216171":[],"21617146":[],"216290":[12,33],"21631786":[],"21635399":[],"21641173":[],"21655059e":[],"216683":[12,33],"21680864":[],"21690241":[],"21699817":[],"2170":23,"2171":23,"2172":23,"21724894e":[],"2173":23,"2174":23,"21742484":[],"21752693":[],"21757908":[],"21776599":[],"21784616":[],"21786054e":[],"2180":[],"21805426":[],"2181":[],"21813899e":[],"21817966":[],"2182":[],"2183":[],"21833102":[],"21838482":[],"2184":[],"21846767e":[],"21894241":32,"21905519":[],"21914111":[],"21919019":29,"21941484e":[],"21949761":[],"21998973":[],"22":[1,2,6,7,10,13,14,22,25,32,33,35,37,38,40,41],"22004411":[],"22004496":[],"22013655":[],"22044605e":[6,33,38,39],"22087319":[],"221":9,"2211133":[],"221180":33,"2212296":[],"22128142":[],"22158725223474995":[],"2216":10,"221666864828611":[],"22169087":[],"2218":10,"221805":[],"221921":6,"22194497":[],"221945":[],"22217237e":[],"22226881":[],"222400":33,"22370461988004753":[],"22377412":[],"22388052":[],"2239":23,"2240":23,"2241":23,"2242":23,"22426265":32,"22462544":[],"22467274":35,"22480853":[],"224809":[],"22481166":[],"2249":[],"225":5,"2250":[],"2251":[],"2252":[],"22523018":[],"22528536":[],"22545286":[],"225453":[],"22569872":[],"2257449":[],"22592946e":[],"226014":[],"22601408":[],"22623421":[],"22625657":32,"22667819":[],"22688898":[],"22690428":6,"22706777":[],"227072949395066":[],"22707295":[],"22715479":[],"22718624":29,"22755839e":[],"22756518":[],"22773898":[],"227738984674172":[],"22783539":[],"22809974":[],"22810365":[],"228104":[],"228216":[],"22821607":[],"22830771":[],"22835717e":[],"22842468702219465":[7,35],"2284246870221947":[7,35],"228467858509994":[],"22846786":[],"22847924":6,"22849742":[],"22851508":[],"22871":37,"22896144":[],"22897303":[],"22919456":[],"22929165":25,"2293":[],"22936132":[],"2294":[],"22946401":[],"22959895":[],"229598952352728":[],"2296":[],"2297":[],"22974904":[],"22977659964799":[],"2297766":[],"22985907723809532":[],"22it":7,"23":[2,7,10,13,16,21,22,25,32,35,38,39,40,41],"23002365e":[7,34],"23006632e":[],"23007231e":[39,40],"230154":[],"23015412":[],"23038906":[],"23040825":[],"230760":12,"23076025":12,"23076923076923078":10,"23077255":[],"23093598e":[],"2310":41,"2311959":[],"23139242":[],"23142852":[],"23164401":[],"23167717":6,"23238973":[],"232435":33,"23262877":[],"23263884":[],"23303727":[],"23326473":[],"23333333333333334":[2,40],"23344210e":[],"233481":[],"23348144":[],"2338675":33,"233868":33,"234":7,"23407025":[],"23423987":[],"234332597247826e":[],"23449217":[],"234496":[],"23449607":[],"2345":23,"2346":23,"23469347":[],"23472994":[],"2348":23,"23481471":[],"2349":23,"23499549":[],"23499891":[],"2350":23,"2351":7,"23526911":[],"23528337":35,"2355":[],"2356":[],"23565337":6,"23570126":[],"2358":[],"2359":[],"235969":[],"23596934":[],"2360":[],"23616967":[],"2364":10,"23663482":[],"23692142":[],"2373286546935638":[],"23770713":38,"23794815":[],"23807169":[],"2383703":[],"23847441":[],"239":35,"23950884":[],"2397":10,"23971032":33,"23994188e":[],"23it":[],"24":[1,2,7,10,22,25,32,35,37,38,40,41],"24000075":[],"2401":26,"2401586":[],"24032019":[],"24033384":[],"24054861":[],"24071032":[],"24088952":[],"24106503":37,"24121476598003452":[],"24140":10,"24149532":[],"24157456":[],"24159785":33,"2416":10,"24164234":[],"24175744e":[7,34],"24185931":[],"24188439":25,"2419":10,"24200844":25,"2420287e":[],"24224776e":[],"24225043":[],"24254071":[],"24284399":[],"24299211":[],"2430":10,"24335376":[],"24344979e":[],"24346138":[],"24369936":[],"24377554":[],"243776":[],"24382159e":[],"24390":10,"24398826":[],"24412628":[],"24427624":[],"244293":[],"24429334":[],"24430166":[],"24433117":[],"24444444444444444":[2,40],"24454968":[],"24485814":37,"24493666":35,"24508783":[],"24512233":[],"24524426":[],"245781":[],"24578123":[],"24582593":[],"2458435":[],"246":[],"24602146":[],"24613581":[],"24629299":[],"24640494":37,"2468382":[],"24688853":[],"24692014":[],"24719968":[],"2475908482537097":[],"24759085":[],"24768421e":37,"24777879":[],"24780663":[],"24816468":[],"2481902":[],"24829629":25,"24829908":6,"24848631e":[],"24856992e":[],"24874378":[],"24906604e":[7,34],"24911588e":[],"24999088":[],"24it":7,"25":[3,4,5,6,7,9,10,12,22,26,33,34,35,37,38,39,41],"250":[3,5,8,10,36,41],"25000":1,"25003261":[],"25007671":[],"250077":[],"25018094":[],"25034533":25,"25068519":[],"25095067":25,"25106936":[],"25111821":[],"25145374e":[],"25185716":34,"251879":33,"25197648":[],"252008":[],"25200841":[],"25206089":[],"25206371":[],"25233253":[],"252333":[],"252436":33,"2527027":[],"252703":[],"25294371":[],"25299716":[],"25337053":[],"25343155":35,"25345918":[],"25354173":[],"25360039":[],"25386687":[],"253891":[],"25389111":[],"25444295":[],"25445":6,"254450":6,"25458506e":[],"2546186":[],"254619":[],"2546817701709956":[],"255":4,"255001":33,"25509222":[],"255107903576545":[],"2551096333979939":[],"25512598":[],"25555619":[],"25568695":[],"255888":6,"25588843":6,"256":5,"2561719":22,"25617655e":[7,34],"25617658e":[7,34],"25646067":[],"25659561":[],"25673276":[],"256733":[],"25680702":[],"25695398":35,"257":[],"25705685":[],"2572":10,"25729558":[],"2575":10,"25767462":22,"25816519":[],"25820894":[],"25849623":[],"25868261":[],"25879315330607":[],"25904489":[],"259153":[12,33],"25915746":38,"25934872":[],"2597":10,"25970297":[],"25982474":[],"259825":[],"25986026":[],"25989189":[],"26":[7,10,17,22,35,37,38],"26001134":[],"26024513":[],"26028795":[],"26048873":[],"26083712":[],"26090013":[],"26113912":[],"26120167":[],"26124692":7,"26128189":[],"26152075":[],"26188646":[],"26191905":[],"26196737":[],"26240613":[],"26254957e":[],"262648":[],"26264849":[],"26280614":[],"26281735":37,"26292364":35,"26292935":32,"26297455":33,"26301436":6,"26307235":[],"2630723514841553":[],"26310762":29,"26321660e":[],"26378176e":[],"26402837":[],"26409315307910025":[7,34],"2640931530791004":[7,34],"26421616":[],"26443391":[],"26464303e":[],"26479234":[],"2650":10,"265109911":5,"2652667":[],"26527089":[],"26536626e":[],"2654":10,"26571335":[],"265868":[],"26586817":[],"26590299":25,"26604611":[],"2662452":[],"26638638":[],"26639089e":[],"26646852":[],"2666905":[],"26674792":[],"26675647":[],"26680052":[],"266801":[],"26692247":[],"26710969":6,"26715191":[],"26767021":[],"26775644":[],"26780278":6,"26781531":38,"2679757":[],"268":10,"268142":[],"26814205":[],"26814548":7,"26817703":[],"26821199":[],"26835242":25,"26897875e":[],"2694078":[],"26953857":[],"26956051":25,"26969649":[],"26995402":33,"27":[1,2,7,21,22,25,35,38,40,41],"27037635":[],"27065633":34,"27097785":[],"270978":[],"27108866":[],"27124938":[],"271260217275557":[],"27126022":[],"27182417":[],"27186049":[],"27223854":22,"27227895":[],"27231287":[],"2729933":[],"27336592":[39,40],"27352535e":[],"27403356":[],"27410687":[],"27425372":[],"274394546800107":[],"27439455":[],"27458376":[],"27465664":[],"27470622":37,"27482829e":[],"2750":10,"2751616":[],"27530242":[],"27597777e":[],"27600259":[],"27602210e":[],"27623637":[],"276263":[12,33],"27633474":[],"2765068":[],"2766892":[],"27677742":[],"27693602e":[38,39],"27698328":[],"27700":10,"27750534":[],"27760":10,"27788872":38,"2781456":[],"27861634e":[],"27874136":[],"27906869":[],"27916593":6,"279166":6,"27924636":6,"27933134":[],"2793874":[],"27952294":[],"27968813e":38,"27993904e":[],"27it":7,"27n_":29,"28":[2,4,5,7,10,22,35,37,38,40,41],"28000355":[],"280004":[],"28021598":[],"2802234e":[],"280464e":[],"280573":6,"280647":[12,33],"2806901924685401":[],"2807716628772665":[],"28078417":32,"2809076":[],"28093636":[],"28098147":25,"28109198":[],"2813829":[],"28158348":[],"281627":6,"28162714":6,"28164443e":[],"28183460e":[],"28192769":[],"28205578e":33,"28210156":[],"28225179":[],"2822642":[],"28244905":[],"28253033":[],"282727":[12,33],"28276275":[],"28292756":[],"28306304e":[],"2830637392":5,"28316949":[],"28336218e":[7,34],"28341218":38,"28341932":25,"28347758":6,"283478":6,"28358004":[],"28364423":[],"28365323e":[],"28366671":[],"2837242622462304":[],"28377068":38,"28390":10,"28391978":33,"28399125":[],"28415767":[],"28443039":35,"28457291":[],"28472162":[],"284722":[],"28475098":9,"28496213":[],"285056":[],"28505619":[],"28519494":[],"28542801":[],"286038":[],"28603828":[],"2861":29,"28614301":[],"2863391":[],"28641033":[],"28645331e":[],"28709718":[],"2871":10,"28726748":[],"2873":10,"28754232":[],"2876045":[],"28766899":[],"28776411":[],"28793787":[],"2882":29,"28833782":[],"28836035":[],"28844641e":[],"2885519":[],"2886":29,"288851":[],"28885103":[],"28886314":[],"2890":[1,32],"28902948":[],"2890577":[],"2892":29,"28923":[],"28924386":[],"28956161":[],"2895786617507":[],"28963909":[],"28973035":[],"28it":7,"29":[7,10,22,35,37,38],"29015871":[],"29030256":[],"29033075e":[],"29078236":[],"29087396":[],"290874":[],"29089154":[],"29091211":[],"29100624e":[],"291056":[],"29107892":[],"29134342":25,"29142853":[],"29147052":[],"2915":29,"29155617":22,"29162184":[],"29167186":6,"29201128":[],"29223521":[],"29234681":[],"29236762":[],"29238145":[],"29253213e":[],"29306584e":[],"2930953":[],"2931":32,"29350":37,"2936657":[],"293665700387852":[],"2938162":32,"29390668":[],"29398019":[],"294":[],"29411652":[],"29414225":[],"29419924":[],"29474356":[],"29491144e":[],"295":[],"29509520e":[],"29541992e":[],"29560433":[],"29561686":[],"29593305":[],"29620357":[],"29627774":[],"29633608":[],"2964378":[],"29645662":[],"29653038":37,"2965315":[],"2965479984453907":[],"29669411":[],"29678339":[],"2968":32,"29687889":32,"297":[],"29716773":37,"29738252":[],"2974512":38,"297476846305301":[],"29747685":[],"297581":[],"29758118":[],"297648":[],"29764815":[],"2976637":[],"29777705":38,"298":[],"2980":[23,32],"298037":[],"29803738":[],"29822833":[7,34],"298499":[],"29849933":[],"29860173":37,"2986082":[],"29861395":[],"29866395":[],"29881308":[],"299":7,"2990":32,"29902518":[],"29950088":[],"299748":33,"29995377":[],"299954":[],"29it":7,"2_":[13,38],"2_1":[13,38],"2_2":[13,38],"2_3":[13,38],"2_i":[13,38],"2_m":[7,29,35],"2_t":[14,37,38],"2_x":29,"2b":29,"2c8f433990d1":[37,38],"2cm":9,"2d":[2,4,12,13,24,32,38,39,40,41],"2e":[7,35],"2f":[0,1,8,10,11,12,13,32,33,36,38,39],"2g":[3,41],"2g_i":[3,41],"2k":4,"2m":[7,35],"2n":[1,3,4,32,33,41],"2nd":[10,38],"2p":[29,39],"2pm":30,"2pt":5,"2x":[1,4,9,14,32,37,38,39],"2x_ix_jy_iy_j":9,"2x_j":9,"2xb":39,"2y_i":11,"2y_j":9,"3":[2,3,4,5,6,7,8,9,10,11,12,13,14,15,18,19,22,24,25,26,27,28,29,30,34,35,36,37,38],"30":[0,1,2,5,7,8,10,11,14,17,22,30,33,35,36,37,40,41],"300":[22,38,39],"30000":[1,32],"30000613":[],"3000498":32,"30005983":[],"30010":10,"30013135":[],"300231":[],"30023144":[],"30043041":[],"30054056":[],"30069427":[],"3008836":[],"30117681":[],"30119421":9,"30124108":[],"30134885":[],"30136959":[],"30141906":[],"30190897":[],"301909":[],"30191327e":[],"30196557e":[],"302":35,"30258509":[25,32],"30269539":[],"30283188":34,"303":7,"30306759":37,"30322722":[],"30332201":[],"30334592e":[],"30387411e":[],"30400113":[],"30442002e":[],"30442964":33,"30450049":[],"30458085":[],"30466214e":[7,34],"30493151":[],"30506075e":[],"30593976":25,"30607268":37,"30616802":[],"30633304":[],"30637636":[],"30655251e":[],"30675215e":[],"3068659":[],"306866":[],"307":[],"3072":4,"3075371":[],"3076923076923077":10,"30782691":[],"30787294":7,"30887484":[],"30894285":[],"30897153":[],"308972":[],"309":[],"30915293":[],"309153":[],"30916402":[],"30916988":[],"309170":[],"30917499e":[],"30940":10,"30959659":[],"3096154":[],"30982436":[],"3099":35,"30it":7,"31":[13,25,29,38],"31008026":[],"310579":33,"3105791":33,"31068981":25,"310957":[],"31095745":[],"31096156":[],"31159615":32,"31160886":[],"31162122":[],"31175179":[],"31186409":25,"312":[],"31210861":[],"31216736":25,"31227657e":[],"3123314713548606":[7,35],"31248425":37,"31276579e":[7,34],"31286143":[],"31294131":[],"31299668e":[],"313":[],"313155":[],"31315504":[],"31318084":6,"31330137":[],"31335664":32,"313568":[],"31356849":[],"31362428":[],"31363366":[],"31367041":[],"31370028":[],"31381731":[],"31436277":[],"31453949":[],"31457796":6,"314869":[],"31486917":[],"315":[7,34],"31541884":[],"3155":[1,6,7,34,35,36],"31555734196707536":[],"31562008":[],"31568288":[],"315683":[],"31571995":[],"316":[],"31601423":[],"31607059":[],"31639624":[],"31664443e":37,"31702609":[],"31711533":[],"31718909":12,"31727523":[],"317367":12,"31739522":25,"31743443":32,"31749096":29,"31758645":[],"31764285":38,"317799":[],"31779928":[],"31787864e":[],"31799354":[],"31812561":[],"31849289":[],"31876431e":[],"31888254":32,"31896852":9,"31897935":[],"31903447":[],"31925135":[],"31936237":[],"31938579":[],"3194467":[],"31950286":[],"31971976e":22,"31973052":[],"31983943":[],"31it":[],"32":[4,5,7,9,13,14,25,29,37,38],"3200":[2,40,41],"32003025":[],"32007041":[],"32052031":[],"320656":[],"32065614":[],"32068736":[],"3207":35,"32084559e":[],"32089655":[],"32114535":[],"32128064":[],"32149601703519115":[7,35],"3214960170351912":[7,35],"3215":10,"3218791":[],"32232922":32,"32262934":37,"32265127e":[],"323059":[],"32305911":[],"323066643161066":[],"323184775146844":[],"32318478":[],"32320616":[],"32329897":[],"32336220689648504":[],"32354671":22,"3236445":[],"32393078":[],"324":[],"32411274":[],"32419549":[],"3241988":[],"32422234e":[],"32447198":29,"3245":[],"32450054":33,"32466925":[],"32493877":[],"3250":[2,40,41],"32513272":[],"325228546249804":[],"32522855":[],"3254":3,"3255":3,"3256":3,"32567311":[],"3257":3,"3258":3,"32588151":22,"3259":3,"3260513":[],"32606139":[],"32606365":[],"32616922":[],"3262":7,"326238":33,"32641434":[],"32641558":[],"326430":[],"32643011":[],"3265097":[],"32651488":[],"32670322":[],"32683063":[],"32684035":[],"32701708":[],"32717946":[],"32733949":[],"32738537":[],"32786329":[],"32798457e":[],"32799141":[],"3280928":[],"328093":[],"32825214":[],"328462040651484":6,"3284789":[],"32906581":[],"32932635":[],"32941912":[],"32962444":[],"32972613":[],"32it":[],"33":[10,13,25,30,37,38],"330":34,"33000573":[],"33006628":[],"33025801":[],"33027933":[],"33045013":[],"33046180e":[],"33049619":[],"33062878":[],"330629":[],"330660516571777":[],"33066907e":[6,33],"33081484":29,"33095972":[],"33096549":[],"331":34,"33107417":[],"331075":[],"33145794":[],"33149531":[],"33163086":[],"331631":[],"33165194":38,"33166055e":6,"33180288":[],"331939":33,"33212971":[],"33217312":[],"33220123":[],"33235647":[],"33241952":[],"33245403":[],"33266422":[],"33282365":22,"333":[8,34,36],"33300231":[],"33329216":[],"33357783":37,"334":34,"33433563":[],"33435637":[],"33477019":[],"33492417e":[],"335144":12,"3351443":12,"33519245":38,"33528448":[],"33532886":[],"335708":[],"33570831":[],"33596404":[],"33604208":[],"33615742":[],"33675308":[],"33700495":[],"33718234":37,"33734605":[],"33742926":[],"33745439":[],"33826500e":[],"33871003":[],"3388112":[],"3391965479837245":[],"33978889":37,"33it":7,"34":[10,25],"340071371496255":33,"34023682":[],"340237":[],"34026086":[],"3403":10,"34031551":38,"3404509":[],"34061974":[],"340620":[],"340782":[12,33],"34098895":[],"34112554":[],"34114547":6,"34121963e":[],"34131583":[],"34134773":[],"34139775e":[39,40],"34182884":[],"34205649":[],"34225662":[],"34249217":[],"342680":33,"34271514":[],"34332677":[],"34346829":[],"3436":[1,32],"34367970":7,"3437":[1,32],"3440539":[],"34412923":33,"344176915013831":[],"34427423":[],"34478594":[],"34483937":[],"34483955":[],"34488863":25,"34549184e":[],"34554342":[],"34568909":[],"34569596":6,"34569632":[],"34577375":[],"34582863":[],"34598722":[],"34599467e":[],"34599598":[],"346000154268618":[],"34601775":[],"34603176":[],"34620097":[],"34654596":[],"346546":[],"34661376":[],"346614":[],"3466389":32,"34704885":[],"34726166":[],"347262":[],"34732758":[],"347328":[],"34759233":[],"3478095":[],"3480527":[],"34832197":25,"34851141":[],"34862407":[],"34871778":[],"34904819":[],"349312":[],"34931214":[],"34951108":[],"3499634874668227":[],"34it":[],"35":[1,7,18,26,30,32],"3500312e":[],"35018806":[],"35098705":[],"35140":10,"3514835":[],"351636":[12,33],"35182854":6,"35195241":[],"35196964":[],"35243372":[],"35268457":12,"352685":12,"35275513":[],"35282003":[],"35313779":[],"35320952":[],"35354073":[],"35388766e":[],"35432563":[],"354360":12,"35436034":12,"3544313922":7,"35468713e":[],"35470445e":[6,33],"35506347e":[],"35520169":[],"35533773":7,"35537114e":[],"355734685106391":[],"35587321":[],"35595337":[],"35613536e":[],"356399":33,"356673":[],"35667342":[],"35689303":[],"35713539":[],"35714377":[],"35731585":[],"35736":37,"357508":33,"3577615":[],"35785072":[],"35798999":[],"357990":[],"35806319":[],"3581341341":5,"35823372e":[],"35826030e":[],"3583246":[],"35870701":[],"3587467":[],"359":[6,34],"35914932":[],"35921835":[],"35956676":[],"35957915":[],"35966688":[],"35980374":[],"35986305":[],"35it":[],"36":[1,6,7,26,29,39,40],"360":[2,40,41],"36014743":[],"36031586":[],"36091273":[],"36116396":[],"3613":10,"3615115":32,"361541":[],"36154104":[],"361556":33,"36161658":[],"36185989":[],"3619775819765686":[],"36209091":[],"3621311":6,"36219301":[],"36235052e":[],"36255393":[],"36278226":[],"36303682e":[],"3632959111950474e":[7,34],"363295916366933e":[7,34],"36339156":[],"36353321":[],"36388542":[],"36401981":[],"36403018e":[],"36439409e":[],"36482524":[],"36490166":22,"36545697e":[],"3655222":6,"36579128":[],"36623476":[],"366375489616074":[],"36658646":[],"36694417":25,"367":[3,10],"36754671":[],"36790471":[],"36841864":[],"36845893":32,"36846593":29,"36880327":[],"368818":[],"36881837":[],"36889994":22,"36895644":35,"369139":[12,33],"36919318":[],"36it":[],"37":[10,26,36,37],"37007195":[],"37008469":[],"37013105":[],"37023353":29,"37023448":22,"37024276":[],"370243":[],"37055062":[],"37057274":[],"370782966":5,"37081457":[],"37093787":[],"37132543":[],"37135473":[],"37137157":[],"37153115":[],"37175575":37,"37175588":[],"37180834e":[],"37215356":[],"37239927e":33,"37255873":[],"3727791":[],"37293361":[],"372934":[],"37343957":[],"37354121":[],"37363888":6,"373639":6,"37396662":[7,34],"37415625e":[],"37449168":[],"37454707":[],"374675":[],"37467515":[],"37468967":25,"37507799e":[],"37532078":[],"37564555":[],"37582891":[],"375829":[],"37600844":[],"37600869":[],"37685349":[],"37700967e":[],"37705573":25,"37717637":[],"37730061":[],"377372":33,"37737221":33,"3777801602":7,"378271":[],"37827145":[],"37837864":[],"37857846":[],"37900111":[7,34],"37919989":[],"38":[10,26,29],"380":10,"3807176":[],"380718":[],"38102552":[],"38123077":[],"38135733e":[7,34],"38155181":29,"3817475779":[7,35],"38198866":[],"38203725":[],"38240257":[],"3824865":[],"38259084":37,"38298234e":[],"38309751":[],"38337498":[],"38381781":[],"38399622":[],"38422061":[],"38440469":[],"38445171":[],"38447892":[],"38460453":[],"38461538461538464":10,"38461539":37,"38506982":38,"38535182":[],"38571859":[],"38592549":[],"386":10,"38612299e":[],"386161":[],"38616125":[],"38618024":[],"38618158":[],"38629436":[25,32],"38675231":[],"38687011":[],"387":35,"38713573":[],"387620":[],"38762019":[],"387672":[],"3881343":[],"3882273":[],"38827994":7,"38853437":[],"38865681e":[],"38916861e":[7,34],"3892":35,"38934047":[],"38943":[],"389430":[],"38949027":[],"38962192e":[7,34],"38992371":[],"38it":[],"39":[1,10,27,30,32,38,39,41],"390334838933238":[],"39033484":[],"39039235":[],"3908470004999506":[],"39090680e":[],"3910042":[],"39184027":32,"39191126":[],"39197698":33,"391977":33,"39214397":33,"392144":33,"39223923":[],"39225935":[],"3924501":[],"39279291":[],"39292847":[],"39376038":[],"39386718":[],"3939184e":[],"39398622":[],"394":10,"39401392":[],"394505933":37,"39503373":25,"39516594":[],"39517073":[],"39525059":[],"39529587":[],"39540235":[],"39579407":6,"39590571":[],"395980256477532":[],"39598026":[],"39614949":[],"39621036e":[],"39646454e":[],"39692815":[],"39697001":[],"39706038":6,"39730902":25,"39741875":32,"397700":[12,33],"39789527":[25,32],"39799981":[],"3983":7,"39855569":[],"39865452":[],"39914635":[],"39919362":25,"39922082":[],"39929428":[],"39931051e":33,"39941008":[],"39952356":[],"399533503407795":[],"399836":33,"39991048":[],"39it":[],"3d":[3,4,5,7,14,26,35,37,41],"3f":[2,4,10,40,41],"3n":25,"3x":[3,9,41],"3x_0x_1":39,"3x_i":[3,41],"3y":9,"3yk470mj5p931p9dtkk0y6jw0000gn":[2,7,32,35,37,40,41],"4":[2,3,4,5,6,7,8,9,10,11,12,13,14,15,18,22,25,26,29,32,34,35,36,37,38,39,41],"40":[2,7,10,26,27,30,32,35,39,40,41],"400":5,"4000":[10,32],"40012736e":[],"40030606e":[],"40037367e":[],"40058341":[],"40072098":[],"4008393":[],"40093324":[],"40101943":[],"40140374":[],"401842":[12,33],"40184681":[],"40209956":[],"40266454":[],"40310129":[],"40354798e":[],"40362686":[],"40415979e":[],"40418675":[],"40431935":[],"40434293":[],"40447213":[],"404783":[],"40478305":[],"40491159":[],"405":10,"4050":[31,32],"405163":[],"4051634":[],"40558184":[],"405890":[12,33],"405964":[],"40596423":[],"40605118":[],"40615633":[],"40620066":33,"406201":33,"40632663":[],"40640797":[],"4069157":[],"407":10,"408":10,"40800396":[],"40833375":[],"40848657":[],"4087":10,"4087793":6,"40903389":[],"40906565":[],"40913138":29,"409131380658218":29,"40927184e":[7,34],"40961688":[],"4096949":[],"40it":[],"41":[1,10,25,27],"41062015":[],"4107":10,"411":[],"41146753":[],"412":[],"4121966338442804":[],"41229845":[],"41259788":[],"41298822":[],"413":[],"41310208":[],"41366128e":[],"41377873":[],"41381309":[],"41388745":[],"41396339":[],"41408221":[],"41416175":[],"41433969":6,"4143542":[],"41452482":[],"41487642":[],"4150026":[],"41511965e":[2,40,41],"41524442":[],"41532521":[],"4155":[3,16,41],"415597":[],"4155974":[],"4162706317":7,"41636775":[],"41664613":[],"4166666666666667":10,"41675053":[],"41705228":[],"41744203":[],"41764479":[],"417644792969273":[],"41788797":6,"417888":6,"41794469":[],"41820116":[],"41842093e":[],"41847686e":[],"418506":[12,33],"4185532":[],"418596857594715":[],"418656":[],"41866748":[],"41869784":[],"41882037e":[7,34],"41925189":[],"419718":[],"41971814":[],"41987877":[],"4198932":[],"42":[2,5,9,10,11,25,27,41],"42013616e":[],"42045168":29,"42083463e":[],"42095725":[],"421":[],"42124182":[],"4214275":[],"4215795":[],"42197297e":[],"422":[],"423":[],"42305151":[],"4230769230769231":10,"423102":[],"42310878":[],"42338272":29,"423646":[],"42364634":[],"42392359":[],"42394588":[],"424274":[],"4242743":[],"42439452":25,"42441033":6,"42450":10,"42497724":[],"42501752":[],"4252885":[],"42553096":[],"42575338":[],"42590512":25,"42592679":[],"426":[0,7,10],"42644801":[],"42670187":[],"42679392":[],"42699034":[39,40],"42772604":[],"42774546":[],"42793344e":[],"42813351":[],"42839645e":[],"42841423":[],"42873361":[],"429":35,"42914251":[],"42945319":[],"43":[0,1,2,8,10,25,36,40],"43007284":[],"4303589":22,"43054282":6,"4310":32,"4313428":32,"43167881e":[],"43208935911369845":32,"43258818":[],"43285487":[],"433":35,"43330971e":[7,34],"43373107":[],"43418708":38,"43492736":[],"43506123":25,"435163":33,"4353359":[],"43563433":[],"43579948e":[7,34],"43639284e":33,"436462435":5,"43703545":[],"43713984":32,"43724406":[],"43727575":[],"43754843":[],"43766686":12,"43779698e":[],"438":[4,5,15],"438060758":7,"438136":33,"43836612":25,"439":[4,5,15],"4390092":[],"439230":7,"43951798e":[],"43962226":[],"43976975777097144":[],"43984553e":[],"43it":[],"44":[1,2,25,40,41],"440":[4,5,15],"44007188":[],"44089210e":[6,33],"4408921e":[],"441":[4,5,15],"44108308":[],"44129766":[],"44137667":[],"442":[4,5,15],"44203729":[],"44239442":[],"44255455":[],"442600":[12,33],"44286735":[],"443":[],"443217":33,"44384079":38,"44395301":34,"444":10,"44418124":37,"44440947":[],"44454123":[],"44456367":[],"44476964":[],"444770":[],"44498054":[],"445":[],"44512218":[],"445128":[],"44536331":[],"44563226":[],"446033":35,"44624525e":33,"44658658e":[],"44756744":29,"44767398":[],"4477214":[],"44830934":[],"44832989":25,"44838546":[],"44845593":[],"44867801":[],"4488194":25,"44895662":[],"44970586e":[2,40,41],"44995924":[],"44it":[],"45":[10,30,32],"450":10,"45007578":[],"450257":[12,33],"4504":10,"45052717":25,"45053025":[],"45054585":[],"45104086":[],"45170202":[],"451702023099923":[],"451834":[],"45183405":[],"45193407":[],"45273617":[],"45397592":[],"45403192":[],"45408759":[],"454126":[],"45412633":[],"45430236":[],"45457427":[],"455173":35,"45538713":25,"45560415":[],"45568883":[],"45572817":[],"45576546":[],"4557763":12,"455947":33,"456":10,"45609936e":[],"45620975":[],"45647803":[],"4565147":[],"4565685":[],"457":[],"45709724":[],"45730151":[],"457311":[],"45731144":[],"45731379":[],"4576811":[],"45781905e":37,"457962":12,"45796224":12,"458027":33,"458078":[12,33],"4580854":[],"45840653":[],"45847011":[],"4591127":[],"45915671e":33,"45921927":[],"45960079":6,"45993741":[],"45it":[],"46":[3,10,23,30,32],"460":[0,10,12],"4601":10,"46049816":[],"46053947":25,"46065811":[],"460688958585373":[],"46068896":[],"46069873e":[],"46086448":[],"46107913":[],"46134248e":[],"46153846153846156":10,"46166231":[],"46170878e":37,"4619526":[],"462":[8,36],"462295":[],"46229544":[],"46242424":[],"46318204":[],"46342934":[],"4635149":[],"4636751":[],"46383925e":[7,34],"46383926e":[7,34],"46387965":[],"46415522":[],"46415888336127775":34,"4644459":29,"46544987":[],"46593404":[],"4659454":[],"46660941e":[],"46713267":34,"46736215":35,"46736822":[],"4676059":33,"467606":33,"46796873":25,"468":41,"46891823":[],"46893501":32,"46911594e":[],"46954702e":[],"46963249":[],"46984559":[],"46984697e":[7,34],"47":[3,10,23,30,32],"47018223":[],"47042744":6,"47051432":35,"47052809":[],"470714":33,"47075725":7,"47116868e":[7,34],"47123617":[],"47125748":6,"47130422":25,"47132891":6,"47187403":32,"47193243":[],"4725246793626128":[],"4729218":[],"47297587e":[],"47308061":[],"47313680":35,"47334547e":[],"4735968":[],"473730000968826":[],"47382705":[],"4742433":32,"47433659":[],"4745474":[],"47478922e":[],"47493254":[],"47544463":[],"47609206":[],"47610036":7,"4761747":[],"476849792":37,"47729832":[],"477405":12,"47740516":12,"47757553":[],"47767335":[],"47815203":12,"47823856":37,"47830372":32,"47851966":[],"47865692":[],"47892661":[],"47900239":32,"47905601e":[],"47910545":[],"479465113":5,"47989224":[],"47991892":[],"47it":[],"48":[3,10,23,35],"48004784":[],"48017159":[],"48062913":[],"48121459":[],"481215":[],"48138036":[],"48151166":[],"48170743":[],"481979":7,"48200792":[],"48233952":[],"482495":[],"48249527":[],"4825169":[],"48257387":[30,32],"48268005e":[],"48306473":[],"48307405":[],"4831139":[],"48326853":[],"48333419":[],"48364163":[],"483642":[],"48399942":[],"48411213":[],"48420061":[],"4843772":[],"48441798":[],"48442601":[],"48455788":[],"48476997":12,"4848097":[],"484810":[],"484905":[],"48490503":[],"48494803":34,"48511333":[],"48521055":[],"48549903":[],"48550":41,"4859188":[],"48597324":22,"48697869":[],"486979":[],"48705697":[],"4870934":[],"48717555":38,"48786826e":[],"48788378":[],"48805429e":[],"48826894e":[],"48827791":[],"48843174e":[],"48853875e":[],"48867802":[],"48875568e":[],"48891141":29,"48900413":[],"48904404":[],"48931464":[],"489315":[],"48944878":25,"48994188":6,"48it":[],"49":[6,7,10,12,26,33,37,38],"49008744":[],"49018385":[],"49035757":[],"49074536":[],"49077038e":[],"49123502e":[],"49152":4,"49156266e":[],"49211524":[],"49224801":[],"49285045e":[],"49312535":[],"49342785":[],"493428":[],"49361065":[],"4937346":[],"49381549":[],"49386718":[],"49392067e":[],"4940954":[1,32],"49470005":[],"49503632":25,"49513019":[],"4951301940723654":[],"49529361":[],"49557088":[],"49584063e":[],"4959161509357395e":[7,34],"4959161509366685e":[7,34],"49608088":[],"49611541":[],"49742477":[],"49761526":[],"49768808":25,"49849206e":[],"49888671":[],"498887":[],"49897504e":[39,40],"4990":29,"49910481e":[],"4992":29,"49939493":[],"499394932852416":[],"4997":29,"49it":[],"4c4c7f":[10,11],"4d":4,"4f":7,"4pm":[30,32],"4y":9,"4y_i":11,"5":[0,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,22,25,26,29,32,33,34,35,36,37,38,39,40,41],"50":[0,2,3,4,5,7,8,9,10,11,14,22,32,33,35,36,37,38,39,40,41],"500":[2,4,5,7,10,11,14,35,36,37,38,40,41],"50000000e":[38,39],"50000455":6,"50000553":6,"50000718":6,"50000855":6,"50000969":6,"50001063":6,"50001142":6,"50001207":6,"50001261":6,"50001306":6,"50001343":6,"50001374":6,"500014":6,"50001414":6,"50001422":6,"50001439":6,"50001454":6,"50001466":6,"50001476":6,"50001485":6,"50001492":6,"50001498":6,"50001502":6,"50001506":6,"5000151":6,"50001512":6,"50001515":6,"50001517":6,"50001518":6,"50001519":6,"50001521":6,"50001522":6,"50001523":6,"50001524":6,"50001525":6,"50029371":[],"50100007":[],"50152186":[],"5018":29,"5020612":[],"50227564e":[7,34],"50233673":[],"50258788e":[],"502698":[],"50269821":[],"50306335":[],"50321091":6,"50327198":[],"50352245":[],"50414881":12,"50427787":35,"50429483":[],"504295":[],"50482139":[],"50505108":25,"50516278":[],"50530832":35,"50558976":[],"5057642":29,"506":1,"50600514e":[],"50610071e":37,"50638863":[],"506389":[],"506553":33,"50655336":33,"50655379":[],"50667812":[],"50714096e":[],"50744967":32,"5076963":[],"507d50":[10,11],"50846111e":[7,34],"50846112e":[7,34],"50850373":[],"50861923":[],"5086728":[],"50887105":[],"5088893":25,"50895251":[],"50911289":[],"50937003":[],"5095016":[],"50it":[],"50j":[14,37],"50x10":[2,40,41],"51":11,"510":[2,40,41],"51019911":32,"51045268":25,"5105332296929628":[],"51069279":[],"51069748":[],"510697484548":[],"51104913":[],"51136852":[],"51154388":[],"5116375":[],"51164347":[],"511888":6,"51262599":[],"51279028":[],"51279273":[],"5128288":[],"51302994":[],"51317506":[],"51364117":[],"51374050":35,"514219":33,"5142856":[],"51437058e":[],"51447004":[],"51470773":[],"514708":[],"51495078":12,"514951":12,"51512235":[],"51532469":[],"51546825":[],"51553626":[],"51561341":[],"51564682":[],"51611732":[],"51615779":[],"51643579":[],"51659846":37,"51690815":[],"51698843":[],"51709529":[],"51713064":22,"51728978":[],"5177783846":5,"51808934e":[39,40],"51857188":[],"51858116":[],"51860019":25,"5187643":[],"51914183e":[],"51928402":[],"51936974":[],"519370":[],"51937727":[],"51it":[],"52":[36,37,38],"52012715":[],"52031552e":[],"52038652":25,"52074297":[],"52087499":[],"52121734e":[],"521422502348998":[],"52154966":32,"521558":[],"52155823":[],"52173381":[],"52200578":[],"52211567":[],"522116":[],"52212442":32,"5222222222222223":[2,40,41],"52228066e":[],"52229337":[],"5223261":[],"52284329e":[],"52301447e":[],"52317963":[],"52359414e":[],"52362157e":33,"5238548":[],"52397611":[],"52407775":[],"52446101":[],"52480767":[],"52488688":[],"52488859":[],"52568928":[],"5257011":[],"52573336":[],"52606333":[],"5260801":[],"52621829":[],"52634209":12,"52635761":[],"52637646":[],"52654886":[],"52657934":[],"5267331":29,"526744":[12,33],"52694246e":[],"52698542":[],"527024":[],"52702419":[],"52706754":[],"527068":[],"52721412":[],"52729975":[],"52732966":[],"52757272":[],"52761872":[],"52780600":35,"52789388":[],"52806096":[],"52829142":29,"52874252":6,"52876551":[],"52888979":[],"5291335":[],"52927353":[],"52989287e":[],"52998985":[],"52it":[],"53":[10,36,37],"5303329":12,"53052247":[],"5305555555555556":[2,40,41],"53105728":[],"53125507":[],"531280":33,"53145384":[],"53149078":[],"532":[],"5321613":[],"53228519":[],"53235952":[],"53269627e":[],"53285379":[],"532854":[],"533":[],"53312817":[],"53332849":[],"53361935":32,"53397188":[],"534":[],"53480404":[],"53488344":[],"535":[],"53512750e":[],"53534008":[],"53585173":[],"535852":[],"53629564":[],"53640693":[],"53684024":[],"537":[],"53703498":7,"53723188":[],"53729411e":[],"53732988":[],"53738247":35,"5378811":12,"538":[],"5384615384615384":10,"53851084":[],"538511":[],"53869015":[],"538811":[],"5388115":[],"539":[],"53911457":[],"539261":[12,33],"53942728":[],"53996947":[],"5399913":[],"53it":[],"54":[7,10,29],"540":10,"54011398":[],"54039921":6,"54041041e":6,"54057582":[],"54150298":[39,40],"541605":33,"54175461":37,"54193001":[],"54214936":32,"54222783":[],"54237093":[],"5426270409877345":[],"54295853":[],"543939":33,"54393936":33,"54406885":[],"5443909":[],"544421":[],"54442191":[],"544422":[],"544439":33,"54453037":[],"54490887":[],"54593165":[],"54637219":35,"54725976":[],"54760497":[],"54804438":[],"54811237":[],"548263635652985":[],"54843215":[],"548432150659993":[],"54856099":[],"54867722":[],"54873252":[],"54874017":25,"54880148":[],"54910928":[],"54921401":[],"54928947e":[39,40],"54935794":[],"54977724":[],"54it":[],"55":[2,10,40,41],"55020405":[],"55020649":[],"55042779":[],"55084297":[],"55094784":[],"55095316":[],"551023":[],"55102329":[],"55148543":25,"55170845":[],"55179339":[],"55196383":[],"55208611e":[],"55225408":[],"55261137":[],"55386681":[],"55440601":[],"5544062":[],"55451651":[],"55460327e":[],"55542568":[],"5555555555555556":[2,40,41],"555617937832463":[],"55565884":[],"55582327":[],"55582742":[],"55643340e":[],"5567":7,"55678525":25,"55688514":[],"557795":[12,33],"55838":37,"55854694":12,"55862544":[],"55900177":[],"55916434":[],"55929651":38,"559791959661649":[],"55988542":[],"55996153":[],"55it":[],"56":[2,10,40,41],"560275":[],"56027537":[],"56033697":6,"560379":[],"56037913":[],"56042437":[],"56072658":[],"56102529":[],"56108455":[],"561410":[],"56141009":[],"56141925":[],"56198284":6,"5620646":[],"56237849":25,"56291214":[],"56342158":[],"564":10,"56408898":[],"56423957":[],"56423957069263":[],"564374":[12,33],"56453869":[],"56469864":23,"564759":[],"56475921":[],"56476512e":[],"565":10,"5653087":[],"56536":[1,32],"56541622e":[],"56576662":29,"56589683":35,"566":10,"56636616e":[7,34],"56665044e":[],"56685379":[],"56690562":32,"567":10,"56763673":[],"5676397":[],"56766321":[],"567753":[],"56775345":[],"56778296":[],"567783":[],"56783103":[],"56790195e":[],"568":[10,35],"56804783":[],"56832496e":[],"56850995":[],"56857739e":[],"5686930390258937":29,"56869304":29,"569":[2,10,40,41],"56912044e":[7,34],"56913182":32,"56922102":[],"56939714":6,"56947078":[],"56965674":35,"56967917":[],"56977352":[],"56990537":[],"56it":[],"57":[1,9,10,30,32],"570":10,"57000682":32,"57016227":[],"57080764":[],"571":[6,34],"571105947979344e":[7,34],"571105947979412e":[7,34],"57120988":29,"5715150087179857":[],"57192197":[],"57201944e":[7,34],"57212292":[],"57231587":22,"57252451e":[],"57261202":[],"57293306":[],"57324367":[],"57331229":[],"57334868":[],"5736973":[],"574465":[12,33],"57467546":[],"57468142":[],"57475388e":[],"57492326":[],"57502449":[],"57521514":[],"57584612":[],"575870409023631":[],"57599494":[],"576":35,"57604071":32,"57610367":[],"5769230769230769":10,"57735536":[],"57803254":[],"57819238e":[],"57833531":[],"57847325":[],"57852915":[],"57854115":[],"57881465":[],"57886743":[],"57914918":[],"57933547":[],"57966619":29,"58":[3,10,11,30,32],"58013829":[],"58084123":[],"58161787":[],"58202531":[],"58218587e":[],"5823186":32,"58257727":[],"58284836":[],"582987":[],"58298728":[],"58304607":[],"5833333333333334":10,"583595":33,"58379362":[],"58408765":[],"58452753":[],"5847256":[],"58492636e":33,"58493493":[],"58493681":[],"58556424":[],"58570955":25,"58583003":[],"58589719":[],"585982":12,"58598227":12,"58641507967355":[],"58735893":[],"58799372":[],"58825417":[],"58827236":7,"5888888888888889":[2,40,41],"58890784":[],"58914105":[],"58933143":[],"589841":[],"58it":[],"59":[3,10],"59001966":[],"59009792":[],"59030403":[],"59078383":[],"590784":[],"59088350e":[],"591317992":5,"59184268":[],"59188296":[],"59206051":32,"59215613":[],"59244881":[],"592766658048914":[],"59348521":[],"59354079":7,"59355752e":[],"59372982":[],"59388777":[],"59391196":[],"593912":[],"59411814":[],"594241":[],"59424119":[],"5944444444444444":[2,40,41],"59465457e":[],"59491404e":[],"59560497":[],"59630232e":[],"5965161e":[39,40],"59683811":[],"59698459":[],"59730078":[],"59741748e":[],"59770229":[],"59779819":37,"59833837":[],"598549916355722":[],"59854992":[],"59875207":[],"59879372":[],"59884169":[],"59898921":[],"59908591":[],"59908884":[],"59958693":25,"59963873":[],"59971727":[],"59it":[],"5cm":29,"5f":[9,37],"5x":9,"5y":9,"6":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,18,22,25,27,29,30,32,33,34,35,36,37,38,39,40,41],"60":[2,4,7,10,40,41],"60000":5,"60001225":[],"60026673":[],"60041459e":[],"600529":[],"60052901":[],"60084189":25,"60116388":[],"601164":[],"60116948":[],"60151468":[],"60160003e":[],"60181469":[],"60184333":[],"6019067271":5,"60193996":[],"6025456":[],"602546":[],"60271731":[],"6029291":[],"60293962":6,"60321647":[],"60342978":35,"60357007":29,"60375434":[],"6038642":[],"60418171":[],"60420593":6,"604213":[],"60421343":[],"60451189e":[],"604731":[],"60480445":25,"60489634":[],"60534421":[],"60543038":33,"60552651":[],"60568686":32,"60570744":[],"60575887":[],"60586595":[],"60588173":[],"60605083e":[],"60610368":23,"606251":[],"60625144":[],"60664448":[],"60672354":29,"60673226":12,"606760":6,"60720029":[],"60737006e":[39,40],"60740312":[],"60753606":38,"60762863":[],"60794187":[],"607942":[],"60837213":32,"6087346":[],"60900861":[],"60907486":[],"60915019":[],"60921896":[],"60929685":35,"60940356":[],"60943791":[25,32],"60999067":[],"61":41,"61069681":[],"61076053":32,"61083596e":[],"61106937e":[],"6111111111111112":[2,40,41],"61113781":[],"61130099":[],"611301":[],"61192203":[],"61192712":[],"61209378e":[39,40],"612939":33,"61321722e":[],"61325173":[],"613579":33,"61366611":[],"61382924":[],"613893902124586":[],"61395701":[],"61402242":[],"61414743":[],"614161296466206":[],"61468323":[],"614808":[12,33],"61501312":[],"6153846153846154":10,"61564092":[],"6156409232714415":[],"61565184":38,"61602014e":[],"6161104":[],"61621902":[],"6177137":6,"61774362":[],"61775176":35,"61860028":[],"618746":[],"61883807":[],"618982":33,"61974366":[],"61979664e":[],"62":41,"6200":32,"6201":32,"6202":32,"6203":32,"62033782":[],"6204":32,"62068649":[],"62113349e":[],"62149436e":[],"621584":[],"62158409":[],"621787":[],"62178739":[],"62198131":[],"62242138e":[],"62246687":25,"62275788":37,"622777986360822":29,"62277799":29,"62286539":[],"62288579":[],"62299126":38,"62299866":[],"622998661867777":[],"62323175":[39,40],"62351789":[],"62373464":12,"6243301":[],"62475001":[],"625":[8,36],"62551462":[],"62571719":25,"62573269":[],"62617788":[],"62620041e":[],"626635268":[7,35],"626678":[],"6266783":[],"6267112":[],"62671667":[],"626889e":22,"62714703":[39,40],"62716271":[],"62725226":34,"628":7,"62816974":[],"62817267":[],"62846663":[],"62863106e":[],"62863887":[],"62886648":[],"62894215":6,"62905281e":[],"62908496":[],"62918713":[],"62927207":33,"62954328e":[],"62977556":29,"62it":[],"63":[1,2,7,10,35,40,41],"63025821e":[7,34],"63030526":[],"63031478":[],"630315":[],"63037846":7,"63042739":[],"63046054":[],"63071772":[],"63116786":32,"63142399":[],"63142731":[],"63144217":[],"631471":6,"63162342":33,"631986":[],"63198618":[],"63204599":[],"63207408":[],"63209435":[],"63249532e":[7,34],"63307537":[],"63309898":[],"63322884":[],"6333349473431832":[],"63347711":[],"63353014":[],"63371136e":[],"63443371":[],"63461575":[],"63483713":[],"63490568":[],"63498144":6,"63500255":[],"63520537":25,"63542565":32,"63546236":[],"63546905":[],"63555267e":[],"63615006":[],"63619709":[],"63693246":[],"6373":[],"63731768":12,"637318":12,"6373454":[],"63760101e":[],"63797452":12,"63801791":[],"6380763200092493":[],"63833441e":[],"6383903":[],"63896182":[],"638962":[],"6389681":[],"63898112e":[],"63919355":[],"639194":[],"63928106":[],"63980097":[],"63980912":[],"63993855":[],"64":[2,4,5,14,25,32,37,38,40,41],"64009263":[],"64012627":6,"6401576":[],"640567":[],"6405671":[],"6407249":[],"64082047":[],"640x480":[7,14],"64192473":[],"64291046":[],"64297976":[],"64316192":35,"64322974":[],"64344541":32,"64361375":[],"64362404":[],"64373221":[],"64373635e":37,"64399916":[],"64408368":[],"64411666":[],"64412879":[],"6441397":[],"64425744":[],"64493447e":[],"64505146":[],"64516328":[],"64564368e":[],"646283":[12,33],"64630711":[],"6463440100796987":12,"64638876":[],"64642277":33,"64672839":[],"64689773":[],"64690121":[],"64713979":[],"64721597":[],"64731484":[],"64742912e":[7,34],"647473":[12,33],"6477208":29,"64796751":[],"64797382":[],"648131":[],"64813145":[],"64836333":[],"64857542":[],"64885265":[],"649":[],"64909048":[],"64911288":[],"649382":[12,33],"64952115":[],"64968814e":[],"64976691":[],"64it":[],"64x50":[2,40,41],"65":[2,9,10,40,41],"650":[],"65003213":[],"65082441":[],"651":[],"65147818":[],"65183066":[],"651831":[],"652":[],"65243388e":[],"65259487":[],"65293772":[],"65295017":25,"653":[],"6533977":[],"653398":[],"65354422":32,"6536392":33,"65396469":29,"654":[],"65410649":[],"65442354":33,"654424":33,"65475653":[],"65476419":[],"6548596":[],"655":[],"65525178":[],"65532045":[],"65543115":[],"65554958":[],"65571174e":33,"65591748":[],"65593984":[],"65599456":33,"656":[],"6561141":25,"65632442":22,"65641173":[],"6564482":32,"65651288":[],"6565507":37,"65672295":[],"65681217":[],"6568154596723088":[],"6568686180915178":[],"65686862":[],"65720152e":[],"6572612e":[39,40],"65738717e":[],"65743689":33,"65744898":[],"65751456":[],"65759127":22,"65759901":[],"6576130374029565":[],"65765268":25,"658":[],"6582728":[],"65833132":33,"65870313":[],"65885453":6,"65889641e":[],"65891389":33,"658914":33,"659":[],"65900052e":[],"65917558":[],"65944342":37,"65971184e":[],"65976951":25,"65it":[],"66":3,"660":[],"66003032":[],"66018725e":[],"66047885e":[],"66063786":[],"66064822":33,"66080313":33,"66130478":[],"66155431":29,"66204648":7,"66215333":[],"662153330760193":[],"66272614":[],"662836197064178":[],"66289235":[],"66294408":33,"6629598":[],"662960":[],"66339452":29,"66340579":32,"66342634e":[],"66363129":[],"66365453e":[],"6638":10,"664314":[],"66431407":[],"66456459":[],"66487438":[],"66498292":[],"6652177":33,"66533454e":[],"66560":10,"66575697":[],"66603461":[],"6663569":[],"66694492":[39,40],"667":10,"66707072e":[],"6671767":[],"66729685":[],"66734487":38,"66757105e":[],"66763324e":[],"66774691":[],"66809369":[],"668172":33,"66841959":[],"6690108":[],"66951679":[],"6696885":[],"66990322":[],"669908227073521":[],"66it":[],"67":[3,10,41],"67015946e":[],"6702":[],"67036507":[],"67047975e":[7,34],"67088558":[],"67100236":[],"67199457":[],"67231298":[],"67238764e":[],"67249437":32,"672721":33,"672759":[],"67275915":[],"67313212":[],"67314874e":6,"67364995":[],"67374815":[],"674":35,"67435043":[],"67464946":[],"67477123e":[],"67490857":37,"67495943":[],"67509321":[],"67511113":[],"67524588":[],"67527216":[],"67555087":37,"67561021":[],"67747547":[],"67819487e":[],"67834752":[],"67849271":32,"67875366":[],"678754":[],"67925887":[],"67955629":[],"6795562921258607":[],"67982841":[],"67it":[],"68":3,"68008931":32,"6801":[],"68012971":32,"68029423":[],"68043896":[],"6805151":[],"68072161":[],"68074588":37,"6809195":[],"680920":[],"68092237e":[],"68110474":[],"68129036":[],"68163369e":[],"6816947":[],"68192193":6,"68257097":[],"68258595":[],"68281739":[],"68286725":33,"68291639":32,"6831505":[],"68352799":[],"68402334":[39,40],"68532803":[],"68534263e":[7,34],"68542204":6,"68591105":[],"68609856":[],"68637642":[],"68640031":[],"68655456":[],"6869":10,"68735473":[],"68748147":[],"68749642":[],"68753434":[],"6876042":[],"68763036":[],"68822559":[],"688226":[],"68824222":[],"68826552":[],"68827947":[],"68830164":[],"68857788":[],"6887363571":5,"68879472":[],"6888043337746685":[],"68891428":[],"68929213e":[7,34],"6893421391051477":[],"689519":[12,33],"68953877":[],"689539":[],"68962656":[],"68989446":[],"69":[3,10,29],"690":10,"6901769":[],"69030225":[],"690617":33,"69069n_":29,"69089532":[],"691":[2,40],"691364":[],"69136414":[],"69164986":32,"69169665":[],"69182892e":[],"69203997":[],"69227923":[],"69230769":37,"6923076923076923":10,"69243548":[],"69247718":[],"6927554":[],"69303453":[],"693035":[],"69306119":[],"69311221":[],"6932681":[],"69391797":[],"69417174":[],"69432059e":[],"69435454":[],"69438869":[],"69463406":[],"69465386":[],"69484813e":6,"69484973":[],"69496248":29,"69504801":7,"69515531":[],"69547403":29,"6955259":[],"69566966":[],"6957182489166325":[],"69634577e":[7,34],"69678073":[],"696781":[],"69695259":6,"697346":[],"69734609":[],"69734853":12,"697349":12,"69749114":[],"69756791":[],"69768266":[],"6980":[22,37,38],"69806017e":[],"69811070e":[],"69824402":[],"6988905":[],"69892546":[],"698996":[],"69899609":[],"69903406":[],"69908819":[],"69925648":[],"69978197":[],"699782":[],"69983227":[],"6999536":12,"69it":[],"6n_":29,"6pm":32,"7":[1,2,3,4,5,6,7,8,9,10,12,13,14,15,16,22,25,27,28,29,31,32,33,34,35,36,37,38,40,41],"70":[2,7,8,10,36,40,41],"70015185":[],"7002164":[],"70037324e":33,"70039784":[],"70040367e":[],"70090777":[],"700908":[],"70135856":[],"701370":6,"70159137":[],"70175778":[],"70241548":[],"7024611":25,"7026722043092946":[],"7026908764394864":[],"70277293":25,"7029975":[],"70301068":[],"70302827":[],"7033198":[],"70334935":[],"70362677":[],"70367841":32,"70408202":[],"70469324":[],"70474145":[],"70485195":[],"7049679":[],"70499955":[],"70514808":[],"70530438":[],"70587399":[],"705946":[],"70594608":[],"70630967":[],"70653767":5,"70665448":[],"70704566":[],"7070883":[],"70710678":[6,33],"70711022":[],"70711514":32,"7074705":[],"70751625":[],"7080469":[],"70811405e":[39,40],"70832814":6,"70857635":[],"7086067479626619":33,"70885855":[],"70928399":[],"709284":[],"70944365":[],"70981984":29,"71":[2,3,40,41],"71007076":[],"7100707615589594":[],"71059619":[],"7112242518460867":10,"71129677":[],"71142161":33,"711422":33,"71143744":25,"7119":10,"712018":[12,33],"71261809":[],"71263085":[],"71263851e":[39,40],"7127128":[],"7129466":[],"712947":[],"71314288":[],"713143":[],"71332117":[],"713763":[],"71376303":[],"71380438":[],"71384267e":37,"71409384":[],"71441086":[],"71460518":[],"71469351":[],"7150195":[],"71505132e":[],"71507714e":[],"71524686":[],"71527295":22,"71582784":[],"71633658":[],"7163835161938888":[],"71640303e":[],"71664506":[],"7167077":[],"71672742e":[],"71672791":[],"71737253":33,"71761101":33,"717794":[],"71779401":[],"718165":6,"71838601":[],"71860732":[],"71957514":[],"7198234":[],"72":3,"72033728":[],"7204561":[],"72047148":[],"72055288":[],"72066289":[],"720889407643293":[],"72174172":12,"72183526":[],"72184754":25,"72219757":[],"72236654":[],"72271878e":[7,34],"7228852":[],"72288611":[],"72309979e":[],"72315862":[],"72364915":[],"72365764":[],"7236674":6,"724":4,"72407267e":[],"72427854":[],"72444668":[],"7249499":[],"72521196":[],"725212":[],"725256":[],"7252563":[],"72577807":[],"72589774":[],"72598313":[],"726017":[],"72601722":[],"72602818":[],"72690169":[],"72710494":[],"727132":[],"7271324":[],"72744372":[],"72754455":25,"72758060e":[],"72770687":[],"727707":[],"72771857":[],"72848736":[],"72859758":6,"72863433":[],"72876728":[],"72882234":[],"72952595":[],"72962885":[],"72964493e":[],"72965547e":22,"72966563":[],"729666":[],"72981762":9,"72992196":[],"72993254":[],"73":[7,10,35],"73018893":[],"73031999":[],"730364":[],"73036439":[],"73062648":[],"73063053":[],"73068535":[],"73071831":[],"731000":33,"73122034":[],"73176319":[],"73203674":[],"73204357":[],"73252708":[],"73265885":[],"73299464":[],"73305640e":[],"733096":33,"73337355":[],"7333854378748542":[],"73374853e":[39,40],"7337962":25,"73380463":[],"73390423":32,"73397147":[],"73423383":[],"73425394":[],"735490390687699":12,"73550536":[],"73565966":[],"73578772":[],"73594414e":[],"73595198":[],"73614157":[],"736282":[],"73628249":[],"73646130e":[],"73690387":37,"737184":[],"73718419":[],"73741131e":[],"73785069":[],"737851":[],"73853838":[],"73854926":[],"73876416607927":[],"73888203":[],"73958682":[],"739705":[],"73970542":[],"73986200e":[],"73994248":[],"74":[7,35,37,38],"740":10,"74028787e":[],"7403629637766833":[],"74038232":[],"74038873":[],"74075864":[],"74081822":9,"740859":12,"7408594":12,"74100159":[],"74101067e":[],"74138002e":[],"74171049":[],"74180248":[],"74208907":[],"74259663":[],"742597":[],"7426664":29,"74272285e":[],"74291603e":[],"74336924":12,"74384858":[],"74384949":33,"74407544":[],"74412221":32,"74463241":[],"74471533":[],"74499034":[],"74506271":[],"74523085":29,"74533876":[],"7458714725617228":12,"74607851":33,"74613334":[],"7461895":[],"74632311":[],"74692641":[],"74724414e":[],"74724774":37,"74761815":[],"74803743":38,"74829661":33,"74840212":6,"7487814387207385":[],"74878144":[],"74883972":[],"74900488":[],"74957448":[],"749765":33,"7499639":[],"75":[6,7,9,10,12,26,33,35,38,39],"75015599e":[],"75018686":[],"750187":[],"75025744":[],"750445":33,"75050135":33,"75080943":[],"7509571":[],"7510197":[],"7511273":[],"75131415":[],"75138064":[],"75146945":[],"7515117":[],"751699":[12,33],"75170092":6,"75216828e":37,"75217508":[],"75240139":[],"75269037":33,"75300068":[],"75345893":[],"7539148284817965":[],"75406226e":[],"75409398":37,"754130":[],"75413023":[],"75424998":[],"75433448":[],"7546022135629743":[],"7546682196464342":[],"75488337e":[],"75532041":[],"75573622e":38,"75576555":33,"75592582":37,"75600095":[],"756352":33,"75654448":[],"75669703":[],"75674251":[],"75677341":[],"756828":[],"75682834":[],"75733813e":[],"7573694380469245":[],"75737136":[],"75760403":[],"7577":35,"75770568":33,"7577113351177733":[],"75956076":[],"75956617":[],"75978215":[],"76":[10,30,32,36,37],"76030841":[],"76065256":[],"76120718":[],"76134837":[],"76140219":[],"761574":[],"76157429":[],"76159011":[],"76183628":32,"76190591":34,"76262775":[],"76277254":[],"762891":[],"76289128":[],"76329289":[],"7642448":[],"764245":[],"76426396":[],"765":[8,36],"76527062":[],"76533281":[],"765333":[],"76542156e":[],"76561858":25,"7658599":[],"76587916":25,"76601702":29,"76612827":[],"766163196245293":[],"76632434":[],"7663967":[],"76686148":[],"76688358":[],"76690383":35,"76692036e":[],"767285":[],"76728511":[],"767367275553824":[],"7674217625964539":[],"76751116":38,"76790092":[],"76807288":[],"76852722":[],"76855921":[],"76880575":[],"76887316e":[],"7692307692307693":10,"76936315":6,"76940759":[],"76942272":[],"7694444444444445":[2,40,41],"76996262e":[],"77":[10,30,32,37,38],"77014839":[],"77030637":[],"77037707":[],"77039278":[],"77106886":[],"77112374":25,"771128":[],"77132083":[],"77152076":6,"77152924":[],"7718":10,"77199379e":[],"7721643":25,"772588722239781":[37,38],"77306239":[],"77313451":25,"77352152e":[],"77354301":[],"774498":[],"77449804":[],"77465911":[],"77482712":[],"77497266":[],"77507653":[],"775204":[],"77551387":[],"7755649":[],"775565":[],"77559615":[],"77579275":[],"77594586":[],"77596142":[],"77597019":[],"77602426e":[],"77618181":[],"77626954":[],"776322":[],"77632217":[],"776352510835556":[],"77635684e":[],"77636e":[37,38],"77647539":[],"77652198e":[],"77659961":[],"7766443":[],"77699722":[],"777112373675558":[],"77714169":9,"77725472":[],"777255":[],"77803692":[],"7782028952":5,"77829399":[],"77845284":[],"77873845e":[],"77878621":25,"77959897":[],"77982629e":[],"77994763":32,"78":[],"78009660e":33,"78010668":[],"78025829":[],"78144936":29,"78156479e":6,"78157908":[],"78184120e":[7,34],"78186404":[],"78198192":[],"78243922":[],"78251088":[],"78254537":[],"78261152":29,"78278086":[],"78326199e":[],"78338459":[],"78340902e":[],"78380113":25,"78385675":[],"78459458":[],"78477759e":[],"78575679":29,"78596158":[],"78617455":[],"78630865":[],"78646096":[],"78671736":[],"78714005":[],"7873282":[],"78745927":[],"78758569":[],"78762558":[],"78810857":38,"78850066":29,"78876155e":[],"78888045":[],"78892891":[],"7889793":32,"78904605":[],"78941903":6,"78945959":[],"78949699e":[],"79":[],"79010601":[],"79041505":[],"79086674":[],"79102612":[],"79111643":6,"79135536e":[],"79171202":[],"79247398":[],"79251831":[],"79280475":[],"793167":33,"79401569":[],"794282":[12,33],"79443618":[],"79469098":[],"794749":12,"7947491":12,"7949388":38,"79503363":[],"79522963":[],"7953443":34,"79540402":[],"79558414":[],"795584143123602":[],"79558423":[],"79619113":[],"79622905":[],"79623487":[],"796235":[],"79628257":[],"79633065":35,"7963438":[],"79673108":[],"79718864":[],"79778008":7,"79787194":[],"797872":[],"797e":7,"7981908":[],"79849321":[],"79866484":[],"79870689e":[],"79873884e":[],"7989356":[],"79898903":[],"79942527e":[],"7d7d58":[10,11],"8":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,19,22,23,25,26,27,29,32,33,34,35,36,37,38,39,40,41],"80":[1,2,6,7,9,10,33,34,35,40,41],"800":[5,8,36],"80011603e":[],"80033616":[],"80056922":[],"80066012":[],"80072356":[],"800724":[],"800782161095708":[],"80118954e":[],"801356":[],"80147746":32,"8030191":[],"80311453":[],"80354994":7,"80369722e":[],"80373997":[],"80374623":[],"80410179":[],"80413696":[],"80421021":25,"80453713":[],"80460179":35,"80469739":6,"80499859":[],"8055555555555556":[2,40,41],"8056560957923695":[],"80576215":[],"80609615e":[7,34],"80609616e":[7,34],"80633871":[],"80637385":[],"8063779":[],"80668002":[],"8073322":[],"80740939":[],"80750113":[],"80835002":32,"80845707":6,"80846898":[],"80847477e":[7,34],"80848093e":[],"8084934":[],"80853029":[],"80855691":[],"808557":[],"80873785":[],"80907609":[],"80979265":[],"80980728":[],"81":[2,40,41],"81006889":[],"81007381":[],"81012733":25,"810150":[],"81015037":[],"81035098":[],"81040897":[],"810409":[],"81042619e":[],"81048318e":[7,34],"81091776":[],"8110093e":[],"81160425":6,"81208109":[],"81218742e":[],"81246697":[],"81283649":[],"81318307":[],"81329201":32,"8133":[],"81333808":7,"81337164":[],"81385326":[],"813957":[],"81395716":[],"81425129e":[],"81478187":[],"81479491":[],"81506255":[],"81522211":[],"81559586":[],"81592028":[],"81598832":[],"815am":[30,32],"81622636":[],"81633628":12,"81642349":[],"816454":33,"816847":33,"81684805":[],"81707832":[],"81737781":[],"81818338":[],"81825652":25,"81826799":[],"81839897":[],"8184887":[],"81889188":[],"81919129":[],"81924504":[],"819253":[],"81925302":[],"81931348":[],"81947005":[],"81979166":[],"819805":[],"81980509":[],"81984817":32,"81987609":[],"82":[],"82004436":[],"82060913e":[],"8207274870895702":[],"82079885":[],"8211787":[],"82121282":[],"82130663":[],"82133401":25,"82146046":[],"82161167":[],"82198978":6,"82204612":25,"82212861e":[],"8221979401496963":[],"82242798":[],"82247747":[],"82259916":[],"8227":[],"8232102961414203":[],"823242":[],"82324216":[],"823417":[],"82341746":[],"82360724":29,"82376174e":[],"82418008":[],"82418352":[],"82435177e":[],"8244":[],"8248473":[],"8250103":[],"82528238":[],"82533209":[],"82551764":[],"82559478":[],"82561703":[],"82620196":[],"82623709e":[],"8265786":6,"82677051":38,"82688363":[],"82749474":29,"82757017":[],"82802004":[],"82832579":[],"82844656":[],"82864173":[],"82891669":38,"82914126":[],"829252":[],"82925222":[],"82934918":25,"83":[],"83020679":[],"8305555555555556":[2,40],"83146596":35,"83164611e":[],"83203197":[],"83220985":[],"83230774":[],"83231208e":[],"83234002":[],"83294718":[],"83317854e":[],"8332972":[],"83363025":[],"83369175":[],"83377498e":[],"83408369":32,"83429536":[],"83463634e":[],"83466807":[],"83512277":6,"83519517":[],"8352834":[],"83641435":[],"83655756":[],"83671148":[],"83681263":[],"83693088":32,"83718206":[],"83741078":[],"83759075e":[],"83785062":32,"83816147":[],"83823317e":[],"83824223":[],"83870189":7,"8387133":[],"83884553e":[],"83915415":25,"839957":[],"83995705":[],"84":[],"84019294":[],"84047912":[],"8406741e":[],"84078641":[],"84155431":[],"84159992":[],"842":[],"8423144":[],"842436":33,"84254917":[],"8426271":[],"84269323":32,"843":[],"84338322":[],"84351715":[],"84355903e":[2,40,41],"844":[],"84401886":[],"84416298":[],"8443626e":[],"84443254e":[2,40,41],"84478178":37,"84514616":[],"84581375":[],"846":[1,8,36],"84605507":[],"8461538461538461":10,"84629286":[],"84634327e":[],"84659427":[],"84661230e":[],"84668873":[],"847":[1,8,36],"84707251e":[],"84711297":[],"84726382":[],"8473429":12,"847343":12,"84745063e":[],"84745525":32,"8475889":[],"84767088":[],"8476874":[],"84780262":7,"84789579e":[],"848":[1,8,36],"8485":[],"84858":35,"84862355":[],"84869686":25,"8488":[],"84907109":[],"84908687":[],"84923989e":[7,34],"8496945":38,"84972787":[],"84973247":[],"84984612":[],"84985833":[],"84992628":38,"84993546e":[],"84994524":6,"85":[2,10,40,41],"85005989":[],"85014487":[],"850164":6,"85018556e":[],"85025162":[],"8502729":12,"85035562":[],"85043503":[],"85049285":32,"85050649":[],"8505072899794293":[],"85064942":[],"85068028":[],"85079059":[],"850859":6,"85085928":6,"8509859":32,"85108001":[],"85112594":[],"85142265":32,"8517571":[],"85189083":25,"85222243":[],"85259666e":[],"85261484":[],"85263220":7,"85276538":[],"85278920e":6,"8529175":[],"85293303e":[],"85294194":[],"85298956":[],"853":[],"85378708":[],"85408468e":37,"85422602":[],"85427641":[],"85437905":[],"85551266e":[],"85601992":6,"85613835":[],"85625134":[],"85703369":[],"85711016":[],"85714286":0,"85724692":25,"85777073":[],"857771":[],"85786373":[],"858":10,"8583333333333333":[2,40],"85859459":[],"85866294e":[],"85871255":[],"85897711":[],"85945208e":[],"85954931":[],"8595836":[],"85975973e":[],"86":[],"86009093":[],"86013055e":[],"86016288":[],"86030593e":[],"8605188":[],"86062977":7,"86066395":[],"86067818":[],"86070736":[],"86084064":[],"86117291":6,"86123808e":[],"86134827":6,"86138568":22,"86143379":[],"86145244":12,"8614682":[],"86174943":[],"86192682e":[],"86197958":[],"86215487e":[],"86240935":[],"86247117":[],"86267497":[],"862675":[],"86273371":34,"86274590e":[],"86284277":[],"8631855":32,"86346614":[],"8638888888888889":[2,40,41],"86416941":[],"86434639":[],"8644215":[],"86445334":[],"86458368e":[39,40],"86459987":[],"86493273":25,"865":[],"86506494":[],"86519139":[],"86567539":[],"866":[],"86602331":[],"86611894":[],"8662":[],"86629161":[],"86630":10,"86632028":[],"86639832e":[],"86643649":[],"86660944":[],"86664528":[],"8666666666666667":[2,40,41],"86666667":0,"86668761":[],"86696026":[],"866960260598739":[],"867":[],"86737671":[],"86757762":[],"86799623":[],"86810":10,"86833907e":[],"86861591e":[],"86867512":[],"86868711":[],"86885547":32,"86891741":[],"869":[1,8,36],"8691787":[],"86943859":[],"86978877":[],"87":10,"870":[1,8,36],"87001185":[],"87020155":[],"8702784034":5,"870337":[],"87068171":34,"87070464":[],"870704644539707":[],"87072815e":33,"87093752":[],"871":[1,8,36],"87148597":[],"87169607e":[],"87171492":[],"87186187":[],"8720648":[],"8722222222222222":[2,40],"87228174":32,"87231849":[],"87238538":[],"87323528":[],"87340054":[],"87349626":[],"87381451":6,"87399194":[],"8745559":38,"8746702":[],"87473452":[],"87478387":25,"87494456e":[],"875":[2,40,41],"87584434":[],"8758851":[],"8759":[37,38],"87616945":[],"8762":[],"87627342":35,"87633005":[],"87637891":[],"87661362":[],"87696461":[],"877":7,"87700882":[],"87732748":[],"87736602":25,"87740526":[],"87788573":[],"87818331":[],"87830277":[],"87878356":[],"87884833":[],"87894459":[],"878945":[],"87981676":[],"88":[32,37,38],"8800906083513416":29,"88009061":29,"88046261":6,"88055343":[],"8805555555555555":[2,40],"88064505":[],"8809429":[],"88135577e":[],"88147037":[],"88157423":[],"88168312e":[7,34],"8817842e":[],"88182591":33,"88208835":[],"88236598e":[],"8825782":[],"88289263":[],"882972378790114":[],"88297238":[],"88323026":33,"88333467":[],"88336879":6,"8834":[],"88355913":[],"88377332":[],"88415819":[],"88445111":32,"884669":33,"88477745e":[],"88501323":[],"88514106":[],"88518":[],"88529063e":[7,34],"88551766e":[],"88569607":[],"88594444":[],"8859754":25,"88613493":33,"88614317":[],"8861811":[],"88632765":[],"88639567":[],"88653291":[],"88685415":[],"88691234":[],"88726714":[],"88738394e":[],"88809826":[],"88860399":[],"8888888888888888":[2,40],"888908":6,"8889085":6,"88930323":[],"88960947":[],"8896094720230128":[],"89":[1,10],"890033":[],"89003324":[],"8901199":[],"89017198e":[],"89044267":[],"89054207":[],"89058005":[],"8905826":[],"89069029":[],"8908983":[],"89100924":[],"89126914e":33,"89155606":[],"891559":[],"89155934":[],"8917148":[],"891715":[],"89184351":[],"89211256":22,"89219980e":[],"89222305":[],"89229722":[],"89288636":12,"8931686":37,"8932188":[],"89378344":[],"89408972":[],"89410423":6,"8944444444444445":[2,40],"89457785":[],"89463761e":[],"89527291":[],"89533642":[],"89540936":[],"89543945":[],"89563074":[39,40],"89576555":29,"8962":[],"89654756":[],"89676583":[],"89692576":[],"89708415":[],"89710385e":[],"89714357":[],"89724282":[],"89730533":[],"89755794":[],"89766337e":[],"89770892":[],"8978":[],"89790867":[],"89845198":[],"89876098":[],"898761":[],"89961861":[],"89989636":[],"8f":[7,35],"8g":[7,35],"8n":25,"8x8":[2,40,41],"9":[0,1,2,3,4,5,6,7,8,9,10,12,13,14,15,19,22,23,25,26,28,29,30,32,33,34,36,37,38,39,40,41],"90":[2,7,10,40,41],"90075537":6,"90105594":29,"9011":7,"90114639":[],"90135464e":[],"90137221":[],"90137794":[],"90140097":[],"90162133":32,"90168941":[],"90182442":[],"90210098":32,"90220243":6,"90232941":25,"90266948":6,"90273253":22,"9027777777777778":[2,40],"90281987":[],"90300219":[],"90300704":[],"90304177e":[],"9031162":22,"9040":10,"90407685":[],"90475506e":[7,34],"90536337":[],"9054":35,"9055555555555556":[2,40],"9057119872872414":32,"906589":[],"9065894":[],"906747":6,"90717653":[],"907177":[],"9074157":[],"90751311":[],"90800497":[],"90842732":[],"90860768":[],"90871271":[],"90885733e":[],"90894281":[],"90903158":[],"90909173":[],"909092":[],"9092793":[],"91":[30,32],"910":10,"910294":[],"91029443":[],"91029782":[],"910297822789896":[],"91099021":[],"91100141e":37,"9111111111111111":[2,40],"91114704":[],"91128596":6,"91152267":[],"91193672":[],"911937":[],"91247848e":[39,40],"91264011":[],"91270333":[],"91312563":[],"91322403":[],"91365263":[],"91410559e":[],"91416944e":[],"91417133":[],"91433102":[],"91434750e":[],"914672":[],"91467223":[],"91483251":[],"91492986e":[7,34],"91496317":25,"91507022":25,"91520631":[],"91527102":[],"91546928":[],"91568958e":[],"91624469":[],"916245":[],"91648657":[],"91650453":[],"9166666666666666":10,"91669165":[],"916692":[],"91672129":[],"91692081":[],"91722512e":[],"91744988":[],"91752854":[],"91760278":6,"91811422":[],"91812702":6,"91854134":[],"91864509":[],"9187554":[],"918785":[],"918992":33,"91933003":[],"919503":[],"9195033":[],"91961463":[],"919895":12,"91989533":12,"92":[7,10,30,32],"92003207":[],"92004534":25,"92021197":[],"9202867":[],"9206617":[],"92080432":[],"92159388":[],"92169638":[],"92192012":[],"9225384545636204":[],"922622":[],"92262217":[],"92292668":[],"92301743":32,"9230769230769231":10,"92308585":[],"92308867":[],"92313548e":[39,40],"92319842":[],"92326255":[],"92341148":[],"92353194":[],"92370959":[],"92376655":[],"924011":[],"92452609":[],"92458892":[],"92466997":[],"92474572":[],"924746":[],"924e":7,"925":[2,40,41],"92507116e":[2,40,41],"92512799":[],"92527681e":[],"92552059":[],"92578916":6,"92599642":[],"92604454":[],"92622842":[],"92656908":29,"92661467e":37,"92663007":[],"92665802":[],"92678221":[],"92713642":[],"9271707090281667":[],"92719687":[],"92756256":34,"92761972":25,"92768548":[],"9277777777777778":[2,40,41],"92779167e":[],"92808577":[],"92857143":0,"928769588424018e":[],"9288904":[],"92891949":[],"92892022":[],"92901115":[],"92905422":[],"92910808":[],"92921498":[],"92929225e":[],"9294527":[],"92991181":[],"93001752":25,"93019783":[],"930536":[],"93053605":[],"9305555555555556":[2,40,41],"93059014":[],"9306628":[],"93074023":[],"9308197":25,"931":[1,32],"93148781":[],"93155188":6,"93158979":6,"93159699":[],"93164276":[],"93164299":25,"93237862":[],"932382":[],"93272428":[],"93277949":[],"93289669":[],"93293067":[],"933":[6,34],"93325371e":[39,40],"93366198":[],"933662":[],"93397254e":[],"93416236e":[],"934339":12,"93460701":[],"9349":[],"93492130e":[7,34],"93518383":[],"93519647":[],"93597168":[],"935972":[],"93601008e":33,"93602214":[],"93619369":[],"9366":[],"93668828":[],"936877889972962":[],"937":29,"937057":[],"937082":33,"93708805":[],"93709777e":[],"93787605":[],"93799826":6,"938":29,"9387":10,"9388939e":[],"93891335e":[],"939":[1,29,32],"93900569e":[],"93909866":[],"93937564":[],"93942872":[],"93967766":33,"93968601":[],"93969971":[],"93970238":[],"93987367":[],"93989887":[],"94":[0,10,29],"94008244":[],"94027059":[],"94033034":[],"94040981":37,"9405":[],"94082113e":[],"94091671":[],"94092702":[],"94096731":[],"94099173":[],"94114646":[],"94121002":[],"9412311":[],"94141801":[],"94217756":[],"942178":[],"94221383":[],"94226022e":[7,34],"94255277":[],"94272259":[],"94273542":33,"942739":[],"94284104":6,"94285762":[],"94291287":[],"94320205":6,"94335":[],"9434":[],"94388948":[],"94419718":[],"9444444444444444":[2,40,41],"944557":[],"94455739":[],"94463114":[],"94466211":[],"94484047e":[38,39],"94485679":[],"94493877":[],"94495682":[],"94499279":37,"9450635":[],"94533329":[],"9458396":[],"94591015":[25,32],"94607554":25,"946167":[],"94616709":[],"94623184":[],"94625725":[],"946276":[],"94627617":[],"946372":[],"94637206":[],"94639099":12,"94652":[],"94662523":[],"946957":6,"94704941e":[],"9472222222222222":[2,40,41],"9473":7,"94735263":[],"947607":[],"94766419e":[],"94781064":34,"94802663":[],"94805555":[],"9482527":6,"9485531":[],"948638":6,"94863824":6,"94897376":22,"9495484":[],"94963563":[],"94973606":[],"95":[2,10,12,35,40,41],"95000259":[],"95001004":[],"95008046":7,"95033155e":[],"95039348e":[],"950423":[],"95053694e":[],"950873":[],"95115528":[],"95132723":25,"95152814":[],"95162248e":[],"95166920e":[],"9518022":[],"952063":[],"95206335":[],"95226607":[],"952268":12,"95226818":12,"95231424":6,"952387":33,"9525328":[],"9527777777777777":[2,40],"95284275":6,"95289375":22,"953065564":[2,40,41],"95317772":38,"95338273":[],"95343966":[],"95351665":6,"95354276":[],"95365435":[],"95395895":[],"953959":[],"954":29,"95400323":[],"954148":12,"95462534":[],"95482298":[],"954823":[],"95497813":[],"95511792":33,"955118":33,"95518112":[],"9555555555555556":[2,40,41],"95563211":[],"95568161":22,"95578775":[],"955820c21e8b":5,"95611042":37,"95647867":[],"956563":[12,33],"95684892":6,"95703":[37,38],"95712462":[],"95715036":[],"957171":32,"95719454":[],"9573":7,"95743466e":[],"95744254":[],"95745476":[],"95745651":[],"957549":[],"95766504e":[],"95770298":[],"95776346":[],"95793988e":[],"9579870417283":23,"958228616652075":6,"95842106":[],"95845431":[],"95872127":[],"95881375":[],"95882223":[],"95882586":25,"95889379":[],"95890587":[],"95906059":37,"959076":[],"95935439":[],"9594":[],"96":[7,10,12,35,41],"960":29,"9601304850035702e":[7,34],"9601304850073734e":[7,34],"96015381":[],"96024953":6,"96029461":32,"96037553":22,"96054705e":[],"96057795e":[],"9606":[],"96067118":[],"96075051":37,"96080358":[],"96084123":25,"96084663":6,"961":29,"961042":[],"961423":[],"96142343":[],"96154578":[],"96157921":33,"96164155":[],"96175251":[],"962":29,"9623":[],"96231225":[],"96241501":[],"96251859":[],"96282505e":[],"96301764":[],"96305058":[],"963187":[],"96330728":22,"96332346":37,"9637117593816477":[7,34],"96379806e":[],"9638104":22,"96385782":[],"96385798":[],"963858":[],"96391888":[],"9639885":[],"9640435":6,"96414331":[],"96417888":[],"96432396":[],"96461948":[],"9647801":[],"9649652536":5,"96504394":[],"96544479":[],"96547946":[],"96548194":[],"965548":33,"9656484356890704":[],"96564844":[],"96581896":[],"96585824":[],"96631321":33,"96660465":[],"96688672":6,"967":[],"96734276":25,"967388":[],"96738845":[],"96745096":[],"9674916":6,"96762953e":[],"96777468":[],"967809":[12,33],"96799937":[],"96801916":22,"96826039":[],"96850191":[],"96853248":[],"968650":[],"96865004":[],"96866031":[],"968704":[],"96870431":[],"96894842":[],"96897719":[],"96899218":29,"96907179":32,"96916206":[],"96917669":[],"969190":[],"96919021":[],"96919836":[],"9692":[],"9693":[],"969413":[],"96961052":[],"96963458":[39,40],"96991192":[],"97005689":6,"97023479":[],"97065359":35,"97068015":[],"970827":[],"97082748":[],"970965":[],"97108e":[37,38],"97120565":[],"9712983":[],"9716":35,"97165491":[],"97169918":[],"97183244":[],"97190351":[],"97202371":[],"972024":[],"97206657":[],"972067":[],"972149":[],"97218946e":[],"9722222222222222":[2,40],"9722424":38,"97228653":[],"9722865317690382":[],"97235642":[],"97243128":6,"97243946":[],"97252343":[],"972533":[],"97253307":[],"97272223":[],"97286326":[],"97300836":6,"973118":[],"97311824":[],"97321835":32,"9734":35,"973663":[],"97366347":[],"97383031":[],"97435908":[],"97477277":29,"97480604":[],"97482498":[],"974825":[],"97486336":[],"97497404e":[7,34],"97498674":[],"975":[2,40],"97507735":6,"97512176":[],"975484":[],"97548406":[],"97570392":[],"97622642":[],"97626519":[],"97628064":[],"97630233":35,"97664837":[],"9767506308987675":[],"97680846":[],"97683575":[],"97696399":[],"977":[],"97705827":6,"97707044":[],"97716508":[],"97758848":6,"9777777777777777":[2,40],"978":35,"97802245":[],"9780387310732":31,"9780387848570":31,"9781098134174":32,"9781492032632":31,"9781801819312":32,"97836358":[],"9784055938098051":29,"978553":6,"97898392":[7,33,34],"97906022e":33,"9792":[],"97926491":6,"97935648":[],"9794096":[],"9796":[],"979656":[],"97965627":[],"97979891":7,"98":[1,2,10,39,40,41],"980":10,"9805555555555555":[2,40],"98091621":6,"98101481":25,"98108401":[],"981321":33,"98139097":6,"98140006e":[],"98155457":29,"981554571728081":29,"98190064":[],"98232596":[],"98244726":32,"98246764":[],"98261559e":[],"98271562":[],"98275501":6,"98281205":37,"98292239":[],"983310":33,"9833205":[],"98338288":[],"983383":[],"983450":[],"9834505":[],"9838929":[],"98413059":6,"98413606":[],"9841870203276504":[],"98454786":6,"98473694":[],"98474769":[],"984748":[],"98481245":[],"985":29,"98533636":[],"98556236":[],"98556486e":[],"98566191":6,"985725":[],"985725003240627":[],"986":29,"9861111111111112":[2,40],"98619989":[],"98621772e":[],"98636199":[],"986362":[],"98650457":[],"98654811":[],"986699":6,"98680716":6,"98716878":6,"98723376":[],"98740373":[],"98751068":[],"98770503":[],"9878":[],"98806419":[],"98808176":6,"98822371":7,"9888888888888889":[2,40],"98893579":[],"988996":[],"9889962":[],"989":29,"9890348":6,"9893447":6,"98938878":[],"98947674e":[],"98947894":33,"98955596":[],"9896":[],"98966074":[],"9898ff":[10,11],"99":[7,10,12,14,22,35,37,38,41],"990":10,"99003341":[],"99009525":6,"9904314810719695":[],"99057275":[],"99071523":[],"99072374":[],"99088801":6,"991":29,"991047":[],"99104747":[],"99114739e":[],"99115119":6,"9912346":[],"99166852":[],"99176998":6,"99194201":[],"992":29,"99201684e":[],"992232777849821":[],"99232771":38,"99242921":6,"99246366":[],"99265097":6,"99284588":[],"9929":[],"99294056":[],"9929666457835595":[],"993":29,"9930344":[],"9930742":[],"99316252":6,"99318183":[],"99348646":[],"99360578e":[],"99362554":[],"99368326":[],"99370678":[],"99371056":6,"99389612":6,"99415296":[],"99418621":[],"99427512":[],"9943201":6,"99434472":[],"994345":[],"994461":[],"99446145":[],"994474":[],"99447418":[],"99449129":1,"99456598":37,"99457257":22,"99458998":[],"99475306":37,"9947756":6,"9948998579029953":[],"99492986":6,"99498272":[],"9950019477819025":[],"9950865473984227":[],"9951059931014423":[],"99512017":[],"9952183728736417":[],"9952505213910134":[],"9952537939995855":[],"99528218":6,"9953466931203151":[],"99539415":6,"995522":[],"9955318336036834":[],"99556258":[],"99562995":[],"99563484":[],"99566069":6,"995702810640425":[],"9957325252769553":[],"9957680433056965":33,"99578809":6,"9958244721767004":[],"9958589197366403":[],"99588057":22,"9958983289118531":[],"9959177108554428":33,"9959308805732706":[],"9959739":[],"9959836634296064":[],"996":[6,34,35],"9960490331171794":33,"99607299":[],"99607686":22,"99608161":6,"9960887274532307":[],"99619572":[],"99624324":[],"9963961":6,"99647992e":[],"99650061":6,"9966129":29,"996626":[],"99662602":[],"99663433":[],"9967181696265849":[],"9967458":6,"9969461173312454":[],"9969513794144311":[],"99700706":6,"99709215":6,"99723951":[],"997252717901263":[],"9972554229935837":[],"99729015":[],"99729756":6,"99751458":6,"99753321e":[],"99755766":[],"99757634":[],"99758326":6,"99759503":[],"99762142":34,"9976783":[],"997746":[],"99774602":[],"99774972":[],"99775587":6,"99782503":22,"99782627":[],"99783284":[],"99788953":22,"99792291":[],"99793613":6,"99799099":6,"99802696":37,"99812668e":[],"99813653":6,"99817613":[],"99828624":6,"9983289":[],"9983295":6,"99833653":38,"99835783":37,"99835872":[],"99839581":[],"99845267":6,"99848215":[],"99848815":37,"99849105":[],"99852187":[],"99854557":33,"9985605":[],"998577":6,"99861053":6,"99861193":[],"99862019":33,"99864751":[],"99869482":33,"99870462":[],"99871521":6,"99876945":33,"99881845":6,"99883628":33,"99884384":6,"99884409":33,"99884986":34,"99887767":[],"99891093":33,"99891873":33,"99893323":6,"99895448":[],"99898558":33,"999":[10,22,29,37,38,40,41],"99901896":6,"99902531":[],"99903755":6,"99906023":33,"9990927157696581":34,"99911427":6,"99912709":33,"99913489":33,"99916114":37,"99918546":6,"99919837":6,"99919981":38,"99920175":33,"99926459":6,"99927642":33,"99929677":[],"9993237":6,"99933188":6,"9993511":33,"99938942":6,"99941797":33,"99942911":[],"99943837":[],"9994385":6,"99944272":6,"9994574":[],"99949266":33,"99949306":6,"99950283":[],"99952851":[],"99953381":6,"99953475":6,"99954278":[],"99954434":[],"99954925":37,"99956033":[],"99956181":[],"9995659":[],"99956735":33,"99957911":6,"99959609":[],"99959681":[],"99961294":6,"99963172":[],"99965056":6,"99966798":[],"99967865":6,"99968451":[],"99968483":[],"99969576":[],"99969642":[],"99969895":[],"99970894":33,"99970988":6,"99973141":[],"9997332":6,"99974365":[],"99975636":[],"99975913":6,"99976375":22,"99977375":[],"99977849":6,"99977853":[],"99978365":33,"99978784":[],"99979896":[],"9997992":[],"99980002":6,"9998161":6,"99983332":[],"99984732":6,"99984836":[],"99985591":[],"99985884":[],"99987324":6,"99989103":37,"99989296":[],"99989476":6,"99989951":29,"99990413":[],"99990706":[],"99991245":[],"99991263":6,"99991295":[],"99991971":[],"99992107":[],"99992477":[],"99992746":6,"99993596":[],"99993978":6,"99994537":[],"99995":6,"99995454":38,"999955585168597":[7,34],"99996471":[],"99996755":[],"99996897":[],"99997244":[],"99997356":[],"99997644":[],"99998727":[],"9999923":38,"99999776":[],"9x":[7,26],"9y":[7,26],"\u00f8yvind":[7,33,34],"\u03b4":[40,41],"abstract":[2,21,37,40,41],"boolean":5,"break":[1,5,7,12,15,32],"byte":[25,32],"case":[0,1,2,3,4,5,6,7,8,12,13,14,15,17,22,24,25,26,27,32,35,38,39,40,41],"catch":1,"class":[1,2,4,5,7,8,9,10,12,13,14,23,29,32,35,37,38,40,41],"default":[1,2,3,5,7,8,9,17,25,26,27,32,33,34,36,39,40,41],"do":[1,3,4,5,6,7,9,10,11,12,13,14,15,16,17,22,23,25,26,27,32,33,36],"ekstr\u00f8m":5,"export":10,"f\u00f8470":[30,32],"final":[0,1,2,3,4,5,6,7,8,9,10,11,12,14,15,17,19,20,21,23,26,28,29,30,32,35,36],"float":[1,4,5,6,10,12,14,15,25,32,33,37,38,40,41],"function":[3,4,5,6,10,15,16,17,18,19,20,23,24,25],"import":[0,1,2,3,4,5,7,8,9,10,11,12,13,14,15,16,17,18,23,26,29,35,36,37,41],"int":[1,2,3,4,5,6,7,12,14,15,22,25,29,33,35,37,38,40,41],"long":[1,2,4,5,13,14,23,32,36,37,38,39,40],"m\u00f8svatn":[7,26],"new":[1,2,3,4,6,7,8,9,10,11,12,14,15,18,22,25,32,33,36,37,38,40,41],"public":[1,24,32],"return":[0,1,2,3,4,5,6,7,8,9,10,12,14,15,22,23,25,26,29,32,33,34,35,36,37,38,39,40,41],"sch\u00f8yen":[7,33,34],"short":[5,6,21,26,27,34,35,40,41],"super":[6,33,34,40,41],"switch":[1,40,41],"throw":[4,7,29,35],"true":[0,1,2,3,4,5,6,7,8,9,10,11,13,14,15,16,17,20,23,25,26,27,29,32,33,34,35,36,37,38,39,40,41],"try":[1,2,3,5,6,7,8,9,10,11,12,14,15,16,17,23,24,25,26,27,29,32,33,34,35,36,37,38,39,40,41],"var":[2,6,7,11,12,19,20,26,29,32,33,34,35,37,40,41],"while":[0,1,2,4,5,6,7,8,9,10,12,13,14,22,23,29,32,33,34,35,36,37,38,39,40,41],A:[3,4,6,7,8,11,12,13,14,17,19,21,24,25,26,27,28,29,30,31,33,37],AND:[0,3,41],And:[1,4,5,6,7,10,14,21,24,26,27,29],As:[1,2,3,4,5,6,7,9,11,13,14,17,25,26,27,29,32,33,34,35,36,37,38,39,40,41],At:[1,5,7,14,26,32,37],BE:[1,32],Be:[3,24,32,41],Being:[14,37],But:[1,2,3,4,6,7,10,11,23,27,29,33,34,35,39,40,41],By:[1,4,6,7,13,14,18,25,32,33,34,35,36,37,38],For:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,22,23,24,25,26,27,28,29,31,32,34,35,36,37,38,39,40,41],IF:[7,34],IN:31,If:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,17,22,23,24,25,26,27,29,32,33,34,35,36,37,38,39,40,41],In:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,20,22,23,24,25,26,27,29,31,32,33,34,35,36,37,38,40,41],Is:12,Ising:[6,13,33,38,39],It:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,22,23,24,25,26,27,29,32,33,34,35,36,37,38,39,40,41],Its:[2,3,5,12,40,41],NO:[0,10,12],No:[1,7,8,10,32,34,36,38,40,41],Not:[1,2,6,7,33,34,35,38,39,40,41],OR:29,Of:29,On:[1,4,10,16,26,28,29,30,31,32],One:[1,2,4,5,6,7,8,9,12,13,14,18,22,26,29,33,35,36,37,38,39,40,41],Or:[1,2,7,26,32,36,39,40],Such:[1,7,13,17,29,35,36,37,38,39,40],TO:[40,41],That:[1,6,8,11,12,13,15,20,26,29,32,34,35,36,39,40],The:[5,11,14,15,16,17,19,20,21,22,23,25,26,27,28,29,30,31],Their:[],Then:[1,2,7,9,10,11,12,13,14,15,23,25,26,32,35,36,37,38,39,40,41],There:[1,4,5,6,7,9,10,12,13,15,25,26,28,29,30,32,33,34,36,37,38,39],These:[1,4,5,6,9,10,11,12,13,14,15,16,17,25,26,27,29,30,32,33,34,37,38,39,40,41],To:[1,2,3,4,5,6,7,8,9,10,11,12,13,14,22,23,25,27,29,34,35,36,37,38,39,40,41],With:[1,6,7,9,10,11,12,13,15,19,23,25,26,27,29,32,33,35,38,39,40,41],_0:[6,9,11,12,14,33,36,37],_1:[3,6,7,9,11,12,13,14,15,25,33,34,36,37,38,39,40,41],_2:[3,6,9,12,13,14,25,33,37,38,41],_3:25,_4:25,_9:[14,37,38],_:[0,1,2,3,5,6,7,8,9,10,11,12,13,14,18,19,20,22,23,25,26,32,33,34,35,36,37,38,40,41],__array_function__:[],__class__:[11,40,41],__doc__:[7,35],__enter__:[1,8,36],__future__:[9,10,39],__getattr__:34,__getattribute__:32,__getitem__:[1,8,36],__init__:[0,1,2,4,5,8,15,34,36,38,40,41],__main__:[0,3,38,41],__mul__:[],__name__:[0,3,11,34,38,40,41],__rsub__:[],__rtruediv__:[],_accessor:32,_accuraci:[40,41],_allclose_dispatch:23,_auto10:[7,13,38],_auto11:[7,38],_auto12:[7,38],_auto1:[3,4,5,6,7,8,13,14,22,25,29,33,36,37,38,39,40,41],_auto2:[3,4,5,6,7,13,14,25,29,37,38,39,40,41],_auto3:[4,5,6,7,13,14,25,37,38,39,40],_auto4:[5,7,13,14,25,37,38],_auto5:[5,7,13,14,25,37,38],_auto6:[5,7,13,25,38],_auto7:[5,7,13,25,38],_auto8:[7,13,38],_auto9:[7,13,38],_backpropag:[40,41],_base:9,_build:[1,18,20,24,26,31,32],_c:[2,40,41],_can_hold_identifiers_and_holds_nam:32,_check_optimize_result:[0,10,12],_class:9,_compon:12,_coordinate_desc:7,_depth:10,_feedforward:[40,41],_format:[40,41],_fraction:10,_generatorcontextmanag:[1,8,36],_i:[1,2,3,6,7,8,9,12,13,14,16,17,18,20,26,32,33,34,35,36,37,38,39,40,41],_info_axi:32,_j:[1,2,3,4,6,7,9,14,18,20,26,33,34,35,37,40,41],_k:[14,36,37,38,40,41],_l:[13,38,39,40,41],_lambda:[7,32],_leaf:10,_ll:[4,5,15],_logist:[0,10,12],_m:11,_make_vjp:[],_multilayer_perceptron:[2,40],_n:[3,6,9,12,14,33,36,37,41],_node:10,_np:[],_o:[4,5,15],_open_file_or_url:[1,8,36],_p:[6,9,33],_plugin_dir:[4,5,15],_progress_bar:[40,41],_ratio:12,_rc_params_in_fil:[1,8,36],_s:[4,5,15],_sampl:10,_set_classif:[40,41],_split:[7,10,26],_src:[22,38],_t:[14,22,37,38],_test:[7,26],_tf_getinputpropertieslist:[4,5,15],_trace:[],_valu:[],_varianc:12,_weight:10,_wrapreduct:[],a0:4,a0faa0:[10,11],a1:[1,23,32],a2:[1,23,32],a3:[1,32],a4:[1,32],a_0:[1,22,27,32,39,40],a_1:[39,40],a_1a:[1,32],a_1x:[22,27],a_2:[39,40],a_2a:[1,32],a_2x:[22,27],a_3:[1,32],a_3a:[1,32],a_4:[1,32],a_4a:[1,32],a_:[1,2,17,25,32,33,39,40,41],a_h:[2,40,41],a_i:[1,2,3,13,32,39,40,41],a_j:[2,13,39,40,41],a_k:[1,2,13,39,40,41],a_matric:[40,41],a_ndim:[],aaron:31,ab:[1,3,6,14,15,22,26,32,33,34,37,38,39,41],ab_channel:[24,38,39,40,41],abandon:[2,39,40],abbrevi:28,abid:29,abil:[1,11],abl:[1,2,5,6,7,8,11,13,14,17,23,26,33,36,37,38,39,40],about:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,24,25,26,27,30,34,35,36,37,38,39,40,41],abov:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,22,23,25,26,27,29,31,32,33,35,36,37,38],abovement:[7,32,35],abscissa:[14,36,37],absolut:[1,3,6,7,14,32,33,34,35,37,38,41],absorb:[33,34],acceler:[14,22,37,38],accept:[1,4,7,10,23,26,32,33],access:[1,4,12,29,32],accid:[5,7,35,36],accompani:[1,32,33],accomplish:[9,10,14,37,38],accord:[1,2,3,6,7,10,13,14,15,16,17,23,29,32,34,35,36,37,38,39,40,41],accordingli:12,account:[1,4,6,14,16,29,32,34,35,37,38],accumul:[13,14,22,29,37,38,39,40,41],accur:[1,4,5,7,11,14,35,37,38],accuraci:[0,1,2,4,5,6,7,8,10,11,12,13,23,27,32,33,36,38,39,40,41],accuracy_scor:[1,2,11,23,32,40,41],accuracy_score_numpi:[2,40,41],acheiv:23,achiev:[1,2,6,7,9,13,25,32,34,35,38,39,40],aco:29,acquaint:24,acquir:[2,24,32,39,40],acr:1,across:[2,4,7,10,24,32,35,40,41],act:[2,4,25,40,41],act_func:[40,41],act_func_deriv:[40,41],actic:23,action:29,activ:[1,3,4,5,10,28,30,32,36,37],activation_func:23,actual:[1,2,5,6,7,9,12,17,23,25,29,32,33,34,35,39,40],ad:[0,2,4,5,6,9,14,16,17,25,34,35,36,37,41],ada_clf:11,adaboostclassifi:11,adadelta:[14,37,38],adagrad:[27,39,40,41],adagradmomentum:[40,41],adam:[2,4,5,23,27,30,32,39,40,41],adam_schedul:[40,41],adap:39,adapt:[5,7,14,31,35,36,39],add:[1,2,3,4,5,6,7,9,11,12,13,16,17,18,22,23,26,27,29,32,33,34,35,37,39,40,41],add_outgrad:[],add_subplot:[0,2,8,13,15,26,36,37,38,39,40,41],add_suplot:41,addendum:6,addit:[0,1,3,4,6,7,8,9,10,11,13,14,16,18,22,23,24,25,26,29,30,31,32,33,34,35,36,37,38,39,40,41],addition:[13,14,36,37,38,39],address:[2,10,12,14,32,37,38,39,40],adjac:[4,13,38,39],adjoint:[6,33,34],adjust:[1,6,13,14,36,37,38],admir:[1,32],advanc:[5,7,13,31,32,35,39],advantag:[2,4,6,7,11,14,25,34,35,36,37,38,40,41],adversari:32,afecionado:32,affect:[4,40,41],affin:[1,4,9,12,33,39],afford:4,aficionado:32,aforement:15,african:1,after:[1,2,3,5,6,7,10,12,13,14,16,22,23,24,25,26,27,29,32,33,34,35,37,38,39,40,41],afterward:[1,32],ag:[1,8,28,32,33,36],ag_0:[3,41],again:[1,2,5,6,7,8,9,11,12,13,14,16,17,22,26,27,29,32,33,34,35,37,38,39,40,41],against:[2,5,8,11,22,27,36,40,41],agegroup:[8,36],agegroupmean:[8,36],aggreg:[10,11],agorithm:11,agre:[6,7,29,34,35],agreement:[14,22,37,38],ahead:10,ai:[1,27,31],aid:[12,21],aim:[1,2,5,7,8,12,15,16,17,24,25,26,27,33,35,36,39,40],ainv:6,airplan:4,aka:[6,34,35],al:[1,3,5,16,17,18,26,27,31,32,33,34,35,36,37,39,40,41],alarm:[6,8,34,35],aldo:33,alexsmola:41,algebra:[1,4,6,14,22,24,33,34,35,37,38],algorithm:[0,1,2,3,5,6,7,8,9,14,15,24,25,26,29,31,32,34,35,36],align:[0,1,3,6,7,8,9,14,26,29,32,33,34,35,36,37,41],all:[1,2,3,4,5,6,7,8,10,11,12,13,14,15,16,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,41],allclos:23,allevi:[2,14,36,37,39,40],alloc:[4,25],allow:[1,2,3,4,6,7,9,11,14,16,17,24,25,26,32,33,34,35,36,37,38,39,40,41],almost:[1,2,7,9,12,14,22,29,35,36,37,38,39,40],alon:[3,10,41],along:[3,4,5,6,7,10,11,12,23,24,25,32,33,34,35,36,40,41],alpha:[0,1,2,3,4,5,7,8,9,10,11,14,15,29,32,33,35,36,37,39,40,41],alpha_0:4,alpha_1:4,alpha_2:4,alpha_:11,alpha_i:[4,14,37],alpha_k:[14,37],alpha_m:11,alpha_n:4,alpha_opt:[14,37],alreadi:[3,4,5,6,7,11,13,24,25,29,32,33,34,35,38,39,41],also:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,22,23,24,25,26,27,28,29,32,33,34,35,36,37,38,39,40,41],alter:[2,40,41],altern:[0,1,2,5,6,7,9,10,12,14,25,26,32,33,34,35,36,37,40,41],although:[1,2,6,7,9,11,14,17,22,32,34,35,37,38,39,40,41],alwai:[1,4,6,7,13,14,17,22,23,26,29,32,33,34,35,36,37,38,39],am:5,ambit:[39,40],ame2016:[1,32],american:1,among:[1,4,6,10,11,13,25,32,33,34,38,39],amongst:[6,34,35],amount:[1,2,4,5,7,9,11,15,24,35,39,40,41],amp:[38,39,40,41],an:[0,2,3,4,6,7,8,9,10,12,13,14,15,16,17,18,20,21,22,23,24,25,26,27,29,30,31,33,34,35,36,37,38,40,41],an_:29,anaconda:[1,2,16,24,26,32,40,41],analog:[14,37,38],analys:[7,34,35],analysi:[0,2,4,5,8,15,16,17,18,20,22,25,31,36,40,41],analyt:[3,4,6,7,8,13,14,16,24,26,27,32,33,34,35,36,37,38,39],analytical_gradi:[22,38],analyz:[1,2,4,5,6,7,17,18,26,27,29,33,34,41],andrew:[2,39,40,41],angl:[1,4,10,33],anharmon:4,ani:[0,1,2,3,4,5,6,7,8,9,10,11,13,15,19,23,29,32,33,34,35,38,39,40,41],anim:[5,13,38,39],ann:[13,38,39],annot:[0,1,2,4,8,9,32,36,40,41],announc:32,anoth:[1,2,3,4,5,6,7,8,9,11,12,13,14,25,26,27,29,32,33,36,37,38,39,40,41],anp:[],ans_vspac:[],ansatz:[1,32],answer:[1,2,4,6,7,25,26,27,30,32,34,35,40,41],antialias:[3,7,26,41],anticip:5,anymor:[2,9,39,40],anyon:[5,9],anyth:[2,23,29,39,40,41],anytim:[30,32],apach:[2,40,41],apart:[12,14,36,37],api:[2,24,32,40,41],appar:[3,41],appear:[1,2,4,14,17,25,29,37,38,39,40,41],append:[2,4,5,9,10,14,22,23,32,37,38,40,41],appendix:26,appli:[1,2,4,5,7,8,9,10,11,12,13,14,16,17,26,27,29,31,32,33,34,35,36,37,38,39,40,41],applic:[0,1,2,4,5,6,7,8,10,13,14,22,25,26,29,31,32,33,35,36,37,38,39,40,41],apply_gradi:5,approach:[0,2,3,5,6,7,10,11,12,13,14,18,22,23,24,29,31,33,36,39,40,41],appropri:[3,7,10,13,14,24,29,35,37,38,41],approv:32,approx:[1,3,4,7,11,12,14,29,32,35,36,37,38,41],approxim:[1,2,3,4,5,6,7,8,11,12,14,19,20,26,29,32,33,34,35,36,37,38,40,41],apt:[1,16,24,26,32],aq:29,ar:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,20,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41],aragorn:32,arang:[0,2,4,5,7,8,10,11,13,14,26,32,36,37,38,39,40,41],arbitrari:[2,5,7,9,13,14,29,35,36,37,38,39,40,41],arbitrarili:[1,2,12,32,40,41],arc:[7,26],architectur:[4,5,13,39],archiv:27,area:[1,4,7,10,26,31,32],arg:[3,23],argmax:[2,12,23,40,41],argmin:[5,11,15],argnum:[],argnum_0:[],argnum_1:[],argsort:12,argu:[2,14,37,38,39,40],argument:[1,3,4,6,7,12,13,14,22,23,32,33,34,35,39,40,41],argval:[],aris:[1,7,13,14,29,32,35,36,37],arithmet:[1,14,25,32,37,38],arm:[7,34],armadillo:25,arnulf:[39,40,41],around:[1,2,5,6,7,12,23,29,32,34,35,39,40,41],arr:[],arr_list:[],arrai:[0,1,2,3,4,5,6,7,8,9,10,12,13,14,15,22,23,24,26,29,33,34,35,36,37,38,39,40,41],arrang:[4,32],array_function_dispatch:23,arraybox:[14,37],arriv:[1,7,10,12,25,29,32,35],arrow:[13,38,39,40,41],arrowprop:9,art3d:37,art:[1,2,16,24,32,39,40],articl:[1,4,5,7,11,19,26,27,32,33,34,35],artifici:[0,1,3,8,13,31,32,36,41],artificialneuron:[13,38,39],arug:[14,37,38],arxiv:[4,5,22,26,37,38,39,41],asanyarrai:23,asarrai:[1,3,7,10,22,33,34,37,38],asid:33,ask:[6,7,12,13,26,34,35,39,40],aspect:[1,7,24,26,32,33,34,39,40,41],assembl:[1,4,32],assert:[5,40,41],assess:[1,7,26,32,33,34,35],assici:5,assign:[0,1,8,9,10,13,14,15,28,30,31,32,36,37,38,39,40],associ:[1,7,10,13,15,29,32,35,38,39],assum:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,20,25,26,29,32,33,34,35,36,37,38,39,40,41],assumpt:[1,4,6,7,10,12,19,26,29,32,33],ast:[1,6,7,19,26,32,34,35,36],astronomi:[0,32,34,36,37,38,40,41],astyp:[5,10,11],asymmetri:[1,32],asymptot:[5,7,35],atleast_2d:[],atol:23,atom:[1,32],attempt:[1,5,7,8,9,11,32,34,36,39,40],attend:28,attent:[1,25,32],attr:34,attract:[1,11,32],attribut:[1,3,10,27,32,34,40,41],attributeerror:[3,32,34],audi:[1,32],audio:[4,5],august:[16,17,32],aurelien:[1,16,28,31,32],austfjel:[7,26],author:[1,2,11,29,39,40],authour:32,auto:[9,10,11,29,40,41],auto_exampl:[23,26,34],autocor:29,autocorrelation_tim:29,autocorrelform:29,autocovari:29,autoencod:[5,24,32],autoencond:24,autograd:[23,24,27,32,39,40],autom:[1,24,31,32],automac:25,automag:32,automat:[1,2,3,4,5,12,17,23,24,25,32,40],automobil:4,autonom:5,avail:[1,2,5,7,8,11,12,16,21,22,24,25,26,27,28,30,31,32,35,36,38,40,41],averag:[1,2,4,7,10,11,14,15,22,29,30,32,33,34,35,36,37,38,40,41],avoid:[1,5,6,7,10,12,14,22,23,25,32,33,35,36,37,38,40,41],awai:[3,4,7,33,34,39,41],awar:[3,11,41],award:[30,32],ax:[0,1,2,3,4,5,7,8,9,10,11,12,13,14,15,23,25,26,27,32,35,36,37,38,39,40,41],axes3d:[3,7,14,26,36,37,41],axes_grid1:7,axhlin:9,axi:[1,2,3,4,5,7,8,9,10,11,12,13,14,15,22,23,25,26,29,32,33,34,35,36,37,38,39,40,41],axiom:[6,34,35],axis_arg:[],axlabel:[1,32],axvlin:[5,9],axvspan:5,b1:[9,23],b2:[9,23],b3:9,b:[1,2,4,5,6,7,9,10,11,13,14,15,16,17,22,23,29,30,32,33,34,35,36,38,39,40,41],b_0:[1,39],b_1:[1,3,13,14,37,38,39,40,41],b_2:[1,14,37,39,40],b_5:[14,37,38],b_:[1,2,25,39,40,41],b_g:23,b_group:10,b_i:[1,2,3,13,32,38,39,40,41],b_ia_:[1,32],b_ia_i:1,b_index:10,b_is_vec:[],b_j:[2,13,38,39,40,41],b_k:[1,2,13,14,37,38,39,40,41],b_m:[13,38],b_meta:[],b_score:10,b_valu:10,babcock:32,bachelor:[28,30],back:[1,4,5,6,7,9,10,11,16,23,25,27,29,32,33,34,36,37,38],backbon:25,backend:[1,2,5,8,22,36,38,40,41],background:[31,32,33,39,40,41],backpropag:[2,23,39,40,41],backtrack:10,backup:25,backward:[2,3,5,13,25,39,40,41],backward_pass:[],bad:[7,33,34,40,41],badli:29,bag:[10,24,32],bag_clf:11,baggin:32,baggingboot:11,baggingclassifi:11,baggingtre:11,balanc:[7,35,36],balanced_eq:[],baluka:41,band:25,bandwidth:25,bar:[1,7,12,16,17,26,32,40,41],barber:31,bare:[5,11],base:[1,2,4,5,6,8,9,10,11,15,24,29,30,31,32,33,34,36,39,40,41],basi:[6,8,9,11,12,13,14,25,33,34,36,37,38,39],basic:[7,9,13,14,15,24,26,29,32,37,38,40],batch:[4,5,12,13,14,22,23,27,36,41],batch_shap:5,batch_siz:[2,4,5,40,41],batchnorm:5,bay:[0,8,36],baydin:39,bayesian:[6,24,31,32,34,35],beam:[0,32,36,37,38,40,41],becattini:30,becaus:[1,2,3,4,5,6,7,9,10,13,14,15,32,34,35,36,37,38,39,40,41],becom:[0,1,2,3,6,7,8,10,13,14,22,29,32,33,34,35,36,37,38,39,40,41],been:[1,2,3,4,5,6,7,12,13,14,24,25,26,27,32,33,34,35,37,38,39,40,41],befor:[1,2,3,4,5,6,7,8,9,13,14,15,21,23,25,26,27,29,32,33,34,35,36,37,38,39,40,41],beforehand:[1,29,32],begin:[0,1,2,3,4,5,6,7,8,9,10,12,13,14,15,22,25,26,29,30,32,33,34,35,36,37,38,39,40,41],behav:[2,7,14,35,36,37,39,40],behavior:[1,2,14,32,36,37,38,39,40],behaviour:[13,38,39,40],behind:[1,2,7,9,14,32,36,37,40,41],being:[0,1,2,3,4,5,6,8,9,11,12,13,14,18,29,32,33,34,35,36,37,38,39,40,41],believ:[10,25],belong:[0,8,9,10,14,15,36,37,40,41],below:[1,2,3,4,5,6,7,8,9,10,11,12,13,14,23,25,26,27,29,32,33,34,35,36,37,38,39,40,41],benchmark:11,benefici:[2,14,37,38,40,41],benefit:[1,2,5,12,14,16,24,32,36,37,38,40,41],bengio:[2,22,27,28,31,32,33,37,38],benign:[0,2,8,10,36,40,41],benno:[39,40,41],berner:[39,40,41],besid:[5,6,34],bessel:[6,33,34,35],best:[0,1,2,3,4,5,6,7,8,9,10,11,13,14,23,26,27,30,32,33,35,36,37,39,40,41],best_estimator_:36,beta1:[22,37,38],beta2:[22,37,38],beta:[0,1,2,4,6,7,8,11,12,14,17,18,19,20,22,26,32,33,36,37,38,40,41],beta_0:[0,1,2,4,6,7,8,14,32,33,34,36,37,40,41],beta_0x_:[1,32,33],beta_1:[0,1,2,4,6,7,8,11,14,32,33,34,36,37,38,40,41],beta_1x_0:[1,32],beta_1x_1:[0,1,8,32,36],beta_1x_2:[1,32],beta_1x_:[1,32,33],beta_1x_i:[0,8,14,33,36,37],beta_2:[1,4,14,32,33,37,38],beta_2x_0:[1,32],beta_2x_1:[1,32],beta_2x_2:[0,1,8,32,36],beta_2x_:[1,32,33],beta_2x_i:33,beta_3:4,beta_3x_i:33,beta_4x_i:33,beta_:[0,1,4,7,8,14,32,33,34,36,37,38],beta_i:[1,4,6,32,33,34],beta_j:[1,6,7,14,19,26,32,33,34,35,37,38],beta_k:[14,36,37],beta_lasso:38,beta_linreg:[14,22,36,37,38],beta_logreg:[0,38],beta_m:11,beta_mg_m:11,beta_n:4,beta_ol:38,beta_p:[0,8,36],beta_px_p:[0,8,36],beta_ridg:38,betavalu:6,better:[1,2,3,4,5,7,10,11,12,13,14,22,32,33,34,35,37,40,41],between:[0,1,2,3,4,5,6,7,8,9,10,12,13,14,15,18,22,26,27,29,32,33,34,35,36,37,38,39,40,41],beyond:[1,2,6,7,9,14,22,26,32,33,36,37,38,39,40],bf:[14,15,25,29,36,37],bg:32,bgd:[14,37,38],bia:[1,2,3,4,6,9,10,11,13,14,20,23,32,33,34,36,37,38,39,40,41],bias:[2,3,4,6,7,10,13,23,27,34,38],bibliographi:26,big:[1,2,3,6,7,15,19,34,35,40,41],bigger:[2,7,33,34,39,40],bigr:[13,38],bike:10,bilbo:32,billion:[4,13,24,38,39],bin:[0,1,8,29,36],binari:[0,1,4,6,8,10,11,13,27,32,34,35,36],binarycrossentropi:5,bind:[1,33],binomi:[24,29,32],binsboot:[7,35],bioinformat:1,biolog:[2,13,38,39,40],bios1100:[24,32],bird:[1,4],birth:32,bishop:[28,31,32],bit:[2,5,23,25,29,32,39,40,41],bitwis:29,bivari:[3,41],bk:[1,14,37,38],bla:[25,32],black:[9,10,15,22,38],blob:[17,21,26,36,37,38],block:[7,11,24,25,29,32,35],blog:[27,32,41],blogpost:5,blue:[1,4],bmatrix:[1,2,4,6,8,9,12,14,25,32,33,34,36,37,39,40,41],bmi:[2,40,41],bn:[],bodi:[1,2,5,13,32,38,39,40,41],bold:[2,40,41],boldfac:[1,6,17,33,34],boldsymbol:[0,1,2,3,4,6,7,8,9,11,12,14,15,16,17,18,19,20,22,26,32,36,37,38,39,40,41],boltzmann:[13,24,32,38,39],book1:31,book:[20,26,27,31,32,33,35],bool:23,boost:[2,10,24,32,39,40],boostrap:11,bootstrap:[2,14,20,24,26,27,32,36,37,38,39,40,41],borrow:32,boston_dataset:1,bot:9,both:[1,2,5,6,7,9,10,11,14,15,16,17,24,25,26,27,29,30,32,33,34,35,36,37,38,39,40,41],bottl:[8,36],bound:[1,9,13,37,38,39,40,41],boundari:[3,5,9,12,13,41],box:[5,10,23],boxed_arg:[],boyd:[9,14,36,37],bracket:[5,29],brain:[2,8,13,36,38,39,40,41],branch:10,breast:[0,6,8,12,27,34,35,36],breat:27,breviti:[14,22,37,38],brew:[1,16,24,26,32],brg:9,brief:[26,27,33],briefli:[1,32],bring:[1,6,7,11,27,33,34],broad:1,broadcast:23,broadcast_ax:[],broadcast_idx:[],broadcast_to:[],brought:[14,22,24,32,37,38],brownle:5,browser:32,brute:[4,6,12,33,34,36,39],bs:[9,10,11],buffer_s:5,bui:5,build:[1,5,6,7,11,25,29,32,34,35,36,37,38,39],built:[1,2,4,5,7,35,40,41],bunch:12,busi:1,bxe2t:[38,39,40],c1:[9,12],c2:[9,12],c:[0,1,2,3,5,6,7,8,9,10,11,12,13,14,15,16,17,20,23,24,25,29,30,31,33,34,35,36,37,38,39,40,41],c_0:29,c_1:[13,38],c_2:[13,38],c_3:[13,38],c_4:[13,38],c_:[1,9,10,11,14,22,29,36,37,38],c_i:[13,14,37,38],c_k:29,ca:[2,32,39],cach:11,cal:[1,9,11,13,14,16,17,32,36,37,39,40,41],calcul:[0,1,2,3,5,6,7,9,10,11,12,13,14,15,16,17,19,22,25,27,29,32,34,35,36,37,38,39,40,41],california:27,call:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,17,19,20,22,23,24,25,26,27,29,30,32,33,34,35,36,37,38,39,40,41],callabl:[40,41],calor:[1,33],cambridg:[14,31,36,37,39,40,41],can:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,23,24,25,26,27,29,30,31,33,34,36,40,41],cancel:[1,14,32,33,37,38],cancer:[6,11,27,34,35],cancerpd:[0,8,36],candid:[9,10,11,37],cannot:[0,1,2,5,6,7,8,9,10,28,29,33,34,36,40,41],canopi:[1,16,24,26,32],canva:[26,27,32],cap:[6,34,35],capabl:[1,2,9,14,24,32,37,38,40,41],capac:[3,30,41],capita:1,caption:[26,27],captur:[5,12,13,38,39],car:[4,5],card:[1,8,32,36],cardin:[2,39,40],care:[12,36],carefulli:[14,37],carlo:[1,7,24,29,31,32,35],carri:[3,7,8,26,35,36,41],cart:11,casella:31,cast:[2,23,39,40],cat:[4,5],categor:[1,2,4,10,12,32,40,41],categori:[0,1,2,4,8,11,13,15,32,36,38,39,40,41],categorical_crossentropi:[2,4,40,41],caus:[1,6,7,8,29,32,33,34,35,36],causal:1,causat:[1,32],cax:[2,40,41],cb:[7,32],cbar:[2,40,41],cc:[1,2,6,14,27,32,33,34,36,37,39,40,41],ccc:[6,13,34,38],cd_fast:7,cdf:29,cdot:[1,3,7,13,14,15,25,29,32,35,36,37,38,41],celebr:[14,36,37],cell:[1,2,3,4,5,7,8,9,10,11,14,15,23,26,34,36,40,41],center:[0,1,2,7,8,9,10,12,15,26,29,32,34,35,36,39,40,41],central:[1,4,6,7,9,17,25,27,32,33,34,39,40],centroid:[15,29],centroid_differ:15,centuri:4,certain:[1,4,7,8,10,23,29,32,33,35,36],certainti:35,cg:[14,37],cha:1,chain:[1,2,14,24,29,32,37,38,41],challeng:[37,38,39],chanc:[2,6,14,29,34,35,37,38,40,41],chang:[1,2,3,4,5,6,7,9,10,12,13,14,15,22,23,25,26,27,29,32,33,34,35,36,37,38,39,40,41],channel:4,chap4:[39,40],chapter3:[1,20,26],chapter:[1,7,11,12,17,19,22,25,26,27,31,32,33,34,35,36,37,38,39,40,41],charact:[1,4,6,9,32,33],character:[9,10,11,13,29,38,39,40,41],characterist:[1,2,4,11,14,22,32,37,38,40,41],charg:[1,32],charl:1,chase:5,chatgpt:26,chd:[8,36],chddata:[8,36],cheap:[6,33],cheaper:[2,14,37,38,39,40],check:[1,2,4,5,6,7,12,14,23,25,32,37,38,39,40,41],checkmark:4,checkpoint:5,checkpoint_dir:5,checkpoint_prefix:5,chen:11,cheng:33,chiaramont:[3,41],choic:[1,2,3,4,5,7,10,13,14,15,20,25,27,32,33,35,36,37,38,41],choleski:[6,25,33],choos:[3,4,7,10,11,12,14,15,23,26,27,35,36,37,38],chosen:[1,2,3,7,9,10,11,14,17,22,27,29,32,35,36,37,38,39,40,41],chosen_datapoint:[2,40,41],christian:31,christoph:[28,31,32],cifar10:4,cifar:4,circ:[2,13,39,40,41],circl:[1,9,13,33,38,39],circuit:4,circumfer:10,circumv:[2,6,14,33,37,38,39,40],cite:26,ckpt:5,clarifi:23,clariti:29,class_nam:[4,10],class_val:10,class_valu:10,classic:[0,8,10,14,36,37,38,39],classif:[1,4,6,7,8,9,12,13,23,24,26,31,32,33,34,35,37,39],classifi:[0,1,2,5,8,10,11,12,27,32,40,41],classificaton:[2,40,41],classifii:11,clean:[2,40,41],clear:[2,6,11,13,14,22,37,38,40,41],clearli:[0,1,4,6,7,8,9,19,29,33,34,35,36],clever:[2,11,40,41],clf3:1,clf:[1,7,9,10,11,32,33],clf_lasso:7,clf_ridg:[7,32],clip:[4,29],clone:30,close:[1,2,3,5,7,9,10,12,13,14,15,26,29,31,32,35,36,37,38,39,40,41],closer:[4,6,14,33,37,38],closest:[9,12,14,15,37],closur:[24,32],cloud:[24,32],cluster:[1,2,5,7,12,24,32,35,40,41],cluster_label:15,cm:[2,3,4,7,9,10,14,26,36,37,40,41],cmap:[1,2,3,4,5,7,9,10,11,26,32,40,41],cmap_arg:7,cmb:28,cmd:10,cn_:29,cnn:[13,38,39],cnn_kera:4,cntk:[24,32],co:[1,3,4,7,10,14,22,32,35,37,38,41],code:[0,4,5,7,8,9,16,17,18,20,23,24,25,26,29,31],codebas:[40,41],coef0:9,coef:[1,32],coef_:[1,6,7,9,10,14,32,33,34,36,37],coeff:6,coeffici:[1,4,6,7,8,9,10,14,16,17,25,32,33,34,35,36,37,38],coerc:[1,7,23,32,35],coin:[11,29],coin_toss:11,col:[1,12,32,33],colab:[23,24,32],cold:10,colinear:1,collabor:[26,27],collaps:9,collect:[1,3,7,11,12,22,24,29,31,32,35,38,39],collinear:[6,33,34],color:[1,4,5,7,9,10,11,22,26,29,37,38],color_channel:4,color_cod:7,colorbar:[2,7,26,40,41],coloumn:[40,41],colsample_bytre:11,colsaobject:11,column:[0,1,2,3,6,7,8,9,10,12,13,18,25,32,33,34,35,36,38,39,40,41],columntransform:10,com:[5,7,17,21,22,23,24,26,27,31,32,35,36,37,38,39,40,41],combin:[2,3,6,7,8,11,29,34,35,36,37,38,39,40,41],come:[1,2,4,5,6,13,14,15,16,27,32,33,34,35,37,38,39,40,41],command:[1,2,32,40,41],comment:[1,5,6,7,16,17,26,27,32,33],commerci:[1,16,24,26,32],commod:[1,32],common:[0,1,2,4,6,7,8,10,12,14,15,26,27,29,32,33,35,36,37,38,39,40,41],commonli:[1,2,5,7,8,10,14,15,33,35,36,37,38,40,41],commun:[1,13,26,38,39],commut:4,commutatitav:4,compact:[1,2,4,6,7,8,10,12,13,14,15,23,32,33,34,35,37,38,41],compair:1,compar:[1,4,5,6,7,12,14,16,17,22,25,26,27,32,33,34,35,36,37,38,39],comparison:[3,5,14,27,37,38,41],compat:[8,36],compet:1,competit:11,compil:[1,2,4,5,14,16,22,24,25,32,37,38,40,41],compl:23,complet:[0,1,3,4,5,10,13,23,32,38,41],completenn:[13,38],complex:[2,6,9,10,12,13,14,16,17,20,27,32,35,37,38,39,40],complic:[1,2,10,14,32,35,36,40,41],compoment:33,compon:[0,1,2,4,5,6,7,8,10,15,17,24,32,33,34,35,36,39,40,41],components_:12,compos:[10,13,14,15,22,24,32,37,38,39],compphys:[1,7,17,18,20,21,24,26,28,30,31,32,36,37,38],compress:[1,32,33],compris:[7,36],compromis:[6,33],compulsori:[24,32],comput:[0,1,2,3,4,5,6,7,8,9,11,12,13,14,16,17,18,22,23,24,25,26,27,28,29,31,32,33,34,35,36,39,40,41],computation:[1,4,7,10,14,29,32,36,37,38,39],computationalscienceuio:32,computerlab:[26,27],con:27,concaten:[3,5,7,15,41],concatenate_arg:[],concav:[2,10,14,33,36,37,40,41],concentr:[1,11],concept:[1,3,24,32,33,41],conceptu:[13,14,36,37,38,39],concern:[1,2,5,8,32,36,40,41],concic:32,conclud:[1,6,14,22,34,35,37,38],conclus:[2,40,41],cond:[3,41],conda:[1,2,16,24,26,32,40,41],condis:33,condit:[1,3,5,6,7,9,10,12,14,29,32,33,38,41],conduct:24,condwav:[3,41],confid:[1,6,7,8,9,19,26,32,33,34,36],configur:[4,22,38],confirm:[6,13,23,34,35,38],confus:[6,7,8,11,25,33,34,35],confusion_matrix:10,congruenti:29,conjug:[5,9],conjugaci:[14,37],conjunct:4,connect:[1,2,4,5,10,12,13,14,25,32,33,36,37,38,39,40,41],consequ:[6,7,9,11,13,14,33,34,35,36,37,38],conserv:[6,15,33],consid:[1,2,3,4,6,7,8,9,10,11,13,14,17,20,22,25,26,27,29,32,33,34,35,36,37,38,39,40,41],consider:[1,2,6,14,32,33,34,35,36,37,41],consist:[1,2,3,4,5,7,13,14,20,22,26,27,29,33,35,36,37,38,39,40,41],constant:[1,3,5,6,7,9,13,14,29,32,33,34,35,36,37,38,39,40,41],constitu:[1,32],constitut:[3,7,35,36,41],constrain:[2,4,6,8,12,34,36,40,41],constraint:[6,7,9,14,33,34,35,37,38],construct:[0,1,2,3,4,6,7,8,9,10,11,12,25,29,32,33,34,35,36],consult:27,contact:[1,32],contain:[0,1,3,4,5,6,7,8,9,10,12,13,14,18,20,22,23,25,26,27,29,31,32,33,34,35,36,37,38,39,40,41],contemporari:32,content:[2,24,25,32,40,41],context:[7,11,14,26,35,36,37,38,39],contextlib:[1,8,36],contigu:25,continu:[1,2,3,4,5,6,7,8,9,10,11,13,14,18,19,22,25,26,27,29,32,33,34,35,36,37,39,40,41],contour:[10,11,14,37],contourf:[9,10,11],contrast:[2,5,10,11,13,38,39,40],contribut:[1,4,6,14,22,29,32,33,34,37,38],contributor:[1,26],control:[1,2,4,10,14,16,24,32,37,38,40,41],conv2d:[4,5],conv2dtranspos:5,conv:[4,5],convei:32,conveni:[1,6,7,13,14,25,26,27,32,34,35,36,37,38],convent:[13,33],converg:[0,2,3,5,6,7,9,10,12,14,15,22,27,33,34,36,37,38,39,40,41],convergencewarn:[0,2,7,9,10,12,40],convers:37,convert:[1,2,5,6,10,12,14,25,32,33,37,38,40,41],converttomatrix:5,convex:[0,5,6,8,33],convinc:[14,36,37],convolut:[2,5,24,32,40,41],cook:26,cool:[5,10],coolwarm:[7,26],coordin:[6,13,15,33,34,38],coorel:1,copi:[1,2,15,33,40,41],copyright:27,core:[1,3,8,11,23,32,36],corel:32,coronari:[8,36],corr:[0,1,6,8,12,33,36],correalt:[12,24],correct:[1,2,3,4,5,6,8,14,23,25,29,32,33,34,35,37,38,40,41],correctli:[2,3,7,8,11,23,26,27,40,41],correl:[1,2,4,6,7,8,11,13,14,22,24,29,32,34,35,37,38,39,40,41],correlation_matrix:[0,1,6,8,12,33,36],correspond:[1,4,6,7,9,10,12,13,16,17,18,24,25,26,29,32,33,34,35,38,39],cortex:[13,38,39],cosin:[4,7,35],cost:[1,3,4,6,7,8,9,10,13,14,17,18,20,22,23,26,27,32,38],cost_deep_grad:[3,41],cost_func:[40,41],cost_func_deriv:[40,41],cost_funct:[3,41],cost_function_deep:[3,41],cost_function_deep_grad:[3,41],cost_function_grad:[3,41],cost_function_train:[40,41],cost_function_v:[40,41],cost_grad:[3,41],cost_sum:[3,41],costcrossentropi:[40,41],costlogreg:[40,41],costol:[14,22,37,38,40,41],could:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,18,23,25,26,27,29,32,33,34,35,36,37,38,39,40,41],coulomb:[1,32],count:[1,10,28,29,30,32],countor:[14,37],coupl:[5,6,7,23,34,35],cours:[1,2,4,6,12,23,26,27,28,30,33,35,40,41],courvil:[22,27,28,31,32,33,37,38],cov:[6,7,12,25,29,32,33,34,35],cov_xi:[6,12,33],cov_xx:[6,12,33],cov_yi:[6,12,33],covari:[0,1,8,24,25,32,34,36],covariance_matrix:[6,12,15,33],cover:[1,6,18,24,30,31,33,34],covert:[1,32],covxi:29,covxx:29,covxz:29,covyi:29,covyz:29,covzz:29,cpu:[2,40,41],cqofi41lfdw:[39,40],craft:4,creat:[2,4,5,6,10,11,12,13,23,24,32,34,36,37,38,39,40,41],create_biases_and_weight:[2,40,41],create_convolutional_neural_network_kera:4,create_lay:23,create_layers_4:[],create_layers_batch:23,create_neural_network_kera:[2,40,41],create_x:[6,12,33,40,41],credit:[1,8,30,32,36],crim:1,crime:1,criteria:[1,5,10,11,15,29,32],criterion:[10,11,14,36,37,39],critic:[7,26,33,34],critiqu:[26,27],cross:[1,2,4,8,10,11,14,23,24,27,29,32,33,34,37,38,39,40,41],cross_entropi:[5,23],cross_val_scor:[7,35,36],cross_valid:[0,8,11,36],crossvalid:[7,35],crucial:[2,29,39,40],cs231:4,cs231n:41,cs231n_2017_lecture4:41,cs:[28,30],csr_matrix:[25,32],csv:[1,5,7,8,10,32,35,36],ctnk:[2,40,41],cube:39,cubic:1,cumbersom:[6,34,35],cumsum:[11,12,32],cumul:[8,11,29],cumulative_heads_ratio:11,cup:[6,34,35],current:[2,3,4,5,14,15,22,31,36,37,38,39,40,41],curs:[1,33],curv:[0,7,8,11,13,26,36,38,39,40],curvatur:[14,36,37,38],custom:[7,15,26],custom_cmap2:[10,11],custom_cmap:[10,11],cutpoint:10,cv:[0,7,8,11,35,36],cvxbook:[14,36],cvxopt:[6,9,33],cybenko:39,cycl:[2,13,38,39,40,41],cyclotron:34,d2_g_t:[3,41],d:[2,3,4,5,6,7,8,9,10,11,12,14,15,23,25,29,30,32,33,34,35,36,37,38,39,40,41],d_f:[14,36,37],d_g_t:[3,41],d_net_out:[3,41],da:[4,39],dagger:[6,25,33],dai:[2,10,24,40,41],damp:4,daniel:30,darget:10,darkr:29,dat:[1,32],dat_id:[1,7,8,10,32,33,35,36],data1:15,data2:15,data3:15,data4:15,data:[3,5,6,9,11,13,14,15,18,19,20,22,25,27,31,35,37,38],data_id:[1,7,8,10,32,33,35,36],data_indic:[2,40,41],data_panda:32,data_path:[1,7,8,10,32,33,35,36],databas:[2,40,41],datafil:[1,7,8,10,26,32,33,35,36],datafram:[0,1,5,6,8,10,12,32,33,36],datapoint:[0,2,6,7,8,12,14,33,35,36,37,38,40,41],dataset:[0,1,5,7,8,9,10,11,12,14,15,16,17,20,23,26,27,32,35,36,37,38,39],datatyp:5,date:[0,16,17,18,19,20,21,22,23,26,27,32,33,34,35,36,37,38,39,40,41],daughter:11,david:[31,32],davison:35,db:39,dbh:[2,40,41],dbo:[2,40,41],dc5e85cd93c3:27,dcomposit:25,ddot:[3,41],dead:[2,39,40,41],deadlin:[17,18,19,20,21,22,23,28],deal:[1,2,4,6,7,9,12,14,15,18,19,25,26,29,32,33,34,36,37,38,39,40,41],dealt:1,debt:[8,36],debug:[1,6,7,34,35,36,40,41],decad:[1,4,38],decai:[1,14,29,32],decemb:[28,30,32],decent:11,decid:[1,3,4,6,7,10,26,34,35,36,40,41],decim:[1,32,40,41],decis:[1,2,9,12,24,31,32,40,41],decision_funct:9,decision_tre:10,decisiontreeclassifi:[10,11],decisiontreeregressor:[1,10,11],decisiontreeregressordecisiontreeregressor:10,declar:[1,5,25,32],decompos:[6,7,25,33,39],decomposit:[1,7,13,26,32,34,38,39],decompost:[6,33],deconvolut:4,decorrel:[11,14,22,37,38],decreas:[2,3,5,6,7,11,12,14,34,35,36,37,38,40,41],deduc:[1,32],deep:[4,8,13,14,22,24,27,28,31,33,37,38],deep_neural_network:[3,41],deep_param:[3,41],deep_tree_clf1:10,deep_tree_clf2:10,deep_tree_clf:[10,11],deepcopi:[40,41],deepen:[6,24,32],deeper:[1,4,5,32],deepimag:41,deeplearningbook:[27,31,32,34],deer:4,def:[0,1,2,3,4,5,6,7,8,9,10,11,12,14,15,22,23,26,29,32,33,34,35,36,37,38,39,40,41],def_covari:29,default_tim:5,defect:[6,33],defici:[6,33],defin:[0,1,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,22,23,25,26,27,29,33,34,35,36,37,38],definit:[2,3,6,7,8,9,11,12,13,14,18,25,29,33,34,35,36,37,41],defint:29,defualt:[40,41],defvjp:[],defvjp_argnum:[],degre:[4,6,7,9,10,11,12,16,17,18,20,26,29,32,34,35,36],deisenroth:33,del:[2,40,41],delet:7,delimit:5,deliv:[28,32],delta:[1,3,4,7,9,13,14,15,22,32,37,38,39,40,41],delta_0:[4,39],delta_1:[4,39,40],delta_2:[4,39,40],delta_2a_1:[39,40],delta_3:4,delta_4:4,delta_5:4,delta_:[2,25,39,40,41],delta_h:[1,2,32,40,41],delta_i:[39,40],delta_j:[4,13,39,40,41],delta_k:[13,39,40,41],delta_l:[2,4,40,41],delta_matrix:[40,41],delta_momentum:[14,22,37,38],delta_n:[1,4,32],delug:24,delv:1,demand:[14,36,37],demonstr:[1,4,6,7,8,12,13,24,32,33,34,35,36,40,41],demystifi:[38,39,40],den:5,denomin:[2,6,34,35,40,41],denot:[2,3,7,8,14,29,36,37,38,40,41],dens:[2,4,5,40,41],densiti:[1,3,7,29,32,35,41],depart:[0,27,30,32,33,34,35,36,37,38,39,40,41],depend:[1,2,3,5,6,7,8,9,12,13,14,16,22,24,25,26,29,32,33,34,35,36,37,38,39,40,41],depict:29,deploy:[1,16,24,26,32],deprec:37,deprecationwarn:37,depth:[1,4,10,11,25,35,40],deriv:[0,1,2,3,7,8,9,11,12,14,17,18,20,22,24,26,32,38,41],derivati:[14,37,38],derivative_fn:[14,37],derivb1:[39,40],derivb2:[39,40],derivw1:[39,40],derivw2:[39,40],descend:[6,10,12,33,34],descent:[1,2,4,8,9,13,32,33,39,40],descent_i:[22,38],descent_x:[22,38],describ:[1,3,5,6,7,9,11,12,13,14,19,22,25,26,27,32,34,35,37,38,39,41],descript:[1,9,10,27,32,40,41],design:[1,2,4,5,6,7,8,11,12,13,14,16,17,18,19,20,26,27,32,34,35,36,37,38,39,40,41],designmatrix:[1,32],desir:[1,3,5,6,14,15,32,33,36,37,40,41],despit:[2,13,38,39,40],destroi:25,det:[6,25,33],detail:[1,7,12,14,15,23,25,26,27,33,36,37,40],detect:[3,4,9,13,38,39],determin:[1,3,4,5,6,7,9,10,11,12,13,14,17,25,29,32,33,34,35,36,37,38,39,40,41],determinist:[0,8,14,29,36,37,38,39],deternin:39,dev:[2,26,39,40],develop:[1,4,6,9,11,12,13,22,24,25,26,27,32,33,34,35,38,39],deviat:[1,2,3,5,6,7,20,26,29,32,33,34,35,39,40,41],devic:[4,5,15],devis:[13,38,39],df1:32,df:[5,9,12,14,32,37,39],di:1,diag:[6,9,33,34],diagnost:[2,11,39,40],diagon:[0,1,6,8,14,19,22,25,26,29,32,33,34,36,37,38,40,41],diagonaliz:[6,33],diagram:11,diagsvd:7,dice:[7,29,35],dict:[7,9,36,40,41],dictionari:[1,40,41],did:[0,1,2,6,7,8,11,12,15,26,27,32,34,35,36,37,40,41],die:[2,39,40,41],diff1:[3,41],diff2:[3,41],diff:[3,39,41],diff_ag:[3,41],diffeent:9,differ:[1,2,3,4,5,6,7,8,10,11,12,13,14,15,16,18,22,23,24,25,26,29,31,32,33,34,35,36,39,40,41],different:[39,40,41],differenti:[1,4,17,22,23,24,25,32,33,36,40],differential_oper:[],difficult:[1,2,7,11,14,22,26,29,32,35,37,38,39,40],difficulti:[1,2,14,32,36,37,38,39,40],diffonedim:[3,41],digit:[1,2,4,5,7,26,27,28,30,32,40,41],digress:39,dilemma:[14,37,38],dilut:[2,39,40],dim:[5,12,15,25,40,41],dimens:[1,2,3,4,5,6,9,10,12,15,17,18,25,32,33,34,39,40,41],dimension:[1,5,6,7,10,12,14,15,20,22,24,25,27,32,34,35,36,37,38],dimensionless:[1,4,32],diment:25,dimnsion:5,diod:4,direct:[1,2,3,5,8,12,13,14,15,22,32,33,36,37,38,39,40,41],directli:[2,5,6,7,29,33,40,41],directori:[1,8,36],disadvantag:[1,32],disappear:[4,7,35],disc_loss:5,disc_tap:5,discard:[7,12,35,36],disciplin:[1,4,13,38,39],disclaim:29,discontinu:39,discord:[23,32],discourag:[14,36,37],discov:[1,32],discover:[6,34],discret:[2,4,6,8,14,34,35,36,37,38,39,40],discrimin:[0,5,8,11,12,36],discriminator_loss:5,discriminator_loss_list:5,discriminator_model:5,discriminator_optim:5,discuss:[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,24,25,26,27,29,31,32,33,34,35,37,38,39,40,41],diseas:[8,36],disguis:[7,33,34],disord:[2,8,36,40,41],dispai:[38,39],displai:[0,1,2,4,5,6,7,8,9,10,11,12,13,15,26,29,32,33,35,36,38,39,40,41],displaystyl:[1,6,18,32,33,34],disregard:[1,32],dissimilar:[12,15],dist:15,distanc:[1,9,10,12,15,29],distance_list:10,distinct:[0,4,8,9,10,11,15,36],distinctli:9,distinguish:[0,1,5,8,9,29,32,36],distplot:1,distribut:[1,2,5,7,8,11,12,14,15,16,17,19,20,23,24,25,26,27,32,33,36,37,39,40,41],distrubut:[1,16,24,26,32],dive:[1,9,25,32],diverg:[2,14,36,37,38,39,40],divid:[1,2,4,6,7,8,9,10,12,13,27,29,32,33,34,35,36,38,39,40,41],divis:[7,9,10,14,22,25,29,35,36,37,38,39,40,41],dlopen:[4,5,15],dna:[8,36],dnn1:5,dnn2_gru2:5,dnn:[1,2,3,5,13,32,38,39,40,41],dnn_kera:[2,40,41],dnn_model:[2,40,41],dnn_numpi:[2,40,41],dnn_scikit:[1,2,32,40,41],doamin:[34,35],doc:[1,7,17,18,20,21,24,26,27,28,30,31,32,36,37,38],document:[0,5,10,12,14,36,41],doe:[1,2,3,4,5,6,7,9,11,12,13,14,15,17,22,23,25,26,29,32,35,36,37,40,41],doesn:[4,10,13,39,40],dog:[2,4,5,40,41],doi:41,domain:[6,9,14,26,27,34,35,36,37],domin:[1,32],don:[1,2,4,6,7,8,9,12,14,16,22,23,24,26,27,32,33,36,37,38,39,40,41],done:[1,3,4,5,6,7,10,11,12,14,18,25,26,32,33,34,35,36,37,39,40,41],dot:[0,1,3,4,6,7,8,9,10,11,12,13,14,22,25,26,29,32,33,34,35,36,38,39,40,41],doubl:[4,5,25,32],doubli:[2,39,40],doubt:26,down:[1,4,7,10,12,13,14,22,36,37,38,41],download:[1,2,4,6,7,25,26,31,32,40,41],downsampl:4,dozen:[2,39,40],dq:[7,35],drag:[14,37,38],dramat:12,drastic:5,draw:[5,7,11,14,35,36,37],drawback:[1,2,4,14,33,36,37,38,39,40],drawn:[2,5,7,8,12,29,32,35,36,39,40,41],drive:[4,5,23],driven:4,drop:[1,2,6,7,12,14,22,29,32,33,34,35,37,38,39,40],dropna:[1,7,32,35],dropout:5,dt:[3,4,14,23,29,37,38,39,41],dtype:[1,2,4,5,15,22,23,25,32,33,38,39,40,41],dual:9,dualiti:7,dub:[1,32],due:[2,3,6,7,9,11,13,14,30,32,33,35,36,37,38,39,40,41],dummi:1,dure:[1,2,3,4,5,9,10,12,18,21,24,26,32,35,37,38,39,40,41],dwell:1,dwh:[2,40,41],dwo:[2,40,41],dx:[3,4,9,29,39,41],dx_1:29,dx_1p:[7,35],dx_2p:[7,35],dx_mp:[7,35],dx_n:29,dxp:[7,35],dy:[2,9,29,39,40,41],dylib:[4,5,15],dynam:5,dz:9,e:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,19,20,22,23,29,30,32,33,34,35,36,37,38,39,40,41],e_:[1,3,32,41],e_z:23,each:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,18,22,24,25,27,28,29,30,32,33,34,35,37,38,39,40,41],eager:35,eapprox:[1,32],earli:[2,14,22,37,38,40,41],earlier:[1,6,8,9,10,12,13,14,21,22,23,32,33,36,37,38,39,40],earthexplor:[7,26],eas:[7,10,15,35],easi:[0,1,6,7,8,9,10,11,12,13,14,19,22,23,24,25,27,32,33,34,35,36,37,38,39,40,41],easier:[6,7,9,10,14,23,27,29,32,33,35,37,38],easiest:[14,36,37],easili:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,25,26,32,33,34,35,36,37,39,40,41],eastern:[30,32],ebind:[1,32],eblock:10,econometr:32,economi:6,ecosystem:[24,32],ect:28,edg:4,edgecolor:[7,35],edit:23,edu:[14,27,36,41],educ:[0,1,26,27,32,36],eff:29,effect:[2,5,11,14,29,37,38,39,40,41],effic:[2,40,41],effici:[1,4,11,14,23,24,25,29,32,36,37,38,39],efron:[7,35],egrad:[14,37,38],eig:[6,12,14,22,25,29,32,33,36,37,38],eigen:29,eigenpair:[6,12,33],eigenvalu:[1,6,9,12,14,22,25,32,33,34,36,37,38],eigenvector:[6,12,14,18,33,34,37],eight:[25,32],eigval:[25,29,32],eigvalu:[12,14,22,36,37,38],eigvec:[25,29,32],eigvector:[12,14,22,36,37,38],eispack:[25,32],either:[0,2,6,7,8,9,10,11,12,14,16,17,26,27,29,32,33,34,35,36,37,39,40,41],elabor:29,elarn:4,electr:[1,4,13,32,38,39],electron:32,eleg:12,element:[0,2,3,4,5,6,7,8,9,12,13,14,18,19,21,22,23,24,25,26,27,28,31,33,34,35,36,39,40,41],elementari:[11,14,25,37,38,39],elementwis:[4,14,37,38],elementwise_grad:[3,14,37,38,40,41],elessar:32,elif:[15,22,38,40,41],elim:25,elimin:[4,9],els:[0,2,3,4,5,8,10,13,14,15,23,25,36,37,38,39,40,41],elu:2,elus:[1,32],email:[23,28,30,32],embark:39,embed:[1,12,33],embodi:[7,20,26,35],emit:29,emner:31,emphas:[1,11,24,32],emphasi:[1,24,31,32],empir:[2,12,29,39,40],emploi:[1,2,6,7,12,14,26,27,29,32,33,34,35,36,37,40,41],employ:1,empti:[7,11,35,40,41],emul:[13,38,39],en:[24,31],enabl:[12,22,38,40,41],enbodi:[7,35],encod:[1,4,6,8,10,12,15,18,32,33,34,36],encompass:[1,26,29],encount:[0,1,2,6,7,8,14,22,23,29,32,33,35,36,37,38,39,40,41],encourag:[26,27],end:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,17,22,25,26,29,30,32,33,34,35,36,37,38,39,40,41],end_box:[],end_nod:[],end_valu:[],endpoint:[4,7],energi:[1,5,7,33,35],enet_coordinate_desc:7,enforc:[13,38,39],eng:31,engin:[1,2,4,5,24,32,39,40],english:[26,27],enocurag:26,enoent:[4,5,15],enorm:4,enough:[1,7,14,32,35,36,37],ensembl:[2,10,39,40],ensur:[1,2,3,4,6,7,12,14,22,29,33,34,35,36,37,38,39,40,41],entail:32,enter:[6,7,18,33,34],enthought:[1,16,24,26,32],entir:[2,4,8,10,23,24,29,32,36,37,38,39,40,41],entiti:[10,13,25,32],entri:[1,6,9,12,13,25,32,33,34,35],entropi:[2,4,8,11,14,23,32,37,38,40,41],enumer:[1,2,3,4,5,7,9,23,32,33,34,40,41],env:[0,1,2,3,4,5,7,8,9,10,12,15,22,23,29,32,34,36,38,40],environ:[3,10,22,23,24,26,38,41],eo:[1,7,32,35],eol:[1,32],eosfit:[1,32],epoch:[1,2,4,5,13,14,22,23,27,32,37,38,40,41],epsilon:[1,6,7,8,14,20,26,32,33,34,35,36,37,38,39],epsilon_0:[1,32],epsilon_1:[1,32],epsilon_2:[1,32],epsilon_:[1,32],epsilon_i:[1,32,33],eq:[4,14,15,25,29,36,37],eqnarrai:[4,6,7,34,35],equal:[1,2,3,4,5,6,7,9,10,12,13,14,15,16,23,25,26,27,29,32,33,34,35,36,37,38,39,40,41],equal_nan:23,equat:[0,2,4,5,6,7,8,9,10,11,12,14,15,17,18,19,20,22,25,26,29,35,38],equilibrium:[3,13,38,39,41],equiv:[4,14,25,29,36,37],equival:[1,2,6,8,9,12,14,22,24,25,32,33,34,35,37,38,40,41],equivel:23,erf:29,eriador:32,eric:[40,41],eridg:32,err:[1,8,11,36],err_:[7,35],err_sqr:[3,41],errat:[14,36,37],errno:[1,4,5,8,15,36],erron:[3,41],error:[0,2,3,5,6,7,8,10,12,13,14,16,17,18,19,20,22,23,24,25,26,27,29,34,36,37,38,39,40,41],error_estimate_corr_tim:29,error_hidden:[2,40,41],error_output:[2,40,41],escap:[14,36,37,38],especi:[2,4,10,13,14,26,27,37,38,39,40,41],essenti:[1,6,7,10,11,13,15,26,27,29,33,38,39,40],establish:[1,7,11,12,16,26,27,32],estim:[0,1,2,6,7,8,11,12,14,24,29,32,33,36,37,38,39,40,41],estimated_mse_fold:[7,35,36],estimated_mse_kfold:[7,35,36],estimated_mse_sklearn:[7,35,36],et:[1,3,5,16,17,18,26,27,31,32,33,34,35,36,37,39,40,41],eta0:[9,14,36,37],eta:[1,2,4,9,13,14,22,27,32,36,37,38,39,40,41],eta_:[14,22,37,38],eta_t:[14,37,38],eta_v:[1,2,4,32,40,41],etc:[1,2,4,6,8,9,10,12,13,14,15,16,22,24,25,26,27,29,33,36,37,38,40,41],ethic:24,etsim:35,euclidean:[1,15,33],evalu:[1,3,4,5,6,7,10,14,23,26,29,32,33,34,35,36,37,38],evaluationform:[21,26,37,38],evaluationgrad:[21,26,37,38],evalut:[14,37,38],even:[1,2,4,5,6,7,9,10,11,12,13,14,15,24,25,29,32,33,34,35,36,37,38,39,40],evenli:5,event:[0,6,8,11,29,34,35,36],eventu:[1,6,7,12,13,14,26,27,30,33,34,35,36,37,38,39],everi:[1,2,3,4,5,6,7,10,11,12,13,14,15,23,24,29,30,32,33,34,35,36,37,38,39,40,41],everyth:[5,13,22,23,39,40],everywher:[5,14,36,37],evolv:1,exact:[1,6,12,13,14,25,29,32,33,37,39,40],exactli:[1,4,5,7,13,24,26,33,35,38,39,40],exam:32,examin:[7,35],exampl:[6,12,13,14,16,17,20,21,24,25,26,27,29,31],exce:[2,13,14,22,37,38,39,40],excel:[1,2,5,6,11,27,32,33,39,40],except:[1,3,4,5,7,8,9,10,25,36,40,41],excess:[1,32],excit:1,exclud:[2,7,13,26,33,34,35,36,38,39,40],exclus:[1,2,4,7,29,32,35,36,39,40,41],execut:[3,6,14,33,37,38,41],exemplifi:[14,37,38],exercic:[30,32],exercis:[6,24,26,27,28,30,34,35,36,37,38,40],exercisesweek35:18,exhaust:[7,35,36],exhibit:[1,6,7,9,32,33,35],exist:[1,2,3,4,5,6,7,8,9,10,14,15,19,25,26,27,32,33,34,35,36,37,39,40,41],exit:[6,25,33],exp:[0,1,2,3,6,7,8,9,11,12,13,14,16,17,22,23,26,29,32,33,34,35,36,37,38,39,40,41],exp_term:[2,40,41],expand:[0,6,8,12,14,33,36,37],expandus:[1,8,36],expans:[1,4,6,9,11,13,14,32,33,36,37,39],expect:[1,2,6,7,8,12,13,14,16,17,20,22,24,26,27,32,33,36,37,38,39,40,41],expectation_value_of_h_wrt_p:29,expens:[7,11,14,36,37,38],experi:[1,2,7,9,14,24,26,32,33,35,36,37,38,40,41],experiment:[1,5,7,10,29,32,35],expert:[2,10,39,40],explain:[1,7,10,11,12,14,20,26,32,36,37],explained_variance_ratio_:12,explan:27,explanatori:[1,32],explicit:[1,4,7,14,22,25,26,32,33,34,37,38],explicitli:[1,5,9,22,23,38],explod:2,exploit:[1,4,13,14,32,37,38,39],explor:[2,5,7,9,14,24,26,27,36,37,38,39,40],expon:[2,40,41],exponenti:[1,2,6,7,11,14,26,29,32,34,35,36,37,38,39,40],export_graphviz:10,export_text:10,exporttext:10,expos:24,expr:39,express:[1,3,4,6,7,8,11,13,14,17,22,25,26,27,29,32,34,35,38,41],exptmean:29,exptvari:29,extend:[1,3,8,12,14,18,24,32,41],extens:[1,13,16,24,32,38,39],extent:[1,2,7,31,35,40,41],extern:[4,7,10],extra:[2,4,6,30,32,33,40,41],extract:[0,1,4,6,7,8,9,12,14,25,32,33,36,37,39],extrapol:[1,32],extrem:[1,2,5,6,7,8,9,10,14,17,25,33,34,35,36,37,38,39,40],extremum:[14,36,37],extrins:12,ey:[1,6,7,14,15,25,32,34,36,37,38],f11:[1,32],f12:[1,32],f13:[1,32],f1:[14,37,38],f1_grad:[14,37,38],f1d:[14,37],f2:[14,37,38],f2_grad_x1:[14,37,38],f2_grad_x1_analyt:[14,37,38],f2_grad_x2:[14,37,38],f2_grad_x2_analyt:[14,37,38],f3:[14,37,38],f3_grad:[14,37,38],f3_grad_analyt:[14,37,38],f4:[14,37,38],f4_grad:[14,37,38],f4_grad_analyt:[14,37,38],f5:[14,37,38],f5_grad:[14,37,38],f6:[14,37,38],f6_for:[14,37,38],f6_for_grad:[14,37,38],f6_grad_analyt:[14,37,38],f6_while:[14,37,38],f6_while_grad:[14,37,38],f7:[14,37,38],f7_grad:[14,37,38],f7_grad_analyt:[14,37,38],f8:[14,37],f8_grad:[14,37],f9:[1,14,32,37],f9_altern:[14,37],f9_alternative_grad:[14,37],f9_grad:[14,37],f:[1,2,3,4,5,6,7,8,9,11,13,14,15,17,19,20,22,25,29,30,32,33,34,35,36,37,38,39,40,41],f_0:[4,11],f_1:[11,14,36,37],f_2:[13,14,36,37,38],f_3:[13,38],f_:11,f_d:29,f_grad:[14,37,38],f_grad_analyt:[14,37,38],f_i:[1,7,13,17,35,38],f_m:[4,11],f_n:4,f_raw:[3,23],f_vec:[3,41],f_wrap:[3,23],face:[14,36,37,38],facecolor:[7,9,29,35],facil:[0,1,16,24,32,36,37,38,40,41],facilit:[13,38,39],fact:[1,2,4,6,10,12,13,14,32,33,34,35,36,37,38,39,40],factor:[1,2,4,6,7,10,11,12,14,25,29,32,33,34,35,37,38,40,41],factori:[14,37,38],fade:7,fafab0:[10,11],fahimeh:[30,32],fail:[0,1,7,9,10,12,14,30,32,35,36,37,39],fail_on_error:[1,8,36],failur:[8,36],fairli:[2,3,29,39,40,41],faisal:[17,33],fake:5,fake_loss:5,fake_output:5,fale:27,fall:[9,10,28],fals:[1,2,3,4,5,6,7,8,10,11,15,23,25,26,32,33,34,35,36,37,38,40,41],famili:[1,8,9,29,36],familiar:[1,4,6,7,9,16,24,25,26,29,32,34,35,39],famou:[7,13,40,41],far:[1,4,5,6,7,9,12,13,14,15,23,32,33,36,37,38,39],fashion:[1,10,11,32],fast:[2,4,7,11,13,14,24,29,32,35,36,37,39,40],faster:[2,12,14,23,37,38,39,40,41],fastest:[14,25,36,37],favor:[0,8,36],favorit:29,fc:4,fd:[1,8,36],featur:[0,1,2,4,6,7,8,9,11,12,13,14,19,23,24,26,27,29,32,34,35,36,37,38,39,40,41],feature_nam:[0,1,2,8,10,23,36,40,41],feautur:10,fed:[2,39,40,41],feed:[1,3,4,12,23,24,27,32],feed_forward:[2,23,40,41],feed_forward_4:[],feed_forward_all_relu:23,feed_forward_batch:23,feed_forward_out:[2,40,41],feed_forward_train:[2,40,41],feedback:[5,21,32],feeddorward:5,feedforward:[2,5,13,39,40,41],feel:[1,6,7,12,14,16,17,22,23,24,26,27,30,32,34,37,38,39],feet:1,felt:[26,27],fetch:[7,26],few:[2,4,5,6,10,29,32,37,39,40,41],fewer:[1,10,12,32],ffnn:[2,13,27,38,39,40,41],field:[1,4,7,13,24,38,39],fifteen:39,fifth:[1,7,16,17,26,32],fig:[0,1,2,3,4,5,7,8,13,14,15,26,32,36,37,38,39,40,41],fig_id:[1,7,8,10,32,33,35,36],figaxi:29,figsiz:[0,1,2,3,4,5,7,8,9,10,11,32,35,36,40,41],figur:[0,1,2,3,4,5,6,7,8,9,10,11,13,14,15,16,17,22,24,26,27,32,33,34,35,36,37,38,39,40,41],figure_id:[1,7,8,10,32,33,35,36],figurefil:[1,7,8,10,32,33,35,36],file:[1,3,4,5,6,7,8,10,15,23,26,27,32,33,34,35,36],file_prefix:5,filenam:32,filenotfounderror:[1,8,36],fill:[6,10,33,40,41],filter:[1,4,5,8,36],financ:1,find:[1,2,3,4,6,7,8,9,10,11,12,13,14,15,16,17,22,23,24,26,27,29,32,33,34,36,38,39,40,41],fine:[1,15,16,32],finicki:[],finish:[3,23,40,41],finit:[4,6,7,13,14,18,29,33,34,35,37,38,39],first:[0,1,2,3,4,6,7,8,9,10,11,12,14,15,16,17,18,22,23,25,26,29,30,31,35,38],first_moment:[22,37,38],first_term:[22,37,38],firsteigvector:12,fit:[0,2,4,5,6,7,8,9,10,12,13,14,16,17,18,22,26,27,29,33,35,36,37,38,39,40,41],fit_beta:[7,33,34],fit_intercept:[1,6,7,33,34,35],fit_mod:10,fit_transform:[1,7,9,10,12,35,36],fiti:[1,32],five:[1,10,18,26,32,33,36,39],fix:[1,4,5,7,11,12,13,14,22,26,27,32,35,36,37,38,40,41],fixedloc:7,flag:5,flat:[4,5,13,14,15,22,36,37,38],flatten:[2,4,5,6,25,40,41],flexibl:[2,7,9,11,13,27,32,35,38,39,40,41],flip:[23,30,32],float32:[5,10,22,38,40,41],float64:[5,22,25,32,33,38,39,40,41],floatingpointerror:[40,41],floor:[40,41],flop:[6,25,33],flow:[2,5,13,38,39,40,41],flower:23,fluctuat:[6,34],flush:[40,41],fly:12,fm:[1,32],fmax:4,fmesh:[14,37],fn:8,fname:[1,8,36],focu:[1,4,5,6,7,16,24,27,31,32,33,34,35],focus:[0,2,7,8,25,33,34,36,40,41],fold:[7,10,26],folder:[1,2,4,5,7,15,26,27,32,35,37,40,41],follow:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41],font:[1,8,29,32,36],fontdict:29,fontsiz:[2,7,9,10,11,29,40,41],fontweight:[2,40,41],footprint:4,foral:[9,33,39],forc:[1,6,7,11,12,32,33,34,36,39],forcast:5,forecast:[5,13,38,39],forest:[1,2,10,24,32,39,40],forget:12,form:[0,1,4,5,6,7,8,9,10,12,13,14,18,22,24,25,26,27,29,32,33,34,35,36,37,38,39,40,41],formal:[4,5,15,29,39],format:[0,1,2,4,5,7,8,9,10,11,12,24,29,31,33,34,35,36,38,39,40,41],format_data:5,formatstrformatt:[7,14,26,36,37],formul:[5,7,12,15],formula:[4,14,29,36,37,38,39],forth:[5,13,38],fortran2003:[24,32],fortran2008:[26,27],fortran90:29,fortran:[1,16,24,25,32],fortun:[1,12,33],forward:[1,4,7,23,24,25,27,32,35],forwardpropag:[39,40],foster:32,found:[2,3,4,5,6,7,13,14,15,20,21,23,26,27,32,33,34,35,37,38,39,40,41],foundat:[24,32],four:[5,6,7,9,13,23,25,26,28,30,32,34,38,39,40],fourier:[1,32,39],fourierdef1:4,fourierdef2:4,fourierseriessign:4,fourth:[13,32,33],fp:8,frac:[0,1,2,3,4,6,7,8,9,10,11,12,13,14,15,16,17,18,20,23,25,26,27,29,32,33,34,35,36,37,38,39,40,41],fractal:10,fraction:10,frame:[0,8,36],framework:[2,4,5,9,11,15,29,40,41],frank:[6,12,22,27],frankefunct:[6,7,12,26,33],fredli:[23,30,32],free:[1,7,12,14,16,17,22,23,24,25,26,27,29,30,31,32,37,38,39],freecodecamp:24,freedom:[6,34],freeli:[1,26],frequenc:[0,4,7,8,29,35,36],frequent:[1,9,10,14,36,37,38],frequentist:24,fresh:11,frf4l5qax1m:41,fridai:[17,18,19,20,21,22,23,30,32],friedman:[7,19,26,28,31,32],friendli:5,frodo:32,frog:4,from:[1,2,3,4,5,7,8,9,10,12,14,15,17,18,19,20,23,24,25,26,29,30,31,32,41],from_cod:10,from_logit:[4,5],from_tensor_slic:5,fromnumer:3,front:[1,5,6,16,17,32,33,34],fulfil:[3,6,13,33,38,39,40,41],full:[1,2,4,6,8,10,11,14,23,29,32,33,36,37,38],full_matric:[6,33],fulli:[4,7,13,28,29,35,36,38,39],fun:[24,32],fun_nam:[],func:[3,23,40,41],functionali:12,fundament:[1,7,24,32,35],funtion:[3,41],furthemor:39,further:[3,8,10,32,39,41],furthermor:[0,1,4,6,7,8,12,13,14,18,24,26,32,33,34,35,36,37,38,39,40],futur:[1,5,9,10,32,37],futurewarn:9,fy:[16,23,26,27,28,30,31,32],fys4155:[26,27],fys5419:[31,32],fys5429:[31,32],g0:[3,41],g:[1,2,3,4,5,7,9,10,11,12,14,29,32,34,35,36,37,38,39,40,41],g_0:[3,41],g_1:[3,11,41],g_2:[3,11,41],g_3:39,g_:[3,10,11,41],g_analyt:[3,41],g_dnn_ag:[3,41],g_euler:[3,41],g_i:[3,39,41],g_j:39,g_m:[4,11],g_n:4,g_re:[3,41],g_t:[3,40,41],g_t_d2t:[3,41],g_t_d2x:[3,41],g_t_dt:[3,41],g_t_hessian:[3,41],g_t_hessian_func:[3,41],g_t_invers:[40,41],g_t_jacobian:[3,41],g_t_jacobian_func:[3,41],g_trial:[3,41],g_trial_deep:[3,41],g_vec:[3,41],gain:[2,6,8,10,11,14,33,34,35,37,38,40,41],galleri:[1,32],game:5,gamge:32,gamma1:9,gamma2:9,gamma:[1,3,9,10,11,12,14,22,32,36,37,38,41],gamma_0:11,gamma_1:11,gamma_1x:11,gamma_:[1,32],gamma_i:[1,9,29,32],gamma_j:[14,37,38],gamma_k:[14,36,37],gamma_m:11,gamma_x:[1,32],gap:[7,9],gate:[0,5,13],gather:[1,2,13,33,38,39,40,41],gaug:[13,38,39],gaussbacksub:25,gaussian:[5,6,7,9,15,29,34,35],gaussian_point:15,gaussian_rbf:9,gave:[14,37,38],gavra:32,gbc:[28,32],gca:[3,7,9,14],gd:[2,27,36,39,40],gd_clf:11,gdclassiffiercgain:11,gdclassiffierconfus:11,gdclassiffierroc:11,gdfit:[0,38],gdm:[14,22,37,38],gdregress:11,ge:[2,6,8,29,33,36,39,40,41],gen:[1,8,36],gen_loss:5,gen_tap:5,gender:[1,32],genener:5,gener:[0,1,2,3,4,6,7,9,11,12,13,14,15,16,17,18,20,21,22,23,25,26,29,31,33,34,35,36,37,38,40,41],generaliz:[40,41],generallay:[13,38],generate_and_save_imag:5,generate_imag:5,generate_latent_point:5,generate_simple_clustering_dataset:15,generated_imag:5,generator_loss:5,generator_loss_list:5,generator_model:5,generator_optim:5,genom:24,geodes:12,geometr:[1,14,32],geometri:[6,34,35],georg:31,geotif:[7,26],geq:[3,6,9,10,14,33,34,36,37,41],geron:[1,16,22,28,31,32,37,38],get:[1,2,3,4,5,6,7,8,10,11,12,14,16,21,23,24,25,26,27,29,30,32,33,34,35,36,37,40,41],get_dummi:10,get_paramet:[3,41],get_split:10,get_yaxi:9,get_yticklabel:7,getattr:[],getval:[],gg:32,gibb:[24,32],gif:5,gini:11,gini_index:10,ginvers:14,git:[1,16,24,32],giter:[14,22,37,38],github:[1,7,10,16,17,18,20,21,22,24,26,27,28,30,31,32,33,36,37,38,39,40,41],gitlab:[1,16,24,26,27,32],gitta:[39,40,41],give:[1,2,3,4,6,7,8,9,10,11,13,14,15,18,24,26,27,29,32,33,34,35,36,37,38,39,40,41],given:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,22,23,25,26,27,29,32,33,34,35,36,37,38,39,40,41],glkfgrjhtlnplbx4:23,global:[0,7,8,14,26,36,37,38],gloriou:27,glorot:2,gnew:14,go:[1,2,4,6,7,9,10,12,13,14,23,26,32,33,34,35,36,37,39,40,41],goal:[1,8,10,32,36],goe:[1,2,3,6,7,14,15,19,25,32,33,34,35,36,37,38,40,41],golden:[14,37],gone:[6,33,34],gong:[2,40,41],good:[2,4,5,6,7,10,11,12,14,18,22,23,24,29,31,33,34,35,36,37,38,39,40,41],goodfellow:[5,27,28,31,32,33,34,36,37,39,40,41],googl:[2,5,22,23,24,32,38,40,41],got:[2,7,14,23,26,40,41],gotcha:[22,38],gotten:[32,40,41],gov:[7,26],gp:31,gpu:[2,14,22,24,32,37,38,40,41],grad:[3,14,22,23,37,38,40,41],grad_analyt:[14,37,38],grad_broadcast_to:[],grad_concatenate_arg:[],grad_np_sum:[],grade:[26,28],gradient:[1,4,5,8,9,10,13,23,24,32,33],gradient_bia:[40,41],gradient_desc:[22,37,38],gradient_func:23,gradient_weight:[40,41],gradientboostingclassifi:11,gradientboostingregressor:11,gradients_of_discrimin:5,gradients_of_gener:5,gradienttap:5,gradual:[2,15,39,40],grai:[5,7,26],graph:[2,10,12,13,14,36,37,38,39,40,41],graph_from_dot_data:10,graphic:[1,2,10,32,40,41],grasp:1,gray_r:[2,4,40,41],grayscal:4,great:[6,14,23,36,37,39,41],greater:[0,2,8,29,36,39,40,41],greatli:[14,37,38],greedi:[10,36],green:[1,4,10,29],gregor:[40,41],grei:5,grid:[0,2,4,7,8,9,13,29,33,34,35,38,39,40,41],gridsearch:36,gridsearchcv:36,groh:[39,40,41],grossli:[14,36,37],ground:[1,32],group:[1,7,8,10,15,24,26,27,28,30,32,35],groupbi:[1,32],grow:[2,4,10,11,39,40,41],growth:[1,32],gru:5,guarante:[1,5,14,29,32,33,36,37],guess:[2,5,11,14,15,22,27,36,37,38,40,41],guestrin:11,guid:[2,23,40,41],guidanc:26,guidelin:[21,37,38],h1:[3,41],h:[1,2,6,7,9,14,17,19,22,23,26,29,30,31,32,33,36,37,38,40,41],h_1:[3,14,36,37,41],h_2:[3,14,36,37,41],h_:[1,14,32,36,37],h_m:11,ha:[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,17,22,23,25,26,27,29,32,33,34,35,36,37,38,39,40,41],haa:30,habit:[1,33],had:[1,2,7,8,14,32,35,36,37,39,40,41],hadamard:[2,13,14,22,37,38,39,40,41],half:[2,9,10,39,40,41],halv:11,hand:[1,2,3,4,6,12,13,14,16,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38],handi:[4,26,27],handl:[1,2,3,6,10,12,24,34,39,40,41],handle_unknown:10,handsid:[13,39,40],handwrit:[13,38,39],handwritten:[2,6,40,41],handwrittennot:[36,38],happen:[2,3,4,5,6,7,11,14,29,33,37,38,39,40,41],hard:[0,2,8,9,11,14,23,36,37,39,40,41],hardcopi:[24,32],harder:[1,2,23,33,39,40],harmon:4,hasn:[2,40],hassl:[1,16,24,32],hast:[24,32],hasti:[1,7,16,17,18,19,26,28,31,32,33,35,36],hat:[1,2,6,7,8,10,11,12,13,14,18,19,25,26,33,34,35,37,38,39],have:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,22,23,24,25,26,27,29,30,32,33,34,35,36,37,38,39,40,41],haven:[2,40,41],he:[8,36],head:[1,5,11,29],header:[1,32],heads_proba:11,health:[1,33],hear:[1,14,32,37,38],heart:[1,8,32,36],heatmap:[0,1,2,4,8,32,36,40,41],heavili:1,heavisid:[2,40,41],height:[2,4,7,33,34,40,41],hein:41,held:[14,22,37,38],help:[1,2,5,13,14,17,22,26,27,32,37,38,39,40],helper:[5,15],henc:[1,6,7,9,10,11,13,14,19,26,32,33,34,35,36,37,38],henrik:[30,32],her:[8,36],here:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,22,23,24,25,26,27,29,32,33,34,35,36,37,38,39,40,41],hereaft:[1,9,13,32],hermitian:25,hessenberg:25,hessian:[1,3,6,14,17,22,34,38,41],heterogen:[10,11],hi:[8,36],hidden:[2,4,5,13,23,27,38],hidden_bia:[2,40,41],hidden_bias_gradi:[2,39,40,41],hidden_deriv:[40,41],hidden_func:[40,41],hidden_layer_s:[1,2,32,40,41],hidden_neuron:5,hidden_nodes1:[40,41],hidden_nodes2:[40,41],hidden_weight:[2,40,41],hidden_weights_gradi:[2,39,40,41],hierarch:[6,33,34],high:[1,2,3,4,5,6,7,10,11,12,14,15,22,23,24,25,26,32,33,35,36,37,38,39,40,41],higher:[1,2,4,6,7,9,14,22,26,27,32,33,34,35,36,37,38,40,41],highest:[2,3,40,41],highli:[1,4,5,11,19,24,25,27,31,32,33,34,35],highwai:1,hing:9,hint:[14,17,23,27,33,34,36,37],hip:[24,41],hire:1,hist:[0,5,7,8,29,35,36],histogram:[0,1,7,8,29,36],histor:[8,12,36],histori:[4,5,13,38,39],hitherto:[6,34],hjorth:[0,30,32,33,34,35,36,37,38,39,40,41],hobbi:29,hoc:[6,33],hoff:31,hold:[2,4,7,14,15,22,35,36,37,38,40,41],holder:[1,32],holm:41,holomorphic_grad:[],home:1,homepag:[26,27,32],homework:[7,14],homogen:[2,4,10,11,14,22,37,38],honchar:[3,41],hopefulli:[1,12,29,32,38],horizont:12,hornik:39,hors:[4,8,36],hot:[2,10,40,41],hour:[2,24,28,29,30,32,40,41],how:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,22,23,24,25,26,27,29,32,33,34,35,37,38,39,40,41],howev:[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,22,23,24,25,26,27,29,32,33,34,35,36,37,38,39,40,41],hspace:[1,5,9,11,29,32,39,40],hstack:[2,40,41],htf:[28,32],html:[0,1,10,12,18,20,23,24,26,27,28,30,31,32,34,39,40,41],http:[0,1,4,5,7,10,12,14,17,18,20,21,22,23,24,25,26,27,28,30,31,32,33,34,35,36,37,38,39,40,41],huang:[1,32],huber:[1,32],huge:[2,4,5,24,39,40,41],human:[1,2,4,7,10,13,33,34,38,39,40,41],humid:10,hundr:[2,39,40],hungri:[2,39,40],huse:32,hvae:38,hybrid:28,hydrogen:[1,32],hyper:[22,27],hyperbol:[2,5,13,41],hyperparam:9,hyperparamat:39,hyperparamet:[4,5,6,7,10,14,18,32,33,34,36,37,38],hyperplan:12,i0:[1,32],i1:[1,7,9,13,32,33,34,38],i2:[1,9,13,32,38],i3:[1,13,32,38],i5:[1,32],i:[0,1,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,22,23,25,26,27,29,30,32,33,34,35,36,37,38,39,40],i_1:[6,7,34,35],i_2:[6,7,34,35],i_:[14,36,37],i_siz:23,ian:31,iayaan2:23,ic:[2,27,40,41],id:[8,14,36,37,38],ida:[30,32],idea:[1,2,3,4,5,7,10,11,13,14,25,26,27,33,34,35,36,37,38,39,40,41],ideal:[1,3,7,9,14,29,32,35,38,40,41],idem:[7,35],ident:[6,7,13,14,18,25,26,33,37,38,40,41],identical:[34,35],identifi:[1,2,8,10,12,13,14,15,32,33,36,37,38,40,41],ieor:29,ifi:[31,41],ifs:[24,32],ignor:[1,2,4,10,33,39,40,41],ii:[25,29,40],iii:[25,32,40],ij:[1,2,4,7,9,13,15,17,19,25,26,29,32,33,34,38,39,40,41],ik:[1,25,32,33],ilg3ggewq5u:[39,40],illustr:[6,8,11,13,14,15,24,32,35,36,37,40],ilsvrc:[37,38],im:7,imag:[2,4,5,7,10,12,13,15,27,31,32,38,39,40,41],image_at_epoch_:5,image_batch:5,image_height:4,image_path:[1,7,8,10,32,33,35,36],image_width:4,imageio:[7,26],images_from_seed_imag:5,imagin:[2,39,40],immedi:[1,4,5,7,16,24,32],implement:[1,3,4,5,6,7,9,10,11,12,13,14,15,18,22,23,26,27,29,32,33,34,36,37,39],impli:[4,6,7,8,14,25,33,34,35,36,37],implicit:4,implicitli:[12,29],importantli:4,impos:[1,7,12,13,32,38,39,40,41],imposs:[1,6,32,33],impress:[1,13,32,38,39],improv:[1,5,6,10,11,12,14,22,23,26,27,33,38],impur:10,imread:[7,26],imshow:[2,4,5,7,26,40,41],in1:[],in2:[],in3050:[31,32],in3310:32,in4080:[31,32],in4300:[31,32],in4310:31,in5400:4,in5550:31,in_out_neuron:5,inaccur:[14,36,37],inact:[13,38,39,40],inadequ:[1,32],inch:[7,33,34],includ:[1,2,3,4,5,6,7,8,12,13,16,17,18,21,23,24,26,27,29,30,31,32,33,34,35,40,41],include_bia:[7,10,35],inclus:[18,38],incom:[13,38,39],incorrect:[2,40,41],incoveni:9,increas:[0,1,2,4,5,6,7,9,10,12,13,14,22,26,29,32,33,34,35,37,38,39,40,41],increasingli:29,increment:37,ind:7,inde:[1,3,5,6,7,14,32,33,39,41],indefinit:5,independ:[1,6,7,8,9,13,14,29,32,33,36,37,38],index:[1,2,4,5,11,15,24,25,27,29,31,32,40,41],index_col:[1,32],indic:[1,2,4,5,6,7,10,11,12,14,17,22,26,27,32,33,37,38,39,40,41],indispens:[7,35],individu:[2,7,8,11,13,29,32,33,35,36,38,39,40,41],indu:1,indx1:[3,41],indx2:[3,41],indx3:[3,41],indx:25,ineffici:[4,14,37,38],inequ:[9,14],inequaltii:[36,37],inertia:[14,22,37,38],inf1000:[24,32],inf1100:[24,32],inf1100l:[24,32],inf1110:[24,32],inf3000:32,infeas:10,infer:[1,2,5,7,31,32,35,40,41],inferenc:[2,40,41],infil:[1,7,8,10,32,35,36],infin:[6,7,8,12,19,33,34,35,36,39,40],infinit:4,infinitesim:29,influenc:[7,11,35,36],influenti:[2,40,41],info:32,inform:[1,2,4,5,7,10,12,13,14,15,22,25,26,27,31,32,35,36,37,38,39,40,41],infti:[4,7,14,29,35,36,37,39],ingeni:[14,36,37,38],ingrad:[],ingredi:[1,10,32],inher:[7,35],inherit:[25,32],inhomogen:3,init:[],initi:[1,2,3,7,11,14,15,25,27,29,32,35,36,37,38,39,40,41],inititi:[40,41],inject:15,inlin:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,22,25,26,29,32,33,34,35,36,37,38,39,40,41],inner:[1,14,33,37],inp:5,inplac:14,inpput:39,input:[0,1,2,4,5,6,7,8,9,13,14,15,22,26,27,29,32,33,34,35,36,37,38],input_dim:[2,40,41],input_nod:[40,41],input_s:23,input_shap:[4,5],inputs:[2,40,41],inputs_shuffl:[1,2,33,40,41],insert:[4,6,7,9,11,29,33,34,35],insid:[0,1,5,8,23,36],insight:[1,2,6,22,24,27,32,33,34,35],insist:[7,14,33,34,37],inspect:36,inspir:[1,2,13,27,32,38,39,40,41],instabl:[3,41],instal:[1,2,6,7,10,16,40,41],instanc:[1,2,3,5,7,10,12,14,32,33,35,36,37,40,41],instanti:11,instead:[1,2,3,4,5,6,7,9,10,12,14,15,22,23,25,26,29,32,33,35,36,37,38,40,41],institut:[2,40,41],instruct:[1,2,16,17,32,40,41],int32:11,int_0:29,int_:[4,7,29,35,39],int_a:29,intak:[1,33],integ:[2,3,14,15,25,29,32,37,38,40,41],integer_vector:[2,40,41],integr:[4,7,29,32,35],intellig:[1,15,31,32],intend:11,intens:[2,39,40],intention:15,interact:[1,7,10,13,24,26,27,32,38,39],intercept:[1,7,9,12,14,17,26,32,34,35,36,37,38],intercept_:[1,7,9,10,14,32,33,34,36,37],interchang:[6,13,25,34,35,38,39],interconnect:[2,39,40],interest:[0,1,2,3,4,5,6,7,8,9,10,13,22,24,26,27,29,32,33,34,35,36,37,38,39,40,41],interfac:[1,2,25,33,40,41],interior:[1,10,32],intermedi:[25,33],intermediari:23,intern:[2,11,13,38,39,40,41],interpol:[2,4,5,7,13,26,38,39,40,41],interpr:[6,33,34],interpret:[1,2,7,10,11,13,14,18,20,23,25,26,27,29,39,40,41],interv:[1,4,6,7,8,14,19,26,29,32,33,34,36,37],intial:[14,36,37],intract:[1,5,33],intrins:[4,12,25,29,32],intro:[24,31,32,41],introduc:[1,2,6,7,9,11,13,18,25,26,29,32,35,36,39,40,41],introduct:[2,3,5,14,18,21,31,33,36,37,40,41],introductori:[1,5,25,31,32,33],intuit:[1,6,7,9,13,14,22,26,32,34,35,37,38,39,40],inv:[1,6,14,22,32,33,34,36,37,38],invalid:[2,9,40,41],invalu:[1,14,16,24,32,36,37],invari:[2,40,41],invd:[6,34],inver:[9,38,39],invers:[1,4,7,14,16,17,18,22,26,27,32,33,36,37,38],inverse_transform:9,invert:[1,6,8,11,14,22,32,34,36,37,38],invh:[14,22,37,38],invok:[1,9],involv:[0,1,3,7,8,12,13,33,35,36,37,38,39,40,41],io:[1,18,20,24,26,28,30,31,32,33,41],ion:38,ip:[1,9,29,32],ipca:12,ipykernel_12550:[],ipykernel_12588:[],ipykernel_1323:[],ipykernel_1390:[],ipykernel_1539:[],ipykernel_1579:[],ipykernel_22411:[],ipykernel_22458:[],ipykernel_22556:[],ipykernel_22575:[],ipykernel_47048:[],ipykernel_55841:[],ipykernel_57122:[],ipykernel_57183:[],ipykernel_57294:[],ipykernel_57311:[],ipykernel_57345:[],ipykernel_58656:[],ipykernel_58739:[],ipykernel_58840:[],ipykernel_58862:[],ipykernel_59478:[],ipykernel_59625:[],ipykernel_59734:[],ipykernel_59752:[],ipykernel_59845:[],ipykernel_6316:[],ipykernel_6403:[],ipykernel_64157:[],ipykernel_64203:[],ipykernel_64318:[],ipykernel_64336:[],ipykernel_64372:[],ipykernel_6690:[],ipykernel_6729:[],ipykernel_87117:[],ipykernel_87155:[],ipykernel_87222:2,ipykernel_87367:7,ipykernel_87525:32,ipykernel_87545:35,ipykernel_87566:37,ipykernel_87597:40,ipykernel_87605:[],ipykernel_92604:41,ipykernel_95354:[],ipykernel_95419:[],ipykernel_95519:[],ipykernel_95542:[],ipynb:[24,26,32],ipython:[1,6,8,10,12,15,16,24,26,27,32,33,36],iq:[7,35],iri:[9,10,23],irreduc:[7,35],irrelev:[6,33],irrespect:[1,32],irvin:27,isbox:[],isclos:23,iscomplexobj:[],iseffici:[37,38],isfinit:23,isn:[6,34,35],isnan:[40,41],isnul:1,isomap:12,isotop:[0,32,36,37,40,41],issu:[2,10,25,34,39,40,41],it_arrai:[14,37],item:[1,14,32,37],items:[25,32],iter:[0,2,3,5,7,9,10,12,14,15,22,29,35,36,38,39,40,41],its:[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,19,23,24,25,26,27,29,32,34,35,36,37,38,39,40,41],itself:[6,7,13,20,26,27,29,33,34,35,39],iv:40,ix:[40,41],j1:25,j:[1,2,3,4,5,6,7,9,10,12,13,14,15,17,18,19,20,25,26,29,31,32,33,34,35,36,37,38,39,40,41],j_:7,j_lasso_sk:7,j_ridge_sk:7,j_sk:7,jackknif:[7,24,32,35],jacobian:[3,14,36,37],jacobian_shap:[],jakobsen:30,jason:5,jax:[24,27,32,39],jax_descend_i:[22,38],jax_descend_x:[22,38],jax_enable_x64:[22,38],jax_grad:[22,38],jensen:[0,30,32,33,34,35,36,37,38,39,40,41],jentzen:[39,40,41],jerom:[19,26,31],ji:[13,25,39,40],jit:[14,37],jj:[1,6,7,32,34,35],jk:[1,2,7,13,25,32,38,39,40,41],jl:[1,32],jm:25,jmlr:41,jnp:[14,22,37,38],job:[3,9,11,41],johansen:32,join:[1,4,5,7,8,10,15,32,33,35,36],joint:[5,6,34,35],judg:[14,36,37],judgement:[7,26],julia:[24,25,26],juliu:[39,40,41],jump:[26,29],junk:5,jupit:32,jupyt:[1,10,16,20,24,26,31,32,39,40,41],just:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,22,23,24,26,27,29,32,33,34,35,36,37,38,39,40,41],justif:1,justifi:[4,11],k0:[0,8,36],k1:[0,8,36],k:[0,1,2,4,6,7,8,9,10,11,12,13,14,15,23,24,25,26,29,30,32,33,34,37,38,41],kaggl:[7,26,27],kajda:[40,41],kappa_d:29,karl:[30,32],karush:9,katrin:[30,32],keep:[1,2,5,6,7,12,14,15,23,25,26,27,32,33,34,35,36,37,38,39,40],keepdim:[2,7,11,25,35,40,41],kei:[1,2,4,7,13,33,38,40,41],kept:[5,7,15,35],kera:[1,5,24,26,27,32],kernel:[1,2,4,5,15,24,32,33,40,41],kernel_librari:[4,5,15],kernel_regular:[2,4,40,41],kernel_s:5,kernelpca:12,kev:[1,32],kevin:[31,32],keyboardinterrupt:[40,41],keyerror:[],keyword:[7,14,25,32,40,41],kfold:[7,35,36],kg:[2,40,41],ki:25,kick:[2,14,37,38,39,40],kiener:[3,41],kilomet:[7,33,34],kind:[1,3,4,5,9,13,14,15,32,33,37,38,39,40,41],kj:[7,13,25,33,34,39,40,41],kjm:[24,32],kkt:9,kl:29,km:[13,32,38],kmean:15,kmeanspoint:15,kn_k:15,know:[1,2,3,6,7,9,14,24,32,33,34,36,37,38,39,40,41],knowledg:[1,24,32],known:[0,2,4,5,6,7,8,9,10,13,25,26,27,29,31,33,34,35,36,38,39,40,41],kondev:[1,32],kp:29,kpca:12,kristin:41,kroneck:15,kuckuck:[39,40,41],kuhn:9,kumar:41,kutyniok:[39,40,41],kvalsund:[30,32],kwarg:[3,23,40,41],kwown:[1,32],l0:[0,8,36],l1:[0,1,2,4,8,32,36,40,41],l1_l2:[2,4,40,41],l1regl:6,l2:[2,4,40,41],l:[0,1,2,3,4,6,7,8,9,11,12,13,14,20,25,26,29,32,33,36,37,38,41],l_1:[0,8,36,39],l_2:[0,8,14,27,36,37,39],l_:25,l_j:[13,39,40],l_ja:[40,41],la:[14,37],la_i:[13,39,40,41],la_k:[13,39],lab:[21,24,26,32],label:[1,2,3,4,5,6,7,8,9,10,11,13,14,15,22,24,25,26,27,29,32,33,34,35,36,37,38,39,40,41],label_prob:[37,38],labelencod:[0,8,11,36],labels:[7,9,10],labels_shuffl:[1,2,33,40,41],laboratori:[28,34],lack:[1,32],lagari:[3,41],lagrang:[9,12],lam:[40,41],lambda:[0,1,2,3,4,6,7,8,9,11,13,14,18,19,22,26,27,29,32,33,34,35,37,38,39,40,41],lambda_0:12,lambda_1:[6,9,12,33],lambda_2:[9,12],lambda_:12,lambda_i:[9,12],lambda_iy_i:9,lambda_jy_iy_j:9,lambda_k:9,lambda_n:[6,9,33],lamda:[2,40,41],land:[1,9],landmark:9,landscap:[14,22,36,37,38],langl:[1,7,12,29,32,33],languag:[1,2,5,9,16,24,25,26,27,31,32,39,40],lapack:[25,32],laplac:[6,34,35],laptop:24,larg:[1,2,3,5,6,7,9,10,11,12,14,16,24,25,26,29,31,32,33,35,36,37,38,39,40,41],larger:[1,4,6,7,9,11,12,14,18,29,32,33,34,35,36,37,38],largest:[5,9,12],lass:27,lasso:[0,1,8,24,27,32],lasso_sk:7,last:[1,2,3,4,5,6,7,8,9,11,13,14,15,18,19,23,25,26,29,30,32,34,35,37,41],latent:5,latent_dim:5,latent_point:5,latent_space_value_rang:5,later:[0,1,2,5,8,9,13,14,15,16,23,24,26,27,32,36,37,38,39,41],latest:[5,24],latest_checkpoint:5,latex:32,latter:[1,4,7,8,9,12,14,17,25,26,29,32,33,34,35,36,37,38,39],lattic:[13,38,39],law:1,lax_numpi:[22,38],layer:[1,5,14,22,27,32,37,38],layer_output_s:23,layers_grad:23,lbfg:[0,8,10,11,12,36],lcc:[6,7,34,35],lda:12,ldot:[1,7,12,20,26,32,35,36],le:[6,8,11,14,18,22,29,33,34,36,37,38],lead:[0,1,2,4,6,7,8,9,10,11,12,13,14,17,18,23,25,29,32,33,34,35,36,37,38,39,40,41],leaf:10,leaki:[2,27,39,40,41],leakyrelu:5,lear:[14,36,37],learn:[0,4,5,6,7,8,9,10,11,13,19,23,25,28,30,31],learnabl:4,learner:11,learnig:32,learning_r:[0,9,11,22,23,38,41],learning_rate_init:[1,2,32,40,41],learning_schedul:[14,22,37,38],learnt:[26,27],least:[1,8,9,11,12,16,17,18,20,22,24,25,27,29,35,36],leat:[14,22,37,38],leav:[1,2,4,6,7,10,12,23,32,34,35,36,39,40],lectur:[1,2,6,11,12,13,14,16,18,22,24,25,26,27,28,30,31],lecture_11_backpropag:41,lecturenot:[1,18,20,24,26,31,32],left:[0,1,2,3,4,6,7,8,9,10,11,12,13,14,15,17,18,20,25,26,29,32,33,34,35,36,37,38,39,40,41],leftarrow:[9,13,39,40,41],legend:[0,1,3,4,5,6,7,8,9,10,11,14,23,32,33,34,35,36,37,38,41],legend_el:23,len:[1,2,3,4,5,6,7,9,10,11,12,13,23,25,32,33,34,35,37,38,39,40,41],length:[1,2,4,5,9,10,14,17,23,24,32,33,36,37,38,40,41],length_of_sequ:5,leq:[1,6,8,9,14,15,29,32,33,34,36,37],less:[1,2,4,5,6,7,9,10,14,24,29,32,33,34,35,36,37,38,40,41],lessen:[2,39,40],let:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,25,29,32,33,34,35,36,37,38,39,40,41],letter:[1,17,25,29,32,33],level:[1,2,6,7,10,24,25,26,27,28,30,32,35,39,40,41],li:[9,12],lib:[0,1,2,3,4,5,7,8,9,10,12,15,22,23,32,34,36,38,40],liblinear:[9,11],libmetal_plugin:[4,5,15],librari:[1,2,3,4,5,6,7,8,10,11,12,15,16,22,25,26,27,29,31,33,34,36,40,41],library_loc:[4,5,15],licens:[1,2,16,24,26,27,32,40,41],lie:[1,7,12,29,32,33,35],life:[1,2,9,13,32,38,39,40,41],lifetim:[14,37,38],like:[1,2,3,4,5,6,7,8,10,11,12,13,14,16,17,22,23,24,25,26,27,29,32,33,34,35,36,37,38,39,40,41],likelihood:[1,2,6,10,32,33,37,38,40,41],lim_:29,lima:30,limit:[0,1,6,7,9,10,12,13,22,25,26,27,32,33],lin_clf:9,lin_model:1,lin_reg:10,linalg:[1,3,6,7,9,12,14,22,25,29,32,33,34,36,37,38,39,41],line1:9,line2:9,line2d:[22,37,38],line3:9,line:[1,2,3,4,5,7,8,9,11,12,14,15,22,23,26,32,33,34,35,36,37,38,39,40,41],linear:[0,2,4,6,7,8,10,11,12,13,18,19,22,23,24,26,29,35,37,38,39,40,41],linear_model:[0,1,6,7,8,9,10,11,12,14,32,33,34,35,36,37,38,39],linear_regress:[7,35,40,41],linearli:[6,33],linearloc:[7,14,26,36,37],linearregress:[1,7,8,10,32,33,34,35,36],linearsvc:9,lineat:34,liner:[2,4,40,41],linerar:11,linewidth:[1,3,5,7,9,10,11,26,35,41],link:[1,5,10,13,23,24,26,27,30,32,39],linlag:[6,34],linpack:[25,32],linreg:[1,32],linspac:[1,3,4,5,7,9,10,11,14,16,17,22,25,29,32,33,34,35,37,38,41],linu:5,linux:[1,2,16,24,26,32,40,41],liquid:[1,32],list:[1,2,3,4,5,8,10,23,24,26,27,32,33,36,37,38,41],listcomp:[],listedcolormap:[10,11],literatur:[2,8,15,31,35,36,39,40],littl:[2,4,10,13,39,40],live:9,ll:[1,29,32,33],lle:[1,33],lloyd:[5,15],lmb:[1,3,6,7,34,35,36,41],lmbd:[1,2,4,32,40,41],lmbd_val:[1,2,4,32,40,41],lmbda:[14,36,37],ln:[2,14,36,37,40,41],load:[0,1,2,4,5,7,8,10,11,15,26,36,40,41],load_boston:1,load_breast_canc:[0,2,8,10,11,12,36,40,41],load_data:[4,5],load_digit:[2,4,40,41],load_iri:[9,10,23],load_librari:[4,5,15],load_pluggable_device_librari:[4,5,15],loc:[0,1,4,7,8,9,10,11,23,32,35,36],local:[0,1,2,3,4,8,13,14,23,33,36,37,38,39,40,41],locat:[3,4,9,41],log10:[1,6,7,34,35,36,40,41],log:[0,1,2,3,5,6,7,8,10,11,12,14,23,25,26,27,32,33,34,35,36,37,38,40,41],log_:[1,32],log_clf:11,logaddexp2:[],logaddexp:[],logarithm:[0,1,6,8,25,32,34,35,36],logbook:[26,27],logic:[1,2,10,32,39,40],logist:[1,2,3,9,10,11,12,13,14,22,24,33,41],logistic_predict:[37,38],logistic_regress:[40,41],logisticregress:[0,8,10,11,12,36,38,39],logit:[0,8,36],logreg:[0,8,10,11,12,36,38,39],logspac:[1,2,4,6,7,32,34,35,36,40,41],longer:[3,4,9,11,15,25,27,29,32,41],loocv:[7,35,36],look:[1,2,3,4,5,6,7,8,9,10,11,12,14,17,19,22,25,26,27,29,32,33,34,35,36,37,38,39,40,41],loop:[2,5,7,11,13,15,24,25,32,35,40,41],lose:[2,40,41],loss:[0,1,2,4,5,6,7,8,9,11,12,14,23,25,26,27,32,34,35,38,39,40,41],loss_fil:5,lossfil:5,lost:5,lot:[1,2,5,7,35,37,38,39,40,41],low:[1,7,10,11,12,26,32,33,35],lower:[1,2,4,7,10,11,17,23,25,33,39,40,41],lowercas:[25,32],lowest:[10,14,29,37,38],lr:[2,4,5,11,40],lrelu:[40,41],lstat:1,lstm:5,lstm_2layer:5,lstsq:[1,32,33],lt:[7,35],lu:[1,6,32,33],lubksb:25,luckili:[3,41],ludcmp:25,lux:25,lvert:[2,40,41],lw:[1,32],m:[1,2,3,4,6,7,9,10,11,12,13,14,22,25,28,29,30,31,32,33,34,35,36,37,38,39,40,41],m_1:15,m_:[10,13,39,40],m_h:[1,32],m_k:15,m_l:[13,39,40],m_n:[1,32],m_p:[1,32],m_t:[14,37,38],ma:12,machin:[2,4,5,6,7,8,10,11,12,13,16,22,25,28,31,33,34,35,38,39,40,41],machinelearn:[1,7,17,18,20,21,24,26,28,30,31,32,36,37,38],mackai:31,made:[1,2,4,5,6,7,8,10,12,13,19,26,27,32,33,36,38,39,40],mae:[1,32],magic:5,magnitud:[0,2,7,8,14,23,33,34,36,37,38,39,40,41],mai:[0,1,2,3,4,6,7,8,9,10,12,13,14,19,22,24,25,26,27,29,32,33,34,35,36,37,38,39,40,41],mail:[28,30],main:[1,2,4,5,6,7,8,10,25,26,27,31,33,36,39,40],mainli:[0,1,6,7,8,10,32,33,34,35,36],maintain:[7,35],major:[2,7,10,11,14,25,32,35,36,37,38,40,41],make:[0,2,3,4,5,6,7,8,9,12,13,14,22,23,24,25,26,27,29,31,34,35,36,37,38,39,40,41],make_axes_locat:7,make_classif:[38,39],make_moon:[9,10,11],make_pipelin:[1,7,11,33,35],make_vjp:[],makedir:[1,7,8,10,32,33,35,36],makeplot:[1,32],malcondit:25,malign:[0,2,8,10,36,40,41],mammographi:[6,34,35],manag:[1,3,4,16,24,26,32,41],mandatori:[30,32],mani:[0,1,2,4,5,6,7,8,9,10,12,14,15,22,23,24,25,26,27,29,31,32,33,34,35,36,37,38,39,40,41],manifold:12,manner:4,manual:[7,23,33,34,36],map:[1,2,3,7,8,9,12,13,15,26,29,32,36,38,40,41],marc:33,margin:[1,6,9],marit:[1,32],mariu:41,mark:32,marker:[1,8,22,25,32,36,38],markov:[24,32],marsaglia:29,mass:[1,2,6,14,33,37,38,40,41],massag:[1,32],masses2016:[1,32],masses2016ol:[1,32],masses2016tre:1,masseval2016:[1,32],master:[7,17,21,26,28,30,36,37,38,41],mat1100:[24,32],mat1110:[24,32],mat1120:[24,32],mat:[24,32],match:[2,5,6,14,15,33,36,37,38,39,40],match_complex:[],materi:[5,6,8,14,18,25,30,37,38],math:[0,4,8,13,14,22,25,29,31,32,36,37,38,39,40,41],mathbb:[1,5,6,7,8,9,12,13,14,15,18,19,20,25,26,29,32,33,34,35,36,37,38,39],mathbf:[0,1,6,7,8,9,14,19,20,22,25,26,32,33,34,35,36,37,38,39],mathcal:[0,2,6,7,8,14,20,26,34,35,36,37,40,41],matheemat:4,mathemat:[1,7,12,13,14,18,23,24,25,29,31,32,34,36],mathemati:32,mathmlbook:17,mathrm:[0,1,2,4,5,6,7,8,9,10,11,12,13,14,15,18,19,20,26,29,32,33,34,35,36,37,38,39,40,41],matmul:[2,3,6,34,39,40,41],matmul_adjoint_1:[],matmul_vjp_1:[],matnat:31,matplotlib:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,22,23,24,25,26,29,32,33,34,35,36,37,38,39,40,41],matric:[1,2,4,5,7,8,9,12,14,17,18,24,33,36,37,39,40,41],matrix:[1,3,4,5,7,8,9,11,14,16,17,18,19,20,22,23,26,27,29,35,39],matshow:[2,40,41],matter:[3,4,14,33,36,37,38,39,41],max:[1,2,3,4,5,10,11,13,14,22,23,30,32,36,37,38,39,40,41],max_depth:[1,10,11],max_diff1:[3,41],max_diff2:[3,41],max_diff:[3,41],max_it:[0,1,2,9,10,12,14,32,36,37,38,39,40,41],max_iter:15,max_leaf_nod:11,max_sampl:11,maxdegre:[1,7,11,33,35],maxdepth:11,maxim:[0,2,5,6,8,9,12,34,35,36,40,41],maximum:[0,1,2,3,4,6,8,9,10,11,14,15,32,33,37,38,40,41],maxpolydegre:[6,7,34,35,36],maxpooling2d:4,mbox:[6,7,19,26,33,34,35],mcculloch:[13,38,39],md:[12,21,26,37,38],mdoel:5,mean:[2,3,4,5,6,7,8,10,11,12,13,14,15,16,17,18,19,20,22,24,25,26,27,29,32,35,37,38,39,40,41],mean_absolute_error:[1,32],mean_divisor:15,mean_i:29,mean_matrix:15,mean_squared_error:[1,5,7,8,11,32,33,35,36],mean_squared_log_error:[1,32],mean_vector:15,mean_x:29,meaning:[1,5,8,32,36],meansquarederror:[1,32],meant:[0,4,8,11,14,36,37,38,39],measur:[1,2,3,6,7,10,12,13,15,20,26,27,29,32,33,34,35,39,40,41],mechan:[1,5,29,32,41],median:[1,32,33],medicin:[13,38,39],medium:[5,9,14,27,37,38],medv:1,meet:[1,30],mehta:[1,32,33,34],memori:[4,5,12,13,14,22,25,37,38,39],mention:[1,13,14,26,27,29,32,36,37,38,39],mere:[1,16,27,32],meshgrid:[3,6,7,9,10,11,12,26,33,40,41],mess:[],messag:[6,14,37,38],messi:[3,41],met:[1,4,9,33],metadata:[],meteorolog:10,meter:[7,33,34],method:[0,1,2,3,4,5,6,8,9,12,13,15,17,18,19,20,23,24,25,27,29,31,33,34,39],metion:[7,26],metric:[0,1,2,4,7,8,10,11,15,16,17,23,32,33,35,36,40,41],metropoli:[24,32],mev:[1,29,32],mgd:[14,37,38],mglearn:[24,32],mgrid:[14,37],mhjensen:[0,2,4,5,7,9,10,12,15,22,38,40],mi:11,mia:[30,32],michael:[27,39,40],michigan:[0,32,34,36,37,38,40,41],microsoft:31,mid:[2,40,41],midel:5,midnight:[17,18,19,20,21,22,23],midpoint:10,might:[1,2,3,5,7,10,14,33,34,36,37,38,39,40,41],mild:10,millimet:[7,33,34],million:[1,32,33,37,38],mimic:[13,38,39],min:[1,3,6,9,10,41],min_:[1,3,6,15,18,32,33,34,41],min_samples_leaf:10,mind:[1,7,14,23,26,32,33,34,35,36,37],mindboard:5,mine:[24,32],mini:[2,12,13,14,22,27,36,39,40],minibatch:[2,12,14,40,41],minibathc:[14,37,38],miniforge3:[0,1,2,3,4,5,7,8,9,10,12,15,22,23,32,34,36,38,40],minim:[1,2,3,4,6,7,8,9,10,11,12,13,14,15,18,19,22,26,33,34,35,37,38,39,40],minima:[1,2,8,14,32,36,37,38,40,41],minimum:[1,2,3,7,9,10,12,14,33,35,36,37,38,40,41],minmaxscal:[1,33,40,41],minor:29,minst:[2,40,41],minu:[8,36],mirjalili:32,mirror:10,misc:[7,26],misclassif:[9,10,11],misclassifi:[9,11],miser:1,mismatch:[2,39,40],miss:[1,8,11],mistak:5,mit:31,mix:[2,3,32,39,40,41],mixtur:[14,22,37,38],mk:[10,25],mkdir:[1,7,8,10,32,33,35,36],ml:[1,2,11,14,25,26,27,33,36,37,38,39,40,41],mlab:29,mlbook:32,mle:[6,8,36],mlp:[2,38,39,40,41],mlpclassifi:[2,38,39,40,41],mlpregressor:[1,32],mm:25,mml:33,mn:[13,29,38],mnist:[2,12,27,39,40,41],mod:29,mode:[28,30,32,40,41],model:[0,3,4,6,8,9,10,11,12,14,15,16,17,19,20,22,23,24,26,29,31,33,34,35,36,37],model_select:[0,1,2,4,6,7,8,10,11,12,32,33,34,35,36,40,41],moder:11,modern:[1,7,8,24,32,35,36,39,40,41],modif:[3,13,14,37,38,41],modifi:[0,1,2,4,6,8,9,11,13,14,32,33,34,36,37,38,39,40,41],modul:[0,1,10,12,25,32,34],modular:29,modulo:29,moe:[12,33],moment:[6,7,14,22,29,34,35,40,41],moment_correct:[40,41],momentum:[27,39,40,41],momentum_schedul:[40,41],mondai:32,monitor:[14,22,37,38],monoton:[6,13,29,34,35,38,39,40,41],mont:[1,7,24,29,31,32,35],moor:[6,7,34],more:[1,2,3,5,6,8,9,10,11,12,13,14,15,19,23,24,27,29],moreov:[1,4],morten:[0,30,32,33,34,35,36,37,38,39,40,41],most:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,22,23,24,26,29,32,33,34,35,36,37,38,39,40,41],mostli:[2,12,39,40],motion:[1,14,37,38],motiv:[2,5,39,40],move:[1,5,6,7,8,10,13,14,15,22,23,26,29,33,34,35,36,37,38,39,40,41],mpl:[1,8,32,36],mpl_toolkit:[3,7,14,26,36,37,41],mplot3d:[3,7,14,26,36,37,41],mplregressor:[2,40,41],mse:[1,5,6,7,10,11,16,17,18,20,26,27,32,33,34,35,36,40,41],mse_simpletre:11,mselassopredict:[6,34],mselassotrain:[6,34],mseownridgepredict:[7,34],msepredict:[6,34],mseridgepredict:[1,6,7,34,36],msetrain:[6,34],msle:[1,32],mt:[0,8,13,36,38,39,40],mu0:29,mu1:29,mu2:29,mu:[1,7,12,14,29,32,35,37,38],mu_1:33,mu_:[7,29,33,34,35],mu_i:[7,33,34],mu_n:12,mu_x:29,much:[1,2,3,4,5,6,7,9,10,11,12,13,14,23,25,26,29,32,33,34,35,37,38,39,40,41],multi:[1,2,4,8,24,32,36],multiarrai:23,multiclass:[0,2,8,36],multidimension:[12,13,32,38,39],multilay:[2,40,41],multinomi:[0,8,36],multipl:[0,3,5,6,7,8,13,14,29,33,35,36,37,38,39],multipli:[4,6,7,12,14,25,29,33,34,37],multiplum:9,multivari:[1,3,11,12,24,29,32,41],multivariate_norm:[12,15],murphi:[12,31,32],must:[2,3,6,7,9,11,13,14,15,29,33,35,36,37,38,39,40,41],mutat:[8,36],mutual:[2,4,7,14,35,36,37,39,40,41],mx_:29,myenv:[0,1,2,3,4,5,7,8,9,10,12,15,22,23,32,34,36,38,40],myriad:[1,16,24,32],mz1:29,mz2:29,n1:25,n2:25,n:[0,1,2,3,4,5,6,7,8,9,11,12,13,14,15,16,17,18,19,20,22,25,26,27,29,32,33,34,35,36,37,38,39,40,41],n_0:[13,29,38],n_:[2,3,4,9,13,29,38,40,41],n_boostrap:[7,11,35],n_bootstrap:[7,35],n_categori:[2,4,40,41],n_cluster:15,n_compon:12,n_data:[0,38],n_epoch:[14,22,37,38,40,41],n_estim:11,n_examples_to_gener:5,n_featur:[2,39,40,41],n_filter:4,n_hidden:[3,41],n_hidden_neuron:[1,2,32,39,40,41],n_i:29,n_input:[1,2,4,33,39,40,41],n_instanc:10,n_iter:[36,37],n_iter_i:[0,10,12],n_job:11,n_k:15,n_l:[13,29,38],n_layer:[2,40,41],n_m:10,n_neuron:[2,40,41],n_neurons_connect:4,n_neurons_layer1:[2,40,41],n_neurons_layer2:[2,40,41],n_output:[39,40],n_point:15,n_sampl:[7,9,10,11,15,35,38,39],n_split:[7,35,36],n_step:5,n_t:[3,41],n_x:[3,41],nabla:[2,14,36,37,40,41],nabla_:[3,14,22,36,37,38,41],nabla_w:[14,37,38],nafter:[40,41],nag:[14,37,38],naimi:[1,32],naiv:[0,8,36],naive_kmean:15,najafi:[30,32],name:[1,2,4,5,6,7,8,9,10,11,13,14,15,16,23,24,25,26,29,30,32,33,34,35,36,37,38,39,40,41],nameerror:[11,26],namespac:[4,5,15],nan:[40,41],narrow:[14,37,38],nary_f:[],nary_op_arg:[],nary_op_kwarg:[],nary_oper:[],nation:[2,6,34,35,40,41],nativ:[24,32],natur:[1,2,5,9,10,13,14,26,27,29,31,32,36,37,38,39,40,41],navier:[13,38,39],nb:29,nb_:25,nbconvert:32,nbviewer:10,nd:15,ndarrai:[7,40,41],nderiv:[40,41],ndim:37,ne:[10,11,25,29,33],nearest:[2,4,7,12,40,41],nearli:[14,36,37],neat:32,neatli:[],neccesari:[7,35],necess:[3,41],necessari:[1,2,4,5,9,15,32,39,40,41],necessarili:[1,5,12,29,32],necesserali:[6,34,35],neck:[8,36],need:[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,18,22,23,25,27,29,33,34,35,36,37,38,39,40,41],neg:[0,1,2,4,6,7,8,11,14,25,29,32,34,35,36,37,38,39,40,41],neg_mean_squared_error:[7,35,36],neglect:29,neglig:29,neighbor:[4,7,12],neither:[5,14,37,38],neq:[14,15,29,36,37],nerual:[40,41],nervou:[13,38,39],nest:[10,13,38],nesterov:[14,37,38],net:[3,5,13,38,39,41],netlib:[25,32],network:[1,10,14,22,23,24,31,33,37],network_input_s:23,neural:[0,1,14,22,23,24,31,33,36,37],neural_network:[1,2,3,32,38,39,40,41],neuralnetwork:[2,40,41],neuralnetworksanddeeplearn:[27,39,40],neuron:[2,3,4,5,13,40,41],neutal:[],neutral:[1,32],neutron:[1,32],never:[2,5,7,10,29,35,39,40,41],new_box:[3,23],new_chang:[14,22,37,38],new_hobbit:32,new_root:[],new_trac:[],newaxi:[1,4,7,10,23,35,36],newli:[1,32],newton:[2,8,9,14,29,39,40,41],next:[1,2,3,4,5,6,7,8,9,10,14,15,22,23,32,33,34,36,37,38,39,40,41],next_guess:[14,37],next_input:5,ng:[2,39,40,41],ngini:10,nhow:[40,41],ni:[15,22],nice:[1,2,6,12,32,33,34,35,40,41],nielsen:[27,39,40],nine:[39,40],niter:[14,22,36,37,38],nitric:1,nlambda:[1,6,7,34,35,36],nlp:31,nm:29,nm_n:[1,32],nmse:[7,35],nn:[3,6,7,13,25,34,35,38],nn_model:[2,40,41],nnmin:[3,41],node:[2,3,4,10,11,13,23,27,38,41],node_constructor:[],nois:[1,5,6,7,9,10,11,14,16,17,20,26,32,33,34,35,36,37],noise_dimens:5,noisi:[2,7,20,26,35,40,41],non:[1,2,4,6,7,8,10,11,12,13,14,15,22,23,25,29,32,33,34,35,36,37,38,39,40,41],noncommerci:27,none:[0,1,2,3,5,6,10,11,14,29,32,33,34,35,36,39,40,41],nonlinear:[4,7,9,10,12,13,35,38,39],nonneg:[7,10,14,35,36,37],nonparametr:7,nonsens:29,nonsingular:25,nonumb:[0,4,8,9,14,22,25,36,37,38],nor:[2,5,14,37,38,39,40],norm:[1,2,6,7,9,12,14,18,32,33,34,35,36,37,38,39,40,41],normal:[0,4,5,6,7,8,9,10,11,12,13,14,16,17,19,20,22,23,24,25,26,27,29,32,33,34,36,37,38,41],normali:[25,32],norwai:[7,26,27,32,35,38],notat:[1,3,6,7,14,15,22,29,32,33,34,35,37,39,40,41],note:[0,1,2,3,4,5,6,7,8,9,12,13,14,15,16,17,24,25,27,29,31,32,35,36,37,39,40,41],notebook:[1,2,4,10,16,23,24,26,27,32,39,40,41],notesseptember16:36,notesseptember30:38,notfounderror:[4,5,15],noth:[2,3,6,9,13,15,29,33,34,38,39,40,41],notic:[5,6,13,14,25,29,32,34,37,38,39,40],notimplementederror:[40,41],notion:4,notrace_primit:[],noutput:[40,41],novel:[4,7,11,36],novemb:[2,28,30,32,40,41],now:[0,1,3,5,6,7,8,9,11,12,13,15,16,17,18,23,24,25,26,27,29,32,33,36,39],nowadai:[1,2,4,10,24,32,39,40,41],nox:1,np:[0,1,2,3,4,5,6,7,8,9,10,11,12,14,15,16,17,22,23,25,26,29,32,33,34,35,36,37,38,39,40,41],npr:[3,41],nsampl:[7,10,35,36],nt:[3,41],ntrained_model:[7,34],nu:29,nuclear:[6,33],nuclei:[1,29,32],nucleon:[1,32],nucleu:[1,32],num:5,num_coordin:[3,41],num_equ:[40,41],num_featur:[0,38],num_hidden_neuron:[3,41],num_it:[3,41],num_iter:[0,22,38],num_neuron:[3,41],num_neurons_hidden:[3,41],num_not:[40,41],num_point:[3,41],num_tre:11,num_valu:[3,41],number:[0,2,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,20,23,25,26,27,28,30,32,34,35,36,40],numberid:[8,36],numberparamet:4,numer:[1,6,7,10,11,12,13,14,18,22,23,24,25,31,32,33,34,35,36,37,38,39],numpi:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,23,24,26,29,33,34,35,36,39,40,41],numpy_box:[],numpy_vjp:[],numpy_wrapp:[],nunmpi:[6,33],nvalu:10,nx:[3,41],nx_test:[7,34],nx_train:[7,34],nx_train_mean:[7,34],ny:[29,40,41],ny_pr:[7,34],ny_train:[7,34],ny_train_mean:[7,34],o:[1,7,8,9,10,12,16,22,25,30,31,32,36,38],obei:[7,12,14,33,34,37],obj:[],object:[1,2,3,5,7,9,11,25,32,37,39],obliqu:[6,33],observ:[1,2,4,6,7,8,9,10,11,12,13,14,15,29,34,35,36,37,38],obtain:[0,1,2,6,7,8,9,10,11,13,14,15,18,22,25,26,27,29,32,33,34,35,36,37,38,39,40,41],obviou:[6,7,12,29,33],obviouli:32,obvious:[1,5,6,7,22,25,27,32,34,35],oc:33,occupi:1,occur:[1,3,7,9,10,25,29,32],oct:27,octob:[0,22,23,28,30,32,38],od:1,odd:[0,1,4,8,32,33,36],odenum:[3,41],odesi:[3,41],odin:32,oen:1,off:[2,4,5,6,10,14,20,29,34,35,37,38,40,41],offer:[7,12,24,25,28,30,32,35],offic:[30,32],offici:[28,32],offlin:23,often:[0,1,2,4,5,6,7,8,9,10,11,12,13,14,15,17,23,24,25,26,29,32,33,35,36,37,38,39,40,41],ofter:[25,32],og:[40,41],ol:[1,14,16,17,18,19,27,32,33],old:[2,6,11,14,34,35,36,37,40,41],ols_sk:7,ols_svd:7,olsbeta:[1,6,34],omega:[3,4,7,41],omega_0:4,omit:[1,6,32,33,34,35],omlmp_kkie0:36,onc:[2,7,10,12,14,35,36,37,38,39,40,41],one:[0,1,2,4,5,6,7,8,9,10,11,12,14,15,20,22,23,24,25,26,27,29,30,32,33,34,35,38],one_hot_predict:23,onehot:[2,40,41],onehot_vector:[2,40,41],onehotencod:10,ones:[1,3,6,7,9,10,11,12,14,22,23,25,26,32,33,34,35,36,37,41],ones_lik:5,ong:33,onl:4,onli:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,18,20,22,23,25,26,27,29,32,33,34,35,36,37,38,39,40,41],onlin:[12,28,39,40,41],onto:[6,12,33],open:[1,2,5,7,8,10,16,24,26,28,30,32,35,36,40,41],oper:[1,2,4,6,7,11,12,13,14,16,23,24,29,32,33,34,35,37,38,40,41],operation:29,oplu:29,opmiz:[14,37,38],opportun:1,oppos:[7,14,37,38],opposit:[2,6,9,33,39,40],opt:[2,6,27,32,34,39,40,41],optim:[1,3,4,5,6,7,8,10,11,12,15,16,17,18,19,20,22,23,26,27,34,35],optimis:[2,4,40,41],option:[0,1,2,4,6,7,9,10,12,22,25,26,27,33,34,35,38,40,41],optmiz:[2,9,14,22,33,37,38,39,40,41],oral:32,orang:1,order:[1,2,3,4,6,7,8,9,10,11,12,13,16,17,23,25,26,27,29,32,33,34,35,36,39,40,41],ordinari:[1,3,4,8,12,14,16,17,18,20,22,24,27,35,36,37,38],oreilli:[31,32],org:[0,1,4,5,10,12,22,23,24,25,26,27,31,32,34,37,38,39,41],organ:[7,8,11,25,35,36],orgin:39,orient:[2,6,29,33,34],origin:[1,4,6,7,9,12,13,14,25,32,33,35,37,38],orthogn:[6,33],orthogon:[1,6,7,9,12,14,18,25,32,33,34,37],orthonorm:[6,33,34],os:[1,2,5,6,7,8,9,10,30,32,33,34,35,36,40,41],oscar:[2,39,40],oscil:[4,14,37,38],oserror:[1,4,5,8,15,36],oslo:[0,1,16,24,26,27,28,30,32,33,34,35,36,37,38,39,40,41],osx:[1,16,24,26,32],other:[1,2,3,4,6,7,8,9,11,14,15,16,20,23,24,27,28,29,30,31,34,35,37],otherwis:[1,2,5,8,14,22,25,27,32,36,37,38,40,41],ouput:[6,8,13,34,35,36,41],our:[0,2,3,4,7,8,9,10,11,13,15,16,17,18,19,20,22,23,24,25,26,27,29,34,35,38,41],ourmodel:1,ourselv:[1,6,7,9,12,14,32,33,34,35,36,37],out1:[],out2:[],out:[1,2,3,5,6,7,8,9,10,11,12,13,14,17,22,23,24,25,26,27,29,32,33,34,35,36,37,38,39,40,41],out_deriv:[40,41],out_fil:10,outcom:[1,8,10,11,13,29,33,36],outdoor:10,outer:[7,13,14],outfil:5,outgrad:[],outlier:[1,9,32,33],outlin:[7,11,12,35],outlook:10,outperform:11,output:[0,1,2,4,5,6,7,8,9,10,11,13,14,17,23,25,26,27,29,32,33,34,35,36,37,38],output_bia:[2,40,41],output_bias_gradi:[2,39,40,41],output_func:[40,41],output_nod:[40,41],output_s:[],output_shap:5,output_weight:[2,40,41],output_weights_gradi:[2,39,40,41],outputlayer1:[13,38],outputlayer2:[13,38],outsid:5,over1:[14,37,38],over:[1,2,4,5,6,7,10,11,13,14,22,25,26,32,33,34,35,36,37,38,41],overal:[2,11,40,41],overcast:10,overcom:[13,14,37,38,39],overdetermin:[1,32],overfit:[1,2,4,7,10,11,14,22,35,37,38,39,40,41],overflow:[2,6,34,35,40,41],overflowerror:[40,41],overhead:[13,39,40],overlap:[0,4,8,9,10,36],overlin:[1,6,7,10,11,12,15,25,32,33,34],overst:1,overtrain:5,overview:[4,36],overwritten:[40,41],own:[5,6,7,9,13,14,22,24,25,26,34,37,38,39,40,41],owner:1,ownmsepredict:1,ownmsetrain:1,ownridgebeta:[1,7,34],ownypredictridg:1,ownytilderidg:1,oxid:1,p0:[3,41],p1:[3,41],p:[0,1,2,3,4,5,6,7,8,9,10,11,12,14,15,18,25,29,32,33,34,35,36,37,39,40,41],p_:[3,5,9,10,41],p_hidden:[3,41],p_i:[6,29,34,35],p_j:29,p_n:29,p_output:[3,41],p_x:29,pa:39,pack:[1,32],packag:[0,1,2,3,4,5,6,7,8,9,10,12,14,15,16,22,23,24,26,27,29,33,34,36,37,38,40,41],packtpub:32,packtpublish:32,pad:[4,5],page:[1,10,24,32,34,35,36,37],pai:[1,2,10,14,37,38,39,40],pair:[1,3,4,10,24,29,32,41],panda:[0,1,5,6,7,8,10,12,16,24,26,34,35,36],panel:32,paper:[2,41],paradigm:[1,32],parallel:[11,14,22,24,25,32,37,38],param:[3,41],param_distribut:36,param_grid:36,paramat:[3,41],paramet:[1,2,3,4,5,6,7,8,9,10,11,13,14,17,18,19,20,22,23,26,27,29,34,35,41],parameter:[1,7,11,26,32,33],parametr:[1,7,16,17,32,33,35],paramt:[4,6,34,35,39],parent:39,parent_argnum:[],part:[1,2,3,4,6,7,11,18,19,22,23,25,28,29,30,32,34,35,38],partial:[0,1,2,6,7,8,9,11,12,13,14,17,23,29,32,33,34,36,37,39,40],particip:[24,28,30,32],particl:[1,5,14,29,32,37,38],particular:[1,2,3,4,6,7,10,11,12,13,14,17,26,29,31,32,33,34,35,36,37,38,39,40,41],particularli:[6,7,9,12,14,22,29,33,35,36,37,38],partit:[2,5,10,40,41],partli:[7,32,35],pass:[3,4,13,15,23,36,39],passkwarg:[],password:[26,27],past:[11,29],patch:[7,29,35],path:[1,4,5,7,8,10,15,16,24,32,33,35,36],pathcollect:[22,38],patient:[8,36],patter:5,pattern:[1,4,5,13,28,31,32,38,39],pauli:[1,32],pc:[12,24],pca:[0,1,8,24,32,33,36],pd:[0,1,5,6,7,8,10,12,32,33,34,35,36],pde:[3,41],pdf:[1,4,5,6,7,10,17,21,26,27,31,32,34,35,36,38,41],pedagog:[1,32,33],penal:[7,33,34],penalti:[7,14,26,33,34,36,37],penros:[6,7,34],pentagon:[14,36,37],peopl:[1,2,10,14,24,37,38,39,40,41],per:[1,2,7,23,28,30,32,35,39,40,41],perc_print:[40,41],percentag:[1,11,12,30,40,41],perceptron:[0,1,2,8,32,36],peregrin:32,perfect:[1,2,14,22,32,37,38,40,41],perfectli:[5,7,35],perform:[1,3,4,5,6,7,9,11,12,13,14,15,16,17,18,20,22,23,24,25,26,27,29,32,33,34,35,36,37,38,39],performac:5,perhap:[1,6,14,32,33,34,36,37],perimet:[2,10,40,41],period:[2,5,29,40,41],permut:12,persist:[14,37,38],person:[6,7,8,28,30,32,33,34,35,36],perspect:31,pertin:[13,27,32,39,40,41],petal:[9,10],peter:[31,33],petersen:[39,40,41],phantom:29,phase:[7,13,38,39],phd:41,phenomena:29,phi:9,phi_k:9,philipp:[39,40,41],philosophi:[14,37],phone:[30,32],photo:5,php:27,phrase:[1,32],physic:[0,1,2,5,8,13,14,27,29,30,31,32,33,34,35,36,37,38,39,40,41],pi:[0,3,4,6,7,8,10,13,14,29,34,35,36,37,38,39,40,41],pick:[2,10,11,12,14,15,37,38,40,41],pickl:[2,40,41],pictur:[1,32],pie:[24,32],piec:[12,15,23],pillow:[1,16,24,26,32],pinv:[6,7,14,22,33,34,37,38,39],pip3:[1,2,16,26,32,40,41],pip:[1,2,16,24,26,32,40,41],pipelin:[1,7,9,11,33,35],pippin:32,pit:5,pitfal:[7,33,34],pitt:[13,38,39],pixel:[2,4,5,40,41],pixel_height:[2,4,40,41],pixel_width:[2,4,40,41],place:[1,5,7,9,14,25,26,32,35,36,37],plai:[1,4,5,6,7,9,12,24,32,33,34,35,36,39,40],plain:[9,11,13,14,15,22,27,36,37,39,40],plan:[7,10,30,31,32,40],plane:[9,10],plateau:[6,34],platform:[24,32],plausibl:[13,38,39,40],pleas:[0,10,12,14,26,27,30,32,37,38],plenti:[2,39,40],plethora:[4,13,38,39],pliahhy2ibx9hdharr6b7xevztgzra1p:[38,39,40],plot:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,22,23,24,25,26,27,29,32,33,34,37,38,39,40,41],plot_all_sc:[26,34],plot_confusion_matrix:[0,8,11,36],plot_count:7,plot_cumulative_gain:[0,8,11,36],plot_data:[2,40,41],plot_dataset:9,plot_decision_boundari:[10,11],plot_import:11,plot_iris_dataset:23,plot_max:5,plot_min:5,plot_model:5,plot_numb:5,plot_predict:9,plot_regression_predict:10,plot_result:5,plot_roc:[0,8,11,36],plot_surfac:[3,7,14,26,37,41],plot_train:10,plot_tre:[10,11],plqvvvaa0qudcjd5baw2dxe6of2tius3v3:[38,39,40],plt:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,22,23,25,26,29,32,33,34,35,36,37,38,39,40,41],plu:[1,4,6,8,32,33,34,36],pluggabl:[4,5,15],plugin:[4,5,15],pm:[9,35],pmatrix:[3,41],pml:31,pn:4,png:[1,5,7,8,10,32,33,35,36],point:[0,1,2,3,4,6,7,8,9,10,11,12,14,15,16,17,18,20,22,25,26,29,30,32,33,35,36,37,38,40,41],point_1:5,point_2:5,poisson:[24,29,32],poli:[7,9,35,36],poly100_kernel_svm_clf:9,poly3:1,poly3_plot:1,poly3dcollect:37,poly_degre:[40,41],poly_featur:[9,10],poly_features10:10,poly_fit10:10,poly_fit:10,poly_kernel_svm_clf:9,polydegre:[1,6,7,11,33,34,35],polygon:[14,36,37],polym:[13,38,39],polymi:26,polynomi:[1,6,7,8,9,10,11,12,16,17,18,20,22,26,27,32,33,35,36,38,39],polynomial_featur:[7,35],polynomial_svm_clf:9,polynomialfeatur:[1,7,9,10,33,35,36],polytrop:[1,7,32,35],pool:4,pool_siz:4,poor:[2,14,22,36,37,38,39,40],poorli:[1,33],pop:[],popul:[1,6,32,33,34,35],popular:[1,2,4,7,8,9,10,12,13,16,24,25,26,29,33,36,38,39,40],popularli:[1,32],portabl:11,portion:[12,14,22,37,38],pose:[1,5,6,7,12,29,32,35],posit:[1,2,3,4,6,8,9,11,12,14,15,23,25,29,32,33,34,35,36,37,38,39,40,41],possibl:[1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,20,22,23,24,25,26,27,29,30,32,33,34,35,36,37,38,39,40],posterior:[6,34,35],postpon:[1,32,33],postscript:[26,27],postul:[6,34,35],potenti:[1,4,6,7,13,14,32,33,34,35,37,38,39],pott:[13,38,39],power:[1,2,6,7,9,10,13,14,32,33,35,37,38,39,40,41],pp:[6,7,19,34,35,36,39,40,41],practic:[0,1,6,7,8,9,23,26,27,29,33,34,35,36],practition:[1,2,4,32,39,40,41],pre:32,precalcul:39,preced:[2,12,13,29,38,40,41],preceed:[5,40,41],preceq:9,precis:[1,3,6,12,14,25,26,27,29,32,33,34,35,37,38,39,41],pred:[7,35,37,38],pred_train:[40,41],pred_val:[40,41],predicit:1,prediciton:[40,41],predict:[0,1,2,6,7,8,9,10,11,16,17,24,26,31,32,33,34,35,36,38,39,40,41],predict_prob:[2,40,41],predict_proba:[0,8,11,36,38,39],predictor:[1,6,7,8,10,11,12,32,33,34],prefer:[1,2,7,9,10,12,14,16,24,26,27,32,39,40],prepar:[1,7,25,26,27,32,33],preprocess:[0,1,5,7,8,9,10,11,12,26,34,35,36,40,41],prerequisit:1,prescript:[26,27],presenc:[14,37,38],present:[1,6,7,8,10,13,14,22,25,26,27,29,32,33,34,37,38,39,40,41],preserv:[4,12,25],press:[14,31,36,37,39,40,41],pretrain:[2,5,39,40],pretti:[1,5,9,10,16,23,24,26,32],prev_centroid:15,prevent:[14,29,37,38],previou:[1,2,3,4,5,6,7,9,11,12,13,14,22,23,25,26,27,29,33,36,37,38,39,40,41],previous:[3,4,10,11,29,41],price:[1,5,10,14,37,38],primal:9,primari:[1,8,32,36],prime:29,primit:[3,23],princip:[0,1,6,8,24,32,33,36],principl:[1,7,8,9,15,32,35,36],print:[0,1,2,3,4,5,6,7,8,9,10,11,12,14,15,22,23,25,29,32,33,34,35,36,37,38,39,40,41],print_funct:[9,10],print_length:[40,41],printout:[1,32],prior:[1,6,7,32,34,35],privat:1,pro:27,prob:[2,29,40,41],probabilist:[1,31,32,33],probabl:[0,1,2,4,5,7,8,11,14,23,24,32,33,36,37,38,39,40,41],problem:[0,1,4,5,6,7,8,9,10,11,12,13,17,18,24,25,26,27,29,35],probml:31,proce:[1,6,7,8,9,10,11,12,14,25,32,33,34,35,37,39],procedur:[3,5,6,7,9,11,12,14,22,26,33,34,35,36,37,38,41],proceed:25,process:[1,3,5,7,10,11,13,14,16,22,24,25,26,29,31,32,35,36,37,38,39],prod:31,prod_:[2,6,8,34,35,36,40,41],produc:[1,4,5,6,7,10,11,12,13,14,24,25,26,29,32,33,34,35,37,38,39],product:[1,2,4,6,7,8,9,13,14,17,22,24,25,32,33,34,35,36,38,39,40,41],profess:[1,32],progag:27,program:[1,2,5,6,7,9,13,15,16,17,22,24,25,28,29,30,32,33,38,40],programm:25,progress:[2,5,15,37,39,40,41],prohibit:[7,35],project1:[7,26],project:[1,2,3,4,6,12,14,18,19,20,21,22,24,28,30,33,34,35,37,38,39,40,41],project_root_dir:[1,7,8,10,32,33,35,36],projectwrit:26,promin:[13,38,39],promis:9,promot:[30,32],prone:[10,23,39],pronounc:[14,24,32,37,38],proof:[1,12,13,14,32,35,36,37,39],prop:[40,41],propag:[3,4,14,23,27,37,38],proper:[1,3,7,8,26,35,41],properli:[2,7,9,11,14,22,26,27,37,38,39,40],properti:[1,2,4,13,14,17,18,25,32,35,37,38,39,40,41],propgag:39,proport:[1,2,6,10,12,14,17,29,32,33,37,38,40,41],propos:[2,5,7,11,26,27,32,39,40],propto:[6,14,34,35,36,37,38],proton:[1,32],prove:[4,14,36,37,38],provid:[1,2,4,5,6,7,9,10,11,13,14,16,22,23,24,25,26,27,29,32,33,34,35,36,37,38,39,40,41],proxi:[2,14,22,37,38,39,40],prune:10,pseudo:[25,29,37],pseudocod:[26,27],pseudoinv:[6,34],pseudoinvers:[6,7,34],pseudorandom:[7,29,35],psycholog:[1,32],pt:[14,37],publish:[39,40,41],punish:[1,2,32,40,41],pure:[4,10,29],purest:10,puriti:10,purpos:[1,4,11,13,15,23,32,38,39],put:[2,40,41],py:[0,1,2,3,4,5,6,7,8,9,10,12,15,22,23,32,34,35,36,37,38,40,41],py_tf:[4,5,15],pycod:32,pydata:24,pydot:10,pyhton2:32,pylab:[1,8,32,36],pypi:24,pyplot:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,22,23,25,26,29,32,33,34,35,36,37,38,39,40,41],pythagora:[6,34,35],python2:[1,16,26,32],python3:[0,1,2,3,4,5,7,8,9,10,12,15,16,22,23,24,26,32,34,36,38,40],python:[2,3,4,5,6,7,9,12,13,14,15,17,22,23,26,27,29,33,34,37,38,39,40,41],pytorch:[1,24,26,27,32,39,40,41],q:[6,7,9,12,29,33,35,40,41],qp:9,qquad:[3,12,14,25,37,38,41],qr:[6,7,25,33],quad:[2,14,25,37,40,41],quadrat:[1,9,10,14,16,17,32,37],qualit:[5,10,26,27,29],qualiti:[1,10,16,17,24,32,33,39],quantifi:[2,39,40],quantil:11,quantit:[1,7,10,26,27,32,35],quantiti:[1,3,6,7,8,10,11,12,13,15,17,25,29,32,33,34,35,36,39,40,41],quantum:[5,13,31,32,38,39],quartil:[1,33],quasi:39,quench:6,queri:10,question:[1,6,7,10,12,13,14,26,30,32,33,34,35,37,38,39,40],qugan:5,quick:[5,29],quickli:[2,4,10,12,14,36,37,39,40,41],quit:[2,6,7,10,11,13,33,35,38,39,40,41],quot:5,r2:[1,6,7,27,32,33,34,36],r2_score:[1,32],r2score:[1,32],r:[1,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,22,24,25,26,29,33,34,35,36,37,38,39,40,41],r_1:10,r_2:10,r_j:10,r_m:10,rad:1,radial:[1,9,13,38,39],radioact:29,radiu:[1,2,10,33,40,41],rag:[],rain:10,rais:[1,4,5,8,15,34,36,40,41],ramp:[2,39,40],ran0:29,ran1:29,ran2:29,ran3:29,rand:[1,5,6,7,10,11,14,16,17,22,23,25,32,33,34,35,36,37,38,40,41],randint:[7,10,14,22,35,37,38],randn:[1,2,3,6,7,10,12,14,16,17,22,23,32,33,34,35,36,37,38,39,40,41],random:[1,2,3,4,5,6,7,9,10,14,15,16,17,22,23,24,25,26,32,33,34,35,37,38,39,40,41],random_forest_model:11,random_index:[14,22,37,38],random_indic:[2,4,40,41],random_st:[0,1,8,9,10,11,12,36,38,39],randomforestclassifi:11,randomizedsearchcv:36,randomli:[2,7,10,14,15,22,35,36,37,38,40,41],randuniform:36,rang:[0,1,2,3,4,5,6,7,8,10,11,12,13,14,15,22,23,25,29,32,33,34,35,36,37,38,39,40,41],rangl:[1,7,12,29,32,33],rangle_x:29,rank:[6,33],rankdir:5,raphson:[2,9,14,40,41],rapidli:1,rare:[0,2,14,32,36,37,38,39,40,41],rasbt:41,raschka:[27,32,33,34,35,36,37,38],rasckha:32,rashcka:[39,40,41],rashkca:[39,40,41],rate:[1,2,3,4,5,9,10,11,13,14,27,36,39],rather:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,25,29,32,33,34,35,36,37,38,39,40,41],ratio:[0,5,8,10,11,12,36],rational:[1,32],ravel:[0,6,7,8,9,10,11,12,14,25,33,35,36,37,40,41],raw:4,rbf:[9,12,13,38,39],rbf_kernel_svm_clf:9,rbf_pca:12,rc:[1,29],rc_temp:[1,8,36],rcond:[1,32,33],rcparam:[1,2,4,8,9,10,11,29,32,36,40,41],re:[3,5,14,23,36,37,41],reach:[0,2,5,6,7,10,11,12,13,14,15,34,35,36,37,38,39,40,41],read:[1,3,4,5,6,7,8,9,12,13,17,22,25,26,27,28,29,31,34,37],read_csv:[1,7,8,10,32,35,36],read_fwf:[1,32],reader:[1,7,25,29,32,33,34],readi:[1,2,6,7,9,11,12,13,25,26,32,34,35,39,40,41],readili:[2,40,41],readthedoc:24,real:[0,1,2,5,8,11,12,13,25,33,35,36,38,40,41],real_loss:5,real_output:5,realist:9,realiti:29,realiz:[2,13,38,40,41],realli:[1,2,32,39,40,41],rearrang:[14,37,38],reason:[1,2,4,5,11,14,31,32,36,37,38,39,40],reassign:[2,40,41],reat:[40,41],reber:[40,41],recal:[6,7,10,11,12,13,25,29,32,33,34,35,36,37,39,40],recast:4,receiv:[2,4,11,13,29,38,39,40,41],recent:[1,3,4,5,7,8,11,14,15,22,23,26,31,32,34,35,36,37,38,39,40,41],recept:[4,13,38,39],receptive_field:4,recip:[1,7,8,25,26,27,32,33,36],reciproc:[6,34],recogn:[1,5,6,11,32,34,35],recognit:[1,2,4,13,28,31,32,38,39,40],recommen:32,recommend:[1,3,4,5,6,7,9,14,16,19,22,23,24,25,26,27,31,34,35,37,38],reconsid:10,reconstruct:12,record:[11,26,27,28,30,32,37,38],recreat:23,rectangl:[10,14,36,37],rectangular:[6,33],rectifi:[2,4,13,38,39,40,41],recur:[1,24,32],recurr:[1,2,24,32,40],recurs:[10,24,25,32],red:[1,4,5,7,9,10,22,35,37,38],redefin:[1,11,32,33],redefinit:34,reduc:[2,4,6,7,10,11,12,14,22,32,34,35,36,37,38,40,41],reduct:[1,11,12,24,29,32,33],refer:[0,1,2,3,4,6,7,10,12,13,14,15,21,25,27,31,32,33,35,36,37,38,39,40,41],referenc:[3,39,40,41],refin:[13,38,39],refit:[7,35],reflect:[1,2,5,6,26,27,29,32,40,41],refresh:[24,32],refreshprogrammingskil:32,reg:[11,12],regard:[2,10,14,37,39,40],regardless:[13,38,39,40],region:[4,5,7,10,13,26,38,39],regist:[7,26,29],reglasso:[6,34],regr_1:[1,10],regr_2:[1,10],regr_3:[1,10],regress:[2,9,12,13,16,17,22,24,25,40,41],regressor:[1,8,11,32,36,40,41],regridg:[1,6,7,34,36],regular:[0,1,4,5,6,7,8,10,14,18,27,30,32,33,34,35,37,38,39],regularis:7,reilli:[1,16,31,32],reinforc:[1,9,24,32,41],reiniti:[40,41],reiter:[2,40,41],reject:8,rel:[0,1,5,7,8,10,13,14,23,29,32,33,35,36,37,38,39,40],relat:[1,2,4,5,6,12,14,15,25,29,32,34,35,37,38,39,40],relationship:[1,5,10,32],relativeerror:[1,32,33],releas:[2,24,27,32,40,41],relev:[0,1,2,6,8,12,16,22,24,26,27,29,32,34,41],reli:[1,7,9],reliabilti:[26,27],reliabl:[8,29,36],relu:[4,5,23,27,32],remain:[2,3,5,7,13,25,29,33,34,35,36,38,39,40,41],remaind:29,reman:[3,41],remark:[2,39,40],rememb:[1,9,14,23,25,26,27,32,37,38],remind:[1,6,12,14,25,29,33,34,35],remov:[1,5,6,7,33,34],render:[1,10,32,33],reorder:[0,6,8,33,34,36],reorgan:[1,32],repeat:[1,2,4,5,6,7,10,11,12,14,15,22,25,26,27,29,32,34,35,36,37,38,39,40,41],repeat_to_match_shap:[],repeated:32,repeatedli:[1,7,11,14,35,37,38],repet:4,repetit:[7,32,33,35,36,37],rephras:[14,36,37],replac:[1,2,4,5,6,7,11,13,15,16,17,22,24,26,27,32,33,34,35,36,39,40,41],replica:[7,35],repo:[26,27],report:[21,32,37,38],reportexampl:[21,26],reportsampl:21,repositori:[1,5,26,27,32],repres:[0,1,2,3,4,5,6,7,8,9,10,11,13,14,27,29,32,33,34,35,36,37,38,39,40,41],represent:[1,2,4,7,10,29,32,35,36,40,41],representd:4,reproduc:[1,6,7,10,13,16,17,24,26,29,32,33,39,40,41],repuls:[1,32],request:[1,3,14,22,37,38],requir:[1,2,4,5,6,7,9,10,12,13,14,18,25,32,33,34,35,36,37,38,39,40],rerun:[10,40,41],res1:[3,41],res2:[3,41],res3:[3,41],res_analyt:[3,41],res_analytical1:[3,41],res_analytical2:[3,41],res_analytical3:[3,41],resaml:[7,26],resampl:[1,8,11,20,24,32,33,36,37,40,41],rescal:[1,12,13,33,38],rescu:[6,34,35],reseach:[7,26],research:[1,5,14,22,23,24,31,32,37,38],resembl:[7,29,35],reserv:[2,6,7,29,34,35,40,41],reset:[40,41],reset_weight:[40,41],reshap:[1,2,3,4,5,7,9,10,11,16,17,25,32,33,35,40,41],residenti:1,residu:[1,6,14,32,37],resiz:[6,33],resourc:32,respect:[0,1,2,3,4,6,7,8,9,11,12,13,14,15,17,18,22,23,26,29,32,33,34,35,36,37,38,39,40,41],respond:[13,38,39],respons:[1,8,10,13,32,33,36,38,39],rest:[1,6,23,33],restat:[1,13,32],restor:5,restored_discrimin:5,restored_gener:5,restrict:[1,4,10,13,32,38,39,40,41],result:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,22,23,24,25,26,27,29,32,35,36,37,38,41],result_typ:23,retail:1,retain:[6,7,33,35,36],rethink:35,retriev:32,return_data:15,return_sequ:5,return_x_i:10,reus:[2,4,7,19,20,21,26,27,39,40],reveal:[1,13,32,38,39],revers:[2,25,40,41],review:[24,25],revisit:[15,39],revolut:32,reward:[1,5,32],rewrit:[0,1,4,6,7,8,9,11,12,13,14,20,25,26,29,34,36,37,38,39,40],rewritten:[3,7,9,11,29,35,41],rewrot:[14,36,37],rf:11,rgb:4,rgoj5yh7evk:24,rh:[7,35],rho2:[40,41],rho:[1,11,14,22,32,37,38,40,41],rho_1:11,rho_2:11,rho_m:11,rich:[1,32],ride:10,rideclass:10,ridedata:10,ridg:[0,8,12,14,22,24,27,32],ridge_sk:7,ridgebeta:[6,34],ridgecv:36,right:[0,1,2,3,4,6,7,8,9,10,11,13,14,15,17,18,20,23,25,26,29,32,33,34,35,36,37,38,39,40,41],right_sid:[3,41],rightarrow:[1,2,6,7,9,12,13,14,29,32,33,35,36,37,38,39,40,41],rigor:[1,32],ring:7,rise:[1,32],risk:[1,14,16,17,32,36,37,38],rival:5,river:1,rlm:32,rm:[1,29,40,41],rms_prop:[40,41],rmse:1,rmsporp:[14,22,37,38],rmsprop:[2,4,5,14,27,39,40,41],rnd_clf:11,rng:29,rnn1:5,rnn2:5,rnn:[5,13,38,39],rnn_2layer:5,rnn_input:5,rnn_output:5,rnn_train:5,rntrick1:29,rntrick2:29,rntrick3:29,rntrick4:29,ro:[1,14,22,32,36,37,38],robert:[19,26,31],robust:[1,32],robustscal:[1,33],roc:[8,11],role:[1,3,6,7,9,24,32,33,34,35,36,39,40,41],roll:7,room:[1,30,32],root:[1,6,10,14,29,33,34,36,37,38],rot:32,rotat:[2,9,10,11,40,41],rotation_matrix:10,roughli:[2,4,39,40],round:[0,1,8,10,14,36,37,38,40,41],routin:[14,25,32,36,37],row:[1,2,3,6,7,10,12,23,25,32,33,34,35,40,41],row_stack:[],rr:[6,33],rrr:[6,33],rtol:23,rug:[14,36,37,38],rule:[1,2,6,7,14,23,26,32,33,34,35,37,38,41],run:[1,2,3,5,6,7,9,10,12,14,16,22,23,24,26,27,32,33,34,35,36,37,38,40,41],runtim:[2,7,15,39,40,41],runtimeerror:34,runtimewarn:[2,7,35,40,41],rust:[1,16,24,25,32],rvert:[2,40,41],rvert_2:[2,40,41],s:[1,2,3,4,5,6,7,8,10,12,13,14,17,18,19,23,24,25,26,27,29,32,33,34,39,40,41],s_1:7,s_:[4,7],s_i:[7,8,36],s_j:7,s_k:7,saddl:[14,36,37,38],safe:23,sai:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,25,26,29,32,33,34,35,36,39,40,41],said:[7,10,14,36,37],sake:[0,1,6,8,12,32,33,34,36,39,40],sale:[1,32],sam:32,same:[1,2,3,4,5,6,7,9,10,12,13,15,18,23,25,26,29,32,33,34,36,39,40,41],samm:11,sampl:[0,1,2,3,4,5,6,7,8,9,10,11,14,15,16,17,20,22,24,25,26,29,32,33,34,35,36,37,38,39,40,41],sample_vari:15,sampleexptvari:29,samwis:32,sandboxmod:23,sastri:12,satisfactori:[1,32],satisfi:[2,3,4,7,9,14,25,29,35,36,37,40,41],satur:[2,7,35,39,40],save:[1,5,7,8,10,14,22,32,33,35,36,37,38],save_fig:[1,7,8,10,11,32,33,35,36],savefig:[1,5,7,8,10,29,32,33,35,36],savetxt:5,saw:[6,33],scalabl:11,scalar:[3,6,7,11,33,34,35,37,39,40,41],scale:[0,1,2,4,6,7,8,9,10,11,12,13,14,16,17,18,22,24,25,26,27,30,32,34,36,37,38,39,40,41],scale_mean:5,scale_std:5,scaler:[1,8,9,10,11,12,26,33,34,40,41],scan:[6,8,34,35,36],scari:[6,34,35],scatter:[1,2,7,8,9,10,15,22,23,32,33,34,35,36,38,40,41],scenario:[7,14,36,37,38],schedul:[14,30,37,38],scheduler_arg:[40,41],schedulers_bia:[40,41],schedulers_weight:[40,41],scheme:[2,14,36,37,38,39,40],schrage:29,scienc:[0,1,2,11,13,14,24,28,29,30,31,36,37,38,39,40],scientif:[1,16,24,26,27,32,37,38],scientist:[1,32],scikit:[0,4,6,7,9,10,11,14,22,23,24,25,26,27,28,31,38,39],scikit_learn:[1,17],scikitlearn:32,scikitplot:[0,8,11,36],scipi:[1,4,6,7,14,16,24,25,26,32,33,34,35,36,37],scl:7,score:[0,1,2,4,7,8,10,11,12,16,17,23,26,27,30,32,33,35,36,37,38,39,40,41],scores_kfold:[7,35,36],scratch:[2,14,38,39,40],sdg:[14,22,37,38],sdv4f4s2sb8:38,seaborn:[0,1,2,4,7,8,22,27,32,36,40,41],seamless:[1,16,24,26,32],seamlessli:[40,41],search:[1,2,4,6,10,14,32,34,37,38,39,40,41],sebastian:[32,39,40,41],sebastianraschka:[27,32,41],sec:7,second:[0,1,3,4,5,6,7,8,9,10,12,13,15,16,17,23,24,25,26,27,29,30,32,33,35,36,39,40,41],second_correct:[40,41],second_mo:[22,37,38],second_term:[22,37,38],secondeigvector:12,secondli:[13,39,40,41],section:[5,12,17,18,25,26,29,33,34,36,37,38],sector:1,see:[0,1,2,3,4,5,6,7,8,9,11,12,13,14,16,17,18,19,22,23,24,25,26,27,29,32,33,34,35,36,37,38,39,40,41],seed:[1,2,3,4,5,6,7,9,10,12,14,15,16,17,22,23,26,29,32,33,34,35,36,37,38,39,40,41],seed_imag:5,seek:[2,3,9,39,40,41],seem:[2,4,5,37,38,39,40,41],seemingli:[1,32],seen:[1,2,4,6,11,13,18,29,32,40,41],segment:[14,36,37,40,41],seismic:7,seldomli:[1,32],select:[2,6,7,9,10,11,12,18,26,27,28,29,30,31,32,33,34,35,40,41],self:[0,1,2,6,8,22,32,33,34,36,38,40,41],sell:5,semest:[0,8,28,36],semi:[9,14,36,37],semilogx:7,send:[6,13,14,23,30,32,37,38,39],senior:[28,30],sens:[1,5,7,9,23,26,32,35],sensibl:[4,23],sensit:[1,6,7,10,14,32,33,34,35,38],sent:[3,23,39,40,41],sentdex:[38,39,40],sentenc:[5,13,38,39],separ:[1,2,3,5,7,9,10,13,15,16,23,24,26,29,32,35,38,39,40,41],septemb:[18,19,20,21,26,32],sequenc:[3,4,5,8,10,11,13,14,24,25,29,32,36,37,38,39],sequenti:[2,4,5,11,13,29,38,39,40,41],seri:[1,2,3,4,5,6,7,11,12,13,14,25,32,33,34,35,36,37,38,39,40,41],serif:[1,8,29,32,36],serv:[1,2,3,4,6,8,14,22,26,31,32,33,34,35,36,37,38,39,40,41],servic:[26,27],session:[2,21,26,28,30,32],set:[0,2,5,6,7,8,9,11,12,14,15,17,18,22,23,24,25,26,27,29,30,34,35,37,38],set_major_formatt:[7,26],set_major_loc:[7,26],set_tick:[2,7,9,40,41],set_ticklabel:[2,7,40,41],set_titl:[0,1,2,3,4,8,13,15,32,36,38,39,40,41],set_xlabel:[0,1,2,3,4,8,13,32,36,38,39,40,41],set_xlim:[0,8,13,36,38,39,40],set_xticklabel:[2,40,41],set_ylabel:[0,1,2,3,4,8,32,36,40,41],set_ylim:[0,8,13,36,38,39,40],set_ytick:[0,8,36],set_yticklabel:[2,7,40,41],set_zlim:[7,26],seth:5,setminu:[7,36],setosa:[9,10],setosa_or_versicolor:9,setp:[7,35],setup:[2,5,7,9,22,24,27,32,33,39,40],sever:[1,4,6,7,8,9,10,12,13,14,22,24,25,27,29,32,33,34,35,36,37,38,39],sgd:[2,4,22,27,36,40,41],sgd_clf:9,sgdclassifi:9,sgdreg:[14,36,37],sgdregressor:[14,36,37],sgn:[6,33,34],shallow:[14,37,38],shape:[0,1,2,3,4,5,6,7,8,9,10,11,12,14,15,16,17,23,25,32,33,34,35,36,37,38,39,40,41],shape_bas:[],share:[2,4,32,39,40,41],she:[8,36],shell:[22,38],shift:[2,7,13,29,34,38,39,40],ship:4,shire:32,shortcom:[14,36,37,38],shorten:5,shorter:29,shorthand:[32,35],shortli:[25,32],should:[1,3,4,6,7,9,10,12,13,16,17,22,23,25,26,27,29,32,33,34,35],show:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,17,18,19,20,22,25,26,29,32,33,34,35,36,37,38,39,40,41],show_shap:5,shown:[0,1,5,6,9,10,12,13,14,22,25,33,36,37,38,39,40,41],shrink:[4,6,7,9,12,26,33,34],shrinkag:[6,7,33,34],shrunk:12,shuffl:[1,2,5,7,14,22,33,35,36,37,38,40,41],sickit:[39,40],side:[1,3,6,9,13,14,25,27,32,34,35,36,37,38,39,40,41],sigh:[24,32],sigma0:29,sigma1:29,sigma2:29,sigma:[0,1,2,6,7,8,11,12,13,14,18,19,20,25,26,29,32,33,34,35,36,37,38,39,40,41],sigma_0:[6,33,34],sigma_1:[6,33,34,39,40],sigma_2:[6,33,34,39,40],sigma_:[6,25,32,33,34,35],sigma_fn:[0,8,13,36,38,39,40],sigma_i:[1,6,32,33,34],sigma_j:[6,18,33,34],sigma_m:[7,29,35],sigma_n:[12,29],sigma_t:[14,37,38],sigma_x:29,sigmoid:[0,2,3,5,8,9,11,13,23,27,36,37,38,41],sigmundson:[7,33,34],sign:[2,3,8,9,11,29,30,36,38,40,41],signal:[2,4,11,13,38,39,40],signifi:5,signific:[2,39,40],significantli:[2,14,22,29,36,37,38,39,40,41],sigurd:32,sim:[5,6,7,14,19,26,29,34,35,37,38],similar:[1,2,3,4,5,6,7,8,9,10,11,12,15,18,24,25,26,27,32,34,35,36,39,40,41],similarli:[1,2,4,6,9,11,14,21,29,32,33,34,39,40],similiar:[40,41],simpl:[0,2,3,4,6,7,8,9,11,12,13,15,16,17,18,20,22,24,25,26,27,29,35,41],simplefilt:[40,41],simplepredict:11,simpler:[1,2,6,7,8,14,17,20,22,24,26,27,32,34,37,38,40,41],simplernn:5,simplest:[1,2,4,5,10,11,13,15,32,38,39,40,41],simpletre:11,simpli:[1,2,3,5,6,7,9,10,11,12,13,16,24,25,26,27,29,32,33,34,35,38,39,40,41],simplic:[0,3,6,7,8,9,10,11,12,13,15,33,34,36,38,39,40,41],simplicti:[6,33],simplif:39,simplifi:[1,7,10,16,24,26,32,33,34,35,39],simplist:[4,7,29,35],simul:[7,35],simultan:[7,35],sin:[1,2,3,4,5,10,13,14,22,25,32,37,38,39,40,41],sinc:[0,1,2,3,4,6,7,8,9,10,11,12,14,22,23,25,27,29,31,32,33,34,35,36,37,38,39,40,41],sine:[4,13,38,39,40],singl:[0,1,2,3,4,6,7,8,9,10,13,14,23,25,27,29,32,33,34,35,36,37,40,41],singular:[1,7,14,25,26,32,34,35,36,37],sinusoid:4,site:[0,1,2,3,4,5,7,8,9,10,12,15,22,23,26,27,28,32,34,36,38,40,41],situat:[1,5,6,8,14,22,29,32,33,36,37,38],six:[4,29,39],size:[1,2,3,4,5,6,7,9,10,11,12,14,22,23,25,26,27,29,32,34,35,36,37,38,39,40,41],sketch:11,ski:10,skill:1,skip:12,skl:[1,7,32,33,34],sklearn:[0,1,2,4,6,7,8,9,10,11,12,14,15,23,32,33,34,35,36,37,38,39,40,41],skplt:[0,8,11,36],skrankefunct:[40,41],sl:[7,34,36],slack:9,slice:[3,25,32,41],slide:[1,4,16,17,26,27,29,32,33,35,37,39,40,41],slight:[7,14,35,37,38],slightli:[0,2,3,4,6,7,8,11,29,33,34,35,36,39,40,41],slope:[9,12,13,38],slow:[1,3,9,14,33,36,37,38,41],slower:[6,25,32,33,34],slowest:25,slowli:13,slp:[2,40,41],small:[1,2,3,4,6,7,9,10,11,12,13,14,22,23,24,25,26,29,32,33,34,35,36,37,38,39,40,41],smaller:[1,2,3,6,7,9,10,12,14,23,29,32,33,34,35,36,37,38,39,40,41],smallest:[1,5,15,16,17,32,36],smallest_row_index:15,smooth:[1,4,7,10,14,26,32,36,37],sn:[0,1,2,4,7,8,32,36,40,41],sne:12,sneak:32,so:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,17,19,20,22,23,24,25,26,27,29,30,32,33,34,35,36,37,38,39,40,41],soar:7,social:1,soft:[0,2,8,11,13,36,38,39,40,41],soften:9,softmax:[0,4,8,23,36,39],softmax_vec:23,softwar:[1,9,16,24,25,39],sol1:23,sol:9,sole:[1,7,32],solid:[1,8,36],solut:[1,2,3,4,6,7,9,11,12,14,18,19,23,25,26,27,29,32,33,34,35,36,37,38,39,40],solution_ev:37,soluton:[3,41],solv:[1,2,4,6,7,9,11,12,13,14,25,26,27,32,33,38,39,40],solve_expdec:[3,41],solve_ode_deep_neural_network:[3,41],solve_ode_neural_network:[3,41],solve_pde_deep_neural_network:[3,41],solveod:[3,41],solveode_popul:[3,41],solver:[0,3,8,9,10,11,12,22,25,32,36,38,41],some:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,15,23,26,29,32,34,35,38,40,41],some_model:[7,33,34],somehow:5,someth:[0,1,2,4,5,8,10,12,26,27,29,32,33,36,39,40,41],sometim:[1,2,12,13,14,15,32,33,37,38,39,40],somewhat:[38,39],soon:[25,30,33],sophist:[1,32],sopt:[14,37],sort:[6,7,10,12,29,33,35],sound:[4,6,34,35],sourc:[1,2,4,7,16,24,25,26,27,29,32,35,40,41],space:[1,2,5,6,9,10,12,13,14,15,22,29,33,34,35,36,37,38,39,40],span:[1,4,6,10,12,25,32,33],spare:[2,39,40,41],spars:[4,7,25,32],sparse_mtx:[25,32],sparsecategoricalcrossentropi:4,sparsiti:11,spatial:[2,3,4,13,38,39,40,41],speak:29,special:[7,8,11,13,14,25,29,32,33,34,36,37,39,40],specif:[0,1,2,3,4,5,6,7,8,9,10,12,13,17,24,25,26,27,29,31,32,33,34,35,36,38,39,40],specifi:[0,1,4,6,7,8,10,12,14,15,22,26,29,32,34,35,36,37,38,40],specifici:[1,11,32],spectacular:4,spectral:[2,40,41],speech:[1,2,4,5,13,38,39,40],speed:[2,3,5,14,37,38,39,40,41],spend:29,spent:[26,27],sphere:[1,33],spin:7,spite:1,spline:9,split:[2,4,5,6,7,9,10,11,12,15,18,23,26,29,34,35,36,40,41],splite:1,splitter:[2,11,40,41],spontan:29,spot:4,spread:[1,12,29,32,33],spring:[40,41],springer:[19,26,31,32,35],spuriou:[14,22,37,38],sqquar:34,sqrsignal:4,sqrt:[1,4,5,6,7,9,11,12,14,18,22,29,33,34,35,37,38,39,40,41],squar:[0,2,3,4,5,8,9,10,12,14,15,16,17,18,20,22,24,25,27,29,35,36,37,38,39,40,41],squarederror:11,squaredeuclidean:15,squash:[13,38,39,40],squeez:[],srtm:[7,26],srtm_data_norway_1:[7,26],stabil:[6,26,27],stabl:[0,1,5,6,7,10,12,24,26,32,33,34],stack:[4,5],stage:[6,14,22,26,27,37,38,39,40,41],stai:[1,3,5,6,12,18,32,33,40,41],stand:[1,6,10,13,32,33,34,38],standadscal:33,standard:[1,2,5,6,7,8,9,11,13,16,17,18,20,22,25,26,27,29,32,34,36,38,39,40,41],standard_basi:[],standardscal:[1,7,8,9,10,11,12,33,34],stanford:[14,36,41],start:[1,2,3,4,5,6,7,9,10,11,12,13,14,15,21,23,25,27,29,30,32,33,35,36,37,39,40,41],start_box:[],start_nod:[],start_tim:15,starting_point:[22,38],stat:[7,35,36],state:[0,2,3,5,6,7,8,9,11,12,13,14,24,27,29,33,34,35,36,37,38,39,40,41],statement:[0,1,8,25,32,36],stationari:[36,37],statist:[0,1,2,4,5,8,10,11,12,13,14,15,19,25,26,28,31,33,36,37,38,39,40],statu:[0,1,8,10,12,32,36],stavang:[7,26],std:[1,5,7,32,33,35],stdout:[40,41],steep:[14,22,36,37,38],steepest:38,step:[0,1,2,3,5,7,8,10,11,12,13,14,15,22,25,26,32,39,40,41],step_fn:[0,8,13,36,38,39,40],step_length:[14,37,38],step_siz:37,steps_list:10,stereo:4,still:[1,3,4,6,7,12,14,18,23,29,35,36,37,38,39,41],stimuli:[13,38,39],stk2100:[31,32],stk3155:[16,26,27,28,30],stk4021:[31,32],stk4051:[31,32],stk4155:[28,30],stk5000:31,stk:[31,32],stochast:[1,2,6,7,9,12,13,16,17,26,32,34,35,36,39,40,41],stock:5,stoke:[13,38,39],stone:[1,8,26,36,39],stop:[0,2,5,10,12,14,15,22,36,39,40,41],stopiter:[1,8,36],storag:[6,33],store:[1,2,3,4,7,12,14,26,29,32,37,38,40,41],storehaug:[30,32],str:[2,4,5,40,41],straight:[1,7,9,14,32,35,36,37],straightforward:[1,3,4,6,7,9,10,11,14,25,32,33,34,35,36,37,41],strategi:[1,2,10,32,40,41],stratifi:[7,35],strength:[1,6,15,33],stretch:12,strict:[9,14,36,37],strictli:[9,14,36,37],stride:[5,25],strike:7,string:[2,40,41],stroke:[8,36],strong:[4,7,10,11,13,25,29,35,38,39],strongli:[1,9,24,25,27,40,41],stronli:1,structur:[1,2,3,4,7,10,11,13,24,32,35,36,37,38,40,41],stuck:[2,14,36,37,38,39,40,41],student:[1,26,27,28,30,31,32,41],studi:[1,4,5,6,7,8,9,12,13,14,18,20,22,24,26,27,31,32,33,34,37,38,39,40,41],studier:31,stuff:23,style:[1,8,10,25,32,36],sub:[10,13,38,39],subarg:[],subdivid:[1,25,32],subfield:1,subject:[7,9,29],submit:32,subplot:[0,1,2,4,5,7,8,9,10,11,15,23,32,35,36,40,41],subplots_adjust:[9,29],subprogram:[25,32],subract:[1,33],subroutin:[1,32],subscript:[2,40,41],subsequ:[2,5,6,7,13,25,29,33,35,38,39,40,41],subset:[2,7,10,13,14,24,32,35,36,37,38,39,40,41],subspac:[1,9,12,33],substanti:[10,11],substep:12,substitut:[4,7,13,25,35,38],subsubset:10,subtask:7,subtl:[2,39,40],subtract:[1,5,6,7,12,14,18,22,25,26,29,34,35,37,38,40,41],subtre:10,subval:[],succeed:[1,5,32],success:[4,8,10,14,29,36,37],successfulli:[5,10],sudo:[1,16,24,26,32],suffer:[1,2,3,6,11,32,33,34,39,40,41],suffici:[2,7,9,12,14,35,36,37,40,41],suggest:[2,14,26,27,31,37,39,40],suit:[9,13,38,39],suitabl:[1,29,33],sum:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,22,23,25,26,29,32,33,34,36,37,38,41],sum_:[0,1,2,3,4,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,25,26,27,29,32,33,34,35,36,37,38,39,40,41],sum_i:[1,3,6,7,9,14,18,20,26,33,34,35,37,38,41],sum_j:7,sum_ja_:1,sum_k:[7,9,13,25,39,40,41],sum_logist:[14,37],sum_m:4,sum_n:4,sum_nx_:4,summar:[6,7,10,27,34,35,36,38],summari:[2,4,5,11,22,27,28,34,40,41],summat:[1,4,17,33],sunni:10,superconduct:34,superfici:4,superscript:[2,13,38,39,40,41],supervis:[1,6,7,8,10,13,24,32,33,35,36,38,41],supplement:[0,8,26,36],support:[1,2,10,11,12,14,23,24,32,33,37,39,40,41],suppos:[0,1,6,7,8,9,11,12,13,14,25,32,33,34,35,36,37,38,39],suppress:[6,9,14,34,37,38],sure:[1,2,5,7,23,26,39,40,41],surf:[7,26],surfac:[1,7,26,32],surpass:7,surpris:[1,32],surround:[4,24],survei:[1,6,7,32,33,34,35],svc:[9,10,11],svd:[1,7,12,18,32,35],svdinv:[6,34],svm:[9,10,11,12],svm_clf:[9,11],swap:23,swath:[6,33],sy:[14,36,37,40,41],symbol:[2,4,5,6,12,14,15,24,29,32,33,34,37,38,39,40,41],symmeteri:[2,40,41],symmetr:[1,6,9,12,13,14,25,32,33,37,38,39],symmetri:[7,10],sympi:[1,16,24,26,32,39],synonim:29,syntax:[2,14,40],syntaxerror:[2,9,40],system:[0,1,2,4,5,7,8,10,11,13,14,16,24,25,26,32,36,37,38,39,40,41],systemat:[5,7,35],t0:[4,7,14,22,37,38],t1:[3,14,22,37,38,41],t2:[3,41],t3:[3,41],t:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,22,23,24,25,26,27,29,30,32,35,36,37,38,39,40,41],t_0:[3,10,14,37,38,41],t_1:[14,37,38],t_:[3,41],t_b:11,t_batch:[40,41],t_i:[2,3,6,13,27,33,40,41],t_j:13,t_k:10,t_test:[40,41],t_train:[40,41],t_val:[40,41],tabl:[10,26,27,29,30,32,38,39],tabul:[1,32],tabular:32,tackl:5,tag:[3,4,5,6,7,8,13,14,15,22,25,29,33,36,37,38,39,40,41],tagrget:39,taht:[1,32],tail:29,tailor:[3,9,12,32,39,41],taiwan:[1,32],take:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,17,22,23,24,25,29,32,33,34,35,36,37,38,39,40,41],taken:[1,2,4,7,11,14,23,25,32,35,37,40,41],tan:4,tangent:[2,5,13,14,36,37,38,39,40,41],tanh:[0,2,5,8,9,13,36,37,38,39,40,41],target:[0,1,2,4,5,6,7,8,9,10,11,12,13,23,27,32,33,34,35,36,37,38,39,40,41],target_iscomplex:[],target_meta:[],target_nam:[10,23],target_ndim:[],target_shap:[],task:[1,2,4,7,10,12,13,15,21,23,26,32,35,38,39,40,41],tau:[4,6,29,34,35],taught:32,tax:1,taylor:[3,14,36,37,39,41],taylornr:[14,36,37],tba:[],tc:9,td:[2,7,32,35,37,40,41],teach:32,team:[2,40,41],teaser:1,technic:[1,6,7,14,16,26,32,33,37,38],techniqu:[1,2,9,11,14,24,29,31,32,33,35,36,37,38,39,40],technolog:[1,2,40,41],tell:[1,5,7,11,12,14,29,35,37,38],temp1:[2,40,41],temp2:[2,40,41],temp:[2,40,41],temperatur:[1,10,32],temporarili:[2,39,40],ten:[4,18,32,39],tend:[4,6,7,9,10,11,13,14,15,22,33,34,35,37,38],tendenc:[1,32],tension:[7,35],tensor:4,tensordot:[],tensordot_adjoint_0:[],tensordot_adjoint_1:[],tensorflow:[1,3,5,9,15,16,24,25,26,27,28,31,32,33],term1:[6,7,12,26,33],term2:[6,7,12,26,33],term3:[6,7,12,26,33],term4:[6,7,12,26,33],term:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,20,22,26,29,32,33,34,36,37,38,41],termin:[1,5,6,10,11,14,16,22,32,33,37,38],terrain1:[7,26],terrain:[7,22,26,27],test:[0,4,5,6,7,8,9,10,11,14,18,20,22,23,25,26,29,35,36,37,38,39],test_acc:4,test_accuraci:[2,4,40,41],test_error:7,test_imag:[4,5],test_ind:[7,35,36],test_input:5,test_label:[4,5],test_loss:4,test_pr:[2,40,41],test_predict:[2,40,41],test_rnn:5,test_scor:[0,8,11,36],test_siz:[1,2,4,6,7,11,32,33,34,35,36,40,41],test_split:10,tester:34,testerror:[1,7,33,35],testi:5,testpredict:5,testx:5,text:[1,2,3,5,6,9,10,12,14,16,22,25,27,29,31,33,35,36,37,38,40,41],textbook:[17,18,22,26,27,33,35],textual:10,textur:[2,10,40,41],tf:[2,4,5,14,15,36,37,40,41],tf_loadlibrari:[4,5,15],th:[0,1,2,3,6,7,8,10,13,14,15,16,17,25,26,29,32,33,34,35,36,37,38,39,40,41],than:[0,1,2,3,4,5,6,7,8,10,11,12,13,14,18,23,24,27,29,32,33,34,35,38,39,40,41],thank:[5,7,33,34,41],thats:[40,41],theano:[2,24,32,40,41],thei:[0,1,2,3,4,5,6,7,8,9,10,12,13,14,25,26,29,32,33,34,35,36,37,38,39,40,41],them:[1,2,4,5,7,9,10,11,12,13,14,23,25,26,27,32,33,34,37,38,39,40,41],theme:[1,32,37],themselv:[1,29,32],thenc:[7,35],theorem:[0,3,7,8,33,36,38,40,41],theoret:[1,5,11,32],theori:[1,2,4,9,10,13,14,19,24,26,31,32,34,37,38,39,40,41],thereaft:[1,6,7,12,13,16,17,18,25,26,27,32,35,39,40,41],therebi:[1,6,8,12,32,33,34,36,39],therefor:[0,1,2,3,4,5,7,8,9,12,14,29,32,33,35,36,37,38,40,41],therein:12,thereof:[1,7,14,32,35,37,38],thesi:41,theta:[2,5,14,22,29,37,38,39,40,41],theta_:[2,14,37,38,40,41],theta_i:[2,40,41],theta_linreg:[14,22,37,38],theta_t:[14,37,38],thetaand:[38,39],thetaor:[38,39],thetaxor:[38,39],thi:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,24,25,26,27,28,29,31,33,34,35,36,37,38,40,41],thing:[0,1,2,3,5,6,8,10,23,29,32,34,35,36,39,40,41],think:[1,2,4,5,7,10,13,14,15,29,32,35,36,37,38,39,40,41],third:[1,4,7,14,22,30,32,36,37,38],thirti:[0,8,36],thorughout:32,those:[1,4,6,7,9,10,11,12,22,25,26,27,28,32,33,35,39],though:[2,3,4,5,14,23,25,29,37,38,40,41],thought:[7,15,26,27,29,35],thousand:[1,2,26,33,37,38,39,40],three:[1,2,4,6,7,9,10,13,16,23,25,26,28,29,30,32,33,34,35,36,38],threshold:[2,4,10,11,12,13,14,37,38,39,40,41],through:[1,2,3,4,5,6,7,9,12,13,14,15,23,24,25,29,32,33,34,35,36,37,38,40,41],throughout:[1,5,6,15,24,25,29,32,40,41],thu:[1,2,3,6,7,8,9,11,12,13,14,22,26,30,32,33,34,35,36,37,38,39,40,41],thumb:[1,7,26,32,33],thursdai:[30,33],tibshirani:[7,19,26,28,31,32,35],tick:7,tick_param:7,ticker:[7,14,26,29,36,37],tif:[7,26],tight_layout:[0,2,8,36,40,41],tightli:12,tild:[1,6,7,8,12,16,17,18,19,20,26,29,32,33,34,35,39,40,41],till:[0,1,5,8,9,10,11,13,25,32,33,34,36,39,40,41],time:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,17,18,22,23,24,25,26,27,29,32,33,34,35,36,39,40,41],timeit:5,timer:5,tini:[2,39,40],tip:4,titl:[1,2,3,4,5,7,8,9,10,11,14,22,23,26,29,32,35,36,37,38,39,40,41],tmp:[14,37,38],tn:[3,4,8,41],to_categor:[2,4,5,40,41],to_categorical_numpi:[2,40,41],to_numer:[1,7,32,35],todai:4,togeth:[1,4,7,9,12,14,22,24,32,37,38,41],toi:[15,37,38],told:[14,37],toler:[3,7,15,23,41],tolist:5,tomographi:[13,38,39],too:[1,3,5,6,7,10,12,14,16,29,31,32,33,35,36,37,38,41],took:[9,32],tool:[1,2,4,7,14,24,33,35,37,38,39,40],toolbox:9,top:[1,4,6,7,10,11,19,24,32,34,35],topic:[1,6,7,8,9,24,26,27,33,34,36],topograph:26,topolog:[2,4,13,38,39,40],toposort:[],torkjellsdatt:[30,32],toss:[11,29],total:[0,1,2,3,4,5,7,8,9,10,11,12,13,14,15,16,17,25,27,29,30,32,33,35,36,37,39,40,41],total_loss:5,totalclustervari:15,totalscatt:15,toward:[2,3,8,13,14,26,36,37,40,41],towardsdatasci:[37,38],town:1,tp:[5,8],tpng:10,tpu:[14,22,24,32,37,38],tqdm:7,tr:33,trace:[3,23],trace_stack:[],traceback:[1,3,4,5,7,8,11,14,15,23,26,32,34,36,41],tracer:[3,23],track:[4,14,15,25,33,36,37,38],tract:1,tractabl:[1,32,33],trade:[6,10,20,34,35],tradeoff:[1,6,20,26,32,33,34,36],tradit:[1,2,5,7,32,35,40,41],train:[3,4,6,7,9,10,11,12,13,14,18,20,22,26,27,34,35,36,37,38],train_acc:[40,41],train_accuraci:[1,2,4,32,40,41],train_dataset:5,train_end:[1,2,33,40,41],train_error:[7,40,41],train_imag:[4,5],train_ind:[7,35,36],train_label:[4,5],train_network:23,train_pr:[2,40,41],train_siz:[1,2,4,33,40,41],train_step:5,train_test_split:[0,1,2,4,6,7,8,10,11,12,32,33,34,35,36,40,41],train_test_split_numpi:[1,2,33,40,41],trainable_vari:5,trained_model:[7,33,34],trainerror:[1,33],traini:5,training_checkpoint:5,training_dataset:5,training_gradi:[14,22,37,38],training_gradient_fun:[37,38],training_loss:[37,38],trainingerror:[7,35],trainpredict:5,trainscor:5,trainx:5,trait:[1,32],trajectori:5,trajectory_i:[22,38],trajectory_x:[22,38],transfer:10,transform:[0,1,6,7,8,9,10,11,12,13,14,22,23,24,25,32,33,34,35,36,37,38,39,40,41],transit:[7,13,38,39],translat:[2,5,7,11,32,33,34,40,41],transpos:[2,6,12,23,25,33,34,40,41],travers:[1,6],treat:[1,2,4,7,13,14,23,29,32,33,34,35,36,37,38,39,40,41],tree:[1,2,7,24,26,32,39,40,41],tree_clf:[10,11],tree_clf_:10,tree_clf_sr:10,tree_reg1:10,tree_reg2:10,tree_reg:10,trend:29,treue:8,trevor:[19,26,31],tri:[3,4,5,10,14,36,37,41],triain:1,trial:[1,3,5,7,14,29,32,35,36,37],triangl:[14,36,37],triangular:25,trick:[4,5,9,12,14,22,29,37,38],trickier:29,tridiagon:25,trigger:[1,8,36],trillion:24,trivial:[1,2,6,12,29,32,34,40,41],troubl:[1,9,13,23,33,39,40],truck:4,true_beta:[7,33,34],true_divid:[],true_fun:[7,35],truncat:[22,38,39],trust:10,tucker:9,tuesdai:[30,32,35,36,37,41],tumor:[0,8,10,27,36],tumour:[0,8,36],tunabl:[2,22,27,40,41],tune:[5,10,14,22,25,27,32,36,37,38],tup:[],tupl:[23,40,41],turn:[0,1,2,6,7,8,9,10,11,12,13,14,20,25,26,29,32,33,35,36,37,38,39,40,41],tutori:[2,5,40,41],tv:[3,41],tveito:[3,41],tvw1zdmznwm:38,tweak:[2,5,11,29,39,40,41],twice:[14,36,37],twist:12,two:[1,2,3,5,6,7,8,10,11,12,13,14,16,17,23,25,28,29,31,32,34,35,37,38,41],tx:[14,36,37,38,39],tx_1:[14,36,37],txt:5,ty:[14,36,37],type:[1,2,4,7,9,11,14,17,23,25,26,29,33,34,35,36,37,40],typeerror:[7,14,23],typic:[1,2,3,4,5,6,8,10,11,13,14,22,27,29,32,33,34,35,36,37,38,39,40,41],typo:[26,27],u:[1,3,6,7,9,11,12,13,18,25,32,33,34,38,39,41],u_:25,u_i:[13,38],u_m:11,ua:[1,32],ubuntu:[1,16,24,26,32],uci:[1,27],ufunc:23,uio:[23,26,27,30,31],un:15,unabl:[10,23],unari:[25,32],unary_f:[],unary_oper:[],unary_to_nari:[],unbalanc:[7,10,35,36],unbias:[1,6,7,32,34,35],unbroadcast:[],unbroadcast_f:[],uncent:[7,34],uncertainti:[1,6,32,34,35],uncertitud:29,unchang:[2,4,39,40],uncorrel:[11,29],undefin:[6,33],under:[1,2,6,7,11,14,16,17,24,26,27,32,33,34,35,36,37,40,41],underdetermin:[1,32],underfit:[2,7,35,39,40],underflowproblem:[6,34,35],undergo:[6,23,34,35],undergradu:[28,30],underli:[1,2,10,14,29,32,37,38,39,40],underset:[5,15],understand:[1,2,4,6,7,11,14,15,22,23,24,32,33,34,36,37,38,39,40,41],understood:[9,14,37,38],undesir:9,undetermin:[6,9,34,35],undo:5,unexpect:[7,14,35],unexpected:29,unfair:[7,33,34],unfortun:[2,9,10,11,39,40],unicode_liter:[9,10],uniform:[1,2,6,7,12,14,16,17,26,29,32,33,36,37,40,41],uniformli:[14,29,36,37,38],unifrompdf:29,unimport:[14,36,37],union:[6,7,34,35,36],uniqu:[1,3,7,14,15,25,32,35,36,37,41],unique_cluster_label:15,unit:[1,2,4,5,6,11,13,29,32,33,34,38,39,40,41],unitari:[6,7,25,33],unitarili:[25,32],uniti:29,univari:29,univers:[0,1,2,3,14,16,24,26,27,28,30,32,33,34,35,36,37,38,40,41],unix:[2,40,41],unknow:[1,25,32],unknown:[1,2,4,5,6,7,9,11,14,17,22,25,32,33,34,35,37,38,39,40,41],unknowwn:13,unlabel:[2,39,40],unless:[1,4,7,12,14,26,27,32,35,36,37,39],unlik:[2,4,9,14,36,37,38,39,40,41],unnecessarili:10,unord:4,unravel:[2,40,41],unrol:[4,12],unseen:[1,8,10,36],unstabl:[2,39,40],unsupervis:[1,2,5,13,24,32,38,39,40],unsymmetr:[25,32],until:[2,3,5,10,13,14,15,23,36,37,38,39,40,41],untouch:1,unusu:[13,38,39],up:[2,4,5,6,7,9,11,12,14,15,17,19,23,24,25,26,27,29,30,34,37],updat:[0,2,3,11,13,14,15,22,23,37,38],update_chang:[40,41],update_matrix:[40,41],uploa:32,upload:[24,26,27,31],upon:[1,2,7,8,12,25,26,39,40,41],upper:[1,9,10,17,25,33],uppercas:[25,32],upsampl:5,upscal:5,uptad:39,ur:40,url:[1,8,32,33,36,38],us:[5,6,7,9,10,11,12,13,15,16,17,18,19,20,23,25,28,29,31,34,35],usa:[32,38],usag:[0,1,9,24,32,33,38,39],usd10000:1,usd:1,use_bia:5,usecol:[1,32],useless:[2,40,41],user:[0,1,2,3,4,5,7,8,9,10,12,15,16,22,24,25,26,32,33,34,36,38,40,41],usernam:[26,27],userwarn:[7,22,38],usetex:29,usg:[7,26],usr:29,usual:[0,1,4,5,8,13,14,15,32,36,37,38,39],ut:[6,34],utf:[1,8,36],util:[2,4,5,7,8,11,15,27,32,35,36,39,40,41],ux:25,v0:29,v1:29,v2:29,v8xr:[38,39,40],v:[3,5,6,7,12,14,18,22,24,33,34,37,38,39,40,41],v_0:12,va:[2,40,41],vahid:32,val:[14,37,38],val_acc:[40,41],val_accuraci:4,val_error:[40,41],val_loss:5,val_set:[40,41],vale:[3,41],valid:[0,1,2,5,8,10,11,14,22,24,27,29,32,33,37,38,39,40,41],validation_data:4,validation_split:5,valu:[0,1,2,3,4,5,7,8,9,10,11,13,14,15,16,17,20,22,23,24,25,26,32,37,38,39,40,41],valuat:10,valued_at_a:[40,41],valued_at_z:[40,41],valueerror:3,valy:5,van:[1,19,26,32,33,34,35],vandenbergh:[9,14,36,37],vandermond:[1,32],vanilla:[1,7,12,15,33,34],vanish:[2,5,14,29,36,37],var_x:29,varabl:9,varepsilon:[6,7,19,26,34,35],varepsilon_:[6,7,34,35],varepsilon_i:[6,7,34,35],vari:[1,2,4,6,7,11,16,17,23,32,35,39,40,41],variabl:[1,2,3,6,7,8,9,11,12,13,14,15,22,23,25,32,33,34,35,38,39,40,41],varianc:[1,2,6,8,10,11,12,14,15,19,20,22,24,25,29,32,33,36,37,38,39,40],variance_i:[6,12,33],variance_x:[6,12,33],variant:[1,2,7,9,13,14,32,33,36,37,38,39,40,41],variat:[4,5,12,32],varieti:[1,4,13,16,24,26,32,38,39],variou:[0,2,4,6,7,8,9,10,12,13,14,17,19,22,24,25,26,29,33,34,37,38,39,40,41],varydimens:5,vastli:4,vaue:[2,40,41],vault:1,vdot:[3,14,36,37,41],ve:[26,27],vec:[7,35],vector:[0,1,2,3,4,5,6,7,8,10,11,12,14,15,17,18,22,23,24,34,35,36,37,39,40,41],vector_mean:15,ventur:[1,9,16,24,32],verbos:[2,4,5,40,41],veri:[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,23,26,27,29,31,32,33,34,35,36,37,38,39,40,41],verifi:[4,12,25,32],versatil:[9,32],versicolor:[9,10],version:[1,4,11,14,15,16,23,24,25,26,27,29,32,37],versu:[2,39,40],vert:[0,1,2,6,7,8,9,10,12,14,18,32,33,34,35,36,37,39,40,41],vert_1:[6,7,33,34,35],vert_2:[6,7,12,18,33,34,35],vi:[40,41],via:[0,1,6,7,8,9,10,11,12,13,16,20,24,25,26,28,29,30,32,33,34,35,36,38,39,40,41],vidal:12,video:[1,2,13,24,28,30,32,33,34,35,39,40,41],view:[2,4,6,7,13,14,22,29,31,32,34,35,37,38,40],vii:[40,41],viii:[40,41],violat:9,virginica:10,viridi:[1,2,3,4,32,40,41],virtual:[2,39,40],viscos:[14,37,38],viscou:[14,37,38],visibledeprecationwarn:[],vision:[1,4],visual:[1,4,12,13,16,24,32,33,38,39],visualis:[2,40,41],viz:[7,9,29],vjp:[],vjp_0:[],vjp_0_fun:[],vjp_1:[],vjp_1_fun:[],vjp_argnum:[],vjpfun:[],vjpmaker:[],vjpnode:[],vjps_dict:[],vmap:[14,37],vmax:[2,7,40,41],vmh0zpt0tli:38,vmin:[2,7,40,41],voic:4,volum:[1,4,32],volume18:41,von:[39,40,41],vote:11,voting_clf:11,votingclassifi:11,votingsimpl:11,vs:[1,5,7,37],vscode:23,vspace:[],vstack:[6,12,25,29,32,33,40,41],vt:[6,33,34],w1:[9,23],w2:[9,12,23],w3:9,w:[0,1,2,3,4,5,6,7,8,9,11,12,13,14,15,23,25,29,32,33,35,36,37,38,39,40,41],w_0:39,w_1:[9,25,39,40],w_1a_0:[39,40],w_1x:[39,40],w_1x_1:9,w_1x_:9,w_2:[9,25,39,40],w_2a_1:[39,40],w_2x_2:9,w_2x_:9,w_3:25,w_4:25,w_:[2,13,38,39,40,41],w_g:23,w_hidden:[3,41],w_i:[2,3,11,39,40,41],w_ix_i:[13,38,39],w_j:25,w_m:25,w_output:[3,41],w_px_:9,w_px_p:9,wa:[1,2,3,4,5,6,7,8,11,12,13,15,18,23,25,26,32,33,34,35,36,38,39,40,41],wai:[1,2,3,4,5,6,7,8,9,11,12,13,14,15,18,23,25,27,29,32,33,34,36,37,40,41],walk:10,walker:29,wang:[1,32],want:[1,2,3,4,5,6,7,9,10,11,12,13,14,15,16,17,18,23,24,26,27,29,32,33,34,35,36,37,38,39,40,41],warn:[2,5,9,40,41],warrant:[7,35,36],wast:[4,37,38],watch:[24,38,39,40,41],wave:4,wavelet:9,we:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,23,24,25,26,27,28,29,30,31,33,34,35,36],weak:[10,11,15],weather:[2,13,38,39,40,41],web:[24,28,30,32],weblink:27,webpag:32,websit:[7,25,26,28,32],wedg:[9,29,39,40],wednesdai:[30,32,35,36,37,41],wee:12,week:[1,6,7,8,26,27,28,30],weekli:[17,24,26,30,31,32,38],weierstrass:39,weight:[0,1,2,3,4,7,8,10,11,13,14,23,26,27,29,36,37,38,39],weight_arrai:[40,41],weigth:[3,41],welchlab:[38,39,40],welcom:[9,24],well:[0,1,2,3,4,5,6,7,8,9,10,11,13,14,16,17,20,21,22,23,24,25,26,27,29,31,32,33,34,35,36,37,38,39,40,41],went:9,were:[1,2,4,5,6,7,8,9,11,12,13,15,29,32,33,35,36,38,39,40,41],wessel:[1,19,26,32,33,34,35],what:[2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,20,23,24,25,26,27,29,36,37,38,39,40,41],whatev:[4,23],when:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,17,18,19,22,23,25,26,27,29,32,33,34,35,36,39,40,41],whenev:[14,22,29,37,38,39],where:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,22,23,24,25,26,27,29,30,32,33,34,35,36,37,38,39,40,41],wherea:[7,29,35],wherefrom:26,wherein:[2,13,38,39,40,41],whether:[1,4,6,8,10,26,27,29,32,34,36],which:[0,1,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,23,24,25,26,27,28,29,30,32,33,34,35,36],whichev:[2,4,40,41],white:10,whiteboard:[34,35,36,37,38,39,40],who:[1,28],whole:[2,4,5,6,10,12,14,23,34,35,37,38,40,41],whose:[1,7,11,29,33,35],whow:[12,33],why:[1,2,4,7,14,22,23,26,34,36,37,40],wide:[0,1,2,4,7,8,13,16,24,25,26,32,35,36,38,39,40,41],widehat:[7,35],width:[1,4,9,10,23,32],wieringen:[1,19,26,32,33,34,35],win:11,wind:10,wing:[30,32],winther:[3,41],wiothout:[7,33],wiscons:[0,8,36],wisconsin:[11,27,40,41],wisdom:[7,34],wise:[1,2,6,13,14,23,33,37,38,40,41],wish:[0,1,3,6,8,9,12,14,15,25,26,27,32,33,36,37,39,40,41],with_std:[1,33],wither:7,within:[0,1,3,4,5,8,10,13,14,15,23,29,31,32,36,37,41],withinclust:15,without:[1,2,6,7,9,10,12,13,14,22,27,32,33,34,35,37,38,39,40,41],wo5dmep_bbi:[38,39,40],won:[1,32,39],wonder:9,word:[1,2,4,5,6,7,8,15,29,32,33,34,35,40,41],work:[1,2,5,7,8,9,10,14,16,22,23,24,26,27,28,29,30,32,33,35,36,37,38,39,40,41],workshop:32,world:[1,9,33],worldwid:[1,32],wors:[1,2,4,5,7,32,35,39,40],worst:10,worth:[10,23],worthi:[26,27],would:[1,2,4,6,7,8,9,10,11,12,13,14,25,26,27,29,32,33,34,35,36,37,38,39,40,41],wrap:[7,25,32],wrap_util:[],wrapper:23,write:[1,2,3,4,6,7,8,9,13,14,16,17,18,21,22,23,25,32,33,35,36,37,38,39,41],written:[1,3,4,6,12,13,14,17,24,25,26,27,29,32,33,34,35,36,37,38,39,40,41],wrong:[2,9,39,40,41],wrongli:11,wrote:[6,12,33],wrt:[11,14,22,23,37,38,39,40],wth:[11,14,22,37,38],wurstemberg:[39,40,41],www:[24,25,26,27,31,32,34,38,39,40,41],wx_1:9,x0:9,x1:[5,9,10,11,14,37,38],x1_exampl:9,x1d:9,x2:[9,10,11,14,37,38],x2d:[9,12],x2d_train:12,x2dsl:12,x3:[9,10],x4:10,x:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,22,23,25,26,27,29,32,35,36,37,38,39,40,41],x_0:[1,6,12,25,32,33,34,35,39],x_1:[0,1,3,6,7,8,9,10,11,12,14,25,29,32,33,34,35,36,37,38,39,40,41],x_2:[0,1,3,6,7,8,9,10,11,12,14,25,29,32,33,35,36,37,38,39,40,41],x_3:[9,25,29,39],x_4:[25,39],x_5:39,x_:[1,3,4,6,7,9,11,12,14,15,19,25,26,29,32,33,34,35,36,37,39,41],x_batch:[40,41],x_center:12,x_data:[2,40,41],x_data_ful:[2,40,41],x_hidden:[3,41],x_i:[0,1,2,3,6,7,8,9,10,11,12,13,14,15,18,25,29,32,33,34,35,36,37,38,39,40,41],x_input:[3,41],x_ix_:[1,32],x_iy_i:9,x_j:[1,3,9,10,13,17,29,33,38,39,41],x_jy_j:9,x_k:[13,15,25,29,33,38],x_l:[29,39],x_m:[7,13,25,29,35,38],x_n:[1,3,4,7,9,12,13,14,25,29,32,35,36,37,38,39,41],x_new:[10,11],x_offset:[7,33,34],x_output:[3,41],x_p:[0,4,8,10,36],x_poli:10,x_poly10:10,x_pred:5,x_prev:[3,41],x_reduc:12,x_scale:9,x_small:[14,37],x_test:[0,1,2,4,6,7,8,10,11,12,32,33,34,35,36,40,41],x_test_own:7,x_test_scal:[1,7,8,10,11,12,33,34],x_tot:5,x_train:[0,1,2,4,5,6,7,8,10,11,12,32,33,34,35,36,40,41],x_train_mean:[7,33,34],x_train_own:7,x_train_scal:[1,7,8,10,11,12,33,34],x_val:[2,40,41],xarrai:[24,32],xavier:[2,39,40],xbkjd4ccgq:32,xbnew:[14,36,37],xcode:[1,16,24,26,32],xdclassiffierconfus:11,xdclassiffierroc:11,xfin:23,xg_clf:11,xgb:11,xgbclassifi:11,xgboost:10,xgboot:11,xgbregressor:11,xgparam:11,xgtree:11,xi:[9,14,22,37,38],xi_1:9,xi_:9,xi_i:9,xinv:[34,38,39],xk:9,xla:[14,22,24,32,37,38],xlabel:[1,2,3,4,5,6,7,8,9,10,11,14,22,23,26,29,32,34,35,36,37,38,40,41],xlim:[7,11,35],xm:10,xmesh:[14,37],xnew:[1,14,22,32,36,37,38],xp:29,xpanda:[1,33],xpd:[6,12,33],xplot:1,xs:10,xscale:[1,33],xsr:10,xt_x:[14,22,36,37,38],xtest:[7,35,36],xtick:[4,7,9,10,35],xtrain:[7,35,36],xu:[1,32],xx:[1,25,32],xy:[1,7,9,25,26,32],xytext:9,xz:[25,32],y1:5,y2:5,y3:5,y:[0,1,2,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,22,23,25,26,29,32,33,34,35,36,37,38,39,40,41],y_0:[1,6,12,25,32,33,34,35],y_1:[1,6,9,10,12,14,25,32,33,34,35,36,37],y_1y_1:9,y_1y_1k:9,y_1y_2:9,y_1y_2k:9,y_1y_n:9,y_1y_nk:9,y_2:[1,6,9,10,12,25,32,33],y_2y_1:9,y_2y_1k:9,y_2y_2:9,y_2y_2k:9,y_3:[1,10,25],y_4:25,y_:[1,2,6,7,11,12,25,32,33,34,35,40,41],y_data:[1,2,6,7,32,33,34,36,40,41],y_data_ful:[2,40,41],y_decis:9,y_fit:[1,33],y_i:[0,1,2,6,7,8,9,10,11,12,13,14,16,17,18,19,20,25,26,27,32,33,34,35,36,37,38,39,40,41],y_if_:11,y_ix_:[1,32],y_ix_i:[0,8,9,14,33,36,37],y_iy_jk:9,y_j:[7,9,13,20,26,35,38,39,40],y_k:[13,38],y_m:25,y_model:[1,5,6,7,32,33,34,36],y_n:[9,14,36,37],y_ny_1:9,y_ny_1k:9,y_ny_2:9,y_ny_2k:9,y_ny_n:9,y_ny_nk:9,y_offset:[7,33,34],y_plot:10,y_pred1:10,y_pred2:10,y_pred:[0,1,2,5,7,8,9,10,11,33,34,35,36,40,41],y_pred_rf:11,y_pred_tre:11,y_predict:[0,38],y_proba:[0,8,11,36],y_scaler:[7,34],y_test:[0,1,2,4,5,6,7,8,10,11,12,32,33,34,35,36,40,41],y_test_onehot:[2,40,41],y_test_predict:1,y_tot:5,y_train:[0,1,2,4,5,6,7,8,10,11,12,32,33,34,35,36,40,41],y_train_mean:[7,33,34],y_train_onehot:[2,40,41],y_train_predict:1,y_train_scal:[7,34],y_val:[2,40,41],yadav:41,yand:[38,39],ye:[4,7,8,35,36],year:[1,24,32],yet:[1,2,7,9,12,14,23,32,36,37,39,40,41],yfin:23,yi:[14,22,37,38],yield:[1,3,6,7,8,9,11,13,14,15,25,29,32,34,35,36,37,38,39,40,41],yk:9,ylabel:[1,2,3,4,5,6,7,8,9,10,11,14,22,23,26,29,32,34,35,36,37,38,40,41],ylim:[4,7,35],ym:10,ymesh:[14,37],yn:1,yo:[9,10,11],yor:[38,39],yoshua:[2,31,39,40],you:[1,2,4,5,6,7,9,10,11,12,14,16,17,18,20,21,22,23,24,25,26,27,29,30,31,32,33,34,35,36,37,38,39,40,41],young:1,your:[2,3,5,6,7,9,12,14,18,20,21,22,23,24,25,26,34,35,36,37,38,39,40,41],yourself:[12,14,32,37],youtu:[36,38],youtub:[24,38,39,40,41],ypred:[7,35,36],ypredict2:[14,22,36,37,38],ypredict:[1,14,22,32,33,36,37,38],ypredictlasso:[6,34],ypredictol:[1,6,34],ypredictown:[7,33,34],ypredictownridg:[7,34],ypredictridg:[1,6,7,34,36],ypredictskl:[7,33,34],yridg:32,ys:10,ytest:[7,35,36],ytick:[4,7,9,10,35],ytild:[1,7,32,33,35],ytildelasso:[6,34],ytildenp:[1,32,33],ytildeol:[1,6,34],ytildeownridg:[7,34],ytilderidg:[6,7,34],ytrain:[7,35,36],yuxi:32,yx:[25,32],yxor:[38,39,40,41],yy:[25,32],yz:[25,32],z1:23,z2:23,z:[0,1,2,3,4,5,6,7,8,9,10,12,13,14,23,25,26,29,32,33,35,36,37,38,39,40,41],z_0:[25,32,39],z_1:[25,32,39,40],z_2:[25,32,39,40],z_:[2,3,13,25,32,39,40,41],z_c:[2,40,41],z_h:[2,40,41],z_hidden:[3,41],z_i:[2,13,38,40,41],z_j:[2,13,41],z_k:[13,33,39,40],z_m:[2,40,41],z_matric:[40,41],z_mod:10,z_o:[2,40,41],z_output:[3,41],zaman:29,zaxi:[7,26],zero:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,18,19,20,22,23,25,26,29,32,33,34,35,36,37,38,39,40,41],zeros_lik:5,zeroth:33,zfill:5,zip:[5,7,23],zm_h:[1,32],zn:1,zone:1,zoom:32,zx:[25,32],zy:[25,32],zz:[25,32]},titles:["Exercises Week 42: Logistic Regression and Optimization, reminders from week 38 and week 40","3. Linear Regression","14. Building a Feed Forward Neural Network","15. Solving Differential Equations with Deep Learning","16. Convolutional Neural Networks","17. Recurrent neural networks: Overarching view","4. Ridge and Lasso Regression","5. Resampling Methods","6. Logistic Regression","8. Support Vector Machines, overarching aims","9. Decision trees, overarching aims","10. Ensemble Methods: From a Single Tree to Many Trees and Extreme Boosting, Meet the Jungle of Methods","11. Basic ideas of the Principal Component Analysis (PCA)","13. Neural networks","7. Optimization, the central part of any Machine Learning algortithm","12. Clustering and Unsupervised Learning","Exercises week 34","Exercises week 35","Exercises week 36","Exercises week 37","Exercises week 38","Exercises week 39","Exercises week 41","Exercises week 42","Applied Data Analysis and Machine Learning","2. Linear Algebra, Handling of Arrays and more Python Features","Project 1 on Machine Learning, deadline October 7 (midnight), 2024","Project 2 on Machine Learning, deadline November 4 (Midnight)","Teaching schedule with links to material","1. Elements of Probability Theory and Statistical Data Analysis","Teachers and Grading","Textbooks","Week 34: Introduction to the course, Logistics and Practicalities","Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression","Week 36: Linear Regression and Statistical interpretations","Week 37: Statistical interpretations and Resampling Methods","Week 38: Logistic Regression and Optimization","Week 39: Optimization and Gradient Methods","Week 40: Gradient descent methods (continued) and start Neural networks","Week 41 Neural networks and constructing a neural network code","Week 42 Constructing a Neural Network code with examples","Week 43: Deep Learning: Constructing a Neural Network code and solving differential equations"],titleterms:{"0":40,"1":[1,16,17,18,19,23,26,32,33,39,40,41],"11":39,"14":40,"16":36,"2":[1,16,17,18,19,23,27,32,33,34,39,40,41],"2023":30,"2024":[26,37,38,39,40],"21":41,"23":37,"26":33,"27":37,"3":[1,16,17,23,32,33,39,40,41],"30":38,"34":[16,32],"35":[17,33],"36":[18,34],"37":[19,35],"38":[0,20,36],"39":[21,22,37],"4":[1,23,27,33,40],"40":[0,22,38],"41":[22,39],"42":[0,23,40],"43":41,"5":[1,23],"6":23,"7":[23,26,39],"9":35,"case":[9,11,29,33,34,36,37],"class":[0,36,39],"do":[2,34,35,37,38,39,40,41],"final":[13,22,27,33,34,37,38,39,40,41],"float":39,"function":[0,1,2,7,8,9,11,12,13,14,22,26,27,29,32,33,34,35,36,37,38,39,40,41],"import":[6,22,25,32,33,34,38,39,40],"new":[5,34,35,39],A:[0,1,2,5,9,10,22,32,34,35,36,38,39,40,41],AND:[38,39],And:[22,32,33,34,36,37,38],But:[22,37,38],For:33,In:[30,39],Is:[39,40],Ising:7,OR:[38,39],The:[0,1,2,3,4,6,7,8,9,10,12,13,18,24,32,33,34,35,36,37,38,39,40,41],To:[32,33],With:[5,34],about:[32,33],abov:[34,39,40,41],activ:[2,13,23,27,34,38,39,40,41],ad:[1,7,18,26,32,33,38,39,40],adaboost:11,adagrad:[14,22,37,38],adam:[14,22,37,38],adapt:[11,22,37,38],adjust:[2,40,41],advanc:[22,38],adversari:5,again:[0,4,10,36],ai:[26,32],aim:[9,10,18,19,20,21,22,23,32],aka:[32,33],al:[22,38],algebra:[25,32],algorithm:[10,11,12,13,22,27,33,37,38,39,40,41],algortithm:[14,36,37],all:[9,39,40],an:[1,5,11,32,39],analys:[6,33],analysi:[1,6,7,12,24,26,27,29,32,33,34,35,39],analyt:[1,17,18,22,41],analyz:[39,40],ani:[14,36,37],anoth:[10,34,35],appli:24,approach:[1,9,15,32,35,37,38],approxim:[13,39],architectur:[2,40,41],argument:[37,38],arrai:[25,32],artifici:[38,39],assist:30,assumpt:[34,35],august:33,autocorrel:29,autograd:[3,14,22,37,38,41],automat:[14,22,37,38,39,41],avoid:[],b:[18,26,27,37],back:[2,12,13,39,40,41],background:[24,26,27,35],bag:11,base:[14,22,35,37,38],basic:[1,6,8,10,11,12,25,33,34,35,36,39],batch:[2,37,38,39,40],bay:[6,34,35],befor:12,bengio:[39,40],beta:[34,35],better:[9,38,39],bia:[7,26,35],bias:[39,40,41],binari:[2,40,41],bind:32,bird:11,boldsymbol:[33,34,35],book:[39,40,41],boost:11,bootstrap:[7,11,35],boston:1,breast:[2,40,41],brief:[32,35,36,37],bring:[13,39,40],build:[2,4,10,40,41],c:[26,27,32],calcul:33,can:[22,32,35,37,38,39],cancer:[0,2,8,10,12,36,40,41],cart:10,center:33,central:[14,24,29,35,36,37],chain:[13,39,40],challeng:36,chang:11,channel:32,chi:[1,32],choic:[39,40],choos:[2,40,41],cifar01:4,classic:12,classif:[0,2,10,11,27,36,40,41],classifi:[9,36],clip:[2,39,40],cluster:15,cnn:4,code:[1,2,3,6,10,12,13,14,15,22,27,32,33,34,35,36,37,38,39,40,41],collect:[2,4,40,41],come:36,commun:32,compact:[0,36,39,40],compar:[3,11,41],comparison:34,compet:[22,37,38],complet:[33,39,40],complex:[1,7,26,33],complic:[7,37,38,39],compon:12,comput:[10,37,38],computation:35,computerlab:32,con:10,concept:29,condit:[34,35,36,37],confid:35,conjug:[14,37],consider:[39,40],construct:[39,40,41],continu:38,contn:32,convex:[9,14,36,37],convolut:[4,13,38,39],correctli:[34,35],correl:[0,12,33,36],correspond:[36,37],cost:[0,2,11,33,34,35,36,37,39,40,41],count:39,cours:[24,31,32],covari:[6,12,29,33],cover:32,critic:27,cross:[0,7,26,35,36],custom:23,cython:32,d:[26,27],data:[0,1,2,4,7,8,10,12,16,17,23,24,26,29,32,33,34,36,39,40,41],dataset:[2,4,40,41],deadlin:[26,27,32],decai:[3,37,38,41],decis:[10,11],decomposit:[6,12,18,25,33],deep:[2,3,32,36,39,40,41],defin:[2,32,39,40,41],definit:[39,40],degre:[1,33],deliveri:[26,27],delta:35,dens:[1,32],deriv:[6,13,33,34,35,36,37,39,40],descent:[0,3,11,14,22,27,36,37,38,41],descript:26,design:33,detail:[4,32,41],develop:[2,40,41],diagon:12,differ:[9,27,37,38],different:22,differenti:[3,14,37,38,39,41],diffus:[3,41],dimension:[3,4,9,26,33,41],directli:[37,38],disadvantag:10,discret:29,discuss:[0,36],distribut:[6,29,34,35],distrubut:35,doe:[33,34,38,39],domain:29,dot:37,down:[2,39,40],dropout:[2,39,40],e:[26,27],each:[23,36],economi:33,electron:[26,27],element:[1,29,32,37,38],elimin:25,elu:[39,40,41],energi:32,ensembl:11,entri:[39,40],entropi:[0,10,36],environ:[1,16,32],equat:[1,3,13,32,33,34,36,37,39,40,41],error:[1,11,32,33,35],essenti:32,estim:[34,35],et:[22,38],etc:32,euler:[3,41],evalu:[2,27,39,40,41],exampl:[0,1,2,3,4,5,7,8,9,10,11,22,32,33,34,35,36,37,38,39,40,41],exercis:[0,1,7,16,17,18,19,20,21,22,23,32,33,39,41],expect:[19,29,34,35],expens:35,experi:29,explicit:[39,40],explod:[39,40],explor:[1,16,17,32],exponenti:[3,41],express:[0,18,19,33,36,37,39,40],extend:[0,36,37,39],extrapol:5,extrem:[11,32],ey:11,f:[26,27],fall:30,famili:[2,32,33,39,40,41],famou:25,fantast:33,featur:[10,25,33],feed:[2,13,38,39,40,41],find:[35,37],fine:[2,39,40],first:[5,13,27,32,33,34,36,37,39,40,41],fit:[1,11,32,34],fix:33,fold:[35,36],forc:4,forest:11,format:[26,27,32],forward:[2,3,13,38,39,40,41],fourier:4,frank:[7,26,33],freedom:[1,33],frequent:33,frequentist:[1,32],fridai:[],from:[0,6,11,13,22,27,33,34,35,36,37,38,39,40],full:[3,40,41],funtion:[39,40],further:[4,6,33],g:26,gan:5,gate:[38,39,40,41],gaussian:25,gd:[14,22,37,38],gener:[5,10,32,39],geometr:[12,36,37],get:[22,38,39],gini:10,glorot:[39,40],good:[1,32],goodfellow:[22,38],grade:[30,32],gradient:[0,2,3,11,14,22,27,36,37,38,39,40,41],grid:36,group:36,growth:[3,41],ha:24,hand:[39,40,41],handl:[25,32,33],happen:[34,35],hard:[],hessian:[33,36,37],hidden:[3,39,40,41],histogram:35,homework:[36,37],homogen:[39,40],hous:1,how:36,hyperbol:[38,39,40],hyperparamet:[2,39,40,41],hyperplan:9,i:[2,41],id3:10,idea:12,ideal:[36,37],ident:[34,35],identifi:35,ii:[32,41],iid:[34,35],iii:41,illustr:[34,38,39],implement:[2,38,40,41],implic:[6,33],improv:[2,37,40,41],includ:[0,14,22,36,37,38,39],increment:12,independ:[34,35],index:10,inform:30,ingredi:39,init:[40,41],input:[3,23,39,40,41],insight:[39,40],instal:[24,26,32],instructor:30,intercept:33,intermedi:39,interpret:[6,12,32,33,34,35,36,37],interv:35,introduc:[12,14,22,33,37,38],introduct:[1,7,24,25,26,27,32,38,39],invers:[6,25,34],invert:33,iter:[11,37],its:33,iv:41,jacobian:[33,41],jax:[14,22,37,38],job:[38,39],julia:32,jungl:11,k:[35,36,39,40],kera:[2,4,40,41],kernel:[9,12],l:[39,40],lab:[35,36,37,38,41],lagrangian:9,lambda:36,lasso:[6,7,26,33,34,35,36,38],last:[33,36,38,39,40],later:[6,33,40],layer:[2,3,4,13,23,39,40,41],layout:[39,40],learn:[1,2,3,12,14,15,16,17,22,24,26,27,32,33,34,35,36,37,38,39,40,41],least:[6,7,19,26,32,33,34],lectur:[32,33,34,35,36,37,38,39,40,41],level:11,librari:[24,32],likelihood:[0,8,34,35,36],limit:[2,14,29,35,36,37,38,39,40],linear:[1,9,14,25,27,32,33,34,36],link:[6,12,28,31,33,34,35],list:[39,40],literatur:[26,27],logist:[0,8,27,32,36,37,38,39,40],loop:[37,38],loss:[33,36,37],lu:25,machin:[1,9,14,24,26,27,32,36,37],made:[34,35],main:[29,32],make:[1,10,11,16,17,32,33],mani:[11,13],manipul:33,margin:[34,35],mass:32,materi:[26,28,32,33,34,35,36,39,40,41],math:[6,33],mathemat:[4,6,9,33,37,38,39,40,41],matric:[6,25,32,34],matrix:[0,2,6,12,13,25,32,33,34,36,37,38,40,41],matter:[1,32],max:33,maximum:[34,35,36],mean:[1,33,34,36],measur:[0,36],meet:[6,11,29,32,33],mercer:9,method:[7,10,11,14,22,26,32,35,36,37,38,40,41],midnight:[26,27],min:33,mini:[37,38],minibatch:[22,37,38],minim:[0,32,36,41],ml:32,mle:[34,35],mlp:13,mnist:[4,5],mode:39,model:[1,2,5,7,13,32,38,39,40,41],moment:[37,38],momentum:[14,22,37,38],mondai:[33,34,35,36,37,38,39,41],moon:[9,10],more:[0,4,7,22,25,26,32,33,34,35,36,37,38,39,40,41],multi:[38,39,40,41],multiclass:[40,41],multilay:[13,38,39],multipl:[2,4,23,40,41],multipli:9,multivari:39,need:[26,32],network:[0,2,3,4,5,8,13,27,32,36,38,39,40,41],neural:[2,3,4,5,8,13,27,32,38,39,40,41],neuron:[38,39],newton:[22,36,37,38],nn:[39,40,41],node:[39,40],noeds:40,noen:22,non:9,none:[37,38],normal:[1,2,35,39,40],notat:[13,38],note:[22,26,33,34,38],novemb:27,now:[2,10,14,22,34,35,37,38,40,41],nuclear:[1,32],nueral:[0,36],numba:32,number:[1,3,22,29,33,37,38,39,41],numer:[3,26,27,29,41],numpi:[22,25,32,37,38],object:[4,40,41],observ:[39,40],obtain:12,octob:[26,39,40,41],od:[3,41],off:[7,26],ol:[6,7,22,26,34,35,37,38],onc:23,one:[3,13,36,37,39,40,41],ones:[38,39,40],oper:[25,39],optim:[0,2,9,14,24,32,33,36,37,38,39,40,41],option:23,order:[14,22,37,38],ordinari:[6,7,19,26,32,33,34,41],organ:[1,32],orient:[40,41],oslo:31,other:[0,5,10,12,13,25,26,32,33,36,38,39,40,41],ouput:[39,40],our:[1,5,6,12,14,32,33,36,37,39,40],outcom:[24,32],output:[3,39,40,41],over:[39,40],overarch:[1,5,9,10,18,19,20,21,22,23,32,33,39],overview:[11,32,37,38],own:[1,11,12,16,17,27,32,33],packag:[25,32],panda:[32,33],paper:[26,39,40],parallel:39,paramet:[0,32,33,36,37,38,39,40],part:[14,24,26,27,33,36,37,39,40,41],partial:[3,41],pass:[2,40,41],pca:12,pdf:29,pencil:26,percepetron:[39,40],perceptron:[13,38,39,40,41],perform:[2,10,40,41],period:4,perspect:[2,39,40],plan:[33,34,35,36,37,38,39,41],plot:[35,36],point:[5,39],poisson:[3,41],polynomi:[4,34],popul:[3,41],popular:32,possibl:41,practic:[14,22,30,32,37,38],pre:[2,4,40,41],preambl:26,predict:[5,23],predictor:[0,36],preprocess:33,prerequisit:[4,24,32],princip:12,principl:4,pro:10,probabl:[6,29,34,35],problem:[2,3,14,22,32,33,34,36,37,38,39,40,41],procedur:[10,32],process:[2,4,23,40,41],product:37,program:[3,14,26,27,36,37,39,41],project:[7,26,27,32,36],prop:[14,37,38],propag:[2,13,39,40,41],properti:[6,29,33,36],python:[1,10,16,24,25,32],quick:9,r:32,random:[11,12,29,36],raphson:[36,37],rate:[22,37,38,40,41],read:[10,32,33,35,36,38,39,40,41],real:[7,23,26,32,39],recommend:[32,33,39,40,41],rectangular:34,recurr:[5,13,38,39],recurs:[37,38],reduc:[1,33,39],reduct:4,refer:26,reformul:[3,41],regress:[0,1,6,7,8,10,11,14,18,19,26,27,32,33,34,35,36,37,38,39],regular:[2,36,40,41],relat:33,relev:[31,33,36,38,39,40],relu:[2,39,40,41],remark:4,remind:[0,7,9,32,36,37,39,40,41],repeat:33,replac:[14,37,38],report:[26,27],repositori:35,repres:[],requir:[3,24,41],resampl:[7,26,35],rescal:[7,34],residu:33,resourc:[3,41],result:[33,34,39,40],revers:39,revisit:[14,36,37],rewrit:[32,33,35],rewritten:36,ridg:[1,6,7,18,19,26,33,34,35,36,37,38],rm:[14,37,38],rmsprop:[22,37,38],root:[39,40],rule:[13,39,40],s:[9,11,22,35,36,37,38],same:[14,22,35,37,38],sampl:12,scale:33,schedul:[28,32,40,41],schemat:10,scheme:[3,41],scienc:32,scikit:[1,2,12,16,17,32,33,34,35,36,37,40,41],search:36,second:[14,22,37,38],select:36,semest:30,sensit:[36,37],septemb:[34,35,36,37,38],seriou:39,session:[33,34,35,36,37,38,39,40,41],set:[1,3,4,10,13,16,32,33,36,39,40,41],setup:41,sgd:[14,37,38],should:[2,39,40,41],sigmoid:[39,40],similar:[14,22,37,38],simpl:[1,5,10,14,32,33,34,36,37,38,39,40],simpler:39,singl:[11,38,39],singular:[6,12,18,33],size:33,slightli:[37,38],smarter:39,soft:9,softmax:[2,40,41],softwar:[26,32],solut:41,solv:[3,34,36,37,41],solver:14,some:[14,25,33,36,37,39],specif:41,specifi:[3,41],split:[1,16,17,32,33],squar:[1,6,7,11,19,26,32,33,34],standard:[14,33,35,37],start:[22,38],state:[1,32],statist:[6,7,24,29,32,34,35],steepest:[11,14,36,37],step:[27,35,36,37,38],still:33,stochast:[14,22,27,29,37,38],stop:[37,38],strongli:32,studi:[0,36],subtract:33,suggest:[32,36,38],sum:[35,39,40],summari:[30,32,38],superposit:4,supervis:[2,39,40],support:9,svd:[6,33,34],syntax:37,systemat:4,t:[33,34],taken:[22,38],teach:[28,30],teacher:[30,32],technic:[34,41],techniqu:[7,12,26],technolog:24,tensorflow:[2,4,40,41],tent:32,term:[35,39,40],test:[1,2,16,17,27,32,33,34,40,41],text:32,textbook:[31,32],than:[36,37],theorem:[6,9,12,13,29,34,35,39],theori:29,thi:[18,19,20,21,22,23,32,39],think:33,three:[39,40],through:39,thursdai:[],time:[37,38],tip:[14,22,37,38],togeth:[13,39,40],tool:[26,32],top:[2,39,40],topic:32,toward:12,trade:[7,26],tradeoff:[7,35],train:[1,2,5,16,17,23,32,33,39,40,41],transform:4,tree:[10,11],trial:41,tuesdai:[34,38,39,40],tune:[2,39,40],two:[0,4,9,24,26,33,36,39,40],type:[3,5,13,32,38,39,41],uio:32,understand:35,univers:[13,31,39],unsupervis:15,unsupport:37,up:[1,3,10,13,16,22,32,33,35,36,38,39,40,41],updat:[39,40,41],us:[0,1,2,3,4,8,14,22,24,26,27,32,33,36,37,38,39,40,41],usag:[34,35,40,41],valid:[7,26,35,36],valu:[6,12,18,19,29,33,34,35,36],vanish:[39,40],vari:[22,37,38],variabl:[29,36,37],varianc:[7,26,34,35],variou:[1,16,27,32,35,36],vector:[9,13,25,32,33,38],veri:[],video:[36,37,38],view:[1,5,11,33,39],visual:[2,10,40,41],vs:4,wai:[10,22,35,38,39],warm:[22,38],wave:[3,41],we:[22,32,37,38,39,40,41],wednesdai:[34,38,39,40],week:[0,16,17,18,19,20,21,22,23,32,33,34,35,36,37,38,39,40,41],weekend:36,weekli:28,weight:[40,41],what:[1,32,33,34,35],when:[37,38],which:[2,22,37,38,39,40,41],why:[32,33,35,38,39,41],wisconsin:[0,8,36],word:39,wrap:[33,35],write:[5,12,26,27,34,40],x:[33,34],xgboost:11,xor:[38,39,40,41],yet:34,your:[1,11,16,17,27,32,33],yourself:36,z_j:[39,40]}}) \ No newline at end of file diff --git a/doc/LectureNotes/_build/html/week43.html b/doc/LectureNotes/_build/html/week43.html index 8c52d4663..acc647cab 100644 --- a/doc/LectureNotes/_build/html/week43.html +++ b/doc/LectureNotes/_build/html/week43.html @@ -711,13 +711,13 @@ const thebe_selector_output = ".output, .cell_output"
  • - - Solving ODEs with Deep Learning + + Solving differential equations with Deep Learning
  • - - Ordinary Differential Equations + + Ordinary Differential Equations first
  • @@ -1250,13 +1250,13 @@ const thebe_selector_output = ".output, .cell_output"
  • - - Solving ODEs with Deep Learning + + Solving differential equations with Deep Learning
  • - - Ordinary Differential Equations + + Ordinary Differential Equations first
  • @@ -2346,7 +2346,7 @@ the Hadamard product, meaning element-wise multiplication.

    Old accuracy on training data: 0.1440501043841336
    -
    /var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp
+
    /var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/953065564.py:4: RuntimeWarning: overflow encountered in exp
   return 1/(1 + np.exp(-x))
 
    
 
    
@@ -2682,7 +2682,7 @@ Lambda =  10.0
 Accuracy score on test set:  0.19166666666666668
    -
    /var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp
+
    /var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/953065564.py:4: RuntimeWarning: overflow encountered in exp
   return 1/(1 + np.exp(-x))
 
    
 
    
@@ -2691,7 +2691,7 @@ Lambda =  1e-05
 Accuracy score on test set:  0.10555555555555556
    -
    /var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp
+
    /var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/953065564.py:4: RuntimeWarning: overflow encountered in exp
   return 1/(1 + np.exp(-x))
 
    
 
    
@@ -2700,7 +2700,7 @@ Lambda =  0.0001
 Accuracy score on test set:  0.08611111111111111
    -
    /var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp
+
    /var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/953065564.py:4: RuntimeWarning: overflow encountered in exp
   return 1/(1 + np.exp(-x))
 
    
 
    
@@ -2709,7 +2709,7 @@ Lambda =  0.001
 Accuracy score on test set:  0.10555555555555556
    -
    /var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp
+
    /var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/953065564.py:4: RuntimeWarning: overflow encountered in exp
   return 1/(1 + np.exp(-x))
 
    
 
    
@@ -2718,7 +2718,7 @@ Lambda =  0.01
 Accuracy score on test set:  0.08888888888888889
    -
    /var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp
+
    /var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/953065564.py:4: RuntimeWarning: overflow encountered in exp
   return 1/(1 + np.exp(-x))
 
    
 
    
@@ -2727,7 +2727,7 @@ Lambda =  0.1
 Accuracy score on test set:  0.08611111111111111
    -
    /var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp
+
    /var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/953065564.py:4: RuntimeWarning: overflow encountered in exp
   return 1/(1 + np.exp(-x))
 
    
 
    
@@ -2736,7 +2736,7 @@ Lambda =  1.0
 Accuracy score on test set:  0.08888888888888889
    -
    /var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp
+
    /var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/953065564.py:4: RuntimeWarning: overflow encountered in exp
   return 1/(1 + np.exp(-x))
 
    
 
    
@@ -2745,11 +2745,11 @@ Lambda =  10.0
 Accuracy score on test set:  0.09166666666666666
    -
    /var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp
+
    /var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/953065564.py:4: RuntimeWarning: overflow encountered in exp
   return 1/(1 + np.exp(-x))
-/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:43: RuntimeWarning: overflow encountered in exp
+/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/1630775253.py:43: RuntimeWarning: overflow encountered in exp
   exp_term = np.exp(self.z_o)
-/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:44: RuntimeWarning: invalid value encountered in divide
+/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/1630775253.py:44: RuntimeWarning: invalid value encountered in divide
   self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
 
    
 
    
@@ -2758,11 +2758,11 @@ Lambda =  1e-05
 Accuracy score on test set:  0.07777777777777778
    -
    /var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp
+
    /var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/953065564.py:4: RuntimeWarning: overflow encountered in exp
   return 1/(1 + np.exp(-x))
-/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:43: RuntimeWarning: overflow encountered in exp
+/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/1630775253.py:43: RuntimeWarning: overflow encountered in exp
   exp_term = np.exp(self.z_o)
-/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:44: RuntimeWarning: invalid value encountered in divide
+/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/1630775253.py:44: RuntimeWarning: invalid value encountered in divide
   self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
 
    
 
    
@@ -2771,11 +2771,11 @@ Lambda =  0.0001
 Accuracy score on test set:  0.07777777777777778
    -
    /var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp
+
    /var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/953065564.py:4: RuntimeWarning: overflow encountered in exp
   return 1/(1 + np.exp(-x))
-/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:43: RuntimeWarning: overflow encountered in exp
+/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/1630775253.py:43: RuntimeWarning: overflow encountered in exp
   exp_term = np.exp(self.z_o)
-/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:44: RuntimeWarning: invalid value encountered in divide
+/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/1630775253.py:44: RuntimeWarning: invalid value encountered in divide
   self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
 
    
 
    
@@ -2784,11 +2784,11 @@ Lambda =  0.001
 Accuracy score on test set:  0.07777777777777778
    -
    /var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp
+
    /var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/953065564.py:4: RuntimeWarning: overflow encountered in exp
   return 1/(1 + np.exp(-x))
-/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:43: RuntimeWarning: overflow encountered in exp
+/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/1630775253.py:43: RuntimeWarning: overflow encountered in exp
   exp_term = np.exp(self.z_o)
-/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:44: RuntimeWarning: invalid value encountered in divide
+/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/1630775253.py:44: RuntimeWarning: invalid value encountered in divide
   self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
 
    
 
    
@@ -2797,130 +2797,37 @@ Lambda =  0.01
 Accuracy score on test set:  0.07777777777777778
    -
    /var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp
+
    /var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/953065564.py:4: RuntimeWarning: overflow encountered in exp
   return 1/(1 + np.exp(-x))
-/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:43: RuntimeWarning: overflow encountered in exp
+/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/1630775253.py:43: RuntimeWarning: overflow encountered in exp
   exp_term = np.exp(self.z_o)
-/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:44: RuntimeWarning: invalid value encountered in divide
+/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/1630775253.py:44: RuntimeWarning: invalid value encountered in divide
   self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
 
    
 
    
-
    Learning rate  =  1.0
-Lambda =  0.1
-Accuracy score on test set:  0.07777777777777778
-
    
-
    
-
    /var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp
-  return 1/(1 + np.exp(-x))
-
    
-
    
-
    Learning rate  =  1.0
-Lambda =  1.0
-Accuracy score on test set:  0.10555555555555556
-
    
-
    
-
    /var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp
-  return 1/(1 + np.exp(-x))
-/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:43: RuntimeWarning: overflow encountered in exp
-  exp_term = np.exp(self.z_o)
-/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:44: RuntimeWarning: invalid value encountered in divide
-  self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
-
    
-
    
-
    Learning rate  =  1.0
-Lambda =  10.0
-Accuracy score on test set:  0.07777777777777778
-
    
-
    
-
    /var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp
-  return 1/(1 + np.exp(-x))
-/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:43: RuntimeWarning: overflow encountered in exp
-  exp_term = np.exp(self.z_o)
-/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:44: RuntimeWarning: invalid value encountered in divide
-  self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
-
    
-
    
-
    Learning rate  =  10.0
-Lambda =  1e-05
-Accuracy score on test set:  0.07777777777777778
-
    
-
    
-
    /var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp
-  return 1/(1 + np.exp(-x))
-/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:43: RuntimeWarning: overflow encountered in exp
-  exp_term = np.exp(self.z_o)
-/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:44: RuntimeWarning: invalid value encountered in divide
-  self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
-
    
-
    
-
    Learning rate  =  10.0
-Lambda =  0.0001
-Accuracy score on test set:  0.07777777777777778
-
    
-
    
-
    /var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp
-  return 1/(1 + np.exp(-x))
-/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:43: RuntimeWarning: overflow encountered in exp
-  exp_term = np.exp(self.z_o)
-/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:44: RuntimeWarning: invalid value encountered in divide
-  self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
-
    
-
    
-
    Learning rate  =  10.0
-Lambda =  0.001
-Accuracy score on test set:  0.07777777777777778
-
    
-
    
-
    /var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp
-  return 1/(1 + np.exp(-x))
-/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:43: RuntimeWarning: overflow encountered in exp
-  exp_term = np.exp(self.z_o)
-/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:44: RuntimeWarning: invalid value encountered in divide
-  self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
-
    
-
    
-
    Learning rate  =  10.0
-Lambda =  0.01
-Accuracy score on test set:  0.07777777777777778
-
    
-
    
-
    /var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp
-  return 1/(1 + np.exp(-x))
-/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:43: RuntimeWarning: overflow encountered in exp
-  exp_term = np.exp(self.z_o)
-/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:44: RuntimeWarning: invalid value encountered in divide
-  self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
-
    
-
    
-
    Learning rate  =  10.0
-Lambda =  0.1
-Accuracy score on test set:  0.07777777777777778
-
    
-
    
-
    /var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp
-  return 1/(1 + np.exp(-x))
-/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:43: RuntimeWarning: overflow encountered in exp
-  exp_term = np.exp(self.z_o)
-/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:44: RuntimeWarning: invalid value encountered in divide
-  self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
-
    
-
    
-
    Learning rate  =  10.0
-Lambda =  1.0
-Accuracy score on test set:  0.07777777777777778
-
    
-
    
-
    /var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp
-  return 1/(1 + np.exp(-x))
-/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:43: RuntimeWarning: overflow encountered in exp
-  exp_term = np.exp(self.z_o)
-/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:44: RuntimeWarning: invalid value encountered in divide
-  self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
-
    
-
    
-
    Learning rate  =  10.0
-Lambda =  10.0
-Accuracy score on test set:  0.07777777777777778
+
    ---------------------------------------------------------------------------
+KeyboardInterrupt                         Traceback (most recent call last)
+Cell In[8], line 11
+      8 for j, lmbd in enumerate(lmbd_vals):
+      9     dnn = NeuralNetwork(X_train, Y_train_onehot, eta=eta, lmbd=lmbd, epochs=epochs, batch_size=batch_size,
+     10                         n_hidden_neurons=n_hidden_neurons, n_categories=n_categories)
+---> 11     dnn.train()
+     13     DNN_numpy[i][j] = dnn
+     15     test_predict = dnn.predict(X_test)
+
+Cell In[6], line 99, in NeuralNetwork.train(self)
+     96 self.Y_data = self.Y_data_full[chosen_datapoints]
+     98 self.feed_forward()
+---> 99 self.backpropagation()
+
+Cell In[6], line 64, in NeuralNetwork.backpropagation(self)
+     61 self.output_weights_gradient = np.matmul(self.a_h.T, error_output)
+     62 self.output_bias_gradient = np.sum(error_output, axis=0)
+---> 64 self.hidden_weights_gradient = np.matmul(self.X_data.T, error_hidden)
+     65 self.hidden_bias_gradient = np.sum(error_hidden, axis=0)
+     67 if self.lmbd > 0.0:
+
+KeyboardInterrupt: 
 
    
 
    
 
    
@@ -2966,22 +2873,6 @@ Accuracy score on test set:  0.07777777777777778
    -
    -
    /var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp
-  return 1/(1 + np.exp(-x))
-/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp
-  return 1/(1 + np.exp(-x))
-/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp
-  return 1/(1 + np.exp(-x))
-/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp
-  return 1/(1 + np.exp(-x))
-/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp
-  return 1/(1 + np.exp(-x))
-
    -
    -_images/week43_96_1.png -_images/week43_96_2.png -
    @@ -3017,328 +2908,6 @@ performance overall.

    -
    -
    /Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
-  warnings.warn(
-
    -
    -
    Learning rate  =  1e-05
-Lambda =  1e-05
-Accuracy score on test set:  0.18333333333333332
-
    -
    -
    /Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
-  warnings.warn(
-
    -
    -
    Learning rate  =  1e-05
-Lambda =  0.0001
-Accuracy score on test set:  0.18611111111111112
-
    -
    -
    /Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
-  warnings.warn(
-
    -
    -
    Learning rate  =  1e-05
-Lambda =  0.001
-Accuracy score on test set:  0.13055555555555556
-
    -
    -
    /Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
-  warnings.warn(
-
    -
    -
    Learning rate  =  1e-05
-Lambda =  0.01
-Accuracy score on test set:  0.24444444444444444
-
    -
    -
    /Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
-  warnings.warn(
-
    -
    -
    Learning rate  =  1e-05
-Lambda =  0.1
-Accuracy score on test set:  0.23333333333333334
-
    -
    -
    /Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
-  warnings.warn(
-
    -
    -
    Learning rate  =  1e-05
-Lambda =  1.0
-Accuracy score on test set:  0.12777777777777777
-
    -
    -
    /Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
-  warnings.warn(
-
    -
    -
    Learning rate  =  1e-05
-Lambda =  10.0
-Accuracy score on test set:  0.1527777777777778
-
    -
    -
    /Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
-  warnings.warn(
-
    -
    -
    Learning rate  =  0.0001
-Lambda =  1e-05
-Accuracy score on test set:  0.9111111111111111
-
    -
    -
    /Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
-  warnings.warn(
-
    -
    -
    Learning rate  =  0.0001
-Lambda =  0.0001
-Accuracy score on test set:  0.8888888888888888
-
    -
    -
    /Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
-  warnings.warn(
-
    -
    -
    Learning rate  =  0.0001
-Lambda =  0.001
-Accuracy score on test set:  0.8722222222222222
-
    -
    -
    /Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
-  warnings.warn(
-
    -
    -
    Learning rate  =  0.0001
-Lambda =  0.01
-Accuracy score on test set:  0.8305555555555556
-
    -
    -
    /Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
-  warnings.warn(
-
    -
    -
    Learning rate  =  0.0001
-Lambda =  0.1
-Accuracy score on test set:  0.8888888888888888
-
    -
    -
    /Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
-  warnings.warn(
-
    -
    -
    Learning rate  =  0.0001
-Lambda =  1.0
-Accuracy score on test set:  0.8805555555555555
-
    -
    -
    /Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
-  warnings.warn(
-
    -
    -
    Learning rate  =  0.0001
-Lambda =  10.0
-Accuracy score on test set:  0.8944444444444445
-
    -
    -
    /Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
-  warnings.warn(
-
    -
    -
    Learning rate  =  0.001
-Lambda =  1e-05
-Accuracy score on test set:  0.975
-
    -
    -
    /Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
-  warnings.warn(
-
    -
    -
    Learning rate  =  0.001
-Lambda =  0.0001
-Accuracy score on test set:  0.9777777777777777
-
    -
    -
    /Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
-  warnings.warn(
-
    -
    -
    Learning rate  =  0.001
-Lambda =  0.001
-Accuracy score on test set:  0.9805555555555555
-
    -
    -
    /Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
-  warnings.warn(
-
    -
    -
    Learning rate  =  0.001
-Lambda =  0.01
-Accuracy score on test set:  0.9861111111111112
-
    -
    -
    /Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
-  warnings.warn(
-
    -
    -
    Learning rate  =  0.001
-Lambda =  0.1
-Accuracy score on test set:  0.9805555555555555
-
    -
    -
    /Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
-  warnings.warn(
-
    -
    -
    Learning rate  =  0.001
-Lambda =  1.0
-Accuracy score on test set:  0.9777777777777777
-
    -
    -
    /Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
-  warnings.warn(
-
    -
    -
    Learning rate  =  0.001
-Lambda =  10.0
-Accuracy score on test set:  0.9444444444444444
-
    -
    -
    /Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
-  warnings.warn(
-
    -
    -
    Learning rate  =  0.01
-Lambda =  1e-05
-Accuracy score on test set:  0.9861111111111112
-
    -
    -
    Learning rate  =  0.01
-Lambda =  0.0001
-Accuracy score on test set:  0.9888888888888889
-
    -
    -
    Learning rate  =  0.01
-Lambda =  0.001
-Accuracy score on test set:  0.9888888888888889
-
    -
    -
    Learning rate  =  0.01
-Lambda =  0.01
-Accuracy score on test set:  0.9861111111111112
-
    -
    -
    Learning rate  =  0.01
-Lambda =  0.1
-Accuracy score on test set:  0.9888888888888889
-
    -
    -
    Learning rate  =  0.01
-Lambda =  1.0
-Accuracy score on test set:  0.9722222222222222
-
-Learning rate  =  0.01
-Lambda =  10.0
-Accuracy score on test set:  0.9527777777777777
-
    -
    -
    Learning rate  =  0.1
-Lambda =  1e-05
-Accuracy score on test set:  0.9027777777777778
-
-Learning rate  =  0.1
-Lambda =  0.0001
-Accuracy score on test set:  0.8583333333333333
-
    -
    -
    Learning rate  =  0.1
-Lambda =  0.001
-Accuracy score on test set:  0.8722222222222222
-
-Learning rate  =  0.1
-Lambda =  0.01
-Accuracy score on test set:  0.9055555555555556
-
    -
    -
    Learning rate  =  0.1
-Lambda =  0.1
-Accuracy score on test set:  0.8805555555555555
-
-Learning rate  =  0.1
-Lambda =  1.0
-Accuracy score on test set:  0.8722222222222222
-
    -
    -
    Learning rate  =  0.1
-Lambda =  10.0
-Accuracy score on test set:  0.8666666666666667
-
-Learning rate  =  1.0
-Lambda =  1e-05
-Accuracy score on test set:  0.08611111111111111
-
-Learning rate  =  1.0
-Lambda =  0.0001
-Accuracy score on test set:  0.10555555555555556
-
    -
    -
    Learning rate  =  1.0
-Lambda =  0.001
-Accuracy score on test set:  0.10555555555555556
-
-Learning rate  =  1.0
-Lambda =  0.01
-Accuracy score on test set:  0.17777777777777778
-
-Learning rate  =  1.0
-Lambda =  0.1
-Accuracy score on test set:  0.08333333333333333
-
    -
    -
    Learning rate  =  1.0
-Lambda =  1.0
-Accuracy score on test set:  0.08888888888888889
-
-Learning rate  =  1.0
-Lambda =  10.0
-Accuracy score on test set:  0.09444444444444444
-
    -
    -
    Learning rate  =  10.0
-Lambda =  1e-05
-Accuracy score on test set:  0.17222222222222222
-
-Learning rate  =  10.0
-Lambda =  0.0001
-Accuracy score on test set:  0.11666666666666667
-
-Learning rate  =  10.0
-Lambda =  0.001
-Accuracy score on test set:  0.10555555555555556
-
-Learning rate  =  10.0
-Lambda =  0.01
-Accuracy score on test set:  0.1388888888888889
-
    -
    -
    Learning rate  =  10.0
-Lambda =  0.1
-Accuracy score on test set:  0.11388888888888889
-
-Learning rate  =  10.0
-Lambda =  1.0
-Accuracy score on test set:  0.10555555555555556
-
    -
    -
    Learning rate  =  10.0
-Lambda =  10.0
-Accuracy score on test set:  0.09444444444444444
-
    -
    -
    @@ -3382,10 +2951,6 @@ Accuracy score on test set: 0.09444444444444444
    -
    -_images/week43_100_0.png -_images/week43_100_1.png -
    @@ -3424,14 +2989,6 @@ how simple solving a machine learning problem can be.

    -
    -
      Cell In[12], line 1
-    pip3 install tensorflow
-         ^
-SyntaxError: invalid syntax
-
    -
    -

    and/or if you use anaconda, just write (or install from the graphical user interface) (current release of CPU-only TensorFlow)

    @@ -3558,7 +3115,7 @@ If you have Anaconda installed you may run the following command

    model.add(Dense(n_neurons_layer2, activation='sigmoid', kernel_regularizer=regularizers.l2(lmbd))) model.add(Dense(n_categories, activation='softmax')) - sgd = optimizers.SGD(lr=eta) + sgd = optimizers.SGD(learning_rate=eta) model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy']) return model @@ -3738,7 +3295,7 @@ If you have Anaconda installed you may run the following command

    else: #Subsequent layers are capable of automatic shape inferencing model.add(Dense(n_neuron,activation='relu',kernel_regularizer=regularizers.l2(lamda))) model.add(Dense(2,activation='softmax')) #2 outputs - ordered and disordered (softmax for prob) - sgd=optimizers.SGD(lr=eta) + sgd=optimizers.SGD(learning_rate=eta) model.compile(loss='categorical_crossentropy',optimizer=sgd,metrics=['accuracy']) return model @@ -4828,8 +4385,8 @@ digits between the range of 0 to 9.

    Not bad, but the results depend strongly on the learning reate. Try different learning rates.

    -
    -

    Solving ODEs with Deep Learning¶

    +
    +

    Solving differential equations with Deep Learning¶

    The Universal Approximation Theorem states that a neural network can approximate any function at a single hidden layer along with one input and output layer to any given precision.

    @@ -4841,8 +4398,8 @@ and output layer to any given precision.

    The lectures on differential equations were developed by Kristine Baluka Hein, now PhD student at IFI. A great thanks to Kristine.

    -
    -

    Ordinary Differential Equations¶

    +
    +

    Ordinary Differential Equations first¶

    An ordinary differential equation (ODE) is an equation involving functions having one variable.

    In general, an ordinary differential equation looks like

    @@ -5350,8 +4907,8 @@ P_{\text{output},\text{new}} &= P_{\text{output}} - \lambda \nabla_{P_{\text # but with number of hidden layers specified by the user. def deep_neural_network(deep_params, x): # N_hidden is the number of hidden layers - - N_hidden = np.size(deep_params) - 1 # -1 since params consists of + # deep_params is a list, len() should be used + N_hidden = len(deep_params) - 1 # -1 since params consists of # parameters to all the hidden # layers AND the output layer. @@ -5572,9 +5129,12 @@ g(t) = \frac{Ag_0}{g_0 + (A - g_0)\exp(-\alpha A t)} g0 = 1.2 return alpha, A, g0 -def deep_neural_network(P, x): +def deep_neural_network(deep_params, x): # N_hidden is the number of hidden layers - N_hidden = np.size(P) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer + # deep_params is a list, len() should be used + N_hidden = len(deep_params) - 1 # -1 since params consists of + # parameters to all the hidden + # layers AND the output layer. # Assumes input x being an one-dimensional array num_values = np.size(x) @@ -5591,7 +5151,7 @@ g(t) = \frac{Ag_0}{g_0 + (A - g_0)\exp(-\alpha A t)} for l in range(N_hidden): # From the list of parameters P; find the correct weigths and bias for this layer - w_hidden = P[l] + w_hidden = deep_params[l] # Add a row of ones to include bias x_prev = np.concatenate((np.ones((1,num_values)), x_prev ), axis = 0) @@ -5605,7 +5165,7 @@ g(t) = \frac{Ag_0}{g_0 + (A - g_0)\exp(-\alpha A t)} ## Output layer: # Get the weights and bias for this layer - w_output = P[-1] + w_output = deep_params[-1] # Include bias: x_prev = np.concatenate((np.ones((1,num_values)), x_prev), axis = 0) @@ -5616,6 +5176,8 @@ g(t) = \frac{Ag_0}{g_0 + (A - g_0)\exp(-\alpha A t)} return x_output + + def cost_function_deep(P, x): # Evaluate the trial function with the current parameters P @@ -5906,7 +5468,10 @@ g(x) = x(1 - x)\exp(x) def deep_neural_network(deep_params, x): # N_hidden is the number of hidden layers - N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer + # deep_params is a list, len() should be used + N_hidden = len(deep_params) - 1 # -1 since params consists of + # parameters to all the hidden + # layers AND the output layer. # Assumes input x being an one-dimensional array num_values = np.size(x) @@ -5947,6 +5512,7 @@ g(x) = x(1 - x)\exp(x) return x_output + def solve_ode_deep_neural_network(x, num_neurons, num_iter, lmb): # num_hidden_neurons is now a list of number of neurons within each hidden layer @@ -6146,7 +5712,10 @@ f(x_{N_x - 2}) def deep_neural_network(deep_params, x): # N_hidden is the number of hidden layers - N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer + # deep_params is a list, len() should be used + N_hidden = len(deep_params) - 1 # -1 since params consists of + # parameters to all the hidden + # layers AND the output layer. # Assumes input x being an one-dimensional array num_values = np.size(x) @@ -6187,6 +5756,7 @@ f(x_{N_x - 2}) return x_output + def solve_ode_deep_neural_network(x, num_neurons, num_iter, lmb): # num_hidden_neurons is now a list of number of neurons within each hidden layer @@ -6456,7 +6026,7 @@ network at each possible pair \((x,t) num_points = np.size(x,1) # N_hidden is the number of hidden layers - N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer + N_hidden = len(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer # Assume that the input layer does nothing to the input x x_input = x @@ -6607,7 +6177,7 @@ Using TensorFlow results in a much better execution time. Try it!

    num_points = np.size(x,1) # N_hidden is the number of hidden layers - N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer + N_hidden = len(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer # Assume that the input layer does nothing to the input x x_input = x @@ -6750,7 +6320,7 @@ Using TensorFlow results in a much better execution time. Try it!

    T,X = np.meshgrid(t,x) fig = plt.figure(figsize=(10,10)) - ax = fig.gca(projection='3d') + ax = fig.add_suplot(projection='3d') ax.set_title('Solution from the deep neural network w/ %d layer'%len(num_hidden_neurons)) s = ax.plot_surface(T,X,g_dnn_ag,linewidth=0,antialiased=False,cmap=cm.viridis) ax.set_xlabel('Time $t$') @@ -6758,14 +6328,14 @@ Using TensorFlow results in a much better execution time. Try it!

    fig = plt.figure(figsize=(10,10)) - ax = fig.gca(projection='3d') + ax = fig.add_suplot(projection='3d') ax.set_title('Analytical solution') s = ax.plot_surface(T,X,G_analytical,linewidth=0,antialiased=False,cmap=cm.viridis) ax.set_xlabel('Time $t$') ax.set_ylabel('Position $x$'); fig = plt.figure(figsize=(10,10)) - ax = fig.gca(projection='3d') + ax = fig.add_suplot(projection='3d') ax.set_title('Difference') s = ax.plot_surface(T,X,diff_ag,linewidth=0,antialiased=False,cmap=cm.viridis) ax.set_xlabel('Time $t$') @@ -6943,7 +6513,7 @@ g(x,t) = \sin(\pi x)\cos(\pi t) - \sin(\pi x)\sin(\pi t) num_points = np.size(x,1) # N_hidden is the number of hidden layers - N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer + N_hidden = len(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer # Assume that the input layer does nothing to the input x x_input = x @@ -7048,7 +6618,7 @@ g(x,t) = \sin(\pi x)\cos(\pi t) - \sin(\pi x)\sin(\pi t) T,X = np.meshgrid(t,x) fig = plt.figure(figsize=(10,10)) - ax = fig.gca(projection='3d') + ax = fig.add_suplot(projection='3d') ax.set_title('Solution from the deep neural network w/ %d layer'%len(num_hidden_neurons)) s = ax.plot_surface(T,X,res,linewidth=0,antialiased=False,cmap=cm.viridis) ax.set_xlabel('Time $t$') @@ -7056,7 +6626,7 @@ g(x,t) = \sin(\pi x)\cos(\pi t) - \sin(\pi x)\sin(\pi t) fig = plt.figure(figsize=(10,10)) - ax = fig.gca(projection='3d') + ax = fig.add_suplot(projection='3d') ax.set_title('Analytical solution') s = ax.plot_surface(T,X,res_analytical,linewidth=0,antialiased=False,cmap=cm.viridis) ax.set_xlabel('Time $t$') @@ -7064,7 +6634,7 @@ g(x,t) = \sin(\pi x)\cos(\pi t) - \sin(\pi x)\sin(\pi t) fig = plt.figure(figsize=(10,10)) - ax = fig.gca(projection='3d') + ax = fig.add_suplot(projection='3d') ax.set_title('Difference') s = ax.plot_surface(T,X,diff,linewidth=0,antialiased=False,cmap=cm.viridis) ax.set_xlabel('Time $t$') diff --git a/doc/LectureNotes/_build/jupyter_execute/week43.ipynb b/doc/LectureNotes/_build/jupyter_execute/week43.ipynb index 035fa507f..0824e4c7e 100644 --- a/doc/LectureNotes/_build/jupyter_execute/week43.ipynb +++ b/doc/LectureNotes/_build/jupyter_execute/week43.ipynb @@ -2,8 +2,10 @@ "cells": [ { "cell_type": "markdown", - "id": "cda62a86", - "metadata": {}, + "id": "677a195d", + "metadata": { + "editable": true + }, "source": [ "\n", @@ -12,8 +14,10 @@ }, { "cell_type": "markdown", - "id": "bdae1fa5", - "metadata": {}, + "id": "e943e8f0", + "metadata": { + "editable": true + }, "source": [ "# Week 43: Deep Learning: Constructing a Neural Network code and solving differential equations\n", "**Morten Hjorth-Jensen**, Department of Physics, University of Oslo and Department of Physics and Astronomy and Facility for Rare Isotope Beams, Michigan State University\n", @@ -23,8 +27,10 @@ }, { "cell_type": "markdown", - "id": "e2c09dd5", - "metadata": {}, + "id": "22cdced0", + "metadata": { + "editable": true + }, "source": [ "## Plans for week 43\n", "\n", @@ -39,8 +45,10 @@ }, { "cell_type": "markdown", - "id": "7de8bfe0", - "metadata": {}, + "id": "a57ef6df", + "metadata": { + "editable": true + }, "source": [ "## Exercises and lab session week 43\n", "**Lab sessions on Tuesday and Wednesday.**\n", @@ -54,8 +62,10 @@ }, { "cell_type": "markdown", - "id": "2664d901", - "metadata": {}, + "id": "c4a134ee", + "metadata": { + "editable": true + }, "source": [ "## Mathematics of deep learning\n", "\n", @@ -68,8 +78,10 @@ }, { "cell_type": "markdown", - "id": "46bb5558", - "metadata": {}, + "id": "dd73045e", + "metadata": { + "editable": true + }, "source": [ "## Reminder on books with hands-on material and codes\n", "* Sebastian Rashcka et al, Machine learning with Scikit-Learn and PyTorch at " @@ -77,8 +89,10 @@ }, { "cell_type": "markdown", - "id": "1c6ad86d", - "metadata": {}, + "id": "372f4aa4", + "metadata": { + "editable": true + }, "source": [ "## Reading recommendations\n", "\n", @@ -89,8 +103,10 @@ }, { "cell_type": "markdown", - "id": "9fbf4898", - "metadata": {}, + "id": "05efbc6f", + "metadata": { + "editable": true + }, "source": [ "## Using Automatic differentiation\n", "\n", @@ -100,8 +116,10 @@ }, { "cell_type": "markdown", - "id": "135a7122", - "metadata": {}, + "id": "d04b0948", + "metadata": { + "editable": true + }, "source": [ "## Back propagation and automatic differentiation\n", "\n", @@ -115,16 +133,20 @@ }, { "cell_type": "markdown", - "id": "45236eaf", - "metadata": {}, + "id": "b9d41f5c", + "metadata": { + "editable": true + }, "source": [ "## Lecture Monday October 21" ] }, { "cell_type": "markdown", - "id": "8997a10d", - "metadata": {}, + "id": "68606991", + "metadata": { + "editable": true + }, "source": [ "## Setting up the back propagation algorithm and algorithm for a feed forward NN, initalizations\n", "This is a reminder from where we ended last week.\n", @@ -148,8 +170,10 @@ }, { "cell_type": "markdown", - "id": "df88cb72", - "metadata": {}, + "id": "a564a394", + "metadata": { + "editable": true + }, "source": [ "## Setting up the back propagation algorithm, part 1\n", "\n", @@ -169,8 +193,10 @@ }, { "cell_type": "markdown", - "id": "46d7ebde", - "metadata": {}, + "id": "43ab4381", + "metadata": { + "editable": true + }, "source": [ "## Setting up the back propagation algorithm, part 2\n", "\n", @@ -179,8 +205,10 @@ }, { "cell_type": "markdown", - "id": "3ad237f5", - "metadata": {}, + "id": "f578cee4", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\delta_j^L = \\sigma'(z_j^L)\\frac{\\partial {\\cal C}}{\\partial (a_j^L)}.\n", @@ -189,16 +217,20 @@ }, { "cell_type": "markdown", - "id": "f6906530", - "metadata": {}, + "id": "fbc05943", + "metadata": { + "editable": true + }, "source": [ "Then we compute the back propagate error for each $l=L-1,L-2,\\dots,1$ as" ] }, { "cell_type": "markdown", - "id": "e9613a49", - "metadata": {}, + "id": "6ad3c28f", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\delta_j^l = \\sum_k \\delta_k^{l+1}w_{kj}^{l+1}\\sigma'(z_j^l).\n", @@ -207,8 +239,10 @@ }, { "cell_type": "markdown", - "id": "dba468b9", - "metadata": {}, + "id": "135513b5", + "metadata": { + "editable": true + }, "source": [ "## Setting up the Back propagation algorithm, part 3\n", "\n", @@ -219,8 +253,10 @@ }, { "cell_type": "markdown", - "id": "21c684a1", - "metadata": {}, + "id": "f643ecdf", + "metadata": { + "editable": true + }, "source": [ "$$\n", "w_{ij}^l\\leftarrow = w_{ij}^l- \\eta \\delta_j^la_i^{l-1},\n", @@ -229,8 +265,10 @@ }, { "cell_type": "markdown", - "id": "6acf04a3", - "metadata": {}, + "id": "077ef1c2", + "metadata": { + "editable": true + }, "source": [ "$$\n", "b_j^l \\leftarrow b_j^l-\\eta \\frac{\\partial {\\cal C}}{\\partial b_j^l}=b_j^l-\\eta \\delta_j^l,\n", @@ -239,16 +277,20 @@ }, { "cell_type": "markdown", - "id": "b0558fbe", - "metadata": {}, + "id": "50cf7472", + "metadata": { + "editable": true + }, "source": [ "with $\\eta$ being the learning rate." ] }, { "cell_type": "markdown", - "id": "6b424064", - "metadata": {}, + "id": "a10f406c", + "metadata": { + "editable": true + }, "source": [ "## Updating the gradients\n", "\n", @@ -257,8 +299,10 @@ }, { "cell_type": "markdown", - "id": "7bc86e31", - "metadata": {}, + "id": "626cba8f", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\delta_j^l = \\sum_k \\delta_k^{l+1}w_{kj}^{l+1}\\sigma'(z_j^l),\n", @@ -267,16 +311,20 @@ }, { "cell_type": "markdown", - "id": "f7a10087", - "metadata": {}, + "id": "2ebe05ad", + "metadata": { + "editable": true + }, "source": [ "we update the weights and the biases using gradient descent for each $l=L-1,L-2,\\dots,1$ and update the weights and biases according to the rules" ] }, { "cell_type": "markdown", - "id": "2bf12837", - "metadata": {}, + "id": "bfbbac35", + "metadata": { + "editable": true + }, "source": [ "$$\n", "w_{ij}^l\\leftarrow = w_{ij}^l- \\eta \\delta_j^la_i^{l-1},\n", @@ -285,8 +333,10 @@ }, { "cell_type": "markdown", - "id": "ad07731e", - "metadata": {}, + "id": "ac515399", + "metadata": { + "editable": true + }, "source": [ "$$\n", "b_j^l \\leftarrow b_j^l-\\eta \\frac{\\partial {\\cal C}}{\\partial b_j^l}=b_j^l-\\eta \\delta_j^l,\n", @@ -295,8 +345,10 @@ }, { "cell_type": "markdown", - "id": "30c6ae90", - "metadata": {}, + "id": "4703cc51", + "metadata": { + "editable": true + }, "source": [ "## Activation functions\n", "\n", @@ -316,8 +368,10 @@ }, { "cell_type": "markdown", - "id": "5dfa6d1c", - "metadata": {}, + "id": "cab6ad03", + "metadata": { + "editable": true + }, "source": [ "### Activation functions, examples\n", "\n", @@ -326,8 +380,10 @@ }, { "cell_type": "markdown", - "id": "e8d0be63", - "metadata": {}, + "id": "ad6042eb", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\sigma(x) = \\frac{1}{1 + e^{-x}},\n", @@ -336,16 +392,20 @@ }, { "cell_type": "markdown", - "id": "b3ec0efa", - "metadata": {}, + "id": "1e96b7e7", + "metadata": { + "editable": true + }, "source": [ "and the *hyperbolic tangent* function" ] }, { "cell_type": "markdown", - "id": "78c46915", - "metadata": {}, + "id": "c600792f", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\sigma(x) = \\tanh(x)\n", @@ -354,8 +414,10 @@ }, { "cell_type": "markdown", - "id": "87e6973b", - "metadata": {}, + "id": "76d1da8e", + "metadata": { + "editable": true + }, "source": [ "## The RELU function family\n", "\n", @@ -373,8 +435,10 @@ }, { "cell_type": "markdown", - "id": "9c990bc8", - "metadata": {}, + "id": "8e7e0e0c", + "metadata": { + "editable": true + }, "source": [ "## ELU function\n", "\n", @@ -385,8 +449,10 @@ }, { "cell_type": "markdown", - "id": "9b7760f9", - "metadata": {}, + "id": "4b1f77d6", + "metadata": { + "editable": true + }, "source": [ "$$\n", "ELU(z) = \\left\\{\\begin{array}{cc} \\alpha\\left( \\exp{(z)}-1\\right) & z < 0,\\\\ z & z \\ge 0.\\end{array}\\right.\n", @@ -395,8 +461,10 @@ }, { "cell_type": "markdown", - "id": "47927a82", - "metadata": {}, + "id": "10826ba9", + "metadata": { + "editable": true + }, "source": [ "## Which activation function should we use?\n", "\n", @@ -415,8 +483,10 @@ }, { "cell_type": "markdown", - "id": "52554330", - "metadata": {}, + "id": "79d307b6", + "metadata": { + "editable": true + }, "source": [ "## More on activation functions, output layers\n", "\n", @@ -435,8 +505,10 @@ }, { "cell_type": "markdown", - "id": "4b7d3292", - "metadata": {}, + "id": "429b8d4e", + "metadata": { + "editable": true + }, "source": [ "## Setting up a Multi-layer perceptron model for classification\n", "\n", @@ -461,8 +533,10 @@ }, { "cell_type": "markdown", - "id": "38715b52", - "metadata": {}, + "id": "90d9f195", + "metadata": { + "editable": true + }, "source": [ "$$\n", "P(y = 0 \\mid \\boldsymbol{x}, \\boldsymbol{\\theta}) = \\frac{1}{1 + \\exp{(- \\boldsymbol{x}})} ,\n", @@ -471,16 +545,20 @@ }, { "cell_type": "markdown", - "id": "dcc0cbfa", - "metadata": {}, + "id": "73d2763e", + "metadata": { + "editable": true + }, "source": [ "and" ] }, { "cell_type": "markdown", - "id": "c686ecf6", - "metadata": {}, + "id": "e2a70a54", + "metadata": { + "editable": true + }, "source": [ "$$\n", "P(y = 1 \\mid \\boldsymbol{x}, \\boldsymbol{\\theta}) = 1 - P(y = 0 \\mid \\boldsymbol{x}, \\boldsymbol{\\theta}) ,\n", @@ -489,8 +567,10 @@ }, { "cell_type": "markdown", - "id": "79a8b207", - "metadata": {}, + "id": "7ed6c0dc", + "metadata": { + "editable": true + }, "source": [ "where $y \\in \\{0, 1\\}$ and $\\boldsymbol{\\theta}$ represents the weights and biases\n", "of our network." @@ -498,8 +578,10 @@ }, { "cell_type": "markdown", - "id": "3218b074", - "metadata": {}, + "id": "a42826b9", + "metadata": { + "editable": true + }, "source": [ "## Defining the cost function\n", "\n", @@ -508,8 +590,10 @@ }, { "cell_type": "markdown", - "id": "c0e50336", - "metadata": {}, + "id": "d8e401c5", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\mathcal{C}(\\boldsymbol{\\theta}) = - \\ln P(\\mathcal{D} \\mid \\boldsymbol{\\theta}) = - \\sum_{i=1}^n\n", @@ -519,8 +603,10 @@ }, { "cell_type": "markdown", - "id": "79138403", - "metadata": {}, + "id": "17808105", + "metadata": { + "editable": true + }, "source": [ "This last equality means that we can interpret our *cost* function as a sum over the *loss* function\n", "for each point in the dataset $\\mathcal{L}_i(\\boldsymbol{\\theta})$. \n", @@ -542,8 +628,10 @@ }, { "cell_type": "markdown", - "id": "1b3db35e", - "metadata": {}, + "id": "5a9b90ce", + "metadata": { + "editable": true + }, "source": [ "$$\n", "P(y_{ic} = 1 \\mid \\boldsymbol{x}_i, \\boldsymbol{\\theta}) = \\frac{\\exp{((\\boldsymbol{a}_i^{hidden})^T \\boldsymbol{w}_c)}}\n", @@ -553,8 +641,10 @@ }, { "cell_type": "markdown", - "id": "90626f76", - "metadata": {}, + "id": "57f33029", + "metadata": { + "editable": true + }, "source": [ "which reduces to the logistic function in the binary case. \n", "The likelihood of this $C$-class classifier\n", @@ -563,8 +653,10 @@ }, { "cell_type": "markdown", - "id": "596eb444", - "metadata": {}, + "id": "422eb23f", + "metadata": { + "editable": true + }, "source": [ "$$\n", "P(\\mathcal{D} \\mid \\boldsymbol{\\theta}) = \\prod_{i=1}^n \\prod_{c=0}^{C-1} [P(y_{ic} = 1)]^{y_{ic}} .\n", @@ -573,16 +665,20 @@ }, { "cell_type": "markdown", - "id": "5529d979", - "metadata": {}, + "id": "0eb634c3", + "metadata": { + "editable": true + }, "source": [ "Again we take the negative log-likelihood to define our cost function:" ] }, { "cell_type": "markdown", - "id": "f7d2523f", - "metadata": {}, + "id": "b3b208fc", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\mathcal{C}(\\boldsymbol{\\theta}) = - \\log{P(\\mathcal{D} \\mid \\boldsymbol{\\theta})}.\n", @@ -591,8 +687,10 @@ }, { "cell_type": "markdown", - "id": "7d8d61f1", - "metadata": {}, + "id": "3788aba1", + "metadata": { + "editable": true + }, "source": [ "See the logistic regression lectures for a full definition of the cost function.\n", "\n", @@ -601,8 +699,10 @@ }, { "cell_type": "markdown", - "id": "9751fb82", - "metadata": {}, + "id": "4d38ceec", + "metadata": { + "editable": true + }, "source": [ "## Example: binary classification problem\n", "\n", @@ -611,8 +711,10 @@ }, { "cell_type": "markdown", - "id": "3e6587c7", - "metadata": {}, + "id": "cee503b9", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\mathcal{C}(\\boldsymbol{\\beta}) = - \\sum_{i=1}^n \\left(y_i\\log{p(y_i \\vert x_i,\\boldsymbol{\\beta})}+(1-y_i)\\log{1-p(y_i \\vert x_i,\\boldsymbol{\\beta})}\\right),\n", @@ -621,16 +723,20 @@ }, { "cell_type": "markdown", - "id": "39bb5ba4", - "metadata": {}, + "id": "19b29e02", + "metadata": { + "editable": true + }, "source": [ "where we had defined the logistic (sigmoid) function" ] }, { "cell_type": "markdown", - "id": "fcb1f3d9", - "metadata": {}, + "id": "19507159", + "metadata": { + "editable": true + }, "source": [ "$$\n", "p(y_i =1\\vert x_i,\\boldsymbol{\\beta})=\\frac{\\exp{(\\beta_0+\\beta_1 x_i)}}{1+\\exp{(\\beta_0+\\beta_1 x_i)}},\n", @@ -639,16 +745,20 @@ }, { "cell_type": "markdown", - "id": "20fc130a", - "metadata": {}, + "id": "a15191e4", + "metadata": { + "editable": true + }, "source": [ "and" ] }, { "cell_type": "markdown", - "id": "66c7c0e9", - "metadata": {}, + "id": "c9df7a22", + "metadata": { + "editable": true + }, "source": [ "$$\n", "p(y_i =0\\vert x_i,\\boldsymbol{\\beta})=1-p(y_i =1\\vert x_i,\\boldsymbol{\\beta}).\n", @@ -657,8 +767,10 @@ }, { "cell_type": "markdown", - "id": "e757c738", - "metadata": {}, + "id": "0cae75ec", + "metadata": { + "editable": true + }, "source": [ "The parameters $\\boldsymbol{\\beta}$ were defined using a minimization method like gradient descent or Newton-Raphson's method. \n", "\n", @@ -668,8 +780,10 @@ }, { "cell_type": "markdown", - "id": "d048e79e", - "metadata": {}, + "id": "36b0506b", + "metadata": { + "editable": true + }, "source": [ "$$\n", "a_i^l = y_i = \\frac{\\exp{(z_i^l)}}{1+\\exp{(z_i^l)}},\n", @@ -678,16 +792,20 @@ }, { "cell_type": "markdown", - "id": "b97827e7", - "metadata": {}, + "id": "0c729c48", + "metadata": { + "editable": true + }, "source": [ "with" ] }, { "cell_type": "markdown", - "id": "c11528eb", - "metadata": {}, + "id": "060813af", + "metadata": { + "editable": true + }, "source": [ "$$\n", "z_i^l = \\sum_{j}w_{ij}^l a_j^{l-1}+b_i^l,\n", @@ -696,8 +814,10 @@ }, { "cell_type": "markdown", - "id": "94839c96", - "metadata": {}, + "id": "f2a5059b", + "metadata": { + "editable": true + }, "source": [ "where the superscript $l-1$ indicates that these are the outputs from layer $l-1$.\n", "Our cost function at the final layer $l=L$ is now" @@ -705,8 +825,10 @@ }, { "cell_type": "markdown", - "id": "694596c9", - "metadata": {}, + "id": "6763074d", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\mathcal{C}(\\boldsymbol{W}) = - \\sum_{i=1}^n \\left(t_i\\log{a_i^L}+(1-t_i)\\log{(1-a_i^L)}\\right),\n", @@ -715,16 +837,20 @@ }, { "cell_type": "markdown", - "id": "b4c8232e", - "metadata": {}, + "id": "f498b6b5", + "metadata": { + "editable": true + }, "source": [ "where we have defined the targets $t_i$. The derivatives of the cost function with respect to the output $a_i^L$ are then easily calculated and we get" ] }, { "cell_type": "markdown", - "id": "9d611a08", - "metadata": {}, + "id": "fedce928", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\frac{\\partial \\mathcal{C}(\\boldsymbol{W})}{\\partial a_i^L} = \\frac{a_i^L-t_i}{a_i^L(1-a_i^L)}.\n", @@ -733,16 +859,20 @@ }, { "cell_type": "markdown", - "id": "cb7847ef", - "metadata": {}, + "id": "429c11ae", + "metadata": { + "editable": true + }, "source": [ "In case we use another activation function than the logistic one, we need to evaluate other derivatives." ] }, { "cell_type": "markdown", - "id": "cd5be179", - "metadata": {}, + "id": "ddde8370", + "metadata": { + "editable": true + }, "source": [ "## The Softmax function\n", "In case we employ the more general case given by the Softmax equation, we need to evaluate the derivative of the activation function with respect to the activation $z_i^l$, that is we need" @@ -750,8 +880,10 @@ }, { "cell_type": "markdown", - "id": "96b7fe79", - "metadata": {}, + "id": "637bd194", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\frac{\\partial f(z_i^l)}{\\partial w_{jk}^l} =\n", @@ -761,16 +893,20 @@ }, { "cell_type": "markdown", - "id": "1040c590", - "metadata": {}, + "id": "0792109c", + "metadata": { + "editable": true + }, "source": [ "For the Softmax function we have" ] }, { "cell_type": "markdown", - "id": "3162bd5e", - "metadata": {}, + "id": "10f14691", + "metadata": { + "editable": true + }, "source": [ "$$\n", "f(z_i^l) = \\frac{\\exp{(z_i^l)}}{\\sum_{m=1}^K\\exp{(z_m^l)}}.\n", @@ -779,16 +915,20 @@ }, { "cell_type": "markdown", - "id": "b7f2b7b8", - "metadata": {}, + "id": "186529b5", + "metadata": { + "editable": true + }, "source": [ "Its derivative with respect to $z_j^l$ gives" ] }, { "cell_type": "markdown", - "id": "735fd9c0", - "metadata": {}, + "id": "68afc0da", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\frac{\\partial f(z_i^l)}{\\partial z_j^l}= f(z_i^l)\\left(\\delta_{ij}-f(z_j^l)\\right),\n", @@ -797,16 +937,20 @@ }, { "cell_type": "markdown", - "id": "c3cbbb04", - "metadata": {}, + "id": "cf48ea60", + "metadata": { + "editable": true + }, "source": [ "which in case of the simply binary model reduces to having $i=j$." ] }, { "cell_type": "markdown", - "id": "6462edbb", - "metadata": {}, + "id": "861a3bbe", + "metadata": { + "editable": true + }, "source": [ "## Developing a code for doing neural networks with back propagation\n", "\n", @@ -827,8 +971,10 @@ }, { "cell_type": "markdown", - "id": "7915dc1d", - "metadata": {}, + "id": "c8007ce7", + "metadata": { + "editable": true + }, "source": [ "## Collect and pre-process data\n", "\n", @@ -875,8 +1021,11 @@ { "cell_type": "code", "execution_count": 1, - "id": "eb736d79", - "metadata": {}, + "id": "29678f97", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [ { "name": "stdout", @@ -951,8 +1100,10 @@ }, { "cell_type": "markdown", - "id": "01fcb50b", - "metadata": {}, + "id": "3ba6d7d6", + "metadata": { + "editable": true + }, "source": [ "## Train and test datasets\n", "\n", @@ -970,8 +1121,11 @@ { "cell_type": "code", "execution_count": 2, - "id": "e6b7f2e3", - "metadata": {}, + "id": "7df51dd1", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [ { "name": "stdout", @@ -1014,8 +1168,10 @@ }, { "cell_type": "markdown", - "id": "1ca032f8", - "metadata": {}, + "id": "249898fd", + "metadata": { + "editable": true + }, "source": [ "## Define model and architecture\n", "\n", @@ -1056,8 +1212,10 @@ }, { "cell_type": "markdown", - "id": "6e4e1e18", - "metadata": {}, + "id": "2d30ddd3", + "metadata": { + "editable": true + }, "source": [ "## Layers\n", "\n", @@ -1094,8 +1252,10 @@ }, { "cell_type": "markdown", - "id": "ab0ee6d8", - "metadata": {}, + "id": "ce9f14e4", + "metadata": { + "editable": true + }, "source": [ "## Weights and biases\n", "\n", @@ -1113,8 +1273,11 @@ { "cell_type": "code", "execution_count": 3, - "id": "15ad1599", - "metadata": {}, + "id": "c3f75b32", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "# building our neural network\n", @@ -1136,8 +1299,10 @@ }, { "cell_type": "markdown", - "id": "3efd576a", - "metadata": {}, + "id": "4ebf4f68", + "metadata": { + "editable": true + }, "source": [ "## Feed-forward pass\n", "\n", @@ -1162,8 +1327,10 @@ }, { "cell_type": "markdown", - "id": "715c5d46", - "metadata": {}, + "id": "c6760d1f", + "metadata": { + "editable": true + }, "source": [ "## Matrix multiplications\n", "\n", @@ -1197,8 +1364,11 @@ { "cell_type": "code", "execution_count": 4, - "id": "ecaa833e", - "metadata": {}, + "id": "98f3a5d5", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [ { "name": "stdout", @@ -1257,8 +1427,10 @@ }, { "cell_type": "markdown", - "id": "c0319e65", - "metadata": {}, + "id": "19cd49ec", + "metadata": { + "editable": true + }, "source": [ "## Choose cost function and optimizer\n", "\n", @@ -1286,8 +1458,10 @@ }, { "cell_type": "markdown", - "id": "98c6696e", - "metadata": {}, + "id": "b0342f54", + "metadata": { + "editable": true + }, "source": [ "## Optimizing the cost function\n", "\n", @@ -1322,8 +1496,10 @@ }, { "cell_type": "markdown", - "id": "b5d1145a", - "metadata": {}, + "id": "7219731b", + "metadata": { + "editable": true + }, "source": [ "## Regularization\n", "\n", @@ -1354,8 +1530,10 @@ }, { "cell_type": "markdown", - "id": "b73247fd", - "metadata": {}, + "id": "248590b9", + "metadata": { + "editable": true + }, "source": [ "## Matrix multiplication\n", "\n", @@ -1393,8 +1571,11 @@ { "cell_type": "code", "execution_count": 5, - "id": "3d61438f", - "metadata": {}, + "id": "1810a36c", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [ { "name": "stdout", @@ -1407,7 +1588,7 @@ "name": "stderr", "output_type": "stream", "text": [ - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp\n", + "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/953065564.py:4: RuntimeWarning: overflow encountered in exp\n", " return 1/(1 + np.exp(-x))\n" ] }, @@ -1492,8 +1673,10 @@ }, { "cell_type": "markdown", - "id": "6a20a495", - "metadata": {}, + "id": "e094a404", + "metadata": { + "editable": true + }, "source": [ "## Improving performance\n", "\n", @@ -1511,8 +1694,10 @@ }, { "cell_type": "markdown", - "id": "717b63a1", - "metadata": {}, + "id": "1fb3aab5", + "metadata": { + "editable": true + }, "source": [ "## Full object-oriented implementation\n", "\n", @@ -1523,8 +1708,11 @@ { "cell_type": "code", "execution_count": 6, - "id": "9d6c9929", - "metadata": {}, + "id": "20b3c187", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "class NeuralNetwork:\n", @@ -1630,8 +1818,10 @@ }, { "cell_type": "markdown", - "id": "e70667c5", - "metadata": {}, + "id": "f85564ae", + "metadata": { + "editable": true + }, "source": [ "## Evaluate model performance on test data\n", "\n", @@ -1647,8 +1837,11 @@ { "cell_type": "code", "execution_count": 7, - "id": "083dfe41", - "metadata": {}, + "id": "9be7082e", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [ { "name": "stdout", @@ -1679,8 +1872,10 @@ }, { "cell_type": "markdown", - "id": "bffeadd3", - "metadata": {}, + "id": "255b5b3d", + "metadata": { + "editable": true + }, "source": [ "## Adjust hyperparameters\n", "\n", @@ -1691,8 +1886,11 @@ { "cell_type": "code", "execution_count": 8, - "id": "6513ac74", - "metadata": {}, + "id": "fc014642", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [ { "name": "stdout", @@ -1978,7 +2176,7 @@ "name": "stderr", "output_type": "stream", "text": [ - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp\n", + "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/953065564.py:4: RuntimeWarning: overflow encountered in exp\n", " return 1/(1 + np.exp(-x))\n" ] }, @@ -1996,7 +2194,7 @@ "name": "stderr", "output_type": "stream", "text": [ - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp\n", + "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/953065564.py:4: RuntimeWarning: overflow encountered in exp\n", " return 1/(1 + np.exp(-x))\n" ] }, @@ -2014,7 +2212,7 @@ "name": "stderr", "output_type": "stream", "text": [ - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp\n", + "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/953065564.py:4: RuntimeWarning: overflow encountered in exp\n", " return 1/(1 + np.exp(-x))\n" ] }, @@ -2032,7 +2230,7 @@ "name": "stderr", "output_type": "stream", "text": [ - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp\n", + "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/953065564.py:4: RuntimeWarning: overflow encountered in exp\n", " return 1/(1 + np.exp(-x))\n" ] }, @@ -2050,7 +2248,7 @@ "name": "stderr", "output_type": "stream", "text": [ - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp\n", + "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/953065564.py:4: RuntimeWarning: overflow encountered in exp\n", " return 1/(1 + np.exp(-x))\n" ] }, @@ -2068,7 +2266,7 @@ "name": "stderr", "output_type": "stream", "text": [ - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp\n", + "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/953065564.py:4: RuntimeWarning: overflow encountered in exp\n", " return 1/(1 + np.exp(-x))\n" ] }, @@ -2086,7 +2284,7 @@ "name": "stderr", "output_type": "stream", "text": [ - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp\n", + "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/953065564.py:4: RuntimeWarning: overflow encountered in exp\n", " return 1/(1 + np.exp(-x))\n" ] }, @@ -2104,11 +2302,11 @@ "name": "stderr", "output_type": "stream", "text": [ - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp\n", + "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/953065564.py:4: RuntimeWarning: overflow encountered in exp\n", " return 1/(1 + np.exp(-x))\n", - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n", + "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n", " exp_term = np.exp(self.z_o)\n", - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:44: RuntimeWarning: invalid value encountered in divide\n", + "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/1630775253.py:44: RuntimeWarning: invalid value encountered in divide\n", " self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)\n" ] }, @@ -2126,11 +2324,11 @@ "name": "stderr", "output_type": "stream", "text": [ - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp\n", + "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/953065564.py:4: RuntimeWarning: overflow encountered in exp\n", " return 1/(1 + np.exp(-x))\n", - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n", + "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n", " exp_term = np.exp(self.z_o)\n", - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:44: RuntimeWarning: invalid value encountered in divide\n", + "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/1630775253.py:44: RuntimeWarning: invalid value encountered in divide\n", " self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)\n" ] }, @@ -2148,11 +2346,11 @@ "name": "stderr", "output_type": "stream", "text": [ - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp\n", + "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/953065564.py:4: RuntimeWarning: overflow encountered in exp\n", " return 1/(1 + np.exp(-x))\n", - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n", + "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n", " exp_term = np.exp(self.z_o)\n", - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:44: RuntimeWarning: invalid value encountered in divide\n", + "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/1630775253.py:44: RuntimeWarning: invalid value encountered in divide\n", " self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)\n" ] }, @@ -2170,11 +2368,11 @@ "name": "stderr", "output_type": "stream", "text": [ - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp\n", + "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/953065564.py:4: RuntimeWarning: overflow encountered in exp\n", " return 1/(1 + np.exp(-x))\n", - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n", + "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n", " exp_term = np.exp(self.z_o)\n", - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:44: RuntimeWarning: invalid value encountered in divide\n", + "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/1630775253.py:44: RuntimeWarning: invalid value encountered in divide\n", " self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)\n" ] }, @@ -2192,216 +2390,25 @@ "name": "stderr", "output_type": "stream", "text": [ - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp\n", + "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/953065564.py:4: RuntimeWarning: overflow encountered in exp\n", " return 1/(1 + np.exp(-x))\n", - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n", + "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n", " exp_term = np.exp(self.z_o)\n", - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:44: RuntimeWarning: invalid value encountered in divide\n", + "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_92604/1630775253.py:44: RuntimeWarning: invalid value encountered in divide\n", " self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)\n" ] }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 1.0\n", - "Lambda = 0.1\n", - "Accuracy score on test set: 0.07777777777777778\n", - "\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp\n", - " return 1/(1 + np.exp(-x))\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 1.0\n", - "Lambda = 1.0\n", - "Accuracy score on test set: 0.10555555555555556\n", - "\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp\n", - " return 1/(1 + np.exp(-x))\n", - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n", - " exp_term = np.exp(self.z_o)\n", - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:44: RuntimeWarning: invalid value encountered in divide\n", - " self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 1.0\n", - "Lambda = 10.0\n", - "Accuracy score on test set: 0.07777777777777778\n", - "\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp\n", - " return 1/(1 + np.exp(-x))\n", - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n", - " exp_term = np.exp(self.z_o)\n", - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:44: RuntimeWarning: invalid value encountered in divide\n", - " self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 10.0\n", - "Lambda = 1e-05\n", - "Accuracy score on test set: 0.07777777777777778\n", - "\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp\n", - " return 1/(1 + np.exp(-x))\n", - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n", - " exp_term = np.exp(self.z_o)\n", - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:44: RuntimeWarning: invalid value encountered in divide\n", - " self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 10.0\n", - "Lambda = 0.0001\n", - "Accuracy score on test set: 0.07777777777777778\n", - "\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp\n", - " return 1/(1 + np.exp(-x))\n", - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n", - " exp_term = np.exp(self.z_o)\n", - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:44: RuntimeWarning: invalid value encountered in divide\n", - " self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 10.0\n", - "Lambda = 0.001\n", - "Accuracy score on test set: 0.07777777777777778\n", - "\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp\n", - " return 1/(1 + np.exp(-x))\n", - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n", - " exp_term = np.exp(self.z_o)\n", - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:44: RuntimeWarning: invalid value encountered in divide\n", - " self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 10.0\n", - "Lambda = 0.01\n", - "Accuracy score on test set: 0.07777777777777778\n", - "\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp\n", - " return 1/(1 + np.exp(-x))\n", - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n", - " exp_term = np.exp(self.z_o)\n", - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:44: RuntimeWarning: invalid value encountered in divide\n", - " self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 10.0\n", - "Lambda = 0.1\n", - "Accuracy score on test set: 0.07777777777777778\n", - "\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp\n", - " return 1/(1 + np.exp(-x))\n", - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n", - " exp_term = np.exp(self.z_o)\n", - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:44: RuntimeWarning: invalid value encountered in divide\n", - " self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 10.0\n", - "Lambda = 1.0\n", - "Accuracy score on test set: 0.07777777777777778\n", - "\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp\n", - " return 1/(1 + np.exp(-x))\n", - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n", - " exp_term = np.exp(self.z_o)\n", - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/1630775253.py:44: RuntimeWarning: invalid value encountered in divide\n", - " self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 10.0\n", - "Lambda = 10.0\n", - "Accuracy score on test set: 0.07777777777777778\n", - "\n" + "ename": "KeyboardInterrupt", + "evalue": "", + "output_type": "error", + "traceback": [ + "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[0;31mKeyboardInterrupt\u001b[0m Traceback (most recent call last)", + "Cell \u001b[0;32mIn[8], line 11\u001b[0m\n\u001b[1;32m 8\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m j, lmbd \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28menumerate\u001b[39m(lmbd_vals):\n\u001b[1;32m 9\u001b[0m dnn \u001b[38;5;241m=\u001b[39m NeuralNetwork(X_train, Y_train_onehot, eta\u001b[38;5;241m=\u001b[39meta, lmbd\u001b[38;5;241m=\u001b[39mlmbd, epochs\u001b[38;5;241m=\u001b[39mepochs, batch_size\u001b[38;5;241m=\u001b[39mbatch_size,\n\u001b[1;32m 10\u001b[0m n_hidden_neurons\u001b[38;5;241m=\u001b[39mn_hidden_neurons, n_categories\u001b[38;5;241m=\u001b[39mn_categories)\n\u001b[0;32m---> 11\u001b[0m \u001b[43mdnn\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mtrain\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 13\u001b[0m DNN_numpy[i][j] \u001b[38;5;241m=\u001b[39m dnn\n\u001b[1;32m 15\u001b[0m test_predict \u001b[38;5;241m=\u001b[39m dnn\u001b[38;5;241m.\u001b[39mpredict(X_test)\n", + "Cell \u001b[0;32mIn[6], line 99\u001b[0m, in \u001b[0;36mNeuralNetwork.train\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m 96\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mY_data \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mY_data_full[chosen_datapoints]\n\u001b[1;32m 98\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mfeed_forward()\n\u001b[0;32m---> 99\u001b[0m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mbackpropagation\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n", + "Cell \u001b[0;32mIn[6], line 64\u001b[0m, in \u001b[0;36mNeuralNetwork.backpropagation\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m 61\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39moutput_weights_gradient \u001b[38;5;241m=\u001b[39m np\u001b[38;5;241m.\u001b[39mmatmul(\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39ma_h\u001b[38;5;241m.\u001b[39mT, error_output)\n\u001b[1;32m 62\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39moutput_bias_gradient \u001b[38;5;241m=\u001b[39m np\u001b[38;5;241m.\u001b[39msum(error_output, axis\u001b[38;5;241m=\u001b[39m\u001b[38;5;241m0\u001b[39m)\n\u001b[0;32m---> 64\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mhidden_weights_gradient \u001b[38;5;241m=\u001b[39m \u001b[43mnp\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mmatmul\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mX_data\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mT\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43merror_hidden\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 65\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mhidden_bias_gradient \u001b[38;5;241m=\u001b[39m np\u001b[38;5;241m.\u001b[39msum(error_hidden, axis\u001b[38;5;241m=\u001b[39m\u001b[38;5;241m0\u001b[39m)\n\u001b[1;32m 67\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mlmbd \u001b[38;5;241m>\u001b[39m \u001b[38;5;241m0.0\u001b[39m:\n", + "\u001b[0;31mKeyboardInterrupt\u001b[0m: " ] } ], @@ -2430,8 +2437,10 @@ }, { "cell_type": "markdown", - "id": "a6e305f6", - "metadata": {}, + "id": "96089af8", + "metadata": { + "editable": true + }, "source": [ "## Visualization" ] @@ -2439,54 +2448,12 @@ { "cell_type": "code", "execution_count": 9, - "id": "d920e285", - "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp\n", - " return 1/(1 + np.exp(-x))\n", - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp\n", - " return 1/(1 + np.exp(-x))\n", - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp\n", - " return 1/(1 + np.exp(-x))\n", - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp\n", - " return 1/(1 + np.exp(-x))\n", - "/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_87605/953065564.py:4: RuntimeWarning: overflow encountered in exp\n", - " return 1/(1 + np.exp(-x))\n" - ] - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAyIAAANdCAYAAABoOchKAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAABdaklEQVR4nO3debhVZdk/8O+R6aCAioXggCIqiGlooFKgqGWT5fj+ysS0xCEHylnLLJHSV1ESDc0ErYyw0nBO1KzUlHBITQLNmWRwQhSQ4Zzz+wM5b0dA2bBZ+3DO53Nd+7rgWWvt596r1ZZ73/d6VlVdXV1dAAAACrROpQMAAACaH4kIAABQOIkIAABQOIkIAABQOIkIAABQOIkIAABQOIkIAABQOIkIAABQOIkIAABQOIkIsFY788wz06NHjw98HXbYYas1x2WXXZYePXqs8WNW1wMPPJAePXrkS1/6UqHzAsCqqKqrq6urdBAAq+qll17KG2+8Uf/3UaNGZfLkybn88svrx9q1a5ett956leeYMWNGZsyYkd69e6/RY1bXySefnGeeeSZPP/10xo4dm0984hOFzQ0ApWpZ6QAAVkfXrl3TtWvX+r937NgxrVu3LmsC0Llz53Tu3HmNH7M65syZk7vvvjvnnntufvazn2XcuHESEQAaNa1ZQLNw4403plevXvnd736XT33qU9lll13y73//OzU1Nbnqqquy7777Zscdd0zv3r3z1a9+NQ899FD9se9vszrssMPyve99L1dddVUGDhyYHXbYIV/96lfzxBNPrNYxSfLnP/85Bx54YHbcccd89rOfza233prPfOYzueyyyz7w891yyy1ZvHhxBgwYkC9/+cu58847M3v27GX2e+6553LCCSdkl112Sd++fXPMMcfk2Wefrd/+zjvv5LzzzsuAAQPSu3fvHHTQQfnzn/9cv71Hjx7LxPL+z3rmmWfm8MMPzw9+8IPsvPPO+cIXvpCampq88cYbOffcc7PnnnvmYx/7WHbZZZccf/zxmTZtWoP3Gz9+fA444IB8/OMfz8CBA3PxxRdn4cKFeeaZZ9KjR49cf/31DfafPn16tttuu9x8880feI4AaFwkIkCzUVNTkzFjxuRHP/pRzjrrrHTv3j3Dhw/PqFGj8pWvfCVXX311zjvvvMyePTvf/va3M3/+/BW+15133pl77rknZ599di655JK89tprOfHEE1NTU7PKxzz00EM57rjj0qVLl1x22WU59NBD84Mf/CDTp0//0M92ww03ZMCAAfnIRz6S/fffP4sWLcof/vCHBvvMnDkzX/nKV/LCCy/khz/8YS666KK89tprOfzwwzN79uzU1NTkm9/8Zm655ZYcc8wxGTVqVLbaaqscf/zxefjhh1fyLC/x8MMPZ/r06fnpT3+aU045Jeuss06OOeaYPPDAAzn11FMzevTonHDCCXnwwQfzgx/8oP64X//61znjjDOy/fbb5/LLL8/RRx+dX/3qVxk2bFi22WabfPzjH89NN93UYK7x48dn3XXXzT777FNSjABUltYsoFk59thjM3DgwPq/z5o1KyeddFKDG9rbtGmTE088MVOnTl1hi9fixYszevTotGvXLkkyd+7cnHHGGfnXv/6Vj33sY6t0zGWXXZZtttkml19+eaqqqpIkG220UU4++eQP/ExTp07NU089lZEjRyZJNtlkk+y22265/vrr841vfKN+v2uvvTYLFy7MNddck49+9KNJkp49e+aQQw7J448/ntra2jz++OP56U9/mk9/+tNJkt122y0vv/xyHnroofTp0+cD43j/Zx06dGh9e9rMmTPTtm3bnHHGGfXvs+uuu+all16qr3DU1tbWzz1s2LD695o/f35uu+22LFq0KAcddFB+8IMf5OWXX87mm2+eZEki8sUvfjHV1dUrHR8AlaciAjQr2223XYO/X3zxxTn88MPzxhtv5OGHH84NN9xQ3+KzcOHCFb7P1ltvXZ9QJMnGG2+cJB9YRfmgYxYuXJjHHnss++yzT30SkiSf+9zn0rLlB/9mdMMNN6RDhw7p06dP5syZkzlz5uSzn/1snn/++QYtZo888kh69+5dn4QkS+5luffee7PHHnvkkUceSatWrbLXXnvVb19nnXUybty4nHDCCR8Yw/ttsMEGDe6R2XjjjfPLX/4yn/jEJzJt2rQ88MAD+dWvfpVHH320/jw///zzef311/OZz3ymwXsdeeSRufHGG9OqVav6hGNpVeTRRx/NCy+8kAMOOKCk+ACoPBURoFlZd911G/z9ySefzLnnnpsnn3wybdu2zdZbb51NNtkkSfJBiwq2bdu2wd/XWWfJ7zq1tbWrdMzS1qiNNtqowT4tWrTIBhtssML3XLRoUW6++ebMmTMnn/zkJ5fZPm7cuOy2225JktmzZ2ezzTZb4XvNnj07G2ywQX1cq2O99dZbZuzmm2/OJZdckunTp2eDDTbIdttt16CKsfSelvefg//Wrl27fO5zn8vNN9+cE044IePHj0+3bt2y0047rXbMABRLIgI0W++8804GDx6cHj165LbbbstWW22VddZZJ3/5y19y5513FhrLRhttlFatWuW1115rML40SVmRe++9N2+++WbOO++8bLHFFg22/eY3v8ndd9+d119/PRtttFHat2/fYKnjpR588MFsttlmad++fWbPnp26uroGVZnJkyenrq4u22+/fZIscx/MvHnzPvTzPfzwwznjjDNy2GGH5cgjj6yvBl144YV55JFHkiQdOnRIkmVifPPNNzN58uTstNNOWXfddXPQQQflD3/4Q5544onceeedOfLIIz90fgAaH61ZQLP13HPPZfbs2fn617+erbfeur4S8Ne//jXJB1c3yq1FixbZeeedc8899zQY/9Of/pTFixev8LgbbrghnTt3zv/8z/9k1113bfA67LDDsmjRotxwww1Jkj59+uTxxx9v8A/9119/PYMHD85f/vKX9OnTJ4sWLar//MmSqtBZZ52Vn/3sZ0mWVCRmzpzZIIZHH330Qz/fY489ltra2px44on1SUhNTU3+9re/JVlyrrfaaqtsuOGGuffeexsce9NNN+Xoo4/OokWLkiR9+/bNlltumYsuuihvv/129ttvvw+dH4DGR0UEaLa6deuWdu3a5corr0zLli3TsmXL3Hnnnfn973+f5IPv91gThgwZksMOOyxDhgzJwQcfnFdeeSWXXnppkjSoUCw1a9as3HfffTn88MOXu/0Tn/hEunbtmuuvvz5HHXVUjjjiiIwfPz6DBw/OMccck1atWuWKK65I586d86UvfSnt2rXLTjvtlDPPPDPf+c53svnmm+emm27Ks88+m/POOy9JMnDgwNx22235+Mc/ni222CI33nhjXnzxxQ/9bDvuuGOSZOjQoTnooIPy1ltv5de//nWmTJmSZElVpV27djnxxBMzdOjQbLTRRtlrr73y/PPPZ+TIkTn00EOz/vrr17/fQQcdlIsvvji77757fWIDwNpFRQRottq3b59Ro0alrq4u3/72t3P66afnlVdeyXXXXZf11luv5CVrV1efPn1y2WWX5fnnn89xxx2Xa665Jt///veTLP+ei/Hjx6empiZf+MIXVvie++23X6ZNm5b77rsvXbp0ydixY9OpU6eceeaZOeuss9KlS5f84he/yPrrr58WLVrk5z//efbZZ59ceumlOf744/Piiy9mzJgx9YnEWWedlT333DP/+7//myFDhmTdddfNKaec8qGfbdddd80555yTxx57LEcddVQuuOCCbLLJJrn88suTpL4969BDD80FF1yQiRMn5phjjsm1116bo446KqeffnqD99tjjz2SJAceeOBKnFkAGqOqug+6GxOAwtxzzz3p3Llz/b0YSfLMM89k3333zahRo7L33ntXMLrG5aqrrsq1116bP//5z2ndunWlwwFgFWjNAmgk7r///tx+++059dRT061bt8ycOTNXXHFFttpqq/Tv37/S4TUKf/jDH/L0009n7NixOe644yQhAGsxiQhAI3HGGWekuro6V1xxRWbNmpUNNtggAwYMyCmnnJI2bdpUOrxGYcqUKRk3blw+85nP5Jvf/GalwwFgNWjNAgAACudmdQAAoHASEQAAoHASEQAAoHASEQAAoHDNctWsz3X4RqVDaNKqOrSvdAhNXt3b71Q6hCatqtoKVTQBVlpbo2o3Wr/SITR5dz52bqVDWKHaGdtWOoTlWqfz05UOoSQqIgAAQOEkIgAAQOGaZWsWAACsqtrUVjqE5VrbKgxrW7wAAEATIBEBAAAKpzULAABKUFPXOFuz1rZ/2KuIAAAAhZOIAAAAhVvbKjgAAFBRtamrdAhNgooIAABQOIkIAABQOK1ZAABQgsb6QMO1jYoIAABQOIkIAABQOK1ZAABQgpo6q2aVg4oIAABQOIkIAABQOK1ZAABQAg80LA8VEQAAoHASEQAAoHBaswAAoAQ1WrPKQkUEAAAonEQEAAAonNYsAAAogVWzykNFBAAAKJxEBAAAKJzWLAAAKEFNndasclARAQAACicRAQAACqc1CwAASlBb6QCaCBURAACgcBIRAACgcFqzAACgBDUeaFgWKiIAAEDhJCIAAEDhtGYBAEAJanRmlYWKCAAAUDiJCAAAUDitWQAAUAIPNCwPFREAAKBwEhEAAKBwWrMAAKAENamqdAhNgooIAABQOIkIAABQOK1ZAABQgloPNCwLFREAAKBwEhEAAKBwWrMKtvNe2+eIcw5K156bZPasObnl53/KDZf9caWO7b5j11x67/dz5E5nZuZLrzfY9pmvfSoHDflcunTrlFenvZ5bfv6n3HTl3WviIzRqO+/eM4ef/sV07dEls199O7f+4r7c8LM/rdSx3bffLD+59ZQcOeC8zJr2RoNtm3XvlCO/t1927LdNampq8uRD/87Ph47PjPf979Ac7LzX9jni+wcuuYZfXXoN37lSx3bfsWsu/dPZOXLns5Z/DZ/42feu4TeWXMM/a4bX8B7b5fCzvpyuPTbJ7Nfm5NYxf8kNV6zceej+sc3ykz+emSP7nZNZLze8hn/12I/zkU02XOaYr2x3aua8Mbcssa8tnOM1a+cBPZZ8D2/becn38C/vzw1X3btSx3bfftP85OaTc+QeP6r/Hu60Wcf84m/nrPCYCb+dmBGn/qYssa8tdt6te444Ye9ssdVHM/uNubnl+r/n97/62wr3b9WqRQ467JP59L4fz0c7r59XZ87JvXc8kevH3J/Fi2vq9xvw6V75nyP6Z/MtP5K5b7+bxyY+l9Ej78rsZnT9riyrZpWHRKRAPftulXN/+5389ca/5xfDbszH+m2bI8/7n7RouU5+O+L2Dzx2i+02zdDffSctWy37P9nnvr57vnP5N/LbEbfn0T/9Mz36bJWjf/zVVK/XJtdffNua+jiNTs+dt8wPrz06f73lsfxy+O3Zvu9W+eb3vpx1Wq6T3/30g/+RsUWPLjn3F8ekZasWy2z7SJcNcvH4kzLt2Vm54Phr06Zt6xx++hfzo7HH5VufviAL3120pj5So9Ozz1Y59/pvv3cN/yEf67dNjhz6P2nRssXKXcO/XdE1PCDfuewb+e1Pbs+jf3rqvWv4K6lu18yu4U90yw+vOy5/vemR/PJ/b8n2u26db55zwJJr+LIJH3jsFj03ybm/Pn6513CHjuvlI5tsmKt/eEOe+vuzDba989b8sn6Gxs45XrN67rRFfnjNUfnrrf/1PfzdLy05v6Pu+cBjt9i2c8699uhlzu+bs97KSfuNWGb/fQ/vn9333SkTrn+orJ+hseu5w2YZOvJr+cudT+WXo/6U7Xt3zZHf+UzWablOfnvN/cs95lunfz57f/Hj+fXP/5Knn/pPtu21aQ49eo906rJBRpx7U5Jkj89+LN+94H9y2+8n5drL70nHj7TL14/bKxdedUSO/9rPsmjh4iI/Js2ERKRAh333gDz7xIu56OifJ0keufufadGyRb5yyr4Zf8Vdy/0HbctWLfLlYz6dr3/vgCxcsPx/8H7llC/mvj9Mypgf/C5J8o+//Cubbd05+x3z6Wb1j7hBJ38+zz41LcO//askySN//ldatmqRr56wT24a/ZcVn99v7J7DTv1CFi5Y/pfsoFM+n7lz5ue7X708C957j5kvv54fjDk62+y4eZ76+3Nr7kM1Mod9d/88+8RLueiYq5Mkj9zzz7Ro1SJfOfmLH34Nf3f/D7iG98194ydlzA9+n+S9a7j7xtnv6L2b1zV82r559p8vZ/gJ1yZJHrl3clq2XCdf/fbnctPP713x+T1yYA4740srTIq3+tjmSZK/3f6PTH/xtTUW/9rAOV6zlnwP/yfDv/PrJMkjf5my5Hv4+M/kptF/Xe53QMtWLfLlIwbksFM+v9zv4UULazLlsRcbjG29w2bZfd+d8osLb8tTk55fMx+mkTrs2D3z7JQZuej7NyZJHv7bv9OyZYt89ZsDMn7sQ8ucw/brt83nD/xERl96d37/yweSJP/4+5JzduS3P5MxI+/KW2/Oy1e/OSAT73s6I390a/2xL7/wWkb+6ujsuvu2uf/uyQV9QpoT94gUpFXrltmhf4/87ZZHG4zff9PDWa9D22zfb5vlHtd3nx1z6Jn7ZdzFt2bMOb9b7j7n/M9PcvX3r28wtmjh4rRq06o8wa8FWrVumR37bZO//fGJBuP33/aPrNu+Otv33Wq5x/Xdq1cOPenzuf6yuzLmxzcvd59Pff7jmXD9Q/VJSJI888TLGdTn+80qCam/hm9dhWv4jC9n3CW31SfL77fkGv5tg7FFi2rSqrqZXcOf3CZ/u/0fDcbvv/WxrNu+bbbfpftyj+v76Y/l0FO/mOsv/WPGDBu/3H26b79Z5r09v1n/Azlxjte0Vq1bZMfdts7f7lzB9/Au3ZZ7XN89e+XQ73wu119+d8acf8tKzXX8eQfnpWdm5g9X/3l1w16rtGrVIjv22TIP3PuvBuP33f1U1mtXnY/17rrMMeuu1ya3/f7hPPSXKQ3GX37h1SRJ5003TFVVVR6b+FzuuPHh9+2z5HreZLOO5fwYTUJNqhrla20jESlI5y0/mtZtWuU//57ZYPyV55b8fbNtuiz3uKcffT6H73Baxg2/NTX/1cf5315+enp9v327DdfL576+ez59yKdy69Urd29EU9C560Zp1aZl/vPcrAbjr7z3RbtZ907LPe7px1/K4f1+mHGXTVju+d14845pt/66mfWfN3PcsP/Jb/95fm7698U5Z/RR+UiXDcr+ORqz/7uGZzQYf+W9c77Z1p2Xe9zTjz6fw3c8/b1ruHa5+yx7DQ/Ip7/6ydx69cr1lTcFnbf4SFq1aZX/PPu+a/j5967hrTde7nFPP/ZCDu9zdsb95I8r/I7Y6mOb5e035+V7o4/O75+5JDc+NyJnXnVkNuzUobwfopFzjteszl0/8t738KsNxl95LznbbKsP+B7+5NCMu/yuFX5H/Lc9vrRTeu68ZX527h9S28zWUO282YZp3bpl/vNiw3vsXnnvfqXNtvzIMsfMfGV2Lj//tkx73zGfHLhdFi1anP+8+Hrq6upy1SV35sE/T224z549kyQvvu//M1AujaY1a/HixZkwYUImTZqU6dOnZ+HChWnbtm023njj9O3bN/vss09atFi2L3dtsd76bZMk895u2Cs87+13kyTrtq9e7nGvT5+90nNst0v3jLj77CRL/vF3w+UrdwNxU7Beh6Xn990G4/PeWZAkWbfdCs7vjLc+8H3X36hdkuSbZ305U//xYi447hdZ/yPt8o0zv5QLfntCjt/nwiyYv3B1w18rrPAcL72G39v+fiVdw327Z8Td30vSjK/hd95/DS/9jljB+f2QazhZ0ja0UZcNcsd192f8VX9K1207Z9Dp++bC8SfnhE//OAvmNbNr2DleI9Z7779jy57fD/kenvnh5/e/HXTsXnlq0nN58qF/r0KUa7f13juH8+YuaDA+773ra9312qzU+3xyz5759Jc+npuv/3veed93+lJdNtswR5302fx7yvT8/f5nViNqWLFGkYhMmzYtRx55ZGbOnJlevXqlU6dOWX/99bNgwYJMmTIlN954Yy677LJcffXV2WSTTSod7iqpqvrg4lNdGX7VmfnS6znt8xek8xYfyde/f2BG3PW9nDDgh83iH8pVVR9cjqytW7Xz2+q9G6tnv/Z2hh01OnXvvc/0F17LiJtPzl4H9skdv17xSiVNSdU6H3yOy3INv/x6TvvCBem8xUfz9bMPyIgJ380Ju5/rGk5SW/vhvxSvyKWnXJfaxbV5+h9L+uyfmvjvvDjllVx862n59P/sltt+8ddVfu+1iXO8Zn3Yd8Sqfg//t+0+sWW22WHznHvk1av9Xmujdcpwjj+113Y588cH5al/vJSrf3LXcvfZfMuP5MejDkvN4toMO+36+v/28X9q69a+NqjGqFEkIkOHDs1mm22W3//+92nfvv0y2+fMmZOTTjopQ4cOzZVXXlmBCFffvDnzkiRt3/eL0NJfkefOWf1VVd6YMTtvzJidJx+YmukvvJrhfzwr/ffvk3t+0/T/oTz3vUrT+39xW1ppmjdn+b/4fJilv+xNundygy/iKY++kHfempfu22+2Su+7Npr33jW64mt43mrP8X/X8NNLruE7zkz//frknnHN6Bp+3y+aq3sNJ8mUh5e9mXfypOfyzlvz0m37TVf5fdc2zvGaNXdpdfT957fd6p/fpfp/oXfenj03k+5tnjdOLz3Hbddt3WB8vffO+dLq04occGi/HHXSPnni4Rdy7sm/We5KWDt+Ysucc/FXMn/ewpxx9LWZPu3NMkUPy2oU94hMmjQpp59++nKTkCTp0KFDTjvttEyaNKngyMrnlednpWZxTTZ5X4/s0r+/PPWVVXrf6vXaZOD/7JYu73vffz++5Fe5jTpvsErvu7aZ/uJrqVlcky7v64/d5L2/v/S++xpKed/a2tq0ar1szt6iZYsGN7A3dSu8hrstvYanr9L7Vq/XJgMP3nXF13AzuRdn+guvLrmGu73//H40SfLSM6t2ftdtX519DumXLXo2rCZXVVWlZauWeev1d1Yt4LWQc7xmffj38MzlHVaSXfbulQfv/OdK3UvSFL0y7c0l38Obb9RgfJPNl9xM/tL77s/5b986/fM59tTP5a8T/pmzT7gu85fTLjjwcx/Lj684LK/OnJPvHH51/c3qsKY0ikSkffv2mTnzg7+gXnnllVRXL7+/dG2waMHiPPnA0/nUlz/RYLz/l/vkndnzMvWRVVt+sGZxTb5z2RH5nyGfbzD+ib0+liR5/qlpqxbwWmbRgsV5cuKz+dTnP95gvP8Xeuedt+bl6fct/biy3p23MP98733/Oxnp/alt03a9Nss8L6ApW7RgcZ7829P51Jd2bjDef79PlOka/lyD8U/stX2S5Pl/vrxqAa9lFi1YnCcf+nc+9cXeDcb7f3Gn967hF1btfRcuzrd+/NV8ZchnG4zv9rkdU71u6zzxwNOrGPHaxzles+q/hz/3/u/hj+edt+bXt62tqnbrr5vNtuqUpx5uPqsVvt+ihYvz5KMv5lN7b9dgvP/evfLO2/Mz9an/LPe4b5z46ex/yG654Vd/ywXfvaHBQwyX6tt/m5x+3oGZ/PjLOeWbY/L6q2+vkc/QVFR6daymsmpWo2jNOvjgg3PmmWfm29/+dnbbbbd06dIlrVu3zsKFCzNz5sz8/e9/z/Dhw3PwwQdXOtTV8puLbsn5N5+a7/3iuNx53X3ptevWOfjbn8uYH/w+C+YvzLrtq9O15yaZ/tyreev1lfsCWLRgcX57yW0Z9N39M/u1OXn8r1Oy1Q6b59Az9sujf3oqkyY88eFv0kSMu/TO/Hjc8fnuld/IhOsfynaf6JaDjt0r15x/Sxa8uyjrtqtO1207Z/oLr+WtN1b+F8przr8lF/5uSIb+8pjc8LM/ZYOPtM83v/vlTHn0hTw04ck1+Ikan99cdEvOv+nUfO8X38qdv7p/yTU85HMZ88P/uoZ7bJLpz5d6Dd+eQd/dL7NffTuP3/evbPWx967he5/KpLuazzkeN+KO/Ph3Q/Ldnw/OhN88mO36bpWDjv9Mrhk2Pgvmv3cN9+iS6S+8utK/si9asDi/u+zOHHbGl/Lmq3My6Z6nsuV2m2TQqfvmb3f8I4/fP/XD36QJcY7XrHGX3ZUfj/1WvnvFEZlw/cRs94ktc9Axe+aaC25973u4Tbpu0znTX3wtb5X4tO5uPZesLvnSM6tfWVmbjb36r7ngyq/nexf+v9x506Pp9fGuOfjwT2bMyLuXnOP12qTrVh/N9Glv5K0352WrbTvn/x3xqUz957T89a6n0nOHhi3FLz33ahYtqslJ53w58+YtzLir/5quW320wT6vzZyT12bNKfJj0kw0ikTkxBNPzDrrrJMLL7ww8+Yt22e+3nrr5dBDD823v/3tCkRXPo//9V8ZNuinOey7++ecsSfm9VfezNVn/zY3vrcy0NYf3yIX3n5mLj726tw19oGVft+xF96S2a+9nS8fvXcOOvFzeeu1t3PbmHtz3fnj19AnaZwe/9sz+dHRYzLolM/nnKuPymszZmf0sJty41VLloDtvsNmufB3Q3LxSdfl7t/9faXfd8qjL+SM/3dZDj9j33zvqiOzYP7CPHjnk7n6vPHNbunIx/86JcMOG5XDztov54w9Ia9Pn52rv/+7htfwbWfk4m+NLu0avuiWzH797Xz5qL1y0ImfXXINX/Pn5ncN3z81P/rmVRl0+r4559pj8tqMtzL63Btz45VLnkjdfcfNc+EfTs7FQ36Ru0t4mvRvRtyRt15/J/t+c4988fDd8/abc3P7L/6a64Y3n4dFLuUcr1mP/+2Z/OiYazLo5M/nnJ8fmddmzs7oH92cG3/+5yRJ949tngt/e0IuPnls7v79yn8PJ8kGH13Svv3OW6t/P9ra7PFJz+e8U6/PYcfumR9cckhenzUnV//krtzwqyX30m3ds0suuvobGX7OH3LXLf9I/723yzrrrJMeH9ssl/7yqGXe77TB16Rqnaps9NElS02ff+Xhy+zzqyvvzXU/+/Ma/Vw0T1V1jWgphEWLFuVf//pXZs6cmfnz56e6ujqdO3dOz54907p16w9/g5X0uQ7fKNt7sayqDsu/14fyqXu7efScV0pV9cotgQmNWhvX8ZpUu9H6lQ6hybvzsXMrHcIKTXppy0qHsFx9u75Q6RBK0igqIku1atUqO+64Y6XDAAAA1rBGcbM6AADQvDSqiggAADR2HmhYHioiAABA4SQiAABA4bRmAQBACdbGhwc2RioiAABA4SQiAABA4bRmAQBACWrq/JZfDs4iAABQOIkIAABQOK1ZAABQglq/5ZeFswgAABROIgIAABROaxYAAJTAAw3LQ0UEAAAonEQEAAAonNYsAAAogQcaloezCAAAzVxtbW1GjhyZAQMGpHfv3jnqqKPy8ssvr3D/F154IUcffXT69OmT3XffPSNHjszixYtLmlMiAgAAzdyoUaMyduzYnHfeeRk3blxqa2szePDgLFy4cJl933rrrRx66KGZP39+fvGLX+SSSy7JHXfckXPOOaekOSUiAABQgtpUNcrXqlq4cGHGjBmTIUOGZODAgenZs2dGjBiRGTNmZMKECcvs/4c//CHz5s3LpZdemu233z59+vTJsGHDcsMNN2TatGkrPa9EBAAAmrEpU6Zk7ty56devX/1Yhw4d0qtXr0yaNGmZ/V988cVstdVW6dixY/1Yr169kiQPP/zwSs/rZnUAAGgC9t577w/cfs899yx3fMaMGUmSLl26NBjv1KlT/bb3j8+aNSs1NTVp0aJFkuQ///lPkuT1119f6XhVRAAAoAQ1WadRvlbV/PnzkyStW7duMN6mTZssWLBgmf0///nPZ/bs2Tn//PMzb968vPbaaxk2bFhatmyZRYsWrfS8KiIAANAErKji8WGqq6uTLLlXZOmfk2TBggVp27btMvtvueWWufTSS3POOefk17/+ddZdd92ceOKJ+fe//5327duv9LwSEQAAaMaWtmTNmjUrXbt2rR+fNWtWevTosdxj9tprr+y1116ZNWtWNthggyxevDgXXHBBNt9885WeV2sWAACUoKZunUb5WlU9e/ZMu3btMnHixPqxOXPmZPLkyenbt+8y+z/88MM57LDDsnjx4nTq1CmtW7fOhAkT0rZt2+y8884rPa+KCAAANGOtW7fOoEGDMnz48HTs2DGbbrppLrroonTu3Dn77LNPampq8sYbb6R9+/aprq7OVlttlalTp+Z///d/8/Wvfz1Tp07NsGHDcswxx6Rdu3YrPa9EBAAAmrkhQ4Zk8eLFOfvss/Puu++mb9++GT16dFq1apVp06Zl7733zvnnn58DDzwwHTt2zJVXXpkLLrgg++67bz760Y/mhBNOyBFHHFHSnFV1dXV1a+bjNF6f6/CNSofQpFV1WPmblFg1dW+/U+kQmrSq6jaVDgFWXxvX8ZpUu9H6lQ6hybvzsXMrHcIK3fRc70qHsFz7bfWPSodQEveIAAAAhZOIAAAAhXOPCAAAlKCmrqrSITQJKiIAAEDhJCIAAEDhtGYBAEAJavyWXxbOIgAAUDiJCAAAUDiJCAAAUDj3iAAAQAlq6/yWXw7OIgAAUDiJCAAAUDitWQAAUALL95aHswgAABROIgIAABROaxYAAJSgpq6q0iE0CSoiAABA4SQiAABA4bRmAQBACWr9ll8WzTIRmXJ5z0qH0KS1Xm9hpUNo8ha+s2mlQ2jSOn707UqH0OQtWNQs//NTqLmz21Y6hCbtmL5/qXQIsNaTzgEAAIXzkxQAAJSgps5v+eXgLAIAAIWTiAAAAIXTmgUAACWojQcaloOKCAAAUDiJCAAAUDitWQAAUAKrZpWHswgAABROIgIAABROaxYAAJSgxm/5ZeEsAgAAhZOIAAAAhdOaBQAAJait80DDclARAQAACicRAQAACqc1CwAASmDVrPJwFgEAgMJJRAAAgMJpzQIAgBLU1vktvxycRQAAoHASEQAAoHBaswAAoAQ18UDDclARAQAACicRAQAACqc1CwAASmDVrPJwFgEAgMJJRAAAgMJpzQIAgBJYNas8VEQAAIDCSUQAAIDCac0CAIASWDWrPJxFAACgcBIRAACgcFqzAACgBDVas8rCWQQAAAonEQEAAAqnNQsAAEpQ64GGZaEiAgAAFE4iAgAAFE5rFgAAlMCqWeXhLAIAAIWTiAAAAIXTmgUAACWorbNqVjmoiAAAAIWTiAAAAIXTmgUAACWo8Vt+WTiLAABA4SQiAABA4bRmAQBACayaVR4qIgAAQOEkIgAAQOG0ZgEAQAlq/ZZfFs4iAAA0c7W1tRk5cmQGDBiQ3r1756ijjsrLL7+8wv1ff/31nHLKKdltt92y66675qSTTsrMmTNLmlMiAgAAzdyoUaMyduzYnHfeeRk3blxqa2szePDgLFy4cLn7f+c738krr7ySa665Jtdcc01eeeWVHH/88SXN2Whasw477LBUVa3cCgS//OUv13A0AACwfDVNbNWshQsXZsyYMTn11FMzcODAJMmIESMyYMCATJgwIfvuu2+D/efMmZO///3vueKKK7LddtslSY4++ugcd9xxmT17djbYYIOVmrfRVET69++fhx9+OK+//no23XTTD3wBAADlMWXKlMydOzf9+vWrH+vQoUN69eqVSZMmLbN/dXV11ltvvYwfPz7vvPNO3nnnndx0003p1q1bOnTosNLzNpqKyDHHHJN27drl4osvzs9+9rNsttlmlQ4JAACavBkzZiRJunTp0mC8U6dO9dv+W+vWrXPBBRfknHPOSZ8+fVJVVZVOnTrluuuuyzrrrHydo9EkIkly6KGH5r777suFF16YkSNHVjocAABYRmN9oOHee+/9gdvvueee5Y7Pnz8/yZIE47+1adMmb7311jL719XV5V//+ld22mmnDB48ODU1NRkxYkSOO+64/OY3v0m7du1WKt5GlYgkydChQ/PUU09VOgwAAGgWqqurkyy5V2Tpn5NkwYIFadu27TL733HHHbnuuuty77331icdV155Zfbcc8/8/ve/zxFHHLFS8za6RKRTp07p1KlTpcMAAIC1yooqHh9maUvWrFmz0rVr1/rxWbNmpUePHsvs//DDD6dbt24NKh/rr79+unXrlhdffHGl5200N6sDAMDaoLZunUb5WlU9e/ZMu3btMnHixPqxOXPmZPLkyenbt+8y+3fu3DkvvvhiFixYUD82b968TJs2LVtuueVKzysRAQCAZqx169YZNGhQhg8fnnvuuSdTpkzJSSedlM6dO2efffZJTU1NXn311bz77rtJkv333z/JkmeJTJkyJVOmTMnJJ5+cNm3a5MADD1zpeSUiAADQzA0ZMiQHH3xwzj777BxyyCFp0aJFRo8enVatWmX69Onp379/br/99iRLbqUYO3Zs6urqcvjhh+cb3/hGWrVqlbFjx6Z9+/YrPWeju0cEAAAas5o0zlWzVkeLFi1y2mmn5bTTTltm22abbZapU6c2GOvevXuuvPLK1ZpTRQQAACicRAQAACic1iwAAChBY32g4dpGRQQAACicRAQAACic1iwAACjB6jw8kP/jLAIAAIWTiAAAAIXTmgUAACWobYIPNKwEFREAAKBwEhEAAKBwWrMAAKAENR5oWBYqIgAAQOEkIgAAQOG0ZgEAQAk80LA8nEUAAKBwEhEAAKBwWrMAAKAEtVbNKgsVEQAAoHASEQAAoHBaswAAoAS10ZpVDioiAABA4SQiAABA4bRmAQBACayaVR4qIgAAQOEkIgAAQOG0ZgEAQAlq6/yWXw7OIgAAUDiJCAAAUDitWQAAUAKrZpWHiggAAFA4iQgAAFA4rVkAAFCC2mjNKgcVEQAAoHASEQAAoHBaswAAoARWzSoPFREAAKBwEhEAAKBwWrMAAKAEWrPKQ0UEAAAonEQEAAAonNYsAAAogdas8lARAQAACicRAQAACtcsW7Pu2/snlQ6hSaupq3QEsHoWRcl9TWsRXxRrWpcW1ZUOoUmbVrOg0iFQQVqzykNFBAAAKJxEBAAAKFyzbM0CAIBVVauFtyxURAAAgMJJRAAAgMJpzQIAgBJYNas8VEQAAIDCSUQAAIDCac0CAIASaM0qDxURAACgcBIRAACgcFqzAACgBFqzykNFBAAAKJxEBAAAKJzWLAAAKIHWrPJQEQEAAAonEQEAAAqnNQsAAEpQpzWrLFREAACAwklEAACAwmnNAgCAEtRGa1Y5qIgAAACFk4gAAACF05oFAAAl8EDD8lARAQAACicRAQAACqc1CwAASuCBhuWhIgIAABROIgIAABROaxYAAJSgKa6aVVtbm8svvzy/+93v8vbbb6dv374555xzsvnmmy+z72WXXZbLL798ue9z4IEH5vzzz1+pOVVEAACgmRs1alTGjh2b8847L+PGjUttbW0GDx6chQsXLrPvN7/5zdx///0NXkceeWTWXXfdHHHEESs9p0QEAACasYULF2bMmDEZMmRIBg4cmJ49e2bEiBGZMWNGJkyYsMz+6623Xj760Y/Wv1599dX88pe/zDnnnJMePXqs9LwSEQAAKEFdXVWjfK2qKVOmZO7cuenXr1/9WIcOHdKrV69MmjTpQ48fOnRo+vTpkwMOOKCked0jAgAAzdiMGTOSJF26dGkw3qlTp/ptK3Lvvffmsccey/jx40ueVyICAABNwN577/2B2++5557ljs+fPz9J0rp16wbjbdq0yVtvvfWB73nNNddkzz33zHbbbVdCpEtIRAAAoARNbdWs6urqJEvuFVn65yRZsGBB2rZtu8LjXnnllUycODFXXXXVKs0rEQEAgCZgRRWPD7O0JWvWrFnp2rVr/fisWbM+8Obzu+++Ox07dsynPvWpVZrXzeoAANCM9ezZM+3atcvEiRPrx+bMmZPJkyenb9++Kzzu4Ycfzi677JKWLVettqEiAgAAJairq3QE5dW6desMGjQow4cPT8eOHbPpppvmoosuSufOnbPPPvukpqYmb7zxRtq3b9+gdWvy5Mk56KCDVnleFREAAGjmhgwZkoMPPjhnn312DjnkkLRo0SKjR49Oq1atMn369PTv3z+33357g2NeffXVbLDBBqs8Z1VdXVPL6T7cy//p8uE7scpqmt0VRVOzKE3rJsTGqEV8UaxpXVpUf/hOrLJpNQsqHUKTt81mr1Q6hBXqe8d3Kx3Cck36/I8rHUJJtGYBAEAJav1gVRZaswAAgMJJRAAAgMJpzQIAgBLUNbEHGlaKiggAAFA4iQgAAFA4rVkAAFCCWq1ZZaEiAgAAFE4iAgAAFE5rFgAAlKCurtIRNA0qIgAAQOEaTSLywgsv5LLLLsuwYcPy17/+dZnt77zzTs4666wKRAYAAJRbo0hEHnnkkey///655ZZbct999+WYY47Jt7/97SxcuLB+n3fffTfjx4+vXJAAAJAlDzRsjK+1TaNIRC6++OIcdNBBmTBhQu6888785Cc/yf3335/jjjsuixcvrnR4AABAmTWKRGTq1Kn5+te/Xv/3z372s/n5z3+eRx55JGeccUYFIwMAANaERpGItGvXLq+//nqDsZ133jkXXXRR7rjjjpx//vkVigwAABqqdAuW1qwy2mOPPXLuuefmH//4RxYtWlQ//ulPfzrf/e5384tf/CJDhw6tYIQAAEA5NYpE5JRTTslGG22UQw45JA8++GCDbYMGDco555yTP/3pTxWKDgAAKLdG8UDD9ddfP2PGjMlLL72UDTfccJntX/va19KvX79MmDChAtEBAMD/qV0L26Aao0aRiCzVtWvXFW7r1q1bjjnmmAKjAQAA1pRG0ZoFAAA0L42qIgIAAI1dXV2lI2gaVEQAAIDCSUQAAIDCac0CAIASrI0PD2yMVEQAAIDCSUQAAIDCac0CAIASaM0qDxURAACgcBIRAACgcFqzAACgBJ5nWB4qIgAAQOEkIgAAQOG0ZgEAQAmsmlUeKiIAAEDhJCIAAEDhtGYBAEApLJtVFioiAABA4SQiAABA4bRmAQBACayaVR4qIgAAQOEkIgAAQOG0ZgEAQAnqrJpVFioiAABA4SQiAABA4bRmAQBACayaVR4qIgAAQOEkIgAAQOG0ZgEAQCm0ZpWFiggAAFA4iQgAAFA4rVkAAFACDzQsDxURAACgcBIRAACgcFqzAACgFFqzykJFBAAAKJxEBAAAKJzWLAAAKEGdBxqWhYoIAABQOIkIAABQOK1ZAABQCqtmlYWKCAAAUDiJCAAAUDitWQAAUAKrZpWHiggAAFA4iQgAAFA4rVkAAFAKq2aVhYoIAABQuGZZEdm0RftKhwAAa9TQ17ardAhN2kNf26HSITR5f3yi0hGwpjXLRAQAAFadVbPKQWsWAABQOIkIAABQOK1ZAABQCqtmlYWKCAAAUDiJCAAANHO1tbUZOXJkBgwYkN69e+eoo47Kyy+/vML9Fy1alIsvvrh+/0GDBuVf//pXSXNKRAAAoBR1jfS1GkaNGpWxY8fmvPPOy7hx41JbW5vBgwdn4cKFy93/hz/8YW688cb8+Mc/zg033JCOHTvmqKOOyttvv73Sc0pEAACgGVu4cGHGjBmTIUOGZODAgenZs2dGjBiRGTNmZMKECcvs//LLL+eGG27Ij370owwYMCDdu3fPsGHD0rp16/zzn/9c6XklIgAA0IxNmTIlc+fOTb9+/erHOnTokF69emXSpEnL7P/AAw+kffv22X333Rvs/6c//anBe3wYq2YBAEAp6hrnAw333nvvD9x+zz33LHd8xowZSZIuXbo0GO/UqVP9tv/2/PPPZ/PNN8+ECRNy1VVXZebMmenVq1fOPPPMdO/efaXjVREBAIBmbP78+UmS1q1bNxhv06ZNFixYsMz+77zzTl588cWMGjUqJ598cq644oq0bNkyX/va1/L666+v9LwqIgAA0ASsqOLxYaqrq5MsuVdk6Z+TZMGCBWnbtu0y+7ds2TLvvPNORowYUV8BGTFiRPbYY4/84Q9/yODBg1dqXhURAAAoQV1d43ytqqUtWbNmzWowPmvWrGy88cbL7N+5c+e0bNmyQRtWdXV1Nt9880ybNm2l55WIAABAM9azZ8+0a9cuEydOrB+bM2dOJk+enL59+y6zf9++fbN48eI8+eST9WPvvvtuXn755WyxxRYrPa/WLAAAaMZat26dQYMGZfjw4enYsWM23XTTXHTRRencuXP22Wef1NTU5I033kj79u1TXV2dPn365JOf/GTOOOOMDB06NBtssEFGjhyZFi1aZL/99lvpeVVEAACgFJV+cOEaeKDhkCFDcvDBB+fss8/OIYcckhYtWmT06NFp1apVpk+fnv79++f222+v3/+yyy7LLrvskhNOOCEHH3xw3nnnnfzyl79Mx44dV3rOqrq61ekoWzvVzti20iEAwBo19LXtKh1Ck/bQ13aodAhN3h+fGFbpEFZoi9EXVjqE5XrxyNMrHUJJVEQAAIDCuUcEAABK0UgfaLi2UREBAAAKJxEBAAAKpzULAABKUNXslnpaM1REAACAwklEAACAwmnNAgCAUmjNKgsVEQAAoHASEQAAoHBaswAAoBQeaFgWKiIAAEDhJCIAAEDhtGYBAEAprJpVFioiAABA4SQiAABA4bRmAQBAKbRmlYWKCAAAUDiJCAAAUDitWQAAUAqtWWWhIgIAABROIgIAABROaxYAAJSirqrSETQJKiIAAEDhJCIAAEDhtGYBAEAJqqyaVRYqIgAAQOEkIgAAQOG0ZgEAQCm0ZpWFiggAAFA4iQgAAFA4iQgAAFA4iQgAAFC4km5Wf/vtt/PLX/4yTz75ZKqrq9OpU6dsu+226dGjR7bZZptUV1evqTgBAIAmpKRE5LTTTstf/vKXfOQjH8kGG2yQBx98MG+99VaqqqpSVVWVzTffvD4x2XbbbbPttttmiy22SFVV1ZqKHwAACuWBhuVRUiLy4IMP5gtf+EIuvvji+rGZM2fm6aefztSpU/P000/n6aefzp///OcsWrQoVVVVqa6uzmOPPfah771gwYI888wz2XrrrVNdXZ1//etfue666zJz5sxss802Ofzww9O5c+fSPyEAANDolJSItG/fPrvsskuDsY033jgbb7xxBgwYUD9WU1OT559/vj45+TDPPfdcjjjiiMyaNSubbLJJhg0bluOOOy6bbrpptt5669x999258cYbM3bs2HTv3r2UkAEAgEaopJvVP/e5z+Xxxx//0P1atGiRrbfeOl/84hdz0kknfej+//u//5vevXtn/Pjx2WWXXfKtb30rX/jCF3Lrrbfm0ksvzR133JH+/fvn/PPPLyVcAAAov7qqxvlay5SUiBx00EF5+umnc+edd5Y1iL///e/5zne+k549e+b000/PggULMmjQoPp7S1q2bJljjjkmjzzySFnnBQAAKqOk1qwDDjgg66yzTr7zne9k1113zZ577pntt98+PXr0SPv27Vc5iOrq6syfPz9J0rFjx/y///f/0qZNmwb7zJkzZ7XmAAAAGo+SEpHzzz+//ob0qVOn5qGHHqqvWnTu3Dk9evRIjx49Vqod67/1798/5513XoYNG5att946Q4cOrd9WW1ubBx98MOedd14+/elPl/S+AABQdlbNKouSKyL/7Y033sjUqVPrb0qfOnVqJk6cWHIictZZZ+W4447LlVdemeHDhzfY9sc//jEnn3xy9thjj5x88sklvS8AANA4lZSIvF/Hjh3Tr1+/9OvXr36srq70FLFjx44ZN25c5syZs8y2fv365ZZbbsk222yzOqECAACNyGolIsuzOg8v7NChwzJjG264YTbccMPVCQkAAMpHa1ZZlLRqFgAAQDlIRAAAgMKVvTULAACasiqtWWWhIgIAABROIgIAABROaxYAAJRCa1ZZqIgAAACFk4gAAACF05oFAACl0JpVFioiAABA4SQiAABA4bRmAQBACTzQsDxURAAAgMJJRAAAgMJpzQIAgFLUVVU6giZBRQQAACicRAQAACic1iwAACiFVbPKQkUEAAAonEQEAAAonNYsAAAogQcaloeKCAAAUDiJCAAAUDitWQAAUAqtWWWhIgIAABROIgIAABROaxYAAJTAqlnloSICAAAUTiICAAAUTmsWAACUQmtWWaiIAABAM1dbW5uRI0dmwIAB6d27d4466qi8/PLLK9z/5ptvTo8ePZZ5TZs2baXnVBEBAIBmbtSoURk7dmwuuOCCdO7cORdddFEGDx6cW265Ja1bt15m/6lTp2aXXXbJJZdc0mC8Y8eOKz2niggAAJSirpG+VtHChQszZsyYDBkyJAMHDkzPnj0zYsSIzJgxIxMmTFjuMU8//XR69OiRj370ow1eLVq0WOl5JSIAANCMTZkyJXPnzk2/fv3qxzp06JBevXpl0qRJyz1m6tSp6d69+2rNqzULAACagL333vsDt99zzz3LHZ8xY0aSpEuXLg3GO3XqVL/tv7311luZOXNmHn744YwdOzZvvvlmdtxxx5x22mnp1q3bSserIgIAACWoqmucr1U1f/78JFnmXpA2bdpkwYIFy+z/zDPPJEnq6upy/vnn5yc/+UkWLFiQr33ta3nttddWel4VEQAAaAJWVPH4MNXV1UmW3Cuy9M9JsmDBgrRt23aZ/fv06ZMHH3wwG264YaqqqpIkl19+eQYOHJgbb7wxRx999ErNqyICAADN2NKWrFmzZjUYnzVrVjbeeOPlHtOxY8f6JCRJ2rZtm8022ywzZ85c6XklIgAA0Iz17Nkz7dq1y8SJE+vH5syZk8mTJ6dv377L7H/99ddn1113zbx58+rH3nnnnbzwwgvZeuutV3peiQgAADRjrVu3zqBBgzJ8+PDcc889mTJlSk466aR07tw5++yzT2pqavLqq6/m3XffTZLsvvvuqa2tzemnn55nnnkmTz75ZE488cR07NgxBx544ErPKxEBAIBmbsiQITn44INz9tln55BDDkmLFi0yevTotGrVKtOnT0///v1z++23J1nSynXttddm3rx5OeSQQ3LEEUekffv2+eUvf5k2bdqs9JxVdXV1q3GP/dqpdsa2lQ4BANaooa9tV+kQmrSHvrZDpUNo8v74xLBKh7BCPYaOqHQIyzX1nJMqHUJJVEQAAIDCSUQAAIDCeY4IAACUYHUeHsj/UREBAAAKJxEBAAAKpzULAABKoTWrLJplIvLZTT5e6RCAxqxKsXiNq6utdARNn+t4zaqbWukIYK3nWwoAAChcs6yIAADAKtOaVRYqIgAAQOEkIgAAQOG0ZgEAQAk80LA8VEQAAIDCSUQAAIDCac0CAIBSaM0qCxURAACgcBIRAACgcFqzAACgBFbNKg8VEQAAoHASEQAAoHBaswAAoBRas8pCRQQAACicRAQAACic1iwAACiF1qyyUBEBAAAKJxEBAAAKpzULAABK4IGG5aEiAgAAFE4iAgAAFE5rFgAAlEJrVlmoiAAAAIWTiAAAAIXTmgUAAKXQmlUWKiIAAEDhJCIAAEDhtGYBAEAJPNCwPFREAACAwklEAACAwmnNAgCAUmjNKgsVEQAAoHASEQAAoHBaswAAoARWzSoPFREAAKBwEhEAAKBwWrMAAKAUWrPKQkUEAAAonEQEAAAonEQEAAAonHtEAACgFO4RKQsVEQAAoHASEQAAoHBaswAAoARVlQ6giVARAQAACicRAQAACqc1CwAASmHVrLJQEQEAAAonEQEAAAqnNQsAAEpQpTWrLFREAACAwjX6ROToo4/OrFmzKh0GAABQRo2iNWv8+PEr3DZx4sTceuut6dixY5Jk//33LyYoAABYHq1ZZdEoEpFzzz037777bpKkrm7Z/2UvvPDCJElVVZVEBAAAmoBGkYjceOONOfXUU9OhQ4dccMEF2Xjjjeu37bTTTrn55puz+eabVzBCAACgnBrFPSLdunXL9ddfnx122CH77bdfbr/99kqHBAAAy1fXSF9rmUaRiCRJy5Ytc/LJJ+eyyy7L8OHDc+qpp+btt9+udFgAAMAa0GgSkaX69u2b8ePHp66uLvvuu28WLVpU6ZAAAIAyaxT3iLxfhw4dcvHFF2f8+PG58cYb06ZNm0qHBAAASTzQsFwaZSKy1P7772+VLAAAaIIaXWsWAADQ9DXqiggAADQ6WrPKQkUEAAAonEQEAAAonNYsAAAogVWzykNFBAAAmrna2tqMHDkyAwYMSO/evXPUUUfl5ZdfXqljb7755vTo0SPTpk0raU6JCAAANHOjRo3K2LFjc95552XcuHGpra3N4MGDs3Dhwg887j//+U+GDh26SnNKRAAAoBR1jfS1ihYuXJgxY8ZkyJAhGThwYHr27JkRI0ZkxowZmTBhwgqPq62tzWmnnZbtt99+leaViAAAQDM2ZcqUzJ07N/369asf69ChQ3r16pVJkyat8Lgrr7wyixYtyjHHHLNK87pZHQAAmrEZM2YkSbp06dJgvFOnTvXb3u+JJ57ImDFj8vvf/z4zZ85cpXklIgAAUILGumrW3nvv/YHb77nnnuWOz58/P0nSunXrBuNt2rTJW2+9tcz+8+bNy6mnnppTTz01W2655SonIlqzAACgGauurk6SZW5MX7BgQdq2bbvM/sOGDUu3bt3y1a9+dbXmVREBAIAmYEUVjw+ztCVr1qxZ6dq1a/34rFmz0qNHj2X2v+GGG9K6devstNNOSZKampokyb777ptjjz02xx577ErNKxEBAIBSNNLWrFXVs2fPtGvXLhMnTqxPRObMmZPJkydn0KBBy+z//pW0Hn/88Zx22mm56qqrsu222670vBIRAABoxlq3bp1BgwZl+PDh6dixYzbddNNcdNFF6dy5c/bZZ5/U1NTkjTfeSPv27VNdXZ0tttiiwfFLb2jfZJNNssEGG6z0vO4RAQCAZm7IkCE5+OCDc/bZZ+eQQw5JixYtMnr06LRq1SrTp09P//79c/vtt5d1zqq6uromVlz6cJ9Z538qHQLQmFX5jWaNq6utdARNn+t4zXINr3F31f6u0iGs0M7HjKh0CMv16M9OqnQIJfEtBQAAFE4iAgAAFM7N6gAAUILG+kDDtY2KCAAAUDiJCAAAUDitWQAAUAqtWWWhIgIAABROIgIAABROaxYAAJSgqvk9D3yNUBEBAAAKJxEBAAAKpzULAABKoTOrLFREAACAwklEAACAwmnNAgCAElRpzSoLFREAAKBwEhEAAKBwWrMAAKAUWrPKQkUEAAAonEQEAAAonNYsAAAogVWzykNFBAAAKJxEBAAAKJzWLAAAKIXWrLJQEQEAAAonEQEAAAqnNQsAAEpg1azyUBEBAAAKJxEBAAAKpzULAABKoTWrLFREAACAwjXPikiV/AugonwPr3l1tZWOoGlzDcNqa56JCAAArCKrZpWHdB4AACicRAQAACic1iwAAChFnd6sclARAQAACicRAQAACqc1CwAASmDVrPJQEQEAAAonEQEAAAqnNQsAAEqhNassVEQAAIDCSUQAAIDCac0CAIASVNVWOoKmQUUEAAAonEQEAAAonNYsAAAohVWzykJFBAAAKJxEBAAAKJzWLAAAKEGV1qyyUBEBAAAKJxEBAAAKpzULAABKUac3qxxURAAAgMJJRAAAgMJpzQIAgBJYNas8VEQAAIDCSUQAAIDCac0CAIBSaM0qCxURAACgcBIRAACgcFqzAACgBFbNKg8VEQAAoHASEQAAoHBaswAAoBR1erPKQUUEAAAonEQEAAAonNYsAAAogVWzykNFBAAAKJxEBAAAKJzWLAAAKIXWrLJQEQEAAAonEQEAAAonEQEAgBJU1TXO1+qora3NyJEjM2DAgPTu3TtHHXVUXn755RXu/9RTT+Xwww/PTjvtlN122y3nnHNO3n777ZLmlIgAAEAzN2rUqIwdOzbnnXdexo0bl9ra2gwePDgLFy5cZt/XXnst3/jGN7LpppvmxhtvzKhRo/LII4/kzDPPLGlOiQgAADRjCxcuzJgxYzJkyJAMHDgwPXv2zIgRIzJjxoxMmDBhmf3/85//pH///hk6dGi6deuWnXfeOf/v//2/PPDAAyXNKxEBAIBS1NY1ztcqmjJlSubOnZt+/frVj3Xo0CG9evXKpEmTltn/4x//eC655JK0bLlkAd5nn302N910Uz71qU+VNK/lewEAoAnYe++9P3D7Pffcs9zxGTNmJEm6dOnSYLxTp07121bks5/9bF544YVsuummufzyy0uIVkUEAACatfnz5ydJWrdu3WC8TZs2WbBgwQceO3z48PzqV7/KRhttlK9//euZO3fuSs+rIgIAAKVopA80XFHF48NUV1cnWXKvyNI/J8mCBQvStm3bDzx2hx12SJJcfvnl2WOPPXLXXXdl//33X6l5VUQAAKAZW9qSNWvWrAbjs2bNysYbb7zM/s8991z+/Oc/NxjbeOONs8EGG2TmzJkrPa9EBAAAmrGePXumXbt2mThxYv3YnDlzMnny5PTt23eZ/f/2t79lyJAhmTNnTv3YSy+9lDfffDPdu3df6XklIgAAUIJKP7iw3A80bN26dQYNGpThw4fnnnvuyZQpU3LSSSelc+fO2WeffVJTU5NXX3017777bpJk3333zQYbbJDTTjstzzzzTB5++OEMGTIkO+64Y/bcc8+VnrdRJCLjx49f5mEpDz30UI4++uh8+ctfzimnnJJnn322QtEBAEDTNmTIkBx88ME5++yzc8ghh6RFixYZPXp0WrVqlenTp6d///65/fbbkyQbbLBBfvGLXyRJDjnkkBx//PHp1atXRo8enRYtWqz0nFV1dXUVv91mu+22y/3335+NNtooSXLffffl6KOPTv/+/bPNNtvkySefzBNPPJFrrrkmO++882rP95kWX1nt9wCARq2uttIRNG1VjeK33CbtrprrKx3CCg38/IWVDmG5/nzH6ZUOoSSNYtWs9+dCV1xxRY444oicccYZ9WPnn39+hg8fnrFjxxYdHgAA/J/K/47fJDTKdP7FF1/Ml770pQZjX/nKVzJ58uQKRQQAAJRTo0hEqqqqGvy9W7dueeeddxqMvfHGG2nfvn2RYQEAAGtIo2nN2nvvvbPlllume/fuadmyZS644IKMGzcurVu3zqRJkzJ06NDsvvvulQ4VAIBmbnVWqOL/NIpE5C9/+UumTp2ap59+OlOnTs2bb76Z5557LjU1NUmSY489Nt27d88pp5xS4UgBAIByaBSJyMYbb5yNN964QcWjpqamfvmv66+/Pt27d1+mhQsAAFg7NYpEZHn+ew3irbfeuoKRAADAf9GaVRaN4mZ1AACgeZGIAAAAhWu0rVkAANAYVXmgYVmoiAAAAIWTiAAAAIXTmgUAAKWorXQATYOKCAAAUDiJCAAAUDitWQAAUAKrZpWHiggAAFA4iQgAAFA4rVkAAFAKnVlloSICAAAUTiICAAAUTmsWAACUwqpZZaEiAgAAFE4iAgAAFE5rFgAAlKBKZ1ZZqIgAAACFk4gAAACF05oFAAClsGpWWaiIAAAAhZOIAAAAhdOaBQAAJaiqrXQETYOKCAAAUDiJCAAAUDitWQAAUAqrZpWFiggAAFA4iQgAAFA4rVkAAFAKnVlloSICAAAUTiICAAAUTmsWAACUoMqqWWWhIgIAABROIgIAABROaxYAAJRCa1ZZqIgAAACFk4gAAACF05oFAAClqK10AE2DiggAAFA4iQgAAFA4rVkAAFACDzQsDxURAACgcBIRAACgcFqzAACgFFqzykJFBAAAKJxEBAAAKJzWLAAAKIXWrLKQiABAU1Sl6QFo3HxLAQAAhVMRAQCAUtRWOoCmQUUEAAAonEQEAAAonNYsAAAoQZVVs8pCRQQAACicRAQAACic1iwAACiF1qyyUBEBAAAKJxEBAAAKpzULAABKoTWrLFREAACAwklEAACAwmnNAgCAUmjNKgsVEQAAoHASEQAAoHBaswAAoBS1lQ6gaVARAQAACicRAQCAZq62tjYjR47MgAED0rt37xx11FF5+eWXV7j/M888k6OPPjq77rpr+vXrlyFDhuSVV14paU6JCAAAlKCqrq5RvlbHqFGjMnbs2Jx33nkZN25camtrM3jw4CxcuHCZfd9888184xvfSHV1dX71q1/l5z//ed54440MHjw4CxYsWOk5JSIAANCMLVy4MGPGjMmQIUMycODA9OzZMyNGjMiMGTMyYcKEZfa/++67M2/evFx44YXZdttt87GPfSwXXXRRnn322Tz66KMrPa9EBAAAmrEpU6Zk7ty56devX/1Yhw4d0qtXr0yaNGmZ/fv165dRo0alurq6fmyddZakFXPmzFnpea2aBQAApWikDzTce++9P3D7Pffcs9zxGTNmJEm6dOnSYLxTp0712/7bZpttls0226zB2FVXXZXq6ur07dt3peNVEQEAgGZs/vz5SZLWrVs3GG/Tps1K3fPxq1/9Ktddd11OPfXUdOzYcaXnVREBAIAmYEUVjw+ztMVq4cKFDdqtFixYkLZt267wuLq6ulx66aW54oor8q1vfSuHHXZYSfNKRAAAoBS1jbM1a1UtbcmaNWtWunbtWj8+a9as9OjRY7nHLFq0KGeddVZuvfXWnHXWWTniiCNKnldrFgAANGM9e/ZMu3btMnHixPqxOXPmZPLkySu85+P000/PH//4x1x88cWrlIQkKiIAANCstW7dOoMGDcrw4cPTsWPHbLrpprnooovSuXPn7LPPPqmpqckbb7yR9u3bp7q6OjfeeGNuv/32nH766dlll13y6quv1r/X0n1WhooIAACUoq6ucb5Ww5AhQ3LwwQfn7LPPziGHHJIWLVpk9OjRadWqVaZPn57+/fvn9ttvT5LceuutSZILL7ww/fv3b/Baus/KqKqra6Trj61Bn2nxlUqHAADAB7ir5vpKh7BCn9/2jEqHsFx3PP2/lQ6hJCoiAABA4dwjAgAApWh+DUVrhIoIAABQOIkIAABQOK1ZAABQCq1ZZaEiAgAAFE4iAgAAFE5rFgAAlKJWa1Y5qIgAAACFk4gAAACF05oFAAClqKutdARNgooIAABQOIkIAABQOK1ZAABQCg80LAsVEQAAoHASEQAAoHBaswAAoBQeaFgWKiIAAEDhJCIAAEDhtGYBAEAprJpVFioiAABA4SQiAABA4RpNa9bjjz+eiRMn5uijj06SPPTQQ7n22mszbdq0dO3aNd/85jfTp0+fCkcJAECzpzWrLBpFReSPf/xjDjnkkPz9739Pktx77735xje+kbq6uuyxxx5ZtGhRDj/88Nx7770VjhQAACiHRlERufzyyzNkyJAce+yxSZIrrrgixx57bL797W/X73PFFVdk5MiR2XPPPSsVJgAAUCaNoiLy0ksv5Ytf/GL936dNm5bPfvazDfbZd9998+yzzxYdGgAANFRX1zhfa5lGkYhsvvnmeeCBB+r/vt1222XKlCkN9nniiSey8cYbFx0aAACwBjSK1qyjjjoqZ599dqZNm5Z99903xx13XM4888wsWLAg22yzTR5//PH89Kc/zQknnFDpUAEAgDJoFInI/vvvn6qqqowcOTJXX311qqqqUldXlx/84AdJkvXWWy+DBw/OEUccUdlAAQCgtrbSETQJjSIRSZL99tsv++23X55//vk8//zzeeedd9KyZct07tw522+/fdq0aVPpEAEAgDJpNInIUt26dUu3bt0qHQYAALAGNbpEBAAAGrW1cIWqxqhRrJoFAAA0LxIRAACgcFqzAACgFFqzykJFBAAAKJxEBAAAKJzWLAAAKEWt1qxyUBEBAAAKJxEBAAAKpzULAABKUFdXW+kQmgQVEQAAoHASEQAAoHBaswAAoBRWzSoLFREAAKBwEhEAAKBwWrMAAKAUdVqzykFFBAAAKJxEBAAAKJzWLAAAKEWtBxqWg4oIAABQOIkIAABQOK1ZAABQCqtmlYWKCAAAUDiJCAAAUDitWQAAUII6q2aVhYoIAABQOIkIAABQOK1ZAABQCqtmlYWKCAAAUDiJCAAAUDitWQAAUIparVnloCICAAAUTiICAAAUTmsWAACUos4DDctBRQQAACicRAQAACic1iwAAChBnVWzykJFBAAAKJxEBAAAKJzWLAAAKIVVs8pCRQQAACicRAQAACicRAQAAEpQV1vXKF+ro7a2NiNHjsyAAQPSu3fvHHXUUXn55ZdX6rjBgwfnsssuK3lOiQgAADRzo0aNytixY3Peeedl3Lhx9QnGwoULV3jMwoUL893vfjf33XffKs0pEQEAgGZs4cKFGTNmTIYMGZKBAwemZ8+eGTFiRGbMmJEJEyYs95hHH300Bx54YB5++OF06NBhleaViAAAQCnqahvnaxVNmTIlc+fOTb9+/erHOnTokF69emXSpEnLPeYvf/lLBgwYkPHjx6d9+/arNK/lewEAoBmbMWNGkqRLly4Nxjt16lS/7f1OOumk1Z5XIgIAAE3A3nvv/YHb77nnnuWOz58/P0nSunXrBuNt2rTJW2+9VZ7glqNZJiJ31Vxf6RAAAFhL3VX7u0qHsFwfloisSHV1dZIl94os/XOSLFiwIG3bti1LbMvTLBMRAABoalZU8fgwS1uyZs2ala5du9aPz5o1Kz169ChLbMvjZnUAAGjGevbsmXbt2mXixIn1Y3PmzMnkyZPTt2/fNTaviggAADRjrVu3zqBBgzJ8+PB07Ngxm266aS666KJ07tw5++yzT2pqavLGG2+kffv2DVq3VpeKCAAANHNDhgzJwQcfnLPPPjuHHHJIWrRokdGjR6dVq1aZPn16+vfvn9tvv72sc1bV1dWt3vPgAQAASqQiAgAAFE4iAgAAFE4iAgAAFE4iAgAAFE4iAgAAFE4iAgAAFE4iAgAAFE4iAgAAFE4i0kjV1tZm5MiRGTBgQHr37p2jjjoqL7/8cqXDarJ+9rOf5bDDDqt0GE3K7Nmzc84552T33XfPzjvvnEMOOSQPP/xwpcNqUl5//fWcdtpp2W233bLTTjvl6KOPzrPPPlvpsJqk559/PjvttFNuvPHGSofSpMycOTM9evRY5uU8l9f48ePzhS98ITvssEO++MUv5o477qh0SJBEItJojRo1KmPHjs15552XcePGpba2NoMHD87ChQsrHVqT8+tf/zo/+clPKh1Gk3PyySfnscceyyWXXJIbbrgh2223XY488sg899xzlQ6tyTj++OPz4osv5qqrrsrvf//7VFdX54gjjsj8+fMrHVqTsmjRopx66qmZN29epUNpcqZMmZI2bdrkvvvuy/3331//+sIXvlDp0JqMm266Kd/73vdy6KGH5rbbbsu+++5b//0MlSYRaYQWLlyYMWPGZMiQIRk4cGB69uyZESNGZMaMGZkwYUKlw2syZs6cmWOPPTbDhw/PlltuWelwmpQXX3wxDzzwQH74wx+mT58+6datW77//e+nU6dOueWWWyodXpPw1ltvZdNNN82wYcOy4447pnv37jnuuOMya9asPPPMM5UOr0m57LLL0q5du0qH0SQ9/fTT2XLLLdOpU6d89KMfrX9VV1dXOrQmoa6uLpdeemm+/vWv59BDD03Xrl3zrW99K5/85Cfz97//vdLhgUSkMZoyZUrmzp2bfv361Y916NAhvXr1yqRJkyoYWdPy1FNPpVWrVrn55pvz8Y9/vNLhNCkbbrhhrrrqquywww71Y1VVVamqqsqcOXMqGFnTsf766+fiiy/OtttumyR54403cu2116Zz587ZeuutKxxd0zFp0qRcf/31ueCCCyodSpM0derUdO/evdJhNFnPP/98/vOf/+RLX/pSg/HRo0fnmGOOqVBU8H9aVjoAljVjxowkSZcuXRqMd+rUqX4bq2+vvfbKXnvtVekwmqQOHTpkjz32aDB255135sUXX8x3v/vdCkXVdH3/+9/Pb3/727Ru3TpXXHFF1l133UqH1CTMmTMnp59+es4+++xlvo8pj6effjobbrhhDj300Dz//PPZYost8q1vfSu77757pUNrEp5//vkkybx583LkkUdm8uTJ2WyzzfKtb33Lf/9oFFREGqGl/d2tW7duMN6mTZssWLCgEiHBann00Udz1llnZZ999snAgQMrHU6Tc/jhh+eGG27Ivvvum+OPPz5PPfVUpUNqEn74wx9mp512WubXZMpj8eLFee655/LWW2/lxBNPzFVXXZXevXvn6KOPzoMPPljp8JqEd955J0lyxhlnZN99982YMWPyqU99Kscdd5xzTKOgItIILe2NXbhwYYM+2QULFqRt27aVCgtWyd13351TTz01O++8c4YPH17pcJqkpa1YP/rRj/L444/nuuuuy/nnn1/hqNZu48ePz8MPP+yepjWoZcuWmThxYlq0aFH/37qPfexjeeaZZzJ69OgG7cmsmlatWiVJjjzyyBxwwAFJku222y6TJ0/ONddc4xxTcSoijdDSFoBZs2Y1GJ81a1Y23njjSoQEq+S6667LiSeemD333DNXXnll2rRpU+mQmow33ngjt912WxYvXlw/ts4662Trrbde5ruD0t1www15/fXXM3DgwOy0007ZaaedkiQ/+MEPMnjw4ApH13Sst956y9yYvs0222TmzJkViqhpWfpvhqX3ki219dZbZ9q0aZUICRqQiDRCPXv2TLt27TJx4sT6sTlz5mTy5Mnp27dvBSODlbd0+elDDz00l1xyyTKthqye1157LSeffHKD9opFixZl8uTJbv4tg+HDh+f222/P+PHj619JMmTIkPzoRz+qbHBNxDPPPJOdd965wX/rkuSf//ynBRfKZPvtt896662Xxx9/vMH4008/na5du1YoKvg/WrMaodatW2fQoEEZPnx4OnbsmE033TQXXXRROnfunH322afS4cGHev755/PjH/84n/nMZ3LMMcfktddeq99WXV2d9u3bVzC6pmHbbbfN7rvvnmHDhmXYsGFZf/3187Of/Sxz5szJEUccUenw1norqj5vtNFGKtNl0r1792y11VYZOnRozj333Gy44Yb57W9/m3/84x+54YYbKh1ek1BdXZ3Bgwfnpz/9aTbeeOPsuOOOue222/LAAw/k2muvrXR4IBFprIYMGZLFixfn7LPPzrvvvpu+fftm9OjR9f2e0JjdeeedWbRoUe66667cddddDbYdcMABlkItk0suuSQXX3xxTjrppLz99tvp06dPfv3rX2eTTTapdGjwodZZZ51ceeWVufjii/Od73wnc+bMSa9evXLNNdcs00rEqjvuuOPStm3bjBgxIjNnzkz37t1z2WWXZdddd610aJCqurq6ukoHAQAANC/uEQEAAAonEQEAAAonEQEAAAonEQEAAAonEQEAAAonEQEAAAonEQEAAAonEQEAAAonEQFYSy1YsCDbb799+vTpk8suu6zS4QBASSQiAGupxYsXZ9iwYdlkk03y05/+NK+//nqlQwKAlSYRAVhLrbfeejnggAMyePDg1NXVZerUqZUOCQBWmkQEYC23zTbbJEmeffbZCkcCACtPIgKwlluwYEGS5N///neFIwGAlScRAVjLXXDBBUmS5557rsKRAMDKk4gArMVuuummPPbYY9loo420ZgGwVpGIAKyl5s6dm+HDh2f33XfPfvvtl9dffz2zZ8+udFgAsFIkIgBrqZ/97Gd58803873vfS9bb711EveJALD2kIgArIVeeumlXHPNNfnmN7+ZLbfcsn7lLPeJALC2kIgArIXOP//8dOzYMccee2ySZKuttkqiIgLA2qNlpQMAoDT33Xdf/vSnP2XEiBFZd911kyTt2rVLly5d3LAOwFpDRQRgLbJo0aL8+Mc/zq677povfOELDbZ1795daxYAaw2JCMBa5Ne//nVeeumlfP/7319m2zbbbJPp06dn7ty5FYgMAEpTVVdXV1fpIAAAgOZFRQQAACicRAQAACicRAQAACicRAQAACicRAQAACicRAQAACicRAQAACicRAQAACicRAQAACicRAQAACicRAQAACicRAQAACjc/weW4mpJvpCCiQAAAABJRU5ErkJggg==", - "text/plain": [ - "
    " - ] - }, - "metadata": { - "filenames": { - "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/week43_96_1.png" - } - }, - "output_type": "display_data" - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAyIAAANdCAYAAABoOchKAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAABfs0lEQVR4nO3de3zO9f/H8efMTmyTVbMh0RxmyikjRU6lQiX5fr/JRDVErJw6+voWijLk0BzKVA7RLyJSTkmRZlSSNYSctznPZna6rt8fuL7fq23sw+VzXbv2uN9u1+1m78/nc31e18fHLq/P6/V5fzysVqtVAAAAAGCiMs4OAAAAAEDpQyICAAAAwHQkIgAAAABMRyICAAAAwHQkIgAAAABMRyICAAAAwHQkIgAAAABMRyICAAAAwHQkIgBgEp4fCwDAf5V1dgAA4CyvvPKKvvjii8uu07RpU82ZM+ea9xUXFydvb29FR0dfcd2JEydq+vTpioqK0r///e9r3jcAAK7Iw8olOgCl1IEDB3Ty5Enbz3FxcUpKStLUqVNtY/7+/qpZs+Y176tOnToaMGCABg4ceNn1LBaL2rRpo8DAQB09elQ//PCD/Pz8rnn/AAC4GioiAEqtatWqqVq1arafg4KC5O3trYYNGzotpg0bNiglJUUTJkxQVFSUli9frn/84x9OiwcAgOuFe0QA4Aq2bNmiqKgoNWjQQE2bNtXLL79sV0mxWCyaOHGi2rZtq9tvv11t27bV+PHjlZubK+lCNUSSpk6davtzURYtWqTatWvrzjvvVLNmzbRw4cJC11u/fr2eeOIJNWzYUC1atNCIESOUnp5uW753714NGDBATZs2VWRkpPr27as9e/ZIkhISElSnTh0lJCTYvWePHj3Uo0cP289t27bV22+/rZ49e6p+/fp6/fXXJUnJyckaMGCA7rrrLtWrV08tW7bU6NGjdf78edu2OTk5eu+999SuXTvVr19fnTp1srXBzZs3T3Xq1NG+ffvs9r906VLVrVtXR48evewxAgC4BxIRALiMxMRE9erVS76+vnrvvff02muvafPmzXrqqads//H+4IMP9Omnn+r5559XfHy8unXrplmzZmnatGmSZEsmunbtWmRiIUmnT5/Wt99+q86dO0uSHnvsMW3fvl07duywW2/dunXq27evbrzxRr333nsaOnSo1qxZo0GDBkmSUlNT9a9//Ut//fWX3njjDY0bN07Hjx9Xz549dfr0aUOff968ebrjjjsUFxenrl27Ki0tTd27d1dWVpbGjh2rDz74QB07dtScOXP0ySef2LYbOnSoZs+erX/84x+aMWOGWrRooVdeeUXLly/Xww8/LB8fHy1dutRuX0uWLFHz5s0VGhpqKEYAQMlEaxYAXMb48eNVo0YNzZgxQ56enpKkBg0aqGPHjlq0aJG6d++uzZs36/bbb9fjjz8u6cIN7n5+fgoICJAkW6tXSEjIZdu+li1bJovFokcffVSS1L59e40cOVILFizQqFGjbOtNmTJFdevW1dSpU+Xh4SFJ8vb21qRJk3T8+HF99NFHysnJ0ezZs3XzzTdLksLDw9WtWzdt27ZNvr6+xf78lStX1tChQ20/b9iwQXXr1tWkSZPk7+8vSbr77ru1ceNGJSQkqE+fPtq1a5dWrlyp1157TT179pQkNW/eXIcPH1ZCQoI6deqk+++/X19++aVeeOEFeXh4KCUlRT/99JPGjRtX7NgAACUbFREAKEJWVpa2bdumVq1ayWq1Ki8vT3l5ebrlllsUFhamjRs3SpKaNWumjRs36sknn9SHH36oP//8U1FRUbaEorgWLVqkZs2aydvbW+np6crNzVXbtm21fPlyZWRkSJLOnz+vpKQk3XfffbYkRJI6dOiglStX6qabbtLWrVvVsGFDWxIiXUiC1q1bp1atWhmKqW7dunY/t2jRQnPnzpWPj4/+/PNPrV27VtOmTdPJkyeVk5MjSdq6daukC4nU/5oyZYotoeratasOHz6sLVu2SLpQDSlfvrzuv/9+Q/EBAEouKiIAUIT09HRZLBZ98MEH+uCDDwos9/HxkSRFR0erfPnyWrRokWJjYzVu3DjVqlVLw4cP11133VWsfSUlJemPP/6QJEVGRhZY/uWXX+rJJ5/UmTNnZLVadeONNxb5XqdPn1bVqlWLtd8rKVeunN3PFotFEyZM0Lx583Tu3DmFhoaqfv36tmNxaf+SLhvjXXfdpapVq2rJkiWKjIzUkiVL1KFDB7v3AQC4NxIRAChC+fLl5eHhoV69eqljx44Fll+aVrdMmTLq3r27unfvrhMnTmj9+vWaPn26Bg4cqI0bN8rb2/uK+1q8eLHKlSunuLg4lSljX6weMWKEFi5cqCeffFL+/v7y8PCwu1lekrKzs/XTTz+pQYMGCggIKLBckjZt2qSqVavaKikWi8VueWZmpsqXL3/ZOGfOnKmPPvpIb775ptq3b29rP+vatattncDAQEnSyZMnFRISYhvfs2ePTp8+rTvvvFMeHh567LHHNGfOHHXr1k379u3TO++8c6XDBABwI7RmAUAR/P39FRERob179+qOO+6wvWrVqqUpU6bYZp164oknNHr0aEkXqgBdunRR9+7dlZ6ebmup+nty8b9ycnK0bNkytW3bVs2bN1ezZs3sXp07d1ZycrJ+/fVXlS9fXnXr1tW6devs3uP7779Xnz59lJaWpiZNmmjbtm12yciJEycUHR2t9evX2+7tSElJsS0/c+aMbVaty9m6datq1qypxx9/3JaEpKamateuXbbE5s4775Qkffvtt3bbxsbG6q233rL93KVLF6Wnp+udd95RWFiYGjRocMX9AwDcBxURALiMwYMHq0+fPhoyZIgeeeQR5efnKz4+Xtu2bVP//v0lXWilio+P10033aRGjRopNTVVs2fPVtOmTRUUFCTpQpXg559/VmJiopo0aWJ3f8eaNWt0+vRpderUqdAYHn30UU2aNEkLFixQw4YNFRMTo379+mnw4MHq3Lmzjh8/rgkTJui+++5T7dq11atXLy1ZskTR0dHq27evvLy8NG3aNIWEhOjhhx+Wv7+/QkND9f7779sqLDNmzCjWgxPr16+vuLg4zZw5Uw0bNtT+/fs1Y8YM5eTkKCsrS9KFG+MffPBBjRs3TufPn1fdunX1/fffa926dXYPi6xcubLuvvtubdiwwe6GeABA6cCT1QHgoldeeUWbN28ucCV/06ZNmjp1qn7//Xd5eXmpXr16GjhwoJo0aSJJysvL07Rp0/Tll18qJSVFAQEBatu2rYYMGaKKFStKkmbPnq24uDjl5uZqxYoVqly5su39o6Oj9dtvv2njxo3y8vIqNLYePXrot99+0/fff68KFSrou+++09SpU7Vz504FBQWpQ4cOGjhwoO2ejj179mjcuHFKSEiQt7e3mjVrppdeesl278hvv/2mt99+Wzt27NBNN92knj17au/evdq3b5/mzJkj6cJzRJo2baqxY8fa4sjJydHYsWO1atUqnT17VqGhoerYsaMtmdm4caMCAwOVk5OjqVOnaunSpTp16pTCwsLUr1+/Ajewz5kzR2PGjNF3332n4ODga/nrAwCUMCQiAACniY6Olo+Pj95//31nhwIAMBmtWQAA073//vvat2+fNmzYoPnz5zs7HACAE5CIAABM9+233+rAgQN66aWX1LhxY2eHAwBwAlqzAAAAAJiO6XsBAAAAmI5EBAAAAIDpSEQAAAAAmI5EBAAAAIDpSuWsWQ8G9XZ2CG7NGlbV2SG4PY99h50dglvzuPhQQFxH/hzj6y4n19kRuDVLBc7h623l1jedHUKRLCm1nR1CocqE7HJ2CIZQEQEAAABgOhIRAAAAAKYrla1ZAAAAwNWyyOLsEApV0ioMJS1eAAAAAG6ARAQAAACA6WjNAgAAAAzIt7pma1ZJ+489FREAAAAApiMRAQAAAGC6klbBAQAAAJzKIquzQ3ALVEQAAAAAmI5EBAAAAIDpaM0CAAAADHDVBxqWNFREAAAAAJiORAQAAACA6WjNAgAAAAzItzJrliNQEQEAAABgOhIRAAAAAKajNQsAAAAwgAcaOgYVEQAAAACmIxEBAAAAYDpaswAAAAAD8mnNcggqIgAAAABMRyICAAAAwHS0ZgEAAAAGMGuWY1ARAQAAAGA6EhEAAAAApqM1CwAAADAg30prliNQEQEAAABgOhIRAAAAAKajNQsAAAAwwOLsANwEFREAAAAApiMRAQAAAGA6WrMAAAAAA/J5oKFDUBEBAAAAYDoSEQAAAACmozULAAAAMCCfziyHoCICAAAAwHQkIgAAAABMR2sWAAAAYAAPNHQMKiIAAAAATEciAgAAAMB0tGYBAAAABuTLw9khuAUqIgAAAABMRyICAAAAlHIWi0WTJ09Wy5Yt1bBhQ/Xu3VsHDx4scv2//vpLffr0UZMmTXTvvfdq8uTJysvLM7RPEhEAAADAAIvVNV/XIi4uTvPnz9eoUaO0YMECWSwWRUdHKycnp8C6Z86cUffu3ZWVlaWPP/5YEyZM0Ndff60RI0YY2ieJCAAAAFCK5eTkKD4+XjExMWrdurXCw8M1ceJEpaSkaNWqVQXW/+KLL3Tu3DlNmjRJ9erVU5MmTTR69GgtWrRIhw4dKvZ+SUQAAACAUiw5OVmZmZlq3ry5bSwwMFARERFKTEwssP7+/ft12223KSgoyDYWEREhSdqyZUux98usWSZr3CZCvV5/TNXCQ3X62Fktm7VOi6YWzDQLE3bHLZq05jU922S4Ug+esFvW8tE79Y+BD6pqrRBlpp/TL+v/UPybi3T62Nnr8TFcVuNmt6lX/7a69bZgnT6RoWWfJ+rzuZuKXN/Ly1OPd2+u+zo20M2VAnUsLV3rvtmuhR9tUF5ewccV+ZXz1vT5z2nuh+u1evm26/lRXFbj1hHq9XpnVatTWaePpWtZ/Doten91sbYNu+MWTVr1mp5tWsg5/Mid+sfABy6cw2cunsOjFpe+c7hlHfV8qaOq1Q7R6WNntfyTDVo0c12xtg2rV0XvfTlYz7Z6S2mHTtotm5Pwhm4KvaHANv9q8LrST2U6IvQSo/HdNdXzhfaqVvPC74nln/6kRbM3FGvbsLqhem9Bfz370HilHTltt6x5u7rq9lxbVa1xk04dz9DaL3/RZx+sV15u/nX4FK6rcYva6jn4QVWrVUmnj2do+dwftWjW+mJtGxZRWe8titGz972jtMOn7JZVve1mPftyR9VvFqb8PIu2b96rD8YsU8rBk0W8W+nS+K4w9erfTreG3azTJzK17P826/M5Pxa5vpeXpx6Pulv3dWqgmytV0LHUdK375jctnL1BeXml65y9Gq46a1a7du0uu3zt2rWFjqekpEiSQkND7caDg4Nty/4+npaWpvz8fHl6ekqSDh8+LEk6ceJEgfWLQiJiovAmt+nNTwfq+y8S9fGYJbq9WS09+8bj8vQso88mfXPZbW+tW1kjF8SorFfBv7JWXSL16od99NXs9frorS9UMbiCnnrtUb2zdKgGtBml3GxjNw6VVOG3V9HIiU9q/erf9cn0darXoJqeHXi/yniW0Wcfbyx0m35DHlS7DvU1b9b32pV0RLXrVlb33q0UHHKDJo7+0m5d/wBfvRH7hEKqVDTj47ik8Dtr6M35A/T9ki36eMxS3d6spp79z+Py9PTUZ5OvcA6HV9bITweqrJdngWWtHovUqx/01lcfrddHby1RxeBAPfXqo3rniyEa0G506TmHG92qN2b31vfLf9EnsStUL/I2PfPawypTtoz+L67wL49Lbq0dojc/6lPo8Q2sWF43hd6gD0cv1Y7EvXbLMtKzHPoZXF14/Vv0xrSn9P3X2/XJlNWq17i6nhnyoMp4ltH/ffj9Zbe9tWYlvTmtZ6HHuFHzmho+qbu+/3q7Zk9YqVtrBqvXoAdUoWJ5TXtr2fX6OC4nvGE1vTHzaX2/Yps+eW+l6t1ZXc+83OHCOTzj8gn1rbUq6c0Pnin0+N4UWkHjFz6vQ/uOaeyL8+Tj562egx7QWx/1Vr8O45VTSn5HFCX89qoa+d6TWr9qhz6Z9q3qNaymZ2Mufv99VHiS3W/oQ2rXsYHmfbheu3YcVu2IKure5+L336ilJn8COFtW1oXvAm9vb7txHx8fnTlzpsD6Dz30kOLi4jRmzBgNHjxY586d0+jRo1W2bFnl5uYWe78kIibq8coj2rP9gMb1i5ckbV27Q55envrX4A5aMmOtcs4X/Isr6+WpR/q01VOvPKqc7ML/Yp8Y1EGbV/2mKUPm2sYO/ZmiSatfU7MH6mvDlz9fnw/kYnr0aa09O49q3H+WSJK2bNqjsmU99USvllqyIKHAF1VABT899NidmjVljT6fe+Gq0a+J+yRJzw68T/FT1+jM6XOSpLvura1+Qx5SuXL2/0BLmx4vP6I92w9qXP+L5/C3F8/hQQ9pyczLnMO92+qplx8p+hx+8SFtXr1dU4bOs40d+jNVk1a9qmbt62vDstJxDkcNfkh7dhxW7IsXjsPW9ckq6+WpJ56/X0tnfV/o8Svr5alHerVUjyEPFfmfsdvqVZEk/bjyNx3dX/wrVe4oakA77fnjqGJf+T9J0tYNu1W2bBk90ae1ls75sdBjWNbLU490b64eA+5TTk7hx/j+x+7UsaNnNO7lz2SxWPXLpj91w03+6tKzhWa+85XyC6mwuqOoF9prT9IRxQ5dIEna+v3OC7+H+7XV0o9+KPr4PnWPerzwQJG/I6Ji2ivz7Hm99tRMZV/8PZN68KT+M6OXat1xi3Zs2Xf9PlQJ0OO5NtqzM0XjRiyWJG3Z9OeF4/50Sy359KfCv/+63KlZk9fo8zkXLtTZvv9i7lf8lNW27z+ULEVVPK7E19dX0oV7RS79WZKys7Pl5+dXYP3q1atr0qRJGjFihObNm6dy5cpp4MCB+vPPPxUQEFDs/XKPiEm8vMvqjntq68flv9iNb/hyq8oH+KneXTUL3S7y/jvU/aWHtWDiCsW/uajAcg8PD/38XZJWfGx/Je/groslturBDvoErs3Ly1P176yujd8l243/8G2Syvv76PYG1QpsU668j75avEU//bDTbvzgX8clyVb5KO/voxHv/kvbf/5Lr8XMLfA+pYXtHP7qb+fwsovncLMizuH77lD3YZ204L2vFT9ycYHlHh4e+nl9klZ88rdzePelc/hmB30C1+bl7an6d9XUjyt/sxvf8NWvKhfgq3pNaxS6XWSbCHV/8UEtnLpG8WMKv/IeFlFF586eL/VJiJeXp+o3vU0/rtlhN75h1e8q5++reo2rF7pd5L211b1/Wy384DvFjy+88uftU1bns3Jk+Z9pa86ePicv77LyK+/jsM/gyry8PVW/WZh+XP273fiGb367cHybFHEOtwpX94H3a+G0tYoft6LQde554A6t+jzRloRI0u7fDynqntGlPgmxff+t+8Nu/Ie1O1Te31e3Nyzi+2/RFv30vf135sG/jkmSQqqW3sp/ceXLwyVfV+tSS1ZaWprdeFpamipVqlToNm3bttWGDRu0fv16bdq0Sf/85z91/Phx3XLLLcXeL4mISUKq3yRvHy8d3pNqN35k74W/8Ko1Qwrdbtcvf6lng1e0YMKKQq+oWa1WffDv/9NPX9vfr3B3x4aSpP3JRxwQvesLqVJR3t5ldfiA/X+0jlzsHa56640Ftkk9clpT31mhQ3/7z9ndresoNzff9l7Z53PV519xin1zqdJL8RWikFuLOocvfHFd9hxu9KoWTFyh/EL6jq1Wqz4Y8XnBc7hDQ0nS/p2l5ByudpO8fMrq8MXjecmR/RcS46q3FX5RYde2A+p590gtmLq6yKvut0VU0dnT5/T69Kf1+e9jtPiPd/TK+z1VMTjQsR/CxYXcEiQv77I6/Nfffk9c/LdetcZNhW63a/th9bx/nBbM+E75+YUf4+Wf/qTKt96ox59uofIBvgqvf4s697hHm9cnK+NM6Wh/C7nlxgvHd9/fz+GLx/e2wi8q7Np+UD1bva0F074t9ByuVLWi/AP9lHbklPq/8Zg+2/Kmlu54WyOm99JNIRUc/0FKGNv33/6ivv8KntepR05r6tivCvn+q6vc3LwC7wX3Fx4eLn9/fyUkJNjG0tPTlZSUpMjIyALrb9myRT169FBeXp6Cg4Pl7e2tVatWyc/PT40bNy72fl2mNSsvL0+rVq1SYmKijh49qpycHPn5+alSpUqKjIxU+/btbTfDlETlA8tJks6dPW83fi7jws/lAgqWvSTpxNHThvcVWv1mRY/8h/787YASV283vH1JVN7/whXHc5nZduPnzl34uVwxr0je3Tpc93VsqC8/26yMi39XeXmWAr+sS6PygRfO0aLPYd8C20jSiZTThvcVWv1mRb/Z9eI5/PuVN3AD5S8ev0vH85JzGRfPYf8ijm9qwd7dv7utXhXdGFJBX8//UUvi16tazUqKGvyQ3v1sgAY8FKvsrIJzxLuj8heP4bnMvx3jzAufv5x/4b8nTqSlX/G9f/1pjz6f9YOih3VQ9LAOkqQ/kw7rnWELryXkEuW/5/Dffg9nXukcvvzxrRDkL0l6ZlgH7fztoMa+OE8VbvTX00Mf0ti5ffX8wxOVnVX8nnR389/z+u/ffxfP6+J+/7UJ132dGth9/6H08Pb2VlRUlGJjYxUUFKQqVapo3LhxCgkJUfv27ZWfn6+TJ08qICBAvr6+uu2227Rz50698847euqpp7Rz506NHj1affv2lb+/f7H36xKJyKFDh/Tss88qNTVVERERCg4OVoUKFZSdna3k5GQtXrxYU6ZM0YcffqjKlSs7O9yr4lHm8uUyq8Ux/cNVa4Xo7UUvypKXr7d6TZfVeo1Ptykhylzh+FqKcRzuaROuV0Y9rh2/HtCHU4o3C1RpcuVz2DHnWtWaIXr78xdlybforWdmlJpz+ErHtzjncFEmvbxQlrx87frtwhNyd2zeq/27UjR+8Qu67/FIfTW38Mkc3M0Vj/E1nMMD/vOo2ne5U/Onfatff9qjSlUqKur5dho982m9+swsu5Yid+XhcX2Or9fFm9dPn8jQ6P6f2H4nHN1/XBM/H6i2jzTW1wsTLvcWbs0x33919cpbF7//JvH9VxwWq2vOmnUtYmJilJeXp+HDh+v8+fOKjIzUrFmz5OXlpUOHDqldu3YaM2aMunTpoqCgIE2fPl1jx45Vp06ddPPNN2vAgAHq1auXoX26RCIycuRIVa1aVZ9//nmhN7ikp6dr0KBBGjlypKZPn+6ECK/duYsz0/j97YrQpavImQ6Yuab+PbX170/6KyvzvF56dLyO/nXsyhu5icyLV+D8ytlf+Sl/8UrQ36/Q/d1j3e5S7xfu128//6U3hy5Ubg5TF/7df89h+2NsO4fPOugc/qifsjKzS985fPEK5N+vXl66inwu/eqvUCb//FeBsaQt+5RxJks1IkrmxZ2rkXmpelfu78f44u+Js5f/PVGUG4MD9dA/IrVw5nrNmbJGkrQ9cZ92bT+kGcteVPsud2rZ/J+uIfKSwXZ8C5zDl47v1Z3Dl670J65PtrswkfzrAWWkZyksospVva+7uHTc/crbT6ZS7O+/J5ur94vt9dvWv/TmkE+VW8SEDHB/np6eGjZsmIYNG1ZgWdWqVbVzp/09tY0bN9Znn312Tft0iUQkMTFRCxYsKPIu+8DAQA0bNkzdu3c3OTLHObIvTfl5+ar8tx7Zyhf7vg/uOnpN79+6S1MNiXtah3anaPg/J11VS1dJduTQSeXnWVS5apDd+KWfD+wr+j+0/YY8qM5PNNO6b7Yr9s0lhT4/BNKRv45dPIft71WoXMNR53Ckhky9eA7/a/JVtXSVZEf3H1d+Xr5Cq9v3c1e++POBP1ML2+yKygX4qsVDDbTz1/3av+u/c8F7eHiorLenzpzIuPqgS5ijB05eOMZ/u2escrULPx/Ym1bYZld0c2gFlSlTRkm/7LcbP7AnTWdOZerWmoXf6Oluju4/UfjxvXiPwoE9V3cOHz1wQhaLRV7eBf/L4ulZRtlFzLRVWhw5dOrC7+aqfzvutxTj+2/YQ+r8xF1a981viv3PEp4fAtO5xM3qAQEBSk29/C+oI0eO2E0nVtLkZudp+4+7dU8n+xt4Wjx8pzLOnNPOQq5YFlfkfbdr2PRn9MfmPRrS4Z1Sl4RIUm5Ovrb/sl/3tAm3G2/RNkIZZ89r547DhW739PPt1PmJZlo0b5PG/nsxSchl5Gbnafum3bqnYyO78RYPN3bMORz3jP5I3KMhHd8tdUmIdPH4JuzRPQ82sBtv0aGBMs5kadev+4vY8grvm5OnfqMe17+ev89u/K77b5evn7d+2/TnVcdc0uTm5Gn7lr90z3317MZb3F9PGelZ2rX90FW976UEp96d1e3Gq1S/SRUqltfRQ6XjgXu5OXnanrhP9zxwh914iwfuuHB8tx28qvc9fy5Hvyfu0z3t75CX93/vFW3YvKb8yvtoR2LpnjUrNyfvwvdf27p24xe+/7KK/v4bcJ86P3GXFs39UWNfX0QSYpCzZ8dy9KxZzuISFZGuXbvqlVde0QsvvKC77rpLoaGh8vb2Vk5OjlJTU7V582bFxsaqa9euzg71mnw6/iuN+WKQXp/dVyvnbVRE0zB1Hdhe8W8uVnZWjsoF+Kpanco6ui+t2FcpvXzK6sVJPXUu47w+nfCVqtWxb7M4fuSUjh85VcTW7mV+/Pca+/5Ten1MV61c9qsi6t+irj3uVvzUNcrOzlO58t6qVuNmHT10SmdOn9NttSvpn0/do507Duv7NTsUfrt9ef/AvmO2m1hxwacTvtKYRYP0+qy+Wjl/oyIib1PXAe0VP+qLC+ewv6+q1QnV0b+OGTuH33vq4jm8QtXq2D/V9fiRUzpeSpLrBVNW6+35/fTatF5atTBBde+srsf7ttHsscuVfT5X5fx9VK1WiI7uP64zJ4v3NPTc7Dz9X9xa9RjykE4dz1DiuiRVrxOqqEEP6seV27Xtx93X+VO5lgUz1untWc/otYndtGrxVtVtWE2PP9NSsyesvHCMy/uoWliwjh48qTPFfOL8mVOZWjLnR3V9uqUk6Zcf/1Rw5RvUvX9bpR4+pW/+L/F6fiSXsuD9NXr7kz56bUqUVv1fouo2rq7He7fS7HFf//ccrllJRw+cKPY5LEmzY7/Wu/Oe08gPn9WiD9frhpsC9MxLHZT86379tHbHld/Azc2f9b3Gxj2l19/5p1Yu/VkRDaqp61N3K37Kmv+e1zVu1tFDJy9+/4Xonz3v0c7fD+n71TsUfntVu/e78P13da2KgBEukYgMHDhQZcqU0bvvvqtz5wpOj1q+fHl1795dL7zwghOic5xtPyRrdM/p6vHKIxoxp79OHD2tD//zuRa/f+HGsJr1q+ndZcM0/vnZWv3pj8V6z4imNXVj6A2SpDGLBxdYPvedLzX3ndLxVN9tW/7SqJc/U48+rfWfcf/SiWNn9eHk1Vo0b5MkqWadUI2b0Uuxby7R6uXb1KJNXZUp46E69apo0uzoAu83rO9H+u3nq7sK7a62/bBTo3tNV4+XH9GIT/pdOIffWKTFcRfP4QbV9O7SoRo/YLZWL9hUrPeMaBqmG0NukCSNWTSowPK57y7T3HdLyTn842691Xe2ogY/pBEfPKvjqac1660vtfiD7yRJYbffonc/G6Dxg+drzeebi/2+n05epTMnMtSpZwt17HG3zp46pxXzftTcCYU/E8OdbUvYq7demK+oAe00YkqUjqema1bsN1p88enTYRGV9e7HvTX+tc+1ZknxH6T54bivdTzljDr8q5ke79VCJ4+d1c8bd+vjSatt9/+UBtt+2qO3np+jqBfaa8T0Xjqeckaz3vlKi2ddeE5QWL0qendeP41/aaHWLN5S7PdN/mW/Xu4+XT2HPKjX339K2Vm52rTmd304Zvk1TTLgLrYl7tOolxaqR982+s/4bjqRlq4PJ63WoosP660ZHqpxM59W7BtfaPWyX9WibV2VKVNGdW6vqkkf9y7wfsP6zNZvW/8y+VOgNPKwutCUNLm5ufrjjz+UmpqqrKws+fr6KiQkROHh4QUeOX8tHgwq+I8OjmMNq3rllXBNPPYVXmqHY3iUK+fsENyfP8f4ussp3fdOXG+WCpzD19vKrW86O4QiJR6o7uwQChVZ7S9nh2CIS1RELvHy8lL9+vWdHQYAAACA68wlblYHAAAAULq4VEUEAAAAcHXu+EBDZ6AiAgAAAMB0JCIAAAAATEdrFgAAAGBASXx4oCuiIgIAAADAdCQiAAAAAExHaxYAAABgQL6Va/mOwFEEAAAAYDoSEQAAAACmozULAAAAMMDCtXyH4CgCAAAAMB2JCAAAAADT0ZoFAAAAGMADDR2DiggAAAAA05GIAAAAADAdrVkAAACAATzQ0DE4igAAAABMRyICAAAAwHS0ZgEAAAAGWJg1yyGoiAAAAAAwHYkIAAAAANPRmgUAAAAYkM+1fIfgKAIAAAAwHYkIAAAAANPRmgUAAAAYwAMNHYOjCAAAAMB0JCIAAAAATEdrFgAAAGCAhWv5DsFRBAAAAGA6EhEAAAAApqM1CwAAADAg3+rh7BDcAhURAAAAAKYjEQEAAABgOhIRAAAAAKbjHhEAAADAgHyu5TsERxEAAACA6UhEAAAAAJiO1iwAAADAAIuVa/mOwFEEAAAAYDoSEQAAAACmozULAAAAMIBZsxyDowgAAADAdCQiAAAAAExHaxYAAABgQL7Vw9khuAUqIgAAAABMRyICAAAAwHS0ZgEAAAAGWLiW7xClMhFJHhXu7BDcWpmK2c4Owe1ZrTWcHYJbK+ud7+wQ3F5eip+zQ3B7Fj+Ls0Nway/es9rZIQAlHukcAAAAANOVyooIAAAAcLXyrVzLdwSOIgAAAADTkYgAAAAAMB2tWQAAAIABFvFAQ0egIgIAAADAdCQiAAAAAExHaxYAAABgALNmOQZHEQAAAIDpSEQAAAAAmI7WLAAAAMCAfK7lOwRHEQAAAIDpSEQAAAAAmI7WLAAAAMAAi5UHGjoCFREAAAAApiMRAQAAAGA6WrMAAAAAA5g1yzE4igAAAABMRyICAAAAwHS0ZgEAAAAGWKxcy3cEjiIAAABQylksFk2ePFktW7ZUw4YN1bt3bx08eLDI9U+cOKEhQ4borrvuUrNmzTRo0CClpqYa2ieJCAAAAFDKxcXFaf78+Ro1apQWLFggi8Wi6Oho5eTkFLr+iy++qCNHjmj27NmaPXu2jhw5oueff97QPklEAAAAAAPy5eGSr6uVk5Oj+Ph4xcTEqHXr1goPD9fEiROVkpKiVatWFVg/PT1dmzdvVu/evVW3bl1FRESoT58+2r59u06fPl3s/ZKIAAAAAKVYcnKyMjMz1bx5c9tYYGCgIiIilJiYWGB9X19flS9fXkuWLFFGRoYyMjK0dOlS1ahRQ4GBgcXeLzerAwAAAKVYSkqKJCk0NNRuPDg42Lbsf3l7e2vs2LEaMWKEmjRpIg8PDwUHB2vu3LkqU6b4dQ4SEQAAAMAAV501q127dpddvnbt2kLHs7KyJF1IMP6Xj4+Pzpw5U2B9q9WqP/74Q40aNVJ0dLTy8/M1ceJE9e/fX59++qn8/f2LFS+JCAAAAFCK+fr6Srpwr8ilP0tSdna2/Pz8Cqz/9ddfa+7cuVq3bp0t6Zg+fbratGmjzz//XL169SrWfklEAAAAADdQVMXjSi61ZKWlpalatWq28bS0NNWpU6fA+lu2bFGNGjXsKh8VKlRQjRo1tH///mLv1zXrSgAAAICLcvbsWI6eNSs8PFz+/v5KSEiwjaWnpyspKUmRkZEF1g8JCdH+/fuVnZ1tGzt37pwOHTqk6tWrF3u/JCIAAABAKebt7a2oqCjFxsZq7dq1Sk5O1qBBgxQSEqL27dsrPz9fx44d0/nz5yVJnTt3lnThWSLJyclKTk7W4MGD5ePjoy5duhR7vyQiAAAAQCkXExOjrl27avjw4erWrZs8PT01a9YseXl56ejRo2rRooVWrFgh6cJsWvPnz5fValXPnj319NNPy8vLS/Pnz1dAQECx9+lhtVqt1+sDuarq7493dghurUzF7CuvhGtitV59+RVXVtY739khuL28lII3P8KxLH4WZ4fg1l68Z7WzQ3B7g+oWfJCeq3hrRydnh1Co1+std3YIhlARAQAAAGA6EhEAAAAApmP6XgAAAMCAfBd9oGFJw1EEAAAAYDoSEQAAAACmozULAAAAMMByDQ8PxH9REQEAAABgOhIRAAAAAKajNQsAAAAwgFmzHIOjCAAAAMB0JCIAAAAATEdrFgAAAGCAxcqsWY5ARQQAAACA6UhEAAAAAJiO1iwAAADAgHyu5TsERxEAAACA6UhEAAAAAJiO1iwAAADAAGbNcgwqIgAAAABMRyICAAAAwHS0ZgEAAAAGWLiW7xAcRQAAAACmIxEBAAAAYDqXac3q0aOHPDyKNwPBJ598cp2jAQAAAAqXz6xZDuEyiUiLFi00adIk1ahRQ/Xr13d2OAAAAACuI5dJRPr27St/f3+NHz9eM2bMUNWqVZ0dEgAAAIDrxKXuEenevbuaNm2qd99919mhAAAAAIWyWD1c8lXSuExF5JKRI0dqx44dzg4DAAAAwHXkcolIcHCwgoODnR0GAAAAgOvI5RIRAAAAwJVZrC51d0OJxVEEAAAAYDoSEQAAAACmozULAAAAMCBfJW+GKldERQQAAACA6UhEAAAAAJiO1iwAAADAgJL48EBXREUEAAAAgOlIRAAAAACYjtYsAAAAwAAeaOgYHEUAAAAApiMRAQAAAGA6WrMAAAAAAyw80NAhqIgAAAAAMB2JCAAAAADT0ZoFAAAAGJDPAw0dgooIAAAAANORiAAAAAAwHa1ZAAAAgAE80NAxOIoAAAAATEciAgAAAMB0tGYBAAAABliYNcshqIgAAAAAMB2JCAAAAADT0ZoFAAAAGGARrVmOQEUEAAAAgOlIRAAAAACYjtYsAAAAwABmzXIMKiIAAAAATEciAgAAAMB0tGYBAAAABlisXMt3BI4iAAAAANORiAAAAAAwHa1ZAAAAgAHMmuUYVEQAAAAAmI5EBAAAAIDpaM0CAAAADLCI1ixHoCICAAAAwHQkIgAAAABMR2sWAAAAYACzZjkGFREAAAAApiMRAQAAAGA6WrMAAAAAA2jNcgwqIgAAAABMRyICAAAAwHS0ZgEAAAAG0JrlGFREAAAAAJiORAQAAAAo5SwWiyZPnqyWLVuqYcOG6t27tw4ePFjoulOmTFGdOnUKfb366qvF3qeH1Wq1OuoDlBRL9zZ0dgjANQkqk+HsENxarjydHYLby7dyHex6a+eX7+wQ3NoJS6azQ3B7N1c+7OwQitTx+xhnh1Cor+6dfNXbTp06VXPnztXYsWMVEhKicePG6dChQ1q2bJm8vb3t1s3MzNS5c+fsxmbPnq1PP/1UCxYsUJ06dYq1T74JAAAAgFIsJydH8fHxiomJUevWrRUeHq6JEycqJSVFq1atKrB++fLldfPNN9tex44d0yeffKIRI0YUOwmRSEQAAACAUi05OVmZmZlq3ry5bSwwMFARERFKTEy84vYjR45UkyZN9NhjjxnaL7NmAQAAAAZY5JqzZrVr1+6yy9euXVvoeEpKiiQpNDTUbjw4ONi2rCjr1q3TL7/8oiVLlhQ/0IuoiAAAAAClWFZWliQVuBfEx8dH2dnZl9129uzZatOmjerWrWt4v1REAAAAADdQVMXjSnx9fSVduFfk0p8lKTs7W35+fkVud+TIESUkJGjmzJlXtV8SEQAAAMAAd3ug4aWWrLS0NFWrVs02npaWdtmbz9esWaOgoCDdc889V7VfWrMAAACAUiw8PFz+/v5KSEiwjaWnpyspKUmRkZFFbrdlyxY1bdpUZcteXW2DiggAAABQinl7eysqKkqxsbEKCgpSlSpVNG7cOIWEhKh9+/bKz8/XyZMnFRAQYNe6lZSUpMcff/yq90siAgAAABjgbq1ZkhQTE6O8vDwNHz5c58+fV2RkpGbNmiUvLy8dOnRI7dq105gxY9SlSxfbNseOHdMNN9xw1fvkyepACcST1a8vnqx+/fFk9euPJ6tfXzxZ/fpz5Ser3//dIGeHUKjVrSc6OwRD+CYAAAAAYDpaswAAAAAD3LE1yxmoiAAAAAAwHYkIAAAAANPRmgUAAAAYQGuWY1ARAQAAAGA6EhEAAAAApqM1CwAAADDASmuWQ1ARAQAAAGA6EhEAAAAApqM1CwAAADDAIlqzHIGKCAAAAADTkYgAAAAAMB2tWQAAAIABPNDQMaiIAAAAADAdiQgAAAAA09GaBQAAABjAAw0dg4oIAAAAANORiAAAAAAwHa1ZAAAAgAHMmuUYVEQAAAAAmI5EBAAAAIDpaM0CAAAADGDWLMegIgIAAADAdCQiAAAAAExHaxYAAABgALNmOQYVEQAAAACmIxEBAAAAYDpaswAAAAADrFZnR+AeqIgAAAAAMB2JCAAAAADT0ZoFAAAAGGARs2Y5AhURAAAAAKYjEQEAAABgOlqzAAAAAAOsPNDQIaiIAAAAADAdiQgAAAAA09GaBQAAABhgoTXLIaiIAAAAADAdiQgAAAAA09GaBQAAABhgtTo7AvdARQQAAACA6VwmEfnrr780ZcoUjR49Wt9//32B5RkZGXr11VedEBkAAAAAR3OJRGTr1q3q3Lmzli1bph9++EF9+/bVCy+8oJycHNs658+f15IlS5wXJAAAAKALDzR0xVdJ4xKJyPjx4/X4449r1apVWrlypd577z1t2LBB/fv3V15enrPDAwAAAOBgLpGI7Ny5U0899ZTt5wceeEAffPCBtm7dqpdfftmJkQEAAAC4HlwiEfH399eJEyfsxho3bqxx48bp66+/1pgxY5wUGQAAAGDP2S1YtGY5UKtWrfTmm2/q119/VW5urm38vvvu02uvvaaPP/5YI0eOdGKEAAAAABzJJRKRIUOG6MYbb1S3bt20adMmu2VRUVEaMWKEvv32WydFBwAAAMDRXOKBhhUqVFB8fLwOHDigihUrFlj+5JNPqnnz5lq1apUTogMAAAD+y1IC26BckUskIpdUq1atyGU1atRQ3759TYwGAAAAwPXiEq1ZAAAAAEoXl6qIAAAAAK7OanV2BO6BiggAAAAA05GIAAAAADAdrVkAAACAASXx4YGuiIoIAAAAANORiAAAAAAwHa1ZAAAAgAG0ZjkGFREAAAAApiMRAQAAAGA6WrMAAAAAA3ieoWNQEQEAAABgOhIRAAAAAKajNQsAAAAwgFmzHIOKCAAAAADTkYgAAAAAMB2tWQAAAIARTJvlEFREAAAAAJiORAQAAACA6WjNAgAAAAxg1izHoCICAAAAwHQkIgAAAABMR2sWAAAAYICVWbMcgooIAAAAANORiAAAAAAwHYkIAAAAYIDV6uGSr2thsVg0efJktWzZUg0bNlTv3r118ODBItfPzc3V+PHjbetHRUXpjz/+MLRPEhEAAACglIuLi9P8+fM1atQoLViwQBaLRdHR0crJySl0/TfeeEOLFy/W22+/rUWLFikoKEi9e/fW2bNni71PEhEAAACgFMvJyVF8fLxiYmLUunVrhYeHa+LEiUpJSdGqVasKrH/w4EEtWrRIb731llq2bKmwsDCNHj1a3t7e+v3334u9X2bNAgAAAIxwswcaJicnKzMzU82bN7eNBQYGKiIiQomJierUqZPd+hs3blRAQIDuvfdeu/W//fZbQ/slEQEAAADcQLt27S67fO3atYWOp6SkSJJCQ0PtxoODg23L/te+fft0yy23aNWqVZo5c6ZSU1MVERGhV155RWFhYcWOl9YsAAAAoBTLysqSJHl7e9uN+/j4KDs7u8D6GRkZ2r9/v+Li4jR48GBNmzZNZcuW1ZNPPqkTJ04Ue79URAAAAAADXPWBhkVVPK7E19dX0oV7RS79WZKys7Pl5+dXYP2yZcsqIyNDEydOtFVAJk6cqFatWumLL75QdHR0sfZLRQQAAAAoxS61ZKWlpdmNp6WlqVKlSgXWDwkJUdmyZe3asHx9fXXLLbfo0KFDxd4viQgAAABQioWHh8vf318JCQm2sfT0dCUlJSkyMrLA+pGRkcrLy9P27dttY+fPn9fBgwd16623Fnu/tGYBAAAARrhoa9bV8vb2VlRUlGJjYxUUFKQqVapo3LhxCgkJUfv27ZWfn6+TJ08qICBAvr6+atKkie6++269/PLLGjlypG644QZNnjxZnp6eevTRR4u9XyoiAAAAQCkXExOjrl27avjw4erWrZs8PT01a9YseXl56ejRo2rRooVWrFhhW3/KlClq2rSpBgwYoK5duyojI0OffPKJgoKCir1PD6vVVW+3uX6W7m3o7BCAaxJUJsPZIbi1XHk6OwS3l2/lOtj11s4v39khuLUTlkxnh+D2bq582NkhFOm2+W87O4RC7X3yNWeHYAitWQAAAIABVjd7oKGzcEkKAAAAgOlIRAAAAACYjtYsAAAAwIhSd4f19UFFBAAAAIDpSEQAAAAAmI7WLAAAAMAAZs1yDCoiAAAAAExHIgIAAADAdLRmAQAAAEYwa5ZDUBEBAAAAYLpSWRF50C/d2SG4tXRrtrNDcHuBHj7ODsGtZVjPOzsEt1exTDlnh+D2Xkpt5OwQ3NrvT9V2dghu75ttzo4A11upTEQAAACAq8esWY5AaxYAAAAA05GIAAAAADAdrVkAAACAEcya5RBURAAAAACYjkQEAAAAgOlozQIAAACMoDXLIaiIAAAAADAdiQgAAAAA09GaBQAAABhh5YGGjkBFBAAAAIDpSEQAAAAAmI7WLAAAAMAAK7NmOQQVEQAAAACmIxEBAAAAYDpaswAAAAAjaM1yCCoiAAAAAExHIgIAAADAdLRmAQAAAEbwQEOHoCICAAAAwHQkIgAAAABMR2sWAAAAYIAHs2Y5BBURAAAAAKYjEQEAAABgOlqzAAAAACNozXIIKiIAAAAATEciAgAAAMB0tGYBAAAARvBAQ4egIgIAAADAdCQiAAAAAExHaxYAAABgBLNmOQQVEQAAAACmIxEBAAAAYDpaswAAAAAjaM1yCCoiAAAAAExHIgIAAADAdLRmAQAAAEbQmuUQVEQAAAAAmI5EBAAAAIDpaM0CAAAAjLB6ODsCt0BFBAAAAIDpSEQAAAAAmI7WLAAAAMAAD2bNcggqIgAAAABMRyICAAAAwHS0ZgEAAABG0JrlEFREAAAAAJiORAQAAACA6UhEAAAAAJiORAQAAACA6QzdrH727Fl98skn2r59u3x9fRUcHKzatWurTp06qlWrlnx9fa9XnAAAAADciKFEZNiwYVq/fr1uuukm3XDDDdq0aZPOnDkjDw8PeXh46JZbbrElJrVr11bt2rV16623ysPD43rFDwAAAJiKBxo6hqFEZNOmTerQoYPGjx9vG0tNTdWuXbu0c+dO7dq1S7t27dJ3332n3NxceXh4yNfXV7/88ssV3zs7O1u7d+9WzZo15evrqz/++ENz585VamqqatWqpZ49eyokJMT4JwQAAADgcgwlIgEBAWratKndWKVKlVSpUiW1bNnSNpafn699+/bZkpMr2bt3r3r16qW0tDRVrlxZo0ePVv/+/VWlShXVrFlTa9as0eLFizV//nyFhYUZCRkAAACACzJ0s/qDDz6obdu2XXE9T09P1axZUx07dtSgQYOuuP4777yjhg0basmSJWratKn69eunDh06aPny5Zo0aZK+/vprtWjRQmPGjDESLgAAAOB4Vg/XfJUwhhKRxx9/XLt27dLKlSsdGsTmzZv14osvKjw8XC+99JKys7MVFRVlu7ekbNmy6tu3r7Zu3erQ/QIAAABwDkOtWY899pjKlCmjF198Uc2aNVObNm1Ur1491alTRwEBAVcdhK+vr7KysiRJQUFB+uc//ykfHx+7ddLT069pHwAAAABch6FEZMyYMbYb0nfu3KmffvrJVrUICQlRnTp1VKdOnWK1Y/2vFi1aaNSoURo9erRq1qypkSNH2pZZLBZt2rRJo0aN0n333WfofQEAAACHY9YshzBcEflfJ0+e1M6dO203pe/cuVMJCQmGE5FXX31V/fv31/Tp0xUbG2u37JtvvtHgwYPVqlUrDR482ND7AgAAAHBNhhKRvwsKClLz5s3VvHlz25jVajxFDAoK0oIFC5Senl5gWfPmzbVs2TLVqlXrWkIFAAAA4EKuKREpzLU8vDAwMLDAWMWKFVWxYsVrCQkAAABwHFqzHMLQrFkAAAAA4AgkIgAAAABM5/DWLAAAAMCdedCa5RBURAAAAIBSzmKxaPLkyWrZsqUaNmyo3r176+DBg0Wu/+WXX9oe3fG/r0OHDhV7n1REAAAAgFIuLi5O8+fP19ixYxUSEqJx48YpOjpay5Ytk7e3d4H1d+7cqaZNm2rChAl240FBQcXeJxURAAAAwAiri76uUk5OjuLj4xUTE6PWrVsrPDxcEydOVEpKilatWlXoNrt27VKdOnV088032708PT2LvV8SEQAAAKAUS05OVmZmpt2zAQMDAxUREaHExMRCt9m5c6fCwsKuab+0ZgEAAABuoF27dpddvnbt2kLHU1JSJEmhoaF248HBwbZl/+vMmTNKTU3Vli1bNH/+fJ06dUr169fXsGHDVKNGjWLHS0UEAAAAMMLZLVgObs3KysqSpAL3gvj4+Cg7O7vA+rt3775wGKxWjRkzRu+9956ys7P15JNP6vjx48XeLxURAAAAwA0UVfG4El9fX0kX7hW59GdJys7Olp+fX4H1mzRpok2bNqlixYry8PCQJE2dOlWtW7fW4sWL1adPn2Ltl4oIAAAAUIpdaslKS0uzG09LS1OlSpUK3SYoKMiWhEiSn5+fqlatqtTU1GLvl0QEAAAAMMDD6pqvqxUeHi5/f38lJCTYxtLT05WUlKTIyMgC6y9cuFDNmjXTuXPnbGMZGRn666+/VLNmzWLvl0QEAAAAKMW8vb0VFRWl2NhYrV27VsnJyRo0aJBCQkLUvn175efn69ixYzp//rwk6d5775XFYtFLL72k3bt3a/v27Ro4cKCCgoLUpUuXYu+XRAQAAAAo5WJiYtS1a1cNHz5c3bp1k6enp2bNmiUvLy8dPXpULVq00IoVKyRdaOX66KOPdO7cOXXr1k29evVSQECAPvnkE/n4+BR7nx5Wq/UaCjklU/bR25wdgltLtxacXQGOFehR/H/kMC7DmuPsENxexTLlnB2C23sptZGzQ3Brvz9V29khuL1vto1ydghFqvnuRGeHUKg/Xxrk7BAMoSICAAAAwHQkIgAAAABMx3NEAAAAACNK3Y0N1wcVEQAAAACmIxEBAAAAYDpaswAAAAADruXhgfgvKiIAAAAATEciAgAAAMB0tGYBAAAARtCa5RBURAAAAACYjkQEAAAAgOlozQIAAAAMYNYsx6AiAgAAAMB0JCIAAAAATEdrFgAAAGAErVkOQUUEAAAAgOlIRAAAAACYjtYsAAAAwAhasxyCiggAAAAA05GIAAAAADAdrVkAAACAATzQ0DGoiAAAAAAwHYkIAAAAANORiAAAAAAwHYkIAAAAANORiAAAAAAwHbNmAQAAAEYwa5ZDUBEBAAAAYDoSEQAAAACmozULAAAAMIAHGjoGFREAAAAApiMRAQAAAGA6WrMAAAAAI2jNcohSmYg8WqeVs0Nwa5asLGeH4PbK+Pk5OwS3Zjmf7ewQ3J/V4uwI3J8HTQ/Xk0eZvc4OASjx+C0FAAAAwHSlsiICAAAAXDVasxyCiggAAAAA05GIAAAAADAdrVkAAACAATzQ0DGoiAAAAAAwHYkIAAAAANPRmgUAAAAYQWuWQ1ARAQAAAGA6EhEAAAAApqM1CwAAADCAWbMcg4oIAAAAANORiAAAAAAwHa1ZAAAAgBG0ZjkEFREAAAAApiMRAQAAAGA6WrMAAAAAI2jNcggqIgAAAABMRyICAAAAwHS0ZgEAAAAG8EBDx6AiAgAAAMB0JCIAAAAATEdrFgAAAGAErVkOQUUEAAAAgOlIRAAAAACYjtYsAAAAwAhasxyCiggAAAAA05GIAAAAADAdrVkAAACAATzQ0DGoiAAAAAAwHYkIAAAAANPRmgUAAAAYQWuWQ1ARAQAAAGA6EhEAAAAApqM1CwAAADCAWbMcg4oIAAAAANORiAAAAAAwHa1ZAAAAgBG0ZjkEFREAAAAApiMRAQAAAGA6EhEAAAAApiMRAQAAAIywuujrGlgsFk2ePFktW7ZUw4YN1bt3bx08eLBY23755ZeqU6eODh06ZGifJCIAAABAKRcXF6f58+dr1KhRWrBggSwWi6Kjo5WTk3PZ7Q4fPqyRI0de1T5JRAAAAIBSLCcnR/Hx8YqJiVHr1q0VHh6uiRMnKiUlRatWrSpyO4vFomHDhqlevXpXtV8SEQAAAMAADxd9Xa3k5GRlZmaqefPmtrHAwEBFREQoMTGxyO2mT5+u3Nxc9e3b96r2y3NEAAAAgFIsJSVFkhQaGmo3HhwcbFv2d7/99pvi4+P1+eefKzU19ar2SyICAAAAuIF27dpddvnatWsLHc/KypIkeXt72437+PjozJkzBdY/d+6chg4dqqFDh6p69eokIgAAAIAp3OzJ6r6+vpIu3Cty6c+SlJ2dLT8/vwLrjx49WjVq1NATTzxxTfslEQEAAADcQFEVjyu51JKVlpamatWq2cbT0tJUp06dAusvWrRI3t7eatSokSQpPz9fktSpUyc999xzeu6554q1XxIRAAAAoBQLDw+Xv7+/EhISbIlIenq6kpKSFBUVVWD9v8+ktW3bNg0bNkwzZ85U7dq1i71fEhEAAADAAA83a83y9vZWVFSUYmNjFRQUpCpVqmjcuHEKCQlR+/btlZ+fr5MnTyogIEC+vr669dZb7ba/dEN75cqVdcMNNxR7v0zfCwAAAJRyMTEx6tq1q4YPH65u3brJ09NTs2bNkpeXl44ePaoWLVpoxYoVDt2nh9Vqdemcrk+fPho9erSCg4Md9p4PBj7tsPdCQZaLMy/g+ilTyI1jcBzL+Wxnh+D+rBZnR+D+PLjWeD15lLmWpzagOFaen+fsEIrU4MWJzg6hUNveG+TsEAxxidasJUuWFLksISFBy5cvV1BQkCSpc+fO5gQFAAAAFMalL+OXHC6RiLz55ps6f/68JKmwAs27774rSfLw8CARAQAAANyASyQiixcv1tChQxUYGKixY8eqUqVKtmWNGjXSl19+qVtuucWJEQIAAABwJJdoIK1Ro4YWLlyoO+64Q48++qjDb4QBAAAAHMbqoq8SxiUSEUkqW7asBg8erClTpig2NlZDhw7V2bNnnR0WAAAAgOvAZRKRSyIjI7VkyRJZrVZ16tRJubm5zg4JAAAAgIO5xD0ifxcYGKjx48dryZIlWrx4sXx8fJwdEgAAACDJ/R5o6CwumYhc0rlzZ2bJAgAAANyQy7VmAQAAAHB/Ll0RAQAAAFwOrVkOQUUEAAAAgOlIRAAAAACYjtYsAAAAwABmzXIMKiIAAAAATEciAgAAAMB0tGYBAAAARtCa5RBURAAAAACYjkQEAAAAgOlozQIAAAAMYNYsx6AiAgAAAMB0JCIAAAAATEdrFgAAAGAErVkOQUUEAAAAgOlIRAAAAACYjtYsAAAAwAhasxyCiggAAAAA05GIAAAAADAdrVkAAACAATzQ0DGoiAAAAAAwHYkIAAAAANPRmgUAAAAYQWuWQ1ARAQAAAGA6EhEAAAAApqM1CwAAADDAw0pvliNQEQEAAABgOhIRAAAAAKajNQsAAAAwgs4sh6AiAgAAAMB0JCIAAAAATEdrFgAAAGCAB61ZDkFFBAAAAIDpSEQAAAAAmI7WLAAAAMAIWrMcgooIAAAAANORiAAAAAAwHa1ZAAAAgAHMmuUYVEQAAAAAmI5EBAAAAIDpaM0CAAAAjKA1yyGoiAAAAAAwHYkIAAAAANPRmgUAAAAYwKxZjkFFBAAAAIDpSEQAAAAAmI7WLAAAAMAIWrMcgooIAAAAANOVyopIfkaGs0MArgnnMIAr8fT3c3YIbo3fw8C1K5WJCAAAAHC1mDXLMWjNAgAAAGA6EhEAAAAApqM1CwAAADDCSm+WI1ARAQAAAGA6EhEAAAAApqM1CwAAADCAWbMcg4oIAAAAANORiAAAAAAwHa1ZAAAAgBG0ZjkEFREAAAAApiMRAQAAAGA6WrMAAAAAAzwszo7APVARAQAAAGA6EhEAAAAApqM1CwAAADCCWbMcgooIAAAAANORiAAAAAAwHa1ZAAAAgAEetGY5BBURAAAAoJSzWCyaPHmyWrZsqYYNG6p37946ePBgkevv2LFDPXv2VKNGjXTXXXdpxIgROnv2rKF9kogAAAAApVxcXJzmz5+vUaNGacGCBbJYLIqOjlZOTk6BdY8fP66nn35aVapU0eLFixUXF6etW7fqlVdeMbRPEhEAAADACKvVNV9XKScnR/Hx8YqJiVHr1q0VHh6uiRMnKiUlRatWrSqw/uHDh9WiRQuNHDlSNWrUUOPGjfXPf/5TGzduNLRfEhEAAACgFEtOTlZmZqaaN29uGwsMDFRERIQSExMLrN+gQQNNmDBBZcteuN18z549Wrp0qe655x5D++VmdQAAAMANtGvX7rLL165dW+h4SkqKJCk0NNRuPDg42LasKA888ID++usvValSRVOnTjUQLRURAAAAwBAPq2u+rlZWVpYkydvb227cx8dH2dnZl902NjZWc+bM0Y033qinnnpKmZmZxd4vFREAAADADRRV8bgSX19fSRfuFbn0Z0nKzs6Wn5/fZbe94447JElTp05Vq1attHr1anXu3LlY+6UiAgAAAJRil1qy0tLS7MbT0tJUqVKlAuvv3btX3333nd1YpUqVdMMNNyg1NbXY+yURAQAAAIywuujrKoWHh8vf318JCQm2sfT0dCUlJSkyMrLA+j/++KNiYmKUnp5uGztw4IBOnTqlsLCwYu+XRAQAAAAoxby9vRUVFaXY2FitXbtWycnJGjRokEJCQtS+fXvl5+fr2LFjOn/+vCSpU6dOuuGGGzRs2DDt3r1bW7ZsUUxMjOrXr682bdoUe78kIgAAAEApFxMTo65du2r48OHq1q2bPD09NWvWLHl5eeno0aNq0aKFVqxYIUm64YYb9PHHH0uSunXrpueff14RERGaNWuWPD09i71PD6v1Gp5+UkLdX+Yfzg4BAIDrytPf39khuLX8jAxnh+D2Vlv+z9khFKnlY7HODqFQP3wx1NkhGEJFBAAAAIDpSEQAAAAAmI7niAAAAABGlL47G64LKiIAAAAATEciAgAAAMB0tGYBAAAABnjQmeUQVEQAAAAAmI5EBAAAAIDpaM0CAAAAjKA1yyGoiAAAAAAwHYkIAAAAANPRmgUAAAAYwKxZjkFFBAAAAIDpSEQAAAAAmI7WLAAAAMAIC71ZjkBFBAAAAIDpSEQAAAAAmI7WLAAAAMAIOrMcgooIAAAAANORiAAAAAAwHa1ZAAAAgAE80NAxXKIismTJEuXk5NiN/fTTT+rTp48eeeQRDRkyRHv27HFSdAAAAAAczSUSkVdffVVnz561/fzDDz/o6aefltVqVYsWLZSWlqYuXbro559/dmKUAAAAABzFJVqzrFb7+ta0adPUq1cvvfzyy7axMWPGKDY2VvPnzzc7PAAAAOC/rPRmOYJLVET+bv/+/Xr44Yftxv71r38pKSnJSREBAAAAcCSXSEQ8PDzsfq5Ro4YyMjLsxk6ePKmAgAAzwwIAAABwnbhMa1a7du1UvXp1hYWFqWzZsho7dqwWLFggb29vJSYmauTIkbr33nudHSoAAABKOWbNcgyXSETWr1+vnTt3ateuXdq5c6dOnTqlvXv3Kj8/X5L03HPPKSwsTEOGDHFypAAAAAAcwSUSkUqVKqlSpUp2FY/8/Hx5enpKkhYuXKiwsLACLVwAAAAASiaXSEQKcykJkaSaNWs6MRIAAADgf9Ca5RAucbM6AAAAgNKFRAQAAACA6Vy2NQsAAABwRR480NAhqIgAAAAAMB2JCAAAAADT0ZoFAAAAGGFxdgDugYoIAAAAANORiAAAAAAwHa1ZAAAAgAHMmuUYVEQAAAAAmI5EBAAAAIDpaM0CAAAAjKAzyyGoiAAAAAAwHYkIAAAAANPRmgUAAAAYwaxZDkFFBAAAAIDpSEQAAAAAmI7WLAAAAMAADzqzHIKKCAAAAADTkYgAAAAAMB2tWQAAAIARzJrlEFREAAAAAJiORAQAAACA6WjNAgAAAAzwsDg7AvdARQQAAACA6UhEAAAAAJiO1iwAAADACGbNcggqIgAAAABMRyICAAAAwHS0ZgEAAABG0JnlEFREAAAAAJiORAQAAACA6WjNAgAAAAzwYNYsh6AiAgAAAMB0JCIAAAAATEdrFgAAAGAErVkOQUUEAAAAgOlIRAAAAACYjtYsAAAAwAiLswNwD1REAAAAAJiORAQAAACA6WjNAgAAAAzggYaOQUUEAAAAgOlIRAAAAACYjtYsAAAAwAhasxyCiggAAAAA05GIAAAAAKWcxWLR5MmT1bJlSzVs2FC9e/fWwYMHi1x/9+7d6tOnj5o1a6bmzZsrJiZGR44cMbRPEhEAAADACKvVNV/XIC4uTvPnz9eoUaO0YMECWSwWRUdHKycnp8C6p06d0tNPPy1fX1/NmTNHH3zwgU6ePKno6GhlZ2cXe5/cIwIAgBvKz8hwdggASoicnBzFx8dr6NChat26tSRp4sSJatmypVatWqVOnTrZrb9mzRqdO3dO7777rnx9fSVJ48aNU+vWrfXzzz+refPmxdovFREAAACgFEtOTlZmZqZdAhEYGKiIiAglJiYWWL958+aKi4uzJSGSVKbMhbQiPT292PulIgIAAAAYYXF2AIVr167dZZevXbu20PGUlBRJUmhoqN14cHCwbdn/qlq1qqpWrWo3NnPmTPn6+ioyMrLY8VIRAQAAAEqxrKwsSZK3t7fduI+PT7Hu+ZgzZ47mzp2roUOHKigoqNj7pSICAAAAuIGiKh5XcqnFKicnx67dKjs7W35+fkVuZ7VaNWnSJE2bNk39+vVTjx49DO2XRAQAAAAwwMPNHmh4qSUrLS1N1apVs42npaWpTp06hW6Tm5urV199VcuXL9err76qXr16Gd4vrVkAAABAKRYeHi5/f38lJCTYxtLT05WUlFTkPR8vvfSSvvnmG40fP/6qkhCJiggAAABQqnl7eysqKkqxsbEKCgpSlSpVNG7cOIWEhKh9+/bKz8/XyZMnFRAQIF9fXy1evFgrVqzQSy+9pKZNm+rYsWO297q0TnFQEQEAAACMcPaDC6/DAw1jYmLUtWtXDR8+XN26dZOnp6dmzZolLy8vHT16VC1atNCKFSskScuXL5ckvfvuu2rRooXd69I6xeFhtbpZk1sx3F/mH84OAQAAAJex2vJ/zg6hSA82+LezQyjUN9tGOTsEQ6iIAAAAADAd94gAAAAARpS+hqLrgooIAAAAANORiAAAAAAwHa1ZAAAAgBG0ZjkEFREAAAAApiMRAQAAAGA6WrMAAAAAIyzODsA9UBEBAAAAYDoSEQAAAACmozULAAAAMMCDWbMcgooIAAAAANORiAAAAAAwHa1ZAAAAgBG0ZjkEFREAAAAApiMRAQAAAGA6WrMAAAAAIyy0ZjkCFREAAAAApiMRAQAAAGA6WrMAAAAAI5g1yyGoiAAAAAAwHYkIAAAAANPRmgUAAAAYQWuWQ1ARAQAAAGA6EhEAAAAApqM1CwAAADCC1iyHoCICAAAAwHQkIgAAAABMR2sWAAAAYISF1ixHoCICAAAAwHQkIgAAAABMR2sWAAAAYITV4uwI3AIVEQAAAACmIxEBAAAAYDpaswAAAAAjeKChQ1ARAQAAAGA6EhEAAAAApqM1CwAAADCCBxo6BBURAAAAAKYjEQEAAABgOlqzAAAAACOYNcshqIgAAAAAMB2JCAAAAADTuUxr1rZt25SQkKA+ffpIkn766Sd99NFHOnTokKpVq6ZnnnlGTZo0cXKUAAAAKPVozXIIl6iIfPPNN+rWrZs2b94sSVq3bp2efvppWa1WtWrVSrm5uerZs6fWrVvn5EgBAAAAOIJLVESmTp2qmJgYPffcc5KkadOm6bnnntMLL7xgW2fatGmaPHmy2rRp46wwAQAAADiIS1REDhw4oI4dO9p+PnTokB544AG7dTp16qQ9e/aYHRoAAABgz2p1zVcJ4xKJyC233KKNGzfafq5bt66Sk5Pt1vntt99UqVIls0MDAAAAcB24RGtW7969NXz4cB06dEidOnVS//799corryg7O1u1atXStm3b9P7772vAgAHODhUAAACAA7hEItK5c2d5eHho8uTJ+vDDD+Xh4SGr1ar//Oc/kqTy5csrOjpavXr1cm6gAAAAgMXi7AjcgkskIpL06KOP6tFHH9W+ffu0b98+ZWRkqGzZsgoJCVG9evXk4+Pj7BABAAAAOIjLJCKX1KhRQzVq1HB2GAAAAACuI5dLRAAAAACXVgJnqHJFLjFrFgAAAIDShUQEAAAAgOlozQIAAACMoDXLIaiIAAAAADAdiQgAAAAA09GaBQAAABhhoTXLEaiIAAAAADAdiQgAAAAA09GaBQAAABhgtVqcHYJboCICAAAAwHQkIgAAAABMR2sWAAAAYASzZjkEFREAAAAApiMRAQAAAGA6WrMAAAAAI6y0ZjkCFREAAAAApiMRAQAAAGA6WrMAAAAAIyw80NARqIgAAAAAMB2JCAAAAADT0ZoFAAAAGMGsWQ5BRQQAAACA6UhEAAAAgFLOYrFo8uTJatmypRo2bKjevXvr4MGDxdouOjpaU6ZMMbxPEhEAAADAAKvF4pKvaxEXF6f58+dr1KhRWrBggS3ByMnJKXKbnJwcvfbaa/rhhx+uap8kIgAAAEAplpOTo/j4eMXExKh169YKDw/XxIkTlZKSolWrVhW6zc8//6wuXbpoy5YtCgwMvKr9kogAAAAApVhycrIyMzPVvHlz21hgYKAiIiKUmJhY6Dbr169Xy5YttWTJEgUEBFzVfpk1CwAAADDCzWbNSklJkSSFhobajQcHB9uW/d2gQYOueb8kIgAAAIAbaNeu3WWXr127ttDxrKwsSZK3t7fduI+Pj86cOeOY4ApBaxYAAABQivn6+kpSgRvTs7Oz5efnd932S0UEAAAAMMLimq1ZRVU8ruRSS1ZaWpqqVatmG09LS1OdOnUcElthqIgAAAAApVh4eLj8/f2VkJBgG0tPT1dSUpIiIyOv236piAAAAAClmLe3t6KiohQbG6ugoCBVqVJF48aNU0hIiNq3b6/8/HydPHlSAQEBtjYuRyARAQAAAIywXtvDA11RTEyM8vLyNHz4cJ0/f16RkZGaNWuWvLy8dOjQIbVr105jxoxRly5dHLZPD6vVzeYfK4b7y/zD2SEAAADgMlZb/s/ZIRTpAd/uzg6hUCvPz3N2CIZwjwgAAAAA09GaBQAAABhgddFZs0oaKiIAAAAATEciAgAAAMB0tGYBAAAARrjhrFnOQEUEAAAAgOlIRAAAAACYjtYsAAAAwABmzXIMKiIAAAAATEciAgAAAMB0tGYBAAAARjBrlkNQEQEAAABgOhIRAAAAAKbzsFqt3PYPAAAAwFRURAAAAACYjkQEAAAAgOlIRAAAAACYjkQEAAAAgOlIRAAAAACYjkQEAAAAgOlIRAAAAACYjkQEAAAAgOlIRAAAAACYjkQEAAAAgOlIRAAAAACYjkQEAAAAgOlIRAAAAACYjkTERVksFk2ePFktW7ZUw4YN1bt3bx08eNDZYbmtGTNmqEePHs4Ow62cPn1aI0aM0L333qvGjRurW7du2rJli7PDcisnTpzQsGHDdNddd6lRo0bq06eP9uzZ4+yw3NK+ffvUqFEjLV682NmhuJXU1FTVqVOnwIvj7FhLlixRhw4ddMcdd6hjx476+uuvnR0SIIlExGXFxcVp/vz5GjVqlBYsWCCLxaLo6Gjl5OQ4OzS3M2/ePL333nvODsPtDB48WL/88osmTJigRYsWqW7dunr22We1d+9eZ4fmNp5//nnt379fM2fO1Oeffy5fX1/16tVLWVlZzg7NreTm5mro0KE6d+6cs0NxO8nJyfLx8dEPP/ygDRs22F4dOnRwdmhuY+nSpXr99dfVvXt3ffXVV+rUqZPt9zPgbCQiLignJ0fx8fGKiYlR69atFR4erokTJyolJUWrVq1ydnhuIzU1Vc8995xiY2NVvXp1Z4fjVvbv36+NGzfqjTfeUJMmTVSjRg39+9//VnBwsJYtW+bs8NzCmTNnVKVKFY0ePVr169dXWFiY+vfvr7S0NO3evdvZ4bmVKVOmyN/f39lhuKVdu3apevXqCg4O1s0332x7+fr6Ojs0t2C1WjVp0iQ99dRT6t69u6pVq6Z+/frp7rvv1ubNm50dHkAi4oqSk5OVmZmp5s2b28YCAwMVERGhxMREJ0bmXnbs2CEvLy99+eWXatCggbPDcSsVK1bUzJkzdccdd9jGPDw85OHhofT0dCdG5j4qVKig8ePHq3bt2pKkkydP6qOPPlJISIhq1qzp5OjcR2JiohYuXKixY8c6OxS3tHPnToWFhTk7DLe1b98+HT58WA8//LDd+KxZs9S3b18nRQX8V1lnB4CCUlJSJEmhoaF248HBwbZluHZt27ZV27ZtnR2GWwoMDFSrVq3sxlauXKn9+/frtddec1JU7uvf//63PvvsM3l7e2vatGkqV66cs0NyC+np6XrppZc0fPjwAr+P4Ri7du1SxYoV1b17d+3bt0+33nqr+vXrp3vvvdfZobmFffv2SZLOnTunZ599VklJSapatar69evH9x9cAhURF3Spv9vb29tu3MfHR9nZ2c4ICbgmP//8s1599VW1b99erVu3dnY4bqdnz55atGiROnXqpOeff147duxwdkhu4Y033lCjRo0KXE2GY+Tl5Wnv3r06c+aMBg4cqJkzZ6phw4bq06ePNm3a5Ozw3EJGRoYk6eWXX1anTp0UHx+ve+65R/379+cYwyVQEXFBl3pjc3Jy7Ppks7Oz5efn56ywgKuyZs0aDR06VI0bN1ZsbKyzw3FLl1qx3nrrLW3btk1z587VmDFjnBxVybZkyRJt2bKFe5quo7JlyyohIUGenp6277rbb79du3fv1qxZs+zak3F1vLy8JEnPPvusHnvsMUlS3bp1lZSUpNmzZ3OM4XRURFzQpRaAtLQ0u/G0tDRVqlTJGSEBV2Xu3LkaOHCg2rRpo+nTp8vHx8fZIbmNkydP6quvvlJeXp5trEyZMqpZs2aB3x0wbtGiRTpx4oRat26tRo0aqVGjRpKk//znP4qOjnZydO6jfPnyBW5Mr1WrllJTU50UkXu59H+GS/eSXVKzZk0dOnTIGSEBdkhEXFB4eLj8/f2VkJBgG0tPT1dSUpIiIyOdGBlQfJemn+7evbsmTJhQoNUQ1+b48eMaPHiwXXtFbm6ukpKSuPnXAWJjY7VixQotWbLE9pKkmJgYvfXWW84Nzk3s3r1bjRs3tvuuk6Tff/+dCRccpF69eipfvry2bdtmN75r1y5Vq1bNSVEB/0Vrlgvy9vZWVFSUYmNjFRQUpCpVqmjcuHEKCQlR+/btnR0ecEX79u3T22+/rfvvv199+/bV8ePHbct8fX0VEBDgxOjcQ+3atXXvvfdq9OjRGj16tCpUqKAZM2YoPT1dvXr1cnZ4JV5R1ecbb7yRyrSDhIWF6bbbbtPIkSP15ptvqmLFivrss8/066+/atGiRc4Ozy34+voqOjpa77//vipVqqT69evrq6++0saNG/XRRx85OzyARMRVxcTEKC8vT8OHD9f58+cVGRmpWbNm2fo9AVe2cuVK5ebmavXq1Vq9erXdsscee4ypUB1kwoQJGj9+vAYNGqSzZ8+qSZMmmjdvnipXruzs0IArKlOmjKZPn67x48frxRdfVHp6uiIiIjR79uwCrUS4ev3795efn58mTpyo1NRUhYWFacqUKWrWrJmzQwPkYbVarc4OAgAAAEDpwj0iAAAAAExHIgIAAADAdCQiAAAAAExHIgIAAADAdCQiAAAAAExHIgIAAADAdCQiAAAAAExHIgIAAADAdCQiAFBCZWdnq169emrSpImmTJni7HAAADCERAQASqi8vDyNHj1alStX1vvvv68TJ044OyQAAIqNRAQASqjy5cvrscceU3R0tKxWq3bu3OnskAAAKDYSEQAo4WrVqiVJ2rNnj5MjAQCg+EhEAKCEy87OliT9+eefTo4EAIDiIxEBgBJu7NixkqS9e/c6ORIAAIqPRAQASrClS5fql19+0Y033khrFgCgRCERAYASKjMzU7Gxsbr33nv16KOP6sSJEzp9+rSzwwIAoFhIRACghJoxY4ZOnTql119/XTVr1pTEfSIAgJKDRAQASqADBw5o9uzZeuaZZ1S9enXbzFncJwIAKClIRACgBBozZoyCgoL03HPPSZJuu+02SVREAAAlR1lnBwAAMOaHH37Qt99+q4kTJ6pcuXKSJH9/f4WGhnLDOgCgxKAiAgAlSG5urt5++201a9ZMHTp0sFsWFhZGaxYAoMQgEQGAEmTevHk6cOCA/v3vfxdYVqtWLR09elSZmZlOiAwAAGM8rFar1dlBAAAAAChdqIgAAAAAMB2JCAAAAADTkYgAAAAAMB2JCAAAAADTkYgAAAAAMB2JCAAAAADTkYgAAAAAMB2JCAAAAADTkYgAAAAAMB2JCAAAAADTkYgAAAAAMB2JCAAAAADT/T99O0qo9QgBMQAAAABJRU5ErkJggg==", - "text/plain": [ - "
    " - ] - }, - "metadata": { - "filenames": { - "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/week43_96_2.png" - } - }, - "output_type": "display_data" - } - ], + "id": "04c4d29a", + "metadata": { + "collapsed": false, + "editable": true + }, + "outputs": [], "source": [ "# visual representation of grid search\n", "# uses seaborn heatmap, you can also do this with matplotlib imshow\n", @@ -2525,8 +2492,10 @@ }, { "cell_type": "markdown", - "id": "0f1aaadd", - "metadata": {}, + "id": "472a5060", + "metadata": { + "editable": true + }, "source": [ "## scikit-learn implementation\n", "\n", @@ -2546,598 +2515,12 @@ { "cell_type": "code", "execution_count": 10, - "id": "156764f3", - "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", - " warnings.warn(\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 1e-05\n", - "Lambda = 1e-05\n", - "Accuracy score on test set: 0.18333333333333332\n", - "\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", - " warnings.warn(\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 1e-05\n", - "Lambda = 0.0001\n", - "Accuracy score on test set: 0.18611111111111112\n", - "\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", - " warnings.warn(\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 1e-05\n", - "Lambda = 0.001\n", - "Accuracy score on test set: 0.13055555555555556\n", - "\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", - " warnings.warn(\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 1e-05\n", - "Lambda = 0.01\n", - "Accuracy score on test set: 0.24444444444444444\n", - "\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", - " warnings.warn(\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 1e-05\n", - "Lambda = 0.1\n", - "Accuracy score on test set: 0.23333333333333334\n", - "\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", - " warnings.warn(\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 1e-05\n", - "Lambda = 1.0\n", - "Accuracy score on test set: 0.12777777777777777\n", - "\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", - " warnings.warn(\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 1e-05\n", - "Lambda = 10.0\n", - "Accuracy score on test set: 0.1527777777777778\n", - "\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", - " warnings.warn(\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 0.0001\n", - "Lambda = 1e-05\n", - "Accuracy score on test set: 0.9111111111111111\n", - "\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", - " warnings.warn(\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 0.0001\n", - "Lambda = 0.0001\n", - "Accuracy score on test set: 0.8888888888888888\n", - "\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", - " warnings.warn(\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 0.0001\n", - "Lambda = 0.001\n", - "Accuracy score on test set: 0.8722222222222222\n", - "\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", - " warnings.warn(\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 0.0001\n", - "Lambda = 0.01\n", - "Accuracy score on test set: 0.8305555555555556\n", - "\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", - " warnings.warn(\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 0.0001\n", - "Lambda = 0.1\n", - "Accuracy score on test set: 0.8888888888888888\n", - "\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", - " warnings.warn(\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 0.0001\n", - "Lambda = 1.0\n", - "Accuracy score on test set: 0.8805555555555555\n", - "\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", - " warnings.warn(\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 0.0001\n", - "Lambda = 10.0\n", - "Accuracy score on test set: 0.8944444444444445\n", - "\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", - " warnings.warn(\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 0.001\n", - "Lambda = 1e-05\n", - "Accuracy score on test set: 0.975\n", - "\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", - " warnings.warn(\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 0.001\n", - "Lambda = 0.0001\n", - "Accuracy score on test set: 0.9777777777777777\n", - "\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", - " warnings.warn(\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 0.001\n", - "Lambda = 0.001\n", - "Accuracy score on test set: 0.9805555555555555\n", - "\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", - " warnings.warn(\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 0.001\n", - "Lambda = 0.01\n", - "Accuracy score on test set: 0.9861111111111112\n", - "\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", - " warnings.warn(\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 0.001\n", - "Lambda = 0.1\n", - "Accuracy score on test set: 0.9805555555555555\n", - "\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", - " warnings.warn(\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 0.001\n", - "Lambda = 1.0\n", - "Accuracy score on test set: 0.9777777777777777\n", - "\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", - " warnings.warn(\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 0.001\n", - "Lambda = 10.0\n", - "Accuracy score on test set: 0.9444444444444444\n", - "\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py:691: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", - " warnings.warn(\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 0.01\n", - "Lambda = 1e-05\n", - "Accuracy score on test set: 0.9861111111111112\n", - "\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 0.01\n", - "Lambda = 0.0001\n", - "Accuracy score on test set: 0.9888888888888889\n", - "\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 0.01\n", - "Lambda = 0.001\n", - "Accuracy score on test set: 0.9888888888888889\n", - "\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 0.01\n", - "Lambda = 0.01\n", - "Accuracy score on test set: 0.9861111111111112\n", - "\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 0.01\n", - "Lambda = 0.1\n", - "Accuracy score on test set: 0.9888888888888889\n", - "\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 0.01\n", - "Lambda = 1.0\n", - "Accuracy score on test set: 0.9722222222222222\n", - "\n", - "Learning rate = 0.01\n", - "Lambda = 10.0\n", - "Accuracy score on test set: 0.9527777777777777\n", - "\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 0.1\n", - "Lambda = 1e-05\n", - "Accuracy score on test set: 0.9027777777777778\n", - "\n", - "Learning rate = 0.1\n", - "Lambda = 0.0001\n", - "Accuracy score on test set: 0.8583333333333333\n", - "\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 0.1\n", - "Lambda = 0.001\n", - "Accuracy score on test set: 0.8722222222222222\n", - "\n", - "Learning rate = 0.1\n", - "Lambda = 0.01\n", - "Accuracy score on test set: 0.9055555555555556\n", - "\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 0.1\n", - "Lambda = 0.1\n", - "Accuracy score on test set: 0.8805555555555555\n", - "\n", - "Learning rate = 0.1\n", - "Lambda = 1.0\n", - "Accuracy score on test set: 0.8722222222222222\n", - "\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 0.1\n", - "Lambda = 10.0\n", - "Accuracy score on test set: 0.8666666666666667\n", - "\n", - "Learning rate = 1.0\n", - "Lambda = 1e-05\n", - "Accuracy score on test set: 0.08611111111111111\n", - "\n", - "Learning rate = 1.0\n", - "Lambda = 0.0001\n", - "Accuracy score on test set: 0.10555555555555556\n", - "\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 1.0\n", - "Lambda = 0.001\n", - "Accuracy score on test set: 0.10555555555555556\n", - "\n", - "Learning rate = 1.0\n", - "Lambda = 0.01\n", - "Accuracy score on test set: 0.17777777777777778\n", - "\n", - "Learning rate = 1.0\n", - "Lambda = 0.1\n", - "Accuracy score on test set: 0.08333333333333333\n", - "\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 1.0\n", - "Lambda = 1.0\n", - "Accuracy score on test set: 0.08888888888888889\n", - "\n", - "Learning rate = 1.0\n", - "Lambda = 10.0\n", - "Accuracy score on test set: 0.09444444444444444\n", - "\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 10.0\n", - "Lambda = 1e-05\n", - "Accuracy score on test set: 0.17222222222222222\n", - "\n", - "Learning rate = 10.0\n", - "Lambda = 0.0001\n", - "Accuracy score on test set: 0.11666666666666667\n", - "\n", - "Learning rate = 10.0\n", - "Lambda = 0.001\n", - "Accuracy score on test set: 0.10555555555555556\n", - "\n", - "Learning rate = 10.0\n", - "Lambda = 0.01\n", - "Accuracy score on test set: 0.1388888888888889\n", - "\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 10.0\n", - "Lambda = 0.1\n", - "Accuracy score on test set: 0.11388888888888889\n", - "\n", - "Learning rate = 10.0\n", - "Lambda = 1.0\n", - "Accuracy score on test set: 0.10555555555555556\n", - "\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Learning rate = 10.0\n", - "Lambda = 10.0\n", - "Accuracy score on test set: 0.09444444444444444\n", - "\n" - ] - } - ], + "id": "42e40777", + "metadata": { + "collapsed": false, + "editable": true + }, + "outputs": [], "source": [ "from sklearn.neural_network import MLPClassifier\n", "# store models for later use\n", @@ -3159,8 +2542,10 @@ }, { "cell_type": "markdown", - "id": "2e5fec48", - "metadata": {}, + "id": "57fc5d93", + "metadata": { + "editable": true + }, "source": [ "## Visualization" ] @@ -3168,38 +2553,12 @@ { "cell_type": "code", "execution_count": 11, - "id": "ace47e70", - "metadata": {}, - "outputs": [ - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAyIAAANdCAYAAABoOchKAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAABZgUlEQVR4nO3dd5hU9dk/4M/SUcAWKYoogoIdjV3sLa+aWFMMGiwglog1lsRYMTawxppYYgsmomgiRmMvsWvUqIhRLCglVgSRsrvvH8gmK6CMDmeW5b5/117Xu2fOmXnm/Misn3me8z1VtbW1tQEAAChQk0oXAAAALHwEEQAAoHCCCAAAUDhBBAAAKJwgAgAAFE4QAQAACieIAAAAhRNEAACAwgkiAABA4QQRYIF23HHHpUePHl/5s/fee3+r17jooovSo0eP+X7Mt/Xoo4+mR48e+f73v1/o6wLAN1FVW1tbW+kiAL6pt99+Ox9++GHd75dccklefvnl/Pa3v63b1qZNm3Tv3v0bv8a4ceMybty49OrVa74e820deeSRee211zJq1KjceOON+e53v1vYawNAqZpVugCAb6NLly7p0qVL3e9LLrlkWrRoUdYA0LFjx3Ts2HG+H/NtTJw4Mffcc09OOeWUXH755Rk6dKggAkCDZjQLWCjccsstWXXVVfPnP/85m2yySdZff/38+9//TnV1da644orstNNOWXPNNdOrV6/85Cc/yeOPP1537JfHrPbee+/86le/yhVXXJEtttgia6yxRn7yk5/khRde+FbHJMkDDzyQ3XbbLWuuuWa23377/PWvf822226biy666Cvf31/+8pfMmDEjm266aX7wgx/krrvuyscffzzbfm+88UZ+/vOfZ/311896662XAQMG5PXXX697fNKkSTnttNOy6aabplevXtl9993zwAMP1D3eo0eP2Wr58ns97rjj0rdv35x00klZZ511ssMOO6S6ujoffvhhTjnllGy55ZZZffXVs/766+eQQw7JmDFj6j3f8OHDs+uuu2attdbKFltskSFDhmTatGl57bXX0qNHj9x000319h87dmxWWWWV3H777V95jgBoWAQRYKFRXV2dq666KqeffnqOP/74dOvWLYMHD84ll1ySH//4x/n973+f0047LR9//HEOO+ywTJkyZa7Pddddd+Xee+/NCSeckHPPPTfvv/9+Dj300FRXV3/jYx5//PEcfPDB6dSpUy666KL06dMnJ510UsaOHfu1723YsGHZdNNN853vfCe77LJLpk+fnltvvbXePuPHj8+Pf/zjvPnmmzn55JNzzjnn5P3330/fvn3z8ccfp7q6Ovvtt1/+8pe/ZMCAAbnkkkuy4oor5pBDDsnTTz89j2d5pqeffjpjx47NxRdfnKOOOipNmjTJgAED8uijj+boo4/OlVdemZ///Od57LHHctJJJ9Udd8MNN+TYY4/Naqutlt/+9rc54IADct1112XQoEFZaaWVstZaa+W2226r91rDhw/PIossku22266kGgGoLKNZwELlwAMPzBZbbFH3+4QJE3LEEUfUu6C9ZcuWOfTQQ/Pqq6/OdcRrxowZufLKK9OmTZskyeTJk3PsscfmlVdeyeqrr/6Njrnooouy0kor5be//W2qqqqSJEsttVSOPPLIr3xPr776al566aVceOGFSZJlllkmG264YW666absu+++dftdc801mTZtWq6++uosvfTSSZKePXtmzz33zPPPP5+ampo8//zzufjii7PNNtskSTbccMO88847efzxx7Puuut+ZR1ffq+nnnpq3Xja+PHj07p16xx77LF1z7PBBhvk7bffrutw1NTU1L32oEGD6p5rypQpueOOOzJ9+vTsvvvuOemkk/LOO+9kueWWSzIziOy4445p1arVPNcHQOXpiAALlVVWWaXe70OGDEnfvn3z4Ycf5umnn86wYcPqRnymTZs21+fp3r17XaBIkg4dOiTJV3ZRvuqYadOm5bnnnst2221XF0KS5Hvf+16aNfvq74yGDRuWdu3aZd11183EiRMzceLEbL/99hk9enS9EbNnnnkmvXr1qgshycxrWe6///5svvnmeeaZZ9K8efNstdVWdY83adIkQ4cOzc9//vOvrOHLFl988XrXyHTo0CHXXnttvvvd72bMmDF59NFHc9111+XZZ5+tO8+jR4/OBx98kG233bbec+2///655ZZb0rx587rAMasr8uyzz+bNN9/MrrvuWlJ9AFSejgiwUFlkkUXq/f7iiy/mlFNOyYsvvpjWrVune/fuWWaZZZIkX7WoYOvWrev93qTJzO91ampqvtExs0ajllpqqXr7NG3aNIsvvvhcn3P69Om5/fbbM3HixGy88cazPT506NBsuOGGSZKPP/44nTt3nutzffzxx1l88cXr6vo2Fl100dm23X777Tn33HMzduzYLL744llllVXqdTFmXdPy5XPwv9q0aZPvfe97uf322/Pzn/88w4cPT9euXbP22mt/65oBKJYgAiy0Jk2alH79+qVHjx654447suKKK6ZJkyZ58MEHc9dddxVay1JLLZXmzZvn/fffr7d9VkiZm/vvvz8fffRRTjvttCy//PL1HvvjH/+Ye+65Jx988EGWWmqptG3btt5Sx7M89thj6dy5c9q2bZuPP/44tbW19boyL7/8cmpra7PaaqslyWzXwXz22Wdf+/6efvrpHHvssdl7772z//7713WDzj777DzzzDNJknbt2iXJbDV+9NFHefnll7P22mtnkUUWye67755bb701L7zwQu66667sv//+X/v6ADQ8RrOAhdYbb7yRjz/+OD/72c/SvXv3uk7AQw89lOSruxvl1rRp06yzzjq59957622/7777MmPGjLkeN2zYsHTs2DE//OEPs8EGG9T72XvvvTN9+vQMGzYsSbLuuuvm+eefr/cf+h988EH69euXBx98MOuuu26mT59e9/6TmV2h448/PpdffnmSmR2J8ePH16vh2Wef/dr399xzz6WmpiaHHnpoXQiprq7OP/7xjyQzz/WKK66YJZZYIvfff3+9Y2+77bYccMABmT59epJkvfXWyworrJBzzjknn376aXbeeeevfX0AGh4dEWCh1bVr17Rp0yaXXXZZmjVrlmbNmuWuu+7KzTffnOSrr/eYHwYOHJi99947AwcOzB577JH33nsvF1xwQZLU61DMMmHChDz88MPp27fvHB//7ne/my5duuSmm25K//79s88++2T48OHp169fBgwYkObNm+fSSy9Nx44d8/3vfz9t2rTJ2muvneOOOy6HH354lltuudx22215/fXXc9pppyVJtthii9xxxx1Za621svzyy+eWW27JW2+99bXvbc0110ySnHrqqdl9993zySef5IYbbsjIkSOTzOyqtGnTJoceemhOPfXULLXUUtlqq60yevToXHjhhenTp08WW2yxuufbfffdM2TIkGy22WZ1wQaABYuOCLDQatu2bS655JLU1tbmsMMOyzHHHJP33nsv119/fRZddNGSl6z9ttZdd91cdNFFGT16dA4++OBcffXV+fWvf51kztdcDB8+PNXV1dlhhx3m+pw777xzxowZk4cffjidOnXKjTfemPbt2+e4447L8ccfn06dOuUPf/hDFltssTRt2jS/+93vst122+WCCy7IIYcckrfeeitXXXVVXZA4/vjjs+WWW+ass87KwIEDs8gii+Soo4762ve2wQYb5MQTT8xzzz2X/v3758wzz8wyyyyT3/72t0lSN57Vp0+fnHnmmXniiScyYMCAXHPNNenfv3+OOeaYes+3+eabJ0l22223eTizADREVbVfdTUmAIW5995707Fjx7prMZLktddey0477ZRLLrkkW2+9dQWra1iuuOKKXHPNNXnggQfSokWLSpcDwDdgNAuggXjkkUcyYsSIHH300enatWvGjx+fSy+9NCuuuGJ69+5d6fIahFtvvTWjRo3KjTfemIMPPlgIAViACSIADcSxxx6bVq1a5dJLL82ECROy+OKLZ9NNN81RRx2Vli1bVrq8BmHkyJEZOnRott122+y3336VLgeAb8FoFgAAUOfyyy/PI488kuuuu26u+3z00UcZNGhQHnrooVRVVWXHHXfMMcccM9s9s76KjggAAJAkueGGG3L++edn3XXX/cr9Bg4cmClTpuSaa67JxIkT86tf/SqfffZZzjrrrHl+LUEEAAAWcuPHj89JJ52UJ554IiussMJX7vvcc8/lySefzIgRI9KtW7ckM5dn79evX4488sh5Xlbd8r0AALCQe+mll9K8efPcfvvtWWuttb5y36effjpLL710XQhJkvXXXz9VVVV1y7HPCx0RAABoBL5umfd77713ro9ttdVW2WqrrebpdcaPH59OnTrV29aiRYssvvjiGTt27Dw9R7KQBpHv9Tqx0iU0alWTP690CY3fpMmVrqBRm9Zj2UqX0Og1/8C/4fmt+qVRlS6hUWu2/HKVLqHRu3P0uZUuYa5qxq1c6RLmoph/l1OmTJnj8uktW7bM1KlT5/l5FsogAgAAjc1XdTzKqVWrVpk2bdps26dOnZpFFllknp/HNSIAAMA869ixYyZMmFBv27Rp0/Lxxx+nffv28/w8gggAAJSgpoH+v6Kst956GTduXN566626bU8++WSS5Lvf/e48P48gAgAAzFV1dXX+85//5PPPZ14HvNZaa2WdddbJEUcckRdeeCGPP/54TjzxxOyyyy7zvHRvIogAAABfYezYsendu3dGjBiRJKmqqspvf/vbdO7cOX379s3hhx+ezTbbLCeffHJJz1tVW1tbOx/qbdCsmjV/WTWrAFbNmq+smjX/WTVr/rNq1vxl1az5ryGvmjV17IqVLmGOWnZ6o9IllERHBAAAKJwgAgAAFM59RAAAoAQ1WeiubJgvdEQAAIDCCSIAAEDhjGYBAEAJirx5YGOmIwIAABROEAEAAApnNAsAAEpQvfDdD3y+0BEBAAAKJ4gAAACFM5oFAAAlcEPD8tARAQAACieIAAAAhTOaBQAAJag2mlUWOiIAAEDhBBEAAKBwRrMAAKAEVs0qDx0RAACgcIIIAABQOKNZAABQgupao1nloCMCAAAUThABAAAKZzQLAABKUFPpAhoJHREAAKBwgggAAFA4o1kAAFCCajc0LAsdEQAAoHCCCAAAUDijWQAAUIJqk1lloSMCAAAUThABAAAKZzQLAABK4IaG5aEjAgAAFE4QAQAACmc0CwAASlCdqkqX0CjoiAAAAIUTRAAAgMIZzQIAgBLUuKFhWeiIAAAAhRNEAACAwhnNKtg6G3XLPj/fJl1WXDoffzg5f7npyQy79tG57t+8edPs9rNNss1Oa2Xpjovl/fETc9+IF/Knqx7OjBnVs+3fepEWufTPh+SGy+/P32//53x8Jw3TOr1XSt8jvpcuK3XIx+9Pyl9v+EeGXfnQPB3bbdVlcv7Nh2b/bc/OhHc/SpK0X3aJ/OGB4+d6zN3Dnsp5x/25LLUvKNbZvGf6HveDdOnRKR//Z2L+evVDGXbZvfN0bLfVO+f8Ecdk/41PzoQxH851vwNO2T27HrBV/q/TIeUqe4Hx3XW7Zr9+m2eFFZbORx9Nzm3Dn8mfb3pirvs3b940P/zRBtl2+zWy9NJt8/5/Ps2997yUP974j8yY8d9bbi3XZakcMGDLrNVr+VRX1+SF59/OZZfcm7FjPy7gXTV862zUPfsM3DZdurXPxx9Myl+GPpFhf3hkrvs3b940u/XtnW1+sHaW7rBY3h//Se4b8Xz+9PuH5vjZ3Jh9d9s1s++gPbP8asvlo/Gf5PZL/pabh/zlK4/Z8ieb5Ke/2j2dVuyQ8W9OyNCzhufv1z5Yb5+Nd14vfU7YI8v1WCYfjvs491z/YIaeMTwzps+o2+fYaw/NNnttNtvzn/rDIXl42OPleYMN0Dqbrpy+R+2QLit3zMfvf5q/Xvdohv3ugXk6ttuqy+b84Ydn/y1/U/e3bpbOK7bP/sfvlDU36J7q6pq8+MTr+d3pt2XcO3P/vF5YWTWrPASRAvVco3NOubBPHrrrX/nDxfdm9bWXz/6Hb5umTZvkT1c/PMdjDjxmh2y901q58YoHM+qld7PSqsukz4At0qHTYjnvlNvq7dumbaucdP5P03HZJYp4Ow1Oz15dcvLl++ahEc/n2vPvymrrds1+x+yQJk2b5M9XPPCVxy6/UoeccsV+ada8ab3tH/1nYo744W9n23+nPhtlsx3Wyt1/fqqcb6HB67nOCjn52oPy0O3P5tqz/5rV1l8x+/16lzRp1iR//u3fv/LY5Xt0yinXHTTbOf6y1Tfsnp37bVHGqhccq6y6TE4/40d54P6Xc81VD2X1NZbLAQO2StOmTTL0xsfmeMwhh26bbbZdPddf92heHTk2PXp0zN59N02HDu0y+JwRSZKll26bCy/aO++882FOP214WrZsnv323zxnDf5J+u37+0ybNmOOz72w6Lnmcjnl4r3z0N9ezB9+e09WX2f57H/k9mnarEn+NJcvMg48bqdsvVOv3HjF/Rn1rzFZabVl0+fArdKh0+I576RbC34HlbPKBivltL8cnwdv+keuOfGmrN67Z/qftVeaNmuam84aPsdjeu+2QY67fmBuvXBEnv7bP7PxLuvnmGt+nulTp+eBm/6RJFlnmzVz0rCj8+BN/8iVx9+Q5VfrnP1/0yeLfaddLh54Vd1zdeu1Qu678ZEMv2hEvdcYM2rsfHvPldaz1/I5+ff98tAd/8y15/4tq63XNfsdt9PMv3WX3feVxy6/csecclW/OX4Of6fT4hly86EZ88aEnHnYdWnZukX6HvV/Of3aA3PQ987JtKnT59dbYiEmiBRo74O2yusjx+WcE25Jkjzzj3+nabMm+fH+m2b4jY9l2tT6/zHQdrHW+b/dv5urLvh7bv7DzK7JP598I0my/+Hb5aoL/55PPvosSbLh5j1y0LE7pPUiLQt8Rw3LXgO3zeuvvJfBv7gpSfLMw6PSrFmT/OSgrXLbHx6Z7fwmSbPmTfODvTfJ3odtN8cP2enTqjPyn2/X29Z9tWWz2Q5r5Q/n/i0vPfPmfHkvDdVev9gxr/9rTAYf+ockyTP3v5xmzZrmJwO3z22/fyDTPp/9HDZr3jQ/2G+L7H3MjnN8/H+1WqRljjxvr3ww9pMsvRAG6r77bJZ/vzYuZ/5m5rfJTz35Rpo1bZKf9tk4t9z81GyBoV271tlxp7Xzu8vvy5++6Jo89+ybSZL+A7bK7654IJ988ln67rNpJk2eml8cdWOmfvG/g3HjPs5pp/8wPXp0yosvvlPcm2yA9j5467z+ytic88ubkyTPPPpamjZrmh/32zzDr//HnD+b91g3V513V26+ZmbX5J9PfPHZfMT3ctX5d9V9Njd2Pzv5R/n3c6NzVt+LkiRP3/XPNGveNHsev2tuvWBEpn0+bbZj9jv9p3noz4/nsiNnfo48fffzabtEm+xz6k/qgsj2+2yZCW+/nzP3vig1NTV59p4XskSHxbP7ETvlsiP/kOoZ1WnesnmW67FMbjn/jrzyxGvFvekK2+uI7fP6y+9m8JE3JkmeeWjkzM/hg7fJbVc/PMe/Zc2aN80P+vbO3kd8b45/C5Nkr8O2z+RPP88v97osU7/4rB7/zgc56Xf7Z6U1O+elp0bPvzfFQss1IgVp3rxp1lh3hfzjvlfqbX/knpeyaJtWWW3t5Wc7ZpFFW+aOm5/O4w+MrLf9nTffT5J0XHbJJMmibVvl1+fumReeeTO/Ovja+fQOGrbmLZpmzQ265R93/6ve9kf+9mIWadMqq63bdY7Hrbd5z/Q5dJvcdNl9ueqcO+fptQ45eZe8/fqE3DqXLlZj1bxFs6y50Ur5x53/rLf9kb8+l0Xats5q63eb43Hrbb1a+hz1f7npwrty1em3zXGfWfqduGs+nDAxf79pzt/+N2bNmzfNWr265JFHRtXb/tCDI7Pooi2zxhqdZztmkUVa5C+3P5t//KP+f4S9/fYHSZJOyyyeJNl08575250v1IWQJBn16rj8eI+LFvoQ0rx506yxXtf8476X621/5O5/zfxsXmcOn81tWuWOPz01+2fz6C8+mzsvOf8KbkCat2iWNbdYLY8Of7Le9odufjyLtlskq/fuOdsxHZZfOsv1WCaPDq8/bvjwsMey7Eqdsmz3jkmSFq2a5/PJU1NT89/xwokffJoWLZtnkbatkyRdV18uzZo3y+v/fLPM76zhmvm3rnv+cdeL9bY/cufzWaRtq6y23lz+1m2xSvoM3D43XXJvrjrrr3PcZ5P/WzN3//mJuhCSJK+9OCZ7bXiKEDIH1alqkD8LGkGkIB07L5EWLZrl3bfer7f9vbdnzl12Xn6p2Y4Z/97Hufg3f82Ytz6ot33jLXtm+vQZdc81dcr0DNjtogz59a2Z+PHC8S3cl3Vcbqk0b9Es7775pfP7xbnr3HXpOR436sV30neLMzL00vtSXf31c92b77hWevZaPpcPuj01C9nafR2XXyrNWzbPu29MqLf9vTf/kyTp3K3DHI8b9c+30nf9EzP0grtS/T/XLHzZ2pv1zNY/XD/nHXH9Qnduk6RTp8XTokWzjPnSLPa7X8xwd+4y+2fEuHGf5MLz75rtmE16r5zp06sz5p0P07HjYmnTplXGj/skAw/bPrfefkTuvPuYnDpoj3xn6bbz7w0tIDp2XnLmZ/OXPzve+eKzY4XZPzvGv/tRLj799oz50jEbb7VKvc/mxq7Tih3SomXzvDvqvXrb3/v3uCTJcj2Wme2YLqssm2T20al3vzim8xfH3H7J37LsSp2yx1Hfz6KLLZJVNlgpux22Y56449l8+tGkJDPHspLk//ptnaHvXpERn/8x5z54anqu3718b7KB6bjcUmneslneHf2fetvf++LfYucV28/xuFEvvJO+mw7K0IvvmePncIfOS6ZNu9aZ8O5HOfjU3fKn5wbltpFn5cQr9st3Oi5W/jcCX2gwo1kzZszI3Xffnaeeeipjx47NtGnT0rp163To0CHrrbdetttuuzRt+tWz5Q3Zom1aJUk+mzy13vbPPpvZtl7ki8e/zsZbrpJtvt8rfxn6ZCZ9+nmSZMaM6tnCysJm0bZfnN9Jn9fbPut8L9JmziNrH4yfWNLr7N5v87z09Oi8+MWI3MJk0S++hfzs0y+d40lfnOO2c/43/MG4T772uRdp2yqHn9sn1519x2xBZ2Gx6Bf/Rj/77EufEVNm/r7oPI5dbtJ75Wy3/ZoZfuvTmTTp8yzbeeaIW/8BW+bVke9l0KnDs8QSi2T//ltmyHl9MqDflfn8a0bmGrO6z44vfzZP/uKzedF5O+8bb7VqtvnB2vnLHx/PpImff/0BjcCiiy2SJJk8cUq97Z99OvP3Rdq1nusxn33pmClffK4s0m7m48/d96/86ZzbMuCcn2XAOT9Lkrz27Bv5TZ8L6o7p1mvmt/+tFmmZ3/z0/LRbqm1+cuwuOee+kzNwo19m9Iv1x2obg7rP4ZL/1n315/BiS7VJkux37E559fm3c+bAa7PYUm2z7y92yJk3HpxDdhySqVNmH7ODb6tBBJExY8Zk//33z/jx47Pqqqumffv2WWyxxTJ16tSMHDkyt9xySy666KL8/ve/zzLLzP4Ny4KgqslXt8tq5+Eb4E22WiXHnrFHXnru7fz+/LvLVVqjUFX11ee3HN+wr7L28llp9c455cBrvvVzLYi+7t/w/45QlGrAqXvkP+9+lFuv+OoLLRuzr/03XPv1/4Z7b9ojv/r1zvnXi+/kistnnsvmX1yU+tFHk3PSr4dl1tO8++5H+e0l+2TrbVbLHX/957eqfUH2dee9dh7O+yZbr5pjz/pRXnrurfz+3LvKVVqD9/WfCbOfu6omXz2IUfvF58hhl/bP9vtulesH3Zzn7n0xHVdon71P+mHOuPNXOWabUzJ1yrQMv+jOPP6Xp/P03c/XHf/cvS/mmlEX5ae/3D2n73neN3hXDds3OefzYtbnxMfvf5pBB15T9+9+7Jvv57xbD8tWu6yTO//YeFch+yZqahe8MaiGqEEEkVNPPTWdO3fOzTffnLZtZx8VmDhxYo444oiceuqpueyyyypQ4bc369uLL19MPuvbtsmTvvobtF332ij9jtg+Lzz9Zk494sZMX8hXufmyybO+TfvSt5ezvh368rdH30Tv762RTz/+LE89OPLrd26EJn/xLe+Xu3ezOiGffcNvgdffZvVsvvN3M/B7Z6eqSVWqUpUmX/yxbdK0SWpraufpPwYXdJO/+EazdesW9bbP6oRMnjR1tmP+1+57rJcBB22d5//5dk484eZMnzZz1HBW1/WpJ17P/57GV15+L5MmfZ7uK3Us11tYIH3tZ/OnX/PZvPfG6XfU/+WFp0bn1MOuX6g+myd/MnMUeNY1G7Ms+kVXY9bjczqm9Zc6qLO6J5M/+SxLLbNkdui/Tf54xq35w4kzFx954cGXM/LJf+fKl87L9/bbKrdd/LeMGfVexnxpLGzyJ5/lpUdHZsW1Zr+2pzGYPKvbtOiXzt+sqYuv+fc6N7M6Kk89MLLe5+3If76VSROnpNuqy36j54Wv0yCCyFNPPZWhQ4fOMYQkSbt27fKLX/wiffr0Kbiy8nnvnY9SPaM6y3SpfxHjrN/f+dK85/866JgdsvNPN8z9d76QIb++daFbo35ejH37g1TPqE6n5b9Tb/syX/z+9r+//bjP+luuksfueekrr3NozMa+9Z+Z5/hLM/PLfPH726+N+0bP23untdOydYtc/uAJsz12x5iL8vebHs+5h1/3jZ57QfLeex+luromy35ptbBlvvj97bfnft3BIYdum912Xy/33vNSzj7zL/XuH/Leex+lpqY2zZvP/nHftGmThX5Jzvfe+XAun80zr8l5542v+Gw+bsfs3Gfj3D/i+Qz51bCF7rP5vdfHzzx33euH2Vm/v/3KmNmOGfPqzOCwbPeO9S4y/+8x76Z9l++kSZMmeenR+l/6vP3KmHzy/sQsv9pySZLNf7RxJn00Kc/8/YV6+7Vs3SKf/Ke0sdsFxdi3vvhbt8KX/tatMOtv3fhv+Lzvp6amJs1bzvlzYupCPL7J/NUgLlZv27Ztxo//6v/xvPfee2nVat6uo2iIpk+bkReffSubbLVqve29t141kz6dklf/9e4cj9v30G2y8083zLBrH81Zx9+80P2hm1fTp83Ii0+NzibbrV5ve+/tV8+kiVMy6oVvtzJQm8Vap3PXpRe65Xr/1/SpM/Li4//OJjusVW977x17ZdInn2XUN1y55vrBd2Tg986q93Pn9TOXRB34vbNy/eA7vm3pC4Tp06rzwvNvZ9PNetTbvtlmPTNp0ucZ+cp7czxu//5bZLfd18ufb3oivxl0W70QkiSfT5meF154O70361E3fpEka6+zQlq3bpEXvuX/NhZ006fNyIvPvJlNtlmt3vbe266WSROn5NV/zf4f00my72HbZec+G2fYHx7JWcf+aaH8bJ4+dXpeeOiV9N51g3rbN919g0z6eHJeffLfsx3z3uvjMvaN8dl0943qH7Pbhhkz6r2Mf+s/ee/f41I9ozprbLpKvX06r7xMFvtOu4x9Y+Z/L+x4wLY57NID0ux/QvZSyyyZ1TbpmecfeKlcb7NBmT5tRl588o1ssv0a9bb3/t6aM//WPf/Nrov5/LNp+dcXz9u8xX8/J3ptvFJaL9rSqllzUOnVsRrLqlkNoiOyxx575Ljjjsthhx2WDTfcMJ06dUqLFi0ybdq0jB8/Pk8++WQGDx6cPfbYo9Klfit//N2DOePyvvnVOT/KXcOfy6prLZc9+m6Sqy68J1M/n55FFm2ZLisunbFjPswnH32WFXt0zA/37Z1X/zUmD//9pfT80vKdb7/xn9kusFyYDb3k3vzmD/3zywv3yt03P5VV1lk+u/fbPFcPvnPm+W3TMl26d8jYtz/IJx9OLum5u/bolOSbf9vUWAw9/2/5zZ8OzS+v2D93D30sq6y7YnY/eJtcffptmTplehZp0ypdVu6YsW+9n08+mDRPzzlhzIez3WV9/W1mBsrXvuEf1QXV9dc9mnOG/DQnnrxr/jbi+ay6euf86Ccb5vdX3J+pU2dkkUVaZPkVvpP33v04n3zyWbp1b5+f7LlRRr7yXh588JWssmr9a+jeevP9fPbZtFx5xQMZcn6f/ObMH+fPNz2eJZZcNP0P2Covv/xuHvvHwnP/hbn54xUP5Izf7ZtfDflJ7rr12azaq0v22Kd3rjr/7v9+Nndrn7HvfPDFZ3On/HC/TfPqi2Py8N3/Ss81l6v3fG+/PmGh+Wy+8fRhOevvv86vbzoyf7v6vqy6cY/88Ogf5Mrjb8jUKdOySNvWWX7Vznnv9fH55P2ZXYrrT7s5v7j6kEz88NM8dvvT2Xjn9bLFjzfOoJ/MvKbjk/cn5pYL7sgPj/5BkuSZv7+QDssvnb1O/GHGvTkhI353T5LkhkE356y/n5hThh+TWy8ckbZLtsneJ/4wEz/4NH/+mju7L8iG/vbv+c31B+aXF/8sd//pyazy3RWy+wFb5Oqz7/ifv3UdM/bt90v6W3f12Xfk7D8eklOv6p9hv3sgi3+nbfY7bqeMfO6tPH7Pv77+CeAbqKptAMPXtbW1ufjii3P11Vfns89mnylddNFF06dPnxx22GFp8jUXus2L7/U68Vs/xze18ZarZO+DtsyyK3wnH0yYmL/c9GRuuW7mDZzWXHeFnP37/TLkxFvy99v/mb0P2ip9Bmwx1+c6pt9VeeHpN+tt67DM4vnDiCPrnqMSqiZXbsWYjbddLXsN3C6dV1w674//JH+9/rHcctXMOyOvsf6KOfuGAzPk2Jtyzy3PzHbsNrt9N0ed9eP03eKMTPhiydRZNv2/NfPLC/dK/+3PyZivGNUozKTSglQ5bfx/a2Wvo3dM527t8/64T/LXqx/KLZffmyRZY6OVcvYth2fIYdflnj/NfmHjNj/aMEddsHf6rvfr2cLH/+pz1A7Z6+gd83+dDplv7+OrTOtRuXnoTXqvnH323Sydl1sy77//aW4f/kz+/KeZ92lYq1eXnHv+Xjn7zL/krr+9mH323Sx79+091+c68vDr8/wXN+RcdbVls3+/LdJzlWUyder0PPrIqFx26b1fe+3J/NL8g8r9G56TjbdaNXsfsvV/P5v/+HhuuXbmjWTXXLdrzr66X4accHP+fttz2fuQrdPnwK3m+lzH7Pv7vPB05b9Brn5p1NfvVAab7LJ+fnbyj9K5xzL54N0Pc/slf8vN5868V8Wam6+aIfefknP2vTh3/+GBumN2PGCb/PCoH2Tp5ZbK2DcmZOiZt+ae6+vfxX7Xw3bITgO2S8eu7fPh2I/yzN9fyNW/+mNdoEmStbdaPXuf9KOsuObyqampydN/+2d+d9wN+c87838J5WbLL/f1O80nG2+3RvY6Yvt07tp+5t+66x7JLb9/MEmyxgbdcvbQQzLk6D/mnmFPzXbsNruvl6MG75m+vU+b7W/dKuuskL5H75Aevbpk6pRpeezv/8rvT7/9a6+Vml/uHH1uRV53Xjz7dpdKlzBH63RZsL7AaxBBZJbp06fnlVdeyfjx4zNlypS0atUqHTt2TM+ePdOiRYuvf4J5VMkgsjCoZBBZaFQwiCwMKhlEFhYNLYg0RkUFkYVVJYPIwqIhB5Gn3l6h0iXM0Xpd3qx0CSVpEKNZszRv3jxrrrlmpcsAAADmswZxsToAALBwaVAdEQAAaOjc0LA8dEQAAIDCCSIAAEDhjGYBAEAJFsSbBzZEOiIAAEDhBBEAAKBwRrMAAKAE1bW+yy8HZxEAACicIAIAABTOaBYAAJSgxnf5ZeEsAgAAhRNEAACAwhnNAgCAErihYXnoiAAAAIUTRAAAgMIZzQIAgBK4oWF5OIsAAEDhBBEAAKBwRrMAAKAENVbNKgsdEQAAoHCCCAAAUDijWQAAUIJq3+WXhbMIAAAUThABAAAKZzQLAABK4IaG5eEsAgAAhRNEAACAwhnNAgCAEtT4Lr8snEUAAKBwgggAAFA4o1kAAFCC6tqqSpfQKOiIAAAAhRNEAACAwgkiAABA4VwjAgAAJaj2XX5ZOIsAAEDhBBEAAKBwRrMAAKAENbW+yy8HZxEAACicIAIAABTOaBYAAJTAqlnl4SwCAACFE0QAAIDCGc0CAIASVNdWVbqERkFHBAAAKJwgAgAAFM5oFgAAlKDGd/llsVAGkeOHD610CY3a57UL5T+rQn1e26LSJTRqn9c0r3QJjd5yzT+odAmN3qc1rSpdQqNWnVcqXQIs8MQ5AACgcL66BgCAElTX+i6/HJxFAACgcIIIAABQOKNZAABQgpq4oWE56IgAAACFE0QAAIDCGc0CAIASWDWrPJxFAACgcIIIAABQOKNZAABQgmrf5ZeFswgAABROEAEAAApnNAsAAEpQU+uGhuWgIwIAABROEAEAAApnNAsAAEpg1azycBYBAIDCCSIAAEDhjGYBAEAJamp9l18OziIAAFA4QQQAACic0SwAAChBddzQsBx0RAAAgMIJIgAAQOGMZgEAQAmsmlUeziIAAFA4QQQAACic0SwAACiBVbPKQ0cEAAAonCACAAAUzmgWAACUwKpZ5eEsAgAAhRNEAACAwhnNAgCAElQbzSoLZxEAACicIAIAABTOaBYAAJSgxg0Ny0JHBAAAKJwgAgAAFM5oFgAAlMCqWeXhLAIAAIUTRAAAgMIZzQIAgBLU1Fo1qxx0RAAAgMIJIgAAQOGMZgEAQAmqfZdfFs4iAABQOEEEAAAonNEsAAAogVWzykNHBAAAKJwgAgAAFE4QAQCAEtSkSYP8+VbvqaYmF154YTbddNP06tUr/fv3zzvvvDPX/T/44IMcddRR2XDDDbPBBhvkiCOOyPjx40t6TUEEAAAWcpdcckluvPHGnHbaaRk6dGhqamrSr1+/TJs2bY77H3744Xnvvfdy9dVX5+qrr857772XQw45pKTXFEQAAGAhNm3atFx11VUZOHBgtthii/Ts2TPnnXdexo0bl7vvvnu2/SdOnJgnn3wy/fv3zyqrrJJVV101BxxwQF588cV8/PHH8/y6DWbVrL333jtVVfO2AsG11147n6sBAIA5q25kq2aNHDkykydPzkYbbVS3rV27dll11VXz1FNPZaeddqq3f6tWrbLoootm+PDhWX/99ZMkt912W7p27Zp27drN8+s2mCDSu3fvXHDBBenatWvWXHPNSpcDAAALhXHjxiVJOnXqVG97+/bt6x77Xy1atMiZZ56ZE088Meuuu26qqqrSvn37XH/99WnSZN4HrhpMEBkwYEDatGmTIUOG5PLLL0/nzp0rXRIAACwwtt566698/N57753j9ilTpiSZGTD+V8uWLfPJJ5/Mtn9tbW1eeeWVrL322unXr1+qq6tz3nnn5eCDD84f//jHtGnTZp7qbVDXiPTp0yfrr79+zj777EqXAgAAc1RTW9Ugf76pVq1aJclsF6ZPnTo1rVu3nm3/O++8M9dff33OOeecfPe7383666+fyy67LO+++25uvvnmeX7dBtMRmeXUU0/NSy+9VOkyAABggTK3jsfXmTWSNWHChHTp0qVu+4QJE9KjR4/Z9n/66afTtWvXep2PxRZbLF27ds1bb701z6/boDoiycxZtC233LLSZQAAwEKhZ8+eadOmTZ544om6bRMnTszLL7+c9dZbb7b9O3bsmLfeeitTp06t2/bZZ59lzJgxWWGFFeb5dRtcEAEAgIasprZJg/z5plq0aJG99torgwcPzr333puRI0fmiCOOSMeOHbPddtuluro6//nPf/L5558nSXbZZZckM+8lMnLkyIwcOTJHHnlkWrZsmd12222eX1cQAQCAhdzAgQOzxx575IQTTsiee+6Zpk2b5sorr0zz5s0zduzY9O7dOyNGjEgyc4LpxhtvTG1tbfr27Zt99903zZs3z4033pi2bdvO82tW1dbW1s6vN9RQPfjmypUuoVH7vLbBXXrU6Hxe2+Lrd+Ib+7ymeaVLaPSWa/5BpUto9D6taVXpEhq1at/lznc7dP1XpUuYq0Of7VPpEuboonVuqHQJJfFfjAAAUILqNK4bGlaKOA8AABROEAEAAApnNAsAAErwbW4eyH/piAAAAIUTRAAAgMIZzQIAgBJ8m5sH8l/OIgAAUDhBBAAAKJzRLAAAKEGNGxqWhY4IAABQOEEEAAAonNEsAAAoQbUbGpaFjggAAFA4QQQAACic0SwAACiBGxqWh7MIAAAUThABAAAKZzQLAABKUGPVrLLQEQEAAAoniAAAAIUzmgUAACWoidGsctARAQAACieIAAAAhTOaBQAAJbBqVnnoiAAAAIUTRAAAgMIZzQIAgBLU1PouvxycRQAAoHCCCAAAUDijWQAAUAKrZpWHjggAAFA4QQQAACic0SwAAChBTYxmlYOOCAAAUDhBBAAAKJzRLAAAKIFVs8pDRwQAACicIAIAABTOaBYAAJTAaFZ56IgAAACFE0QAAIDCGc0CAIASGM0qDx0RAACgcIIIAABQuIVyNGuTVjWVLqGRm1bpAhq9JplR6RLgW1oo//wUanrt55UuoVH7qMb5XZgZzSoPHREAAKBwgggAAFA4vXEAAChBTYxmlYOOCAAAUDhBBAAAKJzRLAAAKIFVs8pDRwQAACicIAIAABTOaBYAAJTAaFZ56IgAAACFE0QAAIDCGc0CAIASGM0qDx0RAACgcIIIAABQOKNZAABQAqNZ5aEjAgAAFE4QAQAACmc0CwAASlBrNKssdEQAAIDCCSIAAEDhjGYBAEAJamI0qxx0RAAAgMIJIgAAQOGMZgEAQAnc0LA8dEQAAIDCCSIAAEDhjGYBAEAJ3NCwPHREAACAwgkiAABA4YxmAQBACayaVR46IgAAQOEEEQAAoHBGswAAoARWzSoPHREAAKBwgggAAFA4o1kAAFACq2aVh44IAABQOEEEAAAonNEsAAAoQW1tpStoHHREAACAwgkiAABA4YxmAQBACWpi1axy0BEBAAAKJ4gAAACFM5oFAAAlqHVDw7LQEQEAAAoniAAAAIUzmgUAACWoMZpVFjoiAABA4QQRAACgcEazAACgBLW1la6gcdARAQAACtdggsibb76Ziy66KIMGDcpDDz002+OTJk3K8ccfX4HKAACAcmsQQeSZZ57JLrvskr/85S95+OGHM2DAgBx22GGZNm1a3T6ff/55hg8fXrkiAQAgM29o2BB/FjQNIogMGTIku+++e+6+++7cddddOf/88/PII4/k4IMPzowZMypdHgAAUGYNIoi8+uqr+dnPflb3+/bbb5/f/e53eeaZZ3LsscdWsDIAAGB+aBBBpE2bNvnggw/qbVtnnXVyzjnn5M4778wZZ5xRocoAAKC+So9gGc0qo8033zynnHJK/vnPf2b69Ol127fZZpv88pe/zB/+8IeceuqpFawQAAAopwYRRI466qgstdRS2XPPPfPYY4/Ve2yvvfbKiSeemPvuu69C1QEAAOXWIG5ouNhii+Wqq67K22+/nSWWWGK2x3/6059mo402yt13312B6gAA4L9qFsAxqIaoQQSRWbp06TLXx7p27ZoBAwYUWA0AADC/NIjRLAAAYOHSoDoiAADQ0NXWVrqCxkFHBAAAKJwgAgAAFM5oFgAAlGBBvHlgQ6QjAgAAFE4QAQAACmc0CwAASmA0qzx0RAAAgMIJIgAAQOGMZgEAQAncz7A8dEQAAIDCCSIAAEDhjGYBAEAJrJpVHjoiAABA4QQRAACgcEazAACgFJbNKgsdEQAAoHCCCAAAUDijWQAAUAKrZpWHjggAAFA4QQQAACic0SwAAChBrVWzykJHBAAAKJwgAgAAFM5oFgAAlMCqWeWhIwIAABROEAEAAApnNAsAAEphNKssdEQAAIDCCSIAAEDhjGYBAEAJ3NCwPHREAACAwgkiAABA4YxmAQBAKYxmlYWOCAAAUDhBBAAAKJzRLAAAKEGtGxqWhY4IAABQOEEEAAAonNEsAAAohVWzykJHBAAAKJwgAgAAFM5oFgAAlMCqWeWhIwIAABROEAEAAApnNAsAAEph1ayy0BEBAAAKt1B2RJrIXwA0ck2r/K2bn6b7Shy+NZ9SAABQkqoG+vPN1dTU5MILL8ymm26aXr16pX///nnnnXfmuv/06dMzZMiQuv332muvvPLKKyW9piACAAALuUsuuSQ33nhjTjvttAwdOjQ1NTXp169fpk2bNsf9Tz755Nxyyy35zW9+k2HDhmXJJZdM//798+mnn87zawoiAACwEJs2bVquuuqqDBw4MFtssUV69uyZ8847L+PGjcvdd9892/7vvPNOhg0bltNPPz2bbrppunXrlkGDBqVFixb517/+Nc+vK4gAAEApahvozzc0cuTITJ48ORtttFHdtnbt2mXVVVfNU089Ndv+jz76aNq2bZvNNtus3v733Xdfvef4OoIIAAAsxMaNG5ck6dSpU73t7du3r3vsf40ePTrLLbdc7r777uy2227ZZJNN0r9//7z++uslve5CuWoWAAA0NltvvfVXPn7vvffOcfuUKVOSJC1atKi3vWXLlvnkk09m23/SpEl56623cskll+SYY45Ju3btcumll+anP/1pRowYkaWWWmqe6tURAQCAUlR6BKvMo1mtWrVKktkuTJ86dWpat2492/7NmjXLpEmTct5556V3795Zc801c9555yVJbr311nl+XR0RAABoBObW8fg6s0ayJkyYkC5dutRtnzBhQnr06DHb/h07dkyzZs3SrVu3um2tWrXKcsstlzFjxszz6+qIAADAQqxnz55p06ZNnnjiibptEydOzMsvv5z11ltvtv3XW2+9zJgxIy+++GLdts8//zzvvPNOll9++Xl+XR0RAAAoRe23u3lgQ9OiRYvstddeGTx4cJZccsksu+yyOeecc9KxY8dst912qa6uzocffpi2bdumVatWWXfddbPxxhvn2GOPzamnnprFF188F154YZo2bZqdd955nl9XRwQAABZyAwcOzB577JETTjghe+65Z5o2bZorr7wyzZs3z9ixY9O7d++MGDGibv+LLroo66+/fn7+859njz32yKRJk3LttddmySWXnOfXrKqtrf0Wl7YsmGrGrVzpEgBgvqpJTaVLaNTGVk+udAmN3nLLjq10CXO1wtVnV7qEOXpz32MqXUJJjGYBAEAJFr6v8ecPo1kAAEDhBBEAAKBwRrMAAKAURrPKQkcEAAAonCACAAAUzmgWAACUopHd0LBSdEQAAIDCCSIAAEDhjGYBAEAJqqyaVRY6IgAAQOEEEQAAoHBGswAAoBRGs8pCRwQAACicIAIAABTOaBYAAJTCDQ3LQkcEAAAonCACAAAUzmgWAACUwqpZZaEjAgAAFE4QAQAACmc0CwAASmE0qyx0RAAAgMIJIgAAQOGMZgEAQCmMZpWFjggAAFA4QQQAACic0SwAAChFbVWlK2gUdEQAAIDCCSIAAEDhjGYBAEAJqqyaVRY6IgAAQOEEEQAAoHBGswAAoBRGs8pCRwQAACicIAIAABROEAEAAAoniAAAAIUr6WL1Tz/9NNdee21efPHFtGrVKu3bt8/KK6+cHj16ZKWVVkqrVq3mV50AAEAjUlIQ+cUvfpEHH3ww3/nOd7L44ovnscceyyeffJKqqqpUVVVlueWWqwsmK6+8clZeeeUsv/zyqaqqml/1AwBAodzQsDxKCiKPPfZYdthhhwwZMqRu2/jx4zNq1Ki8+uqrGTVqVEaNGpUHHngg06dPT1VVVVq1apXnnnvua5976tSpee2119K9e/e0atUqr7zySq6//vqMHz8+K620Uvr27ZuOHTuW/g4BAIAGp6Qg0rZt26y//vr1tnXo0CEdOnTIpptuWreturo6o0ePrgsnX+eNN97IPvvskwkTJmSZZZbJoEGDcvDBB2fZZZdN9+7dc8899+SWW27JjTfemG7dupVSMgAA0ACVdLH69773vTz//PNfu1/Tpk3TvXv37LjjjjniiCO+dv+zzjorvXr1yvDhw7P++uvnoIMOyg477JC//vWvueCCC3LnnXemd+/eOeOMM0opFwAAyq+2qmH+LGBKCiK77757Ro0albvuuqusRTz55JM5/PDD07NnzxxzzDGZOnVq9tprr7prS5o1a5YBAwbkmWeeKevrAgAAlVHSaNauu+6aJk2a5PDDD88GG2yQLbfcMquttlp69OiRtm3bfuMiWrVqlSlTpiRJllxyyfzoRz9Ky5Yt6+0zceLEb/UaAABAw1FSEDnjjDPqLkh/9dVX8/jjj9d1LTp27JgePXqkR48e8zSO9b969+6d0047LYMGDUr37t1z6qmn1j1WU1OTxx57LKeddlq22Wabkp4XAADKzqpZZVFVW1v7jU/lhx9+mFdffbXuovRXX301b7zxxjytkvXl5zn44IPTuXPnDB48uN5jI0aMyJFHHpnNN988Q4YMSZs2bb5puXVqxq38rZ8DABqymtRUuoRGbWz15EqX0Ogtt+zYSpcwVyuef26lS5ijNw4/stIllORbBZE5qa2t/cb3DZk4cWLatWtXb9tHH32U999/PyuttFI5yksiiADQ+Aki85cgMv8JIqVb0IJISaNZ8+Lb3LzwyyEkSZZYYoksscQS36YkAAAoH6NZZVHSqlkAAADlIIgAAACFK/toFgAANGZVRrPKQkcEAAAonCACAAAUzmgWAACUwmhWWeiIAAAAhRNEAACAwhnNAgCAUhjNKgsdEQAAoHCCCAAAUDijWQAAUAI3NCwPHREAAKBwgggAAFA4o1kAAFCK2qpKV9Ao6IgAAACFE0QAAIDCGc0CAIBSWDWrLHREAACAwgkiAABA4YxmAQBACdzQsDx0RAAAgMIJIgAAQOGMZgEAQCmMZpWFjggAAFA4QQQAACic0SwAACiBVbPKQ0cEAAAonCACAAAUzmgWAACUwmhWWeiIAAAAhRNEAACAwhnNAgCAUhjNKgsdEQAAoHCCCAAAUDijWQAAUAI3NCwPHREAAKBwgggAAFA4QQQAACicIAIAABROEAEAAApn1SwAACiFVbPKQkcEAAAonCACAAAUzmgWAACUwA0Ny0NHBAAAKJwgAgAAFM5oFgAAlMJoVlkslEHk8akzKl1CozZuxmKVLqHRW7TJ1EqX0Ki1bfJ5pUuAb6261tDD/PTi1J6VLqHRO7jSBTDf+ZQCAAAKt1B2RAAA4BszmlUWOiIAAEDhBBEAAKBwRrMAAKAEbmhYHjoiAABA4QQRAACgcEazAACgFEazykJHBAAAKJwgAgAAFM5oFgAAlMCqWeWhIwIAABROEAEAAApnNAsAAEphNKssdEQAAIDCCSIAAEDhjGYBAEApjGaVhY4IAABQOEEEAAAonNEsAAAogRsaloeOCAAAUDhBBAAAKJzRLAAAKIXRrLLQEQEAAAoniAAAAIUzmgUAAKUwmlUWOiIAAEDhBBEAAKBwRrMAAKAEbmhYHjoiAABA4QQRAACgcEazAACgFEazykJHBAAAKJwgAgAAFM5oFgAAlMCqWeWhIwIAABROEAEAAApnNAsAAEphNKssdEQAAIDCCSIAAEDhBBEAAKBwrhEBAIBSuEakLHREAACAwgkiAABA4YxmAQBACaoqXUAjoSMCAAAUThABAAAKZzQLAABKYdWsstARAQAACieIAAAAhTOaBQAAJagymlUWOiIAAEDhGnwQOeCAAzJhwoRKlwEAAJRRgxjNGj58+Fwfe+KJJ/LXv/41Sy65ZJJkl112KaYoAACYE6NZZdEggsgpp5ySzz//PElSWzv7/8+effbZSZKqqipBBAAAGoEGEURuueWWHH300WnXrl3OPPPMdOjQoe6xtddeO7fffnuWW265ClYIAACUU4O4RqRr16656aabssYaa2TnnXfOiBEjKl0SAADMWW0D/VnANIggkiTNmjXLkUcemYsuuiiDBw/O0UcfnU8//bTSZQEAAPNBgwkis6y33noZPnx4amtrs9NOO2X69OmVLgkAACizBhdEkqRdu3YZMmRIjjjiiKyzzjpp2bJlpUsCAIAkM29o2BB/vo2amppceOGF2XTTTdOrV6/0798/77zzzjwde/vtt6dHjx4ZM2ZMSa/ZIIPILLvsskuuvfbatG/fvtKlAABAo3XJJZfkxhtvzGmnnZahQ4empqYm/fr1y7Rp077yuHfffTennnrqN3rNBh1EAACA+WvatGm56qqrMnDgwGyxxRbp2bNnzjvvvIwbNy533333XI+rqanJL37xi6y22mrf6HUFEQAAKEWlV8cq86pZI0eOzOTJk7PRRhvVbWvXrl1WXXXVPPXUU3M97rLLLsv06dMzYMCAb/S6DeI+IgAAwLez9dZbf+Xj99577xy3jxs3LknSqVOnetvbt29f99iXvfDCC7nqqqty8803Z/z48d+gWh0RAABYqE2ZMiVJ0qJFi3rbW7ZsmalTp862/2effZajjz46Rx99dFZYYYVv/Lo6IgAAUIJvu0LV/DK3jsfXadWqVZKZ14rM+r+TZOrUqWnduvVs+w8aNChdu3bNT37yk29W6BcEEQAAWIjNGsmaMGFCunTpUrd9woQJ6dGjx2z7Dxs2LC1atMjaa6+dJKmurk6S7LTTTjnwwANz4IEHztPrCiIAALAQ69mzZ9q0aZMnnniiLohMnDgxL7/8cvbaa6/Z9v/ySlrPP/98fvGLX+SKK67IyiuvPM+vK4gAAEApGuho1jfVokWL7LXXXhk8eHCWXHLJLLvssjnnnHPSsWPHbLfddqmurs6HH36Ytm3bplWrVll++eXrHT/rgvZlllkmiy+++Dy/rovVAQBgITdw4MDsscceOeGEE7LnnnumadOmufLKK9O8efOMHTs2vXv3zogRI8r6mlW1tbWNLNN9vX+8tWKlS2jUxs1YrNIlNHqLNpl9BQvKp22TzytdAnxr1bW+a5yfXpy6XKVLaPQO7nF/pUuYq7UPPq/SJczRc5ccUekSSmI0CwAAStBQV81a0Pi6BAAAKJwgAgAAFM5oFgAAlMJoVlnoiAAAAIUTRAAAgMIZzQIAgFIYzSoLHREAAKBwgggAAFA4o1kAAFACNzQsDx0RAACgcIIIAABQOKNZAABQCqNZZaEjAgAAFE4QAQAACmc0CwAASlBVazarHHREAACAwgkiAABA4YxmAQBAKUxmlYWOCAAAUDhBBAAAKJzRLAAAKEGV0ayy0BEBAAAKJ4gAAACFM5oFAAClMJpVFjoiAABA4QQRAACgcEazAACgBFbNKg8dEQAAoHCCCAAAUDijWQAAUAqjWWWhIwIAABROEAEAAApnNAsAAEpg1azy0BEBAAAKJ4gAAACFM5oFAAClMJpVFjoiAABA4RbKjsgpPTaqdAlAA9akU4dKl9Do1YwdX+kSGr8q3zXOT1XNF8r/hCrUwRMrXQHzm/8VAQBACayaVR6+LgEAAAoniAAAAIUzmgUAAKWoNZtVDjoiAABA4QQRAACgcEazAACgBFbNKg8dEQAAoHCCCAAAUDijWQAAUAqjWWWhIwIAABROEAEAAApnNAsAAEpQVVPpChoHHREAAKBwgggAAFA4o1kAAFAKq2aVhY4IAABQOEEEAAAonNEsAAAoQZXRrLLQEQEAAAoniAAAAIUzmgUAAKWoNZtVDjoiAABA4QQRAACgcEazAACgBFbNKg8dEQAAoHCCCAAAUDijWQAAUAqjWWWhIwIAABROEAEAAApnNAsAAEpg1azy0BEBAAAKJ4gAAACFM5oFAAClqDWbVQ46IgAAQOEEEQAAoHBGswAAoARWzSoPHREAAKBwgggAAFA4o1kAAFAKo1lloSMCAAAUThABAAAKZzQLAABKYNWs8tARAQAACieIAAAAhTOaBQAApagxm1UOOiIAAEDhBBEAAKBwRrMAAKAUJrPKQkcEAAAonCACAAAUzmgWAACUwA0Ny6NBdESGDx+eadOm1dv2+OOP54ADDsgPfvCDHHXUUXn99dcrVB0AAFBuDSKIHH/88fn000/rfn/44Yez7777pra2Nr17986ECROy22675dlnn61glQAAQLk0iNGs2tr6/a1LL700++yzT4499ti6bWeccUYGDx6cG2+8sejyAADgv2rNZpVDg+iIfNlbb72V73//+/W2/fjHP87LL79coYoAAIByahBBpKqqqt7vXbt2zaRJk+pt+/DDD9O2bdsiywIAAOaTBjOatfXWW2eFFVZIt27d0qxZs5x55pkZOnRoWrRokaeeeiqnnnpqNttss0qXCgDAQs6qWeXRIILIgw8+mFdffTWjRo3Kq6++mo8++ihvvPFGqqurkyQHHnhgunXrlqOOOqrClQIAAOXQIIJIhw4d0qFDh3odj+rq6jRt2jRJctNNN6Vbt26zjXABAAALpgYRROZkVghJku7du1ewEgAA+B9Gs8qiQVysDgAALFwEEQAAoHANdjQLAAAaoio3NCwLHREAAKBwgggAAFA4o1kAAFCKmkoX0DjoiAAAAIUTRAAAgMIZzQIAgBJYNas8dEQAAIDCCSIAAEDhjGYBAEApTGaVhY4IAABQOEEEAAAonNEsAAAohVWzykJHBAAAKJwgAgAAFM5oFgAAlKDKZFZZ6IgAAACFE0QAAIDCGc0CAIBSWDWrLHREAACAwgkiAABA4YxmAQBACapqKl1B46AjAgAAFE4QAQAACmc0CwAASmHVrLLQEQEAAAoniAAAAIUzmgUAAKUwmVUWOiIAAEDhBBEAAKBwRrMAAKAEVVbNKgsdEQAAoHCCCAAAUDijWQAAUAqjWWWhIwIAABROEAEAAApnNAsAAEpRU+kCGgcdEQAAoHCCCAAAUDijWQAAUAI3NCwPHREAAKBwgggAAFA4o1kAAFAKo1lloSMCAAAUThABAAAKZzQLAABKYTSrLBbKINKka5dKl9CoVf97dKVLaPSqmjatdAmNm/M73zVZrF2lS2j0aj6ZWOkSGrXa6TMqXQIs8IxmAQAAhVsoOyIAAPCN1VS6gMZBRwQAACicIAIAABTOaBYAAJSgyqpZZaEjAgAAFE4QAQAACieIAABAKWprG+bPt1BTU5MLL7wwm266aXr16pX+/fvnnXfemev+r732Wg444IBssMEG2WijjTJw4MC89957Jb2mIAIAAAu5Sy65JDfeeGNOO+20DB06NDU1NenXr1+mTZs2274fffRR9t1337Rq1SrXXXddfve73+XDDz9Mv379MnXq1Hl+TUEEAAAWYtOmTctVV12VgQMHZosttkjPnj1z3nnnZdy4cbn77rtn2/+ee+7JZ599lrPPPjsrr7xyVl999Zxzzjl5/fXX8+yzz87z6woiAABQikqPYJV5NGvkyJGZPHlyNtpoo7pt7dq1y6qrrpqnnnpqtv032mijXHLJJWnVqlXdtiZNZsaKiRMnzvPrWr4XAAAaga233vorH7/33nvnuH3cuHFJkk6dOtXb3r59+7rH/lfnzp3TuXPnetuuuOKKtGrVKuutt94816sjAgAAC7EpU6YkSVq0aFFve8uWLefpmo/rrrsu119/fY4++ugsueSS8/y6OiIAAFCKBnpDw7l1PL7OrBGradOm1Ru3mjp1alq3bj3X42pra3PBBRfk0ksvzUEHHZS99967pNfVEQEAgIXYrJGsCRMm1Ns+YcKEdOjQYY7HTJ8+Pb/4xS9y2WWX5fjjj8/hhx9e8usKIgAAsBDr2bNn2rRpkyeeeKJu28SJE/Pyyy/P9ZqPY445Jn/7298yZMiQ7LPPPt/odY1mAQBAKWoqXUB5tWjRInvttVcGDx6cJZdcMssuu2zOOeecdOzYMdttt12qq6vz4Ycfpm3btmnVqlVuueWWjBgxIsccc0zWX3/9/Oc//6l7rln7zAsdEQAAWMgNHDgwe+yxR0444YTsueeeadq0aa688so0b948Y8eOTe/evTNixIgkyV//+tckydlnn53evXvX+5m1z7yoqq1toFfbzEf/t8rxlS6hUav+9+hKl9DoVTVtWukSGrUmnZepdAmN36eTKl1Bo1fzybyv5c83UOW73PntrinXVbqEufrear+qdAlz9LeXTq90CSUxmgUAACWoWvi+x58vxHkAAKBwgggAAFA4o1kAAFAKo1lloSMCAAAUThABAAAKZzQLAABKUWM0qxx0RAAAgMIJIgAAQOGMZgEAQCmsmlUWOiIAAEDhBBEAAKBwRrMAAKAURrPKQkcEAAAonCACAAAUzmgWAACUwmhWWeiIAAAAhRNEAACAwhnNAgCAUtQYzSoHHREAAKBwgggAAFA4o1kAAFCK2ppKV9Ao6IgAAACFE0QAAIDCGc0CAIBSuKFhWeiIAAAAhRNEAACAwhnNAgCAUrihYVnoiAAAAIUTRAAAgMIZzQIAgFJYNassdEQAAIDCCSIAAEDhGsxo1vPPP58nnngiBxxwQJLk8ccfzzXXXJMxY8akS5cu2W+//bLuuutWuEoAABZ6RrPKokF0RP72t79lzz33zJNPPpkkuf/++7PvvvumtrY2m2++eaZPn56+ffvm/vvvr3ClAABAOTSIjshvf/vbDBw4MAceeGCS5NJLL82BBx6Yww47rG6fSy+9NBdeeGG23HLLSpUJAACUSYPoiLz99tvZcccd634fM2ZMtt9++3r77LTTTnn99deLLg0AAOqrrW2YPwuYBhFElltuuTz66KN1v6+yyioZOXJkvX1eeOGFdOjQoejSAACA+aBBjGb1798/J5xwQsaMGZOddtopBx98cI477rhMnTo1K620Up5//vlcfPHF+fnPf17pUgEAgDJoEEFkl112SVVVVS688ML8/ve/T1VVVWpra3PSSSclSRZddNH069cv++yzT2ULBQCAmppKV9AoNIggkiQ777xzdt5554wePTqjR4/OpEmT0qxZs3Ts2DGrrbZaWrZsWekSAQCAMmkwQWSWrl27pmvXrpUuAwAAmI8aXBABAIAGbQFcoaohahCrZgEAAAsXQQQAACic0SwAACiF0ayy0BEBAAAKJ4gAAACFM5oFAAClqDGaVQ46IgAAQOEEEQAAoHBGswAAoAS1tTWVLqFR0BEBAAAKJ4gAAACFM5oFAAClsGpWWeiIAAAAhRNEAACAwhnNAgCAUtQazSoHHREAAKBwgggAAFA4o1kAAFCKGjc0LAcdEQAAoHCCCAAAUDijWQAAUAqrZpWFjggAAFA4QQQAACic0SwAAChBrVWzykJHBAAAKJwgAgAAFM5oFgAAlMKqWWWhIwIAABROEAEAAApnNAsAAEpRYzSrHHREAACAwgkiAABA4YxmAQBAKWrd0LAcdEQAAIDCCSIAAEDhjGYBAEAJaq2aVRY6IgAAQOEEEQAAoHBGswAAoBRWzSoLHREAAKBwgggAAFA4o1kAAFACq2aVh44IAABQOEEEAAAonNEsAAAohVWzykJHBAAAKJwgAgAAFK6qtrbWZf8AAEChdEQAAIDCCSIAAEDhBBEAAKBwgggAAFA4QQQAACicIAIAABROEAEAAAoniAAAAIUTRAAAgMIJIgAAQOEEEQAAoHCCCAAAUDhBBAAAKJwg0kDV1NTkwgsvzKabbppevXqlf//+eeeddypdVqN1+eWXZ++99650GY3Kxx9/nBNPPDGbbbZZ1llnney55555+umnK11Wo/LBBx/kF7/4RTbccMOsvfbaOeCAA/L6669XuqxGafTo0Vl77bVzyy23VLqURmX8+PHp0aPHbD/Oc3kNHz48O+ywQ9ZYY43suOOOufPOOytdEiQRRBqsSy65JDfeeGNOO+20DB06NDU1NenXr1+mTZtW6dIanRtuuCHnn39+pctodI488sg899xzOffcczNs2LCsssoq2X///fPGG29UurRG45BDDslbb72VK664IjfffHNatWqVffbZJ1OmTKl0aY3K9OnTc/TRR+ezzz6rdCmNzsiRI9OyZcs8/PDDeeSRR+p+dthhh0qX1mjcdttt+dWvfpU+ffrkjjvuyE477VT3+QyVJog0QNOmTctVV12VgQMHZosttkjPnj1z3nnnZdy4cbn77rsrXV6jMX78+Bx44IEZPHhwVlhhhUqX06i89dZbefTRR3PyySdn3XXXTdeuXfPrX/867du3z1/+8pdKl9cofPLJJ1l22WUzaNCgrLnmmunWrVsOPvjgTJgwIa+99lqly2tULrroorRp06bSZTRKo0aNygorrJD27dtn6aWXrvtp1apVpUtrFGpra3PBBRfkZz/7Wfr06ZMuXbrkoIMOysYbb5wnn3yy0uWBINIQjRw5MpMnT85GG21Ut61du3ZZddVV89RTT1WwssblpZdeSvPmzXP77bdnrbXWqnQ5jcoSSyyRK664ImussUbdtqqqqlRVVWXixIkVrKzxWGyxxTJkyJCsvPLKSZIPP/ww11xzTTp27Jju3btXuLrG46mnnspNN92UM888s9KlNEqvvvpqunXrVukyGq3Ro0fn3Xffzfe///1626+88soMGDCgQlXBfzWrdAHMbty4cUmSTp061dvevn37usf49rbaaqtstdVWlS6jUWrXrl0233zzetvuuuuuvPXWW/nlL39Zoaoar1//+tf505/+lBYtWuTSSy/NIossUumSGoWJEyfmmGOOyQknnDDb5zHlMWrUqCyxxBLp06dPRo8eneWXXz4HHXRQNttss0qX1iiMHj06SfLZZ59l//33z8svv5zOnTvnoIMO8vePBkFHpAGaNd/dokWLettbtmyZqVOnVqIk+FaeffbZHH/88dluu+2yxRZbVLqcRqdv374ZNmxYdtpppxxyyCF56aWXKl1So3DyySdn7bXXnu3bZMpjxowZeeONN/LJJ5/k0EMPzRVXXJFevXrlgAMOyGOPPVbp8hqFSZMmJUmOPfbY7LTTTrnqqquyySab5OCDD3aOaRB0RBqgWbOx06ZNqzcnO3Xq1LRu3bpSZcE3cs899+Too4/OOuusk8GDB1e6nEZp1ijW6aefnueffz7XX399zjjjjApXtWAbPnx4nn76adc0zUfNmjXLE088kaZNm9b9rVt99dXz2muv5corr6w3nsw307x58yTJ/vvvn1133TVJssoqq+Tll1/O1Vdf7RxTcToiDdCsEYAJEybU2z5hwoR06NChEiXBN3L99dfn0EMPzZZbbpnLLrssLVu2rHRJjcaHH36YO+64IzNmzKjb1qRJk3Tv3n22zw5KN2zYsHzwwQfZYostsvbaa2fttddOkpx00knp169fhatrPBZddNHZLkxfaaWVMn78+ApV1LjM+m+GWdeSzdK9e/eMGTOmEiVBPYJIA9SzZ8+0adMmTzzxRN22iRMn5uWXX856661Xwcpg3s1afrpPnz4599xzZxs15Nt5//33c+SRR9Ybr5g+fXpefvllF/+WweDBgzNixIgMHz687idJBg4cmNNPP72yxTUSr732WtZZZ516f+uS5F//+pcFF8pktdVWy6KLLprnn3++3vZRo0alS5cuFaoK/stoVgPUokWL7LXXXhk8eHCWXHLJLLvssjnnnHPSsWPHbLfddpUuD77W6NGj85vf/CbbbrttBgwYkPfff7/usVatWqVt27YVrK5xWHnllbPZZptl0KBBGTRoUBZbbLFcfvnlmThxYvbZZ59Kl7fAm1v3eamlltKZLpNu3bplxRVXzKmnnppTTjklSyyxRP70pz/ln//8Z4YNG1bp8hqFVq1apV+/frn44ovToUOHrLnmmrnjjjvy6KOP5pprrql0eSCINFQDBw7MjBkzcsIJJ+Tzzz/PeuutlyuvvLJu3hMasrvuuivTp0/P3//+9/z973+v99iuu+5qKdQyOffcczNkyJAcccQR+fTTT7PuuuvmhhtuyDLLLFPp0uBrNWnSJJdddlmGDBmSww8/PBMnTsyqq66aq6++erZRIr65gw8+OK1bt855552X8ePHp1u3brnooouywQYbVLo0SFVtbW1tpYsAAAAWLq4RAQAACieIAAAAhRNEAACAwgkiAABA4QQRAACgcIIIAABQOEEEAAAonCACAAAUThABWEBNnTo1q622WtZdd91cdNFFlS4HAEoiiAAsoGbMmJFBgwZlmWWWycUXX5wPPvig0iUBwDwTRAAWUIsuumh23XXX9OvXL7W1tXn11VcrXRIAzDNBBGABt9JKKyVJXn/99QpXAgDzThABWMBNnTo1SfLvf/+7wpUAwLwTRAAWcGeeeWaS5I033qhwJQAw7wQRgAXYbbfdlueeey5LLbWU0SwAFiiCCMACavLkyRk8eHA222yz7Lzzzvnggw/y8ccfV7osAJgnggjAAuryyy/PRx99lF/96lfp3r17EteJALDgEEQAFkBvv/12rr766uy3335ZYYUV6lbOcp0IAAsKQQRgAXTGGWdkySWXzIEHHpgkWXHFFZPoiACw4GhW6QIAKM3DDz+c++67L+edd14WWWSRJEmbNm3SqVMnF6wDsMDQEQFYgEyfPj2/+c1vssEGG2SHHXao91i3bt2MZgGwwBBEABYgN9xwQ95+++38+te/nu2xlVZaKWPHjs3kyZMrUBkAlKaqtra2ttJFAAAACxcdEQAAoHCCCAAAUDhBBAAAKJwgAgAAFE4QAQAACieIAAAAhRNEAACAwgkiAABA4QQRAACgcIIIAABQOEEEAAAonCACAAAU7v8BMskTAMcQ2KAAAAAASUVORK5CYII=", - "text/plain": [ - "
    " - ] - }, - "metadata": { - "filenames": { - "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/week43_100_0.png" - } - }, - "output_type": "display_data" - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAyIAAANdCAYAAABoOchKAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAABV4klEQVR4nO3dd5RV5fk24HvoSFHR0MSCoIBdIyoGe0liiZqQL1FQjMFeYm8xxRI1AmKvERNbMIkGS2yxlyjWqBEBC3aBKAoiSpv5/lDmlxFQjh72GYbrWmvWyuxT9jPbyWbu8zz73VU1NTU1AQAAKFCjShcAAAAseQQRAACgcIIIAABQOEEEAAAonCACAAAUThABAAAKJ4gAAACFE0QAAIDCCSIABXH/WAD4P00qXQBApRx//PH5+9///qXP2WijjXL11Vd/431ddNFFadasWQYNGvSVzx02bFguueSSDBgwIL/61a++8b4BoD6qqvERHbCEeuONNzJ58uTa7y+66KKMHj06F1xwQe221q1bp3v37t94Xz169MghhxySQw899EufV11dna222ipt27bNu+++m4ceeigtW7b8xvsHgPpGRwRYYq200kpZaaWVar9v165dmjVrlvXWW69iNT388MOZMGFCzj777AwYMCC33nprfvzjH1esHgBYVFwjAvAVnnzyyQwYMCDrrrtuNtpooxx33HF1OinV1dUZNmxYtt5666y11lrZeuutM3To0MyaNSvJZ92QJLngggtq//eC3HDDDVl99dXz7W9/OxtvvHGuv/76+T7vgQceyE9/+tOst9566du3b379619n6tSptY+/+uqrOeSQQ7LRRhuld+/e2X///fPKK68kSUaNGpUePXpk1KhRdd5zzz33zJ577ln7/dZbb53TTz89AwcOzDrrrJNf/vKXSZIxY8bkkEMOySabbJI111wzm222WU477bR8+umnta+dOXNmzjnnnGyzzTZZZ511stNOO9WOwV177bXp0aNHxo8fX2f/N910U3r16pV33333S48RAA2DIALwJZ544onsvffeadGiRc4555yceOKJefzxx7PXXnvV/uF9+eWX589//nMOPvjgDB8+PLvvvnuuuOKKXHzxxUlSGyb69eu3wGCRJB9++GHuvffe7LrrrkmS3XbbLc8//3xeeOGFOs+77777sv/++2e55ZbLOeeck6OPPjp33313jjjiiCTJxIkT85Of/CSvvfZafvvb32bw4MF57733MnDgwHz44Ycl/fzXXntt1l577Vx00UXp169fJk2alP79++eTTz7JmWeemcsvvzw77rhjrr766lx11VW1rzv66KNz5ZVX5sc//nEuvfTS9O3bN8cff3xuvfXW7LzzzmnevHluuummOvsaOXJk+vTpk06dOpVUIwCLJ6NZAF9i6NCh6dq1ay699NI0btw4SbLuuutmxx13zA033JD+/fvn8ccfz1prrZUf/ehHST67wL1ly5Zp06ZNktSOenXs2PFLx75uueWWVFdXZ5dddkmSbL/99jnllFMyYsSInHrqqbXPO//889OrV69ccMEFqaqqSpI0a9Ys5557bt5777388Y9/zMyZM3PllVfmW9/6VpKkZ8+e2X333fPss8+mRYsWC/3zd+7cOUcffXTt9w8//HB69eqVc889N61bt06SbLrppnnkkUcyatSo7Lfffhk3blzuvPPOnHjiiRk4cGCSpE+fPnn77bczatSo7LTTTtluu+1y88035xe/+EWqqqoyYcKEPPbYYxk8ePBC1wbA4k1HBGABPvnkkzz77LPZYostUlNTk9mzZ2f27NlZccUV061btzzyyCNJko033jiPPPJI9thjj/zhD3/Iyy+/nAEDBtQGioV1ww03ZOONN06zZs0yderUzJo1K1tvvXVuvfXWTJs2LUny6aefZvTo0dl2221rQ0iS7LDDDrnzzjuz/PLL56mnnsp6661XG0KSz0LQfffdly222KKkmnr16lXn+759++aaa65J8+bN8/LLL+eee+7JxRdfnMmTJ2fmzJlJkqeeeirJZ0Hqf51//vm1gapfv355++238+STTyb5rBvSqlWrbLfddiXVB8DiS0cEYAGmTp2a6urqXH755bn88svnebx58+ZJkkGDBqVVq1a54YYbMmTIkAwePDirrbZaTjrppGyyySYLta/Ro0fnxRdfTJL07t17nsdvvvnm7LHHHpkyZUpqamqy3HLLLfC9Pvzww3Tp0mWh9vtVllpqqTrfV1dX5+yzz861116b6dOnp1OnTllnnXVqj8Xc/Sf50ho32WSTdOnSJSNHjkzv3r0zcuTI7LDDDnXeB4CGTRABWIBWrVqlqqoqe++9d3bcccd5Hp+7rG6jRo3Sv3//9O/fP++//34eeOCBXHLJJTn00EPzyCOPpFmzZl+5rxtvvDFLLbVULrroojRqVLdZ/etf/zrXX3999thjj7Ru3TpVVVV1LpZPkhkzZuSxxx7LuuuumzZt2szzeJI8+uij6dKlS20npbq6us7jH3/8cVq1avWldV522WX54x//mJNPPjnbb7997fhZv379ap/Ttm3bJMnkyZPTsWPH2u2vvPJKPvzww3z7299OVVVVdtttt1x99dXZfffdM378+Pz+97//qsMEQANiNAtgAVq3bp011lgjr776atZee+3ar9VWWy3nn39+7apTP/3pT3Paaacl+awL8MMf/jD9+/fP1KlTa0eqvhgu/tfMmTNzyy23ZOutt06fPn2y8cYb1/naddddM2bMmPz73/9Oq1at0qtXr9x333113uPBBx/Mfvvtl0mTJmXDDTfMs88+WyeMvP/++xk0aFAeeOCB2ms7JkyYUPv4lClTalfV+jJPPfVUunfvnh/96Ee1IWTixIkZN25cbbD59re/nSS5995767x2yJAh+d3vflf7/Q9/+MNMnTo1v//979OtW7esu+66X7l/ABoOHRGAL3HkkUdmv/32y1FHHZUf/OAHmTNnToYPH55nn302Bx10UJLPRqmGDx+e5ZdfPuuvv34mTpyYK6+8MhtttFHatWuX5LMuwdNPP50nnngiG264YZ3rO+6+++58+OGH2WmnneZbwy677JJzzz03I0aMyHrrrZfDDjssBx54YI488sjsuuuuee+993L22Wdn2223zeqrr5699947I0eOzKBBg7L//vunadOmufjii9OxY8fsvPPOad26dTp16pQLL7ywtsNy6aWXLtSNE9dZZ51cdNFFueyyy7Leeuvl9ddfz6WXXpqZM2fmk08+SfLZhfHf+973Mnjw4Hz66afp1atXHnzwwdx33311bhbZuXPnbLrppnn44YfrXBAPwJLBndUBPnf88cfn8ccfn+eT/EcffTQXXHBB/vOf/6Rp06ZZc801c+ihh2bDDTdMksyePTsXX3xxbr755kyYMCFt2rTJ1ltvnaOOOirLLrtskuTKK6/MRRddlFmzZuW2225L586da99/0KBBee655/LII4+kadOm861tzz33zHPPPZcHH3wwSy+9dO6///5ccMEFGTt2bNq1a5cddtghhx56aO01Ha+88koGDx6cUaNGpVmzZtl4441z7LHH1l478txzz+X000/PCy+8kOWXXz4DBw7Mq6++mvHjx+fqq69O8tl9RDbaaKOceeaZtXXMnDkzZ555Zu6666589NFH6dSpU3bcccfaMPPII4+kbdu2mTlzZi644ILcdNNN+eCDD9KtW7cceOCB81zAfvXVV+eMM87I/fffn/bt23+T/3wALGYEEQAqZtCgQWnevHkuvPDCSpcCQMGMZgFQuAsvvDDjx4/Pww8/nOuuu67S5QBQAYIIAIW7995788Ybb+TYY4/NBhtsUOlyAKgAo1kAAEDhLN8LAAAUThABAAAKJ4gAAACFE0QAAIDCLZGrZn2/1wmVLqFh+587RrOI/HdypSto0Gb1WrHSJTR4VbOtk7KoNR73RqVLaNCqWrasdAkN3u1vnlvpEhaoesLqlS5hvhp1HFfpEkqiIwIAABROEAEAAAq3RI5mAQDA11Wd6kqXMF+LW4dhcasXAABoAAQRAACgcEazAACgBHNq6udo1uL2h72OCAAAUDhBBAAAKNzi1sEBAICKqo6bspaDjggAAFA4QQQAACic0SwAAChBfb2h4eJGRwQAACicIAIAABTOaBYAAJRgTo1Vs8pBRwQAACicIAIAABTOaBYAAJTADQ3LQ0cEAAAonCACAAAUzmgWAACUYI7RrLLQEQEAAAoniAAAAIUzmgUAACWwalZ56IgAAACFE0QAAIDCGc0CAIASzKkxmlUOOiIAAEDhBBEAAKBwRrMAAKAE1ZUuoIHQEQEAAAoniAAAAIUzmgUAACWY44aGZaEjAgAAFE4QAQAACmc0CwAASjDHZFZZ6IgAAACFE0QAAIDCGc0CAIASuKFheeiIAAAAhRNEAACAwhnNAgCAEsxJVaVLaBB0RAAAgMIJIgAAQOGMZgEAQAmq3dCwLHREAACAwgkiAABA4YxmFWyDTVfLwMO3z0rd2+fD96fl1useyw1XPrRQr+3Wq1POuf7g/Px7QzLpnQ/rPNZnmzWy+4Fbp8sqy+eD96blnpufyV8uvz+zZ81ZBD9F/bXBpt0z8Bf/c3z//FhuuPLhhXptt16dcs6Ig/Lz7w+d5/huu+sG+dHPNkvnldrl/Ukf5e6RT2fEZfenes6Sd0ujDbbslYEn7pKVenTOh/+dmluHP5AbLvrnQr2229or5pw7j8/PN/51Jr35fp3Htv1pn/zooG3TaZVv5b9vT84twx/IzZfftyh+hHrt2727Zp9BW2blrt/KBx98nJv//mT+OmLUAp/ftGnj9PvJxtnuu2vnW+3b5r3/fpR7/vmfjLj2X5k9e/6/nwcesm1+9P82zrab/25R/Rj12rc3WjU/2//zYzz549x8w5P5258fW+DzmzZtnH67b5Jtv//5MZ70Ue696z8ZcfUjdY7x9jusk357bJLOKyybSROm5uYbn8zIvz5RxI9U72yw5RrZ+5e71p4nbhl+X264cOHPE+fedWJ+vtFJmfiF88R2P+2THx28/f+dJ664Pzddfu+i+BHqtQ0275GBx+6UlVbvmA//+1Fuveqh3HDpwp0vu625Qs655aj8fPPTMumtyXUeu/rxk7N8p2Xmec1P1jkxUz/4uBylNxhWzSoPQaRAPdddMb+9eK88eMfzueq8f2bNb6+cfY7+Xho1bpS//uGBL33tyqt1yMmX7J0mTRvP89j6m3bPSef1z4O3P58rz74jK3fvkL2P+G6WXnapXPy7WxbVj1Pv9Fzn8+N7+/O56vx/Zs0NVsk+R809vg9+6WtX7t4hJ188cL7Hd5cBm+aAE3fKQ3c+nyuG3J6ll22VPQ/dNl17dMzvDr9uUf049VLPb3fNb689OA+OfDJXnXlz1ty4e/b5zW5p1KRR/nrenV/62pV7ds7J1x0832P83QHfyeHD9sxfz7szT98/Oj2+3TX7ndIvLVs1z/Xn3LGofpx6p9canXPamT/J/feOzh+veCBrrbNi9j1gmzRu3Cgjrn10vq856LDts+32a+Xaqx7O2Bffzeo9O2XPvTdLh45LZ+jv/zHP89ded8Xs1m+jRf2j1Fu91lwhpw7+SR64Z3T+eNn9WWvdlbLvwdukcZNGuf7qf833NQcdvn22+d7aufaPD2fc6Heyeq9OGbDP5mnfcemcfcatSZLv77xejjxhp1x/zb/y1KhX03PNFXLAodulZctm+fNVjxT5I1Zcz293zcnXHZIHRz6ZP51xU9bauHt+/psfpXHjxvnLeV/+/+eVe3bOKX8+dL7nie8N6JvDz9krfznvjjx93+fniVN/nBatmuf6c25fVD9OvdNz/ZXz2yv3y4O3PJOrhvwja/ZeNfuc+IM0atw4f73o7i997co9OuXkP+0/3+PbdtlWWb7TMvnDaSPzwuOv1nls2tRPyvozwFyCSIEGHLJtXnnx3Qw57i9JkqceHpcmTRrnp/tvmZuufiQzZ8ye5zVNmjbOD/r3yZ6HbpeZM+d9PEm22+3b+e+7UzL42OtTXV2TZ/71cpZZrnV+uHffXPb7f2TOAj4VbWgGHLLNZ8f3+L8mSZ56+KU0adIoP91vy9x09b++/Pgesu18j2+jRlXZ46Ct8/QjL+X0I/5cu/3l0e/kkpt/kfX7dM8zj7686H6oembAcTvlleffzJCD/5gkeere0Z/9Dh/+vdx02b2Z+emseV7TpGnj/GDQVtnzuJ0zc8a8jyfJT37xvTx081MZfurfkyT/fmhsVujWIT8YtNUSFUQG7rN5Xn5pQn7/u5uTJE88/moaN2mU3Qd8Jzf+9Yl5fkfbtm2ZHXdeP3+45N78ZcRnn+g/8/RrSZJ9D9g6f7jkvkyZMr32+S1aNs0xx++c9977KO3bty3mh6pn9hq0eV4eNyG/P+WmJMmTo15NkyaNsvte38nfr398nmPcpm3L7LDLBvnDRffkr9d9foyfei1JMuigbXLFxfdmyofTs/te38mD947OHy66t/Y5XVZsl136bbjEBZE9j/tBXnn+zQw+aHiS5Kl7X0jjpo3zkyO+n5GX3bPg88S+W2ev436w4PPE4d/PQzc9leGn3Jgk+fdDY9KlW4fssu/WS1QQGXDU9/PKC29lyOHXJEmeun/MZ+fhQ7bLTcMfWPDx/dnm2fOo78/338IkWXXNFZIk/7rjubz7+vvzfQ6Um2tECtK0aeOss9Gq+dfdL9TZ/vCd/8lSrVtkzW+vMt/X9d68R/ofvE2uv+y+DB86/xNts+ZN8uknM1P9P0s4fPTh9DRt1iQtWzUv289Qny3w+N71+fHdYJX5vq735qun/0Fb5/rL78/wofP+wbvMcq3TdpmlMuqBMXW2v/7yxEyZ/HE22qJH2X6G+q5psyZZZ9PV86/b/l1n+8O3PJ2l2rTMmht3n+/rem+7Vvofs2OuP/eO2qDxRb/Z48L84bc31tk2e+bsNG2+5HxW0rRp46yz3sp55KGxdbY/dP+YtGrVPGuts+I8r1mqVbPcetPT+dcj4+psf/ONz/6I6NR5mTrb9z9wm0yePC133vZseYtfTDRt2jjrrL9yHnmw7jF+8L4XPzvG6857jFu1ap5bRz6VRx+ue4zfeL3uMT7p6BG57IJ76jxn1uw5adZsyfkdTj47T6z9ndXzr388U2f7w7c8lVZfep5YO/2P2Skjzrm9Nmh80a93Pz9/+O3f6mybNWsJO080a5x1Nlkt/7rj+TrbH77t2SzVpkXW7L3qfF/Xe+s10v/w7+X6C/6Z4affPN/ndFtzhUz/6FMhZCHNSVW9/FrcCCIF6bhiuzRt1iRvv/5ene3vvPHZ911W+dZ8Xzfu+bcycNuzMuLS+xfY2bj1usfSeaXl8qOfbZZWbVqk57orZte9vpPHHxiTaVOWjHZq7fF9re4J9J3P/yDr0nX5+b5u3PNvZ+B2gz87vvO53uPjjz7N7Flz0v4Lf9C1btsirdu2SMcu7crzAywGOq68fJo2b5q3X5lYZ/s74/+bJOnSvcN8XzfumdczcINfZsSw2xf4O/zmSxNqrxlpvcxS+e6A72Sb/7dJ/nHll4/UNSSdOi+TZs2a5K03685sv/3WB0mSFVdabp7XTHh3Ss4bdsc8r9m07+qZNWtOne0bbNg123537Qw+49bU1CyZ607OPcZvv/GF88SXHuMPc/6QO/LWG3WP8Xc271HnGL/x+vuZOGFKkqRNmxb5/s7rZbvvrZNb/v7UovhR6q2OKy+fZvM7T7w69zzRcb6vG/fMaxm4/gkZMey2zJk9/2sb33xpQu01I62XWSrfG9A32/6/Prl1+JePNjckHVdaPk2bN8nbr06qs/2d1z4/vqu2n+/rxj37RgZuenJGnP/P+f5blySrrtElH334cX556T752wtn5sYxZ+X4iwZm2SW0e0ox6s3HCLNnz85dd92VJ554Iu+++25mzpyZli1bpkOHDundu3e23377NG4870zj4qJVmxZJkunTPq2zffrHM5MkS7Wef+fi/UlTv/K9//3YK/nb8Acz6NgdMujYHZIkL49+O78/esQ3KXmx0qr158f34/Ie3xmfzsqDdzyXH+zRJ2+8PCn/uvuFLN2udQ44YafMmVOdFks1K0P1i4dWbVsmSaZ/9IVj/Pnv9FKf/45/0fsTPlzoffTcsGuG3X5cks/+MFnYi+Abglafdy+nfzyjzvbpn3z2/VIL+bv2nc16ZPvvrZObbnwy0z7/b9OqVfMcfdyO+dPwB/P2Fy5OXZLMPU98/Pl5Ya7p0z8/xgvZQf7O5j2y3ffXyU03PJFpX/j/Q6+1Vsh5l/0sSTL2xXfyt+sWfBF8Q1TEeaLXhqtm2B3HJ5l7nrjra1S6eFrg3xLTPv8dXuDxnfKV773qmitkuY7L5PbrHs3IK+7PSt07ZMBRO+Ssvx6aQ743ODM+mfmV7wGlqhcdkbfeeis77rhjTjzxxIwdOzYtWrTIt771rTRt2jRjxozJCSeckJ133jnvvPNOpUv92qqqvrxdVv0N7oxzyG92Tb99Ns91F9+bY/e6LENP+GvaLL1UTrt8nzRv0fRrv+/ipKrRoju+5598U+695d/5xSm75a+P/ToX3HBIXvz3G3nphXeWqBPzojzGc016c3KO3WVohh76p7TrsHTOvu3YNG/pdzjJQnUx+m7eI7/89a75z/Nv5rJL/m9M6MBDt8ukSVNzw18WvPrWkuArz8MLc4y36JETT94t/3nuzVx+4T3zPD5pwpQcddBVGXzazWm3XOucc9neab4EjQ595e9xGc4TE996P8f8YEiGHnJllu2wdIbdfnyat1wyPhRalOfhc48dkaN2OyfXX/DPvPD4q7n9ukfzu/2Hp8uq7bNtv95f+30bquqaqnr5tbipF2fHU045JV26dMnf/va3tGnTZp7Hp06dmiOOOCKnnHJKLrnkkgpU+M19PPfToC984jb3k/ovfrqxsJZr3zbf/3+9c/1l9+fq8z779Pj5J8Zn3PNv5dJbj8j2P9wwt1w3/9V2GpLa47vUAo7vRzPmec3C+nT6zJzzqxtzyRm3pn3nZTLpnQ/z6fSZ+W6/DfPuG0vOLO3Hn6+aslTrup+4zf0EbnoZVlWZPHFKJk+ckuf/9VImvP7fDL756PTdeYPcswT8Af3x559otvzC73Crz7+f+/iC/OjHG2W/g7bJs/9+Pb858W+ZNfOz8ZaN+3TPVtuskYP2HZ6qRlWpyv/9Qd6ocVVqqmuypExqffzx3PNE3T9a/+8Yf/l5+Ic/2Sj7HbJtnnvm9fzm+L/WHuP/9f570/L+e9Py3L/fyLvvfJizL9orm23VK3d/Yaa/oZp7Hmj5hS703PPExx+V4TwxYUomT5iS5/+VvPv6exlyyzGfnycafvfp4887TQs8D3+D4zvm84Uu/tfoJ8dn2pTp6dprha/9vvBl6kUQeeKJJzJixIj5hpAkadu2bY455pj079+/4MrK5903JmfO7Dnp9IUZ5M6ff//GK5Pm97Kv9K3Oy6RRo0YZ/fTrdba/8cqkTPng46y82vznRRua2uO78gKO76tf7/gmyUZb9Mi0qZ9k9DNv5I2XP3ufpdu1yvId2ubl0Ytvl65U777238+Ocde61zN17vrZ79gb4yZ8rfdt0ap5NvnuOhn7zGt59/PrTZLk5efeTJK067jM1yt4MfPOOx9kzuzqrLDCsnW2d/78OqQ3vnB92f86+LDts1u/3rn3n//JWWfcUufeFptv2TPNmzfNFVftP8/r7rrvxNx5+7MZ/PkStA3dO29/dow7d1nAMX5twR8sHHTE9tntxxvl3rv+k8Gn3VznGLdo2TR9+q6esaPfyTtvf1C7/aWx7yZJlvvW/P9ta4je+fw80fkL1yrMPU+8Oe7dr/W+LVo1zybfWzdjnx7/hfPEG0mS5ZaQ88S7r7/3+b91da977LzKZ9+/8dLE+b3sKy3VpkX67rBuxv77jbw+9v/+G1VVVaVJsyaZMnna1y8avkS9GM1q06ZNJk788v/zvPPOO2nRYv6zj4uDWTNn5/knX8t3tlurzva+26+VaVM/ybjn3/pa7/vu6+9nzuw586y6tcIqy2fpZVvl3TeXjHnw2uO77Zp1tvfdbs1vdHyTZIefbJxBx+xQZ9uue30n1XNq8vgXVtNqyGbNmJ3nH30p39lx/Trb++60fqZNmZ5xz4z/Wu87Z/ac/GLYnul38HZ1tm+wZa8kyWuj3/56BS9mZs2ck+eeeyN9N6+7EttmW/TItI8+zZgX5x96f77fltmtX+/89frHcvqpN81zE8OrrnwoB+07vM7XP27+bEWjg/YdnqsW8oaqDcGsmXPy3LNvpO+WPets32yrnpn20acZu4DftX0O2Cq7/Xij/O3Pj+WM346c5xjPmVOdI0/YKT/u36fO9g03+mwFo/Evf70/DhdHCzxP7LxBpk2ZnrHz+dR9YcyZPSeHD9srPz7ku3W2f3vLNZIk40d//XP84mTWjNl5ftQr+c7316mzve/31/3sPPzv1xfwyq9435mzc+Cp/fKTg7ets32T7ddKi5bN8ty/XvraNTdUlV4dq6GsmlUvOiL9+vXL8ccfn1/84hfZZJNN0qlTpzRr1iwzZ87MxIkT8/jjj2fIkCHp169fpUv9RkZccm9OH/7znDhsj9x145Pptf7K+dE+m+XKs+/MjE9nZalWzbNS9/Z5943JmbKQdzCd8sHHGXnVI+m3z+ZJkmf+9XLad14m/Q/eJhPf/iB3LEF39R1x6X05/Yp9cuKw3XPXjU+l13orzXt8u7XPu28u/PFNkpuv+Vd+94d9st9xO+Sx+8ZkvU265af7bZm/XP7AEhP05hpx9u05/YZf5MQr9s1d1/0rvXqvmh8dsl2uPHVkZnwyK0u1bpGVenTKu6/9N1PeX7hP0GbNmJ2/nHtHBhy3Uz5876M89/C4dF2zS/ofs2Oevv/FPHH3fxbxT1V/XHvVwznr7P751ck/zB23PZs11+qS//fTPvnDpfdmxozZWWqpZll5leXzztsfZsqU6enWvUN+ssemGfPiO3nwvhfTa43Odd7v9dfey8QJU2pXc5prk00/W0J13Niv9+n04uy6Pz6c35/bP7867Ye549Zns8baXfLjPfrkiov/5xh3/VbeefuDTPlwerqt1iE/GbBpxox+Ow/e+2J6rVl3ROX18f/N9OkzM+KqR7LXoC3y4Qcf59mnX8+q3dtnz302z1OPv5rHH32lQj9tZfz57H/kjBuOyC+v2D93XvdI1ui9avodsn2Gn/r3zPhk5jc4T9yeAcftnA//OzXPPjI2q665Yvofs1Oevn/0EnWeGHHeXTn9zwflxIv3zl3Xj0qvDbvmRwdsnSvPuOWzf+taN89Kq3XMu6+/lymTF+7fulkzZuevF96dPY/eIR+891GeuHd0VunZOQOO+F7+dedzeVYQYRGpqqkH6zjW1NTkwgsvzJVXXpnp06fP83irVq3Sv3///OIXv0ijRt+8ifP9Xid84/f4ujbddo0MOGTbdOn6rbw3cWpuve7R3PjHh5Mka/fumrOu2i9DT/hr7h759Dyv3XbXDXLUGT/OwG1+n0nvfFjnsV33+k52+MlG6dilXSb/96M8/chL+dM5d5X0B3fZfMUFoYvSptuskQGHbPN/x/fPj9U9vn/aN0NP/NuCj+/p/TJw27PmOb5b7LBOdj9gq3RYYdlMeufD/GPEqNy8gDtdF+K/lQtAm+6wXgYcu1O6dO+Q9979MLcOfyA3XvzZ3XzX3nT1nHXTkRl66J9y94h5j8+2P+2To84fmIEb/LJ2ud65dhi4WXb++ZbptMq3MuX9abn/xsdzzVm3ZtYCbr61KM3qNe/9JIrync16ZOA+m6XLisvl/fc+yk1/fyp/u/6za2TWXW+lDD1vz5x1+i25647nMnCfzbPn3pst8L2OOuzqPPvvN+bZvtfPNsteP9s8227+u0X2c3yVqtmV+6fnO5v3yF6DNk+XlZbL+//9KDff+GT+9ufPjvE666+coRfumcGn3Zy7bnsuAwdtkQH7fMkxPvjqPPfMZ59C77TrBvlBvw3TeYVlM+WD6bn3n//JVVc8ON9rSYrQeNy8/+2LsukO62XP436QFbp3yPvvfphbht+fGz9fBW+d76yes246OkMPuTL/nM95Yruf9slRF/wsA9c/oXa53rl22Hvz/ODnW9WeJ+67YVSuOeuWipwnqlq2LHyfc236vXUy4Mjvp8uq7fPehA9z61UP58bL7kuSrL1J95z110Mz9Mhrc/dfH5/ntdv+eKMcdXb/DOxzcib9zyp6VVVV2WHAptlp4GbptPJy+eiD6bnv70/mmmF3zPcmiUW4/c1zK7LfhfH0GytVuoT52mClyv3//uuoF0FkrlmzZuXFF1/MxIkT88knn6RFixbp2LFjevbsmWbNyrciRiWDyBKhgkFkiVHBILIkqGQQWVJUMogsKSoZRJYElQwiS4r6HESeeGOVSpcwX71Xeq3SJZSkXoxmzdW0adOss846X/1EAABgsVYvLlYHAACWLPWqIwIAAPXd4njzwPpIRwQAACicIAIAABTOaBYAAJRgcbx5YH2kIwIAABROEAEAAApnNAsAAEowp8Zn+eXgKAIAAIUTRAAAgMIZzQIAgBJU+yy/LBxFAACgcIIIAABQOKNZAABQAjc0LA8dEQAAoHCCCAAAUDijWQAAUAI3NCwPRxEAACicIAIAABTOaBYAAJSg2qpZZaEjAgAAFE4QAQAACmc0CwAASjDHZ/ll4SgCAACFE0QAAIDCGc0CAIASuKFheTiKAABA4QQRAACgcEazAACgBNU+yy8LRxEAACicIAIAABTOaBYAAJRgTk1VpUtoEHREAACAwgkiAABA4YxmAQBACeb4LL8sHEUAAKBwgggAAFA4QQQAACica0QAAKAE1TU+yy8HRxEAACicIAIAABTOaBYAAJTA8r3l4SgCAACFE0QAAIDCGc0CAIASzKmpqnQJDYKOCAAAUDhBBAAAKJzRLAAAKEG1z/LLYokMIr+9Y0SlS2jQ3q9uVekSGryPq5tXuoQG7fWZ4ytdQoO3Rou3K11CgzerZon8J74wS1XNqHQJsNgT5wAAgML5uAQAAEowp8Zn+eXgKAIAAIUTRAAAgMIZzQIAgBJUxw0Ny0FHBAAAKJwgAgAAFM5oFgAAlMCqWeXhKAIAAIUTRAAAgMIZzQIAgBLM8Vl+WTiKAABA4QQRAACgcEazAACgBNU1bmhYDjoiAABA4QQRAACgcEazAACgBFbNKg9HEQAAKJwgAgAAFM5oFgAAlKC6xmf55eAoAgAAhRNEAACAwhnNAgCAEsyJGxqWg44IAABQOEEEAAAonNEsAAAogVWzysNRBAAACieIAAAAhTOaBQAAJbBqVnnoiAAAAIUTRAAAgMIZzQIAgBJYNas8HEUAAKBwgggAAFA4o1kAAFCCOUazysJRBAAACieIAAAAhTOaBQAAJah2Q8Oy0BEBAAAKJ4gAAACFM5oFAAAlsGpWeTiKAABA4QQRAACgcEazAACgBNU1Vs0qBx0RAACgcIIIAAAs4aqrq3Peeedls802y3rrrZd99903b7755gKf//777+eoo47KJptsko033jhHHHFEJk6cWNI+BREAACjBnDSql1/fxEUXXZTrrrsup556akaMGJHq6uoMGjQoM2fOnO/zDz/88Lzzzju58sorc+WVV+add97JwQcfXNI+BREAAFiCzZw5M8OHD89hhx2WLbfcMj179sywYcMyYcKE3HXXXfM8f+rUqXn88cez7777plevXlljjTWy33775fnnn8+HH3640PsVRAAAYAk2ZsyYfPzxx+nTp0/ttrZt22aNNdbIE088Mc/zW7RokVatWmXkyJGZNm1apk2blptuuildu3ZN27ZtF3q/Vs0CAIAS1NdVs7bZZpsvffyee+6Z7/YJEyYkSTp16lRne/v27Wsf+1/NmjXLmWeemV//+tfZcMMNU1VVlfbt2+eaa65Jo0YL3+fQEQEAgCXYJ598kuSzgPG/mjdvnhkzZszz/Jqamrz44otZf/31c+211+ZPf/pTOnfunIMOOijTpk1b6P3qiAAAQAOwoI7HV2nRokWSz64Vmfu/k2TGjBlp2bLlPM+//fbbc8011+S+++5L69atkySXXHJJttpqq/ztb3/L3nvvvVD7FUQAAKAE1Q1sqGjuSNakSZOy0kor1W6fNGlSevToMc/zn3zyyXTt2rU2hCTJ0ksvna5du+b1119f6P02rKMIAACUpGfPnmndunVGjRpVu23q1KkZPXp0evfuPc/zO3bsmNdff73O2Nb06dPz1ltvZZVVVlno/QoiAACwBGvWrFkGDBiQIUOG5J577smYMWNyxBFHpGPHjtl+++0zZ86c/Pe//82nn36aJNl1112TfHYvkTFjxmTMmDE58sgj07x58/zwhz9c6P3Wm9GsPffcM1VVC7cCwVVXXbWIqwEAgPmbU09XzfomDjvssMyePTsnnXRSPv300/Tu3TtXXHFFmjZtmrfeeivbbLNNzjjjjPzwhz9M+/btc91112Xw4MEZOHBgGjVqlA033DDXXXdd2rRps9D7rDdBpG/fvjn33HPTtWvXrLPOOpUuBwAAlhiNGzfOMccck2OOOWaex7p06ZKxY8fW2datW7dccskl32if9SaI7L///mndunWGDh2aSy+9NF26dKl0SQAAwCJSr64R6d+/fzbaaKOcddZZlS4FAADmq7qmql5+LW7qTUdkrlNOOSUvvPBCpcsAAAAWoXoXRNq3b5/27dtXugwAAGARqndBBAAA6rPqmnp1dcNiy1EEAAAKJ4gAAACFM5oFAAAlmJPFb4Wq+khHBAAAKJwgAgAAFM5oFgAAlGBxvHlgfaQjAgAAFE4QAQAACmc0CwAASuCGhuXhKAIAAIUTRAAAgMIZzQIAgBJUu6FhWeiIAAAAhRNEAACAwhnNAgCAEsxxQ8Oy0BEBAAAKJ4gAAACFM5oFAAAlcEPD8nAUAQCAwgkiAABA4YxmAQBACaqtmlUWOiIAAEDhBBEAAKBwRrMAAKAE1TGaVQ46IgAAQOEEEQAAoHBGswAAoARWzSoPHREAAKBwgggAAFA4o1kAAFCC6hqf5ZeDowgAABROEAEAAApnNAsAAEpg1azy0BEBAAAKJ4gAAACFM5oFAAAlqI7RrHLQEQEAAAoniAAAAIUzmgUAACWwalZ56IgAAACFE0QAAIDCGc0CAIASGM0qDx0RAACgcIIIAABQOKNZAABQAqNZ5aEjAgAAFE4QAQAACrdEjmat06zSFTR0H1e6gAavUaZXuoQGrWmrjypdQoM3q2Z2pUto8KpTU+kSGrSJcz6tdAlUkNGs8tARAQAACieIAAAAhVsiR7MAAODrqo7RrHLQEQEAAAoniAAAAIUzmgUAACWwalZ56IgAAACFE0QAAIDCGc0CAIASGM0qDx0RAACgcIIIAABQOKNZAABQAqNZ5aEjAgAAFE4QAQAACmc0CwAASmA0qzx0RAAAgMIJIgAAQOGMZgEAQAlqjGaVhY4IAABQOEEEAAAonNEsAAAoQXWMZpWDjggAAFA4QQQAACic0SwAACiBGxqWh44IAABQOEEEAAAonNEsAAAogRsaloeOCAAAUDhBBAAAKJzRLAAAKIFVs8pDRwQAACicIAIAABTOaBYAAJTAqlnloSMCAAAUThABAAAKZzQLAABKYNWs8tARAQAACieIAAAAhTOaBQAAJaipqXQFDYOOCAAAUDhBBAAAKJzRLAAAKEF1rJpVDjoiAABA4QQRAACgcEazAACgBDVuaFgWOiIAAEDhBBEAAKBwRrMAAKAE1UazykJHBAAAKJwgAgAAFM5oFgAAlKCmptIVNAw6IgAAQOHqTRB57bXXcv755+e0007Lgw8+OM/j06ZNywknnFCBygAAgHKrF0Hkqaeeyq677ppbbrklDz30UPbff//84he/yMyZM2uf8+mnn2bkyJGVKxIAAPLZDQ3r49fipl4EkaFDh+ZHP/pR7rrrrtx5550555xz8vDDD+eggw7K7NmzK10eAABQZvUiiIwdOzZ77bVX7fff/e53c/nll+epp57KcccdV8HKAACARaFeBJHWrVvn/fffr7Ntgw02yODBg3P77bfnjDPOqFBlAABQV6VHsIxmldEWW2yRk08+Of/+978za9as2u3bbrttTjzxxPzpT3/KKaecUsEKAQCAcqoXQeSoo47Kcsstl9133z2PPvponccGDBiQX//617n33nsrVB0AAFBu9eKGhksvvXSGDx+eN954I8suu+w8j++xxx7p06dP7rrrrgpUBwAA/6d6MRyDqo/qRRCZa6WVVlrgY127ds3+++9fYDUAAMCiUi9GswAAgCVLveqIAABAfVdTU+kKGgYdEQAAoHCCCAAAUDijWQAAUILF8eaB9ZGOCAAAUDhBBAAAKJzRLAAAKIHRrPLQEQEAAAoniAAAAIUzmgUAACVwP8Py0BEBAAAKJ4gAAACFM5oFAAAlsGpWeeiIAAAAhRNEAACAwhnNAgCAUlg2qyx0RAAAgMIJIgAAQOGMZgEAQAmsmlUeOiIAAEDhBBEAAKBwRrMAAKAENVbNKgsdEQAAoHCCCAAAUDijWQAAUAKrZpWHjggAAFA4QQQAACic0SwAACiF0ayy0BEBAAAKJ4gAAACFM5oFAAAlcEPD8tARAQAACieIAAAAhTOaBQAApTCaVRY6IgAAQOEEEQAAoHBGswAAoAQ1bmhYFjoiAABA4QQRAACgcEazAACgFFbNKgsdEQAAoHCCCAAAUDijWQAAUAKrZpWHjggAACzhqqurc95552WzzTbLeuutl3333TdvvvnmAp8/a9asDB06tPb5AwYMyIsvvljSPgURAABYwl100UW57rrrcuqpp2bEiBGprq7OoEGDMnPmzPk+/7e//W1uvPHGnH766bnhhhvSrl277Lvvvvnoo48Wep+CCAAAlKKmnn59TTNnzszw4cNz2GGHZcstt0zPnj0zbNiwTJgwIXfdddc8z3/zzTdzww035He/+10222yzdOvWLaeddlqaNWuW//znPwu9X0EEAACWYGPGjMnHH3+cPn361G5r27Zt1lhjjTzxxBPzPP+RRx5JmzZtsvnmm9d5/r333lvnPb7KEnmxevOqppUuAWCJ1rRqifznp1Cf1MyodAkNWrMqFytT/2yzzTZf+vg999wz3+0TJkxIknTq1KnO9vbt29c+9r/Gjx+fFVdcMXfddVcuu+yyTJw4MWussUaOP/74dOvWbaHr1REBAICSVNXTr6/nk08+SZI0a9aszvbmzZtnxox5P9SYNm1aXn/99Vx00UU58sgjc/HFF6dJkybZY4898v777y/0fn0kBQAADcCCOh5fpUWLFkk+u1Zk7v9OkhkzZqRly5bzPL9JkyaZNm1ahg0bVtsBGTZsWLbYYov8/e9/z6BBgxZqvzoiAACwBJs7kjVp0qQ62ydNmpQOHTrM8/yOHTumSZMmdcawWrRokRVXXDFvvfXWQu9XEAEAgFJUenWsMq+a1bNnz7Ru3TqjRo2q3TZ16tSMHj06vXv3nuf5vXv3zuzZs/P888/Xbvv000/z5ptvZuWVV17o/RrNAgCAJVizZs0yYMCADBkyJO3atcsKK6yQwYMHp2PHjtl+++0zZ86cTJ48OW3atEmLFi2y4YYbZtNNN81xxx2XU045Jcsss0zOO++8NG7cOLvssstC71dHBAAAlnCHHXZY+vXrl5NOOim77757GjdunCuuuCJNmzbNu+++m759++a2226rff7555+fjTbaKIccckj69euXadOm5aqrrkq7du0Wep9VNTU136CRs3iqnrB6pUsAgEXK8r2L1pTqWZUuocHrvMI7lS5hgVb54+8rXcJ8vbb3cZUuoSQ6IgAAQOEEEQAAoHAuVgcAgFLUfP2bB/J/dEQAAIDCCSIAAEDhjGYBAEAJlrw1ZxcNHREAAKBwgggAAFA4o1kAAFAKo1lloSMCAAAUThABAAAKZzQLAABK4YaGZaEjAgAAFE4QAQAACmc0CwAASlBl1ayy0BEBAAAKJ4gAAACFM5oFAAClMJpVFjoiAABA4QQRAACgcEazAACgFG5oWBY6IgAAQOEEEQAAoHBGswAAoBRWzSoLHREAAKBwgggAAFA4o1kAAFAKo1lloSMCAAAUThABAAAKZzQLAABKYTSrLHREAACAwgkiAABA4YxmAQBAKWqqKl1Bg6AjAgAAFE4QAQAACmc0CwAASlBl1ayy0BEBAAAKJ4gAAACFM5oFAAClMJpVFjoiAABA4QQRAACgcIIIAABQOEEEAAAoXEkXq3/00Ue56qqr8vzzz6dFixZp3759Vl999fTo0SOrrbZaWrRosajqBAAAGpCSgsgxxxyTBx54IMsvv3yWWWaZPProo5kyZUqqqqpSVVWVFVdcsTaYrL766ll99dWz8sorp6qqalHVDwAAhXJDw/IoKYg8+uij2WGHHTJ06NDabRMnTsy4ceMyduzYjBs3LuPGjcv999+fWbNmpaqqKi1atMgzzzzzle89Y8aMvPTSS+nevXtatGiRF198Mddcc00mTpyY1VZbLQMHDkzHjh1L/wkBAIB6p6Qg0qZNm2y00UZ1tnXo0CEdOnTIZpttVrttzpw5GT9+fG04+Sqvvvpq9t5770yaNCmdO3fOaaedloMOOigrrLBCunfvnrvvvjs33nhjrrvuunTr1q2UkgEAgHqopIvVv/e97+XZZ5/9yuc1btw43bt3z4477pgjjjjiK5//+9//Puutt15GjhyZjTbaKAceeGB22GGH3HrrrTn33HNz++23p2/fvjnjjDNKKRcAAMqvpqp+fi1mSgoiP/rRjzJu3LjceeedZS3i8ccfz+GHH56ePXvm2GOPzYwZMzJgwIDaa0uaNGmS/fffP0899VRZ9wsAAFRGSaNZu+22Wxo1apTDDz88G2+8cbbaaqusueaa6dGjR9q0afO1i2jRokU++eSTJEm7du3y//7f/0vz5s3rPGfq1KnfaB8AAED9UVIQOeOMM2ovSB87dmwee+yx2q5Fx44d06NHj/To0WOhxrH+V9++fXPqqafmtNNOS/fu3XPKKafUPlZdXZ1HH300p556arbddtuS3hcAAMrOqlllUVVTU/O1D+XkyZMzduzY2ovSx44dm1dffXWhVsn64vscdNBB6dKlS4YMGVLnsdtuuy1HHnlktthiiwwdOjStW7f+uuXWqp6w+jd+DwCozz6pmVHpEhq0KdWzKl1Cg9d5hXcqXcICrXrO2ZUuYb5ePfzISpdQkm8UROanpqbma983ZOrUqWnbtm2dbR988EHee++9rLbaauUoL4kgAkDDJ4gsWoLIoieIlG5xCyIljWYtjG9y88IvhpAkWXbZZbPssst+k5IAAKB8jGaVRUmrZgEAAJSDIAIAABSu7KNZAADQkFUZzSoLHREAAKBwgggAAFA4o1kAAFAKo1lloSMCAAAUThABAAAKZzQLAABKYTSrLHREAACAwgkiAABA4YxmAQBACdzQsDx0RAAAgMIJIgAAQOGMZgEAQClqqipdQYOgIwIAABROEAEAAApnNAsAAEph1ayy0BEBAAAKJ4gAAACFM5oFAAAlcEPD8tARAQAACieIAAAAhTOaBQAApTCaVRY6IgAAQOEEEQAAoHBGswAAoARWzSoPHREAAKBwgggAAFA4o1kAAFAKo1lloSMCAAAUThABAAAKZzQLAABKYTSrLHREAACAwgkiAABA4YxmAQBACdzQsDx0RAAAgMIJIgAAQOEEEQAAoHCCCAAAUDhBBAAAKJxVswAAoBRWzSoLHREAAKBwgggAAFA4o1kAAFACNzQsDx0RAACgcIIIAABQOKNZAABQCqNZZbFEBpGHPq10BQ3ba7OWr3QJDd5SjWZUuoQGrWPjKZUuocGbXN260iU0eDNr2lS6hAZtVs0S+SdUofaodAEsckazAACAwonzAABQCqNZZaEjAgAAFE4QAQAACmc0CwAASuCGhuWhIwIAABROEAEAAApnNAsAAEphNKssdEQAAIDCCSIAAEDhjGYBAEAJrJpVHjoiAABA4QQRAACgcEazAACgFEazykJHBAAAKJwgAgAAFM5oFgAAlMJoVlnoiAAAAIUTRAAAgMIZzQIAgBK4oWF56IgAAACFE0QAAIDCGc0CAIBSGM0qCx0RAACgcIIIAABQOKNZAABQCqNZZaEjAgAAFE4QAQAACmc0CwAASuCGhuWhIwIAABROEAEAAApnNAsAAEphNKssdEQAAIDCCSIAAEDhjGYBAEAJrJpVHjoiAABA4QQRAACgcEazAACgFEazykJHBAAAKJwgAgAAFE4QAQAACucaEQAAKIVrRMpCRwQAACicIAIAABTOaBYAAJSgqtIFNBA6IgAAQOEEEQAAoHBGswAAoBRWzSoLHREAAKBwgggAAFA4o1kAAFCCKqNZZaEjAgAAFK7eB5H99tsvkyZNqnQZAABAGdWL0ayRI0cu8LFRo0bl1ltvTbt27ZIku+66azFFAQDA/BjNKot6EUROPvnkfPrpp0mSmpp5/8ueddZZSZKqqipBBAAAGoB6MZp14403Zo011sgmm2ySBx54IGPGjKn9atmyZf75z39mzJgxefHFFytdKgAANDjV1dU577zzstlmm2W99dbLvvvumzfffHOhXnvzzTenR48eeeutt0raZ70IIl27ds3111+ftddeO7vssktuu+22SpcEAADzV1NPv76Biy66KNddd11OPfXUjBgxItXV1Rk0aFBmzpz5pa97++23c8opp3ytfdaLIJIkTZo0yZFHHpnzzz8/Q4YMydFHH52PPvqo0mUBAECDNnPmzAwfPjyHHXZYttxyy/Ts2TPDhg3LhAkTctdddy3wddXV1TnmmGOy5pprfq391psgMlfv3r0zcuTI1NTUZKeddsqsWbMqXRIAADRYY8aMyccff5w+ffrUbmvbtm3WWGONPPHEEwt83SWXXJJZs2Zl//33/1r7rRcXq39R27ZtM3To0IwcOTI33nhjmjdvXumSAAAgSf29oeE222zzpY/fc889890+YcKEJEmnTp3qbG/fvn3tY1/03HPPZfjw4fnb3/6WiRMnfo1q62kQmWvXXXe1ShYAACxCn3zySZKkWbNmdbY3b948U6ZMmef506dPz9FHH52jjz46q6yySsMMIgAAwMJZUMfjq7Ro0SLJZ9eKzP3fSTJjxoy0bNlynuefdtpp6dq1a376059+vUI/J4gAAEAp6ulo1tc1dyRr0qRJWWmllWq3T5o0KT169Jjn+TfccEOaNWuW9ddfP0kyZ86cJMlOO+2UAw44IAcccMBC7VcQAQCAJVjPnj3TunXrjBo1qjaITJ06NaNHj86AAQPmef4XV9J69tlnc8wxx+Syyy7L6quvvtD7FUQAAGAJ1qxZswwYMCBDhgxJu3btssIKK2Tw4MHp2LFjtt9++8yZMyeTJ09OmzZt0qJFi6y88sp1Xj/3gvbOnTtnmWWWWej91rvlewEAoD6rqqmfX9/EYYcdln79+uWkk07K7rvvnsaNG+eKK65I06ZN8+6776Zv375lv+l4VU1NTQObcvtqD7y28C0jSvfarOUrXUKDt1SjGZUuoUHr2HjeFUIor8nVrStdQoM3s6ZxpUto0GbVGCpZ1PboPqrSJSzQ+gcPq3QJ8/XMhUdUuoSS6IgAAACFE+cBAKAUS9w80aKhIwIAABROEAEAAApnNAsAAErwTVeo4jM6IgAAQOEEEQAAoHBGswAAoBRGs8pCRwQAACicIAIAABTOaBYAAJTCaFZZ6IgAAACFE0QAAIDCGc0CAIASuKFheeiIAAAAhRNEAACAwhnNAgCAUhjNKgsdEQAAoHCCCAAAUDijWQAAUIKqGrNZ5aAjAgAAFE4QAQAACmc0CwAASmEyqyx0RAAAgMIJIgAAQOGMZgEAQAmqjGaVhY4IAABQOEEEAAAonNEsAAAohdGsstARAQAACieIAAAAhTOaBQAAJbBqVnnoiAAAAIUTRAAAgMIZzQIAgFIYzSoLHREAAKBwgggAAFA4o1kAAFACq2aVh44IAABQOEEEAAAonNEsAAAohdGsstARAQAACrdEdkROW3XdSpfQoDVq2bLSJcA30miFTpUuocGb/fKrlS6h4avyWeOiVNW4caVLaPD2mFnpCljUlsggAgAAX5dVs8rDxyUAAEDhBBEAAKBwRrMAAKAUNWazykFHBAAAKJwgAgAAFM5oFgAAlMCqWeWhIwIAABROEAEAAApnNAsAAEphNKssdEQAAIDCCSIAAEDhjGYBAEAJqqorXUHDoCMCAAAUThABAAAKZzQLAABKYdWsstARAQAACieIAAAAhTOaBQAAJagymlUWOiIAAEDhBBEAAKBwRrMAAKAUNWazykFHBAAAKJwgAgAAFM5oFgAAlMCqWeWhIwIAABROEAEAAApnNAsAAEphNKssdEQAAIDCCSIAAEDhjGYBAEAJrJpVHjoiAABA4QQRAACgcEazAACgFDVms8pBRwQAACicIAIAABTOaBYAAJTAqlnloSMCAAAUThABAAAKZzQLAABKYTSrLHREAACAwgkiAABA4YxmAQBACayaVR46IgAAQOEEEQAAoHBGswAAoBTVZrPKQUcEAAAonCACAAAUzmgWAACUwmRWWeiIAAAAhRNEAACAwhnNAgCAErihYXnUi47IyJEjM3PmzDrbHnvssey33375wQ9+kKOOOiqvvPJKhaoDAADKrV4EkRNOOCEfffRR7fcPPfRQfvazn6WmpiZ9+/bNpEmT8sMf/jBPP/10BasEAADKpV6MZtXU1O1vXXzxxdl7771z3HHH1W4744wzMmTIkFx33XVFlwcAAP+nxmxWOdSLjsgXvf7669l5553rbPvJT36S0aNHV6giAACgnOpFEKmqqqrzfdeuXTNt2rQ62yZPnpw2bdoUWRYAALCI1JvRrG222SarrLJKunXrliZNmuTMM8/MiBEj0qxZszzxxBM55ZRTsvnmm1e6VAAAlnBWzSqPehFEHnjggYwdOzbjxo3L2LFj88EHH+TVV1/NnDlzkiQHHHBAunXrlqOOOqrClQIAAOVQL4JIhw4d0qFDhzodjzlz5qRx48ZJkuuvvz7dunWbZ4QLAABYPNWLIDI/c0NIknTv3r2ClQAAwP8wmlUW9eJidQAAYMkiiAAAAIWrt6NZAABQH1W5oWFZ6IgAAACFE0QAAIDCGc0CAIBSVFe6gIZBRwQAACicIAIAABTOaBYAAJTAqlnloSMCAAAUThABAAAKZzQLAABKYTKrLHREAACAwgkiAABA4YxmAQBAKayaVRY6IgAAQOEEEQAAoHBGswAAoARVJrPKQkcEAAAonCACAAAUzmgWAACUwqpZZaEjAgAAFE4QAQAACmc0CwAASlBVXekKGgYdEQAAoHCCCAAAUDijWQAAUAqrZpWFjggAAFA4QQQAACic0SwAACiFyayy0BEBAAAKJ4gAAACFM5oFAAAlqLJqVlnoiAAAAIUTRAAAgMIZzQIAgFIYzSoLHREAAKBwgggAAFA4o1kAAFCK6koX0DDoiAAAAIUTRAAAgMIZzQIAgBK4oWF56IgAAACFE0QAAIDCGc0CAIBSGM0qCx0RAACgcIIIAABQOKNZAABQCqNZZbFEBpEm3bpWuoQGrWbSe5UuocGrmT270iU0bFM/qnQFDV7j1q0rXUKDVzNnTqVLaNBqZjkPwzdlNAsAACjcEtkRAQCAr6260gU0DDoiAABA4QQRAACgcEazAACgBFVWzSoLHREAAFjCVVdX57zzzstmm22W9dZbL/vuu2/efPPNBT7/pZdeyn777ZeNN944ffr0yWGHHZZ33nmnpH0KIgAAsIS76KKLct111+XUU0/NiBEjUl1dnUGDBmXmzJnzPPeDDz7Iz372s7Ro0SJXX311Lr/88kyePDmDBg3KjBkzFnqfgggAAJSipqZ+fn1NM2fOzPDhw3PYYYdlyy23TM+ePTNs2LBMmDAhd9111zzPv/vuuzN9+vScddZZWX311bPWWmtl8ODBeeWVV/L0008v9H4FEQAAWIKNGTMmH3/8cfr06VO7rW3btlljjTXyxBNPzPP8Pn365KKLLkqLFi1qtzVq9FmsmDp16kLv18XqAADQAGyzzTZf+vg999wz3+0TJkxIknTq1KnO9vbt29c+9r+6dOmSLl261Nl22WWXpUWLFundu/dC1yuIAABAKRrYqlmffPJJkqRZs2Z1tjdv3jxTpkz5ytdfffXVueaaa3LSSSelXbt2C71fQQQAABqABXU8vsrcEauZM2fWGbeaMWNGWrZsucDX1dTU5Nxzz83FF1+cAw88MHvuuWdJ+3WNCAAALMHmjmRNmjSpzvZJkyalQ4cO833NrFmzcswxx+SSSy7JCSeckMMPP7zk/QoiAABQikqvjlXmVbN69uyZ1q1bZ9SoUbXbpk6dmtGjRy/wmo9jjz02d9xxR4YOHZq99977a+3XaBYAACzBmjVrlgEDBmTIkCFp165dVlhhhQwePDgdO3bM9ttvnzlz5mTy5Mlp06ZNWrRokRtvvDG33XZbjj322Gy00Ub573//W/tec5+zMHREAABgCXfYYYelX79+Oemkk7L77runcePGueKKK9K0adO8++676du3b2677bYkya233pokOeuss9K3b986X3OfszCqamoa2GX/C+H7qx1b6RIatJpJ71W6hAavZvbsSpfQoDVq07rSJTR4NdM/qXQJDV7NnDmVLqFBq5nlPLyo3TXzukqXsEDfW/OXlS5hvu544XeVLqEkOiIAAEDhBBEAAKBwLlYHAIASVC15VzYsEjoiAABA4QQRAACgcEazAACgFEazykJHBAAAKJwgAgAAFM5oFgAAlKLaaFY56IgAAACFE0QAAIDCGc0CAIBSWDWrLHREAACAwgkiAABA4YxmAQBAKYxmlYWOCAAAUDhBBAAAKJzRLAAAKIXRrLLQEQEAAAoniAAAAIUzmgUAAKWoNppVDjoiAABA4QQRAACgcEazAACgFDXVla6gQdARAQAACieIAAAAhTOaBQAApXBDw7LQEQEAAAoniAAAAIUzmgUAAKVwQ8Oy0BEBAAAKJ4gAAACFM5oFAAClsGpWWeiIAAAAhRNEAACAwtWb0axnn302o0aNyn777Zckeeyxx/LHP/4xb731VlZaaaXss88+2XDDDStcJQAASzyjWWVRLzoid9xxR3bfffc8/vjjSZL77rsvP/vZz1JTU5Mtttgis2bNysCBA3PfffdVuFIAAKAc6kVH5IILLshhhx2WAw44IEly8cUX54ADDsgvfvGL2udcfPHFOe+887LVVltVqkwAAKBM6kVH5I033siOO+5Y+/1bb72V7373u3Wes9NOO+WVV14pujQAAKirpqZ+fi1m6kUQWXHFFfPII4/Uft+rV6+MGTOmznOee+65dOjQoejSAACARaBejGbtu+++Oemkk/LWW29lp512ykEHHZTjjz8+M2bMyGqrrZZnn302F154YQ455JBKlwoAAJRBvQgiu+66a6qqqnLeeeflD3/4Q6qqqlJTU5Pf/OY3SZJWrVpl0KBB2XvvvStbKAAAVFdXuoIGoV4EkSTZZZddsssuu2T8+PEZP358pk2bliZNmqRjx45Zc80107x580qXCAAAlEm9CSJzde3aNV27dq10GQAAwCJU74IIAADUa4vhClX1Ub1YNQsAAFiyCCIAAEDhjGYBAEApjGaVhY4IAABQOEEEAAAonNEsAAAoRbXRrHLQEQEAAAoniAAAAIUzmgUAACWoqamudAkNgo4IAABQOEEEAAAonNEsAAAohVWzykJHBAAAKJwgAgAAFM5oFgAAlKLGaFY56IgAAACFE0QAAIDCGc0CAIBSVLuhYTnoiAAAAIUTRAAAgMIZzQIAgFJYNassdEQAAIDCCSIAAEDhjGYBAEAJaqyaVRY6IgAAQOEEEQAAoHBGswAAoBRWzSoLHREAAKBwgggAAFA4o1kAAFCKaqNZ5aAjAgAAFE4QAQAACmc0CwAASlHjhobloCMCAAAUThABAAAKZzQLAABKUGPVrLLQEQEAAAoniAAAAIUzmgUAAKWwalZZ6IgAAACFE0QAAIDCGc0CAIASWDWrPHREAACAwgkiAABA4YxmAQBAKayaVRY6IgAAQOEEEQAAoHBVNTU1LvsHAAAKpSMCAAAUThABAAAKJ4gAAACFE0QAAIDCCSIAAEDhBBEAAKBwgggAAFA4QQQAACicIAIAABROEAEAAAoniAAAAIUTRAAAgMIJIgAAQOEEkXqquro65513XjbbbLOst9562XffffPmm29WuqwG69JLL82ee+5Z6TIalA8//DC//vWvs/nmm2eDDTbI7rvvnieffLLSZTUo77//fo455phssskmWX/99bPffvvllVdeqXRZDdL48eOz/vrr58Ybb6x0KQ3KxIkT06NHj3m+HOfyGjlyZHbYYYesvfba2XHHHXP77bdXuiRIIojUWxdddFGuu+66nHrqqRkxYkSqq6szaNCgzJw5s9KlNTjXXnttzjnnnEqX0eAceeSReeaZZ3L22WfnhhtuSK9evfLzn/88r776aqVLazAOPvjgvP7667nsssvyt7/9LS1atMjee++dTz75pNKlNSizZs3K0UcfnenTp1e6lAZnzJgxad68eR566KE8/PDDtV877LBDpUtrMG666ab88pe/TP/+/fOPf/wjO+20U+35GSpNEKmHZs6cmeHDh+ewww7LlltumZ49e2bYsGGZMGFC7rrrrkqX12BMnDgxBxxwQIYMGZJVVlml0uU0KK+//noeeeSR/Pa3v82GG26Yrl275le/+lXat2+fW265pdLlNQhTpkzJCiuskNNOOy3rrLNOunXrloMOOiiTJk3KSy+9VOnyGpTzzz8/rVu3rnQZDdK4ceOyyiqrpH379vnWt75V+9WiRYtKl9Yg1NTU5Nxzz81ee+2V/v37Z6WVVsqBBx6YTTfdNI8//nilywNBpD4aM2ZMPv744/Tp06d2W9u2bbPGGmvkiSeeqGBlDcsLL7yQpk2b5uabb866665b6XIalGWXXTaXXXZZ1l577dptVVVVqaqqytSpUytYWcOx9NJLZ+jQoVl99dWTJJMnT84f//jHdOzYMd27d69wdQ3HE088keuvvz5nnnlmpUtpkMaOHZtu3bpVuowGa/z48Xn77bez884719l+xRVXZP/9969QVfB/mlS6AOY1YcKEJEmnTp3qbG/fvn3tY3xzW2+9dbbeeutKl9EgtW3bNltssUWdbXfeeWdef/31nHjiiRWqquH61a9+lb/85S9p1qxZLr744iy11FKVLqlBmDp1ao499ticdNJJ85yPKY9x48Zl2WWXTf/+/TN+/PisvPLKOfDAA7P55ptXurQGYfz48UmS6dOn5+c//3lGjx6dLl265MADD/TvH/WCjkg9NHe+u1mzZnW2N2/ePDNmzKhESfCNPP300znhhBOy/fbbZ8stt6x0OQ3OwIEDc8MNN2SnnXbKwQcfnBdeeKHSJTUIv/3tb7P++uvP82ky5TF79uy8+uqrmTJlSg499NBcdtllWW+99bLffvvl0UcfrXR5DcK0adOSJMcdd1x22mmnDB8+PN/5zndy0EEHOcbUCzoi9dDc2diZM2fWmZOdMWNGWrZsWamy4Gu5++67c/TRR2eDDTbIkCFDKl1OgzR3FOt3v/tdnn322VxzzTU544wzKlzV4m3kyJF58sknXdO0CDVp0iSjRo1K48aNa/+tW2uttfLSSy/liiuuqDOezNfTtGnTJMnPf/7z7LbbbkmSXr16ZfTo0bnyyisdYypOR6QemjsCMGnSpDrbJ02alA4dOlSiJPharrnmmhx66KHZaqutcskll6R58+aVLqnBmDx5cv7xj39k9uzZtdsaNWqU7t27z3PuoHQ33HBD3n///Wy55ZZZf/31s/766ydJfvOb32TQoEEVrq7haNWq1TwXpq+22mqZOHFihSpqWOb+zTD3WrK5unfvnrfeeqsSJUEdgkg91LNnz7Ru3TqjRo2q3TZ16tSMHj06vXv3rmBlsPDmLj/dv3//nH322fOMGvLNvPfeeznyyCPrjFfMmjUro0ePdvFvGQwZMiS33XZbRo4cWfuVJIcddlh+97vfVba4BuKll17KBhtsUOffuiT5z3/+Y8GFMllzzTXTqlWrPPvss3W2jxs3LiuttFKFqoL/YzSrHmrWrFkGDBiQIUOGpF27dllhhRUyePDgdOzYMdtvv32ly4OvNH78+Jx++unZbrvtsv/+++e9996rfaxFixZp06ZNBatrGFZfffVsvvnmOe2003Laaadl6aWXzqWXXpqpU6dm7733rnR5i70FdZ+XW245neky6datW1ZdddWccsopOfnkk7PsssvmL3/5S/7973/nhhtuqHR5DUKLFi0yaNCgXHjhhenQoUPWWWed/OMf/8gjjzySP/7xj5UuDwSR+uqwww7L7Nmzc9JJJ+XTTz9N7969c8UVV9TOe0J9duedd2bWrFn55z//mX/+8591Htttt90shVomZ599doYOHZojjjgiH330UTbccMNce+216dy5c6VLg6/UqFGjXHLJJRk6dGgOP/zwTJ06NWussUauvPLKeUaJ+PoOOuigtGzZMsOGDcvEiRPTrVu3nH/++dl4440rXRqkqqampqbSRQAAAEsW14gAAACFE0QAAIDCCSIAAEDhBBEAAKBwgggAAFA4QQQAACicIAIAABROEAEAAAoniAAspmbMmJE111wzG264Yc4///xKlwMAJRFEABZTs2fPzmmnnZbOnTvnwgsvzPvvv1/pkgBgoQkiAIupVq1aZbfddsugQYNSU1OTsWPHVrokAFhoggjAYm611VZLkrzyyisVrgQAFp4gArCYmzFjRpLk5ZdfrnAlALDwBBGAxdyZZ56ZJHn11VcrXAkALDxBBGAxdtNNN+WZZ57JcsstZzQLgMWKIAKwmPr4448zZMiQbL755tlll13y/vvv58MPP6x0WQCwUAQRgMXUpZdemg8++CC//OUv07179ySuEwFg8SGIACyG3njjjVx55ZXZZ599ssoqq9SunOU6EQAWF4IIwGLojDPOSLt27XLAAQckSVZdddUkOiIALD6aVLoAAErz0EMP5d57782wYcOy1FJLJUlat26dTp06uWAdgMWGjgjAYmTWrFk5/fTTs/HGG2eHHXao81i3bt2MZgGw2BBEABYj1157bd5444386le/muex1VZbLe+++24+/vjjClQGAKWpqqmpqal0EQAAwJJFRwQAACicIAIAABROEAEAAAoniAAAAIUTRAAAgMIJIgAAQOEEEQAAoHCCCAAAUDhBBAAAKJwgAgAAFE4QAQAACieIAAAAhfv/MYaJeHyuCBsAAAAASUVORK5CYII=", - "text/plain": [ - "
    " - ] - }, - "metadata": { - "filenames": { - "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/week43_100_1.png" - } - }, - "output_type": "display_data" - } - ], + "id": "90477fe7", + "metadata": { + "collapsed": false, + "editable": true + }, + "outputs": [], "source": [ "# optional\n", "# visual representation of grid search\n", @@ -3239,8 +2598,10 @@ }, { "cell_type": "markdown", - "id": "9d327f9b", - "metadata": {}, + "id": "29af8aff", + "metadata": { + "editable": true + }, "source": [ "## Building neural networks in Tensorflow and Keras\n", "\n", @@ -3255,8 +2616,10 @@ }, { "cell_type": "markdown", - "id": "2280cbce", - "metadata": {}, + "id": "a70cf999", + "metadata": { + "editable": true + }, "source": [ "## Tensorflow\n", "\n", @@ -3288,26 +2651,22 @@ { "cell_type": "code", "execution_count": 12, - "id": "dff81198", - "metadata": {}, - "outputs": [ - { - "ename": "SyntaxError", - "evalue": "invalid syntax (2357089093.py, line 1)", - "output_type": "error", - "traceback": [ - "\u001b[0;36m Cell \u001b[0;32mIn[12], line 1\u001b[0;36m\u001b[0m\n\u001b[0;31m pip3 install tensorflow\u001b[0m\n\u001b[0m ^\u001b[0m\n\u001b[0;31mSyntaxError\u001b[0m\u001b[0;31m:\u001b[0m invalid syntax\n" - ] - } - ], + "id": "812c98fe", + "metadata": { + "collapsed": false, + "editable": true + }, + "outputs": [], "source": [ "pip3 install tensorflow" ] }, { "cell_type": "markdown", - "id": "e9bf9cac", - "metadata": {}, + "id": "694c0bd7", + "metadata": { + "editable": true + }, "source": [ "and/or if you use **anaconda**, just write (or install from the graphical user interface)\n", "(current release of CPU-only TensorFlow)" @@ -3316,8 +2675,11 @@ { "cell_type": "code", "execution_count": 13, - "id": "1ccb8dba", - "metadata": {}, + "id": "1ecd18cc", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "conda create -n tf tensorflow\n", @@ -3326,8 +2688,10 @@ }, { "cell_type": "markdown", - "id": "efc3f4e2", - "metadata": {}, + "id": "584ab0ab", + "metadata": { + "editable": true + }, "source": [ "To install the current release of GPU TensorFlow" ] @@ -3335,8 +2699,11 @@ { "cell_type": "code", "execution_count": 14, - "id": "01556b01", - "metadata": {}, + "id": "a811e23d", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "conda create -n tf-gpu tensorflow-gpu\n", @@ -3345,8 +2712,10 @@ }, { "cell_type": "markdown", - "id": "b900f015", - "metadata": {}, + "id": "ab99008f", + "metadata": { + "editable": true + }, "source": [ "## Using Keras\n", "\n", @@ -3358,8 +2727,11 @@ { "cell_type": "code", "execution_count": 15, - "id": "3fd1f12f", - "metadata": {}, + "id": "fe547415", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "conda install keras" @@ -3367,8 +2739,10 @@ }, { "cell_type": "markdown", - "id": "20ec35c5", - "metadata": {}, + "id": "ed1e0fc6", + "metadata": { + "editable": true + }, "source": [ "You can look up the [instructions here](https://keras.io/) for more information.\n", "\n", @@ -3377,8 +2751,10 @@ }, { "cell_type": "markdown", - "id": "249e56f7", - "metadata": {}, + "id": "6aebcb51", + "metadata": { + "editable": true + }, "source": [ "## Collect and pre-process data\n", "\n", @@ -3388,8 +2764,11 @@ { "cell_type": "code", "execution_count": 16, - "id": "0ff66efc", - "metadata": {}, + "id": "a4a7d68d", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "# import necessary packages\n", @@ -3440,8 +2819,11 @@ { "cell_type": "code", "execution_count": 17, - "id": "2806c62b", - "metadata": {}, + "id": "21335418", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "from tensorflow.keras.layers import Input\n", @@ -3466,8 +2848,11 @@ { "cell_type": "code", "execution_count": 18, - "id": "3bf4c3f6", - "metadata": {}, + "id": "e81d58de", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "\n", @@ -3484,7 +2869,7 @@ " model.add(Dense(n_neurons_layer2, activation='sigmoid', kernel_regularizer=regularizers.l2(lmbd)))\n", " model.add(Dense(n_categories, activation='softmax'))\n", " \n", - " sgd = optimizers.SGD(lr=eta)\n", + " sgd = optimizers.SGD(learning_rate=eta)\n", " model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])\n", " \n", " return model" @@ -3493,8 +2878,11 @@ { "cell_type": "code", "execution_count": 19, - "id": "81a46485", - "metadata": {}, + "id": "b91287cc", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "DNN_keras = np.zeros((len(eta_vals), len(lmbd_vals)), dtype=object)\n", @@ -3517,8 +2905,11 @@ { "cell_type": "code", "execution_count": 20, - "id": "6e619c14", - "metadata": {}, + "id": "d108674e", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "# optional\n", @@ -3556,8 +2947,10 @@ }, { "cell_type": "markdown", - "id": "c303d97b", - "metadata": {}, + "id": "1198d799", + "metadata": { + "editable": true + }, "source": [ "## The Breast Cancer Data, now with Keras" ] @@ -3565,8 +2958,11 @@ { "cell_type": "code", "execution_count": 21, - "id": "03a57bfd", - "metadata": {}, + "id": "eb1eb4fb", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "\n", @@ -3679,7 +3075,7 @@ " else: #Subsequent layers are capable of automatic shape inferencing\n", " model.add(Dense(n_neuron,activation='relu',kernel_regularizer=regularizers.l2(lamda)))\n", " model.add(Dense(2,activation='softmax')) #2 outputs - ordered and disordered (softmax for prob)\n", - " sgd=optimizers.SGD(lr=eta)\n", + " sgd=optimizers.SGD(learning_rate=eta)\n", " model.compile(loss='categorical_crossentropy',optimizer=sgd,metrics=['accuracy'])\n", " return model\n", "\n", @@ -3739,8 +3135,10 @@ }, { "cell_type": "markdown", - "id": "f9ad1085", - "metadata": {}, + "id": "92834502", + "metadata": { + "editable": true + }, "source": [ "## Building a neural network code\n", "\n", @@ -3756,8 +3154,10 @@ }, { "cell_type": "markdown", - "id": "cf4f3c14", - "metadata": {}, + "id": "deb24cc1", + "metadata": { + "editable": true + }, "source": [ "### Learning rate methods\n", "\n", @@ -3776,8 +3176,11 @@ { "cell_type": "code", "execution_count": 22, - "id": "467ea7a2", - "metadata": {}, + "id": "d3619281", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "import autograd.numpy as np\n", @@ -3914,8 +3317,10 @@ }, { "cell_type": "markdown", - "id": "6af64d5a", - "metadata": {}, + "id": "f37fd7bc", + "metadata": { + "editable": true + }, "source": [ "### Usage of the above learning rate schedulers\n", "\n", @@ -3928,8 +3333,11 @@ { "cell_type": "code", "execution_count": 23, - "id": "290e0d00", - "metadata": {}, + "id": "5db34b9b", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "momentum_scheduler = Momentum(eta=1e-3, momentum=0.9)\n", @@ -3938,8 +3346,10 @@ }, { "cell_type": "markdown", - "id": "d1f43cc5", - "metadata": {}, + "id": "8b25a24d", + "metadata": { + "editable": true + }, "source": [ "Here is a small example for how a segment of code using schedulers\n", "could look. Switching out the schedulers is simple." @@ -3948,8 +3358,11 @@ { "cell_type": "code", "execution_count": 24, - "id": "0def5cec", - "metadata": {}, + "id": "2d5f2887", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "weights = np.ones((3,3))\n", @@ -3967,8 +3380,10 @@ }, { "cell_type": "markdown", - "id": "9b27dcd7", - "metadata": {}, + "id": "9070c2d3", + "metadata": { + "editable": true + }, "source": [ "### Cost functions\n", "\n", @@ -3981,8 +3396,11 @@ { "cell_type": "code", "execution_count": 25, - "id": "4217839c", - "metadata": {}, + "id": "c360c736", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "import autograd.numpy as np\n", @@ -4016,8 +3434,10 @@ }, { "cell_type": "markdown", - "id": "ef5f9fe9", - "metadata": {}, + "id": "42c7ec61", + "metadata": { + "editable": true + }, "source": [ "Below we give a short example of how these cost function may be used\n", "to obtain results if you wish to test them out on your own using\n", @@ -4027,8 +3447,11 @@ { "cell_type": "code", "execution_count": 26, - "id": "305e1479", - "metadata": {}, + "id": "cd9cb3a5", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "from autograd import grad\n", @@ -4045,8 +3468,10 @@ }, { "cell_type": "markdown", - "id": "1499b193", - "metadata": {}, + "id": "796aab4a", + "metadata": { + "editable": true + }, "source": [ "### Activation functions\n", "\n", @@ -4059,8 +3484,11 @@ { "cell_type": "code", "execution_count": 27, - "id": "e7b59b25", - "metadata": {}, + "id": "5ea4891d", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "import autograd.numpy as np\n", @@ -4114,8 +3542,10 @@ }, { "cell_type": "markdown", - "id": "57599619", - "metadata": {}, + "id": "2ff3c57c", + "metadata": { + "editable": true + }, "source": [ "Below follows a short demonstration of how to use an activation\n", "function. The derivative of the activation function will be important\n", @@ -4127,8 +3557,11 @@ { "cell_type": "code", "execution_count": 28, - "id": "5ee68acd", - "metadata": {}, + "id": "34289210", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "z = np.array([[4, 5, 6]]).T\n", @@ -4145,8 +3578,10 @@ }, { "cell_type": "markdown", - "id": "2d3295d9", - "metadata": {}, + "id": "0839f98b", + "metadata": { + "editable": true + }, "source": [ "### The Neural Network\n", "\n", @@ -4167,8 +3602,11 @@ { "cell_type": "code", "execution_count": 29, - "id": "c4cfc50a", - "metadata": {}, + "id": "0fdc2707", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "import math\n", @@ -4636,8 +4074,10 @@ }, { "cell_type": "markdown", - "id": "2e6e81c1", - "metadata": {}, + "id": "17bc192e", + "metadata": { + "editable": true + }, "source": [ "Before we make a model, we will quickly generate a dataset we can use\n", "for our linear regression problem as shown below" @@ -4646,8 +4086,11 @@ { "cell_type": "code", "execution_count": 30, - "id": "0d4f1ff8", - "metadata": {}, + "id": "f3f5d088", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "import autograd.numpy as np\n", @@ -4687,8 +4130,10 @@ }, { "cell_type": "markdown", - "id": "a8f1940f", - "metadata": {}, + "id": "53c3b4c2", + "metadata": { + "editable": true + }, "source": [ "Now that we have our dataset ready for the regression, we can create\n", "our regressor. Note that with the seed parameter, we can make sure our\n", @@ -4701,8 +4146,11 @@ { "cell_type": "code", "execution_count": 31, - "id": "ff6bd29b", - "metadata": {}, + "id": "890d682f", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "input_nodes = X_train.shape[1]\n", @@ -4713,8 +4161,10 @@ }, { "cell_type": "markdown", - "id": "a7b906df", - "metadata": {}, + "id": "dc4da852", + "metadata": { + "editable": true + }, "source": [ "We then fit our model with our training data using the scheduler of our choice." ] @@ -4722,8 +4172,11 @@ { "cell_type": "code", "execution_count": 32, - "id": "db7a5bf2", - "metadata": {}, + "id": "1e98a3f8", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "linear_regression.reset_weights() # reset weights such that previous runs or reruns don't affect the weights\n", @@ -4734,8 +4187,10 @@ }, { "cell_type": "markdown", - "id": "508c351b", - "metadata": {}, + "id": "68c5d793", + "metadata": { + "editable": true + }, "source": [ "Due to the progress bar we can see the MSE (train_error) throughout\n", "the FFNN's training. Note that the fit() function has some optional\n", @@ -4748,8 +4203,11 @@ { "cell_type": "code", "execution_count": 33, - "id": "6b0562b2", - "metadata": {}, + "id": "e735086e", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "linear_regression.reset_weights() # reset weights such that previous runs or reruns don't affect the weights\n", @@ -4759,8 +4217,10 @@ }, { "cell_type": "markdown", - "id": "f03172d9", - "metadata": {}, + "id": "360468f6", + "metadata": { + "editable": true + }, "source": [ "We see that given more epochs to train on, the regressor reaches a lower MSE.\n", "\n", @@ -4772,8 +4232,11 @@ { "cell_type": "code", "execution_count": 34, - "id": "26d1d1c3", - "metadata": {}, + "id": "c5937c59", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "from sklearn.datasets import load_breast_cancer\n", @@ -4795,8 +4258,11 @@ { "cell_type": "code", "execution_count": 35, - "id": "6a78c633", - "metadata": {}, + "id": "2e06b929", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "input_nodes = X_train.shape[1]\n", @@ -4807,8 +4273,10 @@ }, { "cell_type": "markdown", - "id": "5feaced2", - "metadata": {}, + "id": "94140fb1", + "metadata": { + "editable": true + }, "source": [ "We will now make use of our validation data by passing it into our fit function as a keyword argument" ] @@ -4816,8 +4284,11 @@ { "cell_type": "code", "execution_count": 36, - "id": "6a9b1538", - "metadata": {}, + "id": "da82b266", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "logistic_regression.reset_weights() # reset weights such that previous runs or reruns don't affect the weights\n", @@ -4828,8 +4299,10 @@ }, { "cell_type": "markdown", - "id": "90c7f8f3", - "metadata": {}, + "id": "19e8d4c2", + "metadata": { + "editable": true + }, "source": [ "Finally, we will create a neural network with 2 hidden layers with activation functions." ] @@ -4837,8 +4310,11 @@ { "cell_type": "code", "execution_count": 37, - "id": "58954140", - "metadata": {}, + "id": "4744accd", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "input_nodes = X_train.shape[1]\n", @@ -4854,8 +4330,11 @@ { "cell_type": "code", "execution_count": 38, - "id": "89536ca9", - "metadata": {}, + "id": "f5b5b198", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "neural_network.reset_weights() # reset weights such that previous runs or reruns don't affect the weights\n", @@ -4866,8 +4345,10 @@ }, { "cell_type": "markdown", - "id": "803f4791", - "metadata": {}, + "id": "1f5fc0a0", + "metadata": { + "editable": true + }, "source": [ "### Multiclass classification\n", "\n", @@ -4879,8 +4360,11 @@ { "cell_type": "code", "execution_count": 39, - "id": "50405951", - "metadata": {}, + "id": "4308d82f", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "from sklearn.datasets import load_digits\n", @@ -4913,8 +4397,10 @@ }, { "cell_type": "markdown", - "id": "d906edf4", - "metadata": {}, + "id": "96fc3508", + "metadata": { + "editable": true + }, "source": [ "## Testing the XOR gate and other gates\n", "\n", @@ -4924,8 +4410,11 @@ { "cell_type": "code", "execution_count": 40, - "id": "414ee306", - "metadata": {}, + "id": "5aa9a2a4", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "X = np.array([ [0, 0], [0, 1], [1, 0],[1, 1]],dtype=np.float64)\n", @@ -4944,18 +4433,22 @@ }, { "cell_type": "markdown", - "id": "982d1747", - "metadata": {}, + "id": "d57346ae", + "metadata": { + "editable": true + }, "source": [ "Not bad, but the results depend strongly on the learning reate. Try different learning rates." ] }, { "cell_type": "markdown", - "id": "616eae3e", - "metadata": {}, + "id": "0219b252", + "metadata": { + "editable": true + }, "source": [ - "## Solving ODEs with Deep Learning\n", + "## Solving differential equations with Deep Learning\n", "\n", "The Universal Approximation Theorem states that a neural network can\n", "approximate any function at a single hidden layer along with one input\n", @@ -4977,10 +4470,12 @@ }, { "cell_type": "markdown", - "id": "43fb869c", - "metadata": {}, + "id": "80032cfc", + "metadata": { + "editable": true + }, "source": [ - "## Ordinary Differential Equations\n", + "## Ordinary Differential Equations first\n", "\n", "An ordinary differential equation (ODE) is an equation involving functions having one variable.\n", "\n", @@ -4989,8 +4484,10 @@ }, { "cell_type": "markdown", - "id": "4855aef1", - "metadata": {}, + "id": "3c5546df", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -5004,8 +4501,10 @@ }, { "cell_type": "markdown", - "id": "d31c26fc", - "metadata": {}, + "id": "e4f540a5", + "metadata": { + "editable": true + }, "source": [ "where $g(x)$ is the function to find, and $g^{(n)}(x)$ is the $n$-th derivative of $g(x)$.\n", "\n", @@ -5018,8 +4517,10 @@ }, { "cell_type": "markdown", - "id": "08d629ec", - "metadata": {}, + "id": "75ec2cfe", + "metadata": { + "editable": true + }, "source": [ "## The trial solution\n", "\n", @@ -5028,8 +4529,10 @@ }, { "cell_type": "markdown", - "id": "b2cfd7bd", - "metadata": {}, + "id": "a71e7eb8", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -5044,8 +4547,10 @@ }, { "cell_type": "markdown", - "id": "f4cd106f", - "metadata": {}, + "id": "9b06d2bf", + "metadata": { + "editable": true + }, "source": [ "where $h_1(x)$ is a function that makes $g_t(x)$ satisfy a given set\n", "of conditions, $N(x,P)$ a neural network with weights and biases\n", @@ -5063,8 +4568,10 @@ }, { "cell_type": "markdown", - "id": "74da655f", - "metadata": {}, + "id": "0aea2f3e", + "metadata": { + "editable": true + }, "source": [ "## Minimization process\n", "\n", @@ -5078,8 +4585,10 @@ }, { "cell_type": "markdown", - "id": "efe0ec18", - "metadata": {}, + "id": "a21c6a43", + "metadata": { + "editable": true + }, "source": [ "$$\n", "C\\left(x, P\\right) = \\big(f\\left(x, \\, g(x), \\, g'(x), \\, g''(x), \\, \\dots \\, , \\, g^{(n)}(x)\\right)\\big)^2\n", @@ -5088,8 +4597,10 @@ }, { "cell_type": "markdown", - "id": "f9c1643a", - "metadata": {}, + "id": "ff939582", + "metadata": { + "editable": true + }, "source": [ "If $N$ inputs are given as a vector $\\boldsymbol{x}$ with elements $x_i$ for $i = 1,\\dots,N$,\n", "the cost function becomes" @@ -5097,8 +4608,10 @@ }, { "cell_type": "markdown", - "id": "9b900e26", - "metadata": {}, + "id": "9f63085a", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -5112,8 +4625,10 @@ }, { "cell_type": "markdown", - "id": "82e8e341", - "metadata": {}, + "id": "2875658a", + "metadata": { + "editable": true + }, "source": [ "The neural net should then find the parameters $P$ that minimizes the cost function in\n", "([3](#cost)) for a set of $N$ training samples $x_i$." @@ -5121,8 +4636,10 @@ }, { "cell_type": "markdown", - "id": "b89f2f97", - "metadata": {}, + "id": "8a63e4c6", + "metadata": { + "editable": true + }, "source": [ "## Minimizing the cost function using gradient descent and automatic differentiation\n", "\n", @@ -5135,8 +4652,10 @@ }, { "cell_type": "markdown", - "id": "cc38c37f", - "metadata": {}, + "id": "2d5b4c88", + "metadata": { + "editable": true + }, "source": [ "## Example: Exponential decay\n", "\n", @@ -5145,8 +4664,10 @@ }, { "cell_type": "markdown", - "id": "3c5f0410", - "metadata": {}, + "id": "daab1d5e", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -5160,8 +4681,10 @@ }, { "cell_type": "markdown", - "id": "4531d4a4", - "metadata": {}, + "id": "fc2cedc3", + "metadata": { + "editable": true + }, "source": [ "with $g(0) = g_0$ for some chosen initial value $g_0$.\n", "\n", @@ -5170,8 +4693,10 @@ }, { "cell_type": "markdown", - "id": "d580caea", - "metadata": {}, + "id": "da284bc1", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -5186,16 +4711,20 @@ }, { "cell_type": "markdown", - "id": "6f384ec5", - "metadata": {}, + "id": "04c2531a", + "metadata": { + "editable": true + }, "source": [ "Having an analytical solution at hand, it is possible to use it to compare how well a neural network finds a solution of ([4](#solve_expdec))." ] }, { "cell_type": "markdown", - "id": "e650af49", - "metadata": {}, + "id": "62a110db", + "metadata": { + "editable": true + }, "source": [ "## The function to solve for\n", "\n", @@ -5204,8 +4733,10 @@ }, { "cell_type": "markdown", - "id": "68e3a73b", - "metadata": {}, + "id": "7b5b722c", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -5219,8 +4750,10 @@ }, { "cell_type": "markdown", - "id": "61f7dce9", - "metadata": {}, + "id": "02446b0e", + "metadata": { + "editable": true + }, "source": [ "where $g(0) = g_0$ with $\\gamma$ and $g_0$ being some chosen values.\n", "\n", @@ -5229,8 +4762,10 @@ }, { "cell_type": "markdown", - "id": "47e985ed", - "metadata": {}, + "id": "205c8e89", + "metadata": { + "editable": true + }, "source": [ "## The trial solution\n", "To begin with, a trial solution $g_t(t)$ must be chosen. A general trial solution for ordinary differential equations could be" @@ -5238,8 +4773,10 @@ }, { "cell_type": "markdown", - "id": "b5787755", - "metadata": {}, + "id": "9e46aee1", + "metadata": { + "editable": true + }, "source": [ "$$\n", "g_t(x, P) = h_1(x) + h_2(x, N(x, P))\n", @@ -5248,16 +4785,20 @@ }, { "cell_type": "markdown", - "id": "efd0d663", - "metadata": {}, + "id": "cb0d1f30", + "metadata": { + "editable": true + }, "source": [ "with $h_1(x)$ ensuring that $g_t(x)$ satisfies some conditions and $h_2(x,N(x, P))$ an expression involving $x$ and the output from the neural network $N(x,P)$ with $P $ being the collection of the weights and biases for each layer. For now, it is assumed that the network consists of one input layer, one hidden layer, and one output layer." ] }, { "cell_type": "markdown", - "id": "de4cfa12", - "metadata": {}, + "id": "8485bdd3", + "metadata": { + "editable": true + }, "source": [ "## Setup of Network\n", "\n", @@ -5274,8 +4815,10 @@ }, { "cell_type": "markdown", - "id": "4de8eeb1", - "metadata": {}, + "id": "730e057b", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -5289,8 +4832,10 @@ }, { "cell_type": "markdown", - "id": "c3076a5d", - "metadata": {}, + "id": "db8907ef", + "metadata": { + "editable": true + }, "source": [ "## Reformulating the problem\n", "\n", @@ -5306,8 +4851,10 @@ }, { "cell_type": "markdown", - "id": "00459420", - "metadata": {}, + "id": "ce4a7022", + "metadata": { + "editable": true + }, "source": [ "$$\n", "g_t(x, P) = g_0 + x \\cdot N(x, P)\n", @@ -5316,16 +4863,20 @@ }, { "cell_type": "markdown", - "id": "493318e4", - "metadata": {}, + "id": "b17ab00d", + "metadata": { + "editable": true + }, "source": [ "has been chosen such that it already solves the condition $g(0) = g_0$. What remains, is to find $P$ such that" ] }, { "cell_type": "markdown", - "id": "b1cd6594", - "metadata": {}, + "id": "f4af75e0", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -5339,16 +4890,20 @@ }, { "cell_type": "markdown", - "id": "bf2aa493", - "metadata": {}, + "id": "4ea72b19", + "metadata": { + "editable": true + }, "source": [ "is fulfilled as *best as possible*." ] }, { "cell_type": "markdown", - "id": "5eda172b", - "metadata": {}, + "id": "428cf327", + "metadata": { + "editable": true + }, "source": [ "## More technicalities\n", "\n", @@ -5361,8 +4916,10 @@ }, { "cell_type": "markdown", - "id": "2bfc8161", - "metadata": {}, + "id": "1b4c9872", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\min_{P}\\Big\\{ \\big(g_t'(x, P) - ( -\\gamma g_t(x, P) \\big)^2 \\Big\\}\n", @@ -5371,8 +4928,10 @@ }, { "cell_type": "markdown", - "id": "9af6a125", - "metadata": {}, + "id": "f12896dd", + "metadata": { + "editable": true + }, "source": [ "(the notation $\\min_{P}\\{ f(x, P) \\}$ means that we desire to find $P$ that yields the minimum of $f(x, P)$)\n", "\n", @@ -5381,8 +4940,10 @@ }, { "cell_type": "markdown", - "id": "bea03f10", - "metadata": {}, + "id": "70e2187e", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\min_{P_{\\text{hidden} }, \\ P_{\\text{output} }}\\Big\\{ \\big(g_t'(x, \\{ P_{\\text{hidden} }, P_{\\text{output} }\\}) - ( -\\gamma g_t(x, \\{ P_{\\text{hidden} }, P_{\\text{output} }\\}) \\big)^2 \\Big\\}\n", @@ -5391,16 +4952,20 @@ }, { "cell_type": "markdown", - "id": "352fb11e", - "metadata": {}, + "id": "e0136d5f", + "metadata": { + "editable": true + }, "source": [ "for an input value $x$." ] }, { "cell_type": "markdown", - "id": "98d5219e", - "metadata": {}, + "id": "ff915c59", + "metadata": { + "editable": true + }, "source": [ "## More details\n", "\n", @@ -5409,8 +4974,10 @@ }, { "cell_type": "markdown", - "id": "83d7b299", - "metadata": {}, + "id": "ca46667e", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -5424,16 +4991,20 @@ }, { "cell_type": "markdown", - "id": "1ba53d11", - "metadata": {}, + "id": "3efa1dc6", + "metadata": { + "editable": true + }, "source": [ "Letting $\\boldsymbol{x}$ be a vector with elements $x_i$ and $C(\\boldsymbol{x}, P) = \\frac{1}{N} \\sum_i \\big(g_t'(x_i, P) - ( -\\gamma g_t(x_i, P) \\big)^2$ denote the cost function, the minimization problem that our network must solve, becomes" ] }, { "cell_type": "markdown", - "id": "3953e778", - "metadata": {}, + "id": "a3d800e8", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\min_{P} C(\\boldsymbol{x}, P)\n", @@ -5442,8 +5013,10 @@ }, { "cell_type": "markdown", - "id": "706a1978", - "metadata": {}, + "id": "8fb679f5", + "metadata": { + "editable": true + }, "source": [ "In terms of $P_{\\text{hidden} }$ and $P_{\\text{output} }$, this could also be expressed as\n", "\n", @@ -5454,8 +5027,10 @@ }, { "cell_type": "markdown", - "id": "8ff57d1a", - "metadata": {}, + "id": "08b2a19b", + "metadata": { + "editable": true + }, "source": [ "## A possible implementation of a neural network\n", "\n", @@ -5468,8 +5043,10 @@ }, { "cell_type": "markdown", - "id": "fb487d3d", - "metadata": {}, + "id": "082074c8", + "metadata": { + "editable": true + }, "source": [ "## Technicalities\n", "\n", @@ -5478,8 +5055,10 @@ }, { "cell_type": "markdown", - "id": "1d6ac6a6", - "metadata": {}, + "id": "b69f9148", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{aligned}\n", @@ -5498,8 +5077,10 @@ }, { "cell_type": "markdown", - "id": "fee86c29", - "metadata": {}, + "id": "9feeefde", + "metadata": { + "editable": true + }, "source": [ "## Final technicalities I\n", "\n", @@ -5508,8 +5089,10 @@ }, { "cell_type": "markdown", - "id": "317dd762", - "metadata": {}, + "id": "cde763a7", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{aligned}\n", @@ -5529,8 +5112,10 @@ }, { "cell_type": "markdown", - "id": "4641c716", - "metadata": {}, + "id": "3c83e5dc", + "metadata": { + "editable": true + }, "source": [ "## Final technicalities II\n", "\n", @@ -5543,8 +5128,10 @@ }, { "cell_type": "markdown", - "id": "a28fd5d3", - "metadata": {}, + "id": "4566531c", + "metadata": { + "editable": true + }, "source": [ "$$\n", "f(z) = \\frac{1}{1 + \\exp{(-z)}}\n", @@ -5553,8 +5140,10 @@ }, { "cell_type": "markdown", - "id": "d2c6ce13", - "metadata": {}, + "id": "e316d1e7", + "metadata": { + "editable": true + }, "source": [ "It is possible to use other activations functions for the hidden layer also.\n", "\n", @@ -5575,8 +5164,10 @@ }, { "cell_type": "markdown", - "id": "5bf0304c", - "metadata": {}, + "id": "41ac9a1c", + "metadata": { + "editable": true + }, "source": [ "## Final technicalities III\n", "\n", @@ -5585,8 +5176,10 @@ }, { "cell_type": "markdown", - "id": "f4ee906f", - "metadata": {}, + "id": "b0a8a3c2", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{aligned}\n", @@ -5604,8 +5197,10 @@ }, { "cell_type": "markdown", - "id": "454cd32f", - "metadata": {}, + "id": "67447b23", + "metadata": { + "editable": true + }, "source": [ "## Final technicalities IV\n", "\n", @@ -5614,8 +5209,10 @@ }, { "cell_type": "markdown", - "id": "49b52638", - "metadata": {}, + "id": "12d3e1ee", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\boldsymbol{z}_{1}^{\\text{output}} =\n", @@ -5631,16 +5228,20 @@ }, { "cell_type": "markdown", - "id": "75d1e0dc", - "metadata": {}, + "id": "562a122c", + "metadata": { + "editable": true + }, "source": [ "In this case we seek a continuous range of values since we are approximating a function. This means that after computing $\\boldsymbol{z}_{1}^{\\text{output}}$ the neural network has finished its feed forward step, and $\\boldsymbol{z}_{1}^{\\text{output}}$ is the final output of the network." ] }, { "cell_type": "markdown", - "id": "243508b9", - "metadata": {}, + "id": "78789a96", + "metadata": { + "editable": true + }, "source": [ "## Back propagation\n", "\n", @@ -5651,8 +5252,10 @@ }, { "cell_type": "markdown", - "id": "cfaa5264", - "metadata": {}, + "id": "76b734fb", + "metadata": { + "editable": true + }, "source": [ "$$\n", "C(\\boldsymbol{x}, P) = \\frac{1}{N} \\sum_i \\big(g_t'(x_i, P) - ( -\\gamma g_t(x_i, P) \\big)^2\n", @@ -5661,8 +5264,10 @@ }, { "cell_type": "markdown", - "id": "188a58e4", - "metadata": {}, + "id": "e2911f97", + "metadata": { + "editable": true + }, "source": [ "In order to minimize the cost function, an optimization method must be chosen.\n", "\n", @@ -5671,8 +5276,10 @@ }, { "cell_type": "markdown", - "id": "6f3c7251", - "metadata": {}, + "id": "5e0c281c", + "metadata": { + "editable": true + }, "source": [ "## Gradient descent\n", "\n", @@ -5686,8 +5293,10 @@ }, { "cell_type": "markdown", - "id": "b48662af", - "metadata": {}, + "id": "077e3318", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\boldsymbol{\\omega}_{\\text{new} } = \\boldsymbol{\\omega} - \\lambda \\nabla_{\\boldsymbol{\\omega}} C(\\boldsymbol{x}, \\boldsymbol{\\omega})\n", @@ -5696,8 +5305,10 @@ }, { "cell_type": "markdown", - "id": "46cc1b9b", - "metadata": {}, + "id": "b6e9387d", + "metadata": { + "editable": true + }, "source": [ "for a number of iterations or until $ \\big|\\big| \\boldsymbol{\\omega}_{\\text{new} } - \\boldsymbol{\\omega} \\big|\\big|$ becomes smaller than some given tolerance.\n", "\n", @@ -5716,8 +5327,10 @@ }, { "cell_type": "markdown", - "id": "018877f6", - "metadata": {}, + "id": "3fa21e60", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{aligned}\n", @@ -5729,8 +5342,10 @@ }, { "cell_type": "markdown", - "id": "35e348bd", - "metadata": {}, + "id": "e05e6028", + "metadata": { + "editable": true + }, "source": [ "## The code for solving the ODE" ] @@ -5738,8 +5353,11 @@ { "cell_type": "code", "execution_count": 41, - "id": "dafe8d25", - "metadata": {}, + "id": "eb77d1cf", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "import autograd.numpy as np\n", @@ -5890,8 +5508,10 @@ }, { "cell_type": "markdown", - "id": "65e4f300", - "metadata": {}, + "id": "bbd6562b", + "metadata": { + "editable": true + }, "source": [ "## The network with one input layer, specified number of hidden layers, and one output layer\n", "\n", @@ -5903,8 +5523,11 @@ { "cell_type": "code", "execution_count": 42, - "id": "5ef5f766", - "metadata": {}, + "id": "24ba1afd", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "import autograd.numpy as np\n", @@ -5919,8 +5542,8 @@ "# but with number of hidden layers specified by the user.\n", "def deep_neural_network(deep_params, x):\n", " # N_hidden is the number of hidden layers\n", - "\n", - " N_hidden = np.size(deep_params) - 1 # -1 since params consists of\n", + " # deep_params is a list, len() should be used\n", + " N_hidden = len(deep_params) - 1 # -1 since params consists of\n", " # parameters to all the hidden\n", " # layers AND the output layer.\n", "\n", @@ -6069,8 +5692,10 @@ }, { "cell_type": "markdown", - "id": "c7ad45ef", - "metadata": {}, + "id": "3974040a", + "metadata": { + "editable": true + }, "source": [ "## Example: Population growth\n", "\n", @@ -6080,8 +5705,10 @@ }, { "cell_type": "markdown", - "id": "2d1376bc", - "metadata": {}, + "id": "a9d203ea", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -6095,8 +5722,10 @@ }, { "cell_type": "markdown", - "id": "1f039785", - "metadata": {}, + "id": "ede35f92", + "metadata": { + "editable": true + }, "source": [ "where $g(t)$ is the population density at time $t$, $\\alpha > 0$ the growth rate and $A > 0$ is the maximum population number in the environment.\n", "Also, at $t = 0$ the population has the size $g(0) = g_0$, where $g_0$ is some chosen constant.\n", @@ -6109,8 +5738,10 @@ }, { "cell_type": "markdown", - "id": "8dfcfb5e", - "metadata": {}, + "id": "a9720dfc", + "metadata": { + "editable": true + }, "source": [ "## Setting up the problem\n", "\n", @@ -6120,8 +5751,10 @@ }, { "cell_type": "markdown", - "id": "3cb5e674", - "metadata": {}, + "id": "5ef6b555", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -6135,8 +5768,10 @@ }, { "cell_type": "markdown", - "id": "56581c9f", - "metadata": {}, + "id": "bd5c646e", + "metadata": { + "editable": true + }, "source": [ "where $g(0) = g_0$.\n", "\n", @@ -6145,8 +5780,10 @@ }, { "cell_type": "markdown", - "id": "ebf7032a", - "metadata": {}, + "id": "cdc5561b", + "metadata": { + "editable": true + }, "source": [ "## The trial solution\n", "\n", @@ -6170,8 +5807,10 @@ }, { "cell_type": "markdown", - "id": "a943c5c6", - "metadata": {}, + "id": "cebde164", + "metadata": { + "editable": true + }, "source": [ "## The program using Autograd\n", "\n", @@ -6181,8 +5820,11 @@ { "cell_type": "code", "execution_count": 43, - "id": "a3620769", - "metadata": {}, + "id": "c71a9592", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "import autograd.numpy as np\n", @@ -6201,9 +5843,12 @@ " g0 = 1.2\n", " return alpha, A, g0\n", "\n", - "def deep_neural_network(P, x):\n", + "def deep_neural_network(deep_params, x):\n", " # N_hidden is the number of hidden layers\n", - " N_hidden = np.size(P) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n", + " # deep_params is a list, len() should be used\n", + " N_hidden = len(deep_params) - 1 # -1 since params consists of\n", + " # parameters to all the hidden\n", + " # layers AND the output layer.\n", "\n", " # Assumes input x being an one-dimensional array\n", " num_values = np.size(x)\n", @@ -6220,7 +5865,7 @@ "\n", " for l in range(N_hidden):\n", " # From the list of parameters P; find the correct weigths and bias for this layer\n", - " w_hidden = P[l]\n", + " w_hidden = deep_params[l]\n", "\n", " # Add a row of ones to include bias\n", " x_prev = np.concatenate((np.ones((1,num_values)), x_prev ), axis = 0)\n", @@ -6234,7 +5879,7 @@ " ## Output layer:\n", "\n", " # Get the weights and bias for this layer\n", - " w_output = P[-1]\n", + " w_output = deep_params[-1]\n", "\n", " # Include bias:\n", " x_prev = np.concatenate((np.ones((1,num_values)), x_prev), axis = 0)\n", @@ -6245,6 +5890,8 @@ " return x_output\n", "\n", "\n", + "\n", + "\n", "def cost_function_deep(P, x):\n", "\n", " # Evaluate the trial function with the current parameters P\n", @@ -6352,8 +5999,10 @@ }, { "cell_type": "markdown", - "id": "9a21afc2", - "metadata": {}, + "id": "bd70d6cc", + "metadata": { + "editable": true + }, "source": [ "## Using forward Euler to solve the ODE\n", "\n", @@ -6370,8 +6019,10 @@ }, { "cell_type": "markdown", - "id": "8a8ad46c", - "metadata": {}, + "id": "ee3bdd02", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{aligned}\n", @@ -6383,8 +6034,10 @@ }, { "cell_type": "markdown", - "id": "7b83336b", - "metadata": {}, + "id": "d81b6054", + "metadata": { + "editable": true + }, "source": [ "along with the condition that $g(0) = g_0$.\n", "\n", @@ -6395,8 +6048,10 @@ }, { "cell_type": "markdown", - "id": "ea68eaac", - "metadata": {}, + "id": "0d7a2272", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{aligned}\n", @@ -6409,16 +6064,20 @@ }, { "cell_type": "markdown", - "id": "c8a2f92c", - "metadata": {}, + "id": "ca29846f", + "metadata": { + "editable": true + }, "source": [ "Now, if $g_i = g(t_i)$ then" ] }, { "cell_type": "markdown", - "id": "4f1a2187", - "metadata": {}, + "id": "1a51d849", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -6437,8 +6096,10 @@ }, { "cell_type": "markdown", - "id": "cb87fc93", - "metadata": {}, + "id": "700a9bac", + "metadata": { + "editable": true + }, "source": [ "for $i \\geq 1$ and $g_0 = g(t_0) = g(0) = g_0$.\n", "\n", @@ -6449,8 +6110,11 @@ { "cell_type": "code", "execution_count": 44, - "id": "f8c976e7", - "metadata": {}, + "id": "482cf93c", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "# Assume that all function definitions from the example program using Autograd\n", @@ -6522,8 +6186,10 @@ }, { "cell_type": "markdown", - "id": "23756e15", - "metadata": {}, + "id": "4ed3cefd", + "metadata": { + "editable": true + }, "source": [ "## Example: Solving the one dimensional Poisson equation\n", "\n", @@ -6532,8 +6198,10 @@ }, { "cell_type": "markdown", - "id": "3a68185f", - "metadata": {}, + "id": "d0bdfcdb", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -6547,8 +6215,10 @@ }, { "cell_type": "markdown", - "id": "dc784286", - "metadata": {}, + "id": "656240e7", + "metadata": { + "editable": true + }, "source": [ "where $f(x)$ is a given function for $x \\in (0,1)$.\n", "\n", @@ -6557,8 +6227,10 @@ }, { "cell_type": "markdown", - "id": "e0f7d593", - "metadata": {}, + "id": "fb310d7c", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{align*}\n", @@ -6570,8 +6242,10 @@ }, { "cell_type": "markdown", - "id": "4cacfaeb", - "metadata": {}, + "id": "e6b102dc", + "metadata": { + "editable": true + }, "source": [ "This equation can be solved numerically using programs where e.g Autograd and TensorFlow are used.\n", "The results from the networks can then be compared to the analytical solution.\n", @@ -6580,8 +6254,10 @@ }, { "cell_type": "markdown", - "id": "69ba705a", - "metadata": {}, + "id": "797d4bb8", + "metadata": { + "editable": true + }, "source": [ "## The specific equation to solve for\n", "\n", @@ -6590,8 +6266,10 @@ }, { "cell_type": "markdown", - "id": "995cfbc9", - "metadata": {}, + "id": "da2e90b8", + "metadata": { + "editable": true + }, "source": [ "$$\n", "-g''(x) = f(x),\\qquad x \\in (0,1)\n", @@ -6600,16 +6278,20 @@ }, { "cell_type": "markdown", - "id": "769f9670", - "metadata": {}, + "id": "318f8f3a", + "metadata": { + "editable": true + }, "source": [ "where $f(x)$ is a given function, along with the chosen conditions" ] }, { "cell_type": "markdown", - "id": "855cddbb", - "metadata": {}, + "id": "c5e8ac9e", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -6623,8 +6305,10 @@ }, { "cell_type": "markdown", - "id": "7006cb54", - "metadata": {}, + "id": "220ceeb4", + "metadata": { + "editable": true + }, "source": [ "In this example, we consider the case when $f(x) = (3x + x^2)\\exp(x)$.\n", "\n", @@ -6633,8 +6317,10 @@ }, { "cell_type": "markdown", - "id": "5216ca22", - "metadata": {}, + "id": "ec8b38dc", + "metadata": { + "editable": true + }, "source": [ "$$\n", "g_t(x) = x \\cdot (1-x) \\cdot N(P,x)\n", @@ -6643,16 +6329,20 @@ }, { "cell_type": "markdown", - "id": "b77670be", - "metadata": {}, + "id": "0ed927a9", + "metadata": { + "editable": true + }, "source": [ "The analytical solution for this problem is" ] }, { "cell_type": "markdown", - "id": "bf2bed23", - "metadata": {}, + "id": "3a51ddcc", + "metadata": { + "editable": true + }, "source": [ "$$\n", "g(x) = x(1 - x)\\exp(x)\n", @@ -6661,8 +6351,10 @@ }, { "cell_type": "markdown", - "id": "a22ccb78", - "metadata": {}, + "id": "d5392b8d", + "metadata": { + "editable": true + }, "source": [ "## Solving the equation using Autograd" ] @@ -6670,8 +6362,11 @@ { "cell_type": "code", "execution_count": 45, - "id": "7b3b5f5a", - "metadata": {}, + "id": "95059f71", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "import autograd.numpy as np\n", @@ -6684,7 +6379,10 @@ "\n", "def deep_neural_network(deep_params, x):\n", " # N_hidden is the number of hidden layers\n", - " N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n", + " # deep_params is a list, len() should be used\n", + " N_hidden = len(deep_params) - 1 # -1 since params consists of\n", + " # parameters to all the hidden\n", + " # layers AND the output layer.\n", "\n", " # Assumes input x being an one-dimensional array\n", " num_values = np.size(x)\n", @@ -6725,6 +6423,7 @@ "\n", " return x_output\n", "\n", + "\n", "def solve_ode_deep_neural_network(x, num_neurons, num_iter, lmb):\n", " # num_hidden_neurons is now a list of number of neurons within each hidden layer\n", "\n", @@ -6828,8 +6527,10 @@ }, { "cell_type": "markdown", - "id": "6668b7ef", - "metadata": {}, + "id": "7c568bf3", + "metadata": { + "editable": true + }, "source": [ "## Comparing with a numerical scheme\n", "\n", @@ -6848,8 +6549,10 @@ }, { "cell_type": "markdown", - "id": "ec1adda2", - "metadata": {}, + "id": "eab0bb68", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -6863,16 +6566,20 @@ }, { "cell_type": "markdown", - "id": "33486af4", - "metadata": {}, + "id": "c0ada0c0", + "metadata": { + "editable": true + }, "source": [ "If $x_i = i \\Delta x = x_{i-1} + \\Delta x$ and $g_i = g(x_i)$ for $i = 1,\\dots N_x - 2$ with $N_x$ being the number of values for $x$, ([15](#approx)) becomes" ] }, { "cell_type": "markdown", - "id": "42c3ef84", - "metadata": {}, + "id": "b483eace", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{aligned}\n", @@ -6884,16 +6591,20 @@ }, { "cell_type": "markdown", - "id": "f454dbea", - "metadata": {}, + "id": "30d45122", + "metadata": { + "editable": true + }, "source": [ "Since we know from our problem that" ] }, { "cell_type": "markdown", - "id": "7411a707", - "metadata": {}, + "id": "52c68e05", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{aligned}\n", @@ -6905,8 +6616,10 @@ }, { "cell_type": "markdown", - "id": "c47c3822", - "metadata": {}, + "id": "1657934d", + "metadata": { + "editable": true + }, "source": [ "along with the conditions $g(0) = g(1) = 0$,\n", "the following scheme can be used to find an approximate solution for $g(x)$ numerically:" @@ -6914,8 +6627,10 @@ }, { "cell_type": "markdown", - "id": "1cca824c", - "metadata": {}, + "id": "9fc46a1b", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -6932,8 +6647,10 @@ }, { "cell_type": "markdown", - "id": "82ecd5a1", - "metadata": {}, + "id": "cd79f75c", + "metadata": { + "editable": true + }, "source": [ "for $i = 1, \\dots, N_x - 2$ where $g_0 = g_{N_x - 1} = 0$ and $f(x_i) = (3x_i + x_i^2)\\exp(x_i)$, which is given for our specific problem.\n", "\n", @@ -6942,8 +6659,10 @@ }, { "cell_type": "markdown", - "id": "39f5fe07", - "metadata": {}, + "id": "501ac880", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{aligned}\n", @@ -6977,16 +6696,20 @@ }, { "cell_type": "markdown", - "id": "5ac54c35", - "metadata": {}, + "id": "8551615b", + "metadata": { + "editable": true + }, "source": [ "which makes it possible to solve for the vector $\\boldsymbol{g}$." ] }, { "cell_type": "markdown", - "id": "79ad3ebc", - "metadata": {}, + "id": "dbd6d2a1", + "metadata": { + "editable": true + }, "source": [ "## Setting up the code\n", "\n", @@ -6996,8 +6719,11 @@ { "cell_type": "code", "execution_count": 46, - "id": "be03bf2d", - "metadata": {}, + "id": "c71a3063", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "import autograd.numpy as np\n", @@ -7010,7 +6736,10 @@ "\n", "def deep_neural_network(deep_params, x):\n", " # N_hidden is the number of hidden layers\n", - " N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n", + " # deep_params is a list, len() should be used\n", + " N_hidden = len(deep_params) - 1 # -1 since params consists of\n", + " # parameters to all the hidden\n", + " # layers AND the output layer.\n", "\n", " # Assumes input x being an one-dimensional array\n", " num_values = np.size(x)\n", @@ -7051,6 +6780,7 @@ "\n", " return x_output\n", "\n", + "\n", "def solve_ode_deep_neural_network(x, num_neurons, num_iter, lmb):\n", " # num_hidden_neurons is now a list of number of neurons within each hidden layer\n", "\n", @@ -7194,8 +6924,10 @@ }, { "cell_type": "markdown", - "id": "faa0daa4", - "metadata": {}, + "id": "da58cdec", + "metadata": { + "editable": true + }, "source": [ "## Partial Differential Equations\n", "\n", @@ -7209,8 +6941,10 @@ }, { "cell_type": "markdown", - "id": "fb7b7ff6", - "metadata": {}, + "id": "df991968", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -7224,16 +6958,20 @@ }, { "cell_type": "markdown", - "id": "71da1640", - "metadata": {}, + "id": "59ef3099", + "metadata": { + "editable": true + }, "source": [ "where $f$ is an expression involving all kinds of possible mixed derivatives of $g(x_1,\\dots,x_N)$ up to an order $n$. In order for the solution to be unique, some additional conditions must also be given." ] }, { "cell_type": "markdown", - "id": "c77e6225", - "metadata": {}, + "id": "a580ae0f", + "metadata": { + "editable": true + }, "source": [ "## Type of problem\n", "\n", @@ -7245,8 +6983,10 @@ }, { "cell_type": "markdown", - "id": "98f82ea2", - "metadata": {}, + "id": "a56f3fe1", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{align*}\n", @@ -7257,8 +6997,10 @@ }, { "cell_type": "markdown", - "id": "bc33501c", - "metadata": {}, + "id": "69586714", + "metadata": { + "editable": true + }, "source": [ "where $h_1(x_1,\\dots,x_N)$ is a function that ensures $g_t(x_1,\\dots,x_N)$ satisfies some given conditions.\n", "The neural network $N(x_1,\\dots,x_N,P)$ has weights and biases described by $P$ and $h_2(x_1,\\dots,x_N,N(x_1,\\dots,x_N,P))$ is an expression using the output from the neural network in some way.\n", @@ -7268,8 +7010,10 @@ }, { "cell_type": "markdown", - "id": "818e8900", - "metadata": {}, + "id": "d8374dce", + "metadata": { + "editable": true + }, "source": [ "## Network requirements\n", "\n", @@ -7286,8 +7030,10 @@ }, { "cell_type": "markdown", - "id": "526d5428", - "metadata": {}, + "id": "f3fe961e", + "metadata": { + "editable": true + }, "source": [ "$$\n", "C\\left(x_1, \\dots, x_N, P\\right) = \\left( f\\left(x_1, \\, \\dots \\, , x_N, \\frac{\\partial g(x_1,\\dots,x_N) }{\\partial x_1}, \\dots , \\frac{\\partial g(x_1,\\dots,x_N) }{\\partial x_N}, \\frac{\\partial g(x_1,\\dots,x_N) }{\\partial x_1\\partial x_2}, \\, \\dots \\, , \\frac{\\partial^n g(x_1,\\dots,x_N) }{\\partial x_N^n} \\right) \\right)^2\n", @@ -7296,8 +7042,10 @@ }, { "cell_type": "markdown", - "id": "279b9296", - "metadata": {}, + "id": "4e453aa1", + "metadata": { + "editable": true + }, "source": [ "## More details\n", "\n", @@ -7306,8 +7054,10 @@ }, { "cell_type": "markdown", - "id": "bcb8ce40", - "metadata": {}, + "id": "e1ebf5bd", + "metadata": { + "editable": true + }, "source": [ "$$\n", "C\\left(\\boldsymbol{x}, P\\right) = f\\left( \\left( \\boldsymbol{x}, \\frac{\\partial g(\\boldsymbol{x}) }{\\partial x_1}, \\dots , \\frac{\\partial g(\\boldsymbol{x}) }{\\partial x_N}, \\frac{\\partial g(\\boldsymbol{x}) }{\\partial x_1\\partial x_2}, \\, \\dots \\, , \\frac{\\partial^n g(\\boldsymbol{x}) }{\\partial x_N^n} \\right) \\right)^2\n", @@ -7316,16 +7066,20 @@ }, { "cell_type": "markdown", - "id": "6549ea7b", - "metadata": {}, + "id": "b8c35553", + "metadata": { + "editable": true + }, "source": [ "If we also have $M$ different sets of values for $x_1, \\dots, x_N$, that is $\\boldsymbol{x}_i = \\big(x_1^{(i)}, \\dots, x_N^{(i)}\\big)$ for $i = 1,\\dots,M$ being the rows in matrix $X$, the cost function can be generalized into" ] }, { "cell_type": "markdown", - "id": "0956062e", - "metadata": {}, + "id": "22a123d5", + "metadata": { + "editable": true + }, "source": [ "$$\n", "C\\left(X, P \\right) = \\sum_{i=1}^M f\\left( \\left( \\boldsymbol{x}_i, \\frac{\\partial g(\\boldsymbol{x}_i) }{\\partial x_1}, \\dots , \\frac{\\partial g(\\boldsymbol{x}_i) }{\\partial x_N}, \\frac{\\partial g(\\boldsymbol{x}_i) }{\\partial x_1\\partial x_2}, \\, \\dots \\, , \\frac{\\partial^n g(\\boldsymbol{x}_i) }{\\partial x_N^n} \\right) \\right)^2.\n", @@ -7334,8 +7088,10 @@ }, { "cell_type": "markdown", - "id": "8fca0166", - "metadata": {}, + "id": "b15eedf5", + "metadata": { + "editable": true + }, "source": [ "## Example: The diffusion equation\n", "\n", @@ -7344,8 +7100,10 @@ }, { "cell_type": "markdown", - "id": "58ad6f25", - "metadata": {}, + "id": "1a92cc8c", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\frac{\\partial g(x,t)}{\\partial t} = \\frac{\\partial^2 g(x,t)}{\\partial x^2}\n", @@ -7354,16 +7112,20 @@ }, { "cell_type": "markdown", - "id": "e6e5728e", - "metadata": {}, + "id": "e4862313", + "metadata": { + "editable": true + }, "source": [ "where a possible choice of conditions are" ] }, { "cell_type": "markdown", - "id": "6921faa1", - "metadata": {}, + "id": "5c186d68", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{align*}\n", @@ -7376,16 +7138,20 @@ }, { "cell_type": "markdown", - "id": "80e632ef", - "metadata": {}, + "id": "57d4f000", + "metadata": { + "editable": true + }, "source": [ "with $u(x)$ being some given function." ] }, { "cell_type": "markdown", - "id": "a2711421", - "metadata": {}, + "id": "7c2cdc6b", + "metadata": { + "editable": true + }, "source": [ "## Defining the problem\n", "\n", @@ -7394,8 +7160,10 @@ }, { "cell_type": "markdown", - "id": "981da4af", - "metadata": {}, + "id": "bde064ea", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -7409,16 +7177,20 @@ }, { "cell_type": "markdown", - "id": "c1b8c16d", - "metadata": {}, + "id": "7a558c97", + "metadata": { + "editable": true + }, "source": [ "and" ] }, { "cell_type": "markdown", - "id": "b8ed6509", - "metadata": {}, + "id": "534a4844", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{align*}\n", @@ -7431,8 +7203,10 @@ }, { "cell_type": "markdown", - "id": "200050ac", - "metadata": {}, + "id": "a07e0730", + "metadata": { + "editable": true + }, "source": [ "with $u(x) = \\sin(\\pi x)$.\n", "\n", @@ -7443,8 +7217,10 @@ }, { "cell_type": "markdown", - "id": "5dca99d2", - "metadata": {}, + "id": "74b72286", + "metadata": { + "editable": true + }, "source": [ "## Setting up the network using Autograd\n", "\n", @@ -7460,8 +7236,11 @@ { "cell_type": "code", "execution_count": 47, - "id": "85f4b843", - "metadata": {}, + "id": "4a0c57e9", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "def sigmoid(z):\n", @@ -7475,7 +7254,7 @@ " num_points = np.size(x,1)\n", "\n", " # N_hidden is the number of hidden layers\n", - " N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n", + " N_hidden = len(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n", "\n", " # Assume that the input layer does nothing to the input x\n", " x_input = x\n", @@ -7512,8 +7291,10 @@ }, { "cell_type": "markdown", - "id": "9dd68ff8", - "metadata": {}, + "id": "350b1d3d", + "metadata": { + "editable": true + }, "source": [ "## Setting up the network using Autograd; The trial solution\n", "\n", @@ -7540,8 +7321,10 @@ }, { "cell_type": "markdown", - "id": "65f68c5a", - "metadata": {}, + "id": "9fd438b5", + "metadata": { + "editable": true + }, "source": [ "## Why the jacobian?\n", "\n", @@ -7567,8 +7350,11 @@ { "cell_type": "code", "execution_count": 48, - "id": "91612f4f", - "metadata": {}, + "id": "d1c81690", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "# Set up the trial function:\n", @@ -7611,8 +7397,10 @@ }, { "cell_type": "markdown", - "id": "ff9df4ba", - "metadata": {}, + "id": "ca22d9ce", + "metadata": { + "editable": true + }, "source": [ "## Setting up the network using Autograd; The full program\n", "\n", @@ -7635,8 +7423,11 @@ { "cell_type": "code", "execution_count": 49, - "id": "e6451b9e", - "metadata": {}, + "id": "5a9146bc", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "import autograd.numpy as np\n", @@ -7659,7 +7450,7 @@ " num_points = np.size(x,1)\n", "\n", " # N_hidden is the number of hidden layers\n", - " N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n", + " N_hidden = len(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n", "\n", " # Assume that the input layer does nothing to the input x\n", " x_input = x\n", @@ -7802,7 +7593,7 @@ " T,X = np.meshgrid(t,x)\n", "\n", " fig = plt.figure(figsize=(10,10))\n", - " ax = fig.gca(projection='3d')\n", + " ax = fig.add_suplot(projection='3d')\n", " ax.set_title('Solution from the deep neural network w/ %d layer'%len(num_hidden_neurons))\n", " s = ax.plot_surface(T,X,g_dnn_ag,linewidth=0,antialiased=False,cmap=cm.viridis)\n", " ax.set_xlabel('Time $t$')\n", @@ -7810,14 +7601,14 @@ "\n", "\n", " fig = plt.figure(figsize=(10,10))\n", - " ax = fig.gca(projection='3d')\n", + " ax = fig.add_suplot(projection='3d')\n", " ax.set_title('Analytical solution')\n", " s = ax.plot_surface(T,X,G_analytical,linewidth=0,antialiased=False,cmap=cm.viridis)\n", " ax.set_xlabel('Time $t$')\n", " ax.set_ylabel('Position $x$');\n", "\n", " fig = plt.figure(figsize=(10,10))\n", - " ax = fig.gca(projection='3d')\n", + " ax = fig.add_suplot(projection='3d')\n", " ax.set_title('Difference')\n", " s = ax.plot_surface(T,X,diff_ag,linewidth=0,antialiased=False,cmap=cm.viridis)\n", " ax.set_xlabel('Time $t$')\n", @@ -7866,8 +7657,10 @@ }, { "cell_type": "markdown", - "id": "7f867cf2", - "metadata": {}, + "id": "e59d2116", + "metadata": { + "editable": true + }, "source": [ "## Example: Solving the wave equation with Neural Networks\n", "\n", @@ -7876,8 +7669,10 @@ }, { "cell_type": "markdown", - "id": "ded83be1", - "metadata": {}, + "id": "e2ffdbaf", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\frac{\\partial^2 g(x,t)}{\\partial t^2} = c^2\\frac{\\partial^2 g(x,t)}{\\partial x^2}\n", @@ -7886,8 +7681,10 @@ }, { "cell_type": "markdown", - "id": "f0aba0c9", - "metadata": {}, + "id": "37b2cc03", + "metadata": { + "editable": true + }, "source": [ "with $c$ being the specified wave speed.\n", "\n", @@ -7896,8 +7693,10 @@ }, { "cell_type": "markdown", - "id": "28a48fe7", - "metadata": {}, + "id": "524df5d3", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{align*}\n", @@ -7911,16 +7710,20 @@ }, { "cell_type": "markdown", - "id": "61ded78d", - "metadata": {}, + "id": "d175b25f", + "metadata": { + "editable": true + }, "source": [ "where $\\frac{\\partial g(x,t)}{\\partial t} \\Big |_{t = 0}$ means the derivative of $g(x,t)$ with respect to $t$ is evaluated at $t = 0$, and $u(x)$ and $v(x)$ being given functions." ] }, { "cell_type": "markdown", - "id": "990695ea", - "metadata": {}, + "id": "11b3289e", + "metadata": { + "editable": true + }, "source": [ "## The problem to solve for\n", "\n", @@ -7929,8 +7732,10 @@ }, { "cell_type": "markdown", - "id": "58841089", - "metadata": {}, + "id": "e469d65e", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -7944,8 +7749,10 @@ }, { "cell_type": "markdown", - "id": "303eb900", - "metadata": {}, + "id": "5f773fa8", + "metadata": { + "editable": true + }, "source": [ "where $c$ is the given wave speed.\n", "The chosen conditions for this equation are" @@ -7953,8 +7760,10 @@ }, { "cell_type": "markdown", - "id": "ce5c5167", - "metadata": {}, + "id": "2952f7b4", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -7971,16 +7780,20 @@ }, { "cell_type": "markdown", - "id": "609a7556", - "metadata": {}, + "id": "99057b48", + "metadata": { + "editable": true + }, "source": [ "In this example, let $c = 1$ and $u(x) = \\sin(\\pi x)$ and $v(x) = -\\pi\\sin(\\pi x)$." ] }, { "cell_type": "markdown", - "id": "5edb8797", - "metadata": {}, + "id": "7debdfea", + "metadata": { + "editable": true + }, "source": [ "## The trial solution\n", "Setting up the network is done in similar matter as for the example of solving the diffusion equation.\n", @@ -8003,8 +7816,10 @@ }, { "cell_type": "markdown", - "id": "c6be7518", - "metadata": {}, + "id": "2d5d5c44", + "metadata": { + "editable": true + }, "source": [ "## The analytical solution\n", "\n", @@ -8017,8 +7832,10 @@ }, { "cell_type": "markdown", - "id": "368bb540", - "metadata": {}, + "id": "f97806ce", + "metadata": { + "editable": true + }, "source": [ "## Solving the wave equation - the full program using Autograd" ] @@ -8026,8 +7843,11 @@ { "cell_type": "code", "execution_count": 50, - "id": "65085f11", - "metadata": {}, + "id": "9337b7c7", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "import autograd.numpy as np\n", @@ -8084,7 +7904,7 @@ " num_points = np.size(x,1)\n", "\n", " # N_hidden is the number of hidden layers\n", - " N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n", + " N_hidden = len(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n", "\n", " # Assume that the input layer does nothing to the input x\n", " x_input = x\n", @@ -8189,7 +8009,7 @@ " T,X = np.meshgrid(t,x)\n", "\n", " fig = plt.figure(figsize=(10,10))\n", - " ax = fig.gca(projection='3d')\n", + " ax = fig.add_suplot(projection='3d')\n", " ax.set_title('Solution from the deep neural network w/ %d layer'%len(num_hidden_neurons))\n", " s = ax.plot_surface(T,X,res,linewidth=0,antialiased=False,cmap=cm.viridis)\n", " ax.set_xlabel('Time $t$')\n", @@ -8197,7 +8017,7 @@ "\n", "\n", " fig = plt.figure(figsize=(10,10))\n", - " ax = fig.gca(projection='3d')\n", + " ax = fig.add_suplot(projection='3d')\n", " ax.set_title('Analytical solution')\n", " s = ax.plot_surface(T,X,res_analytical,linewidth=0,antialiased=False,cmap=cm.viridis)\n", " ax.set_xlabel('Time $t$')\n", @@ -8205,7 +8025,7 @@ "\n", "\n", " fig = plt.figure(figsize=(10,10))\n", - " ax = fig.gca(projection='3d')\n", + " ax = fig.add_suplot(projection='3d')\n", " ax.set_title('Difference')\n", " s = ax.plot_surface(T,X,diff,linewidth=0,antialiased=False,cmap=cm.viridis)\n", " ax.set_xlabel('Time $t$')\n", @@ -8254,8 +8074,10 @@ }, { "cell_type": "markdown", - "id": "41011c90", - "metadata": {}, + "id": "6071d006", + "metadata": { + "editable": true + }, "source": [ "## Resources on differential equations and deep learning\n", "\n", @@ -8270,11 +8092,6 @@ } ], "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, "language_info": { "codemirror_mode": { "name": "ipython", diff --git a/doc/LectureNotes/_build/jupyter_execute/week43.py b/doc/LectureNotes/_build/jupyter_execute/week43.py index f1ed0ece8..7e3c621a5 100644 --- a/doc/LectureNotes/_build/jupyter_execute/week43.py +++ b/doc/LectureNotes/_build/jupyter_execute/week43.py @@ -1366,7 +1366,7 @@ def create_neural_network_keras(n_neurons_layer1, n_neurons_layer2, n_categories model.add(Dense(n_neurons_layer2, activation='sigmoid', kernel_regularizer=regularizers.l2(lmbd))) model.add(Dense(n_categories, activation='softmax')) - sgd = optimizers.SGD(lr=eta) + sgd = optimizers.SGD(learning_rate=eta) model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy']) return model @@ -1542,7 +1542,7 @@ def NN_model(inputsize,n_layers,n_neuron,eta,lamda): else: #Subsequent layers are capable of automatic shape inferencing model.add(Dense(n_neuron,activation='relu',kernel_regularizer=regularizers.l2(lamda))) model.add(Dense(2,activation='softmax')) #2 outputs - ordered and disordered (softmax for prob) - sgd=optimizers.SGD(lr=eta) + sgd=optimizers.SGD(learning_rate=eta) model.compile(loss='categorical_crossentropy',optimizer=sgd,metrics=['accuracy']) return model @@ -2623,7 +2623,7 @@ scores = logistic_regression.fit(X, yXOR, scheduler, epochs=1000) # Not bad, but the results depend strongly on the learning reate. Try different learning rates. -# ## Solving ODEs with Deep Learning +# ## Solving differential equations with Deep Learning # # The Universal Approximation Theorem states that a neural network can # approximate any function at a single hidden layer along with one input @@ -2642,7 +2642,7 @@ scores = logistic_regression.fit(X, yXOR, scheduler, epochs=1000) # The lectures on differential equations were developed by Kristine Baluka Hein, now PhD student at IFI. # A great thanks to Kristine. -# ## Ordinary Differential Equations +# ## Ordinary Differential Equations first # # An ordinary differential equation (ODE) is an equation involving functions having one variable. # @@ -3206,8 +3206,8 @@ def sigmoid(z): # but with number of hidden layers specified by the user. def deep_neural_network(deep_params, x): # N_hidden is the number of hidden layers - - N_hidden = np.size(deep_params) - 1 # -1 since params consists of + # deep_params is a list, len() should be used + N_hidden = len(deep_params) - 1 # -1 since params consists of # parameters to all the hidden # layers AND the output layer. @@ -3436,9 +3436,12 @@ def get_parameters(): g0 = 1.2 return alpha, A, g0 -def deep_neural_network(P, x): +def deep_neural_network(deep_params, x): # N_hidden is the number of hidden layers - N_hidden = np.size(P) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer + # deep_params is a list, len() should be used + N_hidden = len(deep_params) - 1 # -1 since params consists of + # parameters to all the hidden + # layers AND the output layer. # Assumes input x being an one-dimensional array num_values = np.size(x) @@ -3455,7 +3458,7 @@ def deep_neural_network(P, x): for l in range(N_hidden): # From the list of parameters P; find the correct weigths and bias for this layer - w_hidden = P[l] + w_hidden = deep_params[l] # Add a row of ones to include bias x_prev = np.concatenate((np.ones((1,num_values)), x_prev ), axis = 0) @@ -3469,7 +3472,7 @@ def deep_neural_network(P, x): ## Output layer: # Get the weights and bias for this layer - w_output = P[-1] + w_output = deep_params[-1] # Include bias: x_prev = np.concatenate((np.ones((1,num_values)), x_prev), axis = 0) @@ -3480,6 +3483,8 @@ def deep_neural_network(P, x): return x_output + + def cost_function_deep(P, x): # Evaluate the trial function with the current parameters P @@ -3785,7 +3790,10 @@ def sigmoid(z): def deep_neural_network(deep_params, x): # N_hidden is the number of hidden layers - N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer + # deep_params is a list, len() should be used + N_hidden = len(deep_params) - 1 # -1 since params consists of + # parameters to all the hidden + # layers AND the output layer. # Assumes input x being an one-dimensional array num_values = np.size(x) @@ -3826,6 +3834,7 @@ def deep_neural_network(deep_params, x): return x_output + def solve_ode_deep_neural_network(x, num_neurons, num_iter, lmb): # num_hidden_neurons is now a list of number of neurons within each hidden layer @@ -4035,7 +4044,10 @@ def sigmoid(z): def deep_neural_network(deep_params, x): # N_hidden is the number of hidden layers - N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer + # deep_params is a list, len() should be used + N_hidden = len(deep_params) - 1 # -1 since params consists of + # parameters to all the hidden + # layers AND the output layer. # Assumes input x being an one-dimensional array num_values = np.size(x) @@ -4076,6 +4088,7 @@ def deep_neural_network(deep_params, x): return x_output + def solve_ode_deep_neural_network(x, num_neurons, num_iter, lmb): # num_hidden_neurons is now a list of number of neurons within each hidden layer @@ -4358,7 +4371,7 @@ def deep_neural_network(deep_params, x): num_points = np.size(x,1) # N_hidden is the number of hidden layers - N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer + N_hidden = len(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer # Assume that the input layer does nothing to the input x x_input = x @@ -4516,7 +4529,7 @@ def deep_neural_network(deep_params, x): num_points = np.size(x,1) # N_hidden is the number of hidden layers - N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer + N_hidden = len(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer # Assume that the input layer does nothing to the input x x_input = x @@ -4659,7 +4672,7 @@ if __name__ == '__main__': T,X = np.meshgrid(t,x) fig = plt.figure(figsize=(10,10)) - ax = fig.gca(projection='3d') + ax = fig.add_suplot(projection='3d') ax.set_title('Solution from the deep neural network w/ %d layer'%len(num_hidden_neurons)) s = ax.plot_surface(T,X,g_dnn_ag,linewidth=0,antialiased=False,cmap=cm.viridis) ax.set_xlabel('Time $t$') @@ -4667,14 +4680,14 @@ if __name__ == '__main__': fig = plt.figure(figsize=(10,10)) - ax = fig.gca(projection='3d') + ax = fig.add_suplot(projection='3d') ax.set_title('Analytical solution') s = ax.plot_surface(T,X,G_analytical,linewidth=0,antialiased=False,cmap=cm.viridis) ax.set_xlabel('Time $t$') ax.set_ylabel('Position $x$'); fig = plt.figure(figsize=(10,10)) - ax = fig.gca(projection='3d') + ax = fig.add_suplot(projection='3d') ax.set_title('Difference') s = ax.plot_surface(T,X,diff_ag,linewidth=0,antialiased=False,cmap=cm.viridis) ax.set_xlabel('Time $t$') @@ -4859,7 +4872,7 @@ def deep_neural_network(deep_params, x): num_points = np.size(x,1) # N_hidden is the number of hidden layers - N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer + N_hidden = len(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer # Assume that the input layer does nothing to the input x x_input = x @@ -4964,7 +4977,7 @@ if __name__ == '__main__': T,X = np.meshgrid(t,x) fig = plt.figure(figsize=(10,10)) - ax = fig.gca(projection='3d') + ax = fig.add_suplot(projection='3d') ax.set_title('Solution from the deep neural network w/ %d layer'%len(num_hidden_neurons)) s = ax.plot_surface(T,X,res,linewidth=0,antialiased=False,cmap=cm.viridis) ax.set_xlabel('Time $t$') @@ -4972,7 +4985,7 @@ if __name__ == '__main__': fig = plt.figure(figsize=(10,10)) - ax = fig.gca(projection='3d') + ax = fig.add_suplot(projection='3d') ax.set_title('Analytical solution') s = ax.plot_surface(T,X,res_analytical,linewidth=0,antialiased=False,cmap=cm.viridis) ax.set_xlabel('Time $t$') @@ -4980,7 +4993,7 @@ if __name__ == '__main__': fig = plt.figure(figsize=(10,10)) - ax = fig.gca(projection='3d') + ax = fig.add_suplot(projection='3d') ax.set_title('Difference') s = ax.plot_surface(T,X,diff,linewidth=0,antialiased=False,cmap=cm.viridis) ax.set_xlabel('Time $t$') diff --git a/doc/LectureNotes/week43.ipynb b/doc/LectureNotes/week43.ipynb index 47032b0fe..202263be5 100644 --- a/doc/LectureNotes/week43.ipynb +++ b/doc/LectureNotes/week43.ipynb @@ -2,8 +2,10 @@ "cells": [ { "cell_type": "markdown", - "id": "cda62a86", - "metadata": {}, + "id": "677a195d", + "metadata": { + "editable": true + }, "source": [ "\n", @@ -12,8 +14,10 @@ }, { "cell_type": "markdown", - "id": "bdae1fa5", - "metadata": {}, + "id": "e943e8f0", + "metadata": { + "editable": true + }, "source": [ "# Week 43: Deep Learning: Constructing a Neural Network code and solving differential equations\n", "**Morten Hjorth-Jensen**, Department of Physics, University of Oslo and Department of Physics and Astronomy and Facility for Rare Isotope Beams, Michigan State University\n", @@ -23,8 +27,10 @@ }, { "cell_type": "markdown", - "id": "e2c09dd5", - "metadata": {}, + "id": "22cdced0", + "metadata": { + "editable": true + }, "source": [ "## Plans for week 43\n", "\n", @@ -39,8 +45,10 @@ }, { "cell_type": "markdown", - "id": "7de8bfe0", - "metadata": {}, + "id": "a57ef6df", + "metadata": { + "editable": true + }, "source": [ "## Exercises and lab session week 43\n", "**Lab sessions on Tuesday and Wednesday.**\n", @@ -54,8 +62,10 @@ }, { "cell_type": "markdown", - "id": "2664d901", - "metadata": {}, + "id": "c4a134ee", + "metadata": { + "editable": true + }, "source": [ "## Mathematics of deep learning\n", "\n", @@ -68,8 +78,10 @@ }, { "cell_type": "markdown", - "id": "46bb5558", - "metadata": {}, + "id": "dd73045e", + "metadata": { + "editable": true + }, "source": [ "## Reminder on books with hands-on material and codes\n", "* Sebastian Rashcka et al, Machine learning with Scikit-Learn and PyTorch at " @@ -77,8 +89,10 @@ }, { "cell_type": "markdown", - "id": "1c6ad86d", - "metadata": {}, + "id": "372f4aa4", + "metadata": { + "editable": true + }, "source": [ "## Reading recommendations\n", "\n", @@ -89,8 +103,10 @@ }, { "cell_type": "markdown", - "id": "9fbf4898", - "metadata": {}, + "id": "05efbc6f", + "metadata": { + "editable": true + }, "source": [ "## Using Automatic differentiation\n", "\n", @@ -100,8 +116,10 @@ }, { "cell_type": "markdown", - "id": "135a7122", - "metadata": {}, + "id": "d04b0948", + "metadata": { + "editable": true + }, "source": [ "## Back propagation and automatic differentiation\n", "\n", @@ -115,16 +133,20 @@ }, { "cell_type": "markdown", - "id": "45236eaf", - "metadata": {}, + "id": "b9d41f5c", + "metadata": { + "editable": true + }, "source": [ "## Lecture Monday October 21" ] }, { "cell_type": "markdown", - "id": "8997a10d", - "metadata": {}, + "id": "68606991", + "metadata": { + "editable": true + }, "source": [ "## Setting up the back propagation algorithm and algorithm for a feed forward NN, initalizations\n", "This is a reminder from where we ended last week.\n", @@ -148,8 +170,10 @@ }, { "cell_type": "markdown", - "id": "df88cb72", - "metadata": {}, + "id": "a564a394", + "metadata": { + "editable": true + }, "source": [ "## Setting up the back propagation algorithm, part 1\n", "\n", @@ -169,8 +193,10 @@ }, { "cell_type": "markdown", - "id": "46d7ebde", - "metadata": {}, + "id": "43ab4381", + "metadata": { + "editable": true + }, "source": [ "## Setting up the back propagation algorithm, part 2\n", "\n", @@ -179,8 +205,10 @@ }, { "cell_type": "markdown", - "id": "3ad237f5", - "metadata": {}, + "id": "f578cee4", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\delta_j^L = \\sigma'(z_j^L)\\frac{\\partial {\\cal C}}{\\partial (a_j^L)}.\n", @@ -189,16 +217,20 @@ }, { "cell_type": "markdown", - "id": "f6906530", - "metadata": {}, + "id": "fbc05943", + "metadata": { + "editable": true + }, "source": [ "Then we compute the back propagate error for each $l=L-1,L-2,\\dots,1$ as" ] }, { "cell_type": "markdown", - "id": "e9613a49", - "metadata": {}, + "id": "6ad3c28f", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\delta_j^l = \\sum_k \\delta_k^{l+1}w_{kj}^{l+1}\\sigma'(z_j^l).\n", @@ -207,8 +239,10 @@ }, { "cell_type": "markdown", - "id": "dba468b9", - "metadata": {}, + "id": "135513b5", + "metadata": { + "editable": true + }, "source": [ "## Setting up the Back propagation algorithm, part 3\n", "\n", @@ -219,8 +253,10 @@ }, { "cell_type": "markdown", - "id": "21c684a1", - "metadata": {}, + "id": "f643ecdf", + "metadata": { + "editable": true + }, "source": [ "$$\n", "w_{ij}^l\\leftarrow = w_{ij}^l- \\eta \\delta_j^la_i^{l-1},\n", @@ -229,8 +265,10 @@ }, { "cell_type": "markdown", - "id": "6acf04a3", - "metadata": {}, + "id": "077ef1c2", + "metadata": { + "editable": true + }, "source": [ "$$\n", "b_j^l \\leftarrow b_j^l-\\eta \\frac{\\partial {\\cal C}}{\\partial b_j^l}=b_j^l-\\eta \\delta_j^l,\n", @@ -239,16 +277,20 @@ }, { "cell_type": "markdown", - "id": "b0558fbe", - "metadata": {}, + "id": "50cf7472", + "metadata": { + "editable": true + }, "source": [ "with $\\eta$ being the learning rate." ] }, { "cell_type": "markdown", - "id": "6b424064", - "metadata": {}, + "id": "a10f406c", + "metadata": { + "editable": true + }, "source": [ "## Updating the gradients\n", "\n", @@ -257,8 +299,10 @@ }, { "cell_type": "markdown", - "id": "7bc86e31", - "metadata": {}, + "id": "626cba8f", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\delta_j^l = \\sum_k \\delta_k^{l+1}w_{kj}^{l+1}\\sigma'(z_j^l),\n", @@ -267,16 +311,20 @@ }, { "cell_type": "markdown", - "id": "f7a10087", - "metadata": {}, + "id": "2ebe05ad", + "metadata": { + "editable": true + }, "source": [ "we update the weights and the biases using gradient descent for each $l=L-1,L-2,\\dots,1$ and update the weights and biases according to the rules" ] }, { "cell_type": "markdown", - "id": "2bf12837", - "metadata": {}, + "id": "bfbbac35", + "metadata": { + "editable": true + }, "source": [ "$$\n", "w_{ij}^l\\leftarrow = w_{ij}^l- \\eta \\delta_j^la_i^{l-1},\n", @@ -285,8 +333,10 @@ }, { "cell_type": "markdown", - "id": "ad07731e", - "metadata": {}, + "id": "ac515399", + "metadata": { + "editable": true + }, "source": [ "$$\n", "b_j^l \\leftarrow b_j^l-\\eta \\frac{\\partial {\\cal C}}{\\partial b_j^l}=b_j^l-\\eta \\delta_j^l,\n", @@ -295,8 +345,10 @@ }, { "cell_type": "markdown", - "id": "30c6ae90", - "metadata": {}, + "id": "4703cc51", + "metadata": { + "editable": true + }, "source": [ "## Activation functions\n", "\n", @@ -316,8 +368,10 @@ }, { "cell_type": "markdown", - "id": "5dfa6d1c", - "metadata": {}, + "id": "cab6ad03", + "metadata": { + "editable": true + }, "source": [ "### Activation functions, examples\n", "\n", @@ -326,8 +380,10 @@ }, { "cell_type": "markdown", - "id": "e8d0be63", - "metadata": {}, + "id": "ad6042eb", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\sigma(x) = \\frac{1}{1 + e^{-x}},\n", @@ -336,16 +392,20 @@ }, { "cell_type": "markdown", - "id": "b3ec0efa", - "metadata": {}, + "id": "1e96b7e7", + "metadata": { + "editable": true + }, "source": [ "and the *hyperbolic tangent* function" ] }, { "cell_type": "markdown", - "id": "78c46915", - "metadata": {}, + "id": "c600792f", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\sigma(x) = \\tanh(x)\n", @@ -354,8 +414,10 @@ }, { "cell_type": "markdown", - "id": "87e6973b", - "metadata": {}, + "id": "76d1da8e", + "metadata": { + "editable": true + }, "source": [ "## The RELU function family\n", "\n", @@ -373,8 +435,10 @@ }, { "cell_type": "markdown", - "id": "9c990bc8", - "metadata": {}, + "id": "8e7e0e0c", + "metadata": { + "editable": true + }, "source": [ "## ELU function\n", "\n", @@ -385,8 +449,10 @@ }, { "cell_type": "markdown", - "id": "9b7760f9", - "metadata": {}, + "id": "4b1f77d6", + "metadata": { + "editable": true + }, "source": [ "$$\n", "ELU(z) = \\left\\{\\begin{array}{cc} \\alpha\\left( \\exp{(z)}-1\\right) & z < 0,\\\\ z & z \\ge 0.\\end{array}\\right.\n", @@ -395,8 +461,10 @@ }, { "cell_type": "markdown", - "id": "47927a82", - "metadata": {}, + "id": "10826ba9", + "metadata": { + "editable": true + }, "source": [ "## Which activation function should we use?\n", "\n", @@ -415,8 +483,10 @@ }, { "cell_type": "markdown", - "id": "52554330", - "metadata": {}, + "id": "79d307b6", + "metadata": { + "editable": true + }, "source": [ "## More on activation functions, output layers\n", "\n", @@ -435,8 +505,10 @@ }, { "cell_type": "markdown", - "id": "4b7d3292", - "metadata": {}, + "id": "429b8d4e", + "metadata": { + "editable": true + }, "source": [ "## Setting up a Multi-layer perceptron model for classification\n", "\n", @@ -461,8 +533,10 @@ }, { "cell_type": "markdown", - "id": "38715b52", - "metadata": {}, + "id": "90d9f195", + "metadata": { + "editable": true + }, "source": [ "$$\n", "P(y = 0 \\mid \\boldsymbol{x}, \\boldsymbol{\\theta}) = \\frac{1}{1 + \\exp{(- \\boldsymbol{x}})} ,\n", @@ -471,16 +545,20 @@ }, { "cell_type": "markdown", - "id": "dcc0cbfa", - "metadata": {}, + "id": "73d2763e", + "metadata": { + "editable": true + }, "source": [ "and" ] }, { "cell_type": "markdown", - "id": "c686ecf6", - "metadata": {}, + "id": "e2a70a54", + "metadata": { + "editable": true + }, "source": [ "$$\n", "P(y = 1 \\mid \\boldsymbol{x}, \\boldsymbol{\\theta}) = 1 - P(y = 0 \\mid \\boldsymbol{x}, \\boldsymbol{\\theta}) ,\n", @@ -489,8 +567,10 @@ }, { "cell_type": "markdown", - "id": "79a8b207", - "metadata": {}, + "id": "7ed6c0dc", + "metadata": { + "editable": true + }, "source": [ "where $y \\in \\{0, 1\\}$ and $\\boldsymbol{\\theta}$ represents the weights and biases\n", "of our network." @@ -498,8 +578,10 @@ }, { "cell_type": "markdown", - "id": "3218b074", - "metadata": {}, + "id": "a42826b9", + "metadata": { + "editable": true + }, "source": [ "## Defining the cost function\n", "\n", @@ -508,8 +590,10 @@ }, { "cell_type": "markdown", - "id": "c0e50336", - "metadata": {}, + "id": "d8e401c5", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\mathcal{C}(\\boldsymbol{\\theta}) = - \\ln P(\\mathcal{D} \\mid \\boldsymbol{\\theta}) = - \\sum_{i=1}^n\n", @@ -519,8 +603,10 @@ }, { "cell_type": "markdown", - "id": "79138403", - "metadata": {}, + "id": "17808105", + "metadata": { + "editable": true + }, "source": [ "This last equality means that we can interpret our *cost* function as a sum over the *loss* function\n", "for each point in the dataset $\\mathcal{L}_i(\\boldsymbol{\\theta})$. \n", @@ -542,8 +628,10 @@ }, { "cell_type": "markdown", - "id": "1b3db35e", - "metadata": {}, + "id": "5a9b90ce", + "metadata": { + "editable": true + }, "source": [ "$$\n", "P(y_{ic} = 1 \\mid \\boldsymbol{x}_i, \\boldsymbol{\\theta}) = \\frac{\\exp{((\\boldsymbol{a}_i^{hidden})^T \\boldsymbol{w}_c)}}\n", @@ -553,8 +641,10 @@ }, { "cell_type": "markdown", - "id": "90626f76", - "metadata": {}, + "id": "57f33029", + "metadata": { + "editable": true + }, "source": [ "which reduces to the logistic function in the binary case. \n", "The likelihood of this $C$-class classifier\n", @@ -563,8 +653,10 @@ }, { "cell_type": "markdown", - "id": "596eb444", - "metadata": {}, + "id": "422eb23f", + "metadata": { + "editable": true + }, "source": [ "$$\n", "P(\\mathcal{D} \\mid \\boldsymbol{\\theta}) = \\prod_{i=1}^n \\prod_{c=0}^{C-1} [P(y_{ic} = 1)]^{y_{ic}} .\n", @@ -573,16 +665,20 @@ }, { "cell_type": "markdown", - "id": "5529d979", - "metadata": {}, + "id": "0eb634c3", + "metadata": { + "editable": true + }, "source": [ "Again we take the negative log-likelihood to define our cost function:" ] }, { "cell_type": "markdown", - "id": "f7d2523f", - "metadata": {}, + "id": "b3b208fc", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\mathcal{C}(\\boldsymbol{\\theta}) = - \\log{P(\\mathcal{D} \\mid \\boldsymbol{\\theta})}.\n", @@ -591,8 +687,10 @@ }, { "cell_type": "markdown", - "id": "7d8d61f1", - "metadata": {}, + "id": "3788aba1", + "metadata": { + "editable": true + }, "source": [ "See the logistic regression lectures for a full definition of the cost function.\n", "\n", @@ -601,8 +699,10 @@ }, { "cell_type": "markdown", - "id": "9751fb82", - "metadata": {}, + "id": "4d38ceec", + "metadata": { + "editable": true + }, "source": [ "## Example: binary classification problem\n", "\n", @@ -611,8 +711,10 @@ }, { "cell_type": "markdown", - "id": "3e6587c7", - "metadata": {}, + "id": "cee503b9", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\mathcal{C}(\\boldsymbol{\\beta}) = - \\sum_{i=1}^n \\left(y_i\\log{p(y_i \\vert x_i,\\boldsymbol{\\beta})}+(1-y_i)\\log{1-p(y_i \\vert x_i,\\boldsymbol{\\beta})}\\right),\n", @@ -621,16 +723,20 @@ }, { "cell_type": "markdown", - "id": "39bb5ba4", - "metadata": {}, + "id": "19b29e02", + "metadata": { + "editable": true + }, "source": [ "where we had defined the logistic (sigmoid) function" ] }, { "cell_type": "markdown", - "id": "fcb1f3d9", - "metadata": {}, + "id": "19507159", + "metadata": { + "editable": true + }, "source": [ "$$\n", "p(y_i =1\\vert x_i,\\boldsymbol{\\beta})=\\frac{\\exp{(\\beta_0+\\beta_1 x_i)}}{1+\\exp{(\\beta_0+\\beta_1 x_i)}},\n", @@ -639,16 +745,20 @@ }, { "cell_type": "markdown", - "id": "20fc130a", - "metadata": {}, + "id": "a15191e4", + "metadata": { + "editable": true + }, "source": [ "and" ] }, { "cell_type": "markdown", - "id": "66c7c0e9", - "metadata": {}, + "id": "c9df7a22", + "metadata": { + "editable": true + }, "source": [ "$$\n", "p(y_i =0\\vert x_i,\\boldsymbol{\\beta})=1-p(y_i =1\\vert x_i,\\boldsymbol{\\beta}).\n", @@ -657,8 +767,10 @@ }, { "cell_type": "markdown", - "id": "e757c738", - "metadata": {}, + "id": "0cae75ec", + "metadata": { + "editable": true + }, "source": [ "The parameters $\\boldsymbol{\\beta}$ were defined using a minimization method like gradient descent or Newton-Raphson's method. \n", "\n", @@ -668,8 +780,10 @@ }, { "cell_type": "markdown", - "id": "d048e79e", - "metadata": {}, + "id": "36b0506b", + "metadata": { + "editable": true + }, "source": [ "$$\n", "a_i^l = y_i = \\frac{\\exp{(z_i^l)}}{1+\\exp{(z_i^l)}},\n", @@ -678,16 +792,20 @@ }, { "cell_type": "markdown", - "id": "b97827e7", - "metadata": {}, + "id": "0c729c48", + "metadata": { + "editable": true + }, "source": [ "with" ] }, { "cell_type": "markdown", - "id": "c11528eb", - "metadata": {}, + "id": "060813af", + "metadata": { + "editable": true + }, "source": [ "$$\n", "z_i^l = \\sum_{j}w_{ij}^l a_j^{l-1}+b_i^l,\n", @@ -696,8 +814,10 @@ }, { "cell_type": "markdown", - "id": "94839c96", - "metadata": {}, + "id": "f2a5059b", + "metadata": { + "editable": true + }, "source": [ "where the superscript $l-1$ indicates that these are the outputs from layer $l-1$.\n", "Our cost function at the final layer $l=L$ is now" @@ -705,8 +825,10 @@ }, { "cell_type": "markdown", - "id": "694596c9", - "metadata": {}, + "id": "6763074d", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\mathcal{C}(\\boldsymbol{W}) = - \\sum_{i=1}^n \\left(t_i\\log{a_i^L}+(1-t_i)\\log{(1-a_i^L)}\\right),\n", @@ -715,16 +837,20 @@ }, { "cell_type": "markdown", - "id": "b4c8232e", - "metadata": {}, + "id": "f498b6b5", + "metadata": { + "editable": true + }, "source": [ "where we have defined the targets $t_i$. The derivatives of the cost function with respect to the output $a_i^L$ are then easily calculated and we get" ] }, { "cell_type": "markdown", - "id": "9d611a08", - "metadata": {}, + "id": "fedce928", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\frac{\\partial \\mathcal{C}(\\boldsymbol{W})}{\\partial a_i^L} = \\frac{a_i^L-t_i}{a_i^L(1-a_i^L)}.\n", @@ -733,16 +859,20 @@ }, { "cell_type": "markdown", - "id": "cb7847ef", - "metadata": {}, + "id": "429c11ae", + "metadata": { + "editable": true + }, "source": [ "In case we use another activation function than the logistic one, we need to evaluate other derivatives." ] }, { "cell_type": "markdown", - "id": "cd5be179", - "metadata": {}, + "id": "ddde8370", + "metadata": { + "editable": true + }, "source": [ "## The Softmax function\n", "In case we employ the more general case given by the Softmax equation, we need to evaluate the derivative of the activation function with respect to the activation $z_i^l$, that is we need" @@ -750,8 +880,10 @@ }, { "cell_type": "markdown", - "id": "96b7fe79", - "metadata": {}, + "id": "637bd194", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\frac{\\partial f(z_i^l)}{\\partial w_{jk}^l} =\n", @@ -761,16 +893,20 @@ }, { "cell_type": "markdown", - "id": "1040c590", - "metadata": {}, + "id": "0792109c", + "metadata": { + "editable": true + }, "source": [ "For the Softmax function we have" ] }, { "cell_type": "markdown", - "id": "3162bd5e", - "metadata": {}, + "id": "10f14691", + "metadata": { + "editable": true + }, "source": [ "$$\n", "f(z_i^l) = \\frac{\\exp{(z_i^l)}}{\\sum_{m=1}^K\\exp{(z_m^l)}}.\n", @@ -779,16 +915,20 @@ }, { "cell_type": "markdown", - "id": "b7f2b7b8", - "metadata": {}, + "id": "186529b5", + "metadata": { + "editable": true + }, "source": [ "Its derivative with respect to $z_j^l$ gives" ] }, { "cell_type": "markdown", - "id": "735fd9c0", - "metadata": {}, + "id": "68afc0da", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\frac{\\partial f(z_i^l)}{\\partial z_j^l}= f(z_i^l)\\left(\\delta_{ij}-f(z_j^l)\\right),\n", @@ -797,16 +937,20 @@ }, { "cell_type": "markdown", - "id": "c3cbbb04", - "metadata": {}, + "id": "cf48ea60", + "metadata": { + "editable": true + }, "source": [ "which in case of the simply binary model reduces to having $i=j$." ] }, { "cell_type": "markdown", - "id": "6462edbb", - "metadata": {}, + "id": "861a3bbe", + "metadata": { + "editable": true + }, "source": [ "## Developing a code for doing neural networks with back propagation\n", "\n", @@ -827,8 +971,10 @@ }, { "cell_type": "markdown", - "id": "7915dc1d", - "metadata": {}, + "id": "c8007ce7", + "metadata": { + "editable": true + }, "source": [ "## Collect and pre-process data\n", "\n", @@ -875,8 +1021,11 @@ { "cell_type": "code", "execution_count": 1, - "id": "eb736d79", - "metadata": {}, + "id": "29678f97", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "%matplotlib inline\n", @@ -927,8 +1076,10 @@ }, { "cell_type": "markdown", - "id": "01fcb50b", - "metadata": {}, + "id": "3ba6d7d6", + "metadata": { + "editable": true + }, "source": [ "## Train and test datasets\n", "\n", @@ -946,8 +1097,11 @@ { "cell_type": "code", "execution_count": 2, - "id": "e6b7f2e3", - "metadata": {}, + "id": "7df51dd1", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "from sklearn.model_selection import train_test_split\n", @@ -981,8 +1135,10 @@ }, { "cell_type": "markdown", - "id": "1ca032f8", - "metadata": {}, + "id": "249898fd", + "metadata": { + "editable": true + }, "source": [ "## Define model and architecture\n", "\n", @@ -1023,8 +1179,10 @@ }, { "cell_type": "markdown", - "id": "6e4e1e18", - "metadata": {}, + "id": "2d30ddd3", + "metadata": { + "editable": true + }, "source": [ "## Layers\n", "\n", @@ -1061,8 +1219,10 @@ }, { "cell_type": "markdown", - "id": "ab0ee6d8", - "metadata": {}, + "id": "ce9f14e4", + "metadata": { + "editable": true + }, "source": [ "## Weights and biases\n", "\n", @@ -1080,8 +1240,11 @@ { "cell_type": "code", "execution_count": 3, - "id": "15ad1599", - "metadata": {}, + "id": "c3f75b32", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "# building our neural network\n", @@ -1103,8 +1266,10 @@ }, { "cell_type": "markdown", - "id": "3efd576a", - "metadata": {}, + "id": "4ebf4f68", + "metadata": { + "editable": true + }, "source": [ "## Feed-forward pass\n", "\n", @@ -1129,8 +1294,10 @@ }, { "cell_type": "markdown", - "id": "715c5d46", - "metadata": {}, + "id": "c6760d1f", + "metadata": { + "editable": true + }, "source": [ "## Matrix multiplications\n", "\n", @@ -1164,8 +1331,11 @@ { "cell_type": "code", "execution_count": 4, - "id": "ecaa833e", - "metadata": {}, + "id": "98f3a5d5", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "# setup the feed-forward pass, subscript h = hidden layer\n", @@ -1207,8 +1377,10 @@ }, { "cell_type": "markdown", - "id": "c0319e65", - "metadata": {}, + "id": "19cd49ec", + "metadata": { + "editable": true + }, "source": [ "## Choose cost function and optimizer\n", "\n", @@ -1236,8 +1408,10 @@ }, { "cell_type": "markdown", - "id": "98c6696e", - "metadata": {}, + "id": "b0342f54", + "metadata": { + "editable": true + }, "source": [ "## Optimizing the cost function\n", "\n", @@ -1272,8 +1446,10 @@ }, { "cell_type": "markdown", - "id": "b5d1145a", - "metadata": {}, + "id": "7219731b", + "metadata": { + "editable": true + }, "source": [ "## Regularization\n", "\n", @@ -1304,8 +1480,10 @@ }, { "cell_type": "markdown", - "id": "b73247fd", - "metadata": {}, + "id": "248590b9", + "metadata": { + "editable": true + }, "source": [ "## Matrix multiplication\n", "\n", @@ -1343,8 +1521,11 @@ { "cell_type": "code", "execution_count": 5, - "id": "3d61438f", - "metadata": {}, + "id": "1810a36c", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "# to categorical turns our integer vector into a onehot representation\n", @@ -1419,8 +1600,10 @@ }, { "cell_type": "markdown", - "id": "6a20a495", - "metadata": {}, + "id": "e094a404", + "metadata": { + "editable": true + }, "source": [ "## Improving performance\n", "\n", @@ -1438,8 +1621,10 @@ }, { "cell_type": "markdown", - "id": "717b63a1", - "metadata": {}, + "id": "1fb3aab5", + "metadata": { + "editable": true + }, "source": [ "## Full object-oriented implementation\n", "\n", @@ -1450,8 +1635,11 @@ { "cell_type": "code", "execution_count": 6, - "id": "9d6c9929", - "metadata": {}, + "id": "20b3c187", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "class NeuralNetwork:\n", @@ -1557,8 +1745,10 @@ }, { "cell_type": "markdown", - "id": "e70667c5", - "metadata": {}, + "id": "f85564ae", + "metadata": { + "editable": true + }, "source": [ "## Evaluate model performance on test data\n", "\n", @@ -1574,8 +1764,11 @@ { "cell_type": "code", "execution_count": 7, - "id": "083dfe41", - "metadata": {}, + "id": "9be7082e", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "epochs = 100\n", @@ -1598,8 +1791,10 @@ }, { "cell_type": "markdown", - "id": "bffeadd3", - "metadata": {}, + "id": "255b5b3d", + "metadata": { + "editable": true + }, "source": [ "## Adjust hyperparameters\n", "\n", @@ -1610,8 +1805,11 @@ { "cell_type": "code", "execution_count": 8, - "id": "6513ac74", - "metadata": {}, + "id": "fc014642", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "eta_vals = np.logspace(-5, 1, 7)\n", @@ -1638,8 +1836,10 @@ }, { "cell_type": "markdown", - "id": "a6e305f6", - "metadata": {}, + "id": "96089af8", + "metadata": { + "editable": true + }, "source": [ "## Visualization" ] @@ -1647,8 +1847,11 @@ { "cell_type": "code", "execution_count": 9, - "id": "d920e285", - "metadata": {}, + "id": "04c4d29a", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "# visual representation of grid search\n", @@ -1688,8 +1891,10 @@ }, { "cell_type": "markdown", - "id": "0f1aaadd", - "metadata": {}, + "id": "472a5060", + "metadata": { + "editable": true + }, "source": [ "## scikit-learn implementation\n", "\n", @@ -1709,8 +1914,11 @@ { "cell_type": "code", "execution_count": 10, - "id": "156764f3", - "metadata": {}, + "id": "42e40777", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "from sklearn.neural_network import MLPClassifier\n", @@ -1733,8 +1941,10 @@ }, { "cell_type": "markdown", - "id": "2e5fec48", - "metadata": {}, + "id": "57fc5d93", + "metadata": { + "editable": true + }, "source": [ "## Visualization" ] @@ -1742,8 +1952,11 @@ { "cell_type": "code", "execution_count": 11, - "id": "ace47e70", - "metadata": {}, + "id": "90477fe7", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "# optional\n", @@ -1784,8 +1997,10 @@ }, { "cell_type": "markdown", - "id": "9d327f9b", - "metadata": {}, + "id": "29af8aff", + "metadata": { + "editable": true + }, "source": [ "## Building neural networks in Tensorflow and Keras\n", "\n", @@ -1800,8 +2015,10 @@ }, { "cell_type": "markdown", - "id": "2280cbce", - "metadata": {}, + "id": "a70cf999", + "metadata": { + "editable": true + }, "source": [ "## Tensorflow\n", "\n", @@ -1833,8 +2050,11 @@ { "cell_type": "code", "execution_count": 12, - "id": "dff81198", - "metadata": {}, + "id": "812c98fe", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "pip3 install tensorflow" @@ -1842,8 +2062,10 @@ }, { "cell_type": "markdown", - "id": "e9bf9cac", - "metadata": {}, + "id": "694c0bd7", + "metadata": { + "editable": true + }, "source": [ "and/or if you use **anaconda**, just write (or install from the graphical user interface)\n", "(current release of CPU-only TensorFlow)" @@ -1852,8 +2074,11 @@ { "cell_type": "code", "execution_count": 13, - "id": "1ccb8dba", - "metadata": {}, + "id": "1ecd18cc", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "conda create -n tf tensorflow\n", @@ -1862,8 +2087,10 @@ }, { "cell_type": "markdown", - "id": "efc3f4e2", - "metadata": {}, + "id": "584ab0ab", + "metadata": { + "editable": true + }, "source": [ "To install the current release of GPU TensorFlow" ] @@ -1871,8 +2098,11 @@ { "cell_type": "code", "execution_count": 14, - "id": "01556b01", - "metadata": {}, + "id": "a811e23d", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "conda create -n tf-gpu tensorflow-gpu\n", @@ -1881,8 +2111,10 @@ }, { "cell_type": "markdown", - "id": "b900f015", - "metadata": {}, + "id": "ab99008f", + "metadata": { + "editable": true + }, "source": [ "## Using Keras\n", "\n", @@ -1894,8 +2126,11 @@ { "cell_type": "code", "execution_count": 15, - "id": "3fd1f12f", - "metadata": {}, + "id": "fe547415", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "conda install keras" @@ -1903,8 +2138,10 @@ }, { "cell_type": "markdown", - "id": "20ec35c5", - "metadata": {}, + "id": "ed1e0fc6", + "metadata": { + "editable": true + }, "source": [ "You can look up the [instructions here](https://keras.io/) for more information.\n", "\n", @@ -1913,8 +2150,10 @@ }, { "cell_type": "markdown", - "id": "249e56f7", - "metadata": {}, + "id": "6aebcb51", + "metadata": { + "editable": true + }, "source": [ "## Collect and pre-process data\n", "\n", @@ -1924,8 +2163,11 @@ { "cell_type": "code", "execution_count": 16, - "id": "0ff66efc", - "metadata": {}, + "id": "a4a7d68d", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "# import necessary packages\n", @@ -1976,8 +2218,11 @@ { "cell_type": "code", "execution_count": 17, - "id": "2806c62b", - "metadata": {}, + "id": "21335418", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "from tensorflow.keras.layers import Input\n", @@ -2002,8 +2247,11 @@ { "cell_type": "code", "execution_count": 18, - "id": "3bf4c3f6", - "metadata": {}, + "id": "e81d58de", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "\n", @@ -2020,7 +2268,7 @@ " model.add(Dense(n_neurons_layer2, activation='sigmoid', kernel_regularizer=regularizers.l2(lmbd)))\n", " model.add(Dense(n_categories, activation='softmax'))\n", " \n", - " sgd = optimizers.SGD(lr=eta)\n", + " sgd = optimizers.SGD(learning_rate=eta)\n", " model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])\n", " \n", " return model" @@ -2029,8 +2277,11 @@ { "cell_type": "code", "execution_count": 19, - "id": "81a46485", - "metadata": {}, + "id": "b91287cc", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "DNN_keras = np.zeros((len(eta_vals), len(lmbd_vals)), dtype=object)\n", @@ -2053,8 +2304,11 @@ { "cell_type": "code", "execution_count": 20, - "id": "6e619c14", - "metadata": {}, + "id": "d108674e", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "# optional\n", @@ -2092,8 +2346,10 @@ }, { "cell_type": "markdown", - "id": "c303d97b", - "metadata": {}, + "id": "1198d799", + "metadata": { + "editable": true + }, "source": [ "## The Breast Cancer Data, now with Keras" ] @@ -2101,8 +2357,11 @@ { "cell_type": "code", "execution_count": 21, - "id": "03a57bfd", - "metadata": {}, + "id": "eb1eb4fb", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "\n", @@ -2215,7 +2474,7 @@ " else: #Subsequent layers are capable of automatic shape inferencing\n", " model.add(Dense(n_neuron,activation='relu',kernel_regularizer=regularizers.l2(lamda)))\n", " model.add(Dense(2,activation='softmax')) #2 outputs - ordered and disordered (softmax for prob)\n", - " sgd=optimizers.SGD(lr=eta)\n", + " sgd=optimizers.SGD(learning_rate=eta)\n", " model.compile(loss='categorical_crossentropy',optimizer=sgd,metrics=['accuracy'])\n", " return model\n", "\n", @@ -2275,8 +2534,10 @@ }, { "cell_type": "markdown", - "id": "f9ad1085", - "metadata": {}, + "id": "92834502", + "metadata": { + "editable": true + }, "source": [ "## Building a neural network code\n", "\n", @@ -2292,8 +2553,10 @@ }, { "cell_type": "markdown", - "id": "cf4f3c14", - "metadata": {}, + "id": "deb24cc1", + "metadata": { + "editable": true + }, "source": [ "### Learning rate methods\n", "\n", @@ -2312,8 +2575,11 @@ { "cell_type": "code", "execution_count": 22, - "id": "467ea7a2", - "metadata": {}, + "id": "d3619281", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "import autograd.numpy as np\n", @@ -2450,8 +2716,10 @@ }, { "cell_type": "markdown", - "id": "6af64d5a", - "metadata": {}, + "id": "f37fd7bc", + "metadata": { + "editable": true + }, "source": [ "### Usage of the above learning rate schedulers\n", "\n", @@ -2464,8 +2732,11 @@ { "cell_type": "code", "execution_count": 23, - "id": "290e0d00", - "metadata": {}, + "id": "5db34b9b", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "momentum_scheduler = Momentum(eta=1e-3, momentum=0.9)\n", @@ -2474,8 +2745,10 @@ }, { "cell_type": "markdown", - "id": "d1f43cc5", - "metadata": {}, + "id": "8b25a24d", + "metadata": { + "editable": true + }, "source": [ "Here is a small example for how a segment of code using schedulers\n", "could look. Switching out the schedulers is simple." @@ -2484,8 +2757,11 @@ { "cell_type": "code", "execution_count": 24, - "id": "0def5cec", - "metadata": {}, + "id": "2d5f2887", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "weights = np.ones((3,3))\n", @@ -2503,8 +2779,10 @@ }, { "cell_type": "markdown", - "id": "9b27dcd7", - "metadata": {}, + "id": "9070c2d3", + "metadata": { + "editable": true + }, "source": [ "### Cost functions\n", "\n", @@ -2517,8 +2795,11 @@ { "cell_type": "code", "execution_count": 25, - "id": "4217839c", - "metadata": {}, + "id": "c360c736", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "import autograd.numpy as np\n", @@ -2552,8 +2833,10 @@ }, { "cell_type": "markdown", - "id": "ef5f9fe9", - "metadata": {}, + "id": "42c7ec61", + "metadata": { + "editable": true + }, "source": [ "Below we give a short example of how these cost function may be used\n", "to obtain results if you wish to test them out on your own using\n", @@ -2563,8 +2846,11 @@ { "cell_type": "code", "execution_count": 26, - "id": "305e1479", - "metadata": {}, + "id": "cd9cb3a5", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "from autograd import grad\n", @@ -2581,8 +2867,10 @@ }, { "cell_type": "markdown", - "id": "1499b193", - "metadata": {}, + "id": "796aab4a", + "metadata": { + "editable": true + }, "source": [ "### Activation functions\n", "\n", @@ -2595,8 +2883,11 @@ { "cell_type": "code", "execution_count": 27, - "id": "e7b59b25", - "metadata": {}, + "id": "5ea4891d", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "import autograd.numpy as np\n", @@ -2650,8 +2941,10 @@ }, { "cell_type": "markdown", - "id": "57599619", - "metadata": {}, + "id": "2ff3c57c", + "metadata": { + "editable": true + }, "source": [ "Below follows a short demonstration of how to use an activation\n", "function. The derivative of the activation function will be important\n", @@ -2663,8 +2956,11 @@ { "cell_type": "code", "execution_count": 28, - "id": "5ee68acd", - "metadata": {}, + "id": "34289210", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "z = np.array([[4, 5, 6]]).T\n", @@ -2681,8 +2977,10 @@ }, { "cell_type": "markdown", - "id": "2d3295d9", - "metadata": {}, + "id": "0839f98b", + "metadata": { + "editable": true + }, "source": [ "### The Neural Network\n", "\n", @@ -2703,8 +3001,11 @@ { "cell_type": "code", "execution_count": 29, - "id": "c4cfc50a", - "metadata": {}, + "id": "0fdc2707", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "import math\n", @@ -3172,8 +3473,10 @@ }, { "cell_type": "markdown", - "id": "2e6e81c1", - "metadata": {}, + "id": "17bc192e", + "metadata": { + "editable": true + }, "source": [ "Before we make a model, we will quickly generate a dataset we can use\n", "for our linear regression problem as shown below" @@ -3182,8 +3485,11 @@ { "cell_type": "code", "execution_count": 30, - "id": "0d4f1ff8", - "metadata": {}, + "id": "f3f5d088", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "import autograd.numpy as np\n", @@ -3223,8 +3529,10 @@ }, { "cell_type": "markdown", - "id": "a8f1940f", - "metadata": {}, + "id": "53c3b4c2", + "metadata": { + "editable": true + }, "source": [ "Now that we have our dataset ready for the regression, we can create\n", "our regressor. Note that with the seed parameter, we can make sure our\n", @@ -3237,8 +3545,11 @@ { "cell_type": "code", "execution_count": 31, - "id": "ff6bd29b", - "metadata": {}, + "id": "890d682f", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "input_nodes = X_train.shape[1]\n", @@ -3249,8 +3560,10 @@ }, { "cell_type": "markdown", - "id": "a7b906df", - "metadata": {}, + "id": "dc4da852", + "metadata": { + "editable": true + }, "source": [ "We then fit our model with our training data using the scheduler of our choice." ] @@ -3258,8 +3571,11 @@ { "cell_type": "code", "execution_count": 32, - "id": "db7a5bf2", - "metadata": {}, + "id": "1e98a3f8", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "linear_regression.reset_weights() # reset weights such that previous runs or reruns don't affect the weights\n", @@ -3270,8 +3586,10 @@ }, { "cell_type": "markdown", - "id": "508c351b", - "metadata": {}, + "id": "68c5d793", + "metadata": { + "editable": true + }, "source": [ "Due to the progress bar we can see the MSE (train_error) throughout\n", "the FFNN's training. Note that the fit() function has some optional\n", @@ -3284,8 +3602,11 @@ { "cell_type": "code", "execution_count": 33, - "id": "6b0562b2", - "metadata": {}, + "id": "e735086e", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "linear_regression.reset_weights() # reset weights such that previous runs or reruns don't affect the weights\n", @@ -3295,8 +3616,10 @@ }, { "cell_type": "markdown", - "id": "f03172d9", - "metadata": {}, + "id": "360468f6", + "metadata": { + "editable": true + }, "source": [ "We see that given more epochs to train on, the regressor reaches a lower MSE.\n", "\n", @@ -3308,8 +3631,11 @@ { "cell_type": "code", "execution_count": 34, - "id": "26d1d1c3", - "metadata": {}, + "id": "c5937c59", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "from sklearn.datasets import load_breast_cancer\n", @@ -3331,8 +3657,11 @@ { "cell_type": "code", "execution_count": 35, - "id": "6a78c633", - "metadata": {}, + "id": "2e06b929", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "input_nodes = X_train.shape[1]\n", @@ -3343,8 +3672,10 @@ }, { "cell_type": "markdown", - "id": "5feaced2", - "metadata": {}, + "id": "94140fb1", + "metadata": { + "editable": true + }, "source": [ "We will now make use of our validation data by passing it into our fit function as a keyword argument" ] @@ -3352,8 +3683,11 @@ { "cell_type": "code", "execution_count": 36, - "id": "6a9b1538", - "metadata": {}, + "id": "da82b266", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "logistic_regression.reset_weights() # reset weights such that previous runs or reruns don't affect the weights\n", @@ -3364,8 +3698,10 @@ }, { "cell_type": "markdown", - "id": "90c7f8f3", - "metadata": {}, + "id": "19e8d4c2", + "metadata": { + "editable": true + }, "source": [ "Finally, we will create a neural network with 2 hidden layers with activation functions." ] @@ -3373,8 +3709,11 @@ { "cell_type": "code", "execution_count": 37, - "id": "58954140", - "metadata": {}, + "id": "4744accd", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "input_nodes = X_train.shape[1]\n", @@ -3390,8 +3729,11 @@ { "cell_type": "code", "execution_count": 38, - "id": "89536ca9", - "metadata": {}, + "id": "f5b5b198", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "neural_network.reset_weights() # reset weights such that previous runs or reruns don't affect the weights\n", @@ -3402,8 +3744,10 @@ }, { "cell_type": "markdown", - "id": "803f4791", - "metadata": {}, + "id": "1f5fc0a0", + "metadata": { + "editable": true + }, "source": [ "### Multiclass classification\n", "\n", @@ -3415,8 +3759,11 @@ { "cell_type": "code", "execution_count": 39, - "id": "50405951", - "metadata": {}, + "id": "4308d82f", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "from sklearn.datasets import load_digits\n", @@ -3449,8 +3796,10 @@ }, { "cell_type": "markdown", - "id": "d906edf4", - "metadata": {}, + "id": "96fc3508", + "metadata": { + "editable": true + }, "source": [ "## Testing the XOR gate and other gates\n", "\n", @@ -3460,8 +3809,11 @@ { "cell_type": "code", "execution_count": 40, - "id": "414ee306", - "metadata": {}, + "id": "5aa9a2a4", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "X = np.array([ [0, 0], [0, 1], [1, 0],[1, 1]],dtype=np.float64)\n", @@ -3480,18 +3832,22 @@ }, { "cell_type": "markdown", - "id": "982d1747", - "metadata": {}, + "id": "d57346ae", + "metadata": { + "editable": true + }, "source": [ "Not bad, but the results depend strongly on the learning reate. Try different learning rates." ] }, { "cell_type": "markdown", - "id": "616eae3e", - "metadata": {}, + "id": "0219b252", + "metadata": { + "editable": true + }, "source": [ - "## Solving ODEs with Deep Learning\n", + "## Solving differential equations with Deep Learning\n", "\n", "The Universal Approximation Theorem states that a neural network can\n", "approximate any function at a single hidden layer along with one input\n", @@ -3513,10 +3869,12 @@ }, { "cell_type": "markdown", - "id": "43fb869c", - "metadata": {}, + "id": "80032cfc", + "metadata": { + "editable": true + }, "source": [ - "## Ordinary Differential Equations\n", + "## Ordinary Differential Equations first\n", "\n", "An ordinary differential equation (ODE) is an equation involving functions having one variable.\n", "\n", @@ -3525,8 +3883,10 @@ }, { "cell_type": "markdown", - "id": "4855aef1", - "metadata": {}, + "id": "3c5546df", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -3540,8 +3900,10 @@ }, { "cell_type": "markdown", - "id": "d31c26fc", - "metadata": {}, + "id": "e4f540a5", + "metadata": { + "editable": true + }, "source": [ "where $g(x)$ is the function to find, and $g^{(n)}(x)$ is the $n$-th derivative of $g(x)$.\n", "\n", @@ -3554,8 +3916,10 @@ }, { "cell_type": "markdown", - "id": "08d629ec", - "metadata": {}, + "id": "75ec2cfe", + "metadata": { + "editable": true + }, "source": [ "## The trial solution\n", "\n", @@ -3564,8 +3928,10 @@ }, { "cell_type": "markdown", - "id": "b2cfd7bd", - "metadata": {}, + "id": "a71e7eb8", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -3580,8 +3946,10 @@ }, { "cell_type": "markdown", - "id": "f4cd106f", - "metadata": {}, + "id": "9b06d2bf", + "metadata": { + "editable": true + }, "source": [ "where $h_1(x)$ is a function that makes $g_t(x)$ satisfy a given set\n", "of conditions, $N(x,P)$ a neural network with weights and biases\n", @@ -3599,8 +3967,10 @@ }, { "cell_type": "markdown", - "id": "74da655f", - "metadata": {}, + "id": "0aea2f3e", + "metadata": { + "editable": true + }, "source": [ "## Minimization process\n", "\n", @@ -3614,8 +3984,10 @@ }, { "cell_type": "markdown", - "id": "efe0ec18", - "metadata": {}, + "id": "a21c6a43", + "metadata": { + "editable": true + }, "source": [ "$$\n", "C\\left(x, P\\right) = \\big(f\\left(x, \\, g(x), \\, g'(x), \\, g''(x), \\, \\dots \\, , \\, g^{(n)}(x)\\right)\\big)^2\n", @@ -3624,8 +3996,10 @@ }, { "cell_type": "markdown", - "id": "f9c1643a", - "metadata": {}, + "id": "ff939582", + "metadata": { + "editable": true + }, "source": [ "If $N$ inputs are given as a vector $\\boldsymbol{x}$ with elements $x_i$ for $i = 1,\\dots,N$,\n", "the cost function becomes" @@ -3633,8 +4007,10 @@ }, { "cell_type": "markdown", - "id": "9b900e26", - "metadata": {}, + "id": "9f63085a", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -3648,8 +4024,10 @@ }, { "cell_type": "markdown", - "id": "82e8e341", - "metadata": {}, + "id": "2875658a", + "metadata": { + "editable": true + }, "source": [ "The neural net should then find the parameters $P$ that minimizes the cost function in\n", "([3](#cost)) for a set of $N$ training samples $x_i$." @@ -3657,8 +4035,10 @@ }, { "cell_type": "markdown", - "id": "b89f2f97", - "metadata": {}, + "id": "8a63e4c6", + "metadata": { + "editable": true + }, "source": [ "## Minimizing the cost function using gradient descent and automatic differentiation\n", "\n", @@ -3671,8 +4051,10 @@ }, { "cell_type": "markdown", - "id": "cc38c37f", - "metadata": {}, + "id": "2d5b4c88", + "metadata": { + "editable": true + }, "source": [ "## Example: Exponential decay\n", "\n", @@ -3681,8 +4063,10 @@ }, { "cell_type": "markdown", - "id": "3c5f0410", - "metadata": {}, + "id": "daab1d5e", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -3696,8 +4080,10 @@ }, { "cell_type": "markdown", - "id": "4531d4a4", - "metadata": {}, + "id": "fc2cedc3", + "metadata": { + "editable": true + }, "source": [ "with $g(0) = g_0$ for some chosen initial value $g_0$.\n", "\n", @@ -3706,8 +4092,10 @@ }, { "cell_type": "markdown", - "id": "d580caea", - "metadata": {}, + "id": "da284bc1", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -3722,16 +4110,20 @@ }, { "cell_type": "markdown", - "id": "6f384ec5", - "metadata": {}, + "id": "04c2531a", + "metadata": { + "editable": true + }, "source": [ "Having an analytical solution at hand, it is possible to use it to compare how well a neural network finds a solution of ([4](#solve_expdec))." ] }, { "cell_type": "markdown", - "id": "e650af49", - "metadata": {}, + "id": "62a110db", + "metadata": { + "editable": true + }, "source": [ "## The function to solve for\n", "\n", @@ -3740,8 +4132,10 @@ }, { "cell_type": "markdown", - "id": "68e3a73b", - "metadata": {}, + "id": "7b5b722c", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -3755,8 +4149,10 @@ }, { "cell_type": "markdown", - "id": "61f7dce9", - "metadata": {}, + "id": "02446b0e", + "metadata": { + "editable": true + }, "source": [ "where $g(0) = g_0$ with $\\gamma$ and $g_0$ being some chosen values.\n", "\n", @@ -3765,8 +4161,10 @@ }, { "cell_type": "markdown", - "id": "47e985ed", - "metadata": {}, + "id": "205c8e89", + "metadata": { + "editable": true + }, "source": [ "## The trial solution\n", "To begin with, a trial solution $g_t(t)$ must be chosen. A general trial solution for ordinary differential equations could be" @@ -3774,8 +4172,10 @@ }, { "cell_type": "markdown", - "id": "b5787755", - "metadata": {}, + "id": "9e46aee1", + "metadata": { + "editable": true + }, "source": [ "$$\n", "g_t(x, P) = h_1(x) + h_2(x, N(x, P))\n", @@ -3784,16 +4184,20 @@ }, { "cell_type": "markdown", - "id": "efd0d663", - "metadata": {}, + "id": "cb0d1f30", + "metadata": { + "editable": true + }, "source": [ "with $h_1(x)$ ensuring that $g_t(x)$ satisfies some conditions and $h_2(x,N(x, P))$ an expression involving $x$ and the output from the neural network $N(x,P)$ with $P $ being the collection of the weights and biases for each layer. For now, it is assumed that the network consists of one input layer, one hidden layer, and one output layer." ] }, { "cell_type": "markdown", - "id": "de4cfa12", - "metadata": {}, + "id": "8485bdd3", + "metadata": { + "editable": true + }, "source": [ "## Setup of Network\n", "\n", @@ -3810,8 +4214,10 @@ }, { "cell_type": "markdown", - "id": "4de8eeb1", - "metadata": {}, + "id": "730e057b", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -3825,8 +4231,10 @@ }, { "cell_type": "markdown", - "id": "c3076a5d", - "metadata": {}, + "id": "db8907ef", + "metadata": { + "editable": true + }, "source": [ "## Reformulating the problem\n", "\n", @@ -3842,8 +4250,10 @@ }, { "cell_type": "markdown", - "id": "00459420", - "metadata": {}, + "id": "ce4a7022", + "metadata": { + "editable": true + }, "source": [ "$$\n", "g_t(x, P) = g_0 + x \\cdot N(x, P)\n", @@ -3852,16 +4262,20 @@ }, { "cell_type": "markdown", - "id": "493318e4", - "metadata": {}, + "id": "b17ab00d", + "metadata": { + "editable": true + }, "source": [ "has been chosen such that it already solves the condition $g(0) = g_0$. What remains, is to find $P$ such that" ] }, { "cell_type": "markdown", - "id": "b1cd6594", - "metadata": {}, + "id": "f4af75e0", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -3875,16 +4289,20 @@ }, { "cell_type": "markdown", - "id": "bf2aa493", - "metadata": {}, + "id": "4ea72b19", + "metadata": { + "editable": true + }, "source": [ "is fulfilled as *best as possible*." ] }, { "cell_type": "markdown", - "id": "5eda172b", - "metadata": {}, + "id": "428cf327", + "metadata": { + "editable": true + }, "source": [ "## More technicalities\n", "\n", @@ -3897,8 +4315,10 @@ }, { "cell_type": "markdown", - "id": "2bfc8161", - "metadata": {}, + "id": "1b4c9872", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\min_{P}\\Big\\{ \\big(g_t'(x, P) - ( -\\gamma g_t(x, P) \\big)^2 \\Big\\}\n", @@ -3907,8 +4327,10 @@ }, { "cell_type": "markdown", - "id": "9af6a125", - "metadata": {}, + "id": "f12896dd", + "metadata": { + "editable": true + }, "source": [ "(the notation $\\min_{P}\\{ f(x, P) \\}$ means that we desire to find $P$ that yields the minimum of $f(x, P)$)\n", "\n", @@ -3917,8 +4339,10 @@ }, { "cell_type": "markdown", - "id": "bea03f10", - "metadata": {}, + "id": "70e2187e", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\min_{P_{\\text{hidden} }, \\ P_{\\text{output} }}\\Big\\{ \\big(g_t'(x, \\{ P_{\\text{hidden} }, P_{\\text{output} }\\}) - ( -\\gamma g_t(x, \\{ P_{\\text{hidden} }, P_{\\text{output} }\\}) \\big)^2 \\Big\\}\n", @@ -3927,16 +4351,20 @@ }, { "cell_type": "markdown", - "id": "352fb11e", - "metadata": {}, + "id": "e0136d5f", + "metadata": { + "editable": true + }, "source": [ "for an input value $x$." ] }, { "cell_type": "markdown", - "id": "98d5219e", - "metadata": {}, + "id": "ff915c59", + "metadata": { + "editable": true + }, "source": [ "## More details\n", "\n", @@ -3945,8 +4373,10 @@ }, { "cell_type": "markdown", - "id": "83d7b299", - "metadata": {}, + "id": "ca46667e", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -3960,16 +4390,20 @@ }, { "cell_type": "markdown", - "id": "1ba53d11", - "metadata": {}, + "id": "3efa1dc6", + "metadata": { + "editable": true + }, "source": [ "Letting $\\boldsymbol{x}$ be a vector with elements $x_i$ and $C(\\boldsymbol{x}, P) = \\frac{1}{N} \\sum_i \\big(g_t'(x_i, P) - ( -\\gamma g_t(x_i, P) \\big)^2$ denote the cost function, the minimization problem that our network must solve, becomes" ] }, { "cell_type": "markdown", - "id": "3953e778", - "metadata": {}, + "id": "a3d800e8", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\min_{P} C(\\boldsymbol{x}, P)\n", @@ -3978,8 +4412,10 @@ }, { "cell_type": "markdown", - "id": "706a1978", - "metadata": {}, + "id": "8fb679f5", + "metadata": { + "editable": true + }, "source": [ "In terms of $P_{\\text{hidden} }$ and $P_{\\text{output} }$, this could also be expressed as\n", "\n", @@ -3990,8 +4426,10 @@ }, { "cell_type": "markdown", - "id": "8ff57d1a", - "metadata": {}, + "id": "08b2a19b", + "metadata": { + "editable": true + }, "source": [ "## A possible implementation of a neural network\n", "\n", @@ -4004,8 +4442,10 @@ }, { "cell_type": "markdown", - "id": "fb487d3d", - "metadata": {}, + "id": "082074c8", + "metadata": { + "editable": true + }, "source": [ "## Technicalities\n", "\n", @@ -4014,8 +4454,10 @@ }, { "cell_type": "markdown", - "id": "1d6ac6a6", - "metadata": {}, + "id": "b69f9148", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{aligned}\n", @@ -4034,8 +4476,10 @@ }, { "cell_type": "markdown", - "id": "fee86c29", - "metadata": {}, + "id": "9feeefde", + "metadata": { + "editable": true + }, "source": [ "## Final technicalities I\n", "\n", @@ -4044,8 +4488,10 @@ }, { "cell_type": "markdown", - "id": "317dd762", - "metadata": {}, + "id": "cde763a7", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{aligned}\n", @@ -4065,8 +4511,10 @@ }, { "cell_type": "markdown", - "id": "4641c716", - "metadata": {}, + "id": "3c83e5dc", + "metadata": { + "editable": true + }, "source": [ "## Final technicalities II\n", "\n", @@ -4079,8 +4527,10 @@ }, { "cell_type": "markdown", - "id": "a28fd5d3", - "metadata": {}, + "id": "4566531c", + "metadata": { + "editable": true + }, "source": [ "$$\n", "f(z) = \\frac{1}{1 + \\exp{(-z)}}\n", @@ -4089,8 +4539,10 @@ }, { "cell_type": "markdown", - "id": "d2c6ce13", - "metadata": {}, + "id": "e316d1e7", + "metadata": { + "editable": true + }, "source": [ "It is possible to use other activations functions for the hidden layer also.\n", "\n", @@ -4111,8 +4563,10 @@ }, { "cell_type": "markdown", - "id": "5bf0304c", - "metadata": {}, + "id": "41ac9a1c", + "metadata": { + "editable": true + }, "source": [ "## Final technicalities III\n", "\n", @@ -4121,8 +4575,10 @@ }, { "cell_type": "markdown", - "id": "f4ee906f", - "metadata": {}, + "id": "b0a8a3c2", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{aligned}\n", @@ -4140,8 +4596,10 @@ }, { "cell_type": "markdown", - "id": "454cd32f", - "metadata": {}, + "id": "67447b23", + "metadata": { + "editable": true + }, "source": [ "## Final technicalities IV\n", "\n", @@ -4150,8 +4608,10 @@ }, { "cell_type": "markdown", - "id": "49b52638", - "metadata": {}, + "id": "12d3e1ee", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\boldsymbol{z}_{1}^{\\text{output}} =\n", @@ -4167,16 +4627,20 @@ }, { "cell_type": "markdown", - "id": "75d1e0dc", - "metadata": {}, + "id": "562a122c", + "metadata": { + "editable": true + }, "source": [ "In this case we seek a continuous range of values since we are approximating a function. This means that after computing $\\boldsymbol{z}_{1}^{\\text{output}}$ the neural network has finished its feed forward step, and $\\boldsymbol{z}_{1}^{\\text{output}}$ is the final output of the network." ] }, { "cell_type": "markdown", - "id": "243508b9", - "metadata": {}, + "id": "78789a96", + "metadata": { + "editable": true + }, "source": [ "## Back propagation\n", "\n", @@ -4187,8 +4651,10 @@ }, { "cell_type": "markdown", - "id": "cfaa5264", - "metadata": {}, + "id": "76b734fb", + "metadata": { + "editable": true + }, "source": [ "$$\n", "C(\\boldsymbol{x}, P) = \\frac{1}{N} \\sum_i \\big(g_t'(x_i, P) - ( -\\gamma g_t(x_i, P) \\big)^2\n", @@ -4197,8 +4663,10 @@ }, { "cell_type": "markdown", - "id": "188a58e4", - "metadata": {}, + "id": "e2911f97", + "metadata": { + "editable": true + }, "source": [ "In order to minimize the cost function, an optimization method must be chosen.\n", "\n", @@ -4207,8 +4675,10 @@ }, { "cell_type": "markdown", - "id": "6f3c7251", - "metadata": {}, + "id": "5e0c281c", + "metadata": { + "editable": true + }, "source": [ "## Gradient descent\n", "\n", @@ -4222,8 +4692,10 @@ }, { "cell_type": "markdown", - "id": "b48662af", - "metadata": {}, + "id": "077e3318", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\boldsymbol{\\omega}_{\\text{new} } = \\boldsymbol{\\omega} - \\lambda \\nabla_{\\boldsymbol{\\omega}} C(\\boldsymbol{x}, \\boldsymbol{\\omega})\n", @@ -4232,8 +4704,10 @@ }, { "cell_type": "markdown", - "id": "46cc1b9b", - "metadata": {}, + "id": "b6e9387d", + "metadata": { + "editable": true + }, "source": [ "for a number of iterations or until $ \\big|\\big| \\boldsymbol{\\omega}_{\\text{new} } - \\boldsymbol{\\omega} \\big|\\big|$ becomes smaller than some given tolerance.\n", "\n", @@ -4252,8 +4726,10 @@ }, { "cell_type": "markdown", - "id": "018877f6", - "metadata": {}, + "id": "3fa21e60", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{aligned}\n", @@ -4265,8 +4741,10 @@ }, { "cell_type": "markdown", - "id": "35e348bd", - "metadata": {}, + "id": "e05e6028", + "metadata": { + "editable": true + }, "source": [ "## The code for solving the ODE" ] @@ -4274,8 +4752,11 @@ { "cell_type": "code", "execution_count": 41, - "id": "dafe8d25", - "metadata": {}, + "id": "eb77d1cf", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "import autograd.numpy as np\n", @@ -4426,8 +4907,10 @@ }, { "cell_type": "markdown", - "id": "65e4f300", - "metadata": {}, + "id": "bbd6562b", + "metadata": { + "editable": true + }, "source": [ "## The network with one input layer, specified number of hidden layers, and one output layer\n", "\n", @@ -4439,8 +4922,11 @@ { "cell_type": "code", "execution_count": 42, - "id": "5ef5f766", - "metadata": {}, + "id": "24ba1afd", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "import autograd.numpy as np\n", @@ -4455,8 +4941,8 @@ "# but with number of hidden layers specified by the user.\n", "def deep_neural_network(deep_params, x):\n", " # N_hidden is the number of hidden layers\n", - "\n", - " N_hidden = np.size(deep_params) - 1 # -1 since params consists of\n", + " # deep_params is a list, len() should be used\n", + " N_hidden = len(deep_params) - 1 # -1 since params consists of\n", " # parameters to all the hidden\n", " # layers AND the output layer.\n", "\n", @@ -4605,8 +5091,10 @@ }, { "cell_type": "markdown", - "id": "c7ad45ef", - "metadata": {}, + "id": "3974040a", + "metadata": { + "editable": true + }, "source": [ "## Example: Population growth\n", "\n", @@ -4616,8 +5104,10 @@ }, { "cell_type": "markdown", - "id": "2d1376bc", - "metadata": {}, + "id": "a9d203ea", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -4631,8 +5121,10 @@ }, { "cell_type": "markdown", - "id": "1f039785", - "metadata": {}, + "id": "ede35f92", + "metadata": { + "editable": true + }, "source": [ "where $g(t)$ is the population density at time $t$, $\\alpha > 0$ the growth rate and $A > 0$ is the maximum population number in the environment.\n", "Also, at $t = 0$ the population has the size $g(0) = g_0$, where $g_0$ is some chosen constant.\n", @@ -4645,8 +5137,10 @@ }, { "cell_type": "markdown", - "id": "8dfcfb5e", - "metadata": {}, + "id": "a9720dfc", + "metadata": { + "editable": true + }, "source": [ "## Setting up the problem\n", "\n", @@ -4656,8 +5150,10 @@ }, { "cell_type": "markdown", - "id": "3cb5e674", - "metadata": {}, + "id": "5ef6b555", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -4671,8 +5167,10 @@ }, { "cell_type": "markdown", - "id": "56581c9f", - "metadata": {}, + "id": "bd5c646e", + "metadata": { + "editable": true + }, "source": [ "where $g(0) = g_0$.\n", "\n", @@ -4681,8 +5179,10 @@ }, { "cell_type": "markdown", - "id": "ebf7032a", - "metadata": {}, + "id": "cdc5561b", + "metadata": { + "editable": true + }, "source": [ "## The trial solution\n", "\n", @@ -4706,8 +5206,10 @@ }, { "cell_type": "markdown", - "id": "a943c5c6", - "metadata": {}, + "id": "cebde164", + "metadata": { + "editable": true + }, "source": [ "## The program using Autograd\n", "\n", @@ -4717,8 +5219,11 @@ { "cell_type": "code", "execution_count": 43, - "id": "a3620769", - "metadata": {}, + "id": "c71a9592", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "import autograd.numpy as np\n", @@ -4737,9 +5242,12 @@ " g0 = 1.2\n", " return alpha, A, g0\n", "\n", - "def deep_neural_network(P, x):\n", + "def deep_neural_network(deep_params, x):\n", " # N_hidden is the number of hidden layers\n", - " N_hidden = np.size(P) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n", + " # deep_params is a list, len() should be used\n", + " N_hidden = len(deep_params) - 1 # -1 since params consists of\n", + " # parameters to all the hidden\n", + " # layers AND the output layer.\n", "\n", " # Assumes input x being an one-dimensional array\n", " num_values = np.size(x)\n", @@ -4756,7 +5264,7 @@ "\n", " for l in range(N_hidden):\n", " # From the list of parameters P; find the correct weigths and bias for this layer\n", - " w_hidden = P[l]\n", + " w_hidden = deep_params[l]\n", "\n", " # Add a row of ones to include bias\n", " x_prev = np.concatenate((np.ones((1,num_values)), x_prev ), axis = 0)\n", @@ -4770,7 +5278,7 @@ " ## Output layer:\n", "\n", " # Get the weights and bias for this layer\n", - " w_output = P[-1]\n", + " w_output = deep_params[-1]\n", "\n", " # Include bias:\n", " x_prev = np.concatenate((np.ones((1,num_values)), x_prev), axis = 0)\n", @@ -4781,6 +5289,8 @@ " return x_output\n", "\n", "\n", + "\n", + "\n", "def cost_function_deep(P, x):\n", "\n", " # Evaluate the trial function with the current parameters P\n", @@ -4888,8 +5398,10 @@ }, { "cell_type": "markdown", - "id": "9a21afc2", - "metadata": {}, + "id": "bd70d6cc", + "metadata": { + "editable": true + }, "source": [ "## Using forward Euler to solve the ODE\n", "\n", @@ -4906,8 +5418,10 @@ }, { "cell_type": "markdown", - "id": "8a8ad46c", - "metadata": {}, + "id": "ee3bdd02", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{aligned}\n", @@ -4919,8 +5433,10 @@ }, { "cell_type": "markdown", - "id": "7b83336b", - "metadata": {}, + "id": "d81b6054", + "metadata": { + "editable": true + }, "source": [ "along with the condition that $g(0) = g_0$.\n", "\n", @@ -4931,8 +5447,10 @@ }, { "cell_type": "markdown", - "id": "ea68eaac", - "metadata": {}, + "id": "0d7a2272", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{aligned}\n", @@ -4945,16 +5463,20 @@ }, { "cell_type": "markdown", - "id": "c8a2f92c", - "metadata": {}, + "id": "ca29846f", + "metadata": { + "editable": true + }, "source": [ "Now, if $g_i = g(t_i)$ then" ] }, { "cell_type": "markdown", - "id": "4f1a2187", - "metadata": {}, + "id": "1a51d849", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -4973,8 +5495,10 @@ }, { "cell_type": "markdown", - "id": "cb87fc93", - "metadata": {}, + "id": "700a9bac", + "metadata": { + "editable": true + }, "source": [ "for $i \\geq 1$ and $g_0 = g(t_0) = g(0) = g_0$.\n", "\n", @@ -4985,8 +5509,11 @@ { "cell_type": "code", "execution_count": 44, - "id": "f8c976e7", - "metadata": {}, + "id": "482cf93c", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "# Assume that all function definitions from the example program using Autograd\n", @@ -5058,8 +5585,10 @@ }, { "cell_type": "markdown", - "id": "23756e15", - "metadata": {}, + "id": "4ed3cefd", + "metadata": { + "editable": true + }, "source": [ "## Example: Solving the one dimensional Poisson equation\n", "\n", @@ -5068,8 +5597,10 @@ }, { "cell_type": "markdown", - "id": "3a68185f", - "metadata": {}, + "id": "d0bdfcdb", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -5083,8 +5614,10 @@ }, { "cell_type": "markdown", - "id": "dc784286", - "metadata": {}, + "id": "656240e7", + "metadata": { + "editable": true + }, "source": [ "where $f(x)$ is a given function for $x \\in (0,1)$.\n", "\n", @@ -5093,8 +5626,10 @@ }, { "cell_type": "markdown", - "id": "e0f7d593", - "metadata": {}, + "id": "fb310d7c", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{align*}\n", @@ -5106,8 +5641,10 @@ }, { "cell_type": "markdown", - "id": "4cacfaeb", - "metadata": {}, + "id": "e6b102dc", + "metadata": { + "editable": true + }, "source": [ "This equation can be solved numerically using programs where e.g Autograd and TensorFlow are used.\n", "The results from the networks can then be compared to the analytical solution.\n", @@ -5116,8 +5653,10 @@ }, { "cell_type": "markdown", - "id": "69ba705a", - "metadata": {}, + "id": "797d4bb8", + "metadata": { + "editable": true + }, "source": [ "## The specific equation to solve for\n", "\n", @@ -5126,8 +5665,10 @@ }, { "cell_type": "markdown", - "id": "995cfbc9", - "metadata": {}, + "id": "da2e90b8", + "metadata": { + "editable": true + }, "source": [ "$$\n", "-g''(x) = f(x),\\qquad x \\in (0,1)\n", @@ -5136,16 +5677,20 @@ }, { "cell_type": "markdown", - "id": "769f9670", - "metadata": {}, + "id": "318f8f3a", + "metadata": { + "editable": true + }, "source": [ "where $f(x)$ is a given function, along with the chosen conditions" ] }, { "cell_type": "markdown", - "id": "855cddbb", - "metadata": {}, + "id": "c5e8ac9e", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -5159,8 +5704,10 @@ }, { "cell_type": "markdown", - "id": "7006cb54", - "metadata": {}, + "id": "220ceeb4", + "metadata": { + "editable": true + }, "source": [ "In this example, we consider the case when $f(x) = (3x + x^2)\\exp(x)$.\n", "\n", @@ -5169,8 +5716,10 @@ }, { "cell_type": "markdown", - "id": "5216ca22", - "metadata": {}, + "id": "ec8b38dc", + "metadata": { + "editable": true + }, "source": [ "$$\n", "g_t(x) = x \\cdot (1-x) \\cdot N(P,x)\n", @@ -5179,16 +5728,20 @@ }, { "cell_type": "markdown", - "id": "b77670be", - "metadata": {}, + "id": "0ed927a9", + "metadata": { + "editable": true + }, "source": [ "The analytical solution for this problem is" ] }, { "cell_type": "markdown", - "id": "bf2bed23", - "metadata": {}, + "id": "3a51ddcc", + "metadata": { + "editable": true + }, "source": [ "$$\n", "g(x) = x(1 - x)\\exp(x)\n", @@ -5197,8 +5750,10 @@ }, { "cell_type": "markdown", - "id": "a22ccb78", - "metadata": {}, + "id": "d5392b8d", + "metadata": { + "editable": true + }, "source": [ "## Solving the equation using Autograd" ] @@ -5206,8 +5761,11 @@ { "cell_type": "code", "execution_count": 45, - "id": "7b3b5f5a", - "metadata": {}, + "id": "95059f71", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "import autograd.numpy as np\n", @@ -5220,7 +5778,10 @@ "\n", "def deep_neural_network(deep_params, x):\n", " # N_hidden is the number of hidden layers\n", - " N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n", + " # deep_params is a list, len() should be used\n", + " N_hidden = len(deep_params) - 1 # -1 since params consists of\n", + " # parameters to all the hidden\n", + " # layers AND the output layer.\n", "\n", " # Assumes input x being an one-dimensional array\n", " num_values = np.size(x)\n", @@ -5261,6 +5822,7 @@ "\n", " return x_output\n", "\n", + "\n", "def solve_ode_deep_neural_network(x, num_neurons, num_iter, lmb):\n", " # num_hidden_neurons is now a list of number of neurons within each hidden layer\n", "\n", @@ -5364,8 +5926,10 @@ }, { "cell_type": "markdown", - "id": "6668b7ef", - "metadata": {}, + "id": "7c568bf3", + "metadata": { + "editable": true + }, "source": [ "## Comparing with a numerical scheme\n", "\n", @@ -5384,8 +5948,10 @@ }, { "cell_type": "markdown", - "id": "ec1adda2", - "metadata": {}, + "id": "eab0bb68", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -5399,16 +5965,20 @@ }, { "cell_type": "markdown", - "id": "33486af4", - "metadata": {}, + "id": "c0ada0c0", + "metadata": { + "editable": true + }, "source": [ "If $x_i = i \\Delta x = x_{i-1} + \\Delta x$ and $g_i = g(x_i)$ for $i = 1,\\dots N_x - 2$ with $N_x$ being the number of values for $x$, ([15](#approx)) becomes" ] }, { "cell_type": "markdown", - "id": "42c3ef84", - "metadata": {}, + "id": "b483eace", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{aligned}\n", @@ -5420,16 +5990,20 @@ }, { "cell_type": "markdown", - "id": "f454dbea", - "metadata": {}, + "id": "30d45122", + "metadata": { + "editable": true + }, "source": [ "Since we know from our problem that" ] }, { "cell_type": "markdown", - "id": "7411a707", - "metadata": {}, + "id": "52c68e05", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{aligned}\n", @@ -5441,8 +6015,10 @@ }, { "cell_type": "markdown", - "id": "c47c3822", - "metadata": {}, + "id": "1657934d", + "metadata": { + "editable": true + }, "source": [ "along with the conditions $g(0) = g(1) = 0$,\n", "the following scheme can be used to find an approximate solution for $g(x)$ numerically:" @@ -5450,8 +6026,10 @@ }, { "cell_type": "markdown", - "id": "1cca824c", - "metadata": {}, + "id": "9fc46a1b", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -5468,8 +6046,10 @@ }, { "cell_type": "markdown", - "id": "82ecd5a1", - "metadata": {}, + "id": "cd79f75c", + "metadata": { + "editable": true + }, "source": [ "for $i = 1, \\dots, N_x - 2$ where $g_0 = g_{N_x - 1} = 0$ and $f(x_i) = (3x_i + x_i^2)\\exp(x_i)$, which is given for our specific problem.\n", "\n", @@ -5478,8 +6058,10 @@ }, { "cell_type": "markdown", - "id": "39f5fe07", - "metadata": {}, + "id": "501ac880", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{aligned}\n", @@ -5513,16 +6095,20 @@ }, { "cell_type": "markdown", - "id": "5ac54c35", - "metadata": {}, + "id": "8551615b", + "metadata": { + "editable": true + }, "source": [ "which makes it possible to solve for the vector $\\boldsymbol{g}$." ] }, { "cell_type": "markdown", - "id": "79ad3ebc", - "metadata": {}, + "id": "dbd6d2a1", + "metadata": { + "editable": true + }, "source": [ "## Setting up the code\n", "\n", @@ -5532,8 +6118,11 @@ { "cell_type": "code", "execution_count": 46, - "id": "be03bf2d", - "metadata": {}, + "id": "c71a3063", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "import autograd.numpy as np\n", @@ -5546,7 +6135,10 @@ "\n", "def deep_neural_network(deep_params, x):\n", " # N_hidden is the number of hidden layers\n", - " N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n", + " # deep_params is a list, len() should be used\n", + " N_hidden = len(deep_params) - 1 # -1 since params consists of\n", + " # parameters to all the hidden\n", + " # layers AND the output layer.\n", "\n", " # Assumes input x being an one-dimensional array\n", " num_values = np.size(x)\n", @@ -5587,6 +6179,7 @@ "\n", " return x_output\n", "\n", + "\n", "def solve_ode_deep_neural_network(x, num_neurons, num_iter, lmb):\n", " # num_hidden_neurons is now a list of number of neurons within each hidden layer\n", "\n", @@ -5730,8 +6323,10 @@ }, { "cell_type": "markdown", - "id": "faa0daa4", - "metadata": {}, + "id": "da58cdec", + "metadata": { + "editable": true + }, "source": [ "## Partial Differential Equations\n", "\n", @@ -5745,8 +6340,10 @@ }, { "cell_type": "markdown", - "id": "fb7b7ff6", - "metadata": {}, + "id": "df991968", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -5760,16 +6357,20 @@ }, { "cell_type": "markdown", - "id": "71da1640", - "metadata": {}, + "id": "59ef3099", + "metadata": { + "editable": true + }, "source": [ "where $f$ is an expression involving all kinds of possible mixed derivatives of $g(x_1,\\dots,x_N)$ up to an order $n$. In order for the solution to be unique, some additional conditions must also be given." ] }, { "cell_type": "markdown", - "id": "c77e6225", - "metadata": {}, + "id": "a580ae0f", + "metadata": { + "editable": true + }, "source": [ "## Type of problem\n", "\n", @@ -5781,8 +6382,10 @@ }, { "cell_type": "markdown", - "id": "98f82ea2", - "metadata": {}, + "id": "a56f3fe1", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{align*}\n", @@ -5793,8 +6396,10 @@ }, { "cell_type": "markdown", - "id": "bc33501c", - "metadata": {}, + "id": "69586714", + "metadata": { + "editable": true + }, "source": [ "where $h_1(x_1,\\dots,x_N)$ is a function that ensures $g_t(x_1,\\dots,x_N)$ satisfies some given conditions.\n", "The neural network $N(x_1,\\dots,x_N,P)$ has weights and biases described by $P$ and $h_2(x_1,\\dots,x_N,N(x_1,\\dots,x_N,P))$ is an expression using the output from the neural network in some way.\n", @@ -5804,8 +6409,10 @@ }, { "cell_type": "markdown", - "id": "818e8900", - "metadata": {}, + "id": "d8374dce", + "metadata": { + "editable": true + }, "source": [ "## Network requirements\n", "\n", @@ -5822,8 +6429,10 @@ }, { "cell_type": "markdown", - "id": "526d5428", - "metadata": {}, + "id": "f3fe961e", + "metadata": { + "editable": true + }, "source": [ "$$\n", "C\\left(x_1, \\dots, x_N, P\\right) = \\left( f\\left(x_1, \\, \\dots \\, , x_N, \\frac{\\partial g(x_1,\\dots,x_N) }{\\partial x_1}, \\dots , \\frac{\\partial g(x_1,\\dots,x_N) }{\\partial x_N}, \\frac{\\partial g(x_1,\\dots,x_N) }{\\partial x_1\\partial x_2}, \\, \\dots \\, , \\frac{\\partial^n g(x_1,\\dots,x_N) }{\\partial x_N^n} \\right) \\right)^2\n", @@ -5832,8 +6441,10 @@ }, { "cell_type": "markdown", - "id": "279b9296", - "metadata": {}, + "id": "4e453aa1", + "metadata": { + "editable": true + }, "source": [ "## More details\n", "\n", @@ -5842,8 +6453,10 @@ }, { "cell_type": "markdown", - "id": "bcb8ce40", - "metadata": {}, + "id": "e1ebf5bd", + "metadata": { + "editable": true + }, "source": [ "$$\n", "C\\left(\\boldsymbol{x}, P\\right) = f\\left( \\left( \\boldsymbol{x}, \\frac{\\partial g(\\boldsymbol{x}) }{\\partial x_1}, \\dots , \\frac{\\partial g(\\boldsymbol{x}) }{\\partial x_N}, \\frac{\\partial g(\\boldsymbol{x}) }{\\partial x_1\\partial x_2}, \\, \\dots \\, , \\frac{\\partial^n g(\\boldsymbol{x}) }{\\partial x_N^n} \\right) \\right)^2\n", @@ -5852,16 +6465,20 @@ }, { "cell_type": "markdown", - "id": "6549ea7b", - "metadata": {}, + "id": "b8c35553", + "metadata": { + "editable": true + }, "source": [ "If we also have $M$ different sets of values for $x_1, \\dots, x_N$, that is $\\boldsymbol{x}_i = \\big(x_1^{(i)}, \\dots, x_N^{(i)}\\big)$ for $i = 1,\\dots,M$ being the rows in matrix $X$, the cost function can be generalized into" ] }, { "cell_type": "markdown", - "id": "0956062e", - "metadata": {}, + "id": "22a123d5", + "metadata": { + "editable": true + }, "source": [ "$$\n", "C\\left(X, P \\right) = \\sum_{i=1}^M f\\left( \\left( \\boldsymbol{x}_i, \\frac{\\partial g(\\boldsymbol{x}_i) }{\\partial x_1}, \\dots , \\frac{\\partial g(\\boldsymbol{x}_i) }{\\partial x_N}, \\frac{\\partial g(\\boldsymbol{x}_i) }{\\partial x_1\\partial x_2}, \\, \\dots \\, , \\frac{\\partial^n g(\\boldsymbol{x}_i) }{\\partial x_N^n} \\right) \\right)^2.\n", @@ -5870,8 +6487,10 @@ }, { "cell_type": "markdown", - "id": "8fca0166", - "metadata": {}, + "id": "b15eedf5", + "metadata": { + "editable": true + }, "source": [ "## Example: The diffusion equation\n", "\n", @@ -5880,8 +6499,10 @@ }, { "cell_type": "markdown", - "id": "58ad6f25", - "metadata": {}, + "id": "1a92cc8c", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\frac{\\partial g(x,t)}{\\partial t} = \\frac{\\partial^2 g(x,t)}{\\partial x^2}\n", @@ -5890,16 +6511,20 @@ }, { "cell_type": "markdown", - "id": "e6e5728e", - "metadata": {}, + "id": "e4862313", + "metadata": { + "editable": true + }, "source": [ "where a possible choice of conditions are" ] }, { "cell_type": "markdown", - "id": "6921faa1", - "metadata": {}, + "id": "5c186d68", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{align*}\n", @@ -5912,16 +6537,20 @@ }, { "cell_type": "markdown", - "id": "80e632ef", - "metadata": {}, + "id": "57d4f000", + "metadata": { + "editable": true + }, "source": [ "with $u(x)$ being some given function." ] }, { "cell_type": "markdown", - "id": "a2711421", - "metadata": {}, + "id": "7c2cdc6b", + "metadata": { + "editable": true + }, "source": [ "## Defining the problem\n", "\n", @@ -5930,8 +6559,10 @@ }, { "cell_type": "markdown", - "id": "981da4af", - "metadata": {}, + "id": "bde064ea", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -5945,16 +6576,20 @@ }, { "cell_type": "markdown", - "id": "c1b8c16d", - "metadata": {}, + "id": "7a558c97", + "metadata": { + "editable": true + }, "source": [ "and" ] }, { "cell_type": "markdown", - "id": "b8ed6509", - "metadata": {}, + "id": "534a4844", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{align*}\n", @@ -5967,8 +6602,10 @@ }, { "cell_type": "markdown", - "id": "200050ac", - "metadata": {}, + "id": "a07e0730", + "metadata": { + "editable": true + }, "source": [ "with $u(x) = \\sin(\\pi x)$.\n", "\n", @@ -5979,8 +6616,10 @@ }, { "cell_type": "markdown", - "id": "5dca99d2", - "metadata": {}, + "id": "74b72286", + "metadata": { + "editable": true + }, "source": [ "## Setting up the network using Autograd\n", "\n", @@ -5996,8 +6635,11 @@ { "cell_type": "code", "execution_count": 47, - "id": "85f4b843", - "metadata": {}, + "id": "4a0c57e9", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "def sigmoid(z):\n", @@ -6011,7 +6653,7 @@ " num_points = np.size(x,1)\n", "\n", " # N_hidden is the number of hidden layers\n", - " N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n", + " N_hidden = len(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n", "\n", " # Assume that the input layer does nothing to the input x\n", " x_input = x\n", @@ -6048,8 +6690,10 @@ }, { "cell_type": "markdown", - "id": "9dd68ff8", - "metadata": {}, + "id": "350b1d3d", + "metadata": { + "editable": true + }, "source": [ "## Setting up the network using Autograd; The trial solution\n", "\n", @@ -6076,8 +6720,10 @@ }, { "cell_type": "markdown", - "id": "65f68c5a", - "metadata": {}, + "id": "9fd438b5", + "metadata": { + "editable": true + }, "source": [ "## Why the jacobian?\n", "\n", @@ -6103,8 +6749,11 @@ { "cell_type": "code", "execution_count": 48, - "id": "91612f4f", - "metadata": {}, + "id": "d1c81690", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "# Set up the trial function:\n", @@ -6147,8 +6796,10 @@ }, { "cell_type": "markdown", - "id": "ff9df4ba", - "metadata": {}, + "id": "ca22d9ce", + "metadata": { + "editable": true + }, "source": [ "## Setting up the network using Autograd; The full program\n", "\n", @@ -6171,8 +6822,11 @@ { "cell_type": "code", "execution_count": 49, - "id": "e6451b9e", - "metadata": {}, + "id": "5a9146bc", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "import autograd.numpy as np\n", @@ -6195,7 +6849,7 @@ " num_points = np.size(x,1)\n", "\n", " # N_hidden is the number of hidden layers\n", - " N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n", + " N_hidden = len(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n", "\n", " # Assume that the input layer does nothing to the input x\n", " x_input = x\n", @@ -6338,7 +6992,7 @@ " T,X = np.meshgrid(t,x)\n", "\n", " fig = plt.figure(figsize=(10,10))\n", - " ax = fig.gca(projection='3d')\n", + " ax = fig.add_suplot(projection='3d')\n", " ax.set_title('Solution from the deep neural network w/ %d layer'%len(num_hidden_neurons))\n", " s = ax.plot_surface(T,X,g_dnn_ag,linewidth=0,antialiased=False,cmap=cm.viridis)\n", " ax.set_xlabel('Time $t$')\n", @@ -6346,14 +7000,14 @@ "\n", "\n", " fig = plt.figure(figsize=(10,10))\n", - " ax = fig.gca(projection='3d')\n", + " ax = fig.add_suplot(projection='3d')\n", " ax.set_title('Analytical solution')\n", " s = ax.plot_surface(T,X,G_analytical,linewidth=0,antialiased=False,cmap=cm.viridis)\n", " ax.set_xlabel('Time $t$')\n", " ax.set_ylabel('Position $x$');\n", "\n", " fig = plt.figure(figsize=(10,10))\n", - " ax = fig.gca(projection='3d')\n", + " ax = fig.add_suplot(projection='3d')\n", " ax.set_title('Difference')\n", " s = ax.plot_surface(T,X,diff_ag,linewidth=0,antialiased=False,cmap=cm.viridis)\n", " ax.set_xlabel('Time $t$')\n", @@ -6402,8 +7056,10 @@ }, { "cell_type": "markdown", - "id": "7f867cf2", - "metadata": {}, + "id": "e59d2116", + "metadata": { + "editable": true + }, "source": [ "## Example: Solving the wave equation with Neural Networks\n", "\n", @@ -6412,8 +7068,10 @@ }, { "cell_type": "markdown", - "id": "ded83be1", - "metadata": {}, + "id": "e2ffdbaf", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\frac{\\partial^2 g(x,t)}{\\partial t^2} = c^2\\frac{\\partial^2 g(x,t)}{\\partial x^2}\n", @@ -6422,8 +7080,10 @@ }, { "cell_type": "markdown", - "id": "f0aba0c9", - "metadata": {}, + "id": "37b2cc03", + "metadata": { + "editable": true + }, "source": [ "with $c$ being the specified wave speed.\n", "\n", @@ -6432,8 +7092,10 @@ }, { "cell_type": "markdown", - "id": "28a48fe7", - "metadata": {}, + "id": "524df5d3", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{align*}\n", @@ -6447,16 +7109,20 @@ }, { "cell_type": "markdown", - "id": "61ded78d", - "metadata": {}, + "id": "d175b25f", + "metadata": { + "editable": true + }, "source": [ "where $\\frac{\\partial g(x,t)}{\\partial t} \\Big |_{t = 0}$ means the derivative of $g(x,t)$ with respect to $t$ is evaluated at $t = 0$, and $u(x)$ and $v(x)$ being given functions." ] }, { "cell_type": "markdown", - "id": "990695ea", - "metadata": {}, + "id": "11b3289e", + "metadata": { + "editable": true + }, "source": [ "## The problem to solve for\n", "\n", @@ -6465,8 +7131,10 @@ }, { "cell_type": "markdown", - "id": "58841089", - "metadata": {}, + "id": "e469d65e", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -6480,8 +7148,10 @@ }, { "cell_type": "markdown", - "id": "303eb900", - "metadata": {}, + "id": "5f773fa8", + "metadata": { + "editable": true + }, "source": [ "where $c$ is the given wave speed.\n", "The chosen conditions for this equation are" @@ -6489,8 +7159,10 @@ }, { "cell_type": "markdown", - "id": "ce5c5167", - "metadata": {}, + "id": "2952f7b4", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -6507,16 +7179,20 @@ }, { "cell_type": "markdown", - "id": "609a7556", - "metadata": {}, + "id": "99057b48", + "metadata": { + "editable": true + }, "source": [ "In this example, let $c = 1$ and $u(x) = \\sin(\\pi x)$ and $v(x) = -\\pi\\sin(\\pi x)$." ] }, { "cell_type": "markdown", - "id": "5edb8797", - "metadata": {}, + "id": "7debdfea", + "metadata": { + "editable": true + }, "source": [ "## The trial solution\n", "Setting up the network is done in similar matter as for the example of solving the diffusion equation.\n", @@ -6539,8 +7215,10 @@ }, { "cell_type": "markdown", - "id": "c6be7518", - "metadata": {}, + "id": "2d5d5c44", + "metadata": { + "editable": true + }, "source": [ "## The analytical solution\n", "\n", @@ -6553,8 +7231,10 @@ }, { "cell_type": "markdown", - "id": "368bb540", - "metadata": {}, + "id": "f97806ce", + "metadata": { + "editable": true + }, "source": [ "## Solving the wave equation - the full program using Autograd" ] @@ -6562,8 +7242,11 @@ { "cell_type": "code", "execution_count": 50, - "id": "65085f11", - "metadata": {}, + "id": "9337b7c7", + "metadata": { + "collapsed": false, + "editable": true + }, "outputs": [], "source": [ "import autograd.numpy as np\n", @@ -6620,7 +7303,7 @@ " num_points = np.size(x,1)\n", "\n", " # N_hidden is the number of hidden layers\n", - " N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n", + " N_hidden = len(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n", "\n", " # Assume that the input layer does nothing to the input x\n", " x_input = x\n", @@ -6725,7 +7408,7 @@ " T,X = np.meshgrid(t,x)\n", "\n", " fig = plt.figure(figsize=(10,10))\n", - " ax = fig.gca(projection='3d')\n", + " ax = fig.add_suplot(projection='3d')\n", " ax.set_title('Solution from the deep neural network w/ %d layer'%len(num_hidden_neurons))\n", " s = ax.plot_surface(T,X,res,linewidth=0,antialiased=False,cmap=cm.viridis)\n", " ax.set_xlabel('Time $t$')\n", @@ -6733,7 +7416,7 @@ "\n", "\n", " fig = plt.figure(figsize=(10,10))\n", - " ax = fig.gca(projection='3d')\n", + " ax = fig.add_suplot(projection='3d')\n", " ax.set_title('Analytical solution')\n", " s = ax.plot_surface(T,X,res_analytical,linewidth=0,antialiased=False,cmap=cm.viridis)\n", " ax.set_xlabel('Time $t$')\n", @@ -6741,7 +7424,7 @@ "\n", "\n", " fig = plt.figure(figsize=(10,10))\n", - " ax = fig.gca(projection='3d')\n", + " ax = fig.add_suplot(projection='3d')\n", " ax.set_title('Difference')\n", " s = ax.plot_surface(T,X,diff,linewidth=0,antialiased=False,cmap=cm.viridis)\n", " ax.set_xlabel('Time $t$')\n", @@ -6790,8 +7473,10 @@ }, { "cell_type": "markdown", - "id": "41011c90", - "metadata": {}, + "id": "6071d006", + "metadata": { + "editable": true + }, "source": [ "## Resources on differential equations and deep learning\n", "\n", @@ -6805,25 +7490,7 @@ ] } ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.9.15" - } - }, + "metadata": {}, "nbformat": 4, "nbformat_minor": 5 } diff --git a/doc/pub/week43/html/week43-bs.html b/doc/pub/week43/html/week43-bs.html index be20cd0b1..75f394fcb 100644 --- a/doc/pub/week43/html/week43-bs.html +++ b/doc/pub/week43/html/week43-bs.html @@ -488,8 +488,8 @@ MathJax.Hub.Config({
    • Building our own Feed-forward Neural Network with intro to Tensorflow
      • -
    • Solving differential equations with Neural Networks - +
    • Solving differential equations with Neural Networks
      • +
    • "Video of lecture at https://youtu.be/vkBNTn-MLqs
    diff --git a/doc/pub/week43/html/week43-reveal.html b/doc/pub/week43/html/week43-reveal.html index 61513cb07..aae851132 100644 --- a/doc/pub/week43/html/week43-reveal.html +++ b/doc/pub/week43/html/week43-reveal.html @@ -204,8 +204,9 @@ MathJax.Hub.Config({

  • Building our own Feed-forward Neural Network with intro to Tensorflow
    • -

  • Solving differential equations with Neural Networks - +

  • Solving differential equations with Neural Networks
    • + +

  • "Video of lecture at https://youtu.be/vkBNTn-MLqs
    • diff --git a/doc/pub/week43/html/week43-solarized.html b/doc/pub/week43/html/week43-solarized.html index 959c48e25..c74a76ad7 100644 --- a/doc/pub/week43/html/week43-solarized.html +++ b/doc/pub/week43/html/week43-solarized.html @@ -384,8 +384,8 @@ MathJax.Hub.Config({

    diff --git a/doc/pub/week43/html/week43.html b/doc/pub/week43/html/week43.html index bc83851c2..be0d57012 100644 --- a/doc/pub/week43/html/week43.html +++ b/doc/pub/week43/html/week43.html @@ -461,8 +461,8 @@ MathJax.Hub.Config({

    diff --git a/doc/pub/week43/ipynb/ipynb-week43-src.tar.gz b/doc/pub/week43/ipynb/ipynb-week43-src.tar.gz index 7860fbb0f..1842b323b 100644 Binary files a/doc/pub/week43/ipynb/ipynb-week43-src.tar.gz and b/doc/pub/week43/ipynb/ipynb-week43-src.tar.gz differ diff --git a/doc/pub/week43/ipynb/week43.ipynb b/doc/pub/week43/ipynb/week43.ipynb index 202263be5..3a2029a79 100644 --- a/doc/pub/week43/ipynb/week43.ipynb +++ b/doc/pub/week43/ipynb/week43.ipynb @@ -2,7 +2,7 @@ "cells": [ { "cell_type": "markdown", - "id": "677a195d", + "id": "6528e21e", "metadata": { "editable": true }, @@ -14,7 +14,7 @@ }, { "cell_type": "markdown", - "id": "e943e8f0", + "id": "3ee1b36e", "metadata": { "editable": true }, @@ -27,7 +27,7 @@ }, { "cell_type": "markdown", - "id": "22cdced0", + "id": "ce7aadd0", "metadata": { "editable": true }, @@ -39,13 +39,14 @@ " * Building our own Feed-forward Neural Network with intro to Tensorflow\n", "\n", " * Solving differential equations with Neural Networks\n", - "\n", + "\n", + " * \"Video of lecture at \n", "" ] }, { "cell_type": "markdown", - "id": "a57ef6df", + "id": "3b148f15", "metadata": { "editable": true }, @@ -62,7 +63,7 @@ }, { "cell_type": "markdown", - "id": "c4a134ee", + "id": "de5149e1", "metadata": { "editable": true }, @@ -78,7 +79,7 @@ }, { "cell_type": "markdown", - "id": "dd73045e", + "id": "4dfacc15", "metadata": { "editable": true }, @@ -89,7 +90,7 @@ }, { "cell_type": "markdown", - "id": "372f4aa4", + "id": "07178d65", "metadata": { "editable": true }, @@ -103,7 +104,7 @@ }, { "cell_type": "markdown", - "id": "05efbc6f", + "id": "b4b2da11", "metadata": { "editable": true }, @@ -116,7 +117,7 @@ }, { "cell_type": "markdown", - "id": "d04b0948", + "id": "c264977f", "metadata": { "editable": true }, @@ -133,7 +134,7 @@ }, { "cell_type": "markdown", - "id": "b9d41f5c", + "id": "9e588fe4", "metadata": { "editable": true }, @@ -143,7 +144,7 @@ }, { "cell_type": "markdown", - "id": "68606991", + "id": "fd47ec8c", "metadata": { "editable": true }, @@ -170,7 +171,7 @@ }, { "cell_type": "markdown", - "id": "a564a394", + "id": "97fdb112", "metadata": { "editable": true }, @@ -193,7 +194,7 @@ }, { "cell_type": "markdown", - "id": "43ab4381", + "id": "98139ea4", "metadata": { "editable": true }, @@ -205,7 +206,7 @@ }, { "cell_type": "markdown", - "id": "f578cee4", + "id": "6eab3a16", "metadata": { "editable": true }, @@ -217,7 +218,7 @@ }, { "cell_type": "markdown", - "id": "fbc05943", + "id": "3bd14855", "metadata": { "editable": true }, @@ -227,7 +228,7 @@ }, { "cell_type": "markdown", - "id": "6ad3c28f", + "id": "e1ec5ba9", "metadata": { "editable": true }, @@ -239,7 +240,7 @@ }, { "cell_type": "markdown", - "id": "135513b5", + "id": "4a2edd2d", "metadata": { "editable": true }, @@ -253,7 +254,7 @@ }, { "cell_type": "markdown", - "id": "f643ecdf", + "id": "8e4eb5d7", "metadata": { "editable": true }, @@ -265,7 +266,7 @@ }, { "cell_type": "markdown", - "id": "077ef1c2", + "id": "1757a0c5", "metadata": { "editable": true }, @@ -277,7 +278,7 @@ }, { "cell_type": "markdown", - "id": "50cf7472", + "id": "5c1d75f4", "metadata": { "editable": true }, @@ -287,7 +288,7 @@ }, { "cell_type": "markdown", - "id": "a10f406c", + "id": "ddb26b01", "metadata": { "editable": true }, @@ -299,7 +300,7 @@ }, { "cell_type": "markdown", - "id": "626cba8f", + "id": "52e1a3cb", "metadata": { "editable": true }, @@ -311,7 +312,7 @@ }, { "cell_type": "markdown", - "id": "2ebe05ad", + "id": "db17b588", "metadata": { "editable": true }, @@ -321,7 +322,7 @@ }, { "cell_type": "markdown", - "id": "bfbbac35", + "id": "7ddf4cdf", "metadata": { "editable": true }, @@ -333,7 +334,7 @@ }, { "cell_type": "markdown", - "id": "ac515399", + "id": "9a8d5afc", "metadata": { "editable": true }, @@ -345,7 +346,7 @@ }, { "cell_type": "markdown", - "id": "4703cc51", + "id": "5c03ac8f", "metadata": { "editable": true }, @@ -368,7 +369,7 @@ }, { "cell_type": "markdown", - "id": "cab6ad03", + "id": "49f26f27", "metadata": { "editable": true }, @@ -380,7 +381,7 @@ }, { "cell_type": "markdown", - "id": "ad6042eb", + "id": "de917b48", "metadata": { "editable": true }, @@ -392,7 +393,7 @@ }, { "cell_type": "markdown", - "id": "1e96b7e7", + "id": "5222dbb0", "metadata": { "editable": true }, @@ -402,7 +403,7 @@ }, { "cell_type": "markdown", - "id": "c600792f", + "id": "aab49321", "metadata": { "editable": true }, @@ -414,7 +415,7 @@ }, { "cell_type": "markdown", - "id": "76d1da8e", + "id": "50d3fbbc", "metadata": { "editable": true }, @@ -435,7 +436,7 @@ }, { "cell_type": "markdown", - "id": "8e7e0e0c", + "id": "225ad80d", "metadata": { "editable": true }, @@ -449,7 +450,7 @@ }, { "cell_type": "markdown", - "id": "4b1f77d6", + "id": "d00cef28", "metadata": { "editable": true }, @@ -461,7 +462,7 @@ }, { "cell_type": "markdown", - "id": "10826ba9", + "id": "43a6c282", "metadata": { "editable": true }, @@ -483,7 +484,7 @@ }, { "cell_type": "markdown", - "id": "79d307b6", + "id": "9530f930", "metadata": { "editable": true }, @@ -505,7 +506,7 @@ }, { "cell_type": "markdown", - "id": "429b8d4e", + "id": "24c7d25a", "metadata": { "editable": true }, @@ -533,7 +534,7 @@ }, { "cell_type": "markdown", - "id": "90d9f195", + "id": "f2f3871c", "metadata": { "editable": true }, @@ -545,7 +546,7 @@ }, { "cell_type": "markdown", - "id": "73d2763e", + "id": "685fb678", "metadata": { "editable": true }, @@ -555,7 +556,7 @@ }, { "cell_type": "markdown", - "id": "e2a70a54", + "id": "dd107aee", "metadata": { "editable": true }, @@ -567,7 +568,7 @@ }, { "cell_type": "markdown", - "id": "7ed6c0dc", + "id": "dfcc5135", "metadata": { "editable": true }, @@ -578,7 +579,7 @@ }, { "cell_type": "markdown", - "id": "a42826b9", + "id": "fac94809", "metadata": { "editable": true }, @@ -590,7 +591,7 @@ }, { "cell_type": "markdown", - "id": "d8e401c5", + "id": "419655c4", "metadata": { "editable": true }, @@ -603,7 +604,7 @@ }, { "cell_type": "markdown", - "id": "17808105", + "id": "1ee71788", "metadata": { "editable": true }, @@ -628,7 +629,7 @@ }, { "cell_type": "markdown", - "id": "5a9b90ce", + "id": "e03a5a0b", "metadata": { "editable": true }, @@ -641,7 +642,7 @@ }, { "cell_type": "markdown", - "id": "57f33029", + "id": "2fd06b7e", "metadata": { "editable": true }, @@ -653,7 +654,7 @@ }, { "cell_type": "markdown", - "id": "422eb23f", + "id": "2ac0d4f6", "metadata": { "editable": true }, @@ -665,7 +666,7 @@ }, { "cell_type": "markdown", - "id": "0eb634c3", + "id": "7d6f8f75", "metadata": { "editable": true }, @@ -675,7 +676,7 @@ }, { "cell_type": "markdown", - "id": "b3b208fc", + "id": "4f3981af", "metadata": { "editable": true }, @@ -687,7 +688,7 @@ }, { "cell_type": "markdown", - "id": "3788aba1", + "id": "b9a06e11", "metadata": { "editable": true }, @@ -699,7 +700,7 @@ }, { "cell_type": "markdown", - "id": "4d38ceec", + "id": "1496aa66", "metadata": { "editable": true }, @@ -711,7 +712,7 @@ }, { "cell_type": "markdown", - "id": "cee503b9", + "id": "44fd57c8", "metadata": { "editable": true }, @@ -723,7 +724,7 @@ }, { "cell_type": "markdown", - "id": "19b29e02", + "id": "11a6e89d", "metadata": { "editable": true }, @@ -733,7 +734,7 @@ }, { "cell_type": "markdown", - "id": "19507159", + "id": "35f31213", "metadata": { "editable": true }, @@ -745,7 +746,7 @@ }, { "cell_type": "markdown", - "id": "a15191e4", + "id": "ad19c6f2", "metadata": { "editable": true }, @@ -755,7 +756,7 @@ }, { "cell_type": "markdown", - "id": "c9df7a22", + "id": "fc1cc4ca", "metadata": { "editable": true }, @@ -767,7 +768,7 @@ }, { "cell_type": "markdown", - "id": "0cae75ec", + "id": "b1c42a22", "metadata": { "editable": true }, @@ -780,7 +781,7 @@ }, { "cell_type": "markdown", - "id": "36b0506b", + "id": "7c87a4cd", "metadata": { "editable": true }, @@ -792,7 +793,7 @@ }, { "cell_type": "markdown", - "id": "0c729c48", + "id": "ec7fd1f9", "metadata": { "editable": true }, @@ -802,7 +803,7 @@ }, { "cell_type": "markdown", - "id": "060813af", + "id": "0cfc10a7", "metadata": { "editable": true }, @@ -814,7 +815,7 @@ }, { "cell_type": "markdown", - "id": "f2a5059b", + "id": "2d0e5b14", "metadata": { "editable": true }, @@ -825,7 +826,7 @@ }, { "cell_type": "markdown", - "id": "6763074d", + "id": "7669c765", "metadata": { "editable": true }, @@ -837,7 +838,7 @@ }, { "cell_type": "markdown", - "id": "f498b6b5", + "id": "3bbef228", "metadata": { "editable": true }, @@ -847,7 +848,7 @@ }, { "cell_type": "markdown", - "id": "fedce928", + "id": "cf1e2ee2", "metadata": { "editable": true }, @@ -859,7 +860,7 @@ }, { "cell_type": "markdown", - "id": "429c11ae", + "id": "0ba0a6a4", "metadata": { "editable": true }, @@ -869,7 +870,7 @@ }, { "cell_type": "markdown", - "id": "ddde8370", + "id": "e5dc0e2e", "metadata": { "editable": true }, @@ -880,7 +881,7 @@ }, { "cell_type": "markdown", - "id": "637bd194", + "id": "a92ffe46", "metadata": { "editable": true }, @@ -893,7 +894,7 @@ }, { "cell_type": "markdown", - "id": "0792109c", + "id": "32e2777e", "metadata": { "editable": true }, @@ -903,7 +904,7 @@ }, { "cell_type": "markdown", - "id": "10f14691", + "id": "136aa72b", "metadata": { "editable": true }, @@ -915,7 +916,7 @@ }, { "cell_type": "markdown", - "id": "186529b5", + "id": "ce0fb4b1", "metadata": { "editable": true }, @@ -925,7 +926,7 @@ }, { "cell_type": "markdown", - "id": "68afc0da", + "id": "aa242196", "metadata": { "editable": true }, @@ -937,7 +938,7 @@ }, { "cell_type": "markdown", - "id": "cf48ea60", + "id": "655fa566", "metadata": { "editable": true }, @@ -947,7 +948,7 @@ }, { "cell_type": "markdown", - "id": "861a3bbe", + "id": "e3e368cd", "metadata": { "editable": true }, @@ -971,7 +972,7 @@ }, { "cell_type": "markdown", - "id": "c8007ce7", + "id": "b4739d6a", "metadata": { "editable": true }, @@ -1021,7 +1022,7 @@ { "cell_type": "code", "execution_count": 1, - "id": "29678f97", + "id": "bb8dc476", "metadata": { "collapsed": false, "editable": true @@ -1076,7 +1077,7 @@ }, { "cell_type": "markdown", - "id": "3ba6d7d6", + "id": "848728d9", "metadata": { "editable": true }, @@ -1097,7 +1098,7 @@ { "cell_type": "code", "execution_count": 2, - "id": "7df51dd1", + "id": "84b61a0e", "metadata": { "collapsed": false, "editable": true @@ -1135,7 +1136,7 @@ }, { "cell_type": "markdown", - "id": "249898fd", + "id": "9e7816f1", "metadata": { "editable": true }, @@ -1179,7 +1180,7 @@ }, { "cell_type": "markdown", - "id": "2d30ddd3", + "id": "8298469d", "metadata": { "editable": true }, @@ -1219,7 +1220,7 @@ }, { "cell_type": "markdown", - "id": "ce9f14e4", + "id": "8d56cebc", "metadata": { "editable": true }, @@ -1240,7 +1241,7 @@ { "cell_type": "code", "execution_count": 3, - "id": "c3f75b32", + "id": "bf84cbc3", "metadata": { "collapsed": false, "editable": true @@ -1266,7 +1267,7 @@ }, { "cell_type": "markdown", - "id": "4ebf4f68", + "id": "fdb7502c", "metadata": { "editable": true }, @@ -1294,7 +1295,7 @@ }, { "cell_type": "markdown", - "id": "c6760d1f", + "id": "6d645fac", "metadata": { "editable": true }, @@ -1331,7 +1332,7 @@ { "cell_type": "code", "execution_count": 4, - "id": "98f3a5d5", + "id": "9c60a3f7", "metadata": { "collapsed": false, "editable": true @@ -1377,7 +1378,7 @@ }, { "cell_type": "markdown", - "id": "19cd49ec", + "id": "611754b9", "metadata": { "editable": true }, @@ -1408,7 +1409,7 @@ }, { "cell_type": "markdown", - "id": "b0342f54", + "id": "387ee0bf", "metadata": { "editable": true }, @@ -1446,7 +1447,7 @@ }, { "cell_type": "markdown", - "id": "7219731b", + "id": "df14864b", "metadata": { "editable": true }, @@ -1480,7 +1481,7 @@ }, { "cell_type": "markdown", - "id": "248590b9", + "id": "b3fec075", "metadata": { "editable": true }, @@ -1521,7 +1522,7 @@ { "cell_type": "code", "execution_count": 5, - "id": "1810a36c", + "id": "b2d49c0e", "metadata": { "collapsed": false, "editable": true @@ -1600,7 +1601,7 @@ }, { "cell_type": "markdown", - "id": "e094a404", + "id": "955515ec", "metadata": { "editable": true }, @@ -1621,7 +1622,7 @@ }, { "cell_type": "markdown", - "id": "1fb3aab5", + "id": "a91e81d4", "metadata": { "editable": true }, @@ -1635,7 +1636,7 @@ { "cell_type": "code", "execution_count": 6, - "id": "20b3c187", + "id": "6e5bfd57", "metadata": { "collapsed": false, "editable": true @@ -1745,7 +1746,7 @@ }, { "cell_type": "markdown", - "id": "f85564ae", + "id": "5b9ad311", "metadata": { "editable": true }, @@ -1764,7 +1765,7 @@ { "cell_type": "code", "execution_count": 7, - "id": "9be7082e", + "id": "74c2f3d3", "metadata": { "collapsed": false, "editable": true @@ -1791,7 +1792,7 @@ }, { "cell_type": "markdown", - "id": "255b5b3d", + "id": "3c32426e", "metadata": { "editable": true }, @@ -1805,7 +1806,7 @@ { "cell_type": "code", "execution_count": 8, - "id": "fc014642", + "id": "6bd66e9e", "metadata": { "collapsed": false, "editable": true @@ -1836,7 +1837,7 @@ }, { "cell_type": "markdown", - "id": "96089af8", + "id": "9510511f", "metadata": { "editable": true }, @@ -1847,7 +1848,7 @@ { "cell_type": "code", "execution_count": 9, - "id": "04c4d29a", + "id": "f97645b0", "metadata": { "collapsed": false, "editable": true @@ -1891,7 +1892,7 @@ }, { "cell_type": "markdown", - "id": "472a5060", + "id": "ac92ed55", "metadata": { "editable": true }, @@ -1914,7 +1915,7 @@ { "cell_type": "code", "execution_count": 10, - "id": "42e40777", + "id": "cc76da2f", "metadata": { "collapsed": false, "editable": true @@ -1941,7 +1942,7 @@ }, { "cell_type": "markdown", - "id": "57fc5d93", + "id": "4b34882e", "metadata": { "editable": true }, @@ -1952,7 +1953,7 @@ { "cell_type": "code", "execution_count": 11, - "id": "90477fe7", + "id": "b0349437", "metadata": { "collapsed": false, "editable": true @@ -1997,7 +1998,7 @@ }, { "cell_type": "markdown", - "id": "29af8aff", + "id": "3dc3d83e", "metadata": { "editable": true }, @@ -2015,7 +2016,7 @@ }, { "cell_type": "markdown", - "id": "a70cf999", + "id": "530560b4", "metadata": { "editable": true }, @@ -2050,7 +2051,7 @@ { "cell_type": "code", "execution_count": 12, - "id": "812c98fe", + "id": "705deef0", "metadata": { "collapsed": false, "editable": true @@ -2062,7 +2063,7 @@ }, { "cell_type": "markdown", - "id": "694c0bd7", + "id": "4c72975c", "metadata": { "editable": true }, @@ -2074,7 +2075,7 @@ { "cell_type": "code", "execution_count": 13, - "id": "1ecd18cc", + "id": "42580269", "metadata": { "collapsed": false, "editable": true @@ -2087,7 +2088,7 @@ }, { "cell_type": "markdown", - "id": "584ab0ab", + "id": "0b2fd909", "metadata": { "editable": true }, @@ -2098,7 +2099,7 @@ { "cell_type": "code", "execution_count": 14, - "id": "a811e23d", + "id": "e8f23bf0", "metadata": { "collapsed": false, "editable": true @@ -2111,7 +2112,7 @@ }, { "cell_type": "markdown", - "id": "ab99008f", + "id": "22efaba3", "metadata": { "editable": true }, @@ -2126,7 +2127,7 @@ { "cell_type": "code", "execution_count": 15, - "id": "fe547415", + "id": "fa995d4c", "metadata": { "collapsed": false, "editable": true @@ -2138,7 +2139,7 @@ }, { "cell_type": "markdown", - "id": "ed1e0fc6", + "id": "d7e17232", "metadata": { "editable": true }, @@ -2150,7 +2151,7 @@ }, { "cell_type": "markdown", - "id": "6aebcb51", + "id": "18ffc9ed", "metadata": { "editable": true }, @@ -2163,7 +2164,7 @@ { "cell_type": "code", "execution_count": 16, - "id": "a4a7d68d", + "id": "50838e03", "metadata": { "collapsed": false, "editable": true @@ -2218,7 +2219,7 @@ { "cell_type": "code", "execution_count": 17, - "id": "21335418", + "id": "ab5f30fe", "metadata": { "collapsed": false, "editable": true @@ -2247,7 +2248,7 @@ { "cell_type": "code", "execution_count": 18, - "id": "e81d58de", + "id": "30fff6f5", "metadata": { "collapsed": false, "editable": true @@ -2277,7 +2278,7 @@ { "cell_type": "code", "execution_count": 19, - "id": "b91287cc", + "id": "9e412c91", "metadata": { "collapsed": false, "editable": true @@ -2304,7 +2305,7 @@ { "cell_type": "code", "execution_count": 20, - "id": "d108674e", + "id": "329f11e7", "metadata": { "collapsed": false, "editable": true @@ -2346,7 +2347,7 @@ }, { "cell_type": "markdown", - "id": "1198d799", + "id": "041d733f", "metadata": { "editable": true }, @@ -2357,7 +2358,7 @@ { "cell_type": "code", "execution_count": 21, - "id": "eb1eb4fb", + "id": "a6adc86f", "metadata": { "collapsed": false, "editable": true @@ -2534,7 +2535,7 @@ }, { "cell_type": "markdown", - "id": "92834502", + "id": "bde196e8", "metadata": { "editable": true }, @@ -2553,7 +2554,7 @@ }, { "cell_type": "markdown", - "id": "deb24cc1", + "id": "529854a1", "metadata": { "editable": true }, @@ -2575,7 +2576,7 @@ { "cell_type": "code", "execution_count": 22, - "id": "d3619281", + "id": "d62e4240", "metadata": { "collapsed": false, "editable": true @@ -2716,7 +2717,7 @@ }, { "cell_type": "markdown", - "id": "f37fd7bc", + "id": "3d25b135", "metadata": { "editable": true }, @@ -2732,7 +2733,7 @@ { "cell_type": "code", "execution_count": 23, - "id": "5db34b9b", + "id": "678df2fb", "metadata": { "collapsed": false, "editable": true @@ -2745,7 +2746,7 @@ }, { "cell_type": "markdown", - "id": "8b25a24d", + "id": "e781631f", "metadata": { "editable": true }, @@ -2757,7 +2758,7 @@ { "cell_type": "code", "execution_count": 24, - "id": "2d5f2887", + "id": "18f1eb25", "metadata": { "collapsed": false, "editable": true @@ -2779,7 +2780,7 @@ }, { "cell_type": "markdown", - "id": "9070c2d3", + "id": "a667e0e3", "metadata": { "editable": true }, @@ -2795,7 +2796,7 @@ { "cell_type": "code", "execution_count": 25, - "id": "c360c736", + "id": "7c8aa0cf", "metadata": { "collapsed": false, "editable": true @@ -2833,7 +2834,7 @@ }, { "cell_type": "markdown", - "id": "42c7ec61", + "id": "00df836d", "metadata": { "editable": true }, @@ -2846,7 +2847,7 @@ { "cell_type": "code", "execution_count": 26, - "id": "cd9cb3a5", + "id": "5a9d9d29", "metadata": { "collapsed": false, "editable": true @@ -2867,7 +2868,7 @@ }, { "cell_type": "markdown", - "id": "796aab4a", + "id": "dfd4d270", "metadata": { "editable": true }, @@ -2883,7 +2884,7 @@ { "cell_type": "code", "execution_count": 27, - "id": "5ea4891d", + "id": "e7f564f3", "metadata": { "collapsed": false, "editable": true @@ -2941,7 +2942,7 @@ }, { "cell_type": "markdown", - "id": "2ff3c57c", + "id": "ea015309", "metadata": { "editable": true }, @@ -2956,7 +2957,7 @@ { "cell_type": "code", "execution_count": 28, - "id": "34289210", + "id": "feed7401", "metadata": { "collapsed": false, "editable": true @@ -2977,7 +2978,7 @@ }, { "cell_type": "markdown", - "id": "0839f98b", + "id": "1b904924", "metadata": { "editable": true }, @@ -3001,7 +3002,7 @@ { "cell_type": "code", "execution_count": 29, - "id": "0fdc2707", + "id": "41311a29", "metadata": { "collapsed": false, "editable": true @@ -3473,7 +3474,7 @@ }, { "cell_type": "markdown", - "id": "17bc192e", + "id": "ed77ef04", "metadata": { "editable": true }, @@ -3485,7 +3486,7 @@ { "cell_type": "code", "execution_count": 30, - "id": "f3f5d088", + "id": "58ad0c6a", "metadata": { "collapsed": false, "editable": true @@ -3529,7 +3530,7 @@ }, { "cell_type": "markdown", - "id": "53c3b4c2", + "id": "d277ed22", "metadata": { "editable": true }, @@ -3545,7 +3546,7 @@ { "cell_type": "code", "execution_count": 31, - "id": "890d682f", + "id": "a8a25f75", "metadata": { "collapsed": false, "editable": true @@ -3560,7 +3561,7 @@ }, { "cell_type": "markdown", - "id": "dc4da852", + "id": "de607468", "metadata": { "editable": true }, @@ -3571,7 +3572,7 @@ { "cell_type": "code", "execution_count": 32, - "id": "1e98a3f8", + "id": "1031d34f", "metadata": { "collapsed": false, "editable": true @@ -3586,7 +3587,7 @@ }, { "cell_type": "markdown", - "id": "68c5d793", + "id": "122da85d", "metadata": { "editable": true }, @@ -3602,7 +3603,7 @@ { "cell_type": "code", "execution_count": 33, - "id": "e735086e", + "id": "3e88b81a", "metadata": { "collapsed": false, "editable": true @@ -3616,7 +3617,7 @@ }, { "cell_type": "markdown", - "id": "360468f6", + "id": "cbb0c747", "metadata": { "editable": true }, @@ -3631,7 +3632,7 @@ { "cell_type": "code", "execution_count": 34, - "id": "c5937c59", + "id": "9241dfe5", "metadata": { "collapsed": false, "editable": true @@ -3657,7 +3658,7 @@ { "cell_type": "code", "execution_count": 35, - "id": "2e06b929", + "id": "81608731", "metadata": { "collapsed": false, "editable": true @@ -3672,7 +3673,7 @@ }, { "cell_type": "markdown", - "id": "94140fb1", + "id": "2e59060d", "metadata": { "editable": true }, @@ -3683,7 +3684,7 @@ { "cell_type": "code", "execution_count": 36, - "id": "da82b266", + "id": "73d42465", "metadata": { "collapsed": false, "editable": true @@ -3698,7 +3699,7 @@ }, { "cell_type": "markdown", - "id": "19e8d4c2", + "id": "05b7ba07", "metadata": { "editable": true }, @@ -3709,7 +3710,7 @@ { "cell_type": "code", "execution_count": 37, - "id": "4744accd", + "id": "a92ddbbd", "metadata": { "collapsed": false, "editable": true @@ -3729,7 +3730,7 @@ { "cell_type": "code", "execution_count": 38, - "id": "f5b5b198", + "id": "624c51a0", "metadata": { "collapsed": false, "editable": true @@ -3744,7 +3745,7 @@ }, { "cell_type": "markdown", - "id": "1f5fc0a0", + "id": "b504ca17", "metadata": { "editable": true }, @@ -3759,7 +3760,7 @@ { "cell_type": "code", "execution_count": 39, - "id": "4308d82f", + "id": "9b943e8a", "metadata": { "collapsed": false, "editable": true @@ -3796,7 +3797,7 @@ }, { "cell_type": "markdown", - "id": "96fc3508", + "id": "72eb50b2", "metadata": { "editable": true }, @@ -3809,7 +3810,7 @@ { "cell_type": "code", "execution_count": 40, - "id": "5aa9a2a4", + "id": "357b7b74", "metadata": { "collapsed": false, "editable": true @@ -3832,7 +3833,7 @@ }, { "cell_type": "markdown", - "id": "d57346ae", + "id": "30932365", "metadata": { "editable": true }, @@ -3842,7 +3843,7 @@ }, { "cell_type": "markdown", - "id": "0219b252", + "id": "7d24fb51", "metadata": { "editable": true }, @@ -3869,7 +3870,7 @@ }, { "cell_type": "markdown", - "id": "80032cfc", + "id": "cbc97579", "metadata": { "editable": true }, @@ -3883,7 +3884,7 @@ }, { "cell_type": "markdown", - "id": "3c5546df", + "id": "47d2f0fd", "metadata": { "editable": true }, @@ -3900,7 +3901,7 @@ }, { "cell_type": "markdown", - "id": "e4f540a5", + "id": "f2699119", "metadata": { "editable": true }, @@ -3916,7 +3917,7 @@ }, { "cell_type": "markdown", - "id": "75ec2cfe", + "id": "91345435", "metadata": { "editable": true }, @@ -3928,7 +3929,7 @@ }, { "cell_type": "markdown", - "id": "a71e7eb8", + "id": "c0649f33", "metadata": { "editable": true }, @@ -3946,7 +3947,7 @@ }, { "cell_type": "markdown", - "id": "9b06d2bf", + "id": "df06e51e", "metadata": { "editable": true }, @@ -3967,7 +3968,7 @@ }, { "cell_type": "markdown", - "id": "0aea2f3e", + "id": "b4ae5502", "metadata": { "editable": true }, @@ -3984,7 +3985,7 @@ }, { "cell_type": "markdown", - "id": "a21c6a43", + "id": "5fd35302", "metadata": { "editable": true }, @@ -3996,7 +3997,7 @@ }, { "cell_type": "markdown", - "id": "ff939582", + "id": "3297342f", "metadata": { "editable": true }, @@ -4007,7 +4008,7 @@ }, { "cell_type": "markdown", - "id": "9f63085a", + "id": "7c4c10e4", "metadata": { "editable": true }, @@ -4024,7 +4025,7 @@ }, { "cell_type": "markdown", - "id": "2875658a", + "id": "12a6cafe", "metadata": { "editable": true }, @@ -4035,7 +4036,7 @@ }, { "cell_type": "markdown", - "id": "8a63e4c6", + "id": "d37498d3", "metadata": { "editable": true }, @@ -4051,7 +4052,7 @@ }, { "cell_type": "markdown", - "id": "2d5b4c88", + "id": "f796df2e", "metadata": { "editable": true }, @@ -4063,7 +4064,7 @@ }, { "cell_type": "markdown", - "id": "daab1d5e", + "id": "8c2d8264", "metadata": { "editable": true }, @@ -4080,7 +4081,7 @@ }, { "cell_type": "markdown", - "id": "fc2cedc3", + "id": "a34cabe4", "metadata": { "editable": true }, @@ -4092,7 +4093,7 @@ }, { "cell_type": "markdown", - "id": "da284bc1", + "id": "1cc244db", "metadata": { "editable": true }, @@ -4110,7 +4111,7 @@ }, { "cell_type": "markdown", - "id": "04c2531a", + "id": "1254829f", "metadata": { "editable": true }, @@ -4120,7 +4121,7 @@ }, { "cell_type": "markdown", - "id": "62a110db", + "id": "e34c331d", "metadata": { "editable": true }, @@ -4132,7 +4133,7 @@ }, { "cell_type": "markdown", - "id": "7b5b722c", + "id": "9f37fc40", "metadata": { "editable": true }, @@ -4149,7 +4150,7 @@ }, { "cell_type": "markdown", - "id": "02446b0e", + "id": "383957ac", "metadata": { "editable": true }, @@ -4161,7 +4162,7 @@ }, { "cell_type": "markdown", - "id": "205c8e89", + "id": "f499a7d2", "metadata": { "editable": true }, @@ -4172,7 +4173,7 @@ }, { "cell_type": "markdown", - "id": "9e46aee1", + "id": "c6851600", "metadata": { "editable": true }, @@ -4184,7 +4185,7 @@ }, { "cell_type": "markdown", - "id": "cb0d1f30", + "id": "8f4b658f", "metadata": { "editable": true }, @@ -4194,7 +4195,7 @@ }, { "cell_type": "markdown", - "id": "8485bdd3", + "id": "2fb6d7fa", "metadata": { "editable": true }, @@ -4214,7 +4215,7 @@ }, { "cell_type": "markdown", - "id": "730e057b", + "id": "5c6b5421", "metadata": { "editable": true }, @@ -4231,7 +4232,7 @@ }, { "cell_type": "markdown", - "id": "db8907ef", + "id": "57d5ee64", "metadata": { "editable": true }, @@ -4250,7 +4251,7 @@ }, { "cell_type": "markdown", - "id": "ce4a7022", + "id": "05af3ca9", "metadata": { "editable": true }, @@ -4262,7 +4263,7 @@ }, { "cell_type": "markdown", - "id": "b17ab00d", + "id": "42b535c2", "metadata": { "editable": true }, @@ -4272,7 +4273,7 @@ }, { "cell_type": "markdown", - "id": "f4af75e0", + "id": "4ea2bfdd", "metadata": { "editable": true }, @@ -4289,7 +4290,7 @@ }, { "cell_type": "markdown", - "id": "4ea72b19", + "id": "9ae091e4", "metadata": { "editable": true }, @@ -4299,7 +4300,7 @@ }, { "cell_type": "markdown", - "id": "428cf327", + "id": "31523397", "metadata": { "editable": true }, @@ -4315,7 +4316,7 @@ }, { "cell_type": "markdown", - "id": "1b4c9872", + "id": "7ee5ff24", "metadata": { "editable": true }, @@ -4327,7 +4328,7 @@ }, { "cell_type": "markdown", - "id": "f12896dd", + "id": "20529482", "metadata": { "editable": true }, @@ -4339,7 +4340,7 @@ }, { "cell_type": "markdown", - "id": "70e2187e", + "id": "2ded9ab7", "metadata": { "editable": true }, @@ -4351,7 +4352,7 @@ }, { "cell_type": "markdown", - "id": "e0136d5f", + "id": "618955a1", "metadata": { "editable": true }, @@ -4361,7 +4362,7 @@ }, { "cell_type": "markdown", - "id": "ff915c59", + "id": "05348619", "metadata": { "editable": true }, @@ -4373,7 +4374,7 @@ }, { "cell_type": "markdown", - "id": "ca46667e", + "id": "d81308ad", "metadata": { "editable": true }, @@ -4390,7 +4391,7 @@ }, { "cell_type": "markdown", - "id": "3efa1dc6", + "id": "30ee97f0", "metadata": { "editable": true }, @@ -4400,7 +4401,7 @@ }, { "cell_type": "markdown", - "id": "a3d800e8", + "id": "35d14495", "metadata": { "editable": true }, @@ -4412,7 +4413,7 @@ }, { "cell_type": "markdown", - "id": "8fb679f5", + "id": "fbb476e9", "metadata": { "editable": true }, @@ -4426,7 +4427,7 @@ }, { "cell_type": "markdown", - "id": "08b2a19b", + "id": "20410893", "metadata": { "editable": true }, @@ -4442,7 +4443,7 @@ }, { "cell_type": "markdown", - "id": "082074c8", + "id": "fc068610", "metadata": { "editable": true }, @@ -4454,7 +4455,7 @@ }, { "cell_type": "markdown", - "id": "b69f9148", + "id": "6e9e408e", "metadata": { "editable": true }, @@ -4476,7 +4477,7 @@ }, { "cell_type": "markdown", - "id": "9feeefde", + "id": "a9121a15", "metadata": { "editable": true }, @@ -4488,7 +4489,7 @@ }, { "cell_type": "markdown", - "id": "cde763a7", + "id": "df8862c4", "metadata": { "editable": true }, @@ -4511,7 +4512,7 @@ }, { "cell_type": "markdown", - "id": "3c83e5dc", + "id": "92846a59", "metadata": { "editable": true }, @@ -4527,7 +4528,7 @@ }, { "cell_type": "markdown", - "id": "4566531c", + "id": "aa3326c2", "metadata": { "editable": true }, @@ -4539,7 +4540,7 @@ }, { "cell_type": "markdown", - "id": "e316d1e7", + "id": "394a6ab1", "metadata": { "editable": true }, @@ -4563,7 +4564,7 @@ }, { "cell_type": "markdown", - "id": "41ac9a1c", + "id": "f8caaf7d", "metadata": { "editable": true }, @@ -4575,7 +4576,7 @@ }, { "cell_type": "markdown", - "id": "b0a8a3c2", + "id": "2053f828", "metadata": { "editable": true }, @@ -4596,7 +4597,7 @@ }, { "cell_type": "markdown", - "id": "67447b23", + "id": "0f367431", "metadata": { "editable": true }, @@ -4608,7 +4609,7 @@ }, { "cell_type": "markdown", - "id": "12d3e1ee", + "id": "3b451325", "metadata": { "editable": true }, @@ -4627,7 +4628,7 @@ }, { "cell_type": "markdown", - "id": "562a122c", + "id": "a92a67ee", "metadata": { "editable": true }, @@ -4637,7 +4638,7 @@ }, { "cell_type": "markdown", - "id": "78789a96", + "id": "bba24aa2", "metadata": { "editable": true }, @@ -4651,7 +4652,7 @@ }, { "cell_type": "markdown", - "id": "76b734fb", + "id": "53fb58d4", "metadata": { "editable": true }, @@ -4663,7 +4664,7 @@ }, { "cell_type": "markdown", - "id": "e2911f97", + "id": "490c09a0", "metadata": { "editable": true }, @@ -4675,7 +4676,7 @@ }, { "cell_type": "markdown", - "id": "5e0c281c", + "id": "90b6bfb0", "metadata": { "editable": true }, @@ -4692,7 +4693,7 @@ }, { "cell_type": "markdown", - "id": "077e3318", + "id": "285ccc48", "metadata": { "editable": true }, @@ -4704,7 +4705,7 @@ }, { "cell_type": "markdown", - "id": "b6e9387d", + "id": "94baf8e8", "metadata": { "editable": true }, @@ -4726,7 +4727,7 @@ }, { "cell_type": "markdown", - "id": "3fa21e60", + "id": "12025356", "metadata": { "editable": true }, @@ -4741,7 +4742,7 @@ }, { "cell_type": "markdown", - "id": "e05e6028", + "id": "42565de6", "metadata": { "editable": true }, @@ -4752,7 +4753,7 @@ { "cell_type": "code", "execution_count": 41, - "id": "eb77d1cf", + "id": "695b8df0", "metadata": { "collapsed": false, "editable": true @@ -4907,7 +4908,7 @@ }, { "cell_type": "markdown", - "id": "bbd6562b", + "id": "caaa6bf9", "metadata": { "editable": true }, @@ -4922,7 +4923,7 @@ { "cell_type": "code", "execution_count": 42, - "id": "24ba1afd", + "id": "2481727a", "metadata": { "collapsed": false, "editable": true @@ -5091,7 +5092,7 @@ }, { "cell_type": "markdown", - "id": "3974040a", + "id": "94e67eb6", "metadata": { "editable": true }, @@ -5104,7 +5105,7 @@ }, { "cell_type": "markdown", - "id": "a9d203ea", + "id": "3c847d14", "metadata": { "editable": true }, @@ -5121,7 +5122,7 @@ }, { "cell_type": "markdown", - "id": "ede35f92", + "id": "88c997db", "metadata": { "editable": true }, @@ -5137,7 +5138,7 @@ }, { "cell_type": "markdown", - "id": "a9720dfc", + "id": "3ee360d6", "metadata": { "editable": true }, @@ -5150,7 +5151,7 @@ }, { "cell_type": "markdown", - "id": "5ef6b555", + "id": "915df9b1", "metadata": { "editable": true }, @@ -5167,7 +5168,7 @@ }, { "cell_type": "markdown", - "id": "bd5c646e", + "id": "3cce06b5", "metadata": { "editable": true }, @@ -5179,7 +5180,7 @@ }, { "cell_type": "markdown", - "id": "cdc5561b", + "id": "164a26c3", "metadata": { "editable": true }, @@ -5206,7 +5207,7 @@ }, { "cell_type": "markdown", - "id": "cebde164", + "id": "a588ddab", "metadata": { "editable": true }, @@ -5219,7 +5220,7 @@ { "cell_type": "code", "execution_count": 43, - "id": "c71a9592", + "id": "76252b3a", "metadata": { "collapsed": false, "editable": true @@ -5398,7 +5399,7 @@ }, { "cell_type": "markdown", - "id": "bd70d6cc", + "id": "2e9f770a", "metadata": { "editable": true }, @@ -5418,7 +5419,7 @@ }, { "cell_type": "markdown", - "id": "ee3bdd02", + "id": "6613bf81", "metadata": { "editable": true }, @@ -5433,7 +5434,7 @@ }, { "cell_type": "markdown", - "id": "d81b6054", + "id": "4f2611c5", "metadata": { "editable": true }, @@ -5447,7 +5448,7 @@ }, { "cell_type": "markdown", - "id": "0d7a2272", + "id": "fbbfb78c", "metadata": { "editable": true }, @@ -5463,7 +5464,7 @@ }, { "cell_type": "markdown", - "id": "ca29846f", + "id": "fede632e", "metadata": { "editable": true }, @@ -5473,7 +5474,7 @@ }, { "cell_type": "markdown", - "id": "1a51d849", + "id": "63169a3e", "metadata": { "editable": true }, @@ -5495,7 +5496,7 @@ }, { "cell_type": "markdown", - "id": "700a9bac", + "id": "1638bb66", "metadata": { "editable": true }, @@ -5509,7 +5510,7 @@ { "cell_type": "code", "execution_count": 44, - "id": "482cf93c", + "id": "805cdc28", "metadata": { "collapsed": false, "editable": true @@ -5585,7 +5586,7 @@ }, { "cell_type": "markdown", - "id": "4ed3cefd", + "id": "b958d38c", "metadata": { "editable": true }, @@ -5597,7 +5598,7 @@ }, { "cell_type": "markdown", - "id": "d0bdfcdb", + "id": "1208716b", "metadata": { "editable": true }, @@ -5614,7 +5615,7 @@ }, { "cell_type": "markdown", - "id": "656240e7", + "id": "9cd38ea9", "metadata": { "editable": true }, @@ -5626,7 +5627,7 @@ }, { "cell_type": "markdown", - "id": "fb310d7c", + "id": "9694bb51", "metadata": { "editable": true }, @@ -5641,7 +5642,7 @@ }, { "cell_type": "markdown", - "id": "e6b102dc", + "id": "6e07281b", "metadata": { "editable": true }, @@ -5653,7 +5654,7 @@ }, { "cell_type": "markdown", - "id": "797d4bb8", + "id": "db57a833", "metadata": { "editable": true }, @@ -5665,7 +5666,7 @@ }, { "cell_type": "markdown", - "id": "da2e90b8", + "id": "ebdeaa75", "metadata": { "editable": true }, @@ -5677,7 +5678,7 @@ }, { "cell_type": "markdown", - "id": "318f8f3a", + "id": "5f930649", "metadata": { "editable": true }, @@ -5687,7 +5688,7 @@ }, { "cell_type": "markdown", - "id": "c5e8ac9e", + "id": "f7e8ffc6", "metadata": { "editable": true }, @@ -5704,7 +5705,7 @@ }, { "cell_type": "markdown", - "id": "220ceeb4", + "id": "8cfec0e6", "metadata": { "editable": true }, @@ -5716,7 +5717,7 @@ }, { "cell_type": "markdown", - "id": "ec8b38dc", + "id": "1fc6e392", "metadata": { "editable": true }, @@ -5728,7 +5729,7 @@ }, { "cell_type": "markdown", - "id": "0ed927a9", + "id": "8aba4caa", "metadata": { "editable": true }, @@ -5738,7 +5739,7 @@ }, { "cell_type": "markdown", - "id": "3a51ddcc", + "id": "fd018426", "metadata": { "editable": true }, @@ -5750,7 +5751,7 @@ }, { "cell_type": "markdown", - "id": "d5392b8d", + "id": "44764933", "metadata": { "editable": true }, @@ -5761,7 +5762,7 @@ { "cell_type": "code", "execution_count": 45, - "id": "95059f71", + "id": "76170788", "metadata": { "collapsed": false, "editable": true @@ -5926,7 +5927,7 @@ }, { "cell_type": "markdown", - "id": "7c568bf3", + "id": "afe64d3c", "metadata": { "editable": true }, @@ -5948,7 +5949,7 @@ }, { "cell_type": "markdown", - "id": "eab0bb68", + "id": "b579efba", "metadata": { "editable": true }, @@ -5965,7 +5966,7 @@ }, { "cell_type": "markdown", - "id": "c0ada0c0", + "id": "1f9afc2e", "metadata": { "editable": true }, @@ -5975,7 +5976,7 @@ }, { "cell_type": "markdown", - "id": "b483eace", + "id": "cdc8be09", "metadata": { "editable": true }, @@ -5990,7 +5991,7 @@ }, { "cell_type": "markdown", - "id": "30d45122", + "id": "5955fec0", "metadata": { "editable": true }, @@ -6000,7 +6001,7 @@ }, { "cell_type": "markdown", - "id": "52c68e05", + "id": "ce2797bd", "metadata": { "editable": true }, @@ -6015,7 +6016,7 @@ }, { "cell_type": "markdown", - "id": "1657934d", + "id": "dd635489", "metadata": { "editable": true }, @@ -6026,7 +6027,7 @@ }, { "cell_type": "markdown", - "id": "9fc46a1b", + "id": "2eddadc9", "metadata": { "editable": true }, @@ -6046,7 +6047,7 @@ }, { "cell_type": "markdown", - "id": "cd79f75c", + "id": "65a61481", "metadata": { "editable": true }, @@ -6058,7 +6059,7 @@ }, { "cell_type": "markdown", - "id": "501ac880", + "id": "7013915a", "metadata": { "editable": true }, @@ -6095,7 +6096,7 @@ }, { "cell_type": "markdown", - "id": "8551615b", + "id": "6b5d013b", "metadata": { "editable": true }, @@ -6105,7 +6106,7 @@ }, { "cell_type": "markdown", - "id": "dbd6d2a1", + "id": "3877834e", "metadata": { "editable": true }, @@ -6118,7 +6119,7 @@ { "cell_type": "code", "execution_count": 46, - "id": "c71a3063", + "id": "7e63f565", "metadata": { "collapsed": false, "editable": true @@ -6323,7 +6324,7 @@ }, { "cell_type": "markdown", - "id": "da58cdec", + "id": "2592748f", "metadata": { "editable": true }, @@ -6340,7 +6341,7 @@ }, { "cell_type": "markdown", - "id": "df991968", + "id": "f76a2424", "metadata": { "editable": true }, @@ -6357,7 +6358,7 @@ }, { "cell_type": "markdown", - "id": "59ef3099", + "id": "06bb4068", "metadata": { "editable": true }, @@ -6367,7 +6368,7 @@ }, { "cell_type": "markdown", - "id": "a580ae0f", + "id": "cf449250", "metadata": { "editable": true }, @@ -6382,7 +6383,7 @@ }, { "cell_type": "markdown", - "id": "a56f3fe1", + "id": "72fe7bb2", "metadata": { "editable": true }, @@ -6396,7 +6397,7 @@ }, { "cell_type": "markdown", - "id": "69586714", + "id": "97d43e59", "metadata": { "editable": true }, @@ -6409,7 +6410,7 @@ }, { "cell_type": "markdown", - "id": "d8374dce", + "id": "26ed608f", "metadata": { "editable": true }, @@ -6429,7 +6430,7 @@ }, { "cell_type": "markdown", - "id": "f3fe961e", + "id": "ae2b2d38", "metadata": { "editable": true }, @@ -6441,7 +6442,7 @@ }, { "cell_type": "markdown", - "id": "4e453aa1", + "id": "0bc085b8", "metadata": { "editable": true }, @@ -6453,7 +6454,7 @@ }, { "cell_type": "markdown", - "id": "e1ebf5bd", + "id": "1634a37d", "metadata": { "editable": true }, @@ -6465,7 +6466,7 @@ }, { "cell_type": "markdown", - "id": "b8c35553", + "id": "4700b147", "metadata": { "editable": true }, @@ -6475,7 +6476,7 @@ }, { "cell_type": "markdown", - "id": "22a123d5", + "id": "07da99c0", "metadata": { "editable": true }, @@ -6487,7 +6488,7 @@ }, { "cell_type": "markdown", - "id": "b15eedf5", + "id": "7d3e9d8a", "metadata": { "editable": true }, @@ -6499,7 +6500,7 @@ }, { "cell_type": "markdown", - "id": "1a92cc8c", + "id": "77e9e06c", "metadata": { "editable": true }, @@ -6511,7 +6512,7 @@ }, { "cell_type": "markdown", - "id": "e4862313", + "id": "ab19ecd2", "metadata": { "editable": true }, @@ -6521,7 +6522,7 @@ }, { "cell_type": "markdown", - "id": "5c186d68", + "id": "31ff9094", "metadata": { "editable": true }, @@ -6537,7 +6538,7 @@ }, { "cell_type": "markdown", - "id": "57d4f000", + "id": "3896b229", "metadata": { "editable": true }, @@ -6547,7 +6548,7 @@ }, { "cell_type": "markdown", - "id": "7c2cdc6b", + "id": "40d97099", "metadata": { "editable": true }, @@ -6559,7 +6560,7 @@ }, { "cell_type": "markdown", - "id": "bde064ea", + "id": "32cda23a", "metadata": { "editable": true }, @@ -6576,7 +6577,7 @@ }, { "cell_type": "markdown", - "id": "7a558c97", + "id": "140ba57b", "metadata": { "editable": true }, @@ -6586,7 +6587,7 @@ }, { "cell_type": "markdown", - "id": "534a4844", + "id": "68834674", "metadata": { "editable": true }, @@ -6602,7 +6603,7 @@ }, { "cell_type": "markdown", - "id": "a07e0730", + "id": "198be6e0", "metadata": { "editable": true }, @@ -6616,7 +6617,7 @@ }, { "cell_type": "markdown", - "id": "74b72286", + "id": "e56f85f9", "metadata": { "editable": true }, @@ -6635,7 +6636,7 @@ { "cell_type": "code", "execution_count": 47, - "id": "4a0c57e9", + "id": "4ee57304", "metadata": { "collapsed": false, "editable": true @@ -6690,7 +6691,7 @@ }, { "cell_type": "markdown", - "id": "350b1d3d", + "id": "0088cd21", "metadata": { "editable": true }, @@ -6720,7 +6721,7 @@ }, { "cell_type": "markdown", - "id": "9fd438b5", + "id": "c422d21e", "metadata": { "editable": true }, @@ -6749,7 +6750,7 @@ { "cell_type": "code", "execution_count": 48, - "id": "d1c81690", + "id": "86acdd1c", "metadata": { "collapsed": false, "editable": true @@ -6796,7 +6797,7 @@ }, { "cell_type": "markdown", - "id": "ca22d9ce", + "id": "6097ee56", "metadata": { "editable": true }, @@ -6822,7 +6823,7 @@ { "cell_type": "code", "execution_count": 49, - "id": "5a9146bc", + "id": "8cf456d6", "metadata": { "collapsed": false, "editable": true @@ -7056,7 +7057,7 @@ }, { "cell_type": "markdown", - "id": "e59d2116", + "id": "1f09e18d", "metadata": { "editable": true }, @@ -7068,7 +7069,7 @@ }, { "cell_type": "markdown", - "id": "e2ffdbaf", + "id": "2f8d1764", "metadata": { "editable": true }, @@ -7080,7 +7081,7 @@ }, { "cell_type": "markdown", - "id": "37b2cc03", + "id": "2e187f3c", "metadata": { "editable": true }, @@ -7092,7 +7093,7 @@ }, { "cell_type": "markdown", - "id": "524df5d3", + "id": "b9710ff1", "metadata": { "editable": true }, @@ -7109,7 +7110,7 @@ }, { "cell_type": "markdown", - "id": "d175b25f", + "id": "474b116e", "metadata": { "editable": true }, @@ -7119,7 +7120,7 @@ }, { "cell_type": "markdown", - "id": "11b3289e", + "id": "f50a379c", "metadata": { "editable": true }, @@ -7131,7 +7132,7 @@ }, { "cell_type": "markdown", - "id": "e469d65e", + "id": "b9be7c57", "metadata": { "editable": true }, @@ -7148,7 +7149,7 @@ }, { "cell_type": "markdown", - "id": "5f773fa8", + "id": "e9a7c309", "metadata": { "editable": true }, @@ -7159,7 +7160,7 @@ }, { "cell_type": "markdown", - "id": "2952f7b4", + "id": "82102f94", "metadata": { "editable": true }, @@ -7179,7 +7180,7 @@ }, { "cell_type": "markdown", - "id": "99057b48", + "id": "33008606", "metadata": { "editable": true }, @@ -7189,7 +7190,7 @@ }, { "cell_type": "markdown", - "id": "7debdfea", + "id": "a4efa0b7", "metadata": { "editable": true }, @@ -7215,7 +7216,7 @@ }, { "cell_type": "markdown", - "id": "2d5d5c44", + "id": "2a1267ce", "metadata": { "editable": true }, @@ -7231,7 +7232,7 @@ }, { "cell_type": "markdown", - "id": "f97806ce", + "id": "0aa8903b", "metadata": { "editable": true }, @@ -7242,7 +7243,7 @@ { "cell_type": "code", "execution_count": 50, - "id": "9337b7c7", + "id": "40349a4d", "metadata": { "collapsed": false, "editable": true @@ -7473,7 +7474,7 @@ }, { "cell_type": "markdown", - "id": "6071d006", + "id": "9e31fdb4", "metadata": { "editable": true }, diff --git a/doc/src/week43/week43.do.txt b/doc/src/week43/week43.do.txt index d0d35f649..b65ea9b44 100644 --- a/doc/src/week43/week43.do.txt +++ b/doc/src/week43/week43.do.txt @@ -8,7 +8,7 @@ DATE: October 21, 2024 !bblock Material for the lecture on Monday October 21, 2024 * Building our own Feed-forward Neural Network with intro to Tensorflow * Solving differential equations with Neural Networks -# * "Video of lecture to be posted asap":"https://youtu.be/" + * "Video of lecture at URL:"https://youtu.be/vkBNTn-MLqs" # * "Whiteboard notes":"https://github.com/CompPhysics/MachineLearning/blob/master/doc/HandWrittenNotes/2024/NotesOct21.pdf" !eblock