diff --git a/doc/LectureNotes/_build/.doctrees/environment.pickle b/doc/LectureNotes/_build/.doctrees/environment.pickle index 6a3912e3b..adb406d0a 100644 Binary files a/doc/LectureNotes/_build/.doctrees/environment.pickle and b/doc/LectureNotes/_build/.doctrees/environment.pickle differ diff --git a/doc/LectureNotes/_build/.doctrees/statistics.doctree b/doc/LectureNotes/_build/.doctrees/statistics.doctree index 6b79faa32..498f6731f 100644 Binary files a/doc/LectureNotes/_build/.doctrees/statistics.doctree and b/doc/LectureNotes/_build/.doctrees/statistics.doctree differ diff --git a/doc/LectureNotes/_build/html/_images/statistics_181_0.png b/doc/LectureNotes/_build/html/_images/statistics_181_0.png new file mode 100644 index 000000000..ec260ba1d Binary files /dev/null and b/doc/LectureNotes/_build/html/_images/statistics_181_0.png differ diff --git a/doc/LectureNotes/_build/html/_images/statistics_188_1.png b/doc/LectureNotes/_build/html/_images/statistics_188_1.png new file mode 100644 index 000000000..b13e4947f Binary files /dev/null and b/doc/LectureNotes/_build/html/_images/statistics_188_1.png differ diff --git a/doc/LectureNotes/_build/html/_images/statistics_32_0.png b/doc/LectureNotes/_build/html/_images/statistics_32_0.png new file mode 100644 index 000000000..5adbf9cac Binary files /dev/null and b/doc/LectureNotes/_build/html/_images/statistics_32_0.png differ diff --git a/doc/LectureNotes/_build/html/_sources/statistics.ipynb b/doc/LectureNotes/_build/html/_sources/statistics.ipynb index 8690ffe7a..4ea8e3730 100644 --- a/doc/LectureNotes/_build/html/_sources/statistics.ipynb +++ b/doc/LectureNotes/_build/html/_sources/statistics.ipynb @@ -2,18 +2,43 @@ "cells": [ { "cell_type": "markdown", - "metadata": {}, + "id": "1de4abfb", + "metadata": { + "editable": true + }, + "source": [ + "" + ] + }, + { + "cell_type": "markdown", + "id": "9268be0a", + "metadata": { + "editable": true + }, + "source": [ + "# Elements of Probability Theory and Statistical Data Analysis" + ] + }, + { + "cell_type": "markdown", + "id": "b24d433f", + "metadata": { + "editable": true + }, "source": [ - "# Elements of Probability Theory and Statistical Data Analysis\n", - "\n", - "\n", "## Domains and probabilities\n", + "\n", "Consider the following simple example, namely the tossing of two dice, resulting in the following possible values" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "9fa7c59b", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\{2,3,4,5,6,7,8,9,10,11,12\\}.\n", @@ -22,7 +47,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "d1c9912c", + "metadata": { + "editable": true + }, "source": [ "These values are called the *domain*. \n", "To this domain we have the corresponding *probabilities*" @@ -30,7 +58,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "6626ea89", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\{1/36,2/36/,3/36,4/36,5/36,6/36,5/36,4/36,3/36,2/36,1/36\\}.\n", @@ -39,9 +70,11 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "65c54c5b", + "metadata": { + "editable": true + }, "source": [ - "## Tossing the dice\n", "The numbers in the domain are the outcomes of the physical process of tossing say two dice.\n", "We cannot tell beforehand whether the outcome is 3 or 5 or any other number in this domain.\n", "This defines the randomness of the outcome, or unexpectedness or any other synonimous word which\n", @@ -55,7 +88,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "84e29b68", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\{2,3,4,5,6,7,8,9,10,11,12\\},\n", @@ -64,14 +100,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "8ae71934", + "metadata": { + "editable": true + }, "source": [ "appear in a random order. After 11 throws the results may look like" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "c3901951", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\{10,8,6,3,6,9,11,8,12,4,5\\}.\n", @@ -80,16 +122,23 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "9c659ce0", + "metadata": { + "editable": true + }, "source": [ - "## Stochastic variables\n", + "**Random variables are characterized by a domain which contains all possible values that the random value may take. This domain has a corresponding probability distribution function(PDF)**." + ] + }, + { + "cell_type": "markdown", + "id": "eb18a204", + "metadata": { + "editable": true + }, + "source": [ + "### Stochastic variables and the main concepts, the discrete case\n", "\n", - "**Random variables are characterized by a domain which contains all possible values that the random value may take. This domain has a corresponding probability distribution function(PDF)**.\n", - "\n", - "\n", - "\n", - "\n", - "## Stochastic variables and the main concepts, the discrete case\n", "There are two main concepts associated with a stochastic variable. The\n", "*domain* is the set $\\mathbb D = \\{x\\}$ of all accessible values\n", "the variable can assume, so that $X \\in \\mathbb D$. An example of a\n", @@ -104,7 +153,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "2589c0ee", + "metadata": { + "editable": true + }, "source": [ "$$\n", "p(x) = \\mathrm{Prob}(X=x).\n", @@ -113,9 +165,11 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "ef71f105", + "metadata": { + "editable": true + }, "source": [ - "## Stochastic variables and the main concepts, the continuous case\n", "In the continuous case, the PDF does not directly depict the\n", "actual probability. Instead we define the probability for the\n", "stochastic variable to assume any value on an infinitesimal interval\n", @@ -127,7 +181,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "9c5efaee", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\mathrm{Prob}(a\\leq X\\leq b) = \\int_a^b p(x)dx.\n", @@ -136,16 +193,15 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "cbf1d43c", + "metadata": { + "editable": true + }, "source": [ "Qualitatively speaking, a stochastic variable represents the values of\n", "numbers chosen as if by chance from some specified PDF so that the\n", "selection of a large set of these numbers reproduces this PDF.\n", "\n", - "\n", - "\n", - "\n", - "## The cumulative probability\n", "Of interest to us is the *cumulative probability\n", "distribution function* (**CDF**), $P(x)$, which is just the probability\n", "for a stochastic variable $X$ to assume any value less than $x$" @@ -153,7 +209,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "be829d81", + "metadata": { + "editable": true + }, "source": [ "$$\n", "P(x)=\\mathrm{Prob(}X\\leq x\\mathrm{)} =\n", @@ -163,14 +222,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "1f193798", + "metadata": { + "editable": true + }, "source": [ "The relation between a CDF and its corresponding PDF is then" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "3d29d961", + "metadata": { + "editable": true + }, "source": [ "$$\n", "p(x) = \\frac{d}{dx}P(x).\n", @@ -179,9 +244,12 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "fc5b2c18", + "metadata": { + "editable": true + }, "source": [ - "## Properties of PDFs\n", + "### Properties of PDFs\n", "\n", "There are two properties that all PDFs must satisfy. The first one is\n", "positivity (assuming that the PDF is normalized)" @@ -189,7 +257,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "2a465c1b", + "metadata": { + "editable": true + }, "source": [ "$$\n", "0 \\leq p(x) \\leq 1.\n", @@ -198,7 +269,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "d69e520a", + "metadata": { + "editable": true + }, "source": [ "Naturally, it would be nonsensical for any of the values of the domain\n", "to occur with a probability greater than $1$ or less than $0$. Also,\n", @@ -209,7 +283,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "2208b590", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{align*}\n", @@ -221,16 +298,21 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "f3d91dca", + "metadata": { + "editable": true + }, "source": [ - "## Important distributions, the uniform distribution\n", "The first one\n", "is the most basic PDF; namely the uniform distribution" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "0f431dd2", + "metadata": { + "editable": true + }, "source": [ "\n", "
\n", @@ -245,14 +327,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "155c6cf7", + "metadata": { + "editable": true + }, "source": [ "For $a=0$ and $b=1$ we have" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "42993dcc", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{array}{ll}\n", @@ -263,20 +351,22 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "749d5c23", + "metadata": { + "editable": true + }, "source": [ - "The latter distribution is used to generate random numbers. For other PDFs, one needs normally a mapping from this distribution to say for example the exponential distribution.\n", + "The latter distribution is used to generate random numbers. For other PDFs, one needs normally a mapping from this distribution to say for example the exponential distribution. \n", "\n", - "\n", - "\n", - "\n", - "## Gaussian distribution\n", "The second one is the Gaussian Distribution" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "6e2110dc", + "metadata": { + "editable": true + }, "source": [ "$$\n", "p(x) = \\frac{1}{\\sigma\\sqrt{2\\pi}} \\exp{(-\\frac{(x-\\mu)^2}{2\\sigma^2})},\n", @@ -285,14 +375,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "d00fdbe1", + "metadata": { + "editable": true + }, "source": [ "with mean value $\\mu$ and standard deviation $\\sigma$. If $\\mu=0$ and $\\sigma=1$, it is normally called the **standard normal distribution**" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "23c37b45", + "metadata": { + "editable": true + }, "source": [ "$$\n", "p(x) = \\frac{1}{\\sqrt{2\\pi}} \\exp{(-\\frac{x^2}{2})},\n", @@ -301,14 +397,18 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "07558d2c", + "metadata": { + "editable": true + }, "source": [ "The following simple Python code plots the above distribution for different values of $\\mu$ and $\\sigma$." ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 1, + "id": "b99f544d", "metadata": { "collapsed": false, "editable": true @@ -358,15 +458,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "617597ef", + "metadata": { + "editable": true + }, "source": [ - "## Exponential distribution\n", "Another important distribution in science is the exponential distribution" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "18be0515", + "metadata": { + "editable": true + }, "source": [ "$$\n", "p(x) = \\alpha\\exp{-(\\alpha x)}.\n", @@ -375,9 +480,13 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "cafcff78", + "metadata": { + "editable": true + }, "source": [ - "## Expectation values\n", + "### Expectation values\n", + "\n", "Let $h(x)$ be an arbitrary continuous function on the domain of the stochastic\n", "variable $X$ whose PDF is $p(x)$. We define the *expectation value*\n", "of $h$ with respect to $p$ as follows" @@ -385,7 +494,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "f5c68a26", + "metadata": { + "editable": true + }, "source": [ "\n", "
\n", @@ -400,7 +512,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "291e3ff7", + "metadata": { + "editable": true + }, "source": [ "Whenever the PDF is known implicitly, like in this case, we will drop\n", "the index $X$ for clarity. \n", @@ -411,7 +526,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "294080d7", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\langle x^n \\rangle \\equiv \\int\\! x^n p(x)\\,dx\n", @@ -420,9 +538,11 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "9d493962", + "metadata": { + "editable": true + }, "source": [ - "## Stochastic variables and the main concepts, mean values\n", "The zero-th moment $\\langle 1\\rangle$ is just the normalization condition of\n", "$p$. The first moment, $\\langle x\\rangle$, is called the *mean* of $p$\n", "and often denoted by the letter $\\mu$" @@ -430,7 +550,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "6ca38798", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\langle x\\rangle = \\mu \\equiv \\int x p(x)dx,\n", @@ -439,14 +562,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "9a779b37", + "metadata": { + "editable": true + }, "source": [ "for a continuous distribution and" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "9fc21016", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\langle x\\rangle = \\mu \\equiv \\sum_{i=1}^N x_i p(x_i),\n", @@ -455,23 +584,24 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "a146b327", + "metadata": { + "editable": true + }, "source": [ "for a discrete distribution. \n", "Qualitatively it represents the centroid or the average value of the\n", "PDF and is therefore simply called the expectation value of $p(x)$.\n", "\n", - "\n", - "\n", - "\n", - "## Stochastic variables and the main concepts, central moments, the variance\n", - "\n", "A special version of the moments is the set of *central moments*, the n-th central moment defined as" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "0b282f14", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\langle (x-\\langle x\\rangle )^n\\rangle \\equiv \\int\\! (x-\\langle x\\rangle)^n p(x)\\,dx\n", @@ -480,7 +610,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "512e6e16", + "metadata": { + "editable": true + }, "source": [ "The zero-th and first central moments are both trivial, equal $1$ and\n", "$0$, respectively. But the second central moment, known as the\n", @@ -490,7 +623,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "e244a343", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{align*}\n", @@ -505,26 +641,31 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "c0d8e272", + "metadata": { + "editable": true + }, "source": [ "The square root of the variance, $\\sigma =\\sqrt{\\langle (x-\\langle x\\rangle)^2\\rangle}$ is called the \n", "**standard deviation** of $p$. It is the RMS (root-mean-square)\n", "value of the deviation of the PDF from its mean value, interpreted\n", - "qualitatively as the \"spread\" of $p$ around its mean.\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "## Probability Distribution Functions\n", + "qualitatively as the \"spread\" of $p$ around its mean." + ] + }, + { + "cell_type": "markdown", + "id": "12fb667e", + "metadata": { + "editable": true + }, + "source": [ + "### Probability Distribution Functions\n", "\n", "The following table collects properties of probability distribution functions.\n", "In our notation we reserve the label $p(x)$ for the probability of a certain event,\n", "while $P(x)$ is the cumulative probability. \n", "\n", - "\n", - "\n", + "
\n", "\n", "\n", "\n", @@ -539,17 +680,15 @@ "\n", "
Discrete PDF Continuous PDF
\n", "\n", - "\n", - "\n", - "\n", - "\n", - "## Probability Distribution Functions\n", "With a PDF we can compute expectation values of selected quantities such as" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "7d318e93", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\langle x^k\\rangle=\\sum_{i=1}^{N}x_i^kp(x_i),\n", @@ -558,14 +697,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "6ef87b82", + "metadata": { + "editable": true + }, "source": [ "if we have a discrete PDF or" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "d11f1bdb", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\langle x^k\\rangle=\\int_a^b x^kp(x)dx,\n", @@ -574,15 +719,13 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "2a8abb3d", + "metadata": { + "editable": true + }, "source": [ "in the case of a continuous PDF. We have already defined the mean value $\\mu$\n", - "and the variance $\\sigma^2$.\n", - "\n", - "\n", - "\n", - "\n", - "## The three famous Probability Distribution Functions\n", + "and the variance $\\sigma^2$. \n", "\n", "There are at least three PDFs which one may encounter. These are the\n", "\n", @@ -591,7 +734,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "6c14f838", + "metadata": { + "editable": true + }, "source": [ "$$\n", "p(x)=\\frac{1}{b-a}\\Theta(x-a)\\Theta(b-x),\n", @@ -600,7 +746,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "62149f95", + "metadata": { + "editable": true + }, "source": [ "yielding probabilities different from zero in the interval $[a,b]$.\n", "\n", @@ -609,7 +758,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "f8f51c0a", + "metadata": { + "editable": true + }, "source": [ "$$\n", "p(x)=\\alpha \\exp{(-\\alpha x)},\n", @@ -618,14 +770,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "9a038bf3", + "metadata": { + "editable": true + }, "source": [ "yielding probabilities different from zero in the interval $[0,\\infty)$ and with mean value" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "bdaf914a", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\mu = \\int_0^{\\infty}xp(x)dx=\\int_0^{\\infty}x\\alpha \\exp{(-\\alpha x)}dx=\\frac{1}{\\alpha},\n", @@ -634,14 +792,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "74226a02", + "metadata": { + "editable": true + }, "source": [ "with variance" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "32215b82", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\sigma^2=\\int_0^{\\infty}x^2p(x)dx-\\mu^2 = \\frac{1}{\\alpha^2}.\n", @@ -650,15 +814,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "688f69a2", + "metadata": { + "editable": true + }, "source": [ - "## Probability Distribution Functions, the normal distribution\n", "Finally, we have the so-called univariate normal distribution, or just the **normal distribution**" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "082e6778", + "metadata": { + "editable": true + }, "source": [ "$$\n", "p(x)=\\frac{1}{b\\sqrt{2\\pi}}\\exp{\\left(-\\frac{(x-a)^2}{2b^2}\\right)}\n", @@ -667,7 +836,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "ba9c8bd4", + "metadata": { + "editable": true + }, "source": [ "with probabilities different from zero in the interval $(-\\infty,\\infty)$.\n", "The integral $\\int_{-\\infty}^{\\infty}\\exp{\\left(-(x^2\\right)}dx$ appears in many calculations, its value\n", @@ -677,7 +849,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "f9540188", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\mu = \\int_0^{\\infty}xp(x)dx=\\frac{1}{b\\sqrt{2\\pi}}\\int_{-\\infty}^{\\infty}x \\exp{\\left(-\\frac{(x-a)^2}{2b^2}\\right)}dx,\n", @@ -686,14 +861,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "0507fa0a", + "metadata": { + "editable": true + }, "source": [ "which becomes with a suitable change of variables" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "74ee9f1a", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\mu =\\frac{1}{b\\sqrt{2\\pi}}\\int_{-\\infty}^{\\infty}b\\sqrt{2}(a+b\\sqrt{2}y)\\exp{-y^2}dy=a.\n", @@ -702,15 +883,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "0cf42992", + "metadata": { + "editable": true + }, "source": [ - "## Probability Distribution Functions, the normal distribution\n", "Similarly, the variance becomes" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "e3554ca3", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\sigma^2 = \\frac{1}{b\\sqrt{2\\pi}}\\int_{-\\infty}^{\\infty}(x-\\mu)^2 \\exp{\\left(-\\frac{(x-a)^2}{2b^2}\\right)}dx,\n", @@ -719,14 +905,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "728ba052", + "metadata": { + "editable": true + }, "source": [ "and inserting the mean value and performing a variable change we obtain" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "60276744", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\sigma^2 = \\frac{1}{b\\sqrt{2\\pi}}\\int_{-\\infty}^{\\infty}b\\sqrt{2}(b\\sqrt{2}y)^2\\exp{\\left(-y^2\\right)}dy=\n", @@ -736,7 +928,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "c5b45c52", + "metadata": { + "editable": true + }, "source": [ "and performing a final integration by parts we obtain the well-known result $\\sigma^2=b^2$.\n", "It is useful to introduce the standard normal distribution as well, defined by $\\mu=a=0$, viz. a distribution\n", @@ -745,7 +940,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "a005fdfe", + "metadata": { + "editable": true + }, "source": [ "\n", "
\n", @@ -760,10 +958,11 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "12c13ae4", + "metadata": { + "editable": true + }, "source": [ - "## Probability Distribution Functions, the cumulative distribution\n", - "\n", "The exponential and uniform distributions have simple cumulative functions,\n", "whereas the normal distribution does not, being proportional to the so-called\n", "error function $erf(x)$, given by" @@ -771,7 +970,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "68e56af7", + "metadata": { + "editable": true + }, "source": [ "$$\n", "P(x) = \\frac{1}{\\sqrt{2\\pi}}\\int_{-\\infty}^x\\exp{\\left(-\\frac{t^2}{2}\\right)}dt,\n", @@ -780,22 +982,22 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "99071464", + "metadata": { + "editable": true + }, "source": [ - "which is difficult to evaluate in a quick way.\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "## Probability Distribution Functions, other important distribution\n", + "which is difficult to evaluate in a quick way. \n", "\n", "Some other PDFs which one encounters often in the natural sciences are the binomial distribution" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "08b3b2e3", + "metadata": { + "editable": true + }, "source": [ "$$\n", "p(x) = \\left(\\begin{array}{c} n \\\\ x\\end{array}\\right)y^x(1-y)^{n-x} \\hspace{0.5cm}x=0,1,\\dots,n,\n", @@ -804,7 +1006,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "57d2b4a7", + "metadata": { + "editable": true + }, "source": [ "where $y$ is the probability for a specific event, such as the tossing of a coin or moving left or right\n", "in case of a random walker. Note that $x$ is a discrete stochastic variable. \n", @@ -817,18 +1022,16 @@ "\n", " * The probability for every outcome in a single trial has the same value, for example the outcome of tossing (either heads or tails) a coin is always $1/2$.\n", "\n", - "\n", - "\n", - "\n", - "## Probability Distribution Functions, the binomial distribution\n", - "\n", "In order to compute the mean and variance we need to recall Newton's binomial\n", "formula" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "4ce5e39f", + "metadata": { + "editable": true + }, "source": [ "$$\n", "(a+b)^m=\\sum_{n=0}^m \\left(\\begin{array}{c} m \\\\ n\\end{array}\\right)a^nb^{m-n},\n", @@ -837,14 +1040,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "4b87bcf7", + "metadata": { + "editable": true + }, "source": [ "which can be used to show that" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "29e93603", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\sum_{x=0}^n\\left(\\begin{array}{c} n \\\\ x\\end{array}\\right)y^x(1-y)^{n-x} = (y+1-y)^n = 1,\n", @@ -853,7 +1062,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "524d4e2f", + "metadata": { + "editable": true + }, "source": [ "the PDF is normalized to one. \n", "The mean value is" @@ -861,7 +1073,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "d129841d", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\mu = \\sum_{x=0}^n x\\left(\\begin{array}{c} n \\\\ x\\end{array}\\right)y^x(1-y)^{n-x} =\n", @@ -871,14 +1086,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "f68c25df", + "metadata": { + "editable": true + }, "source": [ "resulting in" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "f19452aa", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\mu = \n", @@ -888,14 +1109,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "c75f2dc5", + "metadata": { + "editable": true + }, "source": [ "which we rewrite as" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "c7b760bf", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\mu=ny\\sum_{\\nu=0}^n\\left(\\begin{array}{c} n-1 \\\\ \\nu\\end{array}\\right)y^{\\nu}(1-y)^{n-1-\\nu} =ny(y+1-y)^{n-1}=ny.\n", @@ -904,20 +1131,23 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "6fc3e294", + "metadata": { + "editable": true + }, "source": [ "The variance is slightly trickier to get. It reads $\\sigma^2=ny(1-y)$. \n", "\n", - "\n", - "## Probability Distribution Functions, Poisson's distribution\n", - "\n", "Another important distribution with discrete stochastic variables $x$ is \n", "the Poisson model, which resembles the exponential distribution and reads" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "c67cc7c9", + "metadata": { + "editable": true + }, "source": [ "$$\n", "p(x) = \\frac{\\lambda^x}{x!} e^{-\\lambda} \\hspace{0.5cm}x=0,1,\\dots,;\\lambda > 0.\n", @@ -926,14 +1156,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "3988d52a", + "metadata": { + "editable": true + }, "source": [ "In this case both the mean value and the variance are easier to calculate," ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "9b3e4562", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\mu = \\sum_{x=0}^{\\infty} x \\frac{\\lambda^x}{x!} e^{-\\lambda} = \\lambda e^{-\\lambda}\\sum_{x=1}^{\\infty}\n", @@ -943,16 +1179,13 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "88138c21", + "metadata": { + "editable": true + }, "source": [ - "and the variance is $\\sigma^2=\\lambda$.\n", + "and the variance is $\\sigma^2=\\lambda$. \n", "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "## Probability Distribution Functions, Poisson's distribution\n", "An example of applications of the Poisson distribution could be the counting\n", "of the number of $\\alpha$-particles emitted from a radioactive source in a given time interval.\n", "In the limit of $n\\rightarrow \\infty$ and for small probabilities $y$, the binomial distribution\n", @@ -962,7 +1195,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "36ecffde", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\lim_{n\\rightarrow \\infty}\\left(\\begin{array}{c} n \\\\ x\\end{array}\\right)y^x(1-y)^{n-x} e^{-\\lambda}=\\sum_{x=1}^{\\infty}\\frac{\\lambda^x}{x!} e^{-\\lambda}.\n", @@ -971,9 +1207,13 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "be4e015c", + "metadata": { + "editable": true + }, "source": [ - "## Meet the covariance!\n", + "### Meet the covariance!\n", + "\n", "An important quantity in a statistical analysis is the so-called covariance. \n", "\n", "Consider the set $\\{X_i\\}$ of $n$\n", @@ -984,7 +1224,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "93522854", + "metadata": { + "editable": true + }, "source": [ "\n", "
\n", @@ -999,7 +1242,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "e667c0aa", + "metadata": { + "editable": true + }, "source": [ "\n", "
\n", @@ -1014,14 +1260,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "44f76dd9", + "metadata": { + "editable": true + }, "source": [ "with" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "1041b2af", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\langle x_i\\rangle =\n", @@ -1031,15 +1283,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "e42984a2", + "metadata": { + "editable": true + }, "source": [ - "## Meet the covariance in matrix disguise\n", "If we consider the above covariance as a matrix" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "a46f958e", + "metadata": { + "editable": true + }, "source": [ "$$\n", "C_{ij} =\\mathrm{Cov}(X_i,\\,X_j),\n", @@ -1048,22 +1305,21 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "37763243", + "metadata": { + "editable": true + }, "source": [ "then the diagonal elements are just the familiar\n", "variances, $C_{ii} = \\mathrm{Cov}(X_i,\\,X_i) = \\mathrm{Var}(X_i)$. It turns out that\n", "all the off-diagonal elements are zero if the stochastic variables are\n", - "uncorrelated.\n", - "\n", - "\n", - "\n", - "\n", - "## Covariance" + "uncorrelated." ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 2, + "id": "aedc8b8c", "metadata": { "collapsed": false, "editable": true @@ -1098,16 +1354,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "4ad05ea1", + "metadata": { + "editable": true + }, "source": [ - "## Meet the covariance, uncorrelated events\n", - "\n", "Consider the stochastic variables $X_i$ and $X_j$, ($i\\neq j$). We have" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "3f643cb0", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{align*}\n", @@ -1124,14 +1384,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "bedb81e0", + "metadata": { + "editable": true + }, "source": [ "If $X_i$ and $X_j$ are independent (assuming $i \\neq j$), we have that" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "8a8f5bdd", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\langle x_i x_j\\rangle = \\langle x_i\\rangle\\langle x_j\\rangle,\n", @@ -1140,14 +1406,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "cc3e8c15", + "metadata": { + "editable": true + }, "source": [ "leading to" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "cf107fa4", + "metadata": { + "editable": true + }, "source": [ "$$\n", "Cov(X_i, X_j) = 0 \\hspace{0.1cm} (i\\neq j).\n", @@ -1156,10 +1428,11 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "c3d6eefd", + "metadata": { + "editable": true + }, "source": [ - "## Numerical experiments and the covariance\n", - "\n", "Now that we have constructed an idealized mathematical framework, let\n", "us try to apply it to empirical observations. Examples of relevant\n", "physical phenomena may be spontaneous decays of nuclei, or a purely\n", @@ -1173,7 +1446,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "11084473", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\{x_1, x_2,\\dots\\,x_k,\\dots\\}.\n", @@ -1182,9 +1458,11 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "ec3bb41f", + "metadata": { + "editable": true + }, "source": [ - "## Numerical experiments and the covariance\n", "We will call these\n", "values our *measurements* and the entire set as our measured\n", "*sample*. The action of measuring all the elements of a sample\n", @@ -1198,18 +1476,16 @@ "interested in finding the few lowest moments, like the mean\n", "$\\mu_X^{\\phantom X}$ and the variance $\\sigma_X^{\\phantom X}$.\n", "\n", - "\n", - "\n", - "\n", - "\n", - "## Numerical experiments and the covariance, actual situations\n", "In practical situations however, a sample is always of finite size. Let that\n", "size be $n$. The expectation value of a sample $\\alpha$, the **sample mean**, is then defined as follows" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "9654f276", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\langle x_{\\alpha} \\rangle \\equiv \\frac{1}{n}\\sum_{k=1}^n x_{\\alpha,k}.\n", @@ -1218,14 +1494,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "9c339ccc", + "metadata": { + "editable": true + }, "source": [ "The *sample variance* is:" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "9d523f50", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\mathrm{Var}(x) \\equiv \\frac{1}{n}\\sum_{k=1}^n (x_{\\alpha,k} - \\langle x_{\\alpha} \\rangle)^2,\n", @@ -1234,15 +1516,13 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "2793f264", + "metadata": { + "editable": true + }, "source": [ - "with its square root being the *standard deviation of the sample*.\n", + "with its square root being the *standard deviation of the sample*. \n", "\n", - "\n", - "\n", - "\n", - "\n", - "## Numerical experiments and the covariance, our observables\n", "You can think of the above observables as a set of quantities which define\n", "a given experiment. This experiment is then repeated several times, say $m$ times.\n", "The total average is then" @@ -1250,7 +1530,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "b0432b14", + "metadata": { + "editable": true + }, "source": [ "\n", "
\n", @@ -1265,7 +1548,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "c1981b2d", + "metadata": { + "editable": true + }, "source": [ "where the last sums end at $m$ and $n$.\n", "The total variance is" @@ -1273,7 +1559,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "679d42bc", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\sigma^2_m= \\frac{1}{mn^2}\\sum_{\\alpha=1}^m(\\langle x_{\\alpha} \\rangle-\\langle X_m \\rangle)^2,\n", @@ -1282,14 +1571,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "2bb94e0d", + "metadata": { + "editable": true + }, "source": [ "which we rewrite as" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "0611f240", + "metadata": { + "editable": true + }, "source": [ "\n", "
\n", @@ -1304,16 +1599,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "624f9965", + "metadata": { + "editable": true + }, "source": [ - "## Numerical experiments and the covariance, the sample variance\n", - "\n", "We define also the sample variance $\\sigma^2$ of all $mn$ individual experiments as" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "c0d8ff9e", + "metadata": { + "editable": true + }, "source": [ "\n", "
\n", @@ -1328,18 +1627,26 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "d058f856", + "metadata": { + "editable": true + }, "source": [ "These quantities, being known experimental values or the results from our calculations, \n", "may differ, in some cases\n", "significantly, from the similarly named\n", "exact values for the mean value $\\mu_X$, the variance $\\mathrm{Var}(X)$\n", - "and the covariance $\\mathrm{Cov}(X,Y)$.\n", - "\n", - "\n", - "\n", - "\n", - "## Numerical experiments and the covariance, central limit theorem\n", + "and the covariance $\\mathrm{Cov}(X,Y)$." + ] + }, + { + "cell_type": "markdown", + "id": "76e956db", + "metadata": { + "editable": true + }, + "source": [ + "### Numerical experiments and the covariance, central limit theorem\n", "\n", "The central limit theorem states that the PDF $\\tilde{p}(z)$ of\n", "the average of $m$ random values corresponding to a PDF $p(x)$ \n", @@ -1353,7 +1660,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "bd4e286e", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\sigma_m=\n", @@ -1363,20 +1673,22 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "877355c7", + "metadata": { + "editable": true + }, "source": [ "In many cases the above estimate for the standard deviation, in particular if correlations are strong, may be too simplistic. We need therefore a more precise defintion of the error and the variance in our results.\n", "\n", - "\n", - "\n", - "\n", - "## Definition of Correlation Functions and Standard Deviation\n", "Our estimate of the true average $\\mu_{X}$ is the sample mean $\\langle X_m \\rangle$" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "abc25d1f", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\mu_{X}^{\\phantom X} \\approx X_m=\\frac{1}{mn}\\sum_{\\alpha=1}^m\\sum_{k=1}^n x_{\\alpha,k}.\n", @@ -1385,14 +1697,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "29a4c67d", + "metadata": { + "editable": true + }, "source": [ "We can then use Eq. ([7](#eq:exptvariance))" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "46b6a9e4", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\sigma^2_m=\\frac{1}{mn^2}\\sum_{\\alpha=1}^m\\sum_{kl=1}^n (x_{\\alpha,k}-\\langle X_m \\rangle)(x_{\\alpha,l}-\\langle X_m \\rangle),\n", @@ -1401,14 +1719,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "350ee280", + "metadata": { + "editable": true + }, "source": [ "and rewrite it as" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "321d0a3d", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\sigma^2_m=\\frac{\\sigma^2}{n}+\\frac{2}{mn^2}\\sum_{\\alpha=1}^m\\sum_{k\n", "
\n", @@ -1508,23 +1836,24 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "03919ae2", + "metadata": { + "editable": true + }, "source": [ "which gives us a useful measure of the correlation pair correlation\n", "starting always at $1$ for $d=0$.\n", "\n", - "\n", - "\n", - "\n", - "## Definition of Correlation Functions and Standard Deviation, sample variance\n", - "\n", "The sample variance of the $mn$ experiments can now be\n", "written in terms of the autocorrelation function" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "f4edb79b", + "metadata": { + "editable": true + }, "source": [ "\n", "
\n", @@ -1540,7 +1869,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "209c3c0a", + "metadata": { + "editable": true + }, "source": [ "and we see that $\\sigma_m$ can be expressed in terms of the\n", "uncorrelated sample variance times a correction factor $\\tau$ which\n", @@ -1550,7 +1882,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "d0c7027f", + "metadata": { + "editable": true + }, "source": [ "\n", "
\n", @@ -1565,19 +1900,16 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "8183355a", + "metadata": { + "editable": true + }, "source": [ "\n", "\n", "For a correlation free experiment, $\\tau$\n", - "equals 1.\n", + "equals 1. \n", "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "## Definition of Correlation Functions and Standard Deviation\n", "From the point of view of\n", "Eq. ([10](#eq:error_estimate_corr_time)) we can interpret a sequential\n", "correlation as an effective reduction of the number of measurements by\n", @@ -1586,7 +1918,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "04f300ab", + "metadata": { + "editable": true + }, "source": [ "$$\n", "n_\\mathrm{eff} = \\frac{n}{\\tau}\n", @@ -1595,7 +1930,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "d9bdb1eb", + "metadata": { + "editable": true + }, "source": [ "To neglect the autocorrelation time $\\tau$ will always cause our\n", "simple uncorrelated estimate of $\\sigma_m^2\\approx \\sigma^2/n$ to\n", @@ -1604,18 +1942,13 @@ "autocorrelation time poses an efficiency problem if the set of\n", "measurements is very large. The solution to this problem is given by \n", "more practically oriented methods like the blocking technique.\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "## Code to compute the Covariance matrix and the Covariance" + "" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 3, + "id": "93013c0b", "metadata": { "collapsed": false, "editable": true @@ -1663,9 +1996,12 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "65e8cd76", + "metadata": { + "editable": true + }, "source": [ - "## Random Numbers\n", + "### Random Numbers\n", "\n", "Uniform deviates are just random numbers that lie within a specified range\n", "(typically 0 to 1), with any one number in the range just as likely as any other. They\n", @@ -1678,12 +2014,6 @@ "of this section, is an essential building block for any sort of stochastic modeling\n", "or Monte Carlo computer work.\n", "\n", - "\n", - "\n", - "\n", - "\n", - "## Random Numbers, better name: pseudo random numbers\n", - "\n", "A disclaimer is however appropriate. It should be fairly obvious that \n", "something as deterministic as a computer cannot generate purely random numbers.\n", "\n", @@ -1698,18 +2028,16 @@ "\n", " * the algorithm should be fast.\n", "\n", - "\n", - "\n", - "\n", - "\n", - "## Random number generator RNG\n", " The most common random number generators are based on so-called\n", "Linear congruential relations of the type" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "4daecdd3", + "metadata": { + "editable": true + }, "source": [ "$$\n", "N_i=(aN_{i-1}+c) \\mathrm{MOD} (M),\n", @@ -1718,14 +2046,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "fd1c5e47", + "metadata": { + "editable": true + }, "source": [ "which yield a number in the interval [0,1] through" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "d8c06853", + "metadata": { + "editable": true + }, "source": [ "$$\n", "x_i=N_i/M\n", @@ -1734,7 +2068,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "0d971106", + "metadata": { + "editable": true + }, "source": [ "The number \n", "$M$ is called the period and it should be as large as possible \n", @@ -1743,11 +2080,6 @@ "that is if we were to evaluate $(13)\\mathrm{MOD}(9)$, the outcome is the remainder\n", "of the division $13/9$, namely $4$.\n", "\n", - "\n", - "\n", - "\n", - "## Random number generator RNG and periodic outputs\n", - "\n", "The problem with such generators is that their outputs are periodic;\n", "they \n", "will start to repeat themselves with a period that is at most $M$. If however\n", @@ -1758,7 +2090,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "d12a26d7", + "metadata": { + "editable": true + }, "source": [ "$$\n", "N_i=(6N_{i-1}+7) \\mathrm{MOD} (5),\n", @@ -1767,7 +2102,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "2ee32cc9", + "metadata": { + "editable": true + }, "source": [ "with a seed $N_0=2$. This generator produces the sequence\n", "$4,1,3,0,2,4,1,3,0,2,...\\dots$, i.e., a sequence with period $5$.\n", @@ -1777,7 +2115,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "441be869", + "metadata": { + "editable": true + }, "source": [ "$$\n", "N_i=(27N_{i-1}+11) \\mathrm{MOD} (54),\n", @@ -1786,15 +2127,14 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "c0803129", + "metadata": { + "editable": true + }, "source": [ "which still, with $N_0=2$, results in $11,38,11,38,11,38,\\dots$, a period of\n", "just $2$.\n", "\n", - "\n", - "\n", - "\n", - "## Random number generator RNG and its period\n", "Typical periods for the random generators provided in the program library \n", "are of the order of $\\sim 10^9$ or larger. Other random number generators which have\n", "become increasingly popular are so-called shift-register generators.\n", @@ -1807,7 +2147,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "43aef6df", + "metadata": { + "editable": true + }, "source": [ "$$\n", "N_l=(aN_{l-i}+cN_{l-j})\\mathrm{MOD}(M).\n", @@ -1816,9 +2159,11 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "1c7bddf4", + "metadata": { + "editable": true + }, "source": [ - "## Random number generator RNG, other examples\n", "Such a generator again produces a sequence of pseudorandom numbers\n", "but this time with a period much larger than $M$.\n", "It is also possible to construct more elaborate algorithms by including\n", @@ -1829,7 +2174,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "fed2dc23", + "metadata": { + "editable": true + }, "source": [ "\n", "
\n", @@ -1844,14 +2192,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "dc9d8537", + "metadata": { + "editable": true + }, "source": [ "followed by" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "aa9465e9", + "metadata": { + "editable": true + }, "source": [ "\n", "
\n", @@ -1866,21 +2220,23 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "def99a7e", + "metadata": { + "editable": true + }, "source": [ "which according to the authors has a period larger than $2^{94}$.\n", "\n", - "\n", - "\n", - "\n", - "## Random number generator RNG, other examples\n", "Instead of using modular addition, we could use the bitwise\n", "exclusive-OR ($\\oplus$) operation so that" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "61c8f9ef", + "metadata": { + "editable": true + }, "source": [ "$$\n", "N_l=(N_{l-i})\\oplus (N_{l-j})\n", @@ -1889,7 +2245,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "c59c90b8", + "metadata": { + "editable": true + }, "source": [ "where the bitwise action of $\\oplus$ means that if $N_{l-i}=N_{l-j}$ the result is\n", "$0$ whereas if $N_{l-i}\\ne N_{l-j}$ the result is\n", @@ -1900,20 +2259,17 @@ "\n", "In Fortran90, the bitwise $\\oplus$ operation is coded through the intrinsic\n", "function $\\mathrm{IEOR}(m,n)$ where $m$ and $n$ are the input numbers, while in $C$\n", - "it is given by $m\\wedge n$.\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "## Random number generator RNG, RAN0\n", + "it is given by $m\\wedge n$. \n", "\n", "We show here how the linear congruential algorithm can be implemented, namely" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "06587f00", + "metadata": { + "editable": true + }, "source": [ "$$\n", "N_i=(aN_{i-1}) \\mathrm{MOD} (M).\n", @@ -1922,7 +2278,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "c3255e8e", + "metadata": { + "editable": true + }, "source": [ "However, since $a$ and $N_{i-1}$ are integers and their multiplication \n", "could become greater than the standard 32 bit integer, there is a trick via \n", @@ -1932,7 +2291,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "8e3ec364", + "metadata": { + "editable": true + }, "source": [ "$$\n", "M=aq+r,\n", @@ -1941,14 +2303,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "4b7c18d5", + "metadata": { + "editable": true + }, "source": [ "where we have defined" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "4e58c88d", + "metadata": { + "editable": true + }, "source": [ "$$\n", "q=[M/a],\n", @@ -1957,14 +2325,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "1fecdb9a", + "metadata": { + "editable": true + }, "source": [ "and" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "aa126ce7", + "metadata": { + "editable": true + }, "source": [ "$$\n", "r = M\\hspace{0.1cm}\\mathrm{MOD} \\hspace{0.1cm}a.\n", @@ -1973,24 +2347,23 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "20cab2fa", + "metadata": { + "editable": true + }, "source": [ "where the brackets denote integer division. In the code below the numbers \n", "$q$ and $r$ are chosen so that $r < q$.\n", "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "## Random number generator RNG, RAN0\n", - "\n", "To see how this works we note first that" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "904a6320", + "metadata": { + "editable": true + }, "source": [ "\n", "
\n", @@ -2005,23 +2378,24 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "36c6f816", + "metadata": { + "editable": true + }, "source": [ "since we can add or subtract any integer multiple of $M$ from $aN_{i-1}$.\n", "The last term $[N_{i-1}/q]M\\mathrm{MOD}(M)$ is zero since the integer division \n", - "$[N_{i-1}/q]$ just yields a constant which is multiplied with $M$.\n", + "$[N_{i-1}/q]$ just yields a constant which is multiplied with $M$. \n", "\n", - "\n", - "\n", - "\n", - "\n", - "## Random number generator RNG, RAN0\n", "We can now rewrite Eq. ([14](#eq:rntrick1)) as" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "319af13b", + "metadata": { + "editable": true + }, "source": [ "\n", "
\n", @@ -2036,15 +2410,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "0c426357", + "metadata": { + "editable": true + }, "source": [ - "which results\n", - "in" + "which results" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "52547452", + "metadata": { + "editable": true + }, "source": [ "\n", "
\n", @@ -2059,14 +2438,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "86bf958f", + "metadata": { + "editable": true + }, "source": [ "yielding" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "d2015392", + "metadata": { + "editable": true + }, "source": [ "\n", "
\n", @@ -2081,9 +2466,11 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "83495015", + "metadata": { + "editable": true + }, "source": [ - "## Random number generator RNG, RAN0\n", "The term $[N_{i-1}/q]r$ is always smaller or equal $N_{i-1}(r/q)$ and with $r < q$ we obtain always a \n", "number smaller than $N_{i-1}$, which is smaller than $M$. \n", "And since the number $N_{i-1}\\mathrm{MOD} (q)$ is between zero and $q-1$ then\n", @@ -2095,52 +2482,7 @@ "the starting point for each generation of a random number. The period\n", "of $ran0$ is $\\sim 2.1\\times 10^{9}$. A special feature of this\n", "algorithm is that is should never be called with the initial seed \n", - "set to $0$.\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "## Random number generator RNG, RAN0 code" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - " /*\n", - " ** The function\n", - " ** ran0()\n", - " ** is an \"Minimal\" random number generator of Park and Miller\n", - " ** Set or reset the input value\n", - " ** idum to any integer value (except the unlikely value MASK)\n", - " ** to initialize the sequence; idum must not be altered between\n", - " ** calls for sucessive deviates in a sequence.\n", - " ** The function returns a uniform deviate between 0.0 and 1.0.\n", - " */\n", - " double ran0(long &idum)\n", - " {\n", - " const int a = 16807, m = 2147483647, q = 127773;\n", - " const int r = 2836, MASK = 123459876;\n", - " const double am = 1./m;\n", - " long k;\n", - " double ans;\n", - " idum ^= MASK;\n", - " k = (*idum)/q;\n", - " idum = a*(idum - k*q) - r*k;\n", - " // add m if negative difference\n", - " if(idum < 0) idum += m;\n", - " ans=am*(idum);\n", - " idum ^= MASK;\n", - " return ans;\n", - " } // End: function ran0() \n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Properties of Selected Random Number Generators\n", + "set to $0$. \n", "\n", "As mentioned previously, the underlying PDF for the generation of\n", "random numbers is the uniform distribution, meaning that the \n", @@ -2156,16 +2498,15 @@ "Two additional measures are the standard deviation $\\sigma$ and the mean\n", "$\\mu=\\langle x\\rangle$.\n", "\n", - "\n", - "\n", - "\n", - "## Properties of Selected Random Number Generators\n", "For the uniform distribution, the mean value $\\mu$ is then" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "268aac30", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\mu=\\langle x\\rangle=\\frac{1}{2}\n", @@ -2174,14 +2515,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "25bd4ea3", + "metadata": { + "editable": true + }, "source": [ "while the standard deviation is" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "29e8d709", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\sigma=\\sqrt{\\langle x^2\\rangle-\\mu^2}=\\frac{1}{\\sqrt{12}}=0.2886.\n", @@ -2190,13 +2537,15 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "90a07c88", + "metadata": { + "editable": true + }, "source": [ - "## Properties of Selected Random Number Generators\n", "The various random number generators produce results which agree rather well with\n", "these limiting values. \n", "\n", - "\n", + "
\n", "\n", "\n", "\n", @@ -2216,16 +2565,13 @@ "\n", "
$x$-bin ran0 ran1 ran2 ran3
\n", "\n", - "\n", - "\n", - "\n", - "## Simple demonstration of RNGs using python\n", "The following simple Python code plots the distribution of the produced random numbers using the linear congruential RNG employed by Python. The trend displayed in the previous table is seen rather clearly." ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 4, + "id": "3c296708", "metadata": { "collapsed": false, "editable": true @@ -2258,9 +2604,11 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "7a028dda", + "metadata": { + "editable": true + }, "source": [ - "## Properties of Selected Random Number Generators\n", "Since our random numbers, which are typically generated via a linear congruential algorithm,\n", "are never fully independent, we can then define \n", "an important test which measures the degree of correlation, namely the so-called \n", @@ -2270,7 +2618,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "bc325e1a", + "metadata": { + "editable": true + }, "source": [ "$$\n", "C_k=\\frac{f_d}\n", @@ -2280,7 +2631,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "a7738ea8", + "metadata": { + "editable": true + }, "source": [ "with $C_0=1$. Recall that \n", "$\\sigma^2=\\langle x_i^2\\rangle-\\langle x_i\\rangle^2$ and that" @@ -2288,7 +2642,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "b647158e", + "metadata": { + "editable": true + }, "source": [ "$$\n", "f_d = \\frac{1}{nm}\\sum_{\\alpha=1}^m\\sum_{k=1}^{n-d}(x_{\\alpha,k}-\\langle X_m \\rangle)(x_{\\alpha,k+d}-\\langle X_m \\rangle),\n", @@ -2297,24 +2654,33 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "273afa83", + "metadata": { + "editable": true + }, "source": [ "The non-vanishing of $C_k$ for $k\\ne 0$ means that the random\n", "numbers are not independent. The independence of the random numbers is crucial \n", "in the evaluation of other expectation values. If they are not independent, our\n", - "assumption for approximating $\\sigma_N$ is no longer valid.\n", + "assumption for approximating $\\sigma_N$ is no longer valid." + ] + }, + { + "cell_type": "markdown", + "id": "70391792", + "metadata": { + "editable": true + }, + "source": [ + "### Autocorrelation function\n", "\n", - "\n", - "\n", - "\n", - "\n", - "## Autocorrelation function\n", "This program computes the autocorrelation function as discussed in the equation on the previous slide for random numbers generated with the normal distribution $N(0,1)$." ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 5, + "id": "2b08e1a4", "metadata": { "collapsed": false, "editable": true @@ -2354,529 +2720,17 @@ }, { "cell_type": "markdown", - "metadata": {}, - "source": [ - "As can be seen from the plot, the first point gives back the variance and a value of one. \n", - "For the remaining values we notice that there are still non-zero values for the auto-correlation function.\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "## Correlation function and which random number generators should I use\n", - "The program here computes the correlation function for one of the standard functions included with the c++ compiler." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - " // This function computes the autocorrelation function for \n", - " // the standard c++ random number generator\n", - " \n", - " #include \n", - " #include \n", - " #include \n", - " #include \n", - " using namespace std;\n", - " // output file as global variable\n", - " ofstream ofile; \n", - " \n", - " // Main function begins here \n", - " int main(int argc, char* argv[])\n", - " {\n", - " int n;\n", - " char *outfilename;\n", - " \n", - " cin >> n;\n", - " double MCint = 0.; double MCintsqr2=0.;\n", - " double invers_period = 1./RAND_MAX; // initialise the random number generator\n", - " srand(time(NULL)); // This produces the so-called seed in MC jargon\n", - " // Compute the variance and the mean value of the uniform distribution\n", - " // Compute also the specific values x for each cycle in order to be able to\n", - " // the covariance and the correlation function \n", - " // Read in output file, abort if there are too few command-line arguments\n", - " if( argc <= 2 ){\n", - " cout << \"Bad Usage: \" << argv[0] << \n", - " \t \" read also output file and number of cycles on same line\" << endl;\n", - " exit(1);\n", - " }\n", - " else{\n", - " outfilename=argv[1];\n", - " }\n", - " ofile.open(outfilename); \n", - " // Get the number of Monte-Carlo samples\n", - " n = atoi(argv[2]);\n", - " double *X; \n", - " X = new double[n];\n", - " for (int i = 0; i < n; i++){\n", - " double x = double(rand())*invers_period; \n", - " X[i] = x;\n", - " MCint += x;\n", - " MCintsqr2 += x*x;\n", - " }\n", - " double Mean = MCint/((double) n );\n", - " MCintsqr2 = MCintsqr2/((double) n );\n", - " double STDev = sqrt(MCintsqr2-Mean*Mean);\n", - " double Variance = MCintsqr2-Mean*Mean;\n", - " // Write mean value and standard deviation \n", - " cout << \" Standard deviation= \" << STDev << \" Integral = \" << Mean << endl;\n", - " \n", - " // Now we compute the autocorrelation function\n", - " double *autocor; autocor = new double[n];\n", - " for (int j = 0; j < n; j++){\n", - " double sum = 0.0;\n", - " for (int k = 0; k < (n-j); k++){\n", - " \t sum += (X[k]-Mean)*(X[k+j]-Mean); \n", - " }\n", - " autocor[j] = sum/Variance/((double) n );\n", - " ofile << setiosflags(ios::showpoint | ios::uppercase);\n", - " ofile << setw(15) << setprecision(8) << j;\n", - " ofile << setw(15) << setprecision(8) << autocor[j] << endl;\n", - " }\n", - " ofile.close(); // close output file\n", - " return 0;\n", - " } // end of main program \n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Which RNG should I use?\n", - "* C++ has a class called **random**. The [random class](http://www.cplusplus.com/reference/random/) contains a large selection of RNGs and is highly recommended. Some of these RNGs have very large periods making it thereby very safe to use these RNGs in case one is performing large calculations. In particular, the [Mersenne twister random number engine](http://www.cplusplus.com/reference/random/mersenne_twister_engine/) has a period of $2^{19937}$. \n", - "\n", - "* Add RNGs in Python\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "## How to use the Mersenne generator\n", - "The following part of a c++ code (from project 4) sets up the uniform distribution for $x\\in [0,1]$." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - " /*\n", - " \n", - " // You need this \n", - " #include \n", - " \n", - " // Initialize the seed and call the Mersienne algo\n", - " std::random_device rd;\n", - " std::mt19937_64 gen(rd());\n", - " // Set up the uniform distribution for x \\in [[0, 1]\n", - " std::uniform_real_distribution RandomNumberGenerator(0.0,1.0);\n", - " \n", - " // Now use the RNG\n", - " int ix = (int) (RandomNumberGenerator(gen)*NSpins);\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Why blocking?\n", - "**Statistical analysis.**\n", - "\n", - " * Monte Carlo simulations can be treated as *computer experiments*\n", - "\n", - " * The results can be analysed with the same statistical tools as we would use analysing experimental data.\n", - "\n", - " * As in all experiments, we are looking for expectation values and an estimate of how accurate they are, i.e., possible sources for errors.\n", - "\n", - "A very good article which explains blocking is H. Flyvbjerg and H. G. Petersen, *Error estimates on averages of correlated data*, [Journal of Chemical Physics 91, 461-466 (1989)](http://scitation.aip.org/content/aip/journal/jcp/91/1/10.1063/1.457480).\n", - "\n", - " \n", - "\n", - "\n", - "\n", - "\n", - "## Why blocking?\n", - "**Statistical analysis.**\n", - "\n", - " * As in other experiments, Monte Carlo experiments have two classes of errors:\n", - "\n", - " * Statistical errors\n", - "\n", - " * Systematical errors\n", - "\n", - "\n", - " * Statistical errors can be estimated using standard tools from statistics\n", - "\n", - " * Systematical errors are method specific and must be treated differently from case to case. (In VMC a common source is the step length or time step in importance sampling)\n", - "\n", - " \n", - "\n", - "\n", - "\n", - "## Code to demonstrate the calculation of the autocorrelation function\n", - "The following code computes the autocorrelation function, the covariance and the standard deviation\n", - "for standard RNG. \n", - "The [following file](https://github.com/CompPhysics/ComputationalPhysics2/tree/gh-pages/doc/Programs/LecturePrograms/programs/Blocking/autocorrelation.cpp) gives the code." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - " // This function computes the autocorrelation function for \n", - " // the Mersenne random number generator with a uniform distribution\n", - " #include \n", - " #include \n", - " #include \n", - " #include \n", - " #include \n", - " #include \n", - " #include \n", - " #include \n", - " using namespace std;\n", - " using namespace arma;\n", - " // output file\n", - " ofstream ofile;\n", - " \n", - " // Main function begins here \n", - " int main(int argc, char* argv[])\n", - " {\n", - " int MonteCarloCycles;\n", - " string filename;\n", - " if (argc > 1) {\n", - " filename=argv[1];\n", - " MonteCarloCycles = atoi(argv[2]);\n", - " string fileout = filename;\n", - " string argument = to_string(MonteCarloCycles);\n", - " fileout.append(argument);\n", - " ofile.open(fileout);\n", - " }\n", - " \n", - " // Compute the variance and the mean value of the uniform distribution\n", - " // Compute also the specific values x for each cycle in order to be able to\n", - " // compute the covariance and the correlation function \n", - " \n", - " vec X = zeros(MonteCarloCycles);\n", - " double MCint = 0.; double MCintsqr2=0.;\n", - " std::random_device rd;\n", - " std::mt19937_64 gen(rd());\n", - " // Set up the uniform distribution for x \\in [[0, 1]\n", - " std::uniform_real_distribution RandomNumberGenerator(0.0,1.0);\n", - " for (int i = 0; i < MonteCarloCycles; i++){\n", - " double x = RandomNumberGenerator(gen); \n", - " X(i) = x;\n", - " MCint += x;\n", - " MCintsqr2 += x*x;\n", - " }\n", - " double Mean = MCint/((double) MonteCarloCycles );\n", - " MCintsqr2 = MCintsqr2/((double) MonteCarloCycles );\n", - " double STDev = sqrt(MCintsqr2-Mean*Mean);\n", - " double Variance = MCintsqr2-Mean*Mean;\n", - " // Write mean value and variance\n", - " cout << \" Sample variance= \" << Variance << \" Mean value = \" << Mean << endl;\n", - " // Now we compute the autocorrelation function\n", - " vec autocorrelation = zeros(MonteCarloCycles);\n", - " for (int j = 0; j < MonteCarloCycles; j++){\n", - " double sum = 0.0;\n", - " for (int k = 0; k < (MonteCarloCycles-j); k++){\n", - " sum += (X(k)-Mean)*(X(k+j)-Mean); \n", - " }\n", - " autocorrelation(j) = sum/Variance/((double) MonteCarloCycles );\n", - " ofile << setiosflags(ios::showpoint | ios::uppercase);\n", - " ofile << setw(15) << setprecision(8) << j;\n", - " ofile << setw(15) << setprecision(8) << autocorrelation(j) << endl;\n", - " }\n", - " // Now compute the exact covariance using the autocorrelation function\n", - " double Covariance = 0.0;\n", - " for (int j = 0; j < MonteCarloCycles; j++){\n", - " Covariance += autocorrelation(j);\n", - " }\n", - " Covariance *= 2.0/((double) MonteCarloCycles);\n", - " // Compute now the total variance, including the covariance, and obtain the standard deviation\n", - " double TotalVariance = (Variance/((double) MonteCarloCycles ))+Covariance;\n", - " cout << \"Covariance =\" << Covariance << \"Totalvariance= \" << TotalVariance << \"Sample Variance/n= \" << (Variance/((double) MonteCarloCycles )) << endl;\n", - " cout << \" STD from sample variance= \" << sqrt(Variance/((double) MonteCarloCycles )) << \" STD with covariance = \" << sqrt(TotalVariance) << endl;\n", - " \n", - " ofile.close(); // close output file\n", - " return 0;\n", - " } // end of main program \n", - " \n", - " \n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## What is blocking?\n", - "**Blocking.**\n", - "\n", - " * Say that we have a set of samples from a Monte Carlo experiment\n", - "\n", - " * Assuming (wrongly) that our samples are uncorrelated our best estimate of the standard deviation of the mean $\\langle \\mathbf{M}\\rangle$ is given by" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "$$\n", - "\\sigma=\\sqrt{\\frac{1}{n}\\left(\\langle \\mathbf{M}^2\\rangle-\\langle \\mathbf{M}\\rangle^2\\right)}\n", - "$$" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "* If the samples are correlated we can rewrite our results to show that" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "$$\n", - "\\sigma=\\sqrt{\\frac{1+2\\tau/\\Delta t}{n}\\left(\\langle \\mathbf{M}^2\\rangle-\\langle \\mathbf{M}\\rangle^2\\right)}\n", - "$$" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "where $\\tau$ is the correlation time (the time between a sample and the next uncorrelated sample) and $\\Delta t$ is time between each sample\n", - "\n", - " \n", - "\n", - "\n", - "## What is blocking?\n", - "**Blocking.**\n", - "\n", - " * If $\\Delta t\\gg\\tau$ our first estimate of $\\sigma$ still holds\n", - "\n", - " * Much more common that $\\Delta t<\\tau$\n", - "\n", - " * In the method of data blocking we divide the sequence of samples into blocks\n", - "\n", - " * We then take the mean $\\langle \\mathbf{M}_i\\rangle$ of block $i=1\\ldots n_{blocks}$ to calculate the total mean and variance\n", - "\n", - " * The size of each block must be so large that sample $j$ of block $i$ is not correlated with sample $j$ of block $i+1$\n", - "\n", - " * The correlation time $\\tau$ would be a good choice\n", - "\n", - "\n", - "\n", - "\n", - "## What is blocking?\n", - "**Blocking.**\n", - "\n", - " * Problem: We don't know $\\tau$ or it is too expensive to compute\n", - "\n", - " * Solution: Make a plot of std. dev. as a function of blocksize\n", - "\n", - " * The estimate of std. dev. of correlated data is too low $\\to$ the error will increase with increasing block size until the blocks are uncorrelated, where we reach a plateau\n", - "\n", - " * When the std. dev. stops increasing the blocks are uncorrelated\n", - "\n", - "\n", - "\n", - "\n", - "## Implementation\n", - " * Do a Monte Carlo simulation, storing all samples to file\n", - "\n", - " * Do the statistical analysis on this file, independently of your Monte Carlo program\n", - "\n", - " * Read the file into an array\n", - "\n", - " * Loop over various block sizes\n", - "\n", - " * For each block size $n_b$, loop over the array in steps of $n_b$ taking the mean of elements $i n_b,\\ldots,(i+1) n_b$\n", - "\n", - " * Take the mean and variance of the resulting array\n", - "\n", - " * Write the results for each block size to file for later\n", - " analysis\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "## Actual implementation with code, main function\n", - "When the file gets large, it can be useful to write your data in binary mode instead of ascii characters.\n", - "The [following python file](https://github.com/CompPhysics/MachineLearning/blob/master/doc/Programs/Sampling/analysis.py) reads data from file with the output from every Monte Carlo cycle." - ] - }, - { - "cell_type": "code", - "execution_count": null, + "id": "5848dd32", "metadata": { - "collapsed": false, "editable": true }, - "outputs": [], "source": [ - "# Blocking\n", - " @timeFunction\n", - " def blocking(self, blockSizeMax = 500):\n", - " blockSizeMin = 1\n", - "\n", - " self.blockSizes = []\n", - " self.meanVec = []\n", - " self.varVec = []\n", - "\n", - " for i in range(blockSizeMin, blockSizeMax):\n", - " if(len(self.data) % i != 0):\n", - " pass#continue\n", - " blockSize = i\n", - " meanTempVec = []\n", - " varTempVec = []\n", - " startPoint = 0\n", - " endPoint = blockSize\n", - "\n", - " while endPoint <= len(self.data):\n", - " meanTempVec.append(np.average(self.data[startPoint:endPoint]))\n", - " startPoint = endPoint\n", - " endPoint += blockSize\n", - " mean, var = np.average(meanTempVec), np.var(meanTempVec)/len(meanTempVec)\n", - " self.meanVec.append(mean)\n", - " self.varVec.append(var)\n", - " self.blockSizes.append(blockSize)\n", - "\n", - " self.blockingAvg = np.average(self.meanVec[-200:])\n", - " self.blockingVar = (np.average(self.varVec[-200:]))\n", - " self.blockingStd = np.sqrt(self.blockingVar)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## The Bootstrap method\n", - "\n", - "The Bootstrap resampling method is also very popular. It is very simple:\n", - "\n", - "1. Start with your sample of measurements and compute the sample variance and the mean values\n", - "\n", - "2. Then start again but pick in a random way the numbers in the sample and recalculate the mean and the sample variance.\n", - "\n", - "3. Repeat this $K$ times.\n", - "\n", - "It can be shown, see the article by [Efron](https://projecteuclid.org/download/pdf_1/euclid.aos/1176344552)\n", - "that it produces the correct standard deviation.\n", - "\n", - "This method is very useful for small ensembles of data points. \n", - "\n", - "\n", - "## Bootstrapping\n", - "Given a set of $N$ data, assume that we are interested in some \n", - "observable $\\theta$ which may be estimated from that set. This observable can also be for example the result of a fit based on all $N$ raw data. \n", - "Let us call the value of the observable obtained from the original \n", - "data set $\\hat{\\theta}$. One recreates from the sample repeatedly \n", - "other samples by choosing randomly $N$ data out of the original set. \n", - "This costs essentially nothing, since we just recycle the original data set for the building of new sets. \n", - "\n", - "\n", - "## Bootstrapping, recipe\n", - "Let us assume we have done this $K$ times and thus have $K$ sets of $N$ \n", - "data values each. \n", - "Of course some values will enter more than once in the new sets. For each of these sets one computes the observable $\\theta$ resulting in values $\\theta_k$ with $k = 1,...,K$. Then one determines" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "$$\n", - "\\tilde{\\theta} = \\frac{1}{K} \\sum_{k=1}^K \\theta_k,\n", - "$$" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "and" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "$$\n", - "sigma^2_{\\tilde{\\theta}} = \\frac{1}{K} \\sum_{k=1}^K \\left(\\theta_k-\\tilde{\\theta}\\right)^2.\n", - "$$" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "These are estimators for $\\angle\\theta\\rangle$ and its variance. They are not unbiased and therefore \n", - "$\\tilde{\\theta}\\neq\\hat{\\theta}$ for finite K. \n", - "\n", - "The difference is called bias and gives an idea on how far away the result may be from \n", - "the true $\\angle\\theta\\rangle$. As final result for the observable one quotes $\\angle\\theta\\rangle = \\tilde{\\theta} \\pm \\sigma_{\\tilde{\\theta}}$ .\n", - "\n", - "\n", - "\n", - "## Bootstrapping, [code](https://github.com/CompPhysics/MachineLearning/blob/master/doc/Programs/Sampling/analysis.py)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - " # Bootstrap\n", - " @timeFunction\n", - " def bootstrap(self, nBoots = 1000):\n", - " bootVec = np.zeros(nBoots)\n", - " for k in range(0,nBoots):\n", - " bootVec[k] = np.average(np.random.choice(self.data, len(self.data)))\n", - " self.bootAvg = np.average(bootVec)\n", - " self.bootVar = np.var(bootVec)\n", - " self.bootStd = np.std(bootVec)\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Jackknife, [code](https://github.com/CompPhysics/MachineLearning/blob/master/doc/Programs/Sampling/analysis.py)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - " # Jackknife\n", - " @timeFunction\n", - " def jackknife(self):\n", - " jackknVec = np.zeros(len(self.data))\n", - " for k in range(0,len(self.data)):\n", - " jackknVec[k] = np.average(np.delete(self.data, k))\n", - " self.jackknAvg = self.avg - (len(self.data) - 1) * (np.average(jackknVec) - self.avg)\n", - " self.jackknVar = float(len(self.data) - 1) * np.var(jackknVec)\n", - " self.jackknStd = np.sqrt(self.jackknVar)\n" + "As can be seen from the plot, the first point gives back the variance and a value of one. \n", + "For the remaining values we notice that there are still non-zero values for the auto-correlation function." ] } ], "metadata": {}, "nbformat": 4, - "nbformat_minor": 4 + "nbformat_minor": 5 } diff --git a/doc/LectureNotes/_build/html/searchindex.js b/doc/LectureNotes/_build/html/searchindex.js index 25d4af919..f0005bdfd 100644 --- a/doc/LectureNotes/_build/html/searchindex.js +++ b/doc/LectureNotes/_build/html/searchindex.js @@ -1 +1 @@ -Search.setIndex({docnames:["chapter1","chapter10","chapter11","chapter12","chapter13","chapter2","chapter3","chapter4","chapter5","chapter6","chapter7","chapter8","chapter9","chapteroptimization","clustering","intro","linalg","schedule","statistics","teachers","textbooks"],envversion:{"sphinx.domains.c":2,"sphinx.domains.changeset":1,"sphinx.domains.citation":1,"sphinx.domains.cpp":4,"sphinx.domains.index":1,"sphinx.domains.javascript":2,"sphinx.domains.math":2,"sphinx.domains.python":3,"sphinx.domains.rst":2,"sphinx.domains.std":2,"sphinx.ext.intersphinx":1,sphinx:56},filenames:["chapter1.ipynb","chapter10.ipynb","chapter11.ipynb","chapter12.ipynb","chapter13.ipynb","chapter2.ipynb","chapter3.ipynb","chapter4.ipynb","chapter5.ipynb","chapter6.ipynb","chapter7.ipynb","chapter8.ipynb","chapter9.ipynb","chapteroptimization.ipynb","clustering.ipynb","intro.md","linalg.ipynb","schedule.md","statistics.ipynb","teachers.md","textbooks.md"],objects:{},objnames:{},objtypes:{},terms:{"0":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,18,19],"00":[0,1,3,5,6,11],"000":[1,3],"0000":[],"00000":[],"000000":[5,11],"00000000e":5,"0000e":3,"0001":[1,3],"000148":[],"00019998":5,"00024087":5,"00028369228101198006":[],"00029012":5,"0003043256065368937":9,"00031535148309580783":6,"0003153514830958081":[],"0003153514830958126":[],"0003153514830958235":6,"00034944":5,"000417932":2,"00042089":5,"000464088":2,"00050694":5,"00058016":6,"000599":[],"00060705":[],"00060708":6,"00061058":5,"00062582":6,"00062595":[],"00066668":6,"00068734":6,"00068941":6,"00069103":[],"00073541":5,"00076495":6,"00076919":6,"00076925":[],"0007698473260556325":6,"0007698473260556334":[],"0007698473260556339":[],"0007698473260556344":6,"00079123":[],"00079130":6,"00079968":5,"00084705":[],"00084711":6,"00085883":6,"00085884":[],"00087697":6,"00088573":5,"00092645":6,"00092646":[],"00096314":5,"001":[1,2,8,13],"00100519":6,"0010479245355337968":[],"0010479245887943952":6,"00105081":6,"00106677":5,"00107405":6,"00111756":6,"0011526":6,"00115999":5,"00118504":6,"00118527":[],"001263":[],"00128479":5,"001323":6,"00137823":6,"00137835":[],"00139705":5,"00149311":6,"00149956":6,"00152117":6,"00154733":5,"00156379":[],"00156382":6,"00168251":5,"00174276":6,"00175331":6,"00186347":5,"001988":[],"00198806":[],"00200":8,"00202624":5,"00202756":6,"00217499":6,"00224413":5,"00228741":6,"0023548":6,"002381316302584885":[],"002381316302584886":6,"0023813163025848865":6,"00242999":6,"00243186":6,"0024401":5,"00249435":6,"002526":[],"00266858":2,"00270244":5,"00274989":6,"00275135":[],"00289724":6,"00293838":5,"002948":[],"0030828":[],"003083":[],"00310113":2,"00312361":6,"00315593":6,"003215318065760509":[],"0032153180657605125":6,"00323332":6,"0032542":5,"003301":6,"003395515459204093":6,"0033955154592040944":[],"00353823":5,"0036237":6,"0036367":6,"00369758":6,"003704":[],"003717":[],"003759":[],"003774":[],"003788":[],"0038335":6,"003909404072811217":6,"003909404072811231":[],"00391839":5,"0039987":6,"004":5,"004091940707753948":6,"0040919407077539514":[],"00410387":6,"00410478":6,"004113634617443116":6,"0041136346174431284":[],"004113634617443131":6,"004113634617443135":[],"00411363461744314":6,"004113634617443141":[],"004155986345861364":6,"004155986345861374":[],"00415763":[],"00424909":2,"00424967":6,"00426027":5,"00433417":11,"00440346":6,"00443743":6,"00445655":11,"004579219539673834":6,"004579219539673836":[],"00458878":6,"004610275230656187":6,"004610275230656294":[],"00462287":6,"00471782":5,"00472199":6,"00472512":6,"0049544":6,"004999999999999984":0,"004999999999999994":[],"00512927":5,"00517114":6,"00526348":6,"0053018":6,"00554552":6,"00556826":6,"0056799":5,"00577441":[],"00579953":6,"00588657":6,"00607783":6,"006162":6,"00617499":5,"00630331":6,"00642221":6,"00660427":6,"00672607":6,"00673393":13,"00673407":6,"00676387":6,"0068011":6,"00683748":5,"00683964":6,"007012403613997257":[],"007024126888936694":[],"0070241268889382116":6,"00719176":6,"00727646693":0,"007315":[],"0074331":5,"00751823":18,"00759119":6,"00784393":6,"00803064":6,"0080866254785146":[],"00813803":6,"00817631":6,"00823002":5,"00827728":6,"00831018":6,"00834567":6,"00848904":6,"0086649156":0,"008675369724975977":5,"008675369724976501":[],"00894639":5,"00905423":6,"009163470508352211":[],"009163470508352218":5,"009164545680330616":6,"00917248":6,"00934499":6,"009450756365829578":[],"0096208":6,"00990475":5,"00992331":6,"00996754":6,"00it":6,"01":[0,1,2,5,6,9,11,13,20],"010018312644139205":6,"010018312644139347":[],"0100706":6,"010331721306655144":6,"01033172130665515":[],"010516485576646513":6,"010516485576652856":[],"01066519":6,"01076611":5,"0110":18,"0110407067093945":18,"01104071":18,"011076219011339788":[],"01107621901137465":6,"011225":2,"0113104":6,"01179792":6,"01191824":5,"012073649439965807":[],"012073649472576395":6,"01219292":6,"01223198":6,"01231917":6,"01290947":6,"01295356":5,"013121573975499602":[],"013121573975499604":[],"01312157406137079":6,"013121574061370796":6,"01318643":6,"01347916":6,"01348565":6,"01367553":6,"01397146":6,"01405935":6,"01416528":6,"014209325470380275":[],"01433809":5,"014436800088896274":6,"014436800088969727":[],"01449782":6,"01458337":6,"0146081":6,"01463049":6,"015072388895177088":[],"0150723888951771":6,"015072388895177109":[],"015072388895177239":6,"01509543":13,"01530715721129232":[],"01531845":6,"01538461538462":[],"01549377":6,"01558197":5,"016285782696017055":6,"016285782696075054":[],"01633913":6,"01640891":6,"01655318":6,"016587414993037307":[],"016587414993045405":6,"01691985":6,"01708781":6,"01708852":6,"01713366":6,"01724499":5,"01735584819559184":[],"017355848195591845":[],"017355848195593354":6,"01756972":[],"018232":[],"01831036e":6,"01831130e":[],"01831200e":[],"01831207e":6,"01866537":6,"01873344":11,"01873869":5,"01898855":6,"01905883":6,"01908936":6,"019587":[],"01963611":6,"01969145":6,"019754165271682844":6,"01975416527179247":[],"01975848":6,"02":[0,4,6,7,12],"02024962":6,"02054837e":6,"02068067":6,"02073509":5,"02098261":6,"02123176":6,"021250026482402":11,"02159270458799264":[],"021592704587992645":[],"021592704588021164":6,"021592704588021167":6,"02198702e":6,"022022882954618364":[],"02208512":6,"02228115":6,"02229529":6,"02252765":5,"022556":11,"02255619":11,"02299949826036602":[],"0229994982603662":6,"02348765":6,"02365049":6,"0245528":6,"02492265":5,"02493054":[],"02498832":6,"02503753":6,"02511518":6,"02522069":6,"025709":11,"02586427":6,"0260906":6,"02625193":8,"02625928":13,"026408391362671896":[],"026605727637176654":[],"02660572763718461":6,"026605727637184613":6,"02707227":5,"02723445":6,"027609773491022314":[],"027609773491022387":6,"027609773491022407":6,"02786488":11,"027865":11,"02857":4,"02917662":11,"029177":11,"02926396":[],"029613363972682966":18,"029733":[],"02976145":6,"02994311":5,"02f":6,"03":[1,6],"03032441e":6,"03056589":[],"030566":[],"03076923076924":[],"03077640549":4,"03099776":5,"031":5,"03187339273559067":[],"03196357":6,"03251863":5,"03256632e":1,"03267527":6,"03278964":[],"032790":[],"03279636":6,"0330308045180234":[],"0330308045183163":[],"0330308045183219":6,"0330308045190182":6,"033657685071527485":[],"033657685071527624":6,"03382304823545749":11,"03447512":6,"03562355":6,"03568439":6,"0358909447132981":[],"0359565":5,"03630548":6,"03707133":11,"037559":11,"03755944":11,"03761519":18,"03781367141738885":[],"03781367141738886":[],"037813671417388985":6,"03781367141738899":6,"03794112":16,"03814292":6,"03815288":6,"038300":11,"03850557":11,"038506":11,"039039":5,"03981057":16,"0399676689527968":6,"0399676689527975":6,"04":[1,6,11],"04010697":6,"04063602":6,"041":[],"041148729430502":6,"041148729430523":6,"0411487294305246":[],"0411487294305746":[],"04220758":6,"04284519":[],"043":[],"04315108":5,"0433816":[],"04346721":5,"04355837":6,"04362":[],"04368707":16,"0437499":2,"04389027":6,"043927769551648":[],"04423486":6,"044334":[],"044613":6,"04483457":[],"04537385":6,"04543942":6,"04566964":6,"0458":9,"04615384615386":[],"04621521":[],"04648335":5,"0466":[],"04683565":5,"04775904":16,"04784395":6,"04818727730430296":[],"048187277304303056":6,"048653428302840175":16,"04892055":6,"04909093":6,"04912436":6,"0495569966278238":6,"0495569966278269":6,"0495569966278315":[],"04it":[],"05":[1,3,4,6,13],"05009826":6,"05087958":[],"05100875":6,"051649":11,"0517473":5,"05227921801205691":[],"05227921801205692":[],"052279218012057004":6,"05263":[],"05290417684691035":18,"05302":[],"05364854":8,"05383795":6,"05434571":16,"05447415":6,"054585":[],"054785":[],"054963":[],"05505310046362":[],"05505310046363":2,"055137":[],"055320":[],"05533":[],"055676":[],"05614483":5,"05623":[],"05648":[],"05651951":6,"056528":11,"05667":[],"057088709963182":[],"057163681553428394":6,"05716368155351588":[],"0572":[],"05756733":[],"05785343":6,"05789007":6,"057899":11,"05796251":6,"05807125":6,"05883":[],"05884":[],"059013":5,"059294":11,"059378":11,"059601":5,"059783":[],"05999":[],"06":[6,13],"060080":[],"060183":[],"06020587":6,"06021285":[],"060213":[],"060228":11,"060309":5,"06043581":6,"060507":[],"060655":[],"060781":[],"060841":[],"061025":[],"061048":[],"061149":5,"061321":[],"061390":[],"061509":[],"061604":[],"06160438":[],"061701":[],"061745":[],"061898":[],"061917":[],"061923":11,"06200174":5,"06208238634231944":6,"06208238634231953":[],"062136":5,"062142":[],"062292565":4,"062294":[],"062391":[],"062409":11,"062411":11,"062435":[],"062451":[],"062534":[],"062676":11,"062706":11,"062834":[],"062884":[],"063052":5,"063078":[],"063250":[],"063343":[],"063364":5,"063513":[],"063631":5,"063657":[],"063752":11,"063851":5,"063872":[],"063874":5,"06388888888888888":[],"064052":[],"064108":[],"064110":5,"064184":[],"064274":5,"064384":[],"06444":[],"064469":[],"06453579006728315":[],"06453579006728322":6,"064538":11,"064604":11,"064648":[],"064658":5,"064681":[],"064838":5,"06491736":6,"064918":[],"064931":11,"064964":[],"06497046":18,"065020":5,"065106":[],"065249":11,"065314":11,"065348":5,"06547790180152352":6,"06547790180152353":6,"06547790180152357":[],"06547790180152363":[],"065514":[],"065557":[],"065575":[],"065641":11,"065730":11,"065826":5,"065892":[],"065974":5,"066110":[],"066194":[],"066309":[],"06637":[],"066449":5,"066558":5,"066609":[],"066670":11,"0666807":2,"066729":11,"066753":[],"066789":5,"066837":11,"066954":[],"067079":[],"067213":5,"067236":[],"067240":5,"06724062":5,"067282":11,"067392":11,"067402":11,"067432":11,"067544":[],"067597":5,"067599":5,"067612":5,"067620":5,"067622":5,"067675":[],"067714":[],"067723":5,"067769":5,"067956":11,"068059":11,"068093":11,"068094":[],"068103":[],"068120":[],"068318":5,"068363":5,"068376":5,"06844519414009438":[],"06844519414009442":6,"06844519414009444":6,"068461":11,"068538":11,"06858699":[],"068608":5,"068642":5,"068659":[],"068735":5,"068806":5,"068809":[],"068831":[],"068937":[],"068965":5,"068991":5,"068992":5,"069085":11,"069091":[],"069181":5,"069275":11,"069408":5,"069409":5,"069441":5,"069455":[],"069584":[],"069672":[],"069685":11,"069754":[],"069796":5,"069847":5,"069877":11,"069884":11,"069888":5,"069912":[],"06it":6,"07":6,"070028":[],"070043":[],"070071":[],"070074":5,"070150":5,"07016":[],"070163":11,"07017":[],"070343":11,"070379":[],"07039":[],"070419":11,"070429":11,"070491":5,"070492":11,"070576":[],"070612":[],"07062318":6,"070630":5,"070635":[],"070889":11,"070911":[],"070950":11,"070972":11,"07099747918547346":[],"071022":11,"071086":[],"071112":5,"071136":5,"07115":[],"0713":0,"071387":[],"07145103":11,"071467":5,"0714956":16,"071529":5,"07160048164232467":6,"07160048164248561":[],"071614":5,"071632":5,"071698":[],"071705":[],"071767":5,"071921":5,"071935":[],"07201957":[],"072216":11,"072242":[],"072348":[],"072442":11,"072492":[],"072530":[],"072617":[],"072620":[],"072750":[],"07285":3,"0728785":[],"072928":[],"073016":[],"073062":11,"073136":11,"073225":[],"073230":11,"073280":11,"073362":[],"073368":[],"073423":[],"073436":11,"073449":5,"07345504":[],"073531":[],"073575":11,"073583":[5,11],"073598":11,"073629":11,"073695":[],"073699":5,"073719":5,"073761":11,"073810":11,"073816":5,"073827":11,"073907":[],"073949":5,"074034":5,"074067":11,"074111":11,"07413172":18,"074149":11,"07421084":5,"074237":5,"074336":[],"074375":5,"074488":[],"074530":11,"07456491":5,"074573":11,"074666":[],"074693":5,"07490892":6,"074945":[],"075195":5,"075241":5,"075573":[],"075594":5,"075760":5,"075820":[],"075959":5,"076130":11,"076204":[],"076216":11,"076338":5,"0764924":6,"07656896":[],"076589":5,"076604":11,"076628":5,"07678":[],"076780":11,"076804":11,"076895":11,"076905":11,"076938":11,"076955":5,"077003":5,"077013":11,"077015":11,"077022":[],"077083":11,"077211":11,"077349":11,"07735703":16,"077402":5,"077425":5,"077452":11,"077467":[],"07777777777777778":1,"077793":5,"078040":5,"0782":6,"07820":[],"078253":[],"078254":11,"078269":11,"078299":11,"078314":11,"078436":11,"078548":[],"07864":[],"078667":[],"07871":[],"07878641":16,"078911":[],"078992":[],"07903849":16,"079157":5,"079281":5,"079347":[],"079383":5,"079389":5,"079405":5,"079406":[],"07944154":16,"07968918676726029":[],"0796891867672603":6,"079729":[],"079839":[],"079847":[],"079893":[],"07989327":[],"07999999999998":[],"08":[13,18],"080106":[],"080137":[],"080256":[],"080297":5,"080312":[],"080319":5,"080334":[],"080347":[],"080370":[],"080406":[],"080410":[],"08041015":[],"08043851":5,"080517":[],"080582":11,"080593":11,"080647":11,"080706":11,"080738":11,"080801":[],"080847":5,"08085812":18,"080916":[],"080984":11,"081046":11,"081072":11,"081140":[],"081144":11,"081233":5,"081253":5,"08131003":6,"081388":11,"081431":[],"081483":[],"081533":5,"081540":[],"08156108":6,"081773":[],"082174":[],"082183":[],"082295":11,"082451":[],"082503":11,"08251519":6,"082536":11,"082896":11,"082900":[],"08299273e":6,"083015":5,"083053":[],"08318298e":1,"083220":[],"083227":[],"083276":11,"08328216846752691":11,"08333333333333333":1,"08336233266":4,"083404":[],"083441":[],"083506":[],"083527":[],"083600":[],"083692":[],"08376632":6,"083766322923899":6,"0837663229239016":[],"0837663229239025":[],"0837663229239043":6,"083799":[],"083829":[],"083977":[],"084000":[],"084101":[],"084184":[],"084224":[],"084226":[],"08426840630693411":[],"08426840630693412":6,"08426840630693413":6,"08428156":16,"084282":[],"084364":[],"084400":[],"084408":[],"084414":5,"084444":[],"084484":11,"084536":[],"084549":[],"08455":[],"084570":[],"084604":[],"084629":[],"08464758160254343":5,"084670":[],"084683":[],"084702":5,"08474":[],"084777":[],"084862":[5,11],"084904":5,"084909":5,"084965":[],"085018":5,"085044":[],"085163":11,"085167":[],"085184":[],"085185":[],"085361":[],"085416":[],"08551306":6,"085764":[],"08576932":6,"085879":11,"08593216":6,"086074":11,"086076":5,"08611111111111111":1,"086112":11,"086303":[],"08630331":[],"086567":[],"08673755293381497":[],"086807":5,"086872":5,"086890":11,"08690":[],"086956":11,"087501":[],"08758":[],"08759":[],"087834":[],"08816688":[],"088176":[],"0881981":5,"088212":5,"088526":11,"088611":[],"088631":[],"08871404":5,"088809":[],"088816":[],"088825":[],"088853":[],"08888888888888889":1,"088926":[],"08902":[],"089059":[],"08917679":5,"089177":5,"089212":[],"089329":[],"089425":[],"089501":11,"089614":[],"0896981":6,"08996":[],"09":1,"090028":[],"090229":[],"09085624":16,"090929":[],"091051":11,"091060":[],"09117221":[],"091224":[],"091236":[],"091340":[],"091349":[],"091363":[],"091414":[],"091416":[],"091426":[],"09149881":[],"091630":[],"09166666666666666":1,"091696":11,"0917":9,"09170751":0,"09172408":6,"091891":[],"092066":16,"09216046":18,"0923":[],"09251":[],"09297039":[],"093247":[],"0934597075922044":[],"0938":[],"094082198961999e":6,"0940821989624095e":[],"094082198966615e":6,"0940821989673748e":[],"09444444444444444":1,"094472507965532":11,"095510":[],"095702":[],"095871":[],"09609807":5,"096173":[],"096472":11,"096623":[],"09672929714683368":[],"097360":[],"09744":[],"09744272":6,"09780":[],"09791":[],"09849763":11,"098498":11,"09856879":16,"09861229":16,"098802859381565":16,"099":[],"09903804":8,"0991919894927399":6,"099191989493334":[],"09951287404314545":1,"0n":0,"0s":[3,4],"0x11da42d90":13,"0x11e031e50":13,"0x7fad10f9a280":[],"0x7fad20f69be0":[],"0x7fd098df3280":[],"0x7fd0a9063be0":[],"1":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,17,18,19,20],"10":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,16,18],"100":[0,1,2,3,4,5,6,7,8,9,10,11,13,14,16,18,19],"1000":[0,1,2,4,5,8,11,13,14,15,18],"10000":[2,5,6,10,11,18],"100000":8,"10001":10,"1001":18,"1002":18,"1003":18,"10030":[],"10044225464078282":[],"1005":18,"10077114273548984":6,"1009":18,"1011":18,"1013":18,"1013904243":18,"10141413e":6,"1015":18,"102":3,"1022233262115424":[],"10222333":[],"1023":18,"10230":[],"1023111":16,"1024":3,"1026":18,"1027":18,"1028":[],"103":1,"1030":18,"10302062":16,"10340":[],"1037":18,"10378326e":1,"1038":18,"10391807":6,"10398646080125035":[],"10398646080125036":6,"10398646080125037":6,"1040":18,"10405456":11,"10430":[],"104411":[],"10455924":18,"1047":18,"10505137":16,"10520":[],"10555555555555556":1,"1056":[],"10589577":5,"106":[],"106095":11,"10620135":16,"108":6,"109":[],"10913":6,"10931453":6,"10960":[],"10th":9,"10x":0,"11":[0,2,3,4,5,6,7,8,9,10,11,12,13,16,18,20],"110":[],"1100":18,"1101":18,"11022302e":5,"11046771":16,"111":[1,7,12],"11100":[],"1122558214":[],"112383":11,"1124":[],"1125":3,"11297834":[],"1136":3,"11362930e":[],"11388888888888889":1,"1139":3,"11390":[],"114":3,"11402309":16,"114437":[],"11462415":5,"11482289e":6,"11499517":[],"11507992e":1,"11547777218875695":6,"11547777218875696":6,"11547777218940905":[],"11547777218940906":[],"115822":6,"11590":[],"11660":[],"11666666666666667":1,"117":8,"11744554e":6,"11780":[],"11790868":[],"117986":[],"118":2,"1182":[],"1183":[],"118318":[],"1184":4,"11840":[],"1185":[],"1186":[],"11890":[],"119":2,"1199":[],"119936":2,"119999999999976":[],"11it":6,"12":[0,1,2,3,4,5,6,8,9,11,12,13,16,18,20],"120":[2,3],"1203":[],"1203284":8,"1206":8,"12069773":16,"121":[2,8,9,10],"12182967":6,"12196674":[],"122":[2,8,9,10],"12222222222222222":1,"122439":[],"1224392":[],"12288563":5,"122886":5,"123":[],"12318726e":6,"12333649":6,"12345260617257443":9,"123459876":18,"123711":6,"12380":[],"124":0,"12400":[],"125":[],"1250":[],"125000":[],"12552073e":6,"126":[],"1261":[],"12618549":5,"1265":[],"127":4,"1270":[],"127043":[],"1271":6,"1277":6,"127773":18,"12777777777777777":1,"12790":[],"128":[3,4,13],"128664":6,"12945452":[],"1297":[],"1298":9,"12adb44b1c20":7,"12m":3,"13":[0,2,4,5,6,9,11,12,13,16,18],"130":[],"13003291":6,"13055555555555556":1,"131":[],"1314":[],"132":[],"13220608e":6,"13229545716505003":16,"132360":[],"1323603":[],"1326":[],"13280":[],"133":7,"13333333333333333":[],"135":[],"13519106":[],"13535942":6,"136":[],"1360":[],"1361":[],"1362":[],"1363":[],"1364":[],"13646574":5,"13661243e":6,"13679863":6,"1371":6,"13740":[],"137400784702911":[],"137652":11,"1378":[],"1382":[3,4],"1383":[3,4],"1384":[3,4],"1385":[3,4],"1386":[3,4],"13865173":5,"13876586436927824":5,"138775":11,"1388888888888889":1,"13890":[],"1392559581788775e":[],"1392559584983597e":[],"1392559585048734e":6,"13925595925919e":6,"14":[0,2,4,5,6,8,9,10,11,12,13,16,18,20],"140":[],"14021063":6,"1404":[],"14042769":[],"140428":[],"141":2,"14100":[],"1416398":6,"14174745":6,"1418":[],"142":2,"14250":[],"142857":[],"143":[2,7],"1435666":18,"14360598":[],"1437":1,"144":2,"14400":[],"1440501043841336":1,"14440":[],"1446729567":4,"14484695":16,"145":2,"146":[],"147":[],"14710":[],"14722222222222223":1,"147400":11,"14741468":[],"147420":11,"14783702":18,"1479":[],"148":[3,4],"14812206":6,"14818":[],"1484256":[],"148564":6,"14859":6,"148768":[],"149":[3,4],"149213":6,"14932651":[],"14988578":[],"149886":[],"14g":6,"14it":6,"15":[0,2,4,6,7,8,9,12,13,16,18],"150":[3,4,8],"15005476":5,"15098090e":6,"151":[3,4],"15130074e":6,"151986":[],"152":[3,4],"15200":[],"15258907":[],"1527777777777778":1,"153036":[],"153106":[],"1533795":[],"153760":[],"15384615384616":[],"154":[],"15454301":[],"154720":[],"15483121":18,"154911":[],"155":[],"155491":[],"155687":[],"155883":[],"156":[],"1562":[],"15629539":[],"15680777":16,"157":[],"15717291":16,"1575":[],"158":[],"1583767":16,"1586300629904382":[],"1587":[],"15891336":16,"159":[],"1590":[],"15990":[],"15g":6,"15it":[],"16":[1,2,3,4,5,6,8,9,10,13,18],"160":[],"1603":3,"1604":[],"1605":[],"160539":[],"1606":[],"1607":[],"16111111111111112":1,"1612":[],"16211139":5,"16220":[],"16231451":4,"1625":[],"1628":[],"163":[],"1630775253":[],"16342407":5,"16343471":6,"16384":3,"16500":[],"166":[],"16660817":[],"166667":[],"167":[],"16740002":16,"168":[],"16805821e":6,"16807":18,"16827044":18,"16832385":18,"16b8e3cda33a":1,"17":[1,2,4,5,6,8,13,18],"17084902":[],"1709":[],"17174962e":1,"17222222222222222":1,"1726":[],"17300":[],"1731":[],"17339342":[],"17446471":6,"1752":[],"175300":[],"17641709":6,"176880142835407":[],"17777777777777778":1,"17801022":5,"17861098":6,"17917768":5,"17930649":16,"17949575":5,"17953942":11,"1797":[1,3],"18":[2,4,6,7,8,9,10,13,18],"180":[],"18029127":5,"1807":4,"1809":[],"181":[],"1810":4,"1812":[],"18156717":16,"1821":[],"18276924":[],"18327677":[],"18333333333333332":1,"18393678":18,"184":[],"184519":[],"184895":[],"185":[],"1851":[3,4],"1852":[3,4],"18526":[],"185278747229417":[],"1853":[3,4],"1854":[3,4],"1855":[3,4],"186":[],"1860":[],"1861":[],"18611111111111112":1,"18613217e":6,"18660":[],"18673098":11,"18682538":18,"1887":6,"18912963":16,"189496":[],"189622":[],"18it":[],"19":[2,4,6,13,18],"19003":6,"191262820314401":[],"19166666666666668":1,"19207979":5,"19213479":18,"19220":[],"193":[],"19354258":18,"19394283":[],"194":[],"1940":0,"19404282648955e":6,"194042826653172e":[],"194042826815498e":[],"1940428268204826e":6,"1943":12,"1944":[],"19444444444444445":[],"1950915150":[],"1954":[],"1956":[],"19569961":6,"1961":[],"1962":[],"1963":[],"1964":[],"1965":[],"1970":16,"1973":9,"197370":11,"19740":[],"1979":6,"19800":[],"1989":18,"199":[],"19937":18,"19994371":6,"1_1":12,"1_2":12,"1_3":12,"1cm":[0,8,10,18],"1d":[1,2,3],"1e":[1,2,3,4,14],"1e10":14,"1e4":6,"1f":1,"1k":16,"1n":0,"1s":3,"1x":0,"2":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,18,20],"20":[0,1,2,4,6,7,8,13,18],"200":[0,2,3,4,8,9,10,18],"2000":0,"200000":[],"20015436":6,"2004":13,"2006":20,"2010":1,"2011":1,"2014":4,"2015":1,"2016":0,"20174121":16,"2018":[0,6],"2018906331":[],"2019":[],"2021":[6,14],"2022":17,"2027":[],"20277777777777778":1,"203":6,"20320502":[],"20500":[],"206":[],"2060":[],"20661216":[],"2069":[],"207545":11,"20756638":[],"20833333333333334":1,"20867052175006306":6,"20867052175109335":[],"20980":[],"21":[0,1,2,4,5,6,7,9,12,13,16,18],"210340":11,"21058097":5,"21130":[],"2116753732":4,"21169159e":6,"21265216":[],"213":6,"213103":11,"213743":11,"21472683":16,"2147483647":18,"21519063":16,"21596432":6,"216290":11,"216683":11,"21786964":16,"2188":[],"21919813":18,"21944444444444444":[],"2195":3,"21985165":13,"21997099":16,"22":[0,1,2,4,5,6,12,13,16,18],"22044605e":5,"22094791":18,"221":8,"221180":[],"2216":[],"2218":[],"221805":2,"222400":[],"2225":3,"225":4,"2259440937":[],"226124657522696":[],"22612466":[],"22690428":5,"2284246870217459":6,"22842468702288576":[],"22842468702288582":[],"22847924":5,"2299":3,"22it":6,"23":[1,2,4,6,7,12,13,16,18],"23002365e":6,"231":[],"23103285":[],"2315033":[],"23167717":5,"2321528":5,"232153":5,"232435":[],"23297056":18,"23333333333333334":1,"233528":5,"234":6,"23522201":[],"2357089093":[],"2360682191515046":18,"2361111111111111":[],"2364":[],"237038":5,"23703839":5,"2379":6,"2397":[],"24":[0,1,2,4,6,13,16],"24140":[],"2416":[],"24175744e":6,"2419":[],"24276315":16,"2430":[],"24390":[],"24444444444444444":1,"244858":[],"24485843":[],"24512498":16,"246":2,"2470":[],"24770094":6,"24829908":5,"24906604e":6,"2495":[],"25":[2,3,4,5,6,7,8,9,11,13],"250":[2,4,7,9],"2500":[],"25000":0,"250000":[],"250154":[],"2517560119":[],"251879":[],"252436":[],"25259666":16,"2526":3,"252866":[],"25286618":[],"253775":[],"255":3,"255001":[],"256":[2,4],"25617654e":6,"25617657e":[],"25617658e":6,"256962":[],"257":[],"2572":[],"2572495066":[],"2572e3a4b38d":1,"2575":[],"25792767":16,"25845e8df859":6,"259153":11,"2597":[],"25976336":[],"26":[2,4,6,13],"26153846153846":[],"26186844":16,"2619":[],"26290036":[],"26294938":[],"2629493813057833":[],"26301436":5,"26306244":[],"264":4,"26409315307910025":6,"2640931530791003":[],"2640931530791005":[],"26409315307910053":6,"265":[],"2650":[],"265109911":4,"2654":[],"266":[],"26610075":[],"26666667":13,"26710969":5,"26780278":5,"268":[],"268227":[],"26822717":[],"269":[],"27":[0,1,2,4,6,13],"270":[],"2703":3,"27092897":6,"2750":[],"275341":[],"276263":11,"27692307692308":[],"27700":[],"2772":3,"2775623201":[],"27760":[],"278036":5,"27803645":5,"27n_":18,"28":[1,3,4,6],"28001319":[],"28047021":16,"280647":11,"28067036":16,"28097861":5,"280979":5,"2812":[],"282727":11,"283":[],"2830637392":4,"28336218e":6,"2836":18,"28390":[],"28475098":8,"28590743":[],"2861":18,"28634473":11,"286345":11,"2871":[],"2873":9,"2882":18,"2886":18,"2886847885377843":18,"28868479":18,"28875373":16,"2890":0,"2892":18,"28967287":16,"29":[4,6,7,18],"29149329":[],"2915":18,"29167186":5,"29229741":13,"2923076923077":[],"29512284":11,"295123":11,"2954":[3,4],"2955":[3,4],"2956":[3,4],"2957":[3,4],"2958":[3,4],"29588901":13,"29592539":[],"296247":[],"297":[],"2971492148":[],"2972":[],"29822833":6,"298273":[],"298375":[],"299267190588216":11,"299748":[],"2_":[12,18],"2_1":12,"2_2":12,"2_3":12,"2_i":12,"2_m":[6,18],"2_t":13,"2_x":18,"2b":18,"2cm":8,"2d":[1,3,11,12,15],"2e":6,"2f":[0,7,9,10,11,12],"2ff97f4bf03b":18,"2g":2,"2g_i":2,"2k":3,"2m":6,"2n":[0,2,3],"2nd":9,"2p":18,"2pt":4,"2s":4,"2x":[0,3,8,13],"2x_ix_jy_iy_j":8,"2x_j":8,"2y_i":10,"2y_j":8,"3":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,18,19],"30":[0,1,4,6,7,10,13],"30000":0,"30010":[],"30119421":8,"3017":[],"3018":[],"3019":[],"301927":[],"3020":[],"3021":[],"30258509":16,"302m":4,"303":6,"3037":[],"3038":[],"30384239":16,"3039":[],"303m":4,"3040":[],"3041":[],"30466214e":6,"304m":4,"30535506":[],"30552077":16,"30559547":[],"305m":4,"3063":[],"306m":4,"3072":3,"30787294":6,"307m":4,"308m":4,"309":0,"30940":[],"30b53504a633":0,"31":[4,6,12,16,18],"310":0,"311":0,"311m":4,"312":0,"3123314713548606":6,"31276579e":6,"312m":4,"313":0,"31318084":5,"314":[],"31457796":5,"315":6,"3155":[5,6],"316":[],"31605061":16,"317":[],"31718909":11,"317367":11,"317m":4,"31866499":13,"31896852":8,"319m":4,"32":[3,4,6,12,13,16,18],"3200":1,"32047562":[],"3208":2,"320m":4,"32149601703519115":6,"3214960170351912":6,"3215":[],"32179365":[],"32320052":16,"324":2,"32458459":[],"324m":4,"3250":[1,6],"32512":[],"326238":[],"326m":4,"32721178":11,"327212":11,"32903042":[],"3297":[],"32992274":[],"329923":[],"33":[3,4,12,14,16],"3303":[],"33066907e":5,"3310":[],"33166055e":5,"331939":[],"331m":4,"332331":[],"333":7,"3331":[],"33327369":[],"333274":[],"33333333":13,"3337":[],"33611111111111114":[],"33711888":18,"3384":[],"33900713":18,"33m":4,"34":[4,7,16],"3403":[],"340782":11,"34114547":5,"341m":4,"342680":[],"34305928":[],"3436":0,"3437":0,"3456":[],"34568872":16,"34569596":5,"3457":[],"3458":[],"34585355":16,"3459":[],"3460":[],"346810":[],"3469819128513336":[],"34740615":[],"3498":[],"34it":6,"35":[0,2,4,6,7],"3514":[],"35140":[],"35147135":16,"351636":11,"35182854":5,"35203688":16,"3522":[],"3525":[],"3528556":[],"352856":[],"3529":[],"3530606977":[],"3538":[],"3543":[],"3544313922":[],"3546":[],"35470445e":5,"3548":[],"355":[],"3551":[],"35533773":6,"35533774":[],"3556":[],"3558":[],"3560":[],"356399":[],"3567":[],"3572":[],"3575":[],"357508":[],"3576":[],"3577":[],"35771842":6,"3581341341":4,"3585":[],"3587":[],"35892474":16,"359":5,"3593":[],"3594821":13,"3595":[],"3598":[],"359999999999985":[],"36":[2,4,5,6,7,18],"360":1,"3604":[],"3605":[],"3606":[],"360688":[],"3609":[],"36102113":[],"36117602":18,"3613":[],"3615":[],"361556":[],"3617":[],"3621311":5,"3624":[],"3627":[],"3628":[],"363295916323784e":6,"363295916414895e":[],"3632959170548605e":[],"363295924430451e":6,"363834":11,"36383443":11,"3643":[],"364418e97433":1,"3645":[],"3646":[],"3647":[],"3655":[],"3655222":5,"3659":[],"3668":[],"36689784":[],"366898":[],"3669":[],"367":2,"3672":[],"3673":[],"3674":[],"3679":[],"3684":[],"3687":[],"3688":[],"368m":4,"369139":11,"36it":[],"37":[2,4,6,7,18],"3704":[],"370782966":4,"3711":[],"3713":[],"3718":[],"3721":[],"3722":[],"3725":[],"3729":[],"37307168":16,"3739":[],"37396662":6,"37415316":16,"374291":5,"37429133":5,"3748":[],"3749":[],"3753":[],"3759":[],"3760":[],"3765":[],"376559":[],"37655936":[],"37703055":18,"3772":[],"3773":[],"37732":[],"3776":[],"3777801602":[],"3779":[],"37835429e":[],"3784":[],"37900111":6,"3791":[],"3794":[],"379647":11,"37964744":11,"37992857":[],"38":[2,4,7,18],"380":[],"3802":[],"3803":[],"3804":[],"3805":[],"380739":5,"38073947":5,"3811":[],"38135733e":6,"3815":4,"381627865854956":18,"38162787":18,"3817475779":[],"3818":[],"3819":[],"3820":[],"38246359":[],"3827":[],"382951":11,"38295101":11,"3830":4,"3833":[],"38336316":16,"3834":4,"3837":4,"3838":4,"3839":4,"3842":[],"3850":[],"3851":4,"38533184":6,"3854":[],"3858":4,"386":[],"3861":4,"3862":4,"38629436":16,"3864":4,"3867":[],"3869":4,"387":[],"3871":4,"3872":4,"387482":5,"38748219":5,"3876":[],"388":[],"3881":4,"3882":[],"3885":[],"38868469":16,"3888":4,"3889":4,"389":[],"38901478":[],"3891":4,"38916861e":6,"38962192e":6,"3898":[],"39":[0,2,4,13,19],"390":[],"3906":4,"39095416":[],"3914":4,"3915":4,"3916":4,"391602":[],"3917":4,"3922":4,"3925515884752442":[],"3928":4,"3931":[],"39311435":16,"3943":4,"3944":4,"39456996":16,"3950":4,"3955":4,"39560937":16,"39579407":5,"3958":[],"3960":4,"3962":4,"3970":4,"39706038":5,"3975":4,"397700":11,"39789527":16,"3979":4,"3980313467":[],"3983":4,"3994":4,"3996":4,"399836":[],"3d":[2,3,4,6,13],"3f":[1,3,9],"3n":16,"3x":[2,8],"3x_i":2,"3y":8,"4":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,18],"40":[1,4,6,19],"400":4,"4000":20,"4005":4,"4011":4,"40116777":[],"401168":[],"4012":4,"4017":4,"401842":11,"40214433":16,"4022":[],"4027":4,"4030":4,"40340043":18,"4034668048":[],"4038":4,"4039":4,"404":[],"4045":4,"4048":4,"4055":4,"405890":11,"4059":4,"40599799":[],"4068":4,"4069":4,"4075":4,"4077":[],"408":2,"4082":[4,6],"4084":4,"4087":[],"4088":4,"4089":4,"409":2,"40927184e":6,"40a38ad763f1":6,"41":[2,4,16],"410":2,"4100":4,"4107":[],"411":2,"4115":4,"4117":4,"412":2,"4128":4,"4131":4,"4143":4,"41433969":5,"4144":4,"4146":4,"41511965e":1,"4155":2,"4162706317":[],"4167":4,"4176":4,"4177":4,"418506":11,"4186":4,"41876267":5,"418763":5,"41882036e":6,"41882037e":6,"4192423635":[],"4198":4,"41990268":[],"41it":6,"42":[1,2,4,8,9,10,16],"420":[],"42078103":18,"421":2,"4212":4,"422":2,"4224":4,"423":2,"42394972":16,"424":2,"42441033":5,"42449643":18,"42450":[],"425":2,"4255":[],"4258989918":[],"426":[6,7],"42631342":[],"427017":11,"4275":4,"43":[1,2,4,7,16],"43054282":5,"43135183":16,"43330971e":6,"434932":11,"43493232":11,"435163":[],"43552433":16,"43579948e":6,"436462435":4,"43761347":[],"43766686":11,"4379":4,"438060758":[],"438136":[],"439230":6,"44":[1,2,4,16],"44089210e":5,"4410":[],"4411":[],"442600":11,"443217":[],"444":[],"44970586e":1,"45":[2,4,19],"450":[],"450257":11,"4504":[],"45290829":16,"4557763":11,"455947":[],"456":[],"456418966187335":[],"457":[2,4],"457770268480242":[],"458027":[],"458078":11,"45960079":5,"46":[2,4,19],"4601":[],"461":18,"461175":16,"462":7,"46383925e":6,"464424":[],"4644244":[],"46508305":16,"46567887":16,"466":18,"46602982":[],"469":[],"46932688":[],"4694":[],"46984697e":6,"469868":[],"46986815":[],"47":[4,19],"4703":4,"47042744":5,"470714":[],"47075725":6,"47116868e":6,"47125748":5,"47132891":5,"47387858":18,"47400238":[],"47432993":[],"4744":[],"47478057":[],"475311":[],"47531107":[],"47610036":6,"47815203":11,"47920156":18,"47942814":5,"479465113":4,"48":4,"48089797":[],"48133064":18,"481979":6,"48212873":16,"48257387":19,"483257001":[],"4837":[],"48420165":[],"48476997":11,"48574149":16,"486873":[],"48687342":[],"489":[],"48994188":5,"49":[4,5,6,11,13],"490":[],"491":[],"49152":3,"492":[],"493":[],"4940954":0,"4959161509356135e":6,"495916150936645e":6,"495916150936654e":[],"495916150938325e":[],"49636583":[],"497":[3,4],"498":[3,4],"499":[3,4],"4990":18,"4992":18,"4997":18,"4c4c7f":[9,10],"4d":3,"4f":6,"4y":8,"4y_i":10,"5":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,18],"50":[1,2,3,4,6,7,8,10,13],"500":[1,3,4,6,9,10,13,18],"500000":[],"50000455":5,"50000553":5,"50000718":5,"50000855":5,"50000969":5,"50001063":5,"50001142":5,"50001207":5,"50001261":5,"50001306":5,"50001343":5,"50001374":5,"500014":5,"50001414":5,"50001422":5,"50001439":5,"50001454":5,"50001466":5,"50001476":5,"50001485":5,"50001492":5,"50001498":5,"50001502":5,"50001506":5,"5000151":5,"50001512":5,"50001515":5,"50001517":5,"50001518":5,"50001519":5,"50001521":5,"50001522":5,"50001523":5,"50001524":5,"50001525":5,"501":[3,4],"5018":18,"50227564e":6,"50321091":5,"504167":[],"50416731":[],"506":0,"50727059":16,"50769230769231":[],"507d50":[9,10],"50846111e":6,"50846112e":6,"50it":6,"50j":13,"50x10":1,"51":[4,10],"510":1,"51004249":16,"512132":[],"51214899":6,"51265232":16,"51289697":11,"512897":11,"5138":[],"51389553":16,"514219":[],"51561271":16,"5177783846":4,"518030":11,"51803019":11,"519842":[],"52":[3,4,13],"5222222222222223":1,"522758":[],"52307692307693":[],"5260627":[],"526744":11,"52723079":18,"52799362":11,"528115":11,"52811546":11,"52874252":5,"529":[],"52911941":[],"52945798e":[],"53":[3,4,9],"5303329":11,"5305555555555556":1,"5312":[],"531280":[],"53189647":[],"53312754":16,"53423784":16,"53611562":18,"53697476":[],"53703498":6,"5378811":11,"53794784":11,"537948":11,"53846153846155":[],"539261":11,"54":[3,4,6,18],"540":[],"54016188":11,"540162":11,"54039921":5,"54041041e":5,"541605":[],"54163136":13,"544439":[],"546166676":[],"54780216":[],"54it":[],"55":[1,3,4],"5501":[3,4,14],"550321":[],"5539":[],"55505907":16,"5555555555555556":1,"55578041":[],"5566":[],"557795":11,"55854694":11,"5594":6,"56":[1,3,4],"56033697":5,"561":[],"5611":[],"5615":[],"56198284":5,"5625":[],"56302854":16,"5639":[],"564":[],"564374":11,"565":[],"56536":0,"566":[],"56636616e":6,"567":[],"568":[],"569":1,"56912044e":6,"56939714":5,"56992937":[],"57":[0,4,8,19],"570":[],"5700":[],"571":5,"571105947979336e":6,"571105947979394e":6,"571105947979395e":[],"571105947979439e":[],"57154252":18,"57174058":18,"57201944e":6,"57266138":[],"574465":11,"57673618":16,"57it":[],"58":[4,10,19],"583595":[],"58428804":5,"584804":[],"5888888888888889":1,"589":[],"58986647":18,"59":[2,4],"590":[],"591":[],"591317992":4,"591594":5,"59159438":5,"592":[],"593":[],"5944444444444444":1,"59591979":[],"595920":[],"59592669":18,"59766":5,"597660":5,"59833875":13,"598392":[],"59839245":[],"59955801":13,"5cm":18,"5dd54edf2138":6,"5f":8,"5x":8,"5y":8,"6":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,16,17,18],"60":[1,2,3,4,6],"60000":4,"6019067271":4,"60236938":16,"60293962":5,"60420593":5,"60538875":16,"60619654":6,"606439":[],"60673226":11,"60685864":6,"60791699":18,"6079169911277265":18,"60943791":16,"61":[2,4,7],"6111111111111112":1,"612939":[],"613579":[],"61399851":11,"613999":11,"61463451":16,"614808":11,"618":[],"61808351":[],"618982":[],"6199381169260247":[],"62":[2,3,4],"62168613":18,"62364974":[],"62373464":11,"625":7,"626635268":[],"62894215":5,"629961":[],"63":[0,1,2,3,4,6,7],"6300745149331701":[],"630224":[],"63025821e":6,"63249532e":6,"63270833":18,"63498144":5,"63675140":[],"6371293350711955":[],"63860687":16,"63901111":18,"63it":[],"64":[1,3,4,7,13,16],"64012627":5,"64425009":18,"645":[],"64615384615385":[],"646283":11,"647":6,"64742912e":6,"647473":11,"649339":5,"64933923":5,"649382":11,"64x50":1,"65":[1,2,3,4,7,8,9],"65136857":[],"6562":[],"65626043":5,"65628853":16,"65885453":5,"659306":[],"66":[2,3,4],"66054752":18,"66087937":18,"66204648":6,"66226149":6,"66247212":[],"6628996975186953":[],"66292841":18,"6638":[],"66410989":[],"66560":[],"66619972":11,"666200":11,"667":[],"668172":[],"668186":[],"66818635":[],"66it":6,"67":[2,4],"67047975e":6,"671089":[],"672721":[],"67314874e":5,"67588315":[],"67697934":11,"67838309":[],"67882608":18,"68":[2,4],"68192193":5,"68246089":16,"68279358":16,"68534263e":6,"68542204":5,"68545647e":[],"6869":[],"6887363571":4,"68929213e":6,"689519":11,"69":[2,4,7,18],"690":[],"690569639355314":[],"69061276":11,"690617":[],"69069n_":18,"692":1,"69295955":[],"69484813e":5,"69493539":16,"69504801":6,"69634577e":6,"69695259":5,"69985355":[],"6999536":11,"69it":[],"6e75736fdab1":6,"6f7a6bd7d79f":6,"6m":[3,4],"6n_":18,"7":[0,1,2,3,4,5,6,7,8,9,11,12,13,16,18,20],"70":[1,2,4,6,7],"70127680":6,"70179437":[],"70205195":16,"703716d317a7":8,"7050":[],"70598996":[],"70653767":4,"70710678":5,"70769586":[],"70832814":5,"70980493":16,"71":[1,2,4],"71131626":16,"7119":[],"712018":11,"7134":[],"7151":[],"718697":5,"71869727":5,"72":[2,4],"72174172":11,"722047011333792":5,"72271878e":6,"7236674":5,"72373129":[],"724":3,"72522848":[],"72859758":5,"7293182":[],"72981762":8,"73":[4,6],"731000":[],"73174557":[],"733096":[],"73484667":18,"735738558766299":18,"73573856":18,"73752910":[],"73it":6,"74":[4,6,13],"740":[],"74081822":8,"74368436":16,"743u":[],"7448615806559786":18,"74840212":5,"749765":[],"75":[4,5,6,8,11],"7501749450963715":[],"750445":[],"750u":[],"75106135":16,"75118364":18,"751699":11,"75170092":5,"753846153846155":[],"756352":[],"75838233":[],"75it":[],"76":[2,4,19],"76060096":[],"76066069":16,"763u":[],"7643536":13,"765":7,"76674796":16,"76923076923077":[],"76936315":5,"7694444444444445":1,"77":[2,4,13,19],"77034458":[],"770345":[],"77152076":5,"7718":9,"774300":[],"775":[],"77627886":13,"77636e":13,"7767978193240488":11,"77714169":8,"7782028952":4,"77865169":[],"78":[2,4],"78011544":[],"78156479e":5,"78184120e":6,"7846153846154":[],"78941903":5,"79":[2,4],"79009329":16,"79111643":5,"793167":[],"79394867":18,"7939646":18,"794282":11,"79602861":18,"79754246":[],"797e":6,"7c394b1e8b71":9,"7d7d58":[9,10],"7f2b3a6174c2":13,"7m":4,"8":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,17,18],"80":[0,1,2,4,5,6,8],"800":[4,7],"80315282":5,"803153":5,"80354994":6,"80469739":5,"8055555555555556":1,"80561191":[],"80609615e":6,"80609616e":6,"8064":[],"80734875":[],"807349":[],"80738685":16,"80747253":16,"80842254":11,"808423":11,"80847477e":6,"81":[1,2,4],"81048318e":6,"8108619":[],"81160425":5,"81187794":16,"81333804":6,"81333805":[],"814":[7,11],"815563":[],"81633628":11,"816454":[],"816847":[],"8175":[],"82":[2,4],"8219235992494145":[],"8219236":[],"82198978":5,"82650876":[],"826509":[],"8265786":5,"82747653":[],"827477":[],"82773778":18,"83":4,"8305555555555556":1,"83512277":5,"836874":5,"83687444":5,"83698677":16,"83999999999999":[],"84":4,"84062065":[],"84228957e":[],"84232163":16,"842436":[],"84292394":[],"842u":[],"84355903e":1,"84443254e":1,"84569271":13,"84780262":6,"84854738":[],"84923989e":6,"84924834":11,"84994524":5,"85":[1,4],"8503720991789538":5,"85091337":18,"85263220":6,"8527777777777777":[],"85278920e":5,"853241":[],"85324115":[],"853u":[],"85601992":5,"85758696":16,"858":[],"8583333333333333":1,"85953586":6,"85959007":18,"86":2,"8608479":[],"860848":[],"861":[],"86117291":5,"86134827":5,"86145244":11,"8638888888888889":1,"86478158":18,"86546962":[],"86623151":13,"86630":[],"8666666666666667":1,"86810":[],"869":0,"86925797":5,"869258":5,"87":2,"870":0,"8702784034":4,"871":0,"872":0,"8722222222222222":1,"873":0,"87381451":5,"875":1,"8759":13,"876":6,"87625493":[],"8768":[],"878297":[],"87940752":16,"87it":6,"88":[2,13],"88017651":11,"880177":11,"88046261":5,"8805555555555555":1,"88118231":18,"88168312e":6,"883":[],"88336879":5,"884":[],"88477619":[],"885":[],"88512489":18,"88529063e":6,"8855417382991412":18,"88554174":18,"886":[],"88679947":[],"887":[],"8879":6,"88871662":16,"8888888888888888":1,"8894":6,"89":2,"8916666666666667":[],"89274639":16,"89288636":11,"893489":5,"89348922":5,"89410423":5,"8944444444444445":1,"89550839":16,"89565264":18,"8962476":[],"896248":[],"89873772":16,"8991514":16,"89944994":5,"899450":5,"8f":6,"8g":6,"8n":16,"8x8":1,"9":[0,1,2,3,4,5,6,7,8,9,11,12,13,14,16,18],"90":[0,1,2,6],"900449":[],"90054363":[],"90075537":5,"9011":6,"90220243":5,"90266948":5,"90268858":[],"9027777777777778":1,"903824":[],"90382431":[],"9040":9,"90475506e":6,"90540529":11,"9055555555555556":1,"9083333333333333":[],"90884627":16,"90895045":16,"90940378":16,"91":[18,19],"910":[],"9111111111111111":1,"91128596":5,"912u":[],"913":[3,4],"91379157":[],"913791573406831":[],"91383439":16,"914":[3,4],"91416375":[],"91479093":0,"91492986e":6,"915":[3,4],"91549644":[],"916":[3,4],"9166666666666666":[],"917":[3,4],"91760278":5,"918":[],"91812702":5,"918992":[],"919":[],"92":[6,19],"922u":[],"92343595":[],"924018":[],"92477093":18,"924e":6,"925":1,"92507116e":1,"9252772":11,"92578916":5,"92603747":16,"92626212":16,"92754397":[],"927544":[],"9277777777777778":1,"92819235":[],"92857143":7,"93":[],"9305555555555556":1,"930583":11,"931":0,"93155188":5,"93158979":5,"93248252":5,"932483":5,"932u":[],"933":5,"93414191":18,"93492130e":6,"93579127":16,"935u":[],"9361111111111111":[],"937":18,"937082":[],"93799826":5,"938":18,"9387":[],"939":[0,18],"94":[7,18],"9400":[],"941866404575299":18,"94226022e":6,"94284104":5,"94320205":5,"9444444444444444":1,"945":[3,4],"94548496":[],"94591015":16,"946":[3,4],"94639099":11,"94659383":[],"947":[3,4],"9472222222222222":1,"94735055":[],"947903":[],"94790323":[],"948":[3,4],"9482527":5,"949":[3,4],"95":[1,7,11],"95008046":6,"95079764":6,"95231424":5,"9527777777777777":1,"95284275":5,"953065564":[],"95351665":5,"954":18,"9549351910143222":[],"954u":[],"9555555555555556":1,"955820c21e8b":4,"956563":11,"95684892":5,"95703":13,"95714723":[],"957147232685324":[],"95it":[],"96":[6,7,11],"960":18,"9601304850018328e":6,"960130485007504e":6,"960130485007934e":[],"9601304850213484e":[],"96024953":5,"96032148":16,"96084663":5,"961":18,"962":18,"9637117593816477":6,"9640435":5,"96489434":[],"9649652536":4,"96527903":16,"965548":[],"96606158":16,"96653373":[],"96688672":5,"9674916":5,"967809":11,"96863851":[],"96987657":[],"96992454":[],"97":7,"97005689":5,"97065296":[],"97108e":13,"9722222222222222":1,"97230501":[],"97243128":5,"97300836":5,"97497404e":6,"975":1,"97507735":5,"975510299261579":9,"9760832":[],"97705827":5,"977418":5,"97758848":5,"9777777777777777":1,"977880":[],"97788031":[],"9780387310732":20,"9780387848570":20,"9781492032632":20,"97898392":6,"97926491":5,"98":[0,1,7],"980":[],"98017611":13,"98036405":[],"9805555555555555":1,"98091621":5,"981321":[],"98139097":5,"98266587":16,"98275501":5,"983310":[],"98346748":[],"98399675":[],"98413059":5,"98454786":5,"985":18,"98566191":5,"986":18,"9861111111111112":1,"98680716":5,"98716878":5,"98756882":13,"98794823":16,"9879924":18,"98808176":5,"98822371":6,"9888888888888889":1,"989":18,"9890348":5,"9893149172528393":[],"9893447":5,"9898ff":[9,10],"99":[6,7,11,13],"990":[],"99009525":5,"9902552771282336":18,"99049330":6,"99088801":5,"991":18,"99115119":5,"99176998":5,"992":18,"99242921":5,"99265097":5,"993":18,"99316252":5,"99353454":[],"993535":[],"99371056":5,"99389612":5,"993972":[],"99397245":[],"9943201":5,"9947756":5,"99492986":5,"99519225":13,"99528218":5,"99539415":5,"9955282554647219":[],"99566069":5,"99578809":5,"996":5,"99608161":5,"9963961":5,"99650061":5,"9967458":5,"99700706":5,"99709215":5,"99724883":[],"99729756":5,"99751458":5,"99758326":5,"99775587":5,"99775949":[],"99793613":5,"99799099":5,"99813653":5,"99828624":5,"99829953":[],"9983295":5,"99845267":5,"998577":5,"99861053":5,"99871521":5,"99881845":5,"99884384":5,"99893323":5,"999":[9,18],"99901896":5,"99903755":5,"99911427":5,"99918546":5,"99919837":5,"99926459":5,"9993237":5,"99933188":5,"99938942":5,"9994385":5,"99944272":5,"99949306":5,"99953381":5,"99953475":5,"99957911":5,"99961294":5,"99965056":5,"99967865":5,"99970988":5,"9997332":5,"99975913":5,"99977416":[],"99977849":5,"99980002":5,"9998161":5,"99984732":5,"99987324":5,"99989476":5,"99991263":5,"99992746":5,"99993978":5,"99995":5,"9999555851685968":6,"999955585168597":6,"9999840939906267":[],"9999858320366368":[],"9x":6,"9y":6,"\u00f8yvind":[6,19],"abstract":1,"boolean":4,"break":[0,4,6,11,14],"byte":16,"case":[0,1,2,3,4,5,6,7,11,12,13,14,15,16,17],"catch":0,"char":18,"class":[0,1,3,4,6,7,8,9,11,12,13,18],"const":18,"default":[0,1,2,4,6,7,13,16],"do":[0,2,3,4,5,6,8,9,10,11,12,13,14,16,18],"ekstr\u00f8m":4,"eng\u00f8i":19,"export":9,"f\u00f8470":19,"final":[0,1,2,3,4,5,6,7,8,9,10,11,13,14,17,18,19],"float":[0,3,4,5,9,11,13,14,16,18],"function":[2,3,4,5,9,14,15,16],"import":[0,1,2,3,4,6,7,8,9,10,11,12,13,14],"int":[0,1,2,3,4,5,6,11,13,14,16,18],"long":[0,1,3,4,12,13,18],"m\u00f8svatn":6,"new":[0,1,2,3,5,6,7,8,9,10,11,13,14,16,18],"null":18,"public":[0,15],"return":[0,1,2,3,4,5,6,7,8,9,11,13,14,16,18],"s\u00f8rli":[],"s\u00f8rlie":19,"sch\u00f8yen":[6,19],"short":[4,5],"steinsv\u00e5g":[],"super":[3,5],"switch":0,"throw":[3,6,18],"true":[0,1,2,3,4,5,6,7,8,9,10,12,13,14,16,18],"try":[0,1,2,3,4,5,6,7,8,9,10,11,13,14,15,16,18],"var":[5,6,10,11,18],"while":[0,1,3,4,5,6,7,8,9,11,12,13,18],A:[2,3,5,6,7,10,11,12,13,15,16,17,18,19,20],AND:2,And:[0,3,4,5,6,9,15,18],As:[0,1,2,3,4,5,6,8,10,12,13,16,18],At:[0,4,6,13],BE:0,Be:[2,15],Being:13,But:[0,1,2,3,5,6,9,10,18],By:[0,3,5,6,12,13,16],For:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,20],IF:6,IN:20,If:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,15,16,18],In:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,18,20],Is:11,Ising:[5,12],It:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,18],Its:[1,2,4,11],NO:[7,11],No:[3,4,5,6,7,8,9],Not:[0,1,5,6,17],OR:18,Of:18,On:[0,3,17,18,20],One:[0,1,3,4,5,6,7,8,11,12,13,18],Or:[0,1,6],Such:[6,12,18],That:[0,5,7,10,11,12,14,18],The:[4,10,13,14,16,17,19,20],Then:[0,1,5,6,8,9,10,11,12,13,14,16,18],There:[0,3,4,5,6,8,9,11,12,14,16,17,18,19],These:[0,3,4,5,8,9,10,11,12,13,14,16,18],To:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,16,18],With:[0,5,6,8,9,10,11,12,14,16,18],_0:[5,8,10,11,13],_1:[2,5,6,8,10,11,12,13,14,16],_2:[2,5,8,11,12,13,16],_3:16,_4:16,_9:13,_:[0,1,2,4,5,6,7,8,9,10,11,12,13,16],_________________________________________________________________:4,__call__:[3,4],__class__:10,__doc__:6,__future__:[8,9],__getitem__:[],__init__:[1,3,4,14],__main__:2,__mosek:5,__name__:[2,10],__traceback__:[3,4],_asarrai:[],_auto10:[6,12],_auto12:6,_auto1:[2,3,4,5,6,7,12,13,16,18],_auto2:[2,3,4,5,6,12,13,16,18],_auto3:[3,4,5,6,12,13,16],_auto4:[4,6,12,13,16],_auto5:[4,6,12,13,16],_auto6:[4,6,12,16],_auto7:[4,6,12,16],_auto8:[6,12],_auto9:[6,12],_ax:[],_base:8,_build:[15,20],_build_call_output:[3,4],_c:1,_call:[3,4],_call_flat:[3,4],_check_1d:[],_check_optimize_result:[7,11],_compon:11,_coordinate_desc:6,_copy_docstring_and_deprec:[],_decor:0,_depth:9,_fraction:9,_get_lin:[],_getitem_multilevel:[],_handl:[3,4],_i:[0,1,2,5,6,8,11,12,13,18],_inference_funct:[3,4],_interpolatefunctionerror:[3,4],_is_primit:[],_j:[0,1,2,3,5,6,8,13],_jit_compil:[3,4],_k:13,_l:12,_lambda:6,_leaf:9,_logist:[7,11],_m:10,_make_vjp:[2,13],_maybe_define_funct:[3,4],_multilayer_perceptron:1,_n:[2,5,8,11,13],_node:[2,9,13],_notokstatusexcept:[3,4],_np:[],_num_output:[3,4],_p:[5,8],_plot_arg:[],_process_traceback_fram:[3,4],_r:[3,4],_ratio:11,_sampl:9,_select_forward_and_backward_funct:[3,4],_split:[6,9],_src:[3,4,14],_stateful_fn:[3,4],_stateless_fn:[3,4],_t:13,_test:6,_trace:[2,13],_valu:[2,13],_varianc:11,_weight:9,a0:3,a0faa0:[9,10],a1:0,a2:0,a3:0,a4:0,a_0:0,a_1a:0,a_2a:0,a_3:0,a_3a:0,a_4:0,a_4a:0,a_:[0,1,16],a_h:1,a_i:[0,1,2,12],a_j:[1,12],a_k:[1,12],a_ndim:2,aaron:20,ab:[0,2,5,13,14],ab_channel:15,abandon:1,abbrevi:17,abid:18,abil:[0,10],abl:[1,4,5,6,7,10,12,13,18],abort:18,about:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,15,16,20],abov:[0,1,3,4,5,6,7,8,9,10,11,12,13,14,16,18],abovement:6,abscissa:13,absolut:[0,2,5,6,13],acceler:13,accept:[0,3,6,9],access:[0,3,4,11,18],accid:[4,6],accmod:5,accompani:0,accomplish:[8,9,13],accord:[0,1,2,5,6,9,12,13,14,18],accordingli:11,account:[0,3,5,13,18],accumul:[12,18],accur:[0,3,4,6,10,13,18],accuraci:[0,1,3,4,5,6,7,9,10,11,12],accuracy_scor:[0,1,10],accuracy_score_numpi:1,achiev:[0,1,5,6,8,12,16],aco:18,acquaint:15,acquir:[1,15],acr:0,across:[1,3,6,9,15],act:[1,3,16],action:18,activ:[0,2,3,4,9,17],actual:[0,1,4,5,6,8,11,16],ad:[0,1,3,4,5,8,13,16],ada_clf:10,adaboostclassifi:10,adadelta:13,adagrad:13,adam:[1,3,4],adapt:[4,6,13,20],add:[0,1,2,3,4,5,6,8,10,11,12,18],add_lin:[],add_outgrad:2,add_subplot:[1,7,12,14],addendum:5,addit:[0,2,3,5,6,7,8,9,10,12,13,15,16,18,19,20],addition:[12,13],address:[1,9,11,13,20],adjac:[3,12],adjoint:5,adjust:[0,5,12,13],admir:0,advanc:[4,6,12,20],advantag:[1,3,5,6,10,13,16],affect:3,affin:[0,3,8,11],afford:3,aforement:14,african:0,after:[0,1,2,4,5,6,9,11,12,13,15,16,18],afterward:0,ag:[0,7,17],ag_0:2,again:[0,1,4,5,6,7,8,10,11,12,13,18],against:[1,4,7,10],agegroup:7,agegroupmean:7,aggreg:[9,10],agorithm:10,agre:[5,6,18],ahead:9,ai:[0,20],aid:11,aim:[0,1,4,6,7,11,14,15,16],ainv:5,airplan:3,aka:5,al:[0,2,4,17,20],alarm:5,algebra:[0,3,5,13,15,17],algo:18,algorithm:[0,1,2,4,5,6,7,8,13,14,15,16,17,18,20],align:[0,2,5,6,7,8,13,18],all:[0,1,2,3,4,5,6,7,9,10,11,12,13,14,15,16,17,18,19,20],allevi:[1,13],alloc:[3,16],allow:[0,1,2,3,5,6,8,10,13,15,16],almost:[0,1,6,8,11,13,18],alon:[2,9],along:[2,3,4,5,6,9,10,11,15,16],alpha:[0,1,2,3,4,5,6,7,8,9,10,13,14,18],alpha_0:3,alpha_1:3,alpha_2:3,alpha_:10,alpha_i:[3,13],alpha_k:13,alpha_m:10,alpha_n:3,alpha_opt:13,alreadi:[2,3,4,5,6,10,12,15,16,18],also:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,20],alter:[1,18],altern:[0,1,4,5,6,7,8,9,11,13,16],although:[1,5,6,8,10,13],alwai:[0,3,5,6,12,13,18],am:[4,18],ame2016:0,american:0,among:[0,3,5,9,10,12,16],amongst:5,amount:[0,1,3,4,6,8,10,14,15],an:[1,2,3,5,6,7,8,9,11,12,13,14,15,16,18,19,20],an_:18,anaconda3:[],anaconda:[0,1,15],analog:13,analys:[6,18],analysi:[1,3,4,7,14,16,17,20],analyt:[0,2,3,5,6,7,12,13,15],analyz:[0,1,3,4,5,6,18],andrew:1,angl:[0,3,9,18],anharmon:3,ani:[0,1,2,3,4,5,6,7,8,9,10,12,14,18],anim:[4,12],ann:12,annot:[0,1,3,7,8],anoth:[0,1,3,4,5,6,7,8,10,11,12,13,16,18],anp:2,ans_vspac:2,ansatz:0,answer:[0,1,3,5,6,16],antialias:[2,6],anticip:4,anymor:[1,8],anyon:[4,8],anyth:[1,18],anytim:19,apach:1,apart:[11,13],api:[1,15],appar:2,appear:[0,1,3,13,16,18],append:[1,3,4,8,9,13,18],appendcon:5,appendvar:5,appli:[0,1,2,3,4,6,7,8,9,10,11,12,13,18,20],applic:[0,1,3,4,5,6,7,9,12,13,16,17,18,20],apply_gradi:4,approach:[1,2,4,5,6,9,10,11,12,13,15,18,20],appropri:[2,6,9,12,13,15,18],approx:[0,2,3,6,10,11,13,18],approxim:[0,1,2,3,4,5,6,7,10,11,13,18],apt:[0,15],aq:18,ar:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20],arang:[1,3,4,6,7,9,10,12],arbitrari:[1,4,6,8,12,13,18],arbitrarili:[0,1,11],arc:6,architectur:[3,4,12,20],area:[0,3,6,20],arg:[0,2,3,4,13],argc:18,argmax:[1,11],argmin:[4,10,14],argnum:[2,13],argnum_0:2,argnum_1:2,argsort:11,argu:[1,13],argument:[0,2,3,5,6,11,12,13,18],argv:18,argval:2,aris:[0,6,12,13,18],arithmet:[0,13,16],arm:[3,4,6,14],arma:18,armadillo:[16,18],around:[0,1,4,5,6,11,18],arr:2,arrai:[0,1,2,3,4,5,6,7,8,9,11,12,13,14,15,18],arrang:3,arraybox:13,arriv:[0,6,9,11,16,18],arrow:12,arrowprop:8,art3d:13,art:[0,1,15],articl:[0,3,4,6,10,18],artifici:[0,2,7,12,20],artificialneuron:12,arug:13,arxiv:[3,4],as_fram:[],asarrai:[0,2,6,9],asc:5,ascii:18,ashrafi:19,ask:[5,6,11,12],aspect:[0,6,15],assembl:[0,3],assert:4,assess:[0,6],assici:4,assign:[0,7,8,9,12,13,14,17,20],associ:[0,6,9,12,14,18],assum:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,18],assumpt:[0,3,5,6,9,11,18],ast:[0,5,6],astyp:[4,9,10],asymmetri:0,asymptot:[4,6],async_wait:[3,4],atoi:18,atom:0,attempt:[0,4,6,7,8,10],attend:17,attent:[0,16],attr:[3,4],attract:[0,10],attribut:[0,9],attributeerror:[],audi:0,audio:[3,4],aurelien:[0,17,20],austfjel:6,author:[0,1,10,18],authour:0,auto:[6,9,10,18],autocor:18,autocorrelation_tim:18,autocorrelform:18,autocovari:18,autoencod:[4,15],autoencond:15,autograd:15,autom:[0,15],automac:16,automat:[0,1,2,3,4,11,15,16,17],automobil:3,autonom:[4,20],avail:[0,1,4,6,10,11,15,16,17,20],averag:[0,1,3,6,9,10,13,14,18,19],avg:18,avoid:[0,4,5,6,9,11,13,16],avx2:[],avx:[],awai:[2,3,6,18],awar:[2,10],award:19,ax:[0,1,2,3,4,6,7,8,9,10,11,12,13,14,16],axes3d:[2,6,13],axes_grid1:6,axessubplot:[],axhlin:8,axi:[0,1,2,3,4,6,7,8,9,10,11,12,13,14,16,18],axiom:5,axlabel:0,axvlin:[4,8],axvspan:4,b1:8,b2:8,b3:8,b:[0,1,2,3,4,5,6,8,9,10,12,13,14,18,19],b_1:[2,12,13],b_2:13,b_5:13,b_:[1,16],b_group:9,b_i:[0,1,2,12],b_ia_:0,b_index:9,b_is_vec:2,b_j:[1,12],b_k:[1,12,13],b_m:12,b_meta:2,b_score:9,b_valu:9,bachelor:17,back:[0,3,4,5,6,8,9,10,16,17,18],backbon:16,backend:[1,4],background:[17,20],backpropag:1,backtrack:9,backup:16,backward:[1,2,4,12,16],backward_pass:2,bad:[6,18],badli:18,bag:[9,15,17],bag_clf:10,baggingboot:10,baggingclassifi:10,baggingtre:10,balanc:6,band:16,bandwidth:16,bar:[0,6,11],barber:20,bare:[4,10],base:[0,1,3,4,5,7,8,9,10,14,15,18,19,20],basi:[5,7,8,10,11,12,13,16],basic:[6,8,12,13,14,15,17,18],batch:[3,4,11,12,13],batch_shap:4,batch_siz:[1,3,4],batchnorm:4,bay:7,bayesian:[5,15,20],bc298b802fe2:6,becaus:[0,1,2,3,4,5,6,8,9,12,13,14],becom:[0,1,2,5,6,7,9,12,13,18],been:[0,1,2,3,4,5,6,11,12,13,15,16],befor:[0,1,2,3,4,5,6,7,8,12,13,14,16,18],beforehand:[0,18],begin:[0,1,2,3,4,5,6,7,8,9,11,12,13,14,16,18],behav:[1,6,13],behavior:[0,1,13],behaviour:12,behind:[0,1,6,8,13],behnoosh:19,being:[0,1,2,3,4,5,7,8,10,11,12,13,18],believ:[9,16],belong:[5,7,8,9,13,14],below:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,16,18],benchmark:10,bendik:[],benefici:[1,13],benefit:[0,1,4,11,13,15],bengio:[1,17,20],benign:[1,7],bennosh:19,besid:[4,5],bessel:5,best:[0,1,2,3,4,5,6,7,8,9,10,12,13,18,19],beta:[0,1,3,5,6,7,10,11,13],beta_0:[0,1,3,5,6,7,13],beta_0x_:0,beta_1:[0,1,3,5,6,7,10,13],beta_1x_0:0,beta_1x_1:[0,7],beta_1x_2:0,beta_1x_:0,beta_1x_i:[7,13],beta_2:[0,3,13],beta_2x_0:0,beta_2x_1:0,beta_2x_2:[0,7],beta_2x_:0,beta_3:3,beta_:[0,3,6,7,13],beta_i:[0,3,5],beta_j:[0,5,6,13],beta_k:13,beta_linreg:13,beta_m:10,beta_mg_m:10,beta_n:3,beta_p:7,beta_px_p:7,betavalu:5,better:[0,1,2,3,4,6,9,10,11,12,13],between:[0,1,2,3,4,5,6,7,8,9,11,12,13,14,18],beyond:[0,1,5,6,8,13],bf:[13,14,16,18],bgd:13,bia:[0,1,2,3,5,8,9,10,12,13,17,18],bias:[1,2,3,5,6,9,12],big:[0,1,2,5,6,14],bigger:[1,6],bigr:12,bike:9,bilek:19,billion:[3,12,15],bin:[0,7,18],binari:[0,3,5,7,9,10,12,17,18],binarycrossentropi:4,bind:0,binomi:15,binsboot:6,bioinformat:0,biolog:[1,12,20],bios1100:15,bird:[0,3],bishop:[17,20],bit:[1,4,16,18],bitwis:18,bivari:2,bk:[0,13],bla:[5,16],black:[8,9,14],block:[6,10,15,16],blockingavg:18,blockingstd:18,blockingvar:18,blocksiz:18,blocksizemax:18,blocksizemin:18,blogpost:4,blue:[0,3],bmatrix:[0,1,3,5,7,8,11,13,16],bmi:1,bodi:[0,1,4,12],bold:1,boldfac:[0,5],boldsymbol:[0,1,2,3,5,6,7,8,10,11,13,14],boltzmann:[12,15],book:20,boost:[1,9,15,17],boostrap:10,bootavg:18,bootstd:18,bootstrap:[1,13,15,17],bootvar:18,bootvec:18,boston_dataset:0,bot:8,both:[0,1,4,5,6,8,9,10,13,14,15,16,18,19],bottl:7,bound:[0,5,8,12],boundari:[2,4,8,11,12],boundkei:5,box:[2,4,9],boxed_arg:2,boyd:[8,13],bracket:[4,18],brain:[1,7,12],branch:9,breast:[5,7,11],breviti:13,brew:[0,15],brg:8,briefli:0,bring:[0,5,6,10],broad:[0,3,4],brownle:4,brute:[3,5,11],bs:[8,9,10],buffer_s:4,bui:4,build:[0,4,5,6,10,13,16,18],built:[0,1,3,4,6],bunch:11,busi:0,bx:5,bzl:5,c1:[8,11],c2:[8,11],c95af3df0cdd:3,c:[0,1,2,4,5,6,7,8,9,10,11,12,13,14,15,16,18,19,20],c_0:18,c_1:12,c_2:12,c_3:12,c_4:12,c_:[0,8,9,10,13,18],c_i:[12,13],c_k:18,ca:1,cabc613b8702:13,cach:10,cal:[0,8,10,12,13],calcul:[0,1,2,4,5,6,8,9,10,11,12,13,14,16],california:[],call:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,18,20],callabl:[3,4],callback:[3,4],calor:0,cambridg:[13,20],can:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,18,20],cancel:[0,13],cancellation_manag:[3,4],cancer:[5,10],cancerpd:7,candid:[8,9,10],cannot:[0,1,4,5,6,7,8,9,17,18],canopi:[0,15],cap:5,capabl:[0,1,8,13,15],capac:2,capita:0,captur:[4,11,12],captured_input:[3,4],car:[3,4],card:[0,7],cardin:1,care:11,carefulli:13,carlo:[0,6,15,18,20],carri:[2,6,7],cart:10,carvalho:19,casella:20,cast:1,cat:[3,4],categor:[0,1,3,9,11],categori:[0,1,3,7,10,12,14,17],categorical_crossentropi:[1,3],caus:[0,5,6,18],causal:0,causat:0,cax:1,cb:6,cbar:1,cbook:[],cc:[0,1,5,13],ccc:[5,12],cd_fast:6,cdf:18,cdot:[0,2,6,12,13,14,16,18],celebr:13,cell:4,center:[0,1,6,7,8,9,11,14,18],centr:20,central:[0,3,5,6,8,16],centroid:[14,18],centroid_differ:14,centuri:3,certain:[0,3,6,7,9,18],cg:13,cha:0,chain:[1,13,15,18],chanc:[1,5,13,18],chang:[0,1,2,3,4,5,6,8,9,11,12,13,14,16,18],channel:3,chapter:[0,6,10,11,16,17,20],charact:[0,3,5,18],character:[8,9,10,12,18],characterist:[0,1,3,10,13],charg:0,charl:0,chase:4,chd:7,chddata:7,cheap:5,cheaper:[1,13],check:[0,1,3,4,5,6,11,13,16],checkmark:3,checkpoint:4,checkpoint_dir:4,checkpoint_prefix:4,chemic:18,chen:10,chiaramont:2,choic:[0,1,2,3,4,6,9,12,13,14,16,18],choleski:[5,16],choos:[2,3,6,9,10,11,13,14,18],chosen:[0,1,2,6,8,9,10,13,18],chosen_datapoint:1,christian:20,christoph:[17,20],cifar10:3,cifar:3,cin:18,circ:[1,12],circl:[0,8,12],circuit:3,circumfer:9,circumv:[1,5,13],ckpt:4,clariti:18,class_nam:[3,9],class_val:9,class_valu:9,class_weight:[3,4],classic:[7,9,13],classif:[0,3,5,6,7,8,11,12,15,17,20],classifi:[0,1,4,7,9,10,11],classificaton:1,classifii:10,clean:1,clear:[1,5,10,12,13],clearli:[0,3,5,6,7,8,18],clever:[1,10],clf3:0,clf:[0,6,8,9,10],clf_lasso:6,clf_ridg:6,clip:[3,18],close:[0,1,2,4,6,8,9,11,12,13,14,18,20],closer:[3,5,13],closest:[8,11,13,14],closur:15,cloud:15,cluster:[0,1,4,6,11,15,17],cluster_label:14,cm:[1,2,3,6,8,13],cmap:[0,1,2,3,4,6,8,9,10],cmap_arg:6,cmath:18,cmb:17,cmd:9,cmu:[],cn_:18,cnn:[12,17],cnn_kera:3,cntk:15,co:[0,2,3,6,9,13],code:[3,4,6,7,8,13,15,16,17,20],coef0:8,coef:0,coef_:[0,5,6,8,9,13],coeff:5,coeffici:[0,3,5,6,7,8,9,13,16],coerc:[0,6],coin:[10,18],coin_toss:10,col:[0,11],colab:15,cold:9,colinear:0,collaps:8,collect:[0,2,6,10,11,15,18,20],collinear:5,color:[0,3,4,6,8,9,10,18],color_channel:3,color_cod:6,colorbar:[1,6],colsample_bytre:10,colsaobject:10,colspec:0,column:[0,1,2,5,6,7,8,9,11,12,16],columntransform:9,com:[3,4,6,14,15,20],combin:[1,2,5,6,7,10,18],come:[0,1,3,4,5,12,13,14,17],command:[0,1,18],comment:[0,4,5,6],commerci:[0,15],commod:0,common:[0,1,3,5,6,7,9,11,13,14,18],commonli:[0,1,4,6,7,9,13,14],commun:[0,12],commut:3,commutatitav:3,compact:[0,1,3,5,6,7,9,11,12,13,14],compar:[0,3,4,5,6,11,13,16],comparison:[2,4,13],compat:7,compet:0,competit:10,compil:[0,1,3,4,15,16,18],complet:[0,2,3,4,9,12],completenn:12,complex:[1,5,8,9,11,12,13],complic:[0,1,9,13],compon:[0,1,3,4,5,6,7,9,14,15,17],components_:11,compos:[9,12,14],compphys:[6,15,17,20],compress:0,compris:6,compromis:5,compulsori:15,comput:[0,1,2,3,4,5,6,7,8,10,11,12,13,15,16,17,20],computation:[0,3,6,9,13,18],con:5,concaten:[2,4,6,14],concav:[1,13],concentr:[0,10],concept:[0,2,15],conceptu:[12,13],concern:[0,1,4,7],conclud:[0,5,13],conclus:1,cond:2,conda:[0,1,15],condit:[0,2,4,5,6,8,9,11,13,18],conduct:15,condwav:2,coneqp:5,confid:[0,5,6,7,8],configur:3,confirm:[5,12],confus:[5,6,10,16],confusion_matrix:9,congruenti:18,conjug:[4,8],conjugaci:13,conjunct:3,connect:[0,1,3,4,9,11,12,13,16],consequ:[5,6,8,10,12,13],conserv:[5,14],consid:[0,1,2,3,5,6,7,8,9,10,12,13,16,18],consider:[0,1,5,13],consist:[0,1,2,3,4,6,12,13,18],constant:[0,2,4,5,6,8,12,13,18],constitu:0,constitut:[2,6],constrain:[1,3,5,7,11],constraint:[5,6,8,13],construct:[0,1,2,3,5,6,7,8,9,10,11,16,18,20],contact:0,contain:[0,2,3,4,5,6,7,8,9,11,12,13,16,18,20],contemporari:20,content:[1,15,16],context:[3,4,6,10,13],contigu:16,continu:[0,1,2,3,4,5,6,7,8,9,10,12,13,16],contour:[9,10,13],contourf:[8,9,10],contrast:[1,4,9,10,12],contribut:[0,3,5,13,18],contributor:0,control:[0,1,3,9,13,15],conv2d:[3,4],conv2dtranspos:4,conv:[3,4],conveni:[0,5,6,12,13,16],convent:12,converg:[1,2,4,5,6,7,8,11,13,14],convergencewarn:[1,6,7,8,11],convert:[0,1,4,5,9,11,13,16],converttomatrix:4,convex:[4,5,7],convinc:13,convolut:[1,4,15,17],cool:[4,9],coolwarm:6,coordin:[5,12,14],coorel:0,copi:[0,1,14],core:[2,3,4,10,13],corel:0,coronari:7,corr:[0,5,7,11],correalt:[11,15],correct:[0,1,2,3,4,5,13,16,18],correctli:[1,2,6,10],correl:[0,1,3,5,6,7,10,12,13,15],correlation_matrix:[0,5,7,11],correspond:[0,3,5,6,8,9,11,12,15,16,18],cortex:12,cosin:[3,6],cost:[0,2,3,5,6,7,8,9,12,13,18],cost_deep_grad:2,cost_funct:2,cost_function_deep:2,cost_function_deep_grad:2,cost_function_grad:2,cost_grad:2,cost_sum:2,costol:13,could:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,16,18],coulomb:0,count:[0,9,17,18,19],countor:13,coupl:[4,5,6],cours:[0,1,3,5,11,17,18],courvil:[17,20],cout:18,cov:[5,6,11,16,18],cov_xi:[5,11],cov_xx:[5,11],cov_yi:[5,11],covari:[0,7,15,16],covariance_matrix:[5,11,14],cover:[0,5,15,17,20],covert:0,covxi:18,covxx:18,covxz:18,covyi:18,covyz:18,covzz:18,cpu:1,cpu_feature_guard:[],craft:3,creat:[1,2,3,4,5,6,9,10,11,12,13,15],create_biases_and_weight:1,create_convolutional_neural_network_kera:3,create_neural_network_kera:1,create_x:[5,11],credit:[0,7],crim:0,crime:0,criteria:[0,4,9,10,14,18],criterion:[9,10,13],critic:6,cross:[0,1,3,7,9,10,13,15,17,18],cross_entropi:4,cross_val_scor:6,cross_valid:[7,10],crossvalid:6,crucial:[1,18],cs231:3,cs:17,csr_matrix:16,cstdlib:18,csv:[0,4,6,7,9],ctnk:1,ctx:[3,4],cubic:0,cumbersom:5,cumsum:[10,11],cumul:10,cumulative_heads_ratio:10,cup:5,current:[1,2,3,4,6,13,14],curs:0,curv:[6,7,10,12],curvatur:13,custom:[5,6,14],custom_cmap2:[9,10],custom_cmap:[9,10],cutpoint:9,cv:[6,7,10],cvxbook:13,cvxopt:[5,8],cyber:20,cycl:[1,12,18],d1:5,d2:5,d2_g_t:2,d3:5,d670a873ab0c:5,d985fb40c43d:6,d:[1,2,3,4,5,6,7,8,9,10,11,13,14,16,18,19],d_f:13,d_g_t:2,d_net_out:2,da:3,dagger:[5,16],dai:[1,9,15],dalen:[],damp:3,darget:9,darkr:18,dat:0,dat_id:[0,6,7,9],data1:14,data2:14,data3:14,data4:14,data:[2,4,5,8,10,12,13,14,16,20],data_handl:[3,4],data_id:[0,6,7,9],data_indic:1,data_modul:[],data_path:[0,6,7,9],data_url:[],databas:1,datafil:[0,6,7,9],datafram:[0,4,5,7,9,11],datapoint:[1,5,6,7,11,13],dataset:[0,4,6,7,8,9,10,11,13,14],datatyp:4,date:0,daughter:10,david:20,dbh:1,dbo:1,dcomposit:16,ddot:2,dead:1,deadlin:17,deal:[0,1,3,5,6,8,11,13,14,16,18],debt:7,debug:[5,6],decad:[0,3],decai:[0,13,18],decemb:17,decent:10,decid:[0,2,3,5,6,9],decim:0,decis:[0,1,8,11,15,17,20],decision_funct:8,decision_tre:9,decisiontreeclassifi:[9,10],decisiontreeregressor:[0,9,10],declar:[0,4,16],decompos:[5,6,16],decomposit:[0,6,12,17],decompost:5,deconvolut:3,decorrel:[10,13],decreas:[1,2,4,5,6,10,11,13],deduc:0,deep:[3,7,12,13,15,17,20],deep_neural_network:2,deep_param:2,deep_tree_clf1:9,deep_tree_clf2:9,deep_tree_clf:[9,10],deepen:[5,15],deeper:[0,3,4],deeplearningbook:20,deer:3,def:[0,1,2,3,4,5,6,7,8,9,10,11,13,14,18],def_covari:18,def_funct:[3,4],default_tim:4,defect:5,defici:5,defin:[0,2,3,4,5,6,7,8,9,10,11,12,13,14,16,18],definit:[1,2,5,6,8,10,11,12,13,16],defint:18,defun:[3,4],defvjp:2,degre:[3,5,6,8,9,10,11,18],del:1,delet:[6,18],delimit:4,deliv:17,delta:[0,2,3,6,8,12,13,14,18],delta_0:3,delta_1:3,delta_2:3,delta_3:3,delta_4:3,delta_5:3,delta_:[1,16],delta_h:[0,1],delta_j:[3,12],delta_k:12,delta_l:[1,3],delta_n:[0,3],delug:15,delv:0,demand:13,demonstr:[0,3,5,6,7,11,12,15],den:4,denomin:[1,5],denot:[1,2,6,7,13,18],dens:[1,3,4],dense_1:4,densiti:[0,2,6,18],depart:19,depend:[0,1,2,4,5,6,7,8,11,12,13,15,16,18],depict:18,deploy:[0,15],deprec:[2,3,6,13],depth:[0,3,9,10,16],deriv:[0,1,2,6,7,8,10,11,13,15],derivati:13,descend:[5,9,11],descent:[0,1,3,7,8,12,17],descr:[],describ:[0,2,4,5,6,8,10,11,12,13,16],descript:[0,8,9],design:[0,1,3,4,5,6,7,10,11,12,13],designmatrix:0,desir:[0,2,4,5,13,14],despit:[1,12],destroi:16,det:[5,16],detail:[0,6,11,13,14,16],detect:[3,8,12],determin:[0,2,3,4,5,6,8,9,10,11,12,13,16,18],determinist:[7,13,18],dev:[1,18],develop:[0,3,5,8,10,11,12,15,16,17],deviat:[0,1,2,4,5,6],device_nam:[3,4],devis:12,df:[4,8,11,13],di:[0,5],diag:[5,8],diagnost:[1,10],diagon:[0,5,7,13,16,18],diagonaliz:5,diagram:10,diagsvd:6,dice:6,dict:[6,8],dict_kei:[],dictionari:0,did:[0,1,5,6,7,10,11,14],die:1,diff1:2,diff2:2,diff:2,diff_ag:2,diffeent:8,differ:[0,1,2,3,4,5,6,9,10,11,12,13,14,15,16,18,20],differenti:[3,15,16,17],differential_oper:[2,13],difficult:[0,1,6,10,13,18],difficulti:[0,1,13],diffonedim:2,digit:[0,1,3,4,6,17,19],dilemma:13,dilut:1,dim:[4,11,14,16],dimens:[0,1,2,3,4,5,8,11,14,16],dimension:[0,4,5,6,9,11,13,14,15,16],dimensionless:[0,3],diment:16,dimnsion:4,diod:3,direct:[0,1,2,4,11,12,13,14],directli:[1,4,5,6,18],directori:[],disabl:[3,4],disadvantag:0,disappear:[3,6],disc_loss:4,disc_tap:4,discard:[6,11],disciplin:[0,3,12],disclaim:18,discourag:13,discov:0,discover:5,discret:[1,3,5,7,13],discrimin:[4,7,10,11],discriminator_loss:4,discriminator_loss_list:4,discriminator_model:4,discriminator_optim:4,discuss:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,20],diseas:7,disguis:6,disord:[1,7],displai:[0,1,3,4,5,6,7,8,9,10,11,12,14,18],displaystyl:[0,5],displot:[],disregard:0,dissimilar:[11,14],dist:14,distanc:[0,8,9,11,14,18],distance_list:9,distinct:[3,7,8,9,10,14],distinctli:8,distinguish:[0,4,7,8,18],distplot:0,distribut:[0,1,4,6,7,10,11,13,14,15,16],distrubut:[0,15],div:5,dive:[0,8,16],diverg:[1,13],divid:[0,1,3,5,6,8,9,11,12,18],divis:[6,8,9,13,16,18],dna:7,dnn1:4,dnn2_gru2:4,dnn:[0,1,2,4,12],dnn_kera:1,dnn_model:1,dnn_numpi:1,dnn_scikit:[0,1],doc:[6,15,17,20],document:[4,7,11,13],doe:[0,1,2,3,4,5,6,8,10,11,12,13,16,18],doesn:[3,9,12],dog:[1,3,4],domain:[5,8,13],domin:0,don:[0,1,3,5,6,8,11,13,15,18],done:[0,2,3,4,5,6,9,10,11,13,16,18],dot:[0,2,3,5,6,7,8,9,10,11,12,13,16,18],doubl:[3,4,16,18],doubli:1,down:[0,3,6,9,11,12,13],download:[0,1,3,5,6,16,20],downsampl:3,dozen:1,dq:6,drag:13,dramat:11,drastic:4,draw:[4,6,10,13],drawback:[0,1,3,13],drawn:[1,4,6,7,11,18],drive:[3,4],driven:3,drop:[0,1,5,6,11,13,18],dropna:[0,6],dropout:4,ds:5,dt:[2,3,13,18],dtype:[0,1,2,3,4,14,16],dualiti:6,dub:0,due:[1,2,5,6,8,10,12,13],dummi:0,dure:[0,1,3,4,8,9,11,15],dwell:0,dwh:1,dwo:1,dx:[2,3,8,18],dx_1:18,dx_1p:6,dx_2p:6,dx_mp:6,dx_n:18,dxp:6,dy:[1,8,18],dynam:4,dysth:19,dz:8,e:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,18,19],e_:[0,2],each:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19],eager:[3,4],eapprox:0,earli:[1,3,4,13],earlier:[0,5,7,8,9,11,12,13],earthexplor:6,eas:[6,9,14],easi:[0,5,6,7,8,9,10,11,12,13,15,16],easier:[5,6,8,9,13,18],easiest:13,easili:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,16],eastern:19,ebind:0,eblock:9,economi:5,ecosystem:15,ect:17,edg:3,edgecolor:6,edu:13,educ:0,eface79dac2c:10,eff:18,effect:[1,4,10,13,18],effic:1,effici:[0,3,10,13,15,16,18],efron:[6,18],egrad:13,eig:[5,11,13,16,18],eigen:18,eigenpair:[5,11],eigenvalu:[0,5,8,11,13,16],eigenvector:[5,11,13],eight:16,eigval:[16,18],eigvalu:[11,13],eigvec:[16,18],eigvector:[11,13],eispack:16,either:[1,5,6,7,8,9,10,11,13,18],ekstrom:[],elabor:18,elarn:3,electr:[0,3,12],electur:20,eleg:11,element:[1,2,3,4,5,6,7,8,11,12,13,15,16,17,20],elementari:[10,13,16],elementwis:[3,13],elementwise_grad:[2,13],elif:[2,3,4,14],elim:16,elimin:[3,5,8],els:[1,2,3,4,7,9,12,13,16,18],elu:1,elus:0,email:[17,19],embed:[0,11],embodi:6,emit:18,emner:[17,20],emphas:[0,10,15],emphasi:[0,15,20],empir:[1,11,18],emploi:[0,1,5,6,11,13,18],employ:0,empti:[6,10],emul:12,en:[15,20],enabl:11,enbodi:6,encod:[0,3,5,9,11,14],encompass:[0,18],encount:[0,1,5,6,7,13,18],end:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,18],end_box:[2,13],end_nod:[2,13],end_valu:[2,13],endl:18,endpoint:[3,6,18],energi:[0,4,6],enet_coordinate_desc:6,enforc:12,eng:20,engin:[0,1,3,4,15,18],english:20,enorm:3,enough:[0,6,13],ensembl:[1,9,17,18],ensur:[0,1,2,3,5,6,11,13,18],ensure_initi:[3,4],enter:[5,6,18],enthought:[0,15],entir:[1,3,7,9,15,18],entiti:[9,12,16],entri:[0,5,8,11,12,16],entropi:[1,3,7,10,13],enumer:[0,1,2,3,4,6,8],env:[0,1,2,3,4,5,6,7,8,11,13,14,18],environ:[2,15,20],eo:[0,6],eol:0,eosfit:0,epoch:[0,1,3,4,12,13],epsilon:[0,5,6,7,13],epsilon_0:0,epsilon_1:0,epsilon_2:0,epsilon_:0,epsilon_i:0,eq:[3,13,14,16,18],eqnarrai:[3,5,6],equal:[0,1,2,3,4,5,6,8,9,11,12,13,14,16,18],equat:[1,3,4,5,6,7,8,9,10,11,13,14,16,17,18],equilibrium:[2,12],equiv:[3,13,16,18],equival:[0,1,5,8,11,13,15,16],erf:18,err:[0,10],err_:6,err_sqr:2,errat:13,errno:[],erron:2,error:[1,2,4,5,6,7,9,11,12,13,15,16,18],error_estimate_corr_tim:18,error_handl:[3,4],error_hidden:1,error_output:1,escap:13,especi:[1,3,9,12,13],essenti:[0,5,6,9,10,12,14,17,18],establish:[0,6,10,11],estim:[0,1,5,6,7,10,11,13,15,18],estimated_mse_fold:6,estimated_mse_kfold:6,estimated_mse_sklearn:6,et:[0,2,4,17,20],eta0:[8,13],eta:[0,1,3,8,12,13],eta_:13,eta_t:13,eta_v:[0,1,3],etc:[0,1,3,5,7,8,9,11,12,13,14,15,16,18],ethic:15,etsim:6,euclidean:[0,14],evalu:[0,2,3,4,5,6,9,13,18],evalut:13,even:[0,1,3,4,5,6,8,9,10,11,12,13,14,15,16,18],evenli:4,event:[3,4,5,7,10,14],eventu:[5,6,11,12,13,19],everi:[0,1,2,3,4,5,6,9,10,11,12,13,14,15,18],everyth:[4,12],everywher:[4,13],evolv:0,exact:[0,2,5,11,12,13,16,18],exactli:[0,3,4,6,12,15],examin:6,exampl:[5,11,12,13,15,16,17,20],exce:[1,12,13],excel:[0,1,4,5,10,20],except:[3,4,5,6,8,9,16,18],excess:0,excit:0,exclud:[1,6,12],exclus:[0,1,3,6,18],execut:[2,3,4,5,13],executing_eagerli:[3,4],exemplifi:13,exercis:[5,15,17],exhaust:6,exhibit:[0,5,6,8],exist:[0,1,2,3,5,6,7,8,9,13,16,20],exit:[5,16,18],exp:[0,1,2,5,6,7,8,10,11,12,13,18],exp_term:1,expand:[5,7,11,13],expans:[0,3,5,8,10,12,13],expect:[0,1,5,6,7,11,12,13,15],expectation_value_of_h_wrt_p:18,expens:[6,10,13,18],experi:[0,1,6,8,13,15],experiment:[0,3,4,6,9,14,18],experimental_get_tracing_count:[3,4],expert:[1,9],explain:[0,6,9,10,11,13,18],explained_variance_ratio_:11,explanatori:0,explicit:[0,3,6,13,16],explicitli:[0,4],explod:1,exploit:[0,3,12,13],explor:[1,4,6,8,13,15],expon:1,exponenti:[0,1,5,6,10,13],export_graphviz:9,export_text:9,exporttext:9,expos:15,express:[0,2,3,5,6,7,10,12,13,16,18],exptmean:18,exptvari:18,extend:[2,5,7,11,13,15],extens:[0,12,15],extent:[0,1,6,20],extern:[3,6,9],extra:[1,3,5],extract:[0,3,5,6,7,8,11,13,16],extrapol:0,extrem:[0,1,4,5,6,7,8,9,13,16],extremum:13,extrins:11,ey:[0,5,6,13,14,16],f11:0,f12:0,f13:0,f1:13,f1_grad:13,f1d:13,f2:13,f2_grad_x1:13,f2_grad_x1_analyt:13,f2_grad_x2:13,f2_grad_x2_analyt:13,f3:13,f3_grad:13,f3_grad_analyt:13,f4:13,f4_grad:13,f4_grad_analyt:13,f5:13,f5_grad:13,f6:13,f6_for:13,f6_for_grad:13,f6_grad_analyt:13,f6_while:13,f6_while_grad:13,f6d7a289d493:[],f7:13,f7_grad:13,f7_grad_analyt:13,f8:13,f8_grad:13,f9:[0,13],f9_altern:13,f9_alternative_grad:13,f9_grad:13,f:[0,1,2,3,4,5,6,7,8,10,12,13,14,16,18,19],f_0:[3,10],f_1:[10,13],f_2:[12,13],f_3:12,f_:10,f_d:18,f_grad:13,f_grad_analyt:13,f_i:[6,12],f_m:[3,10],f_n:3,f_raw:2,f_vec:2,f_wrap:2,face:13,facecolor:[6,8,18],facil:[0,15],facilit:12,fact:[0,1,3,5,9,11,12,13],factor:[0,1,3,5,6,9,10,11,13,16,18],factori:13,fade:6,fafab0:[9,10],fail:[0,6,7,8,11,13,19],failur:7,fairli:[1,2,18],fake:4,fake_loss:4,fake_output:4,fall:[8,9,17],fals:[0,1,2,3,4,5,6,7,9,10,13,14,16],famili:[0,7,8,18],familiar:[0,3,5,6,8,15,16,18],famou:[6,12],far:[0,3,4,5,6,8,11,12,13,14,18],fashion:[0,9,10],fast:[1,3,6,10,12,13,15,18],faster:[1,11,13],fastest:[13,16],favor:7,favorit:18,fc:3,fdf8a5d7c717:2,fdfcc778e1f8:4,fe5b9d300cc0:6,featur:[0,1,3,5,6,7,8,10,11,12,13,15,18],feature_nam:[0,1,7,9],feautur:9,fed:1,feed:[0,2,3,11,15,17],feed_forward:1,feed_forward_out:1,feed_forward_train:1,feedback:4,feeddorward:4,feedforward:[1,4,12],feel:[0,5,6,11,13,15,19],feet:0,fetch:6,fetch_california_h:[],fetch_openml:[],few:[1,3,4,5,9,18],fewer:[0,9,11],ffnn:[1,12],field:[0,3,6,12,15],fifth:[0,6],fig:[0,1,2,3,4,6,7,12,13,14],fig_id:[0,6,7,9],figaxi:18,figsiz:[0,1,2,3,4,6,7,8,9,10],figur:[0,1,2,3,4,5,6,7,8,9,10,12,13,14,15],figure_id:[0,6,7,9],figurefil:[0,6,7,9],file:[0,1,2,4,5,6,7,9,18],file_prefix:4,filenam:18,filenotfounderror:[],fileout:18,filepath_or_buff:0,fill:[5,9],filter:[3,4],filtered_flat_arg:[3,4],filtered_tb:[3,4],financ:0,find:[0,1,2,3,5,6,7,8,9,10,11,12,13,14,15,18],find_top_boxed_arg:2,fine:[0,14],finish:2,finit:[3,5,6,12,13,18],first:[0,1,2,3,5,6,7,8,9,10,11,13,14,16,17,18,20],firsteigvector:11,fit:[1,3,4,5,6,7,8,9,11,12,13,18],fit_beta:6,fit_intercept:[0,5,6],fit_mod:9,fit_transform:[0,6,8,9,11],fiti:0,five:[0,9],fix:[0,3,4,6,10,11,12,13],fixedformatt:6,fixedloc:6,fkkt:5,flag:4,flat:[12,13],flatten:[1,3,4,5,16],flexibl:[1,6,8,10,12],float32:[4,9],float64:[4,16],flop:[5,16],flow:[1,4,12],fluctuat:5,fly:11,flyvbjerg:18,fm:0,fma:[],fmax:3,fmesh:13,fn:[3,4],focu:[0,3,4,5,6,15,20],focus:[1,6,7,16],fold:[6,9],folder:[0,4,6],follow:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20],font:[0,7,18],fontdict:18,fontsiz:[1,6,8,9,10,18],fontweight:1,footprint:3,foral:8,forc:[0,5,6,10,11],forcast:4,forecast:[4,12],forelesningsvideo:17,forest:[0,1,9,15,17],forget:11,form:[0,3,4,5,6,7,8,9,11,12,13,15,16,18],formal:[3,4,14,18],format:[0,1,2,3,4,6,7,8,9,10,11,15,18,20],format_data:4,formatstrformatt:[6,13],formul:[4,6,11,14],formula:[3,13,18],forth:[4,12],fortran2003:15,fortran90:18,fortran:[0,15,16],fortun:[0,11],forward:[0,3,6,15,16,17],forward_backward:[3,4],found:[1,2,4,5,6,12,13],foundat:15,four:[4,5,6,8,12,16,17],fourier:0,fourierdef1:3,fourierdef2:3,fourierseriessign:3,fourth:12,fr:5,frac:[0,1,2,3,5,6,7,8,9,10,11,12,13,14,16,18],fractal:[],fraction:9,frame:7,framework:[1,8,10,18],frank:[5,11],frankefunct:[5,6,11],free:[0,6,11,13,15,16,18,19,20],freecodecamp:15,freedom:5,freeli:0,frequenc:[3,6,7,18],frequent:[0,8,9,13],frequentist:15,fresh:10,frida:19,fridai:17,friedman:[6,17,20],friendli:4,frog:3,from:[0,1,2,3,4,6,7,8,9,11,13,14,15,16,17,18,19,20],from_cod:9,from_logit:[3,4],from_tensor_slic:4,fromnumer:2,front:[0,4,5],fstream:18,fulfil:[2,5,12],full:[0,1,3,5,7,9,10,13,18],full_matric:5,fulli:[3,6,12,17,18],fun:[2,13,15],fun_nam:[],func:[0,2],functionali:11,fundament:[0,6,15],funtion:2,further:[2,9],furthermor:[0,3,5,6,7,11,12,13,15],futur:[0,4,8,9],futurewarn:[],fx:5,fy:[17,19],g0:2,g42mrgv128v34gnnhxwk9nrc0000gp:[],g:[0,1,2,3,4,5,6,8,9,10,11,13,18],g_0:2,g_1:[2,10],g_2:[2,10],g_:[2,9,10],g_analyt:2,g_dnn_ag:2,g_euler:2,g_i:2,g_m:[3,10],g_n:3,g_re:2,g_t:2,g_t_d2t:2,g_t_d2x:2,g_t_dt:2,g_t_hessian:2,g_t_hessian_func:2,g_t_jacobian:2,g_t_jacobian_func:2,g_trial:2,g_trial_deep:2,g_vec:2,gain:[1,5,9,10,13],galleri:0,game:4,gamma1:8,gamma2:8,gamma:[0,2,8,9,10,11,13],gamma_0:10,gamma_1:10,gamma_1x:10,gamma_:0,gamma_i:[0,8,18],gamma_j:13,gamma_k:13,gamma_m:10,gamma_x:0,gap:[6,8],gate:[4,12],gather:[0,1,12],gaug:12,gaussbacksub:16,gaussian:[4,5,6,8,14],gaussian_point:14,gaussian_rbf:8,gave:13,gbc:17,gca:[2,6,8,13],gd:1,gd_clf:10,gdclassiffiercgain:10,gdclassiffierconfus:10,gdclassiffierroc:10,gdm:13,gdregress:10,ge:[1,5,7,18],gemv:5,gen:18,gen_loss:4,gen_tap:4,gender:0,genener:4,gener:[0,1,2,3,5,6,8,10,11,12,13,14,16,20],generallay:12,generate_and_save_imag:4,generate_imag:4,generate_latent_point:4,generate_simple_clustering_dataset:14,generated_imag:4,generator_loss:4,generator_loss_list:4,generator_model:4,generator_optim:4,genexpr:[],genom:15,geodes:11,geometr:[0,13],geometri:5,georg:20,geotif:6,geq:[2,5,8,9,13],geron:[0,17,20],get:[0,1,2,3,4,5,6,7,9,10,11,13,15,16,18],get_dummi:9,get_loc:[],get_next_color:[],get_paramet:2,get_split:9,get_yaxi:8,get_yticklabel:6,getsolutionslic:5,getval:[],gg:18,gibb:15,gif:4,gini:10,gini_index:9,git:[0,15],github:[0,3,4,6,14,15,17,20],gitlab:[0,15],give:[0,1,2,3,5,6,7,8,9,10,12,13,14,15,17,18,20],given:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,18],global:[6,7,13,18],glorot:1,gmail:[],go:[0,1,3,5,6,8,9,11,12,13],goal:[0,7,9],goe:[0,1,2,5,6,13,14,16],golden:13,gone:5,gong:1,good:[1,3,4,5,6,9,10,11,13,15,17,18,20],goodfellow:[4,17,20],googl:[1,3,4,14,15],got:[1,6],gov:6,gp:20,gpu:1,grad:[2,13],grad_analyt:13,grade:17,gradient:[0,3,4,7,8,9,12,15,17],gradient_desc:3,gradientboostingclassifi:10,gradientboostingregressor:10,gradients_of_discrimin:4,gradients_of_gener:4,gradienttap:4,gradual:[1,14],grai:[4,6],graph:[1,9,11,12,13],graph_from_dot_data:9,graph_funct:[3,4],graphic:[0,1,9],grasp:0,gray_r:[1,3],grayscal:3,great:[5,13],greater:[1,7,18],greatli:13,greedi:9,green:[0,3,9,18],grei:4,grid:[1,3,6,7,8,12,18],grossli:13,ground:0,group:[0,6,7,9,14,15,17],groupbi:0,grow:[1,3,9,10],growth:0,gru:4,guarante:[0,3,4,13,18],guess:[1,4,10,13,14],guestrin:10,guid:1,guilherm:19,h1:2,h21:17,h:[0,1,5,6,8,13,18,20],h_1:[2,13],h_2:[2,13],h_:[0,13],h_m:10,ha:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,18],habit:0,had:[0,1,6,7,13],hadamard:[1,12],half:[1,8,9],halv:10,hand:[0,1,2,3,5,11,12,13,15,16,17,18,20],handi:3,handl:[0,1,2,5,9,11,15],handle_unknown:9,handsid:12,handwrit:12,handwritten:[1,5],happen:[1,2,3,4,5,6,10,13,18],hard:[1,7,8,10,13],hardcopi:15,harder:[0,1],harmon:3,hasn:1,hassl:[0,15],hast:15,hasti:[0,6,17,20],hat:[1,5,6,7,9,10,11,12,13,16,18],have:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,18],haven:1,he:7,head:[0,4,10,18],header:0,heads_proba:10,health:0,hear:[0,13],heart:[0,7],heatmap:[0,1,3,7],heavili:0,heavisid:1,height:[1,3,6],held:13,help:[0,1,4,12,13],helper:[4,14],henc:[0,5,6,8,9,10,12,13],her:7,here:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,20],hereaft:[0,8,12],hermitian:16,hessenberg:16,hessian:[0,2,5,13],heterogen:[9,10],hi:7,hidden:[1,3,4,12],hidden_bia:1,hidden_bias_gradi:1,hidden_layer_s:[0,1],hidden_neuron:4,hidden_weight:1,hidden_weights_gradi:1,hierarch:5,high:[0,1,2,3,4,5,6,9,10,11,13,14,15,16],higher:[0,1,3,5,6,8,13],highest:[1,2],highli:[0,3,4,10,15,16,18,20],highwai:0,hing:8,hint:13,hip:15,hire:0,hist:[4,6,7,18],histogram:[0,6,7,18],histor:[7,11],histori:[3,4,12],histplot:[],hit:[],hitherto:5,hjorth:19,hobbi:18,hoc:5,hoff:20,hold:[1,3,6,13,14,18],holder:0,home:0,homework:[6,13],homogen:[1,3,9,10,13],honchar:2,hopefulli:[0,11,18],horizont:11,hors:[3,7],hot:[1,9],hour:[1,15,17,18,19],house_pric:[],how:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,20],howev:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,18],hs:[],hspace:[0,4,8,10,18],hstack:1,htf:17,html:[7,11,15,17,20],http:[3,4,6,7,11,13,14,15,16,17,20],huang:0,huber:0,huge:[1,3,4,15],human:[0,1,3,6,9,12],humid:9,hundr:1,hungri:1,hybrid:17,hydrogen:0,hyperbol:[1,4,12],hyperparam:8,hyperparamet:[3,4,5,6,9,13],hyperplan:11,i0:0,i1:[0,6,8,12],i2:[0,8,12],i3:[0,12],i5:0,i:[0,2,3,4,5,6,7,8,9,10,11,12,13,14,16,19],i_1:[5,6],i_2:[5,6],i_:13,ian:20,ic:1,id:[7,13],ida:19,idea:[0,1,2,3,4,6,9,10,12,13,16,18],ideal:[0,2,6,8,13,18],idem:6,ident:[5,6,12,13,16],identical:5,identifi:[0,1,7,9,11,12,13,14],idum:18,ieor:18,ifi:20,ifs:15,ignor:[0,1,3,9],ii:[16,18],iii:16,ij:[0,1,3,6,8,12,14,16,18],ik:[0,16],illustr:[5,7,10,12,13,14,15],im:6,imag:[1,3,4,6,9,11,12,14,20],image_at_epoch_:4,image_batch:4,image_height:3,image_path:[0,6,7,9],image_width:3,imageio:6,images_from_seed_imag:4,imagin:1,immedi:[0,3,4,6,15],implement:[0,2,3,4,5,6,8,9,10,11,12,13,14],impli:[3,5,6,7,13,16],implicit:3,implicitli:[11,18],importantli:3,impos:[0,6,11,12],imposs:[0,5],impress:[0,12],improv:[0,4,5,9,10,11,13],impur:9,imread:6,imshow:[1,3,4,6],in3050:20,in4080:20,in4300:20,in5400:[3,20],in_out_neuron:4,inaccur:13,inacio:19,inact:12,inadequ:0,inch:6,includ:[0,1,2,3,4,5,6,7,11,12,15,18,19,20],include_bia:[6,9],incom:12,incorrect:1,incoveni:8,increas:[0,1,3,4,5,6,7,8,9,11,12,13,18],increasingli:18,ind:6,inde:[0,2,4,5,6],indefinit:4,indent:18,indentationerror:18,independ:[0,5,6,7,8,12,13,18],index:[0,1,3,4,10,14,15,16,18,20],index_col:0,index_of:[],indic:[0,1,3,4,5,6,9,10,11,13],indispens:6,individu:[1,6,7,10,12,18],indu:0,indx1:2,indx2:2,indx3:2,indx:16,ineffici:[3,13],inequ:8,inequaltii:13,inertia:13,inf1000:15,inf1100:15,inf1100l:15,inf1110:15,inf3000:20,inf4490:20,inf5860:20,infeas:9,infer:[0,1,4,6,20],infer_nrow:0,inferenc:1,infil:[0,6,7,9],infin:[5,6,7,11],infinit:3,infinitesim:18,influenc:[6,10],influenti:1,inform:[0,1,3,4,6,9,11,12,13,14,16,20],infti:[3,6,13,18],ingeni:13,ingrad:2,ingredi:[0,9],inher:6,inherit:16,initi:[0,1,2,6,10,13,14,16,18],initial_epoch:[3,4],initialis:18,initialize_root:[],inject:14,inlin:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,18],inner:13,innov:20,inp:4,inplac:13,input:[0,1,3,4,5,6,7,8,9,10,12,13,14,18],input_dim:1,input_shap:[3,4],inputs:1,inputs_shuffl:[0,1],insert:[3,5,6,8,10,18],insid:[0,4,7],insight:[0,1,5,15,20],insist:[6,13],inspir:[0,1,12,20],instabl:2,instal:[0,1,5,6,9],instanc:[0,1,2,4,6,9,11,13],instanti:10,instead:[0,1,2,3,4,5,6,8,9,11,13,14,16,18],institut:1,instruct:[0,1],int32:10,int64:[],int_0:18,int_:[3,6,18],int_a:18,intak:0,integ:[1,2,13,14,16,18],integer_vector:1,integr:[3,6,18],intellig:[0,14,20],intend:10,intens:1,intention:14,interact:[0,6,9,12,15],intercept:[0,6,8,11,13],intercept_:[0,6,8,9,13],interchang:[5,12,16],interconnect:1,interest:[0,1,2,3,4,5,6,7,8,9,12,15,17,18],interfac:[0,1,16],interior:[0,9],intermedi:16,intern:[1,10,12],interpol:[1,3,4,6,12],interpr:5,interpret:[0,1,6,9,10,12,13,16,18],interv:[0,3,5,6,7,13,18],intial:13,intract:[0,4],intrins:[3,11,16,18],intro:[15,20],introduc:[0,1,5,6,8,10,12,13,16,18],introduct:[1,2,4,13,17,20],introductori:[0,4,16,20],intuit:[0,5,6,8,12,13],inv:[0,5,13],invalid:1,invalu:[0,13,15],invari:1,invd:5,inver:8,invers:[0,3,6,13],invers_period:18,inverse_transform:8,invert:[0,5,7,10],invok:[0,8],involv:[0,2,6,7,11,12],io:[0,15,17,18,20],iomanip:18,iostream:18,ip:[0,8,18],ipca:11,ipykernel_42331:[],ipykernel_42376:[],ipykernel_42449:[],ipykernel_42456:[],ipykernel_42530:[],ipykernel_42541:[],ipykernel_42553:[],ipykernel_42573:[],ipykernel_42580:[],ipykernel_42586:[],ipykernel_47411:[],ipykernel_47448:[],ipykernel_47647:[],ipykernel_47724:[],ipykernel_47735:[],ipynb:15,ipython:[0,1,2,3,4,5,6,7,8,9,10,11,13,14,15,18],iq:6,iri:[8,9],irreduc:6,irrelev:5,irrespect:0,is_integ:[],isbox:[2,13],iscomplexobj:[],isinst:[2,13],isn:5,isnul:0,isomap:11,issu:[1,3,4,9,14,16],it_arrai:13,item:[0,13],items:16,iter:[1,2,3,4,6,7,8,11,13,14,18],itr:5,its:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,20],itself:[5,6,12,18],ix:18,j1:16,j:[0,1,2,3,4,5,6,8,9,11,12,13,14,16,18,20],j_:6,j_lasso_sk:6,j_ridge_sk:6,j_sk:6,jackknavg:18,jackknif:[6,15],jackknstd:18,jackknvar:18,jackknvec:18,jacobian:[2,13],jacobian_shap:2,jargon:18,jason:4,jax:[3,4,14],jensen:19,jerom:20,ji:[12,16],jj:[0,5,6],jk:[0,1,6,12,16],jl:0,jm:16,joao:19,joaogca:19,job:[2,8,10],join:[0,4,6,7,9],joint:[4,5],journal:18,judg:13,judgement:6,julia:[15,16],jump:18,junk:4,jupyt:[0,15,20],just:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,18],justif:0,justifi:[3,10],jy:[],k0:7,k1:7,k:[0,1,3,5,6,7,8,9,10,11,12,13,14,15,16,18],kaggl:6,kappa_d:18,karim:[],karlsen:[],karush:8,keep:[0,1,4,5,6,11,13,14,16],keepdim:[1,6,10,16],kei:[0,1,3,6,12,20],kept:[4,6,14],kera:[0,4,15],kernel:[0,1,3,15],kernel_regular:[1,3],kernel_s:4,kernelpca:11,kev:0,kevin:20,keyboardinterrupt:[2,3,4],keyerror:[],keyword:[6,13,16],kfold:6,kg:1,ki:16,kick:[1,13],kiener:2,kilomet:6,kind:[0,2,3,4,8,12,13,14],kj:[6,12,16],kjm:15,kkt:[5,8],kktsolver:5,kl:18,km:12,kmean:14,kmeanspoint:14,kn_k:14,know:[0,1,2,5,6,8,13,15,18],knowledg:[0,15],known:[1,3,4,5,6,7,8,9,12,16,18,20],kondev:0,kp:18,kpca:11,kroneck:14,kuhn:8,kwarg:[0,2,3,4,13],kwd:[0,3,4],kwown:0,l0:7,l1:[0,1,3,5,7],l1_l2:[1,3],l1regl:5,l1regls_mosek2:5,l1regls_mosek:5,l2:[1,3],l:[0,1,2,3,5,6,7,8,10,11,12,13,16,18],l_1:7,l_2:[7,13],l_:16,l_j:12,la:13,la_i:12,la_k:12,lab:[15,17],label:[0,1,2,3,4,5,6,7,8,9,10,12,13,14,15,16,18],label_prob:13,labelencod:[7,10],labels:[6,8,9],labels_shuffl:[0,1],laboratori:17,lack:0,lagari:2,lagrang:[8,11],lambda:[0,1,2,3,5,6,7,8,10,12,13,18],lambda_0:11,lambda_1:[5,8,11],lambda_2:[8,11],lambda_:11,lambda_i:[8,11],lambda_iy_i:8,lambda_jy_iy_j:8,lambda_k:8,lambda_n:[5,8],lamda:1,lamdbda:5,land:[0,8],landmark:8,landscap:13,langl:[0,6,11,18],languag:[0,1,4,8,15,16,20],lapack:[5,16],laplac:5,laptop:15,larg:[0,1,2,4,5,6,8,9,10,11,13,15,16,18,20],larger:[0,3,5,6,8,10,11,13,18],largest:[4,8,11],lasso:[0,7,15,17],lasso_sk:6,last:[0,1,2,3,4,5,6,7,8,9,10,12,13,16,17,18],latent:4,latent_dim:4,latent_point:4,latent_space_value_rang:4,later:[0,1,4,6,7,8,12,13,14,15,18],latest:4,latest_checkpoint:4,latter:[0,3,5,6,7,8,11,13,16,17,18],lattic:12,law:0,layer:[0,4,13],lbfg:[7,9,10,11],lcc:[5,6],lda:11,ldot:[0,6,11,18],le:[5,7,10,13,18],lead:[0,1,3,5,6,7,8,9,10,11,12,13,16,18],leaf:9,leaki:1,leakyrelu:4,lear:13,learn:[3,4,5,6,7,8,9,10,12,16,17,20],learnabl:3,learner:10,learning_r:[3,8,10],learning_rate_init:[0,1],learning_schedul:13,least:[0,2,7,8,10,11,15,16,17,18],leat:13,leav:[0,1,3,5,6,9,11],lectur:[0,1,5,10,11,12,13,15,16,17,20],lecturenot:[15,20],lecturenovember11:[],lecturenovember12:[],lecturenovember19:[],lecturenovember25:[],lecturenovember26:[],lecturenovember4:[],lecturenovember5:[],lectureoctober14:17,lectureoctober15:17,lectureoctober1:[],lectureoctober21:[],lectureoctober22:[],lectureoctober28:[],lectureoctober29:[],lectureoctober7:[],lectureoctober8:[],lectureseptember10:[],lectureseptember16firstpart:[],lectureseptember16secondpart:[],lectureseptember17:[],lectureseptember23:[],lectureseptember24:[],lectureseptember2:[],lectureseptember30:[],lectureseptember3:[],lectureseptember9:[],lecturethursdayaugust26:[],lecturethursdayaugust27:[],left:[0,1,2,3,5,6,7,8,9,10,11,12,13,14,16,18],leftarrow:[8,12],legend:[0,2,3,4,5,6,7,8,9,10,13],len:[0,1,2,3,4,5,6,8,9,10,11,12,16,18],len_index:0,length:[0,1,2,3,4,8,9,13,15,18],length_of_sequ:4,leq:[0,5,7,8,13,14,18],less:[0,1,3,4,5,6,8,9,13,18],lessen:1,let:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,18],letter:[0,16,18],level:[0,1,5,6,9,15,16,17],li:[8,11],lib:[0,1,2,3,4,6,7,8,11,13,14],liblinear:[8,10],librari:[0,1,2,3,4,5,6,9,10,11,16,18,20],licens:[0,1,15],lie:[0,6,11,18],life:[0,1,8,12],lifetim:13,like:[0,1,2,3,4,5,6,7,9,10,11,12,13,15,16,18],likelihood:[0,1,5,9,13],lim_:18,limit:[0,5,6,7,8,11,12,16],lin_clf:8,lin_model:0,lin_reg:9,linalg:[0,2,5,6,8,11,13,16,18],line1:8,line2:8,line2d:13,line3:8,line:[0,1,3,6,8,11,13,18],linear:[1,3,5,6,7,9,10,11,12,15,17,18],linear_model:[0,5,6,7,8,9,10,11,13],linear_regress:6,linearli:5,linearloc:[6,13],linearregress:[0,6,7,9],linearsvc:8,liner:[1,3],linerar:10,linewidth:[0,2,4,6,8,9,10],link:[0,4,9,12,15],linlag:5,linpack:16,linreg:0,linspac:[0,2,3,4,6,8,9,10,13,16,18],linu:4,linuek:[],linux:[0,1,15],liquid:0,list:[0,1,2,3,4,5,9,15],listcomp:2,listedcolormap:[9,10],literatur:[1,7,14,20],littl:[1,3,9,12],live:8,ll:[0,18],lle:0,lloyd:[4,14],lmb:[2,5,6],lmbd:[0,1,3],lmbd_val:[0,1,3],lmbda:13,ln:[1,13],lo:5,load:[0,1,4,6,7,9,10],load_boston:0,load_breast_canc:[1,7,9,10,11],load_data:[3,4],load_digit:[1,3],load_iri:[8,9],loc:[0,3,6,7,8,9,10],local:[0,1,3,7,12,13],locat:[2,3,8],lock:[3,4],log10:[5,6],log:[0,1,2,4,5,6,7,9,10,11,13,16],log_:0,log_clf:10,logarithm:[0,5,7,16],logic:[0,1,9],logist:[0,1,2,8,9,10,11,12,15,17],logistic_predict:13,logisticregress:[7,9,10,11],logit:7,logreg:[7,9,10,11],logspac:[0,1,3,5,6],longer:[2,3,8,10,14,16,18],loocv:6,look:[0,1,2,3,4,5,6,7,8,9,10,11,13,16,18],loop:[1,4,6,10,12,14,15,16,18],lose:1,loss:[0,1,3,4,5,6,7,8,10,11,13,16],loss_fil:4,lossfil:4,lost:4,lot:[0,1,4,6],low:[0,6,9,10,11,18],lower:[0,1,3,6,9,10,16],lowercas:16,lowest:[9,13,18],lr:[1,3,4,10],lstat:0,lstm:4,lstm_2layer:4,lstsq:0,lt:6,lu:[0,5],lubksb:16,luckili:2,ludcmp:16,lux:16,lvert:1,lw:0,m:[0,1,2,3,5,6,8,9,10,11,12,13,16,17,18,19,20],m_1:14,m_:[9,12],m_h:0,m_k:14,m_l:12,m_n:0,m_p:0,m_t:13,ma:11,mac:[3,4,14],machin:[1,3,4,5,6,7,9,10,11,12,14,16,17,20],machinelearn:[6,15,17,20],mackai:20,made:[0,1,3,4,5,6,7,9,11,12],mae:0,magic:4,magnitud:[1,6,7,13],mai:[0,1,2,3,5,6,7,8,9,11,12,13,15,16,18],mail:17,main:[0,1,3,4,5,6,7,9,16,20],mainli:[0,5,6,7,9],maintain:6,major:[1,6,9,10,13,16],make:[1,2,3,4,5,6,7,8,11,12,13,15,16,18,20],make_axes_locat:6,make_moon:[8,9,10],make_pipelin:[0,6,10],make_vjp:[2,13],makedir:[0,6,7,9],makeplot:0,malcondit:16,malign:[1,7,9],mammographi:5,manag:[0,2,3,15],mani:[0,1,3,4,5,6,7,8,9,11,13,14,15,16,18,20],manifold:11,manner:3,manual:6,map:[0,1,2,6,7,8,11,12,14,18],margin:[0,5,8],mari:19,marit:0,marker:[0,7,16],markov:15,marsaglia:18,mask:18,mass:[0,1,5,13],massag:0,masses2016:0,masses2016ol:0,masses2016tre:0,masseval2016:0,master:[6,17],mat1100:15,mat1110:15,mat1120:15,mat3155:[],mat4155:[],mat:15,match:[0,1,4,5,13,14],materi:[4,5,7,13,16],math:[3,7,12,13,16,18,20],mathbb:[0,4,5,6,7,8,11,12,13,14,16,18],mathbf:[0,5,6,7,8,13,16,18],mathcal:[1,5,6,7,13],matheemat:3,mathemat:[0,6,11,12,13,15,16,17,18,20],mathrm:[0,1,3,4,5,6,7,8,9,10,11,12,13,14,18],matmul:[1,2,5],matmul_adjoint_1:2,matmul_vjp_0:2,matmul_vjp_1:2,matnat:[17,19,20],matplotlib:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,18],matplotlibdeprecationwarn:[6,13],matric:[0,1,3,4,6,7,8,11,13,15],matrix:[0,2,3,4,6,7,8,10,13],matshow:1,matter:[2,3,13],max:[0,1,2,3,4,9,10,12,13],max_depth:[0,9,10],max_diff1:2,max_diff2:2,max_diff:2,max_it:[0,1,7,8,11,13],max_iter:14,max_leaf_nod:10,max_queue_s:[3,4],max_sampl:10,maxdegre:[0,6,10],maxdepth:10,maxim:[1,4,5,7,8,11],maximum:[0,1,2,3,5,7,8,9,10,13,14],maxpolydegre:[5,6],maxpooling2d:3,mbox:[5,6],mc:18,mcculloch:12,mcint:18,mcintsqr2:18,md:11,mdoel:4,mean:[1,2,3,4,5,6,7,9,10,11,12,13,14,15,16],mean_absolute_error:0,mean_divisor:14,mean_i:18,mean_matrix:14,mean_squared_error:[0,4,6,7,10],mean_squared_log_error:0,mean_vector:14,mean_x:18,meaning:[0,4,7],meansquarederror:0,meant:[2,3,7,10,13],meantempvec:18,meanvec:18,measur:[0,1,2,5,6,9,11,12,14,18],mechan:[0,4,18],median:0,medicin:12,medium:[4,8,13],medv:0,meet:[0,19],mehta:0,memori:[3,4,11,12,13,16],mention:[0,12,13,18],mere:0,mersienn:18,meshgrid:[2,5,6,8,9,10,11],messag:[5,13],messi:2,met:[0,3,8],metadata:2,meteorolog:9,meter:6,method:[0,1,2,3,4,5,7,8,11,12,14,15,16,17,20],metion:6,metric:[0,1,3,6,7,9,10,14],metropoli:15,mev:[0,18],mgd:13,mglearn:15,mgrid:13,mhjensen:[1,2,3,4,6,7,8,11,14],mi:10,microsoft:20,mid:1,midel:4,midpoint:9,might:[0,1,2,4,6,9,13],mild:9,miller:18,millimet:6,million:0,mimic:12,min:[0,2,5,8,9],min_:[0,2,5,14],min_samples_leaf:9,mind:[0,6,13],mindboard:4,mine:15,mini:[1,11,12,13],minibatch:[1,11,13],minibathc:13,miniforge3:[0,1,2,3,4,6,7,8,11,13,14],minim:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,18],minima:[0,1,7,13],minimum:[0,1,2,6,8,9,11,13],minmaxscal:0,minor:[6,13,18],minst:1,minu:7,mirror:9,misc:6,misclassif:[8,9,10],misclassifi:[8,10],miser:0,mismatch:1,miss:[0,10],mistak:4,mit:20,mix:[1,2],mixtur:13,mk:[9,16],mkdir:[0,6,7,9],ml:[0,1,10,13,16],mlab:18,mle:[5,7],mline:[],mlir:[],mlir_graph_optimization_pass:[],mlp:1,mlpclassifi:1,mlpregressor:0,mm:16,mn:[12,18,20],mnist:[1,11],mod:18,mode:[17,18],model:[2,3,5,7,8,9,10,11,13,14,15,18,20],model_select:[0,1,3,5,6,7,9,10,11],moder:10,modern:[0,6,7,15],modif:[2,12,13],modifi:[0,1,3,5,7,8,10,12,13],modul:[0,2,3,4,5,6,7,8,9,10,11,13,16],modular:18,modulenotfounderror:[5,7,8,9],modulo:18,moe:[5,11],moment:[5,6,13],monitor:13,monoton:[5,12,18],mont:[0,6,15,18,20],montecarlocycl:18,moor:[5,6],more:[0,1,2,4,5,7,8,9,10,11,12,13,14,15,17,18],moreov:[0,3],morten:19,mortenimac:[],mosek:5,most:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,17,18],mostli:[1,11],motion:[0,13],motiv:[1,4],move:[0,4,5,6,7,9,12,13,14,18],mp4:17,mpl:[0,7],mpl_toolkit:[2,6,13],mplot3d:[2,6,13],mplregressor:1,mse:[0,4,5,6,9,10],mse_simpletre:10,mselassopredict:5,mselassotrain:5,mseownridgepredict:6,msepredict:5,mseridgepredict:[5,6],msetrain:5,msg:[],msle:0,mt19937_64:18,mt:[7,12],mu0:18,mu1:18,mu2:18,mu:[0,6,11,13,18],mu_:[6,18],mu_i:6,mu_n:11,mu_x:18,much:[0,1,2,3,4,5,6,8,9,10,11,12,13,16,18],mul:5,multi:[0,1,3,7,15],multiclass:[1,7],multidimension:[11,12],multilay:1,multinomi:7,multipl:[2,4,5,6,7,12,13,18],multipli:[3,5,6,11,13,16,18],multiplum:8,multivari:[0,2,10,11,15,18],multivariate_norm:[11,14],murphi:[11,20],must:[0,1,2,5,6,8,10,12,13,14,18],mutat:7,mutual:[1,3,6,13],mx_:18,myenv:[0,1,2,3,4,6,7,8,11,13,14],myriad:[0,15],mz1:18,mz2:18,n1:16,n2:16,n:[0,1,2,3,4,5,6,7,8,10,11,12,13,14,16,18],n_0:[12,18],n_:[1,2,3,8,12,18],n_b:18,n_boostrap:[6,10],n_bootstrap:6,n_categori:[1,3],n_cluster:14,n_compon:11,n_epoch:13,n_estim:10,n_examples_to_gener:4,n_featur:1,n_filter:3,n_hidden:2,n_hidden_neuron:[0,1],n_i:18,n_input:[0,1,3],n_instanc:9,n_iter_i:[7,11],n_job:10,n_k:14,n_l:[12,18],n_layer:1,n_m:9,n_neuron:1,n_neurons_connect:3,n_neurons_layer1:1,n_neurons_layer2:1,n_point:14,n_sampl:[6,8,9,10,14],n_split:6,n_step:4,n_t:2,n_x:2,nabla:[1,13],nabla_:[2,13],nabla_w:13,nag:13,naimi:0,naiv:7,naive_kmean:14,nall:0,name:[0,1,3,4,5,6,7,8,9,10,12,13,14,15,16,19],nameerror:[6,10],namespac:18,narrow:13,nary_f:[2,13],nary_op_arg:[2,13],nary_op_kwarg:[2,13],nary_oper:[2,13],nation:[1,5],nativ:15,natur:[0,1,4,8,9,12,13,18,20],navier:12,nb:18,nb_:16,nboot:18,nd:14,ndarrai:[2,6],ndim:2,ne:[9,10,16,18],nearest:[1,3,6,11],nearli:13,neccesari:6,necess:2,necessari:[0,1,3,4,8,14],necessarili:[0,4,11,18],necesserali:5,neck:7,need:[0,1,2,3,4,5,6,8,9,10,11,12,13,14,16,18],neg:[0,1,3,5,6,7,10,13,16,18],neg_mean_squared_error:6,neglect:18,neglig:18,neighbor:[3,6,11],neither:[4,13],neq:[13,14,18],nervou:12,nest:[2,9,12],nesterov:13,net:[2,4,12],netlib:16,network:[0,9,13,15,17,20],neural:[0,7,13,15,17,20],neural_network:[0,1,2],neuralnetwork:1,neuron:[1,2,3,4,12],neutral:0,neutron:0,never:[1,3,4,6,9,18],new_box:[2,13],new_root:[2,13],new_trac:[2,13],new_tracing_count:[3,4],newaxi:[0,3,6,9],newli:0,newton:[1,7,8,13,18],next:[0,1,2,3,4,5,6,8,9,13,14,18],next_guess:13,next_input:4,ng:1,ngini:[],ni:14,nian:[],nice:[0,1,5,11],nichola:[],nicholaskarlsen1102:[],niter:13,nitric:0,nlambda:[5,6],nlevel:[],nm:18,nm_n:0,nmse:6,nn:[2,5,6,12,16],nn_model:1,nnmin:2,node:[1,2,3,9,10,12],node_constructor:2,nois:[0,4,5,6,8,9,10,13],noise_dimens:4,noisi:[1,6],non:[0,1,3,4,5,6,7,9,10,11,12,13,14,16,18],none:[0,1,2,3,4,5,9,10,13,18],nonlinear:[3,6,8,9,11,12],nonneg:[6,9,13],nonparametr:6,nonsens:18,nonsingular:16,nonumb:[3,7,8,13,16],nor:[1,4,13],norm:[0,1,5,6,8,11,13],normal:[3,4,5,6,7,8,9,10,11,12,13,15,16],normali:16,normalize_kwarg:[],norwai:6,notat:[0,2,5,6,13,14,18],note:[0,1,2,3,4,5,6,7,8,11,12,13,14,15,16,17,18,20],notebook:[0,1,3,9,15],noth:[1,2,5,8,12,14,18],notic:[4,5,12,13,16,18],notimplementederror:[],notion:3,notrace_primit:[],novel:[3,6,10],novemb:1,now:[0,2,4,5,6,7,8,10,11,12,13,14,16,18],nowadai:[0,1,3,9,15],nox:0,np:[0,1,2,3,4,5,6,7,8,9,10,11,13,14,16,18],npr:2,nsampl:6,nspin:18,nt:2,nthi:0,ntrained_model:6,nu:18,nuclear:5,nuclei:[0,18],nucleon:0,nucleu:0,num:4,num_coordin:2,num_hidden_neuron:2,num_it:2,num_neuron:2,num_neurons_hidden:2,num_output:[3,4],num_point:2,num_tre:10,num_valu:2,number:[1,3,4,5,6,7,8,9,10,11,12,13,14,16,17,19],numberid:7,numberparamet:3,numer:[0,5,6,9,10,11,12,13,15,16,20],numpi:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,18],numpy_vjp:2,numpy_wrapp:2,nunmpi:5,nvalu:[],nx:2,nx_test:6,nx_train:6,nx_train_mean:6,ny:18,ny_pr:6,ny_train:6,ny_train_mean:6,o:[0,6,7,8,9,11,16,20],obei:[6,11,13],object:[0,1,2,4,5,6,8,10,13,16],objsens:5,obliqu:5,observ:[1,3,5,6,7,8,9,10,11,12,13,14],obtain:[0,1,5,6,7,8,9,10,12,13,14,16,18],obviou:[5,6,11,18],obviouli:0,obvious:[0,4,5,6,16],oc:5,occupi:0,occur:[0,6,8,9,16,18],od:0,odd:[0,3,7],odenum:2,odesi:2,oen:0,off:[1,3,4,5,9,13,18],offer:[6,11,15,16,17],offic:19,offici:17,ofil:18,ofstream:18,often:[0,1,3,4,5,6,7,8,9,10,11,12,13,14,15,16,18],ofter:16,ol:0,old:[1,5,10,13],ols_sk:6,ols_svd:6,olsbeta:5,omega:[2,3,6],omega_0:3,omit:[0,5],on_train_batch_begin:[3,4],onc:[1,6,9,11,13,18],one:[0,1,3,4,5,6,7,8,9,10,11,13,14,15,16,18],oneapi:[],onednn:[],onehot:1,onehot_vector:1,onehotencod:9,ones:[0,2,5,6,8,9,10,11,13,16,17],ones_lik:4,onl:3,onli:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,18],onlin:[11,17],onto:[5,11],op_nam:[3,4],open:[0,1,4,6,7,9,15,17,18],oper:[0,1,3,5,6,10,11,12,13,15,18],operation:18,oplu:18,opmiz:13,opportun:0,oppos:[6,13],opposit:[1,5,8],opt:[1,5],optim:[0,2,3,4,5,6,7,9,10,11,14,17],optimis:[1,3],optimizer_v2:3,option:[0,1,3,5,6,7,8,11,16],optionalxlacontext:[3,4],optmiz:[1,8,13],orang:0,order:[0,1,2,3,5,6,7,8,9,10,11,12,13,16,18],ordinari:[0,2,3,7,11,13,15,17],oreilli:20,org:[3,4,7,11,15,16,20],organ:[6,7,10,16],orient:[1,5,18],origin:[0,3,5,6,8,11,12,13,16,18],orthogn:5,orthogon:[0,5,6,8,11,13,16],orthonorm:5,os:[0,1,4,5,6,7,8,9],oscar:1,oscil:[3,13],oslo:[0,15,17,19],osx:[0,15],other:[0,1,2,3,5,6,7,8,10,13,14,15,17,20],otherwis:[0,1,4,7,13,16],ouput:[5,7,12],our:[1,2,3,6,7,8,9,10,12,14,15,16,17],ourmodel:0,ourselv:[0,5,6,8,11,13],out:[0,1,2,4,5,6,7,8,9,10,11,12,13,15,16,18],out_fil:9,outcom:[0,7,9,10,12,18],outdoor:9,outer:[6,12],outfil:4,outfilenam:18,outgrad:2,outlier:[0,8],outlin:[6,10,11],outlook:9,outperform:10,output:[0,1,3,4,5,6,7,8,9,10,12,13,16],output_bia:1,output_bias_gradi:1,output_shap:4,output_weight:1,output_weights_gradi:1,outputlayer1:12,outputlayer2:12,outsid:4,over1:13,over:[0,1,3,4,5,6,9,10,12,13,16,18],overal:[1,10],overcast:9,overcom:[12,13],overdetermin:0,overfit:[0,1,3,6,9,10,13],overflow:[1,5],overhead:12,overlap:[3,7,8,9],overlin:[0,5,6,9,10,11,14,16],overst:0,overtrain:4,overview:[3,20],own:[4,5,6,8,12,13,15,16],owner:0,ownridgebeta:6,oxid:0,oyvinssc:19,p0:2,p1:2,p:[0,1,2,3,4,5,6,7,8,9,10,11,13,14,16,18],p_:[2,4,8,9],p_hidden:2,p_i:[5,18],p_j:18,p_n:18,p_output:2,p_x:18,pack:0,packag:[0,1,2,3,4,5,6,7,8,11,13,14,15,18],pad:[3,4],page:[0,15],pai:[0,1,9,13],pair:[0,2,3,9,15,18],panda:[0,4,5,6,7,9,11,15],paper:1,paradigm:0,parallel:[10,16],param:[2,4],paramat:2,paramet:[0,1,2,3,4,5,6,7,8,9,10,12,13,18],parameter:[0,6,10],parametr:[0,6],paramt:[3,5],parent:2,parent_argnum:[],park:18,parser:0,part:[0,1,3,5,6,10,16,17,18,20],partial:[0,1,5,6,7,8,10,11,12,13,18],particip:[15,17],particl:[0,4,13,18],particular:[0,1,2,3,5,6,9,10,11,12,13,18,20],particularli:[5,6,8,11,13,18],partit:[1,4,9],partli:6,pass:[2,3,12,14,18],past:[10,18],patch:[6,18],path:[0,4,6,7,9,15],patient:7,patter:4,pattern:[0,3,4,12,17,20],pauli:0,pc:[11,15],pca:[0,7,15,17],pcolor:6,pcolormesh:6,pd:[0,4,5,6,7,9,11],pde:2,pdf:[0,3,4,5,6,9,20],pedagog:0,penal:6,penalti:[6,13],penros:[5,6],pentagon:13,peopl:[0,1,9,13,15],per:[0,1,6,17],percentag:[0,10,11],perceptron:[0,1,7],perfect:[0,1],perfectli:[4,6],perform:[0,2,3,4,5,6,8,10,11,12,13,14,15,16,18],performac:4,perhap:[0,5,13],perimet:1,period:[1,4],permut:11,persist:13,person:[5,6,7,17,19],perspect:20,pertin:12,petal:[8,9],peter:20,petersen:18,phantom:18,phase:[6,12],phenomena:18,phi:8,phi_k:8,philip:[],philosophi:13,phone:19,photo:4,phrase:0,physic:[0,1,4,7,12,13,18,19,20],pi:[2,3,5,6,7,9,12,13,18],pick:[1,9,10,11,13,14,18],pickl:1,pictur:0,pie:15,piec:[11,14],pillow:[0,15],pinv:[5,6,13],pip3:[0,1],pip:[0,1,15],pipelin:[0,6,8,10],pit:4,pitfal:6,pitt:12,pixel:[1,3,4],pixel_height:[1,3],pixel_width:[1,3],place:[0,4,6,8,13,16],plai:[0,3,4,5,6,8,11,15],plain:[8,10,12,13,14],plan:[6,9,19,20],plane:[8,9],plateau:[5,18],platform:15,plausibl:12,pleas:[3,4,6,7,11,13,14],plenti:1,plethora:[3,12],plot:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,18],plot_confusion_matrix:[7,10],plot_count:6,plot_cumulative_gain:[7,10],plot_data:1,plot_dataset:8,plot_decision_boundari:[9,10],plot_import:10,plot_max:4,plot_min:4,plot_model:4,plot_numb:4,plot_predict:8,plot_regression_predict:9,plot_result:4,plot_roc:[7,10],plot_surfac:[2,6,13],plot_train:9,plot_tre:[9,10],plt:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,18],plu:[0,3,5,7],pm:[8,18],pmatrix:2,pn:3,png:[0,4,6,7,9],point:[0,1,2,3,5,6,7,8,9,10,11,13,14,16,18,19],point_1:4,point_2:4,poisson:15,poli:[6,8],poly100_kernel_svm_clf:8,poly3:0,poly3_plot:0,poly3dcollect:13,poly_featur:[8,9],poly_features10:9,poly_fit10:9,poly_fit:9,poly_kernel_svm_clf:8,polydegre:[0,5,6,10],polygon:13,polym:12,polynomi:[0,5,6,7,8,9,10,11],polynomial_featur:6,polynomial_svm_clf:8,polynomialfeatur:[0,6,8,9],polytrop:[0,6],pool:3,pool_siz:3,poor:[1,13],poorli:0,pop:2,popul:[0,5],popular:[0,1,3,6,7,8,9,11,12,15,16,18],popularli:0,portabl:10,portion:[11,13],pose:[0,4,5,6,11,18],posit:[0,1,2,3,5,7,8,10,11,13,14,16,18],possibl:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,15,16,18,19],posterior:5,postpon:0,postul:5,potenti:[0,3,5,6,12,13],potr:5,potrf:5,pott:12,power:[0,1,5,6,8,9,12,13],pp:[5,6],practic:[0,5,6,7,8,17,18],practition:[0,1,3],preced:[1,11,12,18],preceed:4,preceq:8,precis:[0,2,5,11,13,16,18],pred:[6,13],predicit:0,predict:[0,1,5,6,7,8,9,10,15,20],predict_prob:1,predict_proba:[7,10],predictor:[0,5,6,7,9,10,11],prefer:[0,1,6,8,9,11,15],prepar:[0,6,16],preprocess:[0,4,6,7,8,9,10,11],prerequisit:0,presenc:13,present:[0,5,6,9,12,13,16,17,18],preserv:[3,11,16],press:[13,20],pretrain:[1,4],pretti:[0,4,8,9,15],prev_centroid:14,prevent:[13,18],previou:[0,1,2,3,4,5,6,8,10,11,12,13,16,18],previous:[2,3,9,10,18],price:[0,4,9,13],primal:8,primari:[0,7],prime:18,primit:2,princip:[0,5,7,15,17],principl:[0,6,7,8,14],print:[0,1,2,3,4,5,6,7,8,9,10,11,13,14,16,18],print_funct:[8,9],printout:0,prior:[0,5,6],privat:0,prob:[1,18],probabilist:[0,20],probabl:[0,1,3,4,6,7,10,13,15,17],problem:[0,3,4,5,6,7,8,9,10,11,12,14,15,16,17,18],proce:[0,5,6,8,9,10,11,13,16],procedur:[2,4,5,6,8,10,11,13],proceed:16,process:[0,2,4,6,9,10,12,13,15,16,18,20],prod:20,prod_:[1,5,7],produc:[0,3,4,5,6,9,10,11,12,13,15,16,18],product:[0,1,3,5,6,7,8,12,13,15,16],profess:0,program:[0,1,4,5,6,8,12,14,15,16,17,18],programm:16,progress:[1,4,14],prohibit:6,project1:6,project:[0,1,2,3,5,11,13,15,17,18,19],project_root_dir:[0,6,7,9],promin:12,promis:8,prone:9,pronounc:[13,15],proof:[0,11,12,13],propag:[2,3,13,17],proper:[0,2,6],properli:[1,6,8,10,13],properti:[0,1,3,12,13,16],proport:[0,1,5,9,11,13,18],propos:[1,4,6,10],propto:[5,13],protect:[3,4],proton:0,prove:[3,13],provid:[0,1,3,4,5,6,8,9,10,12,13,15,16,18,20],proxi:[1,13],prune:9,pseudo:16,pseudoinv:5,pseudoinvers:[5,6],pseudorandom:[6,18],psycholog:0,pt:13,ptratio:[],punish:[0,1],pure:[3,9,18],purest:9,puriti:9,purpos:[0,3,10,12,14],put:1,putarow:5,putboundslic:5,putclist:5,putobjsens:5,putqobj:5,py:[0,1,2,3,4,5,6,7,8,11,13,14],pydata:15,pydot:9,pylab:[0,7],pylint:[3,4],pypi:15,pyplot:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,18],pythagora:5,python2:0,python3:[0,1,2,3,4,6,7,8,11,13,14,15],python:[1,2,3,4,5,6,8,11,12,13,14,17],pytorch:[0,15],pywrap_tf:[3,4],q:[5,6,8,11,18],qp:8,qquad:[2,11,13,16],qr:[5,6,16],quad:[1,13,16],quadrat:[0,5,8,9,13],qualit:[4,9,18],qualiti:[0,9,15],quantifi:1,quantil:10,quantit:[0,6,9],quantiti:[0,2,5,6,7,9,10,11,12,14,16,18],quantum:[4,12],quartil:0,quench:5,queri:9,question:[0,5,6,9,11,12,13],qugan:4,quick:[4,18],quick_execut:[3,4],quickli:[1,3,9,11,13],quit:[1,5,6,9,10,12],quot:[4,18],r2:[0,5,6],r2_score:0,r2score:0,r:[0,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,18],r_1:9,r_2:9,r_j:9,r_m:9,rad:0,radial:[0,8,12],radioact:18,radiu:[0,1],rag:2,rain:9,rais:[0,2,13],ramp:1,ran1:18,ran2:18,ran3:18,rand:[0,4,5,6,9,10,13,16,18],rand_max:18,randint:[6,9,13],randn:[0,1,2,5,6,9,11,13],random:[0,1,2,3,4,5,6,8,9,13,14,15,16,17],random_devic:18,random_forest_model:10,random_index:13,random_indic:[1,3],random_st:[0,7,8,9,10,11],randomforestclassifi:10,randomli:[1,6,9,13,14,18],randomnumbergener:18,rang:[0,1,2,3,4,5,6,7,9,10,11,12,13,14,16,18],rangl:[0,6,11,18],rangle_x:18,rank:5,rankdir:4,raphson:[1,8,13],rapidli:0,rare:[1,13],rate:[0,1,2,3,4,8,9,10,12,13],rather:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,16,18],ratio:[4,7,9,10,11],rational:0,ravel:[5,6,7,8,9,10,11,13,16],raw:[3,18],raw_df:[],rbf:[8,11,12],rbf_kernel_svm_clf:8,rbf_pca:11,rc:[0,18],rcond:0,rcparam:[0,1,3,7,8,9,10,18],rd:18,re:[2,4,13],reach:[1,4,5,6,7,9,10,11,12,13,14,18],read:[0,2,3,4,5,6,7,8,11,12,16,17,18,20],read_csv:[0,6,7,9],read_fwf:0,reader:[0,6,16,18],readi:[0,1,5,6,8,10,11,12,16],readili:1,real:[0,1,2,4,7,10,11,12,13,16],real_loss:4,real_output:4,realist:8,realiti:18,realiz:[1,12],realli:[0,1],rearrang:13,reason:[0,1,3,4,10,13,20],reassign:1,rebuild:[],recal:[5,6,9,10,11,12,16,18],recalcul:18,recast:3,receiv:[1,3,10,12,18],recent:[0,2,3,4,5,6,7,8,9,10,13],recept:[3,12],receptive_field:3,recip:[0,6,7,16],reciproc:5,recogn:[0,4,5,10],recognit:[0,1,3,12,17,20],recommend:[0,2,3,4,5,6,8,13,15,16,17,18,20],reconsid:9,reconstruct:11,record:[10,17],recreat:18,rectangl:[9,13],rectangular:5,rectifi:[1,3,12],recur:[0,15],recurr:[0,1,15,17],recurs:[9,15,16],recycl:18,red:[0,3,4,6,8,9],redefin:[0,10],reduc:[1,3,5,6,9,10,11,13],reduct:[0,10,11,15,18],refer:[0,1,2,3,5,6,7,11,12,13,14,16,20],referenc:2,refin:12,refit:6,reflect:[0,1,4,5,18],refresh:[15,17],reg:[10,11],regard:[1,9,13],regardless:12,region:[3,4,6,9,12],regist:[6,18],reglasso:5,regr_1:[0,9],regr_2:[0,9],regr_3:[0,9],regress:[1,8,11,12,15,16,17],regressor:[0,7,10],regridg:[5,6],regular:[0,3,4,5,6,7,9,13],regularis:6,reilli:[0,20],reinforc:[0,8,15],reiter:1,rel:[0,4,6,7,9,12,13,18],relat:[0,1,3,4,5,11,13,14,16,18],relationship:[0,4,9],relativeerror:0,releas:[1,3,4,6,13,15],relev:[0,1,5,7,11,15,17,18],reli:[0,6,8],reliabl:[7,18],relu:[3,4],remain:[1,2,4,6,12,16,18],remaind:18,reman:2,remark:1,rememb:[0,8,13,16],remind:[0,5,11,13,16,18],remov:[0,4,5,6],render:0,reorder:[5,7],reorgan:0,repeat:[0,1,3,4,5,6,9,10,11,13,14,16,18],repeated:0,repeatedli:[6,10,13,18],repet:3,repetit:[6,17],rephras:13,replac:[0,1,3,4,5,6,10,12,13,14],replica:6,repositori:[0,4],repres:[0,1,2,3,4,5,6,7,8,9,10,12,13,18],represent:[0,1,3,6,18],representd:3,reproduc:[0,5,6,9,12,15,18],repuls:0,request:[0,13],requir:[0,1,3,4,5,6,8,9,11,12,13,16],res1:2,res2:2,res3:2,res_analyt:2,res_analytical1:2,res_analytical2:2,res_analytical3:2,resaml:6,resampl:[0,7,10,15,17,18],rescal:[0,11,12],rescu:5,reseach:6,research:[0,4,15,20],resembl:[6,18],reserv:[1,5,6,18],reset:18,reshap:[0,1,2,3,4,6,8,9,10,16],residenti:0,residu:[0,5,13],resiz:5,respect:[0,1,2,3,5,6,7,8,10,11,12,13,14,18],respond:12,respons:[0,7,9,12],rest:[0,5],restat:[0,12],restor:4,restored_discrimin:4,restored_gener:4,restrict:[0,3,9,12],result:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,18],result_typ:[],retail:0,retain:[5,6],return_data:14,return_kwarg:[],return_sequ:4,return_x_i:9,reus:[1,3,6],reveal:[0,12],revers:[1,16],review:[15,16,17],revisit:14,reward:[0,4],rewrit:[0,3,5,6,7,8,10,11,12,13,16,18],rewritten:[2,6,8,10,18],rewrot:13,rf:10,rgb:3,rgoj5yh7evk:15,rh:[5,6],rho:[0,10],rho_1:10,rho_2:10,rho_m:10,rich:0,ride:9,rideclass:9,ridedata:9,ridg:[0,7,11,13,15,17],ridge_sk:6,ridgebeta:5,right:[0,1,2,3,5,6,7,8,9,10,12,13,14,16,18],right_sid:2,rightarrow:[0,1,5,6,8,11,12,13,18],rigor:0,ring:6,rise:0,risk:[0,13],rival:4,river:0,rm:[0,18],rmse:0,rmsporp:13,rmsprop:[1,3,4,13],rnd_clf:10,rnn1:4,rnn2:4,rnn:[4,12,17],rnn_2layer:4,rnn_input:4,rnn_output:4,rnn_train:4,rntrick1:18,rntrick2:18,rntrick3:18,rntrick4:18,ro:[0,13],robert:20,robust:0,robustscal:0,roc:10,role:[0,2,5,6,8,15],roll:6,room:[0,19],root:[0,5,9,13,18],rotat:[1,8,9,10],rotation_matrix:9,roughli:[1,3],round:[0,7,9,13],routin:[13,16],row:[0,1,2,5,6,7,9,11,16],rr:5,rrr:5,rthe:5,rug:13,rule:[0,1,5,6,13],run:[0,1,2,3,4,5,6,8,9,11,13,15],runtim:[1,6,14],runtimewarn:[1,6],rust:[0,15,16],rustad:19,rvert:1,rvert_2:1,s:[0,1,2,3,4,5,6,7,9,11,12,13,15,16,17,19],s_1:6,s_:[3,6],s_i:[6,7],s_j:6,s_k:6,saddl:13,safe:18,sai:[0,1,2,3,4,5,6,7,8,9,10,11,12,16,18],said:[6,9,13],sake:[0,5,7,11],sale:0,same:[0,1,2,3,4,5,6,8,9,11,12,13,14,16,18],samm:10,sampl:[0,1,2,3,4,5,6,7,8,9,10,13,14,15,16],sample_vari:14,sample_weight:[3,4],sampleexptvari:18,sastri:11,satisfactori:0,satisfi:[1,2,3,6,8,13,16,18],satur:[1,6],save:[0,4,6,7,9],save_fig:[0,6,7,9,10],savefig:[0,4,6,7,9,18],savetxt:4,saw:5,scalabl:10,scalar:[2,5,6,10,13],scale:[0,1,3,5,6,7,8,9,10,11,12,13,15,16,19],scale_mean:4,scale_std:4,scalei:[],scaler:[0,7,8,9,10,11],scalex:[],scan:[5,7],scari:5,scatter:[0,1,6,7,8,9,14],scenario:[6,13],schedul:13,scheme:[1,13],schrage:18,scienc:[0,1,10,12,13,15,17,18,20],scientif:[0,15],scientist:0,scikit:[3,5,6,7,8,9,10,13,15,16,17,20],scikitplot:[7,10],scipi:[0,3,5,6,13,15,16],scl:6,score:[0,1,3,6,7,9,10,11,19],scores_kfold:6,scratch:1,sdg:13,seaborn:[0,1,3,6,7],seamless:[0,15],search:[0,1,3,5,9,13],sec:6,second:[0,2,3,4,5,6,7,8,9,11,12,13,14,15,16,18],secondeigvector:11,secondli:12,section:[4,11,16,17,18],sector:0,see:[0,1,2,3,4,5,6,7,8,10,11,12,13,14,15,16,17,18],seed:[0,1,2,3,4,5,6,8,9,11,13,14,18],seed_imag:4,seek:[1,2,8],seem:[1,3,4],seemingli:0,seen:[0,1,3,5,10,12,18],segment:13,seismic:6,seldomli:0,select:[1,5,6,8,9,10,11,17,20],self:[1,3,4,5,18,20],sell:4,semest:[7,17],semi:[8,13],semilogx:6,send:[5,12,13,19],senior:17,sens:[0,4,6,8],sensibl:3,sensit:[0,5,6,9,13],sent:2,sentenc:[4,12],sep:[],separ:[0,1,2,4,6,8,9,12,14,15,18],sequenc:[2,3,4,7,9,10,12,13,15,16,18],sequenti:[1,3,4,10,12,18],seri:[0,1,2,3,4,5,6,10,11,12,13,16,17],serif:[0,7,18],serv:[0,1,2,3,5,7,13,20],session:[1,17],set:[1,4,5,6,7,8,10,11,13,14,15,16,18],set_major_formatt:6,set_major_loc:6,set_stream:5,set_tick:[1,8],set_ticklabel:1,set_titl:[0,1,2,3,7,12,14],set_xlabel:[0,1,2,3,7,12],set_xlim:[7,12],set_xticklabel:1,set_ylabel:[0,1,2,3,7],set_ylim:[7,12],set_ytick:7,set_yticklabel:[1,6],set_zlim:6,seth:4,setiosflag:18,setminu:6,setosa:[8,9],setosa_or_versicolor:8,setp:6,setprecis:18,setse:5,setup:[1,4,5,6,8,15],setw:18,sever:[0,3,5,6,7,8,9,11,12,13,15,16,17,18],sgd:[1,3],sgd_clf:8,sgdclassifi:8,sgdreg:13,sgdregressor:13,sgn:5,sh:[],shallow:13,shape:[0,1,2,3,4,5,6,7,8,9,10,11,13,14,16],share:[1,3],she:7,shift:[1,6,12,18],ship:3,shortcom:13,shorten:4,shorter:18,shortli:16,should:[0,2,3,5,6,8,9,11,12,13,16],should_sync:[3,4],show:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,18],show_shap:4,shown:[0,4,5,7,8,11,12,13,16,18],showpoint:18,shrink:[3,5,6,8,11],shrinkag:[5,6],shrunk:11,shuffl:[0,1,3,4,6,13],side:[0,2,5,8,12,13,16],sigh:15,sigma0:18,sigma1:18,sigma2:18,sigma:[0,1,5,6,7,10,11,12,13,16,18],sigma_0:5,sigma_1:5,sigma_2:5,sigma_:[5,16,18],sigma_fn:[7,12],sigma_i:[0,5],sigma_j:5,sigma_m:[6,18],sigma_n:[11,18],sigma_t:13,sigma_x:18,sigmoid:[1,2,4,7,8,10,12,13],sigmundson:[6,19],sign:[1,2,7,8,10,18],signal:[1,3,10,12],signatur:[3,4],signifi:4,signific:1,significantli:[1,13,18],sigurd:19,sim:[4,5,6,13,18],similar:[0,1,2,3,4,5,6,7,8,9,10,11,13,14,15,16],similarli:[0,1,3,5,8,10,18],simpl:[1,2,3,5,6,7,8,10,11,12,14,15,16],simple_rnn:4,simplepredict:10,simpler:[0,1,5,6,13,15],simplernn:4,simplest:[0,1,3,4,9,10,12,14],simpletre:10,simpli:[0,1,2,4,5,6,8,9,10,11,12,15,16,18],simplic:[2,5,6,7,8,9,10,11,12,14],simplicti:5,simplifi:[0,6,9,15],simplist:[3,6,18],simul:[6,18],simultan:6,sin:[0,1,2,3,4,9,12,13,16],sinc:[0,1,2,3,5,6,7,8,9,10,11,13,16,18,20],sine:[3,12],singl:[0,1,2,3,5,6,7,8,9,12,13,16,18],singular:[0,6,13,16,17],sinusoid:3,site:[0,1,2,3,4,6,7,8,11,13,14,17],situat:[0,4,5,7,13],six:[3,18],size:[0,1,2,3,4,5,6,8,9,10,11,13,16,18],sketch:10,ski:9,skill:0,skip:[3,4,11],skiprow:[],skl:[0,6],sklearn:[0,1,3,5,6,7,8,9,10,11,13,14],skplt:[7,10],sl:6,slack:8,slice:[2,16],slide:[0,3,18],slight:[6,13],slightli:[1,2,3,5,6,7,10,18],slope:[8,11,12],slow:[0,2,8,13],slower:[5,16],slowest:16,slowli:12,slp:1,small:[0,1,2,3,5,6,8,9,10,11,12,13,15,16,18],smaller:[0,1,2,5,6,8,9,11,13,18],smallest:[0,4,14],smallest_row_index:14,smooth:[0,3,6,13],sn:[0,1,3,6,7],sne:11,so:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,18],soar:6,social:0,soft:[1,7,10,12],soften:8,softmax:[3,7],softwar:[0,8,15,16,17],sol:8,sole:[0,6],solid:[0,7],solitem:5,soltyp:5,solut:[0,1,2,3,5,6,8,10,11,13,16,18],solutionsummari:5,soluton:2,solv:[0,1,3,5,6,8,10,11,12,13,16,17],solve_expdec:2,solve_ode_deep_neural_network:2,solve_ode_neural_network:2,solve_pde_deep_neural_network:2,solveod:2,solveode_popul:2,solver:[2,5,7,8,9,10,11,16],some:[0,1,2,3,4,5,6,7,8,9,10,11,12,14,17,18],some_model:6,somehow:4,someth:[0,1,3,4,7,9,11,18],sometim:[0,1,11,12,13,14],soon:16,sophist:0,sopt:13,sort:[5,6,9,11,18],sound:[3,5],sourc:[0,1,3,6,15,16,18],space:[0,1,4,5,8,9,11,12,13,14,18],span:[0,3,5,9,11,16],spare:1,spars:[3,5,6,16],sparse_mtx:16,sparsecategoricalcrossentropi:3,sparsiti:10,spatial:[1,2,3,12],spdiag:5,speak:18,special:[6,7,10,12,13,16,18],specif:[0,1,2,3,4,5,6,7,8,9,11,12,15,16,18],specifi:[0,3,5,6,7,9,11,13,14,18],specifici:[0,10],spectacular:3,spectral:1,speech:[0,1,3,4,12],speed:[1,2,4,13],spend:18,sphere:0,spin:6,spite:0,spline:8,split:[1,3,4,5,6,8,9,10,11,14,18],splite:0,splitter:[1,10],spmatrix:5,spontan:18,spot:3,spread:[0,11,18],springer:20,spuriou:13,sqquar:5,sqrsignal:3,sqrt:[0,3,4,5,6,8,10,11,13,18],squar:[1,2,3,4,7,8,9,11,13,14,15,16,17,18],squarederror:10,squaredeuclidean:14,squash:12,squeez:2,srand:18,srtm:6,srtm_data_norway_1:6,sse4:[],stabil:5,stabl:[0,4,5,6,7,9,11,15],stack:[2,3,4],stacklevel:0,stage:[5,13],stai:[0,2,4,5,11],stand:[0,5,9,12],standard:[0,1,4,5,6,7,8,10,12,16],standard_basi:2,standardscal:[0,6,7,8,9,10,11],stanford:13,start:[0,1,2,3,4,5,6,8,9,10,11,12,13,14,16,17,18],start_box:[2,13],start_nod:[2,13],start_tim:14,startpoint:18,stat:6,state:[1,2,4,5,6,7,8,10,11,12,13,15,18],statement:[0,7,16],statist:[0,1,3,4,7,9,10,11,12,13,14,16,17,20],statu:[0,7,11],stavang:6,std:[0,4,6,18],stdev:18,stdout:5,steep:13,step:[0,1,2,3,4,6,7,9,10,11,12,13,14,16,18],step_fn:[7,12],step_length:13,steps_list:9,steps_per_epoch:[3,4],stereo:3,stian:19,still:[2,3,5,6,11,13,18],stimuli:12,stk2100:20,stk3155:17,stk4021:20,stk4051:20,stk4155:17,stk5000:20,stk:20,stochast:[0,1,5,6,8,11,12,17],stock:4,stoke:12,stone:[0,7],stop:[1,4,7,9,11,13,14,18],storag:5,store:[0,1,2,3,6,11,13,18],storehaug:19,str:[1,3,4],straight:[0,6,8,13],straightforward:[0,2,3,5,6,8,9,10,13,16],strategi:[0,1,9],stratifi:6,streamtyp:5,strength:[0,5,14],stretch:11,strict:[8,13],strictli:[8,13],stride:[4,16],strike:6,string:[1,18],stroke:7,strong:[3,6,9,10,12,16,18],strongli:[0,8,15,16],stronli:0,structur:[0,1,2,3,6,9,10,12,15],stuck:[1,13],student:[0,17,19,20],studi:[0,3,4,5,6,7,8,11,12,13,15,20],studier:[17,20],style:[0,7,9,16],sub:[9,12],subarg:[2,13],subdivid:[0,16],subfield:0,subject:[5,6,8,18],subplot:[0,1,3,4,6,7,8,9,10,13,14],subplots_adjust:[8,18],subprogram:16,subract:0,subroutin:0,subscript:1,subsequ:[1,4,5,6,12,16,18],subset:[1,6,9,12,13,15],subspac:[0,8,11],substanti:[9,10],substep:11,substitut:[3,6,12,16],subsubset:9,subtask:6,subtl:1,subtract:[0,4,5,6,11,13,16,18],subtre:9,subval:[2,13],succeed:[0,4],success:[3,7,9,13,18],successfulli:[4,9],sucess:18,sudo:[0,15],suffer:[0,1,2,5,10],suffici:[1,6,8,11,13],suggest:[1,13,20],suit:[8,12],suitabl:[0,18],sum:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,18],sum_:[0,1,2,3,5,6,7,8,9,10,11,12,13,14,16,18],sum_i:[2,5,6,8,13],sum_j:6,sum_k:[6,8,12,16],sum_m:3,sum_n:3,sum_nx_:3,summar:[5,6,9],summari:[1,3,4,10,17],summat:3,sunni:9,superfici:3,superscript:[1,12],supervis:[0,5,6,7,9,12,15],supplement:7,support:[0,1,9,10,11,13,15,17],suppos:[0,5,6,7,8,10,11,12,13,16],suppress:[5,13],sure:[1,4,6],surf:6,surfac:[0,6],surpass:6,surpris:0,surround:[3,15],survei:[0,5,6],svc:[8,9,10],svd:[0,6,11,17],svdinv:5,svm:[8,9,10,11],svm_clf:[8,10],swap:2,swapax:2,swath:5,sy:[5,13],symbol:[1,5,11,13,15,18],symmeteri:1,symmetr:[0,5,8,11,12,13,16],symmetri:6,sympi:[0,15],synonim:18,syntax:[1,13],syntaxerror:1,syrk:5,system:[0,1,3,4,5,6,7,9,10,12,13,15,16,20],systemat:[4,6,18],t0:[3,6,13],t1:[2,13],t2:2,t3:2,t:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,18,19],t_0:[2,9,13],t_1:13,t_:2,t_b:10,t_i:[1,2,12],t_j:[5,12],t_k:9,tabl:[9,18,19],tabul:0,tackl:4,tag:[2,3,4,5,6,7,12,13,14,16,18],taht:0,tail:18,tailor:[2,8,11],taiwan:0,take:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,15,16,18],taken:[0,1,3,6,10,13,16],tan:3,tangent:[1,4,12,13],tanh:[1,4,7,8,12,13],tape:[3,4],target:[0,1,3,4,5,6,7,8,9,10,11,12,13],target_nam:9,task:[0,1,3,5,6,9,11,12,14],tau:[3,5,18],tax:0,taylor:[2,13],taylornr:13,tc:8,team:1,teaser:0,technic:[0,5,6,13],techniqu:[0,1,8,10,13,15,17,18,20],technolog:[0,1],tek5040:20,tell:[0,4,6,10,11,13,18],temp1:1,temp2:1,temp:1,temperatur:[0,9],temporarili:1,ten:3,tend:[3,5,6,8,9,10,12,13,14],tendenc:0,tension:6,tensor:[3,4],tensorflow:[0,2,4,8,14,15,16,17,20],term1:[5,6,11],term2:[5,6,11],term3:[5,6,11],term4:[5,6,11],term:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,18],termin:[0,4,5,9,10,13],terrain1:6,terrain:6,test:[3,4,5,6,7,8,9,10,13,14,16,18],test_acc:3,test_accuraci:[1,3],test_error:6,test_imag:[3,4],test_ind:6,test_input:4,test_label:[3,4],test_loss:3,test_pr:1,test_predict:1,test_rnn:4,test_scor:[7,10],test_siz:[0,1,3,5,6,10],test_split:9,testerror:[0,6],testi:4,testpredict:4,testx:4,text:[0,1,2,4,5,8,9,11,13,16,17,18,20],textual:9,textur:1,tf:[1,3,4,13,14],tfe_py_execut:[3,4],th:[0,1,2,5,6,7,9,12,13,14,16,18],than:[0,1,2,3,4,5,6,7,9,10,11,12,13,15,18],thank:[4,6],theano:[1,15],thei:[0,1,2,3,4,5,6,7,8,9,11,12,13,16,18],them:[0,1,3,4,6,8,9,10,11,12,13,16],theme:0,themselv:[0,18],thenc:6,theorem:[2,6,7],theoret:[0,4,10],theori:[0,1,3,8,9,12,13,15,17,20],thereaft:[0,5,6,11,12,16],therebi:[0,5,7,11,18],therefor:[0,1,2,3,4,6,7,8,11,13,18],therein:11,thereof:[0,6,13],theta:[1,4,13,18],theta_:[1,13],theta_i:1,theta_k:18,theta_linreg:13,theta_t:13,thi:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,20],thing:[0,1,2,4,5,7,9,18],think:[0,1,3,4,6,9,12,13,14,18],third:[0,3,6,13],thirti:7,those:[3,5,6,8,9,10,11,16,17],though:[1,2,3,4,13,16,18],thought:[6,14,18],thousand:[0,1],thread:[3,4],three:[0,1,3,5,6,8,9,12,16,17,19],threshold:[1,3,9,10,11,12,13],through:[0,1,2,3,4,5,6,8,11,12,13,14,15,16,18],throughout:[0,4,5,14,15,16,18],thu:[0,1,2,5,6,7,8,10,11,12,13,18,19],thumb:[0,6],thursdai:17,tibshirani:[6,17,20],tick_param:6,ticker:[6,13,18],tif:6,tight_layout:[1,7],tightli:11,tild:[0,5,6,11,18],till:[0,4,7,8,9,10,12,16],time:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,17,18],timefunct:18,timeit:4,timer:4,tini:1,tip:3,titl:[0,1,2,3,4,6,7,8,9,10,13,18],tmp:13,tmp_log:[3,4],tn:[2,3],to_categor:[1,3,4],to_categorical_numpi:1,to_numer:[0,6],to_str:18,todai:3,togeth:[0,3,6,8,11],toi:[13,14],told:13,toler:[2,6,14],tolist:4,tomographi:12,too:[0,2,4,5,6,9,11,13,18,20],took:8,tool:[0,1,3,6,13,15,18],toolbox:8,top:[0,3,5,6,9,10,15],top_node_typ:2,top_trac:2,topic:[0,5,6,7,8,15,17,20],topolog:[1,3,12],toposort:2,torkjellsdatt:19,toss:10,total:[0,1,2,3,4,6,7,8,10,11,12,13,14,16,18,19],total_loss:4,totalclustervari:14,totalscatt:14,totalvari:18,toward:[1,2,7,12,13],town:0,tp:4,tpng:9,tqdm:6,trace:[2,13],trace_stack:[2,13],traceback:[0,2,3,4,5,6,7,8,9,10,13],traceback_util:[3,4],tracer:[2,13],track:[3,13,14,16],tract:0,tractabl:0,trade:[5,9],tradeoff:[0,5,17],tradit:[0,1,4,6],train:[2,3,5,6,8,9,10,11,12,13],train_accuraci:[0,1,3],train_dataset:4,train_end:[0,1],train_error:6,train_funct:[3,4],train_imag:[3,4],train_ind:6,train_label:[3,4],train_pr:1,train_siz:[0,1,3],train_step:4,train_test_split:[0,1,3,5,6,7,9,10,11],train_test_split_numpi:[0,1],trainabl:4,trainable_vari:4,trained_model:6,trainerror:0,traini:4,training_checkpoint:4,training_dataset:4,training_gradi:13,training_gradient_fun:13,training_loss:13,trainingerror:6,trainpredict:4,trainscor:4,trainx:4,trait:0,trajectori:4,tran:5,transfer:9,transform:[0,5,6,7,8,9,10,11,12,13,15,16],transit:[6,12],translat:[1,4,6,10],transpos:[1,5,11,16],travers:[0,5],treat:[0,1,3,6,12,13,18],tree:[0,1,6,15,17],tree_clf:[9,10],tree_clf_:9,tree_clf_sr:9,tree_reg1:9,tree_reg2:9,tree_reg:9,trend:18,trevor:20,tri:[2,3,4,9,13],triain:0,trial:[0,2,4,6,13,18],triangl:13,triangular:16,trick:[3,4,8,11,13,18],trickier:18,tridiagon:16,trillion:15,trivial:[0,1,5,11,18],troubl:[0,8,12],truck:3,true_beta:6,true_divid:1,true_fun:6,tucker:8,tumor:[7,9],tumour:7,tunabl:1,tune:[4,9,13,16],tup:[],tupl:[2,13],turn:[0,1,5,6,7,8,9,10,11,12,13,16,18],tutori:[1,4],tv:2,tveito:2,tweak:[1,4,10,18],twice:13,twist:11,twister:18,two:[0,1,2,4,5,6,7,9,10,11,12,13,16,17,18,20],tx:13,tx_1:13,txt:4,ty:13,type:[0,1,3,6,8,10,13,16,18],typeerror:[2,13],typic:[0,1,2,3,4,5,7,9,10,12,13,18],u:[0,2,5,6,10,11,12,16],u_:16,u_i:12,u_m:10,ua:0,ubuntu:[0,15],uci:0,uio:[17,19,20],un:14,unari:16,unary_f:[2,13],unary_oper:[2,13],unary_to_nari:2,unbalanc:[6,9],unbias:[0,5,6,18],uncent:6,uncertainti:[0,5],uncertitud:18,unchang:[1,3],uncorrel:10,undefin:5,under:[0,1,5,6,10,13,15],underdetermin:0,underfit:[1,6],underflowproblem:5,undergo:5,undergradu:17,underli:[0,1,9,13,18],underset:[4,14],understand:[0,1,3,5,6,10,13,14,15],understood:[8,13],undesir:8,undetermin:[5,8],undo:4,unexpect:[6,18],unexpected:18,unfair:6,unfortun:[1,8,9,10],unicode_liter:[8,9],uniform:[0,1,5,6,11,13],uniform_real_distribut:18,uniformli:[13,18],unifrompdf:18,unimport:13,union:[5,6],uniqu:[0,2,6,13,14,16],unique_cluster_label:14,unit:[0,1,3,4,5,10,12,18],unitari:[5,6,16],unitarili:16,uniti:18,univari:18,univers:[0,1,2,13,15,17,19],unix:1,unknow:[0,16],unknown:[0,1,3,4,5,6,8,10,16],unknowwn:12,unlabel:1,unless:[0,3,6,11,13],unlik:[1,3,8,13,18],unnecessarili:9,unord:3,unravel:1,unrol:[3,11],unseen:[0,7,9],unstabl:1,unsupervis:[0,1,4,12,15,17],unsymmetr:16,until:[1,2,4,9,12,13,14,18],untouch:0,unusu:12,up:[1,3,4,5,6,8,10,11,13,14,15,16,17,18],updat:[1,2,10,12,13,14],upload:[15,20],upon:[1,6,11,16],upper:[0,8,9,16],uppercas:[16,18],upsampl:4,upscal:4,us:[4,5,6,8,9,10,11,12,14,16,17,20],usag:[0,8,15,18],usd10000:0,usd:0,use_bia:4,use_multiprocess:[3,4],usecol:0,useless:1,user:[0,1,2,3,4,6,7,8,11,14,15,16],userwarn:[3,4,6,14],usetex:18,usg:6,usr:18,usual:[0,3,4,7,12,13,14],ut:5,util:[0,1,3,4,6,7,10,14],ux:16,v0:18,v1:18,v2:18,v:[2,4,5,6,11,13,15],v_0:11,va:1,val:13,val_accuraci:3,val_loss:4,vale:2,valid:[0,1,4,7,9,10,13,15,17,18],validation_batch_s:[3,4],validation_data:[3,4],validation_freq:[3,4],validation_split:[3,4],validation_step:[3,4],valu:[0,1,2,3,4,6,7,8,9,10,12,13,14,15,16,17],valuat:9,valueerror:[0,2],valy:4,van:0,vandenbergh:[8,13],vandermond:0,vanilla:[0,6,11,14],vanish:[1,4,13,18],var_x:18,varabl:8,varepsilon:[5,6],varepsilon_:[5,6],varepsilon_i:[5,6],vari:[0,1,3,5,6,10],variabl:[0,1,2,3,4,5,6,7,8,10,11,12,13,14,16],varianc:[0,1,5,7,9,10,11,13,14,15,16,17],variance_i:[5,11],variance_x:[5,11],variant:[0,1,6,8,12,13],variat:[3,4,11],varieti:[0,3,12,15],variou:[1,3,5,6,7,8,9,11,12,13,15,16,18],vartempvec:18,varvec:18,varydimens:4,vastli:3,vaue:1,vault:0,vdot:[2,13],vec:[6,18],vector:[0,1,2,3,4,5,6,7,9,10,11,13,14,15,17],vector_mean:14,ventur:[0,8,15],verbos:[1,3,4],veri:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,18,20],verifi:[3,11,16],versatil:8,versicolor:[8,9],version:[0,3,4,10,13,14,15,16,18],versu:1,vert:[0,1,5,6,7,8,9,11,13],vert_1:[5,6],vert_2:[5,6,11],via:[0,5,6,7,8,9,10,11,12,15,16,17,18],vidal:11,video:[0,1,12,15,17],view:[1,3,5,6,12,13,17,18,20],violat:8,virginica:9,viridi:[0,1,2,3],virtual:1,viscos:13,viscou:13,visibledeprecationwarn:2,vision:[0,3],visual:[0,3,11,12,15],visualis:1,viz:[6,8,18],vjp:[2,13],vjp_0:[],vjp_0_fun:[],vjp_1:[],vjp_1_fun:[],vjp_argnum:[],vjpfun:2,vjpmaker:[],vjpnode:[2,13],vmax:[1,6],vmc:18,vmin:[1,6],voic:3,volum:[0,3],vote:10,voting_clf:10,votingclassifi:10,votingsimpl:10,vrtx:17,vs:[0,4,6],vspace:[2,13],vstack:[5,11,16,18],vt:5,w1:8,w2:[8,11],w3:8,w:[0,1,2,3,4,5,6,7,8,10,11,12,13,14,16,18],w_1:[8,16],w_1x_1:8,w_1x_:8,w_2:[8,16],w_2x_2:8,w_2x_:8,w_3:16,w_4:16,w_:[1,12],w_hidden:2,w_i:[1,2,10],w_ix_i:12,w_j:16,w_m:16,w_output:2,w_px_:8,w_px_p:8,wa:[0,1,3,4,5,6,7,10,11,12,13,14,16],wai:[0,1,2,3,4,5,6,7,8,10,11,12,13,14,16,18],walk:9,walker:18,wang:0,want:[0,1,2,3,4,5,6,8,9,10,11,12,13,14,15,18],warn:[1,3,4,8,14],warrant:6,wast:3,watch:[3,4,15],wave:3,wavelet:8,we:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,20],weak:[9,10,14],weather:[1,12],web:[15,17],webpag:17,websit:[6,16,17],wedg:[8,18],wednesdai:17,wee:11,week:[5,6,7],weekli:[15,20],weight:[0,1,2,3,6,7,9,10,12,13,18],weigth:2,welcom:[8,15],well:[0,1,2,3,4,5,6,7,8,9,10,12,13,15,16,17,18,20],went:8,were:[0,1,3,4,5,6,7,8,10,11,12,14,18],wessel:0,westbi:19,westby:19,what:[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17],whatev:3,when:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,18],whenev:[13,18],where:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,18,19],wherea:[6,18],wherein:[1,12],whether:[0,3,5,7,9,18],which:[0,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,19],whichev:[1,3],white:9,who:[0,17],whole:[1,3,4,5,9,11,13],whose:[0,6,10,18],whow:[5,11],why:[0,1,3,6,13],wide:[0,1,3,6,7,12,15,16],widehat:6,width:[0,3,8,9],wieringen:0,win:10,wind:9,wing:19,winther:2,wiothout:6,wiscons:7,wisconsin:10,wisdom:6,wise:[0,1,5,12,13],wish:[0,2,5,7,8,11,13,14,16],with_std:0,wither:6,within:[0,2,3,4,7,9,12,13,14,18,20],withinclust:14,without:[0,1,5,6,8,9,11,12,13],won:0,wonder:8,word:[0,1,3,4,5,6,14,18],work:[0,1,4,6,7,8,9,13,15,17,18],worker:[3,4],workshop:17,world:[0,8],worldwid:0,wors:[0,1,3,4,6],worst:[],worth:9,would:[0,1,3,5,6,7,8,9,10,11,12,13,16,18],wrap:[2,6,16,17],wrap_util:[2,13],wrapper:0,write:[0,1,2,3,5,6,7,8,12,13,16,18],written:[0,2,3,5,11,12,13,15,16,18],wrong:[1,8],wrongli:[10,18],wrote:[5,11],wrt:[2,10,13],wth:10,www:[15,16,17,20],wx_1:8,x0:8,x1:[4,8,9,10,13],x1_exampl:8,x1d:8,x2:[8,9,10,13],x2d:[8,11],x2d_train:11,x2dsl:11,x3:8,x:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,18],x_0:[0,5,11,16],x_1:[0,2,5,6,7,8,9,10,11,13,16,18],x_2:[0,2,5,6,7,8,9,10,11,13,16,18],x_3:[8,16,18],x_4:16,x_:[0,2,3,5,6,8,10,11,13,14,16,18],x_center:11,x_data:1,x_data_ful:1,x_hidden:2,x_i:[0,1,2,5,6,7,8,9,10,11,12,13,14,16,18],x_input:2,x_ix_:0,x_iy_i:8,x_j:[0,2,8,9,12,18],x_jy_j:8,x_k:[12,14,16,18],x_l:18,x_m:[6,12,16,18],x_n:[0,2,3,6,8,11,12,13,16,18],x_new:[9,10],x_offset:6,x_output:2,x_p:[3,7,9],x_poli:9,x_poly10:9,x_pred:4,x_prev:2,x_reduc:11,x_scale:8,x_test:[0,1,3,5,6,7,9,10,11],x_test_own:6,x_test_scal:[0,6,7,9,10,11],x_tot:4,x_train:[0,1,3,4,5,6,7,9,10,11],x_train_mean:6,x_train_own:6,x_train_scal:[0,6,7,9,10,11],x_val:1,xarrai:15,xavier:1,xbnew:13,xcode:[0,15],xdclassiffierconfus:10,xdclassiffierroc:10,xg_clf:10,xgb:10,xgbclassifi:10,xgboost:9,xgboot:10,xgbregressor:10,xgparam:10,xgtree:10,xi:[8,13],xi_1:8,xi_:8,xi_i:8,xk:8,xlabel:[0,1,2,3,4,5,6,7,8,9,10,13,18],xlim:[6,10],xm:9,xmesh:13,xnew:[0,13],xp:18,xpanda:0,xpd:[5,11],xplot:0,xs:9,xscale:0,xsr:9,xt_x:13,xtest:6,xtick:[3,6,8,9],xtrain:6,xu:0,xx:[0,5,16],xy:[0,6,8,16],xytext:8,xz:16,y1:4,y2:4,y3:4,y:[0,1,3,4,5,6,7,8,9,10,11,12,13,14,16,18],y_0:[0,5,11,16],y_1:[0,5,8,9,11,13,16],y_1y_1:8,y_1y_1k:8,y_1y_2:8,y_1y_2k:8,y_1y_n:8,y_1y_nk:8,y_2:[0,5,8,9,11,16],y_2y_1:8,y_2y_1k:8,y_2y_2:8,y_2y_2k:8,y_3:[0,9,16],y_4:16,y_:[0,1,5,6,10,11,16],y_data:[0,1,5,6],y_data_ful:1,y_decis:8,y_fit:0,y_i:[0,1,5,6,7,8,9,10,11,12,13,16],y_if_:10,y_ix_:0,y_ix_i:[7,8,13],y_iy_jk:8,y_j:[6,8,12],y_k:12,y_m:16,y_model:[0,4,5,6],y_n:[8,13],y_ny_1:8,y_ny_1k:8,y_ny_2:8,y_ny_2k:8,y_ny_n:8,y_ny_nk:8,y_offset:6,y_plot:9,y_pred1:9,y_pred2:9,y_pred:[0,1,4,6,7,8,9,10],y_pred_rf:10,y_pred_tre:10,y_proba:[7,10],y_scaler:6,y_test:[0,1,3,4,5,6,7,9,10,11],y_test_onehot:1,y_test_predict:0,y_tot:4,y_train:[0,1,3,4,5,6,7,9,10,11],y_train_mean:6,y_train_onehot:1,y_train_predict:0,y_train_scal:6,y_val:1,ye:[3,6,7],year:[0,15],yet:[0,1,6,8,11,13],yi:13,yield:[0,2,5,6,8,10,12,13,14,16,18],yk:8,ylabel:[0,1,2,3,4,5,6,7,8,9,10,13,18],ylim:[3,6],ym:9,ymesh:13,yn:0,yo:[8,9,10],yoshua:[1,20],you:[0,1,2,3,4,5,6,8,9,10,11,13,15,16,18,20],young:0,your:[1,2,4,5,6,8,11,13,15,16,18],yourself:[11,13],youtub:15,ypred:6,ypredict2:13,ypredict:[0,13],ypredictlasso:5,ypredictol:5,ypredictown:6,ypredictownridg:6,ypredictridg:[5,6],ypredictskl:6,ys:9,ytest:6,ytick:[3,6,8,9],ytild:[0,6],ytildelasso:5,ytildenp:0,ytildeol:5,ytildeownridg:6,ytilderidg:[5,6],ytrain:6,yx:16,yy:16,yz:16,z:[0,1,2,3,4,5,6,7,8,9,11,12,13,16,18],z_0:16,z_1:16,z_2:16,z_:[1,2,12,16],z_c:1,z_h:1,z_hidden:2,z_i:[1,12],z_j:[1,12],z_k:12,z_m:1,z_mod:9,z_o:1,z_output:2,zaman:18,zaxi:6,zero:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,18],zeros_lik:4,zfill:4,zip:[2,4,6,13],zl:5,zm_h:0,zn:0,zone:0,zx:16,zy:16,zz:16},titles:["3. Linear Regression","14. Building a Feed Forward Neural Network","15. Solving Differential Equations with Deep Learning","16. Convolutional Neural Networks","17. Recurrent neural networks: Overarching view","4. Ridge and Lasso Regression","5. Resampling Methods","6. Logistic Regression","8. Support Vector Machines, overarching aims","9. Decision trees, overarching aims","10. Ensemble Methods: From a Single Tree to Many Trees and Extreme Boosting, Meet the Jungle of Methods","11. Basic ideas of the Principal Component Analysis (PCA)","13. Neural networks","7. Optimization, the central part of any Machine Learning algortithm","12. Clustering and Unsupervised Learning","Applied Data Analysis and Machine Learning","2. Linear Algebra, Handling of Arrays and more Python Features","Teaching schedule with links to material","1. Elements of Probability Theory and Statistical Data Analysis","Teachers and Grading","Textbooks"],titleterms:{"1":[],"10":17,"11":17,"12":17,"13":[],"14":17,"15":[],"16":17,"17":17,"18":17,"19":17,"2":[0,17],"20":[],"2021":[],"2022":19,"21":17,"22":17,"23":17,"24":17,"25":17,"26":17,"27":[],"28":17,"29":17,"3":17,"30":17,"31":17,"34":17,"35":17,"36":17,"37":17,"38":17,"39":17,"4":17,"40":17,"41":17,"4155":[],"42":17,"43":17,"44":17,"45":17,"46":17,"47":17,"5":17,"6":[],"7":17,"8":[],"9":17,"case":[8,10,18],"do":1,"final":12,"function":[0,1,6,7,8,10,11,12,13,18],"import":[5,16,18],"new":4,A:[0,1,4,8,9],And:13,Ising:6,The:[0,1,2,3,5,6,7,8,9,11,12,15,18],With:4,activ:[1,12],actual:18,ad:6,adaboost:10,adam:13,adapt:10,adjust:1,adversari:4,again:[3,9],aim:[8,9],algebra:16,algorithm:[9,10,11,12],algortithm:13,all:8,an:[0,4,10],analys:5,analysi:[0,5,6,11,15,18],ani:13,anoth:9,appli:15,approach:[0,8,14],approxim:12,architectur:1,arrai:16,assist:19,august:17,autocorrel:18,autograd:[2,13],automat:13,back:[1,11,12],background:15,bag:10,base:13,basic:[0,5,7,9,10,11,16],batch:1,bay:5,befor:11,better:[8,18],bia:6,binari:1,binomi:18,bird:10,block:18,boost:10,bootstrap:[6,10,18],boston:0,breast:1,bring:12,build:[1,3,9],calcul:18,cancer:[1,7,9,11],cart:9,central:[13,15,18],chain:12,chang:10,chi:0,choos:1,cifar01:3,classic:11,classif:[1,9,10],classifi:8,clip:1,cluster:14,cnn:3,code:[0,1,2,5,9,11,12,14,18],collect:[1,3],compar:[2,10],complex:[0,6],complic:6,compon:11,comput:[9,18],con:9,concept:18,conjug:13,continu:18,convex:[8,13],convolut:[3,12],correl:[11,18],cost:[1,10],cours:[15,20],covari:[5,11,18],cross:6,cumul:18,data:[0,1,3,6,7,9,11,15,18],dataset:[1,3],decai:2,decis:[9,10],decomposit:[5,11,16],deep:[1,2],defin:1,definit:18,degre:0,demonstr:18,dens:0,deriv:[5,12],descent:[2,10,13],detail:3,develop:1,deviat:18,diagon:11,dice:18,differ:8,differenti:[2,13],diffus:2,dimension:[2,3,8],disadvantag:9,discret:18,disguis:18,distribut:[5,18],domain:18,down:1,dropout:1,element:[0,18],elimin:16,ensembl:10,entropi:9,environ:0,equat:[0,2,12],error:[0,10],euler:2,evalu:1,event:18,exampl:[0,1,2,3,4,6,7,8,9,10,18],exercis:[0,6],expect:18,experi:18,explor:0,exponenti:[2,18],extrapol:4,extrem:10,ey:10,fall:19,famili:1,famou:[16,18],featur:[9,16],feed:[1,12],fine:1,first:[4,12],fit:[0,10],forc:3,forest:10,forward:[1,2,12],fourier:3,frank:6,freedom:0,frequentist:0,from:[5,10,12],full:2,further:[3,5],fy:[],gan:4,gaussian:[16,18],gd:13,gener:[4,9,18],geometr:11,gini:9,good:0,grade:19,gradient:[1,2,10,13],growth:2,ha:15,handl:16,hidden:2,hous:0,how:18,hyperparamet:1,hyperplan:8,i:[1,18],id3:9,idea:11,implement:[1,18],implic:5,improv:1,includ:13,increment:11,index:9,inform:19,input:2,instal:15,instructor:19,interpret:[5,11],introduc:11,introduct:[0,6,15,16],invers:[5,16],iter:10,its:18,jackknif:18,jungl:10,kera:[1,3],kernel:[8,11],lagrangian:8,lasso:[5,6],later:5,layer:[1,2,3,12],learn:[0,1,2,11,13,14,15],least:[5,6],level:10,librari:15,likelihood:7,limit:[1,13,18],linear:[0,8,13,16],link:[5,11,17,20],logist:[7,13],lu:16,machin:[0,8,13,15],main:18,make:[0,9,10],mani:[10,12],materi:17,math:5,mathemat:[3,5,8],matric:[5,16],matrix:[1,5,11,12,16,18],matter:0,mean:[0,18],meet:[5,10,18],mercer:8,mersenn:18,method:[6,9,10,13,18],mlp:12,mnist:[3,4],model:[0,1,4,6,12],moment:18,momentum:13,moon:[8,9],more:[3,6,16],multilay:12,multipl:[1,3],multipli:8,name:18,network:[1,2,3,4,7,12],neural:[1,2,3,4,12],non:8,normal:[0,1,18],norwai:[],notat:12,novemb:17,now:[1,9],nuclear:0,nueral:7,number:[0,2,18],numer:[2,18],numpi:16,object:3,observ:18,obtain:11,octob:17,od:2,off:6,ol:[5,6,13],one:[2,12],oper:16,optim:[1,8,13,15],ordinari:[5,6],organ:0,oslo:20,other:[4,9,11,12,16,18],our:[0,4,5,11,13,18],outcom:15,output:[2,18],overarch:[0,4,8,9],overview:10,own:[0,10,11],packag:16,part:[13,15],partial:2,pass:1,pca:11,pdf:18,perceptron:12,perform:[1,9],period:[3,18],perspect:1,point:4,poisson:[2,18],polynomi:3,popul:2,practic:13,pre:[1,3],predict:4,prerequisit:[3,15],princip:11,principl:3,pro:9,probabl:[5,18],problem:[1,2,13],procedur:9,process:[1,3],program:[2,13],project:6,prop:13,propag:[1,12],properti:[5,18],pseudo:18,python:[0,9,15,16,18],quick:8,ran0:18,random:[10,11,18],read:9,real:6,recip:18,recurr:[4,12],reduc:0,reduct:3,reformul:2,regress:[0,5,6,7,9,10,13],regular:1,relev:20,relu:1,remark:3,remind:[6,8],requir:[2,15],resampl:6,rescal:6,resourc:2,revisit:13,ridg:[5,6],rm:13,rng:18,rule:12,s:[8,10,18],sampl:[11,18],schedul:17,schemat:9,scheme:2,scikit:[0,1,11],select:18,semest:19,septemb:17,set:[0,2,3,9,12],sgd:13,should:[1,18],simpl:[0,4,9,13,18],singl:10,singular:[5,11],situat:18,soft:8,softmax:1,solv:2,solver:13,some:[13,16],specifi:2,split:0,squar:[0,5,6,10],standard:[13,18],state:0,statist:[5,6,15,18],steepest:[10,13],stk3155:[],stochast:[13,18],superposit:3,supervis:1,support:8,svd:5,systemat:3,teach:[17,19],teacher:19,techniqu:[6,11],technolog:15,tensorflow:[1,3],test:[0,1],textbook:20,theorem:[5,8,11,12,18],theori:18,three:18,tip:13,togeth:12,top:1,toss:18,toward:11,trade:6,tradeoff:6,train:[0,1,4],transform:3,tree:[9,10],tune:1,two:[3,8,15],type:[2,4,12],uncorrel:18,uniform:18,univers:[12,20],unsupervis:14,up:[0,2,9,12],us:[0,1,2,3,7,13,15,18],valid:6,valu:[5,11,18],variabl:18,varianc:[6,18],variou:0,vector:[8,12,16],view:[0,4,10],visual:[1,9],vs:3,wai:9,wave:2,week:17,weekli:17,what:[0,18],which:[1,18],why:18,wisconsin:7,write:[4,11],xgboost:10,your:[0,10]}}) \ No newline at end of file +Search.setIndex({docnames:["chapter1","chapter10","chapter11","chapter12","chapter13","chapter2","chapter3","chapter4","chapter5","chapter6","chapter7","chapter8","chapter9","chapteroptimization","clustering","intro","linalg","schedule","statistics","teachers","textbooks"],envversion:{"sphinx.domains.c":2,"sphinx.domains.changeset":1,"sphinx.domains.citation":1,"sphinx.domains.cpp":4,"sphinx.domains.index":1,"sphinx.domains.javascript":2,"sphinx.domains.math":2,"sphinx.domains.python":3,"sphinx.domains.rst":2,"sphinx.domains.std":2,"sphinx.ext.intersphinx":1,sphinx:56},filenames:["chapter1.ipynb","chapter10.ipynb","chapter11.ipynb","chapter12.ipynb","chapter13.ipynb","chapter2.ipynb","chapter3.ipynb","chapter4.ipynb","chapter5.ipynb","chapter6.ipynb","chapter7.ipynb","chapter8.ipynb","chapter9.ipynb","chapteroptimization.ipynb","clustering.ipynb","intro.md","linalg.ipynb","schedule.md","statistics.ipynb","teachers.md","textbooks.md"],objects:{},objnames:{},objtypes:{},terms:{"0":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,18,19],"00":[0,1,3,5,6,11],"000":[1,3],"0000":[],"00000":[],"000000":[5,11],"00000000e":5,"0000e":3,"0001":[1,3],"000148":[],"00019998":5,"00024087":5,"00028369228101198006":[],"00029012":5,"0003043256065368937":9,"00031535148309580783":6,"0003153514830958081":[],"0003153514830958126":[],"0003153514830958235":6,"00034944":5,"000417932":2,"00042089":5,"000464088":2,"00050694":5,"00058016":6,"000599":[],"00060705":[],"00060708":6,"00061058":5,"00062582":6,"00062595":[],"00066668":6,"00068734":6,"00068941":6,"00069103":[],"00073541":5,"00076495":6,"00076919":6,"00076925":[],"0007698473260556325":6,"0007698473260556334":[],"0007698473260556339":[],"0007698473260556344":6,"00079123":[],"00079130":6,"00079968":5,"00084705":[],"00084711":6,"00085883":6,"00085884":[],"00087697":6,"00088573":5,"00092645":6,"00092646":[],"00096314":5,"001":[1,2,8,13],"00100519":6,"0010479245355337968":[],"0010479245887943952":6,"00105081":6,"00106677":5,"00107405":6,"00111756":6,"0011526":6,"00115999":5,"0011828302640124":18,"00118504":6,"00118527":[],"001263":[],"00128479":5,"001323":6,"00137823":6,"00137835":[],"00139705":5,"00149311":6,"00149956":6,"00152117":6,"00154733":5,"00156379":[],"00156382":6,"00168251":5,"00174276":6,"00175331":6,"00186347":5,"001988":[],"00198806":[],"00200":8,"00202624":5,"00202756":6,"00217499":6,"00224413":5,"00228741":6,"0023548":6,"002381316302584885":[],"002381316302584886":6,"0023813163025848865":6,"00242999":6,"00243186":6,"0024401":5,"00249435":6,"002526":[],"00266858":2,"00270244":5,"00274989":6,"00275135":[],"00289724":6,"00293838":5,"002948":[],"0030828":[],"003083":[],"00310113":2,"00312361":6,"00315593":6,"003215318065760509":[],"0032153180657605125":6,"00323332":6,"0032542":5,"003301":6,"003395515459204093":6,"0033955154592040944":[],"00353823":5,"0036237":6,"0036367":6,"00369758":6,"003704":[],"003717":[],"003759":[],"003774":[],"003788":[],"0038335":6,"00390021":18,"003909404072811217":6,"003909404072811231":[],"00391839":5,"0039987":6,"004":5,"004091940707753948":6,"0040919407077539514":[],"00410387":6,"00410478":6,"004113634617443116":6,"0041136346174431284":[],"004113634617443131":6,"004113634617443135":[],"00411363461744314":6,"004113634617443141":[],"004155986345861364":6,"004155986345861374":[],"00415763":[],"00424909":2,"00424967":6,"00426027":5,"00433417":11,"00440346":6,"00443743":6,"00445655":11,"004579219539673834":6,"004579219539673836":[],"00458878":6,"004610275230656187":6,"004610275230656294":[],"00462287":6,"00471782":5,"00472199":6,"00472512":6,"0049544":6,"004999999999999984":0,"004999999999999994":[],"00512927":5,"00517114":6,"00526348":6,"0053018":6,"00554552":6,"00556826":6,"0056799":5,"00577441":[],"00579953":6,"00588657":6,"00607783":6,"006162":6,"00617499":5,"00630331":6,"00642221":6,"00660427":6,"00672607":6,"00673393":13,"00673407":6,"00676387":6,"0068011":6,"00683748":5,"00683964":6,"007012403613997257":[],"007024126888936694":[],"0070241268889382116":6,"00719176":6,"00727646693":0,"007315":[],"0074331":5,"00751823":[],"00759119":6,"00784393":6,"00803064":6,"0080866254785146":[],"00813803":6,"00817631":6,"00823002":5,"00827728":6,"00831018":6,"00834567":6,"00848904":6,"0086649156":0,"008675369724975977":5,"008675369724976501":[],"00894639":5,"00905423":6,"009163470508352211":[],"009163470508352218":5,"009164545680330616":6,"00917248":6,"00934499":6,"009450756365829578":[],"0096208":6,"00990475":5,"00992331":6,"00996754":6,"00it":6,"01":[0,1,2,5,6,9,11,13,20],"010018312644139205":6,"010018312644139347":[],"0100706":6,"010331721306655144":6,"01033172130665515":[],"010516485576646513":6,"010516485576652856":[],"01066519":6,"01076611":5,"0110":18,"0110407067093945":[],"01104071":[],"011076219011339788":[],"01107621901137465":6,"011225":2,"0113104":6,"01179792":6,"01191824":5,"012073649439965807":[],"012073649472576395":6,"01219292":6,"01223198":6,"01231917":6,"012633802944855959":18,"01290947":6,"01295356":5,"013121573975499602":[],"013121573975499604":[],"01312157406137079":6,"013121574061370796":6,"01318643":6,"01347916":6,"01348565":6,"01367553":6,"01397146":6,"01405935":6,"01416528":6,"014209325470380275":[],"01433809":5,"014436800088896274":6,"014436800088969727":[],"01449782":6,"01458337":6,"0146081":6,"01463049":6,"015072388895177088":[],"0150723888951771":6,"015072388895177109":[],"015072388895177239":6,"01509543":13,"01530715721129232":[],"01531845":6,"01538461538462":[],"01549377":6,"01558197":5,"016285782696017055":6,"016285782696075054":[],"01633913":6,"01640891":6,"01655318":6,"016587414993037307":[],"016587414993045405":6,"01691985":6,"01708781":6,"01708852":6,"01713366":6,"01724499":5,"01735584819559184":[],"017355848195591845":[],"017355848195593354":6,"01756972":[],"01817152":18,"018232":[],"01831036e":6,"01831130e":[],"01831200e":[],"01831207e":6,"01866537":6,"01873344":11,"01873869":5,"01898855":6,"01905883":6,"01908936":6,"019587":[],"01963611":6,"01969145":6,"019754165271682844":6,"01975416527179247":[],"01975848":6,"02":[0,4,6,7,12],"02024962":6,"02054837e":6,"02068067":6,"02073509":5,"02098261":6,"02123176":6,"021250026482402":11,"02159270458799264":[],"021592704587992645":[],"021592704588021164":6,"021592704588021167":6,"02198702e":6,"022022882954618364":[],"02208512":6,"02228115":6,"02229529":6,"02252765":5,"022556":11,"02255619":11,"02299949826036602":[],"0229994982603662":6,"02348765":6,"02365049":6,"0245528":6,"02492265":5,"02493054":[],"02498832":6,"02503753":6,"02511518":6,"02522069":6,"025709":11,"02586427":6,"0260906":6,"02625193":8,"02625928":13,"026408391362671896":[],"026605727637176654":[],"02660572763718461":6,"026605727637184613":6,"02707227":5,"02723445":6,"027609773491022314":[],"027609773491022387":6,"027609773491022407":6,"02786488":11,"027865":11,"02857":4,"02917662":11,"029177":11,"02926396":[],"029613363972682966":[],"029733":[],"02976145":6,"02994311":5,"02f":6,"03":[1,6],"03032441e":6,"03056589":[],"030566":[],"03076923076924":[],"03077640549":4,"03099776":5,"031":5,"03187339273559067":[],"03196357":6,"03251863":5,"03256632e":1,"03267527":6,"03278964":[],"032790":[],"03279636":6,"0330308045180234":[],"0330308045183163":[],"0330308045183219":6,"0330308045190182":6,"033657685071527485":[],"033657685071527624":6,"03382304823545749":11,"03447512":6,"03562355":6,"03568439":6,"0358909447132981":[],"0359565":5,"03630548":6,"03707133":11,"037559":11,"03755944":11,"03761519":[],"03781367141738885":[],"03781367141738886":[],"037813671417388985":6,"03781367141738899":6,"03794112":16,"03814292":6,"03815288":6,"038300":11,"03850557":11,"038506":11,"039039":5,"03981057":16,"0399676689527968":6,"0399676689527975":6,"04":[1,6,11],"04010697":6,"04063602":6,"041":[],"041148729430502":6,"041148729430523":6,"0411487294305246":[],"0411487294305746":[],"04179719":18,"041797190646905":18,"04220758":6,"04284519":[],"043":[],"04315108":5,"0433816":[],"04346721":5,"04355837":6,"04362":[],"04368707":16,"0437499":2,"04389027":6,"043927769551648":[],"04423486":6,"044334":[],"0444119":18,"044613":6,"04483457":[],"04537385":6,"04543942":6,"04566964":6,"0458":9,"04615384615386":[],"04621521":[],"04648335":5,"0466":[],"04683565":5,"04775904":16,"04784395":6,"04818727730430296":[],"048187277304303056":6,"048653428302840175":16,"04892055":6,"04909093":6,"04912436":6,"0495569966278238":6,"0495569966278269":6,"0495569966278315":[],"04it":[],"05":[1,3,4,6,13],"05009826":6,"05087958":[],"05100875":6,"051649":11,"0517473":5,"05227921801205691":[],"05227921801205692":[],"052279218012057004":6,"05255759":18,"05263":[],"05290417684691035":[],"05302":[],"05364854":8,"05383795":6,"05434571":16,"05447415":6,"054585":[],"054785":[],"054963":[],"05505310046362":[],"05505310046363":2,"055137":[],"055320":[],"05533":[],"055676":[],"05614483":5,"05623":[],"05648":[],"05651951":6,"056528":11,"05667":[],"057088709963182":[],"057163681553428394":6,"05716368155351588":[],"0572":[],"05756733":[],"05785343":6,"05789007":6,"057899":11,"05796251":6,"05807125":6,"05883":[],"05884":[],"059013":5,"059294":11,"059378":11,"059601":5,"059783":[],"05999":[],"06":[6,13],"060080":[],"060183":[],"06020587":6,"06021285":[],"060213":[],"060228":11,"060309":5,"06043581":6,"060507":[],"060655":[],"060781":[],"060841":[],"061025":[],"061048":[],"061149":5,"061321":[],"061390":[],"061509":[],"061604":[],"06160438":[],"061701":[],"061745":[],"061898":[],"061917":[],"061923":11,"06200174":5,"06208238634231944":6,"06208238634231953":[],"062136":5,"062142":[],"062292565":4,"062294":[],"062391":[],"062409":11,"062411":11,"062435":[],"062451":[],"062534":[],"062676":11,"062706":11,"062834":[],"062884":[],"063052":5,"063078":[],"063250":[],"063343":[],"063364":5,"063513":[],"063631":5,"063657":[],"063752":11,"063851":5,"063872":[],"063874":5,"06388888888888888":[],"064052":[],"064108":[],"064110":5,"064184":[],"064274":5,"064384":[],"06444":[],"064469":[],"06453579006728315":[],"06453579006728322":6,"064538":11,"064604":11,"064648":[],"064658":5,"064681":[],"064838":5,"06491736":6,"064918":[],"064931":11,"064964":[],"06497046":[],"065020":5,"065106":[],"065249":11,"065314":11,"065348":5,"06547790180152352":6,"06547790180152353":6,"06547790180152357":[],"06547790180152363":[],"065514":[],"065557":[],"065575":[],"065641":11,"065730":11,"065826":5,"065892":[],"065974":5,"06602663":18,"066110":[],"066194":[],"066309":[],"06637":[],"066449":5,"066558":5,"066609":[],"066670":11,"0666807":2,"066729":11,"066753":[],"066789":5,"066837":11,"066954":[],"067079":[],"067213":5,"067236":[],"067240":5,"06724062":5,"067282":11,"067392":11,"067402":11,"067432":11,"067544":[],"067597":5,"067599":5,"067612":5,"067620":5,"067622":5,"067675":[],"067714":[],"067723":5,"067769":5,"067956":11,"068059":11,"068093":11,"068094":[],"068103":[],"068120":[],"068318":5,"068363":5,"068376":5,"06844519414009438":[],"06844519414009442":6,"06844519414009444":6,"068461":11,"068538":11,"06858699":[],"068608":5,"068642":5,"068659":[],"068735":5,"068806":5,"068809":[],"068831":[],"068937":[],"068965":5,"068991":5,"068992":5,"069085":11,"069091":[],"069181":5,"069275":11,"069408":5,"069409":5,"069441":5,"069455":[],"069584":[],"069672":[],"069685":11,"069754":[],"069796":5,"069847":5,"069877":11,"069884":11,"069888":5,"069912":[],"06it":6,"07":6,"070028":[],"070043":[],"070071":[],"070074":5,"070150":5,"07016":[],"070163":11,"07017":[],"070343":11,"070379":[],"07039":[],"070419":11,"070429":11,"070491":5,"070492":11,"070576":[],"070612":[],"07062318":6,"070630":5,"070635":[],"070889":11,"070911":[],"070950":11,"070972":11,"07099747918547346":[],"071022":11,"071086":[],"071112":5,"071136":5,"07115":[],"07129539":18,"0712953943627344":18,"0713":0,"071387":[],"07145103":11,"071467":5,"0714956":16,"071529":5,"07160048164232467":6,"07160048164248561":[],"071614":5,"071632":5,"071698":[],"071705":[],"071767":5,"071921":5,"071935":[],"07201957":[],"072216":11,"072242":[],"072348":[],"072442":11,"072492":[],"072530":[],"072617":[],"072620":[],"072750":[],"07285":3,"0728785":[],"072928":[],"073016":[],"073062":11,"073136":11,"073225":[],"073230":11,"073280":11,"073362":[],"073368":[],"073423":[],"073436":11,"073449":5,"07345504":[],"073531":[],"073575":11,"073583":[5,11],"073598":11,"073629":11,"073695":[],"073699":5,"073719":5,"073761":11,"073810":11,"073816":5,"073827":11,"073907":[],"073949":5,"074034":5,"074067":11,"074111":11,"07413172":[],"074149":11,"07421084":5,"074237":5,"074336":[],"074375":5,"074488":[],"074530":11,"07456491":5,"074573":11,"074666":[],"074693":5,"07490892":6,"074945":[],"075195":5,"075241":5,"075573":[],"075594":5,"075760":5,"075820":[],"075959":5,"076130":11,"076204":[],"076216":11,"076338":5,"0764924":6,"07656896":[],"076589":5,"076604":11,"076628":5,"07678":[],"076780":11,"076804":11,"076895":11,"076905":11,"076938":11,"076955":5,"077003":5,"077013":11,"077015":11,"077022":[],"077083":11,"077211":11,"077349":11,"07735703":16,"077402":5,"077425":5,"077452":11,"077467":[],"07777777777777778":1,"077793":5,"078040":5,"0782":6,"07820":[],"078253":[],"078254":11,"078269":11,"078299":11,"078314":11,"078436":11,"078548":[],"07864":[],"078667":[],"07871":[],"07878641":16,"078911":[],"078992":[],"07903849":16,"079157":5,"079281":5,"079347":[],"079383":5,"079389":5,"079405":5,"079406":[],"07944154":16,"07968918676726029":[],"0796891867672603":6,"079729":[],"079839":[],"079847":[],"079893":[],"07989327":[],"07999999999998":[],"08":[13,18],"080106":[],"080137":[],"080256":[],"080297":5,"080312":[],"080319":5,"080334":[],"080347":[],"080370":[],"080406":[],"080410":[],"08041015":[],"08043851":5,"080517":[],"080582":11,"080593":11,"080647":11,"080706":11,"080738":11,"080801":[],"080847":5,"08085812":[],"080916":[],"080984":11,"081046":11,"081072":11,"081140":[],"081144":11,"081233":5,"081253":5,"08131003":6,"081388":11,"081431":[],"081483":[],"081533":5,"081540":[],"08156108":6,"081773":[],"082174":[],"082183":[],"082295":11,"0823185":18,"082451":[],"082503":11,"08251519":6,"082536":11,"082896":11,"082900":[],"08299273e":6,"083015":5,"083053":[],"08318298e":1,"083220":[],"083227":[],"083276":11,"08328216846752691":11,"08333333333333333":1,"08336233266":4,"083404":[],"083441":[],"083506":[],"083527":[],"083600":[],"083692":[],"08376632":6,"083766322923899":6,"0837663229239016":[],"0837663229239025":[],"0837663229239043":6,"083799":[],"083829":[],"083977":[],"084000":[],"084101":[],"084184":[],"084224":[],"084226":[],"08426840630693411":[],"08426840630693412":6,"08426840630693413":6,"08428156":16,"084282":[],"084364":[],"084400":[],"084408":[],"084414":5,"084444":[],"084484":11,"084536":[],"084549":[],"08455":[],"084570":[],"084604":[],"084629":[],"08464758160254343":5,"084670":[],"084683":[],"084702":5,"08474":[],"084777":[],"084862":[5,11],"084904":5,"084909":5,"084965":[],"085018":5,"085044":[],"085163":11,"085167":[],"085184":[],"085185":[],"085361":[],"085416":[],"08551306":6,"085764":[],"08576932":6,"085879":11,"08593216":6,"086074":11,"086076":5,"08611111111111111":1,"086112":11,"086303":[],"08630331":[],"086567":[],"08673755293381497":[],"086807":5,"086872":5,"086890":11,"08690":[],"086956":11,"087501":[],"08758":[],"08759":[],"087834":[],"08816688":[],"088176":[],"0881981":5,"088212":5,"088526":11,"088611":[],"088631":[],"08871404":5,"088809":[],"088816":[],"088825":[],"088853":[],"08888888888888889":1,"088926":[],"08902":[],"089059":[],"08917679":5,"089177":5,"089212":[],"089329":[],"089425":[],"089501":11,"089614":[],"0896981":6,"08996":[],"09":1,"090028":[],"090229":[],"09085624":16,"090929":[],"091051":11,"091060":[],"09117221":[],"091224":[],"091236":[],"091340":[],"091349":[],"091363":[],"091414":[],"091416":[],"091426":[],"09149881":[],"091630":[],"09166666666666666":1,"091696":11,"0917":9,"09170751":0,"09172408":6,"091891":[],"092066":16,"09216046":[],"0923":[],"09251":[],"09297039":[],"093247":[],"0934597075922044":[],"0938":[],"094082198961999e":6,"0940821989624095e":[],"094082198966615e":6,"0940821989673748e":[],"09444444444444444":1,"094472507965532":11,"095510":[],"095702":[],"095871":[],"09609807":5,"096173":[],"096472":11,"096623":[],"09672929714683368":[],"097360":[],"09744":[],"09744272":6,"09780":[],"09791":[],"09849763":11,"098498":11,"09856879":16,"09861229":16,"098802859381565":16,"099":[],"09903804":8,"0991919894927399":6,"099191989493334":[],"09951287404314545":1,"0n":0,"0s":[3,4],"0x11da42d90":13,"0x11e031e50":13,"0x7fad10f9a280":[],"0x7fad20f69be0":[],"0x7fd098df3280":[],"0x7fd0a9063be0":[],"1":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,17,18,19,20],"10":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,16,18],"100":[0,1,2,3,4,5,6,7,8,9,10,11,13,14,16,18,19],"1000":[0,1,2,4,5,8,11,13,14,15,18],"10000":[2,5,6,10,11,18],"100000":8,"10001":10,"1001":18,"1002":18,"1003":18,"10030":[],"10044225464078282":[],"1005":18,"10077114273548984":6,"1009":18,"1011":18,"1013":18,"1013904243":18,"10141413e":6,"1015":18,"102":3,"1022233262115424":[],"10222333":[],"1023":18,"10230":[],"1023111":16,"1024":3,"1026":18,"1027":18,"1028":[],"103":1,"1030":18,"10302062":16,"10340":[],"1037":18,"10378326e":1,"1038":18,"10391807":6,"10398646080125035":[],"10398646080125036":6,"10398646080125037":6,"1040":18,"10405456":11,"10430":[],"104411":[],"10455924":[],"1047":18,"10505137":16,"10520":[],"10555555555555556":1,"1056":[],"10589577":5,"106":[],"106095":11,"10620135":16,"108":6,"109":[],"10913":6,"10931453":6,"10960":[],"10th":9,"10x":0,"11":[0,2,3,4,5,6,7,8,9,10,11,12,13,16,18,20],"110":[],"1100":18,"1101":18,"11022302e":5,"11046771":16,"111":[1,7,12],"11100":[],"1122558214":[],"112383":11,"1124":[],"1125":3,"11297834":[],"1136":3,"11362930e":[],"11388888888888889":1,"1139":3,"11390":[],"114":3,"11402309":16,"114437":[],"11462415":5,"11482289e":6,"11499517":[],"11507992e":1,"11547777218875695":6,"11547777218875696":6,"11547777218940905":[],"11547777218940906":[],"115822":6,"11590":[],"11604844":18,"116048442683864":18,"11660":[],"11666666666666667":1,"117":8,"11744554e":6,"11780":[],"11790868":[],"117986":[],"118":2,"1182":[],"1183":[],"118318":[],"1184":4,"11840":[],"1185":[],"1186":[],"11890":[],"119":2,"1199":[],"119936":2,"119999999999976":[],"11it":6,"12":[0,1,2,3,4,5,6,8,9,11,12,13,16,18,20],"120":[2,3],"1203":[],"1203284":8,"1206":8,"12069773":16,"121":[2,8,9,10],"12182967":6,"12196674":[],"122":[2,8,9,10],"12222222222222222":1,"122439":[],"1224392":[],"12288563":5,"122886":5,"123":[],"12318726e":6,"12333649":6,"12345260617257443":9,"123459876":[],"123711":6,"12380":[],"124":0,"12400":[],"125":[],"1250":[],"125000":[],"12552073e":6,"126":[],"1261":[],"12618549":5,"1265":[],"127":4,"1270":[],"127043":[],"1271":6,"1277":6,"127773":[],"12777777777777777":1,"12788968":18,"12790":[],"128":[3,4,13],"128664":6,"12898627064868978":18,"12937662":18,"12945452":[],"1297":[],"1298":9,"12adb44b1c20":7,"12m":3,"13":[0,2,4,5,6,9,11,12,13,16,18],"130":[],"13003291":6,"13055555555555556":1,"131":[],"1314":[],"132":[],"13220608e":6,"13229545716505003":16,"132360":[],"1323603":[],"1326":[],"13280":[],"133":7,"13333333333333333":[],"135":[],"13519106":[],"13535942":6,"136":[],"1360":[],"1361":[],"1362":[],"1363":[],"1364":[],"13646574":5,"13661243e":6,"13679863":6,"1371":6,"13740":[],"137400784702911":[],"137652":11,"1378":[],"1382":[3,4],"1383":[3,4],"1384":[3,4],"1385":[3,4],"1386":[3,4],"13865173":5,"13876586436927824":5,"138775":11,"1388888888888889":1,"13890":[],"1392559581788775e":[],"1392559584983597e":[],"1392559585048734e":6,"13925595925919e":6,"14":[0,2,4,5,6,8,9,10,11,12,13,16,18,20],"140":[],"14021063":6,"1404":[],"14042769":[],"140428":[],"141":2,"14100":[],"1416398":6,"14174745":6,"1418":[],"142":2,"14250":[],"142857":[],"143":[2,7],"1435666":[],"14360598":[],"1437":1,"144":2,"14400":[],"1440501043841336":1,"14440":[],"1446729567":4,"14484695":16,"145":2,"146":[],"147":[],"14710":[],"14722222222222223":1,"147400":11,"14741468":[],"147420":11,"14783702":[],"1479":[],"148":[3,4],"14812206":6,"14818":[],"1484256":[],"148564":6,"14859":6,"148768":[],"149":[3,4],"149213":6,"14932651":[],"14988578":[],"149886":[],"14g":6,"14it":6,"15":[0,2,4,6,7,8,9,12,13,16,18],"150":[3,4,8],"15005476":5,"15098090e":6,"151":[3,4],"15130074e":6,"151986":[],"152":[3,4],"15200":[],"15258907":[],"1527777777777778":1,"153036":[],"153106":[],"1533795":[],"153760":[],"15384615384616":[],"154":[],"15454301":[],"154720":[],"15483121":[],"154911":[],"155":[],"155491":[],"155687":[],"155883":[],"156":[],"1562":[],"15629539":[],"15680777":16,"157":[],"15717291":16,"1575":[],"158":[],"1583767":16,"1586300629904382":[],"1587":[],"15891336":16,"159":[],"1590":[],"15990":[],"15g":6,"15it":[],"16":[1,2,3,4,5,6,8,9,10,13,18],"160":[],"1603":3,"1604":[],"1605":[],"160539":[],"1606":[],"1607":[],"16111111111111112":1,"1612":[],"16211139":5,"16220":[],"16231451":4,"1625":[],"1628":[],"163":[],"1630775253":[],"16342407":5,"16343471":6,"16384":3,"16500":[],"166":[],"16660817":[],"166667":[],"167":[],"16740002":16,"168":[],"16805821e":6,"16807":[],"16827044":[],"16832385":[],"16b8e3cda33a":1,"17":[1,2,4,5,6,8,13,18],"17084902":[],"1709":[],"17174962e":1,"17222222222222222":1,"1726":[],"17300":[],"1731":[],"17339342":[],"17446471":6,"1752":[],"175300":[],"17641709":6,"176880142835407":[],"17777777777777778":1,"17801022":5,"17861098":6,"17917768":5,"17930649":16,"17949575":5,"17953942":11,"1797":[1,3],"18":[2,4,6,7,8,9,10,13,18],"180":[],"18029127":5,"1807":4,"1809":[],"181":[],"1810":4,"1812":[],"18156717":16,"1821":[],"18276924":[],"18327677":[],"18333333333333332":1,"18393678":[],"184":[],"184519":[],"184895":[],"185":[],"1851":[3,4],"1852":[3,4],"18526":[],"185278747229417":[],"1853":[3,4],"1854":[3,4],"1855":[3,4],"186":[],"1860":[],"1861":[],"18611111111111112":1,"18613217e":6,"18660":[],"18673098":11,"18682538":[],"1887":6,"18912963":16,"189496":[],"189622":[],"18it":[],"19":[2,4,6,13,18],"19003":6,"191262820314401":[],"19166666666666668":1,"19207979":5,"19213479":[],"19220":[],"193":[],"19354258":[],"19379506":18,"19394283":[],"194":[],"1940":0,"19404282648955e":6,"194042826653172e":[],"194042826815498e":[],"1940428268204826e":6,"1943":12,"1944":[],"19444444444444445":[],"19466812":18,"1950915150":[],"1954":[],"1956":[],"19569961":6,"1961":[],"1962":[],"1963":[],"1964":[],"1965":[],"1970":16,"1973":9,"197370":11,"19740":[],"1979":6,"19800":[],"1989":[],"199":[],"19937":[],"19994371":6,"1_1":12,"1_2":12,"1_3":12,"1cm":[0,8,10,18],"1d":[1,2,3],"1e":[1,2,3,4,14],"1e10":14,"1e4":6,"1f":1,"1k":16,"1n":0,"1s":3,"1x":0,"2":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,18,20],"20":[0,1,2,4,6,7,8,13,18],"200":[0,2,3,4,8,9,10],"2000":0,"200000":[],"20015436":6,"2004":13,"2006":20,"2010":1,"2011":1,"2014":4,"2015":1,"2016":0,"20174121":16,"2018":[0,6],"2018906331":[],"2019":[],"2021":[6,14],"2022":17,"2027":[],"20277777777777778":1,"203":6,"20320502":[],"20375361":18,"203753611274545":18,"20500":[],"20509391":18,"206":[],"2060":[],"20661216":[],"2069":[],"207545":11,"20756638":[],"20833333333333334":1,"20867052175006306":6,"20867052175109335":[],"20980":[],"21":[0,1,2,4,5,6,7,9,12,13,16],"210340":11,"21058097":5,"21130":[],"2116753732":4,"21169159e":6,"21265216":[],"213":6,"213103":11,"213743":11,"21472683":16,"2147483647":[],"21519063":16,"21596432":6,"216290":11,"216683":11,"21786964":16,"2188":[],"21919813":[],"21944444444444444":[],"2195":3,"21985165":13,"21997099":16,"22":[0,1,2,4,5,6,12,13,16],"22044605e":5,"22094791":[],"221":8,"221180":[],"2216":[],"2218":[],"221805":2,"222400":[],"2225":3,"225":4,"2259440937":[],"226124657522696":[],"22612466":[],"22690428":5,"2284246870217459":6,"22842468702288576":[],"22842468702288582":[],"22847924":5,"2299":3,"22it":6,"23":[1,2,4,6,7,12,13,16],"23002365e":6,"231":[],"23103285":[],"2315033":[],"23167717":5,"2321528":5,"232153":5,"232435":[],"23297056":[],"23333333333333334":1,"233528":5,"234":6,"23522201":[],"2357089093":[],"2360682191515046":[],"2361111111111111":[],"2364":[],"237038":5,"23703839":5,"2379":6,"2397":[],"24":[0,1,2,4,6,13,16],"24140":[],"2416":[],"24175744e":6,"2419":[],"24276315":16,"2430":[],"24390":[],"24444444444444444":1,"244858":[],"24485843":[],"24512498":16,"246":2,"2470":[],"24770094":6,"24829908":5,"24906604e":6,"24924624":18,"2495":[],"25":[2,3,4,5,6,7,8,9,11,13],"250":[2,4,7,9],"2500":[],"25000":0,"250000":[],"250154":[],"2517560119":[],"251879":[],"252436":[],"25259666":16,"2526":3,"252866":[],"25286618":[],"253775":[],"255":3,"255001":[],"256":[2,4],"25617654e":6,"25617657e":[],"25617658e":6,"256962":[],"257":[],"2572":[],"2572495066":[],"2572e3a4b38d":1,"2575":[],"25792767":16,"25845e8df859":6,"259153":11,"2597":[],"25976336":[],"26":[2,4,6,13],"26079358":18,"26153846153846":[],"26186844":16,"2619":[],"26290036":[],"26294938":[],"2629493813057833":[],"26301436":5,"26306244":[],"264":4,"26409315307910025":6,"2640931530791003":[],"2640931530791005":[],"26409315307910053":6,"265":[],"2650":[],"265109911":4,"2654":[],"266":[],"26610075":[],"26666667":13,"26710969":5,"26780278":5,"268":[],"268227":[],"26822717":[],"269":[],"27":[0,1,2,4,6,13],"270":[],"2703":3,"27092897":6,"2750":[],"275341":[],"276263":11,"27692307692308":[],"27700":[],"2772":3,"2775623201":[],"27760":[],"278036":5,"27803645":5,"27n_":18,"28":[1,3,4,6],"28001319":[],"28047021":16,"280647":11,"28067036":16,"28097861":5,"280979":5,"2812":[],"282727":11,"283":[],"2830637392":4,"28336218e":6,"2836":[],"28390":[],"28475098":8,"28590743":[],"2861":18,"28634473":11,"286345":11,"2871":[],"2873":9,"2882":18,"2886":18,"2886847885377843":[],"28868479":[],"28875373":16,"2890":0,"2892":18,"28967287":16,"29":[4,6,7],"29149329":[],"2915":18,"29167186":5,"29229741":13,"2923076923077":[],"29512284":11,"295123":11,"2954":[3,4],"2955":[3,4],"2956":[3,4],"2957":[3,4],"2958":[3,4],"29588901":13,"29592539":[],"296247":[],"297":[],"2971492148":[],"2972":[],"29822833":6,"298273":[],"298375":[],"299267190588216":11,"299748":[],"2_":12,"2_1":12,"2_2":12,"2_3":12,"2_i":12,"2_m":[6,18],"2_t":13,"2_x":18,"2b":18,"2cm":8,"2d":[1,3,11,12,15],"2e":6,"2f":[0,7,9,10,11,12],"2ff97f4bf03b":[],"2g":2,"2g_i":2,"2k":3,"2m":6,"2n":[0,2,3],"2nd":9,"2p":18,"2pt":4,"2s":4,"2x":[0,3,8,13],"2x_ix_jy_iy_j":8,"2x_j":8,"2y_i":10,"2y_j":8,"3":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,18,19],"30":[0,1,4,6,7,10,13],"30000":0,"30010":[],"30119421":8,"3017":[],"3018":[],"3019":[],"301927":[],"3020":[],"3021":[],"30258509":16,"302m":4,"303":6,"3037":[],"3038":[],"30384239":16,"3039":[],"303m":4,"3040":[],"3041":[],"30466214e":6,"304m":4,"30535506":[],"30552077":16,"30559547":[],"305m":4,"3063":[],"306m":4,"3072":3,"30787294":6,"307m":4,"308m":4,"309":0,"30940":[],"30b53504a633":0,"31":[4,6,12,16,18],"310":0,"311":0,"311m":4,"312":0,"3123314713548606":6,"31276579e":6,"312m":4,"313":0,"31318084":5,"314":[],"31457796":5,"315":6,"3155":[5,6],"316":[],"31605061":16,"317":[],"31718909":11,"317367":11,"317m":4,"31866499":13,"31896852":8,"319m":4,"32":[3,4,6,12,13,16,18],"3200":1,"32047562":[],"3208":2,"320m":4,"32149601703519115":6,"3214960170351912":6,"3215":[],"32179365":[],"32309075":18,"32320052":16,"324":2,"32458459":[],"324m":4,"3250":[1,6],"32512":[],"326238":[],"326m":4,"32721178":11,"327212":11,"32816737":18,"32903042":[],"3297":[],"32992274":[],"329923":[],"33":[3,4,12,14,16],"3303":[],"33066907e":5,"3310":[],"33166055e":5,"331939":[],"331m":4,"332331":[],"333":7,"3331":[],"33327369":[],"333274":[],"33333333":13,"3337":[],"33611111111111114":[],"33711888":[],"3384":[],"33900713":[],"33m":4,"34":[4,7,16],"3403":[],"340782":11,"34114547":5,"341m":4,"342680":[],"34305928":[],"3436":0,"3437":0,"3456":[],"34568872":16,"34569596":5,"3457":[],"3458":[],"34585355":16,"3459":[],"3460":[],"346810":[],"3469819128513336":[],"34740615":[],"3498":[],"34it":6,"35":[0,2,4,6,7],"3514":[],"35140":[],"35147135":16,"351636":11,"35182854":5,"35203688":16,"3522":[],"3525":[],"3528556":[],"352856":[],"3529":[],"3530606977":[],"3538":[],"3543":[],"3544313922":[],"3546":[],"35470445e":5,"3548":[],"355":[],"3551":[],"35533773":6,"35533774":[],"3556":[],"3558":[],"3560":[],"356399":[],"3567":[],"3572":[],"3575":[],"357508":[],"3576":[],"3577":[],"35771842":6,"3581341341":4,"3585":[],"3587":[],"35892474":16,"359":5,"3593":[],"3594821":13,"3595":[],"3598":[],"359999999999985":[],"36":[2,4,5,6,7,18],"360":1,"3604":[],"3605":[],"3606":[],"360688":[],"3609":[],"36102113":[],"36117602":[],"3613":[],"3615":[],"361556":[],"3617":[],"3621311":5,"3624":[],"3627":[],"3628":[],"363295916323784e":6,"363295916414895e":[],"3632959170548605e":[],"363295924430451e":6,"363834":11,"36383443":11,"3643":[],"364418e97433":1,"3645":[],"3646":[],"3647":[],"3655":[],"3655222":5,"3659":[],"3668":[],"36689784":[],"366898":[],"3669":[],"367":2,"3672":[],"3673":[],"3674":[],"3679":[],"3684":[],"3687":[],"3688":[],"368m":4,"369139":11,"36it":[],"37":[2,4,6,7],"3704":[],"370782966":4,"3711":[],"3713":[],"3718":[],"3721":[],"3722":[],"3725":[],"3729":[],"37307168":16,"3739":[],"37396662":6,"37415316":16,"374291":5,"37429133":5,"3748":[],"3749":[],"3753":[],"3759":[],"3760":[],"3765":[],"376559":[],"37655936":[],"37692363":18,"37703055":[],"3772":[],"3773":[],"37732":[],"3776":[],"3777801602":[],"3779":[],"37835429e":[],"3784":[],"37900111":6,"3791":[],"3794":[],"379647":11,"37964744":11,"37992857":[],"38":[2,4,7,18],"380":[],"3802":[],"3803":[],"3804":[],"3805":[],"380739":5,"38073947":5,"3811":[],"38135733e":6,"3815":4,"381627865854956":[],"38162787":[],"3817475779":[],"3818":[],"3819":[],"3820":[],"38246359":[],"3827":[],"382951":11,"38295101":11,"3830":4,"3833":[],"38336316":16,"3834":4,"3837":4,"3838":4,"3839":4,"3842":[],"3850":[],"3851":4,"38533184":6,"3854":[],"3858":4,"386":[],"3861":4,"3862":4,"38629436":16,"3864":4,"3867":[],"3869":4,"387":[],"3871":4,"3872":4,"387482":5,"38748219":5,"3876":[],"388":[],"3881":4,"3882":[],"3885":[],"38868469":16,"3888":4,"3889":4,"389":[],"38901478":[],"3891":4,"38916861e":6,"38962192e":6,"3898":[],"39":[0,2,4,13,19],"390":[],"3906":4,"39095416":[],"3914":4,"3915":4,"3916":4,"391602":[],"3917":4,"3922":4,"3925515884752442":[],"3928":4,"3931":[],"39311435":16,"3943":4,"3944":4,"39456996":16,"3950":4,"3955":4,"39560937":16,"39579407":5,"3958":[],"3960":4,"3962":4,"3970":4,"39706038":5,"3975":4,"397700":11,"39789527":16,"3979":4,"3980313467":[],"3983":4,"3994":4,"3996":4,"399836":[],"3d":[2,3,4,6,13],"3f":[1,3,9],"3n":16,"3x":[2,8],"3x_i":2,"3y":8,"4":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,18],"40":[1,4,6,19],"400":4,"4000":20,"4005":4,"4011":4,"40116777":[],"401168":[],"4012":4,"4017":4,"401842":11,"40214433":16,"4022":[],"4027":4,"4030":4,"40340043":[],"4034668048":[],"4038":4,"4039":4,"404":[],"4045":4,"4048":4,"4055":4,"405890":11,"4059":4,"40599799":[],"4068":4,"4069":4,"4075":4,"4077":[],"408":2,"4082":[4,6],"4084":4,"4087":[],"4088":4,"4089":4,"409":2,"40927184e":6,"40a38ad763f1":6,"41":[2,4,16],"410":2,"4100":4,"4107":[],"411":2,"4115":4,"4117":4,"412":2,"4128":4,"4131":4,"41357064":18,"4143":4,"41433969":5,"4144":4,"4146":4,"41511965e":1,"4155":2,"4162706317":[],"4167":4,"4176":4,"4177":4,"418506":11,"4186":4,"41876267":5,"418763":5,"41882036e":6,"41882037e":6,"4192423635":[],"4198":4,"41990268":[],"41it":6,"42":[1,2,4,8,9,10,16],"420":[],"42078103":[],"421":2,"4212":4,"422":2,"4224":4,"423":2,"42394972":16,"424":2,"42441033":5,"42449643":[],"42450":[],"425":2,"4255":[],"4258989918":[],"426":[6,7],"42631342":[],"427017":11,"4275":4,"43":[1,2,4,7,16],"43054282":5,"43135183":16,"43330971e":6,"434932":11,"43493232":11,"435163":[],"43552433":16,"43579948e":6,"436462435":4,"43761347":[],"43766686":11,"4379":4,"438060758":[],"438136":[],"439230":6,"44":[1,2,4,16],"44089210e":5,"4410":[],"4411":[],"442600":11,"443217":[],"444":[],"44970586e":1,"45":[2,4,19],"450":[],"450257":11,"4504":[],"45290829":16,"4543859":18,"4557763":11,"455947":[],"456":[],"456418966187335":[],"457":[2,4],"457770268480242":[],"458027":[],"458078":11,"45960079":5,"46":[2,4,19],"4601":[],"461":[],"461175":16,"462":7,"46383925e":6,"464424":[],"4644244":[],"46508305":16,"46567887":16,"466":[],"46602982":[],"46696223":18,"469":[],"46932688":[],"4694":[],"46984697e":6,"469868":[],"46986815":[],"47":[4,19],"4703":4,"47042744":5,"470714":[],"47075725":6,"47116868e":6,"47125748":5,"47132891":5,"47387858":[],"47400238":[],"47432993":[],"4744":[],"47478057":[],"47485224":18,"475311":[],"47531107":[],"47610036":6,"47815203":11,"47920156":[],"47942814":5,"479465113":4,"48":4,"48089797":[],"48133064":[],"481979":6,"48212873":16,"48257387":19,"483257001":[],"4837":[],"48420165":[],"48476997":11,"48574149":16,"486873":[],"48687342":[],"489":[],"48994188":5,"49":[4,5,6,11,13],"490":[],"491":[],"49152":3,"492":[],"493":[],"4940954":0,"4959161509356135e":6,"495916150936645e":6,"495916150936654e":[],"495916150938325e":[],"49636583":[],"497":[3,4],"498":[3,4],"499":[3,4],"4990":18,"4992":18,"4997":18,"4c4c7f":[9,10],"4d":3,"4f":6,"4y":8,"4y_i":10,"5":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,18],"50":[1,2,3,4,6,7,8,10,13],"500":[1,3,4,6,9,10,13],"500000":[],"50000455":5,"50000553":5,"50000718":5,"50000855":5,"50000969":5,"50001063":5,"50001142":5,"50001207":5,"50001261":5,"50001306":5,"50001343":5,"50001374":5,"500014":5,"50001414":5,"50001422":5,"50001439":5,"50001454":5,"50001466":5,"50001476":5,"50001485":5,"50001492":5,"50001498":5,"50001502":5,"50001506":5,"5000151":5,"50001512":5,"50001515":5,"50001517":5,"50001518":5,"50001519":5,"50001521":5,"50001522":5,"50001523":5,"50001524":5,"50001525":5,"501":[3,4],"50105159":18,"5018":18,"50227564e":6,"50321091":5,"504167":[],"50416731":[],"506":0,"50727059":16,"50769230769231":[],"507d50":[9,10],"50846111e":6,"50846112e":6,"50it":6,"50j":13,"50x10":1,"51":[4,10],"510":1,"51004249":16,"512132":[],"51214899":6,"51265232":16,"51289697":11,"512897":11,"5138":[],"51389553":16,"514219":[],"51561271":16,"517350858882083":18,"5177783846":4,"518030":11,"51803019":11,"519842":[],"52":[3,4,13],"5222222222222223":1,"522758":[],"52307692307693":[],"5260627":[],"526744":11,"52723079":[],"52773051":18,"52799362":11,"528115":11,"52811546":11,"52874252":5,"529":[],"52911941":[],"52945798e":[],"53":[3,4,9],"5303329":11,"5305555555555556":1,"5312":[],"531280":[],"53189647":[],"53312754":16,"53423784":16,"53603432":18,"53611562":[],"53697476":[],"53703498":6,"5378811":11,"53794784":11,"537948":11,"53846153846155":[],"539261":11,"54":[3,4,6,18],"540":[],"54016188":11,"540162":11,"54039921":5,"54041041e":5,"541605":[],"54163136":13,"544439":[],"546166676":[],"54780216":[],"54it":[],"55":[1,3,4],"5501":[3,4,14],"550321":[],"5539":[],"55505907":16,"5555555555555556":1,"55578041":[],"5566":[],"557795":11,"55854694":11,"5594":6,"56":[1,3,4],"56033697":5,"561":[],"5611":[],"5615":[],"56198284":5,"5625":[],"56302854":16,"5639":[],"564":[],"564374":11,"565":[],"56536":0,"566":[],"56636616e":6,"567":[],"568":[],"569":1,"56912044e":6,"56939714":5,"56992937":[],"57":[0,4,8,19],"570":[],"5700":[],"571":5,"571105947979336e":6,"571105947979394e":6,"571105947979395e":[],"571105947979439e":[],"57154252":[],"57174058":[],"57201944e":6,"57266138":[],"57361898":18,"574465":11,"57673618":16,"57it":[],"58":[4,10,19],"583595":[],"58395707":18,"58428804":5,"584804":[],"5888888888888889":1,"589":[],"58986647":[],"59":[2,4],"590":[],"591":[],"591317992":4,"591594":5,"59159438":5,"592":[],"593":[],"5944444444444444":1,"59591979":[],"595920":[],"59592669":[],"59766":5,"597660":5,"59833875":13,"598392":[],"59839245":[],"59955801":13,"5cm":18,"5dd54edf2138":6,"5f":8,"5x":8,"5y":8,"6":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,16,17,18],"60":[1,2,3,4,6],"60000":4,"6019067271":4,"60236938":16,"60293962":5,"60420593":5,"60538875":16,"60619654":6,"6063219":18,"606439":[],"60673226":11,"60685864":6,"60791699":[],"6079169911277265":[],"60943791":16,"61":[2,4,7],"61033524":18,"6111111111111112":1,"612939":[],"613579":[],"61399851":11,"613999":11,"61463451":16,"614808":11,"618":[],"61808351":[],"618982":[],"6199381169260247":[],"62":[2,3,4],"62168613":[],"62364974":[],"62373464":11,"625":7,"626635268":[],"62767384":18,"62894215":5,"629961":[],"63":[0,1,2,3,4,6,7],"6300745149331701":[],"630224":[],"63025821e":6,"63249532e":6,"63270833":[],"63395187":18,"63498144":5,"63675140":[],"6371293350711955":[],"63860687":16,"63901111":[],"63it":[],"64":[1,3,4,7,13,16],"64012627":5,"64293754":18,"64425009":[],"645":[],"64615384615385":[],"646283":11,"647":6,"64742912e":6,"647473":11,"649339":5,"64933923":5,"649382":11,"64x50":1,"65":[1,2,3,4,7,8,9],"65136857":[],"6530742540053943":18,"6562":[],"65626043":5,"65628853":16,"65885453":5,"659306":[],"66":[2,3,4],"66054752":[],"66087937":[],"66204648":6,"66226149":6,"66247212":[],"6628996975186953":[],"66292841":[],"66310422":18,"6638":[],"66410989":[],"66560":[],"66619972":11,"666200":11,"667":[],"668172":[],"668186":[],"66818635":[],"66it":6,"67":[2,4],"67047975e":6,"671089":[],"672721":[],"67314874e":5,"67588315":[],"67591616":18,"67697934":11,"67838309":[],"67882608":[],"68":[2,4],"68192193":5,"68246089":16,"68279358":16,"6834195":18,"68534263e":6,"68542204":5,"68545647e":[],"6869":[],"6887363571":4,"68929213e":6,"68944595":18,"689519":11,"69":[2,4,7,18],"690":[],"690569639355314":[],"69061276":11,"690617":[],"69069n_":18,"692":1,"69295955":[],"69484813e":5,"69493539":16,"69504801":6,"69634577e":6,"69695259":5,"69985355":[],"6999536":11,"69it":[],"6e75736fdab1":6,"6f7a6bd7d79f":6,"6m":[3,4],"6n_":18,"7":[0,1,2,3,4,5,6,7,8,9,11,12,13,16,18,20],"70":[1,2,4,6,7],"70127680":6,"70179437":[],"70205195":16,"70354373":18,"703716d317a7":8,"70434005":18,"7050":[],"70598996":[],"70653767":4,"70710678":5,"70769586":[],"70831425":18,"70832814":5,"70980493":16,"71":[1,2,4],"71131626":16,"7119":[],"712018":11,"7134":[],"71350226":18,"7151":[],"71721168":18,"718697":5,"71869727":5,"72":[2,4],"72174172":11,"722047011333792":5,"72271878e":6,"72347283":18,"7236674":5,"72373129":[],"724":3,"72522848":[],"72859758":5,"7293182":[],"72981762":8,"73":[4,6],"731000":[],"73174557":[],"733096":[],"73484667":[],"735738558766299":[],"73573856":[],"73752910":[],"73it":6,"74":[4,6,13],"740":[],"74081822":8,"74368436":16,"743u":[],"7448615806559786":[],"7469898175164704":18,"74840212":5,"749765":[],"75":[4,5,6,8,11],"7501749450963715":[],"750445":[],"750u":[],"75106135":16,"75118364":[],"751699":11,"75170092":5,"753846153846155":[],"75517445":18,"756352":[],"75838233":[],"75it":[],"76":[2,4,19],"76060096":[],"76066069":16,"763u":[],"7643536":13,"765":7,"76529528":18,"76674796":16,"76802186":18,"76923076923077":[],"76936315":5,"7694444444444445":1,"77":[2,4,13,19],"77034458":[],"770345":[],"77152076":5,"7718":9,"773329728649545":18,"77332973":18,"774300":[],"775":[],"77627886":13,"77636e":13,"77662945":18,"7767978193240488":11,"77714169":8,"7782028952":4,"77865169":[],"78":[2,4],"78011544":[],"78156479e":5,"78184120e":6,"7846153846154":[],"78941903":5,"79":[2,4],"79009329":16,"79093776":18,"79111643":5,"791123":18,"793167":[],"79394867":[],"7939646":[],"794282":11,"79449156":18,"79602861":[],"79754246":[],"797e":6,"7c394b1e8b71":9,"7d7d58":[9,10],"7f2b3a6174c2":13,"7m":4,"8":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,17,18],"80":[0,1,2,4,5,6,8],"800":[4,7],"80315282":5,"803153":5,"80354994":6,"80469739":5,"8055555555555556":1,"80561191":[],"80609615e":6,"80609616e":6,"8064":[],"80734875":[],"807349":[],"80738685":16,"80747253":16,"80842254":11,"808423":11,"80847477e":6,"81":[1,2,4],"81048318e":6,"8108619":[],"81160425":5,"81187794":16,"81333804":6,"81333805":[],"814":[7,11],"815563":[],"81633628":11,"816454":[],"81671821":18,"816847":[],"8175":[],"82":[2,4],"8219235992494145":[],"8219236":[],"82198978":5,"82650876":[],"826509":[],"8265786":5,"82747653":[],"827477":[],"82773778":[],"83":4,"8305555555555556":1,"83512277":5,"836874":5,"83687444":5,"83698677":16,"83999999999999":[],"84":4,"84062065":[],"84228957e":[],"84232163":16,"842436":[],"84292394":[],"842u":[],"84355903e":1,"84443254e":1,"84569271":13,"84780262":6,"84854738":[],"84923989e":6,"84924834":11,"84994524":5,"85":[1,4],"8503720991789538":5,"85091337":[],"85263220":6,"8527777777777777":[],"85278920e":5,"853241":[],"85324115":[],"853u":[],"85601992":5,"85758696":16,"858":[],"8583333333333333":1,"85953586":6,"85959007":[],"86":2,"8608479":[],"860848":[],"861":[],"86117291":5,"86134827":5,"86145244":11,"8638888888888889":1,"86478158":[],"86546962":[],"86623151":13,"86630":[],"8666666666666667":1,"86810":[],"869":0,"86905621":18,"86925797":5,"869258":5,"87":2,"870":0,"8702784034":4,"871":0,"872":0,"8722222222222222":1,"873":0,"87327414":18,"87381451":5,"875":1,"8759":13,"876":6,"87625493":[],"87647541":18,"8768":[],"878297":[],"87836018":18,"87940752":16,"87it":6,"88":[2,13],"88017651":11,"880177":11,"88031314":18,"88046261":5,"8805555555555555":1,"88118231":[],"88168312e":6,"881788":18,"883":[],"88336879":5,"884":[],"88477619":[],"885":[],"88512489":[],"88529063e":6,"8855417382991412":[],"88554174":[],"886":[],"88679947":[],"887":[],"8879":6,"88871662":16,"8888888888888888":1,"8894":6,"89":2,"8916666666666667":[],"89274639":16,"89288636":11,"893489":5,"89348922":5,"89410423":5,"8944444444444445":1,"89550839":16,"89565264":[],"8962476":[],"896248":[],"89873772":16,"89876748":18,"8991514":16,"89944994":5,"899450":5,"8f":6,"8g":6,"8n":16,"8x8":1,"9":[0,1,2,3,4,5,6,7,8,9,11,12,13,14,16,18],"90":[0,1,2,6],"900449":[],"90054363":[],"90075537":5,"9011":6,"90220243":5,"90266948":5,"90268858":[],"9027777777777778":1,"903824":[],"90382431":[],"9040":9,"90475506e":6,"90540529":11,"9055555555555556":1,"9083333333333333":[],"90884627":16,"90895045":16,"90940378":16,"91":19,"910":[],"9111111111111111":1,"91128596":5,"912u":[],"913":[3,4],"91379157":[],"913791573406831":[],"91383439":16,"91396388":18,"914":[3,4],"91416375":[],"91479093":0,"91492986e":6,"915":[3,4],"91549644":[],"916":[3,4],"9166666666666666":[],"917":[3,4],"91760278":5,"918":[],"91812702":5,"918992":[],"919":[],"92":[6,19],"922u":[],"92343595":[],"924018":[],"92477093":[],"924e":6,"925":1,"92507116e":1,"9252772":11,"92578916":5,"92603747":16,"92626212":16,"92754397":[],"927544":[],"9277777777777778":1,"92819235":[],"92857143":7,"93":[],"9305555555555556":1,"930583":11,"931":0,"93155188":5,"93158979":5,"93248252":5,"932483":5,"932u":[],"933":5,"93414191":[],"93492130e":6,"93579127":16,"935u":[],"9361111111111111":[],"937":18,"937082":[],"93799826":5,"938":18,"9387":[],"939":[0,18],"94":[7,18],"9400":[],"941866404575299":[],"94226022e":6,"94284104":5,"94320205":5,"9444444444444444":1,"945":[3,4],"94548496":[],"94591015":16,"946":[3,4],"94639099":11,"94659383":[],"947":[3,4],"9472222222222222":1,"94735055":[],"947903":[],"94790323":[],"948":[3,4],"9482527":5,"949":[3,4],"95":[1,7,11],"95008046":6,"95079764":6,"95231424":5,"9527777777777777":1,"95284275":5,"953065564":[],"95351665":5,"954":18,"9549351910143222":[],"954u":[],"9555555555555556":1,"95569422":18,"955820c21e8b":4,"956563":11,"95684892":5,"95703":13,"95714723":[],"957147232685324":[],"95it":[],"96":[6,7,11],"960":18,"9601304850018328e":6,"960130485007504e":6,"960130485007934e":[],"9601304850213484e":[],"96024953":5,"96032148":16,"96084663":5,"961":18,"962":18,"9637117593816477":6,"9640435":5,"96489434":[],"9649652536":4,"96527903":16,"965548":[],"96606158":16,"96653373":[],"96688672":5,"9674916":5,"967809":11,"96863851":[],"96987657":[],"96992454":[],"97":7,"97005689":5,"97065296":[],"97108e":13,"9722222222222222":1,"97230501":[],"97243128":5,"97300836":5,"97497404e":6,"975":1,"97507735":5,"975510299261579":9,"9760832":[],"97705827":5,"977418":5,"97758848":5,"9777777777777777":1,"977880":[],"97788031":[],"9780387310732":20,"9780387848570":20,"9781492032632":20,"97898392":6,"97926491":5,"98":[0,1,7],"980":[],"98017611":13,"98036405":[],"9805555555555555":1,"98091621":5,"981321":[],"98139097":5,"98266587":16,"98275501":5,"983310":[],"98346748":[],"98399675":[],"98413059":5,"98454786":5,"985":18,"98566191":5,"986":18,"9861111111111112":1,"98680716":5,"98716878":5,"98756882":13,"98794823":16,"9879924":[],"98808176":5,"98822371":6,"9888888888888889":1,"989":18,"9890348":5,"9893149172528393":[],"9893447":5,"9898254753574576":18,"98982548":18,"9898ff":[9,10],"99":[6,7,11,13],"990":[],"99009525":5,"9902552771282336":[],"99049330":6,"99088801":5,"991":18,"99115119":5,"99176998":5,"992":18,"99242921":5,"99265097":5,"993":18,"99316252":5,"99353454":[],"993535":[],"99371056":5,"99389612":5,"993972":[],"99397245":[],"9943201":5,"99435648":18,"9947756":5,"99492986":5,"99519225":13,"99528218":5,"99539415":5,"9955282554647219":[],"99566069":5,"99578809":5,"996":5,"99608161":5,"9963961":5,"99650061":5,"9967458":5,"99700706":5,"99709215":5,"99724883":[],"99729756":5,"99751458":5,"99758326":5,"99775587":5,"99775949":[],"99793613":5,"99799099":5,"99813653":5,"99828624":5,"99829953":[],"9983295":5,"99845267":5,"998577":5,"99861053":5,"99871521":5,"99881845":5,"99884384":5,"99893323":5,"999":[9,18],"99901896":5,"99903755":5,"99911427":5,"99918546":5,"99919837":5,"99926459":5,"9993237":5,"99933188":5,"99938942":5,"9994385":5,"99944272":5,"99949306":5,"99953381":5,"99953475":5,"99957911":5,"99961294":5,"99965056":5,"99967865":5,"99970988":5,"9997332":5,"99975913":5,"99977416":[],"99977849":5,"99980002":5,"9998161":5,"99984732":5,"99987324":5,"99989476":5,"99991263":5,"99992746":5,"99993978":5,"99995":5,"9999555851685968":6,"999955585168597":6,"9999840939906267":[],"9999858320366368":[],"9x":6,"9y":6,"\u00f8yvind":[6,19],"abstract":1,"boolean":4,"break":[0,4,6,11,14],"byte":16,"case":[0,1,2,3,4,5,6,7,11,12,13,14,15,16,17],"catch":0,"char":[],"class":[0,1,3,4,6,7,8,9,11,12,13,18],"const":[],"default":[0,1,2,4,6,7,13,16],"do":[0,2,3,4,5,6,8,9,10,11,12,13,14,16],"ekstr\u00f8m":4,"eng\u00f8i":19,"export":9,"f\u00f8470":19,"final":[0,1,2,3,4,5,6,7,8,9,10,11,13,14,17,18,19],"float":[0,3,4,5,9,11,13,14,16],"function":[2,3,4,5,9,14,15,16],"import":[0,1,2,3,4,6,7,8,9,10,11,12,13,14,18],"int":[0,1,2,3,4,5,6,11,13,14,16,18],"long":[0,1,3,4,12,13],"m\u00f8svatn":6,"new":[0,1,2,3,5,6,7,8,9,10,11,13,14,16],"null":[],"public":[0,15],"return":[0,1,2,3,4,5,6,7,8,9,11,13,14,16,18],"s\u00f8rli":[],"s\u00f8rlie":19,"sch\u00f8yen":[6,19],"short":[4,5],"steinsv\u00e5g":[],"super":[3,5],"switch":0,"throw":[3,6,18],"true":[0,1,2,3,4,5,6,7,8,9,10,12,13,14,16,18],"try":[0,1,2,3,4,5,6,7,8,9,10,11,13,14,15,16,18],"var":[5,6,10,11,18],"while":[0,1,3,4,5,6,7,8,9,11,12,13,18],A:[2,3,5,6,7,10,11,12,13,15,16,17,18,19,20],AND:2,And:[0,3,4,5,6,9,15,18],As:[0,1,2,3,4,5,6,8,10,12,13,16,18],At:[0,4,6,13],BE:0,Be:[2,15],Being:13,But:[0,1,2,3,5,6,9,10,18],By:[0,3,5,6,12,13,16],For:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,20],IF:6,IN:20,If:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,15,16,18],In:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,18,20],Is:11,Ising:[5,12],It:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,18],Its:[1,2,4,11],NO:[7,11],No:[3,4,5,6,7,8,9],Not:[0,1,5,6,17],OR:18,Of:18,On:[0,3,17,18,20],One:[0,1,3,4,5,6,7,8,11,12,13,18],Or:[0,1,6],Such:[6,12,18],That:[0,5,7,10,11,12,14,18],The:[4,10,13,14,16,17,18,19,20],Then:[0,1,5,6,8,9,10,11,12,13,14,16],There:[0,3,4,5,6,8,9,11,12,14,16,17,18,19],These:[0,3,4,5,8,9,10,11,12,13,14,16,18],To:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,16,18],With:[0,5,6,8,9,10,11,12,14,16,18],_0:[5,8,10,11,13],_1:[2,5,6,8,10,11,12,13,14,16],_2:[2,5,8,11,12,13,16],_3:16,_4:16,_9:13,_:[0,1,2,4,5,6,7,8,9,10,11,12,13,16],_________________________________________________________________:4,__call__:[3,4],__class__:10,__doc__:6,__future__:[8,9],__getitem__:[],__init__:[1,3,4,14],__main__:2,__mosek:5,__name__:[2,10],__traceback__:[3,4],_asarrai:[],_auto10:[6,12],_auto12:6,_auto1:[2,3,4,5,6,7,12,13,16,18],_auto2:[2,3,4,5,6,12,13,16,18],_auto3:[3,4,5,6,12,13,16],_auto4:[4,6,12,13,16],_auto5:[4,6,12,13,16],_auto6:[4,6,12,16],_auto7:[4,6,12,16],_auto8:[6,12],_auto9:[6,12],_ax:[],_base:8,_build:[15,20],_build_call_output:[3,4],_c:1,_call:[3,4],_call_flat:[3,4],_check_1d:[],_check_optimize_result:[7,11],_compon:11,_coordinate_desc:6,_copy_docstring_and_deprec:[],_decor:0,_depth:9,_fraction:9,_get_lin:[],_getitem_multilevel:[],_handl:[3,4],_i:[0,1,2,5,6,8,11,12,13],_inference_funct:[3,4],_interpolatefunctionerror:[3,4],_is_primit:[],_j:[0,1,2,3,5,6,8,13],_jit_compil:[3,4],_k:13,_l:12,_lambda:6,_leaf:9,_logist:[7,11],_m:10,_make_vjp:[2,13],_maybe_define_funct:[3,4],_multilayer_perceptron:1,_n:[2,5,8,11,13],_node:[2,9,13],_notokstatusexcept:[3,4],_np:[],_num_output:[3,4],_p:[5,8],_plot_arg:[],_process_traceback_fram:[3,4],_r:[3,4],_ratio:11,_sampl:9,_select_forward_and_backward_funct:[3,4],_split:[6,9],_src:[3,4,14],_stateful_fn:[3,4],_stateless_fn:[3,4],_t:13,_test:6,_trace:[2,13],_valu:[2,13],_varianc:11,_weight:9,a0:3,a0faa0:[9,10],a1:0,a2:0,a3:0,a4:0,a_0:0,a_1a:0,a_2a:0,a_3:0,a_3a:0,a_4:0,a_4a:0,a_:[0,1,16],a_h:1,a_i:[0,1,2,12],a_j:[1,12],a_k:[1,12],a_ndim:2,aaron:20,ab:[0,2,5,13,14],ab_channel:15,abandon:1,abbrevi:17,abid:18,abil:[0,10],abl:[1,4,5,6,7,10,12,13],abort:[],about:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,15,16,20],abov:[0,1,3,4,5,6,7,8,9,10,11,12,13,14,16,18],abovement:6,abscissa:13,absolut:[0,2,5,6,13],acceler:13,accept:[0,3,6,9],access:[0,3,4,11,18],accid:[4,6],accmod:5,accompani:0,accomplish:[8,9,13],accord:[0,1,2,5,6,9,12,13,14,18],accordingli:11,account:[0,3,5,13,18],accumul:[12,18],accur:[0,3,4,6,10,13],accuraci:[0,1,3,4,5,6,7,9,10,11,12],accuracy_scor:[0,1,10],accuracy_score_numpi:1,achiev:[0,1,5,6,8,12,16],aco:18,acquaint:15,acquir:[1,15],acr:0,across:[1,3,6,9,15],act:[1,3,16],action:18,activ:[0,2,3,4,9,17],actual:[0,1,4,5,6,8,11,16,18],ad:[0,1,3,4,5,8,13,16],ada_clf:10,adaboostclassifi:10,adadelta:13,adagrad:13,adam:[1,3,4],adapt:[4,6,13,20],add:[0,1,2,3,4,5,6,8,10,11,12,18],add_lin:[],add_outgrad:2,add_subplot:[1,7,12,14],addendum:5,addit:[0,2,3,5,6,7,8,9,10,12,13,15,16,18,19,20],addition:[12,13],address:[1,9,11,13,20],adjac:[3,12],adjoint:5,adjust:[0,5,12,13],admir:0,advanc:[4,6,12,20],advantag:[1,3,5,6,10,13,16],affect:3,affin:[0,3,8,11],afford:3,aforement:14,african:0,after:[0,1,2,4,5,6,9,11,12,13,15,16,18],afterward:0,ag:[0,7,17],ag_0:2,again:[0,1,4,5,6,7,8,10,11,12,13,18],against:[1,4,7,10],agegroup:7,agegroupmean:7,aggreg:[9,10],agorithm:10,agre:[5,6,18],ahead:9,ai:[0,20],aid:11,aim:[0,1,4,6,7,11,14,15,16],ainv:5,airplan:3,aka:5,al:[0,2,4,17,20],alarm:5,algebra:[0,3,5,13,15,17],algo:[],algorithm:[0,1,2,4,5,6,7,8,13,14,15,16,17,18,20],align:[0,2,5,6,7,8,13,18],all:[0,1,2,3,4,5,6,7,9,10,11,12,13,14,15,16,17,18,19,20],allevi:[1,13],alloc:[3,16],allow:[0,1,2,3,5,6,8,10,13,15,16],almost:[0,1,6,8,11,13,18],alon:[2,9],along:[2,3,4,5,6,9,10,11,15,16],alpha:[0,1,2,3,4,5,6,7,8,9,10,13,14,18],alpha_0:3,alpha_1:3,alpha_2:3,alpha_:10,alpha_i:[3,13],alpha_k:13,alpha_m:10,alpha_n:3,alpha_opt:13,alreadi:[2,3,4,5,6,10,12,15,16,18],also:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,20],alter:1,altern:[0,1,4,5,6,7,8,9,11,13,16],although:[1,5,6,8,10,13],alwai:[0,3,5,6,12,13,18],am:4,ame2016:0,american:0,among:[0,3,5,9,10,12,16],amongst:5,amount:[0,1,3,4,6,8,10,14,15],an:[1,2,3,5,6,7,8,9,11,12,13,14,15,16,18,19,20],an_:18,anaconda3:[],anaconda:[0,1,15],analog:13,analys:6,analysi:[1,3,4,7,14,16,17,20],analyt:[0,2,3,5,6,7,12,13,15],analyz:[0,1,3,4,5,6,18],andrew:1,angl:[0,3,9],anharmon:3,ani:[0,1,2,3,4,5,6,7,8,9,10,12,14,18],anim:[4,12],ann:12,annot:[0,1,3,7,8],anoth:[0,1,3,4,5,6,7,8,10,11,12,13,16,18],anp:2,ans_vspac:2,ansatz:0,answer:[0,1,3,5,6,16],antialias:[2,6],anticip:4,anymor:[1,8],anyon:[4,8],anyth:[1,18],anytim:19,apach:1,apart:[11,13],api:[1,15],appar:2,appear:[0,1,3,13,16,18],append:[1,3,4,8,9,13],appendcon:5,appendvar:5,appli:[0,1,2,3,4,6,7,8,9,10,11,12,13,18,20],applic:[0,1,3,4,5,6,7,9,12,13,16,17,18,20],apply_gradi:4,approach:[1,2,4,5,6,9,10,11,12,13,15,18,20],appropri:[2,6,9,12,13,15,18],approx:[0,2,3,6,10,11,13,18],approxim:[0,1,2,3,4,5,6,7,10,11,13,18],apt:[0,15],aq:18,ar:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20],arang:[1,3,4,6,7,9,10,12],arbitrari:[1,4,6,8,12,13,18],arbitrarili:[0,1,11],arc:6,architectur:[3,4,12,20],area:[0,3,6,20],arg:[0,2,3,4,13],argc:[],argmax:[1,11],argmin:[4,10,14],argnum:[2,13],argnum_0:2,argnum_1:2,argsort:11,argu:[1,13],argument:[0,2,3,5,6,11,12,13],argv:[],argval:2,aris:[0,6,12,13,18],arithmet:[0,13,16],arm:[3,4,6,14],arma:[],armadillo:16,around:[0,1,4,5,6,11,18],arr:2,arrai:[0,1,2,3,4,5,6,7,8,9,11,12,13,14,15,18],arrang:3,arraybox:13,arriv:[0,6,9,11,16,18],arrow:12,arrowprop:8,art3d:13,art:[0,1,15],articl:[0,3,4,6,10],artifici:[0,2,7,12,20],artificialneuron:12,arug:13,arxiv:[3,4],as_fram:[],asarrai:[0,2,6,9],asc:5,ascii:[],ashrafi:19,ask:[5,6,11,12],aspect:[0,6,15],assembl:[0,3],assert:4,assess:[0,6],assici:4,assign:[0,7,8,9,12,13,14,17,20],associ:[0,6,9,12,14,18],assum:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,18],assumpt:[0,3,5,6,9,11,18],ast:[0,5,6],astyp:[4,9,10],asymmetri:0,asymptot:[4,6],async_wait:[3,4],atoi:[],atom:0,attempt:[0,4,6,7,8,10],attend:17,attent:[0,16],attr:[3,4],attract:[0,10],attribut:[0,9],attributeerror:[],audi:0,audio:[3,4],aurelien:[0,17,20],austfjel:6,author:[0,1,10,18],authour:0,auto:[6,9,10,18],autocor:18,autocorrelation_tim:18,autocorrelform:18,autocovari:18,autoencod:[4,15],autoencond:15,autograd:15,autom:[0,15],automac:16,automat:[0,1,2,3,4,11,15,16,17],automobil:3,autonom:[4,20],avail:[0,1,4,6,10,11,15,16,17,20],averag:[0,1,3,6,9,10,13,14,18,19],avg:[],avoid:[0,4,5,6,9,11,13,16],avx2:[],avx:[],awai:[2,3,6],awar:[2,10],award:19,ax:[0,1,2,3,4,6,7,8,9,10,11,12,13,14,16],axes3d:[2,6,13],axes_grid1:6,axessubplot:[],axhlin:8,axi:[0,1,2,3,4,6,7,8,9,10,11,12,13,14,16,18],axiom:5,axlabel:0,axvlin:[4,8],axvspan:4,b1:8,b2:8,b3:8,b:[0,1,2,3,4,5,6,8,9,10,12,13,14,18,19],b_1:[2,12,13],b_2:13,b_5:13,b_:[1,16],b_group:9,b_i:[0,1,2,12],b_ia_:0,b_index:9,b_is_vec:2,b_j:[1,12],b_k:[1,12,13],b_m:12,b_meta:2,b_score:9,b_valu:9,bachelor:17,back:[0,3,4,5,6,8,9,10,16,17,18],backbon:16,backend:[1,4],background:[17,20],backpropag:1,backtrack:9,backup:16,backward:[1,2,4,12,16],backward_pass:2,bad:6,badli:18,bag:[9,15,17],bag_clf:10,baggingboot:10,baggingclassifi:10,baggingtre:10,balanc:6,band:16,bandwidth:16,bar:[0,6,11],barber:20,bare:[4,10],base:[0,1,3,4,5,7,8,9,10,14,15,18,19,20],basi:[5,7,8,10,11,12,13,16],basic:[6,8,12,13,14,15,17,18],batch:[3,4,11,12,13],batch_shap:4,batch_siz:[1,3,4],batchnorm:4,bay:7,bayesian:[5,15,20],bc298b802fe2:6,becaus:[0,1,2,3,4,5,6,8,9,12,13,14],becom:[0,1,2,5,6,7,9,12,13,18],been:[0,1,2,3,4,5,6,11,12,13,15,16],befor:[0,1,2,3,4,5,6,7,8,12,13,14,16,18],beforehand:[0,18],begin:[0,1,2,3,4,5,6,7,8,9,11,12,13,14,16,18],behav:[1,6,13],behavior:[0,1,13],behaviour:12,behind:[0,1,6,8,13],behnoosh:19,being:[0,1,2,3,4,5,7,8,10,11,12,13,18],believ:[9,16],belong:[5,7,8,9,13,14],below:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,16,18],benchmark:10,bendik:[],benefici:[1,13],benefit:[0,1,4,11,13,15],bengio:[1,17,20],benign:[1,7],bennosh:19,besid:[4,5],bessel:5,best:[0,1,2,3,4,5,6,7,8,9,10,12,13,19],beta:[0,1,3,5,6,7,10,11,13],beta_0:[0,1,3,5,6,7,13],beta_0x_:0,beta_1:[0,1,3,5,6,7,10,13],beta_1x_0:0,beta_1x_1:[0,7],beta_1x_2:0,beta_1x_:0,beta_1x_i:[7,13],beta_2:[0,3,13],beta_2x_0:0,beta_2x_1:0,beta_2x_2:[0,7],beta_2x_:0,beta_3:3,beta_:[0,3,6,7,13],beta_i:[0,3,5],beta_j:[0,5,6,13],beta_k:13,beta_linreg:13,beta_m:10,beta_mg_m:10,beta_n:3,beta_p:7,beta_px_p:7,betavalu:5,better:[0,1,2,3,4,6,9,10,11,12,13],between:[0,1,2,3,4,5,6,7,8,9,11,12,13,14,18],beyond:[0,1,5,6,8,13],bf:[13,14,16,18],bgd:13,bia:[0,1,2,3,5,8,9,10,12,13,17],bias:[1,2,3,5,6,9,12],big:[0,1,2,5,6,14],bigger:[1,6],bigr:12,bike:9,bilek:19,billion:[3,12,15],bin:[0,7,18],binari:[0,3,5,7,9,10,12,17],binarycrossentropi:4,bind:0,binomi:[15,18],binsboot:6,bioinformat:0,biolog:[1,12,20],bios1100:15,bird:[0,3],bishop:[17,20],bit:[1,4,16,18],bitwis:18,bivari:2,bk:[0,13],bla:[5,16],black:[8,9,14],block:[6,10,15,16,18],blockingavg:[],blockingstd:[],blockingvar:[],blocksiz:[],blocksizemax:[],blocksizemin:[],blogpost:4,blue:[0,3],bmatrix:[0,1,3,5,7,8,11,13,16],bmi:1,bodi:[0,1,4,12],bold:1,boldfac:[0,5],boldsymbol:[0,1,2,3,5,6,7,8,10,11,13,14],boltzmann:[12,15],book:20,boost:[1,9,15,17],boostrap:10,bootavg:[],bootstd:[],bootstrap:[1,13,15,17],bootvar:[],bootvec:[],boston_dataset:0,bot:8,both:[0,1,4,5,6,8,9,10,13,14,15,16,18,19],bottl:7,bound:[0,5,8,12],boundari:[2,4,8,11,12],boundkei:5,box:[2,4,9],boxed_arg:2,boyd:[8,13],bracket:[4,18],brain:[1,7,12],branch:9,breast:[5,7,11],breviti:13,brew:[0,15],brg:8,briefli:0,bring:[0,5,6,10],broad:[0,3,4],brownle:4,brute:[3,5,11],bs:[8,9,10],buffer_s:4,bui:4,build:[0,4,5,6,10,13,16,18],built:[0,1,3,4,6],bunch:11,busi:0,bx:5,bzl:5,c1:[8,11],c2:[8,11],c95af3df0cdd:3,c:[0,1,2,4,5,6,7,8,9,10,11,12,13,14,15,16,18,19,20],c_0:18,c_1:12,c_2:12,c_3:12,c_4:12,c_:[0,8,9,10,13,18],c_i:[12,13],c_k:18,ca:1,cabc613b8702:13,cach:10,cal:[0,8,10,12,13],calcul:[0,1,2,4,5,6,8,9,10,11,12,13,14,16,18],california:[],call:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,18,20],callabl:[3,4],callback:[3,4],calor:0,cambridg:[13,20],can:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,18,20],cancel:[0,13],cancellation_manag:[3,4],cancer:[5,10],cancerpd:7,candid:[8,9,10],cannot:[0,1,4,5,6,7,8,9,17,18],canopi:[0,15],cap:5,capabl:[0,1,8,13,15],capac:2,capita:0,captur:[4,11,12],captured_input:[3,4],car:[3,4],card:[0,7],cardin:1,care:11,carefulli:13,carlo:[0,6,15,18,20],carri:[2,6,7],cart:10,carvalho:19,casella:20,cast:1,cat:[3,4],categor:[0,1,3,9,11],categori:[0,1,3,7,10,12,14,17],categorical_crossentropi:[1,3],caus:[0,5,6,18],causal:0,causat:0,cax:1,cb:6,cbar:1,cbook:[],cc:[0,1,5,13],ccc:[5,12],cd_fast:6,cdf:18,cdot:[0,2,6,12,13,14,16,18],celebr:13,cell:4,center:[0,1,6,7,8,9,11,14,18],centr:20,central:[0,3,5,6,8,16],centroid:[14,18],centroid_differ:14,centuri:3,certain:[0,3,6,7,9,18],cg:13,cha:0,chain:[1,13,15,18],chanc:[1,5,13,18],chang:[0,1,2,3,4,5,6,8,9,11,12,13,14,16,18],channel:3,chapter:[0,6,10,11,16,17,20],charact:[0,3,5],character:[8,9,10,12,18],characterist:[0,1,3,10,13],charg:0,charl:0,chase:4,chd:7,chddata:7,cheap:5,cheaper:[1,13],check:[0,1,3,4,5,6,11,13,16],checkmark:3,checkpoint:4,checkpoint_dir:4,checkpoint_prefix:4,chemic:[],chen:10,chiaramont:2,choic:[0,1,2,3,4,6,9,12,13,14,16],choleski:[5,16],choos:[2,3,6,9,10,11,13,14],chosen:[0,1,2,6,8,9,10,13,18],chosen_datapoint:1,christian:20,christoph:[17,20],cifar10:3,cifar:3,cin:[],circ:[1,12],circl:[0,8,12],circuit:3,circumfer:9,circumv:[1,5,13],ckpt:4,clariti:18,class_nam:[3,9],class_val:9,class_valu:9,class_weight:[3,4],classic:[7,9,13],classif:[0,3,5,6,7,8,11,12,15,17,20],classifi:[0,1,4,7,9,10,11],classificaton:1,classifii:10,clean:1,clear:[1,5,10,12,13],clearli:[0,3,5,6,7,8,18],clever:[1,10],clf3:0,clf:[0,6,8,9,10],clf_lasso:6,clf_ridg:6,clip:[3,18],close:[0,1,2,4,6,8,9,11,12,13,14,18,20],closer:[3,5,13],closest:[8,11,13,14],closur:15,cloud:15,cluster:[0,1,4,6,11,15,17],cluster_label:14,cm:[1,2,3,6,8,13],cmap:[0,1,2,3,4,6,8,9,10],cmap_arg:6,cmath:[],cmb:17,cmd:9,cmu:[],cn_:18,cnn:[12,17],cnn_kera:3,cntk:15,co:[0,2,3,6,9,13],code:[3,4,6,7,8,13,15,16,17,18,20],coef0:8,coef:0,coef_:[0,5,6,8,9,13],coeff:5,coeffici:[0,3,5,6,7,8,9,13,16],coerc:[0,6],coin:[10,18],coin_toss:10,col:[0,11],colab:15,cold:9,colinear:0,collaps:8,collect:[0,2,6,10,11,15,18,20],collinear:5,color:[0,3,4,6,8,9,10,18],color_channel:3,color_cod:6,colorbar:[1,6],colsample_bytre:10,colsaobject:10,colspec:0,column:[0,1,2,5,6,7,8,9,11,12,16],columntransform:9,com:[3,4,6,14,15,20],combin:[1,2,5,6,7,10,18],come:[0,1,3,4,5,12,13,14,17],command:[0,1],comment:[0,4,5,6],commerci:[0,15],commod:0,common:[0,1,3,5,6,7,9,11,13,14,18],commonli:[0,1,4,6,7,9,13,14],commun:[0,12],commut:3,commutatitav:3,compact:[0,1,3,5,6,7,9,11,12,13,14],compar:[0,3,4,5,6,11,13,16],comparison:[2,4,13],compat:7,compet:0,competit:10,compil:[0,1,3,4,15,16],complet:[0,2,3,4,9,12],completenn:12,complex:[1,5,8,9,11,12,13],complic:[0,1,9,13],compon:[0,1,3,4,5,6,7,9,14,15,17],components_:11,compos:[9,12,14],compphys:[6,15,17,20],compress:0,compris:6,compromis:5,compulsori:15,comput:[0,1,2,3,4,5,6,7,8,10,11,12,13,15,16,17,18,20],computation:[0,3,6,9,13,18],con:5,concaten:[2,4,6,14],concav:[1,13],concentr:[0,10],concept:[0,2,15],conceptu:[12,13],concern:[0,1,4,7],conclud:[0,5,13],conclus:1,cond:2,conda:[0,1,15],condit:[0,2,4,5,6,8,9,11,13,18],conduct:15,condwav:2,coneqp:5,confid:[0,5,6,7,8],configur:3,confirm:[5,12],confus:[5,6,10,16],confusion_matrix:9,congruenti:18,conjug:[4,8],conjugaci:13,conjunct:3,connect:[0,1,3,4,9,11,12,13,16],consequ:[5,6,8,10,12,13],conserv:[5,14],consid:[0,1,2,3,5,6,7,8,9,10,12,13,16,18],consider:[0,1,5,13],consist:[0,1,2,3,4,6,12,13,18],constant:[0,2,4,5,6,8,12,13,18],constitu:0,constitut:[2,6],constrain:[1,3,5,7,11],constraint:[5,6,8,13],construct:[0,1,2,3,5,6,7,8,9,10,11,16,18,20],contact:0,contain:[0,2,3,4,5,6,7,8,9,11,12,13,16,18,20],contemporari:20,content:[1,15,16],context:[3,4,6,10,13],contigu:16,continu:[0,1,2,3,4,5,6,7,8,9,10,12,13,16,18],contour:[9,10,13],contourf:[8,9,10],contrast:[1,4,9,10,12],contribut:[0,3,5,13,18],contributor:0,control:[0,1,3,9,13,15],conv2d:[3,4],conv2dtranspos:4,conv:[3,4],conveni:[0,5,6,12,13,16],convent:12,converg:[1,2,4,5,6,7,8,11,13,14],convergencewarn:[1,6,7,8,11],convert:[0,1,4,5,9,11,13,16],converttomatrix:4,convex:[4,5,7],convinc:13,convolut:[1,4,15,17],cool:[4,9],coolwarm:6,coordin:[5,12,14],coorel:0,copi:[0,1,14],core:[2,3,4,10,13],corel:0,coronari:7,corr:[0,5,7,11],correalt:[11,15],correct:[0,1,2,3,4,5,13,16,18],correctli:[1,2,6,10],correl:[0,1,3,5,6,7,10,12,13,15,18],correlation_matrix:[0,5,7,11],correspond:[0,3,5,6,8,9,11,12,15,16,18],cortex:12,cosin:[3,6],cost:[0,2,3,5,6,7,8,9,12,13],cost_deep_grad:2,cost_funct:2,cost_function_deep:2,cost_function_deep_grad:2,cost_function_grad:2,cost_grad:2,cost_sum:2,costol:13,could:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,16,18],coulomb:0,count:[0,9,17,18,19],countor:13,coupl:[4,5,6],cours:[0,1,3,5,11,17],courvil:[17,20],cout:[],cov:[5,6,11,16,18],cov_xi:[5,11],cov_xx:[5,11],cov_yi:[5,11],covari:[0,7,15,16],covariance_matrix:[5,11,14],cover:[0,5,15,17,20],covert:0,covxi:18,covxx:18,covxz:18,covyi:18,covyz:18,covzz:18,cpu:1,cpu_feature_guard:[],craft:3,creat:[1,2,3,4,5,6,9,10,11,12,13,15],create_biases_and_weight:1,create_convolutional_neural_network_kera:3,create_neural_network_kera:1,create_x:[5,11],credit:[0,7],crim:0,crime:0,criteria:[0,4,9,10,14,18],criterion:[9,10,13],critic:6,cross:[0,1,3,7,9,10,13,15,17,18],cross_entropi:4,cross_val_scor:6,cross_valid:[7,10],crossvalid:6,crucial:[1,18],cs231:3,cs:17,csr_matrix:16,cstdlib:[],csv:[0,4,6,7,9],ctnk:1,ctx:[3,4],cubic:0,cumbersom:5,cumsum:[10,11],cumul:[10,18],cumulative_heads_ratio:10,cup:5,current:[1,2,3,4,6,13,14],curs:0,curv:[6,7,10,12],curvatur:13,custom:[5,6,14],custom_cmap2:[9,10],custom_cmap:[9,10],cutpoint:9,cv:[6,7,10],cvxbook:13,cvxopt:[5,8],cyber:20,cycl:[1,12],d1:5,d2:5,d2_g_t:2,d3:5,d670a873ab0c:5,d985fb40c43d:6,d:[1,2,3,4,5,6,7,8,9,10,11,13,14,16,18,19],d_f:13,d_g_t:2,d_net_out:2,da:3,dagger:[5,16],dai:[1,9,15],dalen:[],damp:3,darget:9,darkr:18,dat:0,dat_id:[0,6,7,9],data1:14,data2:14,data3:14,data4:14,data:[2,4,5,8,10,12,13,14,16,20],data_handl:[3,4],data_id:[0,6,7,9],data_indic:1,data_modul:[],data_path:[0,6,7,9],data_url:[],databas:1,datafil:[0,6,7,9],datafram:[0,4,5,7,9,11],datapoint:[1,5,6,7,11,13],dataset:[0,4,6,7,8,9,10,11,13,14],datatyp:4,date:0,daughter:10,david:20,dbh:1,dbo:1,dcomposit:16,ddot:2,dead:1,deadlin:17,deal:[0,1,3,5,6,8,11,13,14,16,18],debt:7,debug:[5,6],decad:[0,3],decai:[0,13,18],decemb:17,decent:10,decid:[0,2,3,5,6,9],decim:0,decis:[0,1,8,11,15,17,20],decision_funct:8,decision_tre:9,decisiontreeclassifi:[9,10],decisiontreeregressor:[0,9,10],declar:[0,4,16],decompos:[5,6,16],decomposit:[0,6,12,17],decompost:5,deconvolut:3,decorrel:[10,13],decreas:[1,2,4,5,6,10,11,13],deduc:0,deep:[3,7,12,13,15,17,20],deep_neural_network:2,deep_param:2,deep_tree_clf1:9,deep_tree_clf2:9,deep_tree_clf:[9,10],deepen:[5,15],deeper:[0,3,4],deeplearningbook:20,deer:3,def:[0,1,2,3,4,5,6,7,8,9,10,11,13,14,18],def_covari:18,def_funct:[3,4],default_tim:4,defect:5,defici:5,defin:[0,2,3,4,5,6,7,8,9,10,11,12,13,14,16,18],definit:[1,2,5,6,8,10,11,12,13,16,18],defint:18,defun:[3,4],defvjp:2,degre:[3,5,6,8,9,10,11,18],del:1,delet:6,delimit:4,deliv:17,delta:[0,2,3,6,8,12,13,14],delta_0:3,delta_1:3,delta_2:3,delta_3:3,delta_4:3,delta_5:3,delta_:[1,16],delta_h:[0,1],delta_j:[3,12],delta_k:12,delta_l:[1,3],delta_n:[0,3],delug:15,delv:0,demand:13,demonstr:[0,3,5,6,7,11,12,15],den:4,denomin:[1,5],denot:[1,2,6,7,13,18],dens:[1,3,4],dense_1:4,densiti:[0,2,6,18],depart:19,depend:[0,1,2,4,5,6,7,8,11,12,13,15,16,18],depict:18,deploy:[0,15],deprec:[2,3,6,13],depth:[0,3,9,10,16],deriv:[0,1,2,6,7,8,10,11,13,15],derivati:13,descend:[5,9,11],descent:[0,1,3,7,8,12,17],descr:[],describ:[0,2,4,5,6,8,10,11,12,13,16],descript:[0,8,9],design:[0,1,3,4,5,6,7,10,11,12,13],designmatrix:0,desir:[0,2,4,5,13,14],despit:[1,12],destroi:16,det:[5,16],detail:[0,6,11,13,14,16],detect:[3,8,12],determin:[0,2,3,4,5,6,8,9,10,11,12,13,16,18],determinist:[7,13,18],dev:1,develop:[0,3,5,8,10,11,12,15,16,17],deviat:[0,1,2,4,5,6,18],device_nam:[3,4],devis:12,df:[4,8,11,13],di:[0,5],diag:[5,8],diagnost:[1,10],diagon:[0,5,7,13,16,18],diagonaliz:5,diagram:10,diagsvd:6,dice:[6,18],dict:[6,8],dict_kei:[],dictionari:0,did:[0,1,5,6,7,10,11,14],die:1,diff1:2,diff2:2,diff:2,diff_ag:2,diffeent:8,differ:[0,1,2,3,4,5,6,9,10,11,12,13,14,15,16,18,20],differenti:[3,15,16,17],differential_oper:[2,13],difficult:[0,1,6,10,13,18],difficulti:[0,1,13],diffonedim:2,digit:[0,1,3,4,6,17,19],dilemma:13,dilut:1,dim:[4,11,14,16],dimens:[0,1,2,3,4,5,8,11,14,16],dimension:[0,4,5,6,9,11,13,14,15,16],dimensionless:[0,3],diment:16,dimnsion:4,diod:3,direct:[0,1,2,4,11,12,13,14],directli:[1,4,5,6,18],directori:[],disabl:[3,4],disadvantag:0,disappear:[3,6],disc_loss:4,disc_tap:4,discard:[6,11],disciplin:[0,3,12],disclaim:18,discourag:13,discov:0,discover:5,discret:[1,3,5,7,13],discrimin:[4,7,10,11],discriminator_loss:4,discriminator_loss_list:4,discriminator_model:4,discriminator_optim:4,discuss:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,20],diseas:7,disguis:6,disord:[1,7],displai:[0,1,3,4,5,6,7,8,9,10,11,12,14,18],displaystyl:[0,5],displot:[],disregard:0,dissimilar:[11,14],dist:14,distanc:[0,8,9,11,14,18],distance_list:9,distinct:[3,7,8,9,10,14],distinctli:8,distinguish:[0,4,7,8,18],distplot:0,distribut:[0,1,4,6,7,10,11,13,14,15,16],distrubut:[0,15],div:5,dive:[0,8,16],diverg:[1,13],divid:[0,1,3,5,6,8,9,11,12,18],divis:[6,8,9,13,16,18],dna:7,dnn1:4,dnn2_gru2:4,dnn:[0,1,2,4,12],dnn_kera:1,dnn_model:1,dnn_numpi:1,dnn_scikit:[0,1],doc:[6,15,17,20],document:[4,7,11,13],doe:[0,1,2,3,4,5,6,8,10,11,12,13,16,18],doesn:[3,9,12],dog:[1,3,4],domain:[5,8,13],domin:0,don:[0,1,3,5,6,8,11,13,15],done:[0,2,3,4,5,6,9,10,11,13,16],dot:[0,2,3,5,6,7,8,9,10,11,12,13,16,18],doubl:[3,4,16],doubli:1,down:[0,3,6,9,11,12,13],download:[0,1,3,5,6,16,20],downsampl:3,dozen:1,dq:6,drag:13,dramat:11,drastic:4,draw:[4,6,10,13],drawback:[0,1,3,13],drawn:[1,4,6,7,11,18],drive:[3,4],driven:3,drop:[0,1,5,6,11,13,18],dropna:[0,6],dropout:4,ds:5,dt:[2,3,13,18],dtype:[0,1,2,3,4,14,16],dualiti:6,dub:0,due:[1,2,5,6,8,10,12,13],dummi:0,dure:[0,1,3,4,8,9,11,15],dwell:0,dwh:1,dwo:1,dx:[2,3,8,18],dx_1:18,dx_1p:6,dx_2p:6,dx_mp:6,dx_n:18,dxp:6,dy:[1,8,18],dynam:4,dysth:19,dz:8,e:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,18,19],e_:[0,2],each:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19],eager:[3,4],eapprox:0,earli:[1,3,4,13],earlier:[0,5,7,8,9,11,12,13],earthexplor:6,eas:[6,9,14],easi:[0,5,6,7,8,9,10,11,12,13,15,16],easier:[5,6,8,9,13,18],easiest:13,easili:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,16],eastern:19,ebind:0,eblock:9,economi:5,ecosystem:15,ect:17,edg:3,edgecolor:6,edu:13,educ:0,eface79dac2c:10,eff:18,effect:[1,4,10,13,18],effic:1,effici:[0,3,10,13,15,16,18],efron:6,egrad:13,eig:[5,11,13,16,18],eigen:18,eigenpair:[5,11],eigenvalu:[0,5,8,11,13,16],eigenvector:[5,11,13],eight:16,eigval:[16,18],eigvalu:[11,13],eigvec:[16,18],eigvector:[11,13],eispack:16,either:[1,5,6,7,8,9,10,11,13,18],ekstrom:[],elabor:18,elarn:3,electr:[0,3,12],electur:20,eleg:11,element:[1,2,3,4,5,6,7,8,11,12,13,15,16,17,20],elementari:[10,13,16],elementwis:[3,13],elementwise_grad:[2,13],elif:[2,3,4,14],elim:16,elimin:[3,5,8],els:[1,2,3,4,7,9,12,13,16],elu:1,elus:0,email:[17,19],embed:[0,11],embodi:6,emit:18,emner:[17,20],emphas:[0,10,15],emphasi:[0,15,20],empir:[1,11,18],emploi:[0,1,5,6,11,13,18],employ:0,empti:[6,10],emul:12,en:[15,20],enabl:11,enbodi:6,encod:[0,3,5,9,11,14],encompass:[0,18],encount:[0,1,5,6,7,13,18],end:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,18],end_box:[2,13],end_nod:[2,13],end_valu:[2,13],endl:[],endpoint:[3,6],energi:[0,4,6],enet_coordinate_desc:6,enforc:12,eng:20,engin:[0,1,3,4,15],english:20,enorm:3,enough:[0,6,13],ensembl:[1,9,17],ensur:[0,1,2,3,5,6,11,13,18],ensure_initi:[3,4],enter:[5,6],enthought:[0,15],entir:[1,3,7,9,15,18],entiti:[9,12,16],entri:[0,5,8,11,12,16],entropi:[1,3,7,10,13],enumer:[0,1,2,3,4,6,8],env:[0,1,2,3,4,5,6,7,8,11,13,14,18],environ:[2,15,20],eo:[0,6],eol:0,eosfit:0,epoch:[0,1,3,4,12,13],epsilon:[0,5,6,7,13],epsilon_0:0,epsilon_1:0,epsilon_2:0,epsilon_:0,epsilon_i:0,eq:[3,13,14,16,18],eqnarrai:[3,5,6],equal:[0,1,2,3,4,5,6,8,9,11,12,13,14,16,18],equat:[1,3,4,5,6,7,8,9,10,11,13,14,16,17,18],equilibrium:[2,12],equiv:[3,13,16,18],equival:[0,1,5,8,11,13,15,16],erf:18,err:[0,10],err_:6,err_sqr:2,errat:13,errno:[],erron:2,error:[1,2,4,5,6,7,9,11,12,13,15,16,18],error_estimate_corr_tim:18,error_handl:[3,4],error_hidden:1,error_output:1,escap:13,especi:[1,3,9,12,13],essenti:[0,5,6,9,10,12,14,17,18],establish:[0,6,10,11],estim:[0,1,5,6,7,10,11,13,15,18],estimated_mse_fold:6,estimated_mse_kfold:6,estimated_mse_sklearn:6,et:[0,2,4,17,20],eta0:[8,13],eta:[0,1,3,8,12,13],eta_:13,eta_t:13,eta_v:[0,1,3],etc:[0,1,3,5,7,8,9,11,12,13,14,15,16,18],ethic:15,etsim:6,euclidean:[0,14],evalu:[0,2,3,4,5,6,9,13,18],evalut:13,even:[0,1,3,4,5,6,8,9,10,11,12,13,14,15,16,18],evenli:4,event:[3,4,5,7,10,14,18],eventu:[5,6,11,12,13,19],everi:[0,1,2,3,4,5,6,9,10,11,12,13,14,15,18],everyth:[4,12],everywher:[4,13],evolv:0,exact:[0,2,5,11,12,13,16,18],exactli:[0,3,4,6,12,15],examin:6,exampl:[5,11,12,13,15,16,17,18,20],exce:[1,12,13],excel:[0,1,4,5,10,20],except:[3,4,5,6,8,9,16],excess:0,excit:0,exclud:[1,6,12],exclus:[0,1,3,6,18],execut:[2,3,4,5,13],executing_eagerli:[3,4],exemplifi:13,exercis:[5,15,17],exhaust:6,exhibit:[0,5,6,8],exist:[0,1,2,3,5,6,7,8,9,13,16,20],exit:[5,16],exp:[0,1,2,5,6,7,8,10,11,12,13,18],exp_term:1,expand:[5,7,11,13],expans:[0,3,5,8,10,12,13],expect:[0,1,5,6,7,11,12,13,15],expectation_value_of_h_wrt_p:18,expens:[6,10,13],experi:[0,1,6,8,13,15],experiment:[0,3,4,6,9,14,18],experimental_get_tracing_count:[3,4],expert:[1,9],explain:[0,6,9,10,11,13],explained_variance_ratio_:11,explanatori:0,explicit:[0,3,6,13,16],explicitli:[0,4],explod:1,exploit:[0,3,12,13],explor:[1,4,6,8,13,15],expon:1,exponenti:[0,1,5,6,10,13,18],export_graphviz:9,export_text:9,exporttext:9,expos:15,express:[0,2,3,5,6,7,10,12,13,16,18],exptmean:18,exptvari:18,extend:[2,5,7,11,13,15],extens:[0,12,15],extent:[0,1,6,20],extern:[3,6,9],extra:[1,3,5],extract:[0,3,5,6,7,8,11,13,16],extrapol:0,extrem:[0,1,4,5,6,7,8,9,13,16],extremum:13,extrins:11,ey:[0,5,6,13,14,16],f11:0,f12:0,f13:0,f1:13,f1_grad:13,f1d:13,f2:13,f2_grad_x1:13,f2_grad_x1_analyt:13,f2_grad_x2:13,f2_grad_x2_analyt:13,f3:13,f3_grad:13,f3_grad_analyt:13,f4:13,f4_grad:13,f4_grad_analyt:13,f5:13,f5_grad:13,f6:13,f6_for:13,f6_for_grad:13,f6_grad_analyt:13,f6_while:13,f6_while_grad:13,f6d7a289d493:[],f7:13,f7_grad:13,f7_grad_analyt:13,f8:13,f8_grad:13,f9:[0,13],f9_altern:13,f9_alternative_grad:13,f9_grad:13,f:[0,1,2,3,4,5,6,7,8,10,12,13,14,16,18,19],f_0:[3,10],f_1:[10,13],f_2:[12,13],f_3:12,f_:10,f_d:18,f_grad:13,f_grad_analyt:13,f_i:[6,12],f_m:[3,10],f_n:3,f_raw:2,f_vec:2,f_wrap:2,face:13,facecolor:[6,8,18],facil:[0,15],facilit:12,fact:[0,1,3,5,9,11,12,13],factor:[0,1,3,5,6,9,10,11,13,16,18],factori:13,fade:6,fafab0:[9,10],fail:[0,6,7,8,11,13,19],failur:7,fairli:[1,2,18],fake:4,fake_loss:4,fake_output:4,fall:[8,9,17],fals:[0,1,2,3,4,5,6,7,9,10,13,14,16],famili:[0,7,8,18],familiar:[0,3,5,6,8,15,16,18],famou:[6,12],far:[0,3,4,5,6,8,11,12,13,14],fashion:[0,9,10],fast:[1,3,6,10,12,13,15,18],faster:[1,11,13],fastest:[13,16],favor:7,favorit:18,fc:3,fdf8a5d7c717:2,fdfcc778e1f8:4,fe5b9d300cc0:6,featur:[0,1,3,5,6,7,8,10,11,12,13,15,18],feature_nam:[0,1,7,9],feautur:9,fed:1,feed:[0,2,3,11,15,17],feed_forward:1,feed_forward_out:1,feed_forward_train:1,feedback:4,feeddorward:4,feedforward:[1,4,12],feel:[0,5,6,11,13,15,19],feet:0,fetch:6,fetch_california_h:[],fetch_openml:[],few:[1,3,4,5,9,18],fewer:[0,9,11],ffnn:[1,12],field:[0,3,6,12,15],fifth:[0,6],fig:[0,1,2,3,4,6,7,12,13,14],fig_id:[0,6,7,9],figaxi:18,figsiz:[0,1,2,3,4,6,7,8,9,10],figur:[0,1,2,3,4,5,6,7,8,9,10,12,13,14,15],figure_id:[0,6,7,9],figurefil:[0,6,7,9],file:[0,1,2,4,5,6,7,9],file_prefix:4,filenam:[],filenotfounderror:[],fileout:[],filepath_or_buff:0,fill:[5,9],filter:[3,4],filtered_flat_arg:[3,4],filtered_tb:[3,4],financ:0,find:[0,1,2,3,5,6,7,8,9,10,11,12,13,14,15,18],find_top_boxed_arg:2,fine:[0,14],finish:2,finit:[3,5,6,12,13,18],first:[0,1,2,3,5,6,7,8,9,10,11,13,14,16,17,18,20],firsteigvector:11,fit:[1,3,4,5,6,7,8,9,11,12,13,18],fit_beta:6,fit_intercept:[0,5,6],fit_mod:9,fit_transform:[0,6,8,9,11],fiti:0,five:[0,9],fix:[0,3,4,6,10,11,12,13],fixedformatt:6,fixedloc:6,fkkt:5,flag:4,flat:[12,13],flatten:[1,3,4,5,16],flexibl:[1,6,8,10,12],float32:[4,9],float64:[4,16],flop:[5,16],flow:[1,4,12],fluctuat:5,fly:11,flyvbjerg:[],fm:0,fma:[],fmax:3,fmesh:13,fn:[3,4],focu:[0,3,4,5,6,15,20],focus:[1,6,7,16],fold:[6,9],folder:[0,4,6],follow:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20],font:[0,7,18],fontdict:18,fontsiz:[1,6,8,9,10,18],fontweight:1,footprint:3,foral:8,forc:[0,5,6,10,11],forcast:4,forecast:[4,12],forelesningsvideo:17,forest:[0,1,9,15,17],forget:11,form:[0,3,4,5,6,7,8,9,11,12,13,15,16,18],formal:[3,4,14,18],format:[0,1,2,3,4,6,7,8,9,10,11,15,18,20],format_data:4,formatstrformatt:[6,13],formul:[4,6,11,14],formula:[3,13,18],forth:[4,12],fortran2003:15,fortran90:18,fortran:[0,15,16],fortun:[0,11],forward:[0,3,6,15,16,17],forward_backward:[3,4],found:[1,2,4,5,6,12,13],foundat:15,four:[4,5,6,8,12,16,17],fourier:0,fourierdef1:3,fourierdef2:3,fourierseriessign:3,fourth:12,fr:5,frac:[0,1,2,3,5,6,7,8,9,10,11,12,13,14,16,18],fractal:[],fraction:9,frame:7,framework:[1,8,10,18],frank:[5,11],frankefunct:[5,6,11],free:[0,6,11,13,15,16,18,19,20],freecodecamp:15,freedom:5,freeli:0,frequenc:[3,6,7,18],frequent:[0,8,9,13],frequentist:15,fresh:10,frida:19,fridai:17,friedman:[6,17,20],friendli:4,frog:3,from:[0,1,2,3,4,6,7,8,9,11,13,14,15,16,17,18,19,20],from_cod:9,from_logit:[3,4],from_tensor_slic:4,fromnumer:2,front:[0,4,5],fstream:[],fulfil:[2,5,12],full:[0,1,3,5,7,9,10,13,18],full_matric:5,fulli:[3,6,12,17,18],fun:[2,13,15],fun_nam:[],func:[0,2],functionali:11,fundament:[0,6,15],funtion:2,further:[2,9],furthermor:[0,3,5,6,7,11,12,13,15],futur:[0,4,8,9],futurewarn:[],fx:5,fy:[17,19],g0:2,g42mrgv128v34gnnhxwk9nrc0000gp:[],g:[0,1,2,3,4,5,6,8,9,10,11,13,18],g_0:2,g_1:[2,10],g_2:[2,10],g_:[2,9,10],g_analyt:2,g_dnn_ag:2,g_euler:2,g_i:2,g_m:[3,10],g_n:3,g_re:2,g_t:2,g_t_d2t:2,g_t_d2x:2,g_t_dt:2,g_t_hessian:2,g_t_hessian_func:2,g_t_jacobian:2,g_t_jacobian_func:2,g_trial:2,g_trial_deep:2,g_vec:2,gain:[1,5,9,10,13],galleri:0,game:4,gamma1:8,gamma2:8,gamma:[0,2,8,9,10,11,13],gamma_0:10,gamma_1:10,gamma_1x:10,gamma_:0,gamma_i:[0,8,18],gamma_j:13,gamma_k:13,gamma_m:10,gamma_x:0,gap:[6,8],gate:[4,12],gather:[0,1,12],gaug:12,gaussbacksub:16,gaussian:[4,5,6,8,14,18],gaussian_point:14,gaussian_rbf:8,gave:13,gbc:17,gca:[2,6,8,13],gd:1,gd_clf:10,gdclassiffiercgain:10,gdclassiffierconfus:10,gdclassiffierroc:10,gdm:13,gdregress:10,ge:[1,5,7,18],gemv:5,gen:[],gen_loss:4,gen_tap:4,gender:0,genener:4,gener:[0,1,2,3,5,6,8,10,11,12,13,14,16,18,20],generallay:12,generate_and_save_imag:4,generate_imag:4,generate_latent_point:4,generate_simple_clustering_dataset:14,generated_imag:4,generator_loss:4,generator_loss_list:4,generator_model:4,generator_optim:4,genexpr:[],genom:15,geodes:11,geometr:[0,13],geometri:5,georg:20,geotif:6,geq:[2,5,8,9,13],geron:[0,17,20],get:[0,1,2,3,4,5,6,7,9,10,11,13,15,16,18],get_dummi:9,get_loc:[],get_next_color:[],get_paramet:2,get_split:9,get_yaxi:8,get_yticklabel:6,getsolutionslic:5,getval:[],gg:[],gibb:15,gif:4,gini:10,gini_index:9,git:[0,15],github:[0,3,4,6,14,15,17,20],gitlab:[0,15],give:[0,1,2,3,5,6,7,8,9,10,12,13,14,15,17,18,20],given:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,18],global:[6,7,13],glorot:1,gmail:[],go:[0,1,3,5,6,8,9,11,12,13],goal:[0,7,9],goe:[0,1,2,5,6,13,14,16],golden:13,gone:5,gong:1,good:[1,3,4,5,6,9,10,11,13,15,17,18,20],goodfellow:[4,17,20],googl:[1,3,4,14,15],got:[1,6],gov:6,gp:20,gpu:1,grad:[2,13],grad_analyt:13,grade:17,gradient:[0,3,4,7,8,9,12,15,17],gradient_desc:3,gradientboostingclassifi:10,gradientboostingregressor:10,gradients_of_discrimin:4,gradients_of_gener:4,gradienttap:4,gradual:[1,14],grai:[4,6],graph:[1,9,11,12,13],graph_from_dot_data:9,graph_funct:[3,4],graphic:[0,1,9],grasp:0,gray_r:[1,3],grayscal:3,great:[5,13],greater:[1,7,18],greatli:13,greedi:9,green:[0,3,9,18],grei:4,grid:[1,3,6,7,8,12,18],grossli:13,ground:0,group:[0,6,7,9,14,15,17],groupbi:0,grow:[1,3,9,10],growth:0,gru:4,guarante:[0,3,4,13,18],guess:[1,4,10,13,14],guestrin:10,guid:1,guilherm:19,h1:2,h21:17,h:[0,1,5,6,8,13,18,20],h_1:[2,13],h_2:[2,13],h_:[0,13],h_m:10,ha:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,18],habit:0,had:[0,1,6,7,13],hadamard:[1,12],half:[1,8,9],halv:10,hand:[0,1,2,3,5,11,12,13,15,16,17,18,20],handi:3,handl:[0,1,2,5,9,11,15],handle_unknown:9,handsid:12,handwrit:12,handwritten:[1,5],happen:[1,2,3,4,5,6,10,13,18],hard:[1,7,8,10,13],hardcopi:15,harder:[0,1],harmon:3,hasn:1,hassl:[0,15],hast:15,hasti:[0,6,17,20],hat:[1,5,6,7,9,10,11,12,13,16],have:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,18],haven:1,he:7,head:[0,4,10,18],header:0,heads_proba:10,health:0,hear:[0,13],heart:[0,7],heatmap:[0,1,3,7],heavili:0,heavisid:1,height:[1,3,6],held:13,help:[0,1,4,12,13],helper:[4,14],henc:[0,5,6,8,9,10,12,13],her:7,here:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,20],hereaft:[0,8,12],hermitian:16,hessenberg:16,hessian:[0,2,5,13],heterogen:[9,10],hi:7,hidden:[1,3,4,12],hidden_bia:1,hidden_bias_gradi:1,hidden_layer_s:[0,1],hidden_neuron:4,hidden_weight:1,hidden_weights_gradi:1,hierarch:5,high:[0,1,2,3,4,5,6,9,10,11,13,14,15,16],higher:[0,1,3,5,6,8,13],highest:[1,2],highli:[0,3,4,10,15,16,20],highwai:0,hing:8,hint:13,hip:15,hire:0,hist:[4,6,7,18],histogram:[0,6,7,18],histor:[7,11],histori:[3,4,12],histplot:[],hit:[],hitherto:5,hjorth:19,hobbi:18,hoc:5,hoff:20,hold:[1,3,6,13,14],holder:0,home:0,homework:[6,13],homogen:[1,3,9,10,13],honchar:2,hopefulli:[0,11,18],horizont:11,hors:[3,7],hot:[1,9],hour:[1,15,17,18,19],house_pric:[],how:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,18,20],howev:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,18],hs:[],hspace:[0,4,8,10,18],hstack:1,htf:17,html:[7,11,15,17,20],http:[3,4,6,7,11,13,14,15,16,17,20],huang:0,huber:0,huge:[1,3,4,15],human:[0,1,3,6,9,12],humid:9,hundr:1,hungri:1,hybrid:17,hydrogen:0,hyperbol:[1,4,12],hyperparam:8,hyperparamet:[3,4,5,6,9,13],hyperplan:11,i0:0,i1:[0,6,8,12],i2:[0,8,12],i3:[0,12],i5:0,i:[0,2,3,4,5,6,7,8,9,10,11,12,13,14,16,18,19],i_1:[5,6],i_2:[5,6],i_:13,ian:20,ic:1,id:[7,13],ida:19,idea:[0,1,2,3,4,6,9,10,12,13,16],ideal:[0,2,6,8,13,18],idem:6,ident:[5,6,12,13,16],identical:5,identifi:[0,1,7,9,11,12,13,14],idum:[],ieor:18,ifi:20,ifs:15,ignor:[0,1,3,9],ii:[16,18],iii:16,ij:[0,1,3,6,8,12,14,16,18],ik:[0,16],illustr:[5,7,10,12,13,14,15],im:6,imag:[1,3,4,6,9,11,12,14,20],image_at_epoch_:4,image_batch:4,image_height:3,image_path:[0,6,7,9],image_width:3,imageio:6,images_from_seed_imag:4,imagin:1,immedi:[0,3,4,6,15],implement:[0,2,3,4,5,6,8,9,10,11,12,13,14,18],impli:[3,5,6,7,13,16],implicit:3,implicitli:[11,18],importantli:3,impos:[0,6,11,12],imposs:[0,5],impress:[0,12],improv:[0,4,5,9,10,11,13],impur:9,imread:6,imshow:[1,3,4,6],in3050:20,in4080:20,in4300:20,in5400:[3,20],in_out_neuron:4,inaccur:13,inacio:19,inact:12,inadequ:0,inch:6,includ:[0,1,2,3,4,5,6,7,11,12,15,18,19,20],include_bia:[6,9],incom:12,incorrect:1,incoveni:8,increas:[0,1,3,4,5,6,7,8,9,11,12,13,18],increasingli:18,ind:6,inde:[0,2,4,5,6],indefinit:4,indent:[],indentationerror:[],independ:[0,5,6,7,8,12,13,18],index:[0,1,3,4,10,14,15,16,18,20],index_col:0,index_of:[],indic:[0,1,3,4,5,6,9,10,11,13],indispens:6,individu:[1,6,7,10,12,18],indu:0,indx1:2,indx2:2,indx3:2,indx:16,ineffici:[3,13],inequ:8,inequaltii:13,inertia:13,inf1000:15,inf1100:15,inf1100l:15,inf1110:15,inf3000:20,inf4490:20,inf5860:20,infeas:9,infer:[0,1,4,6,20],infer_nrow:0,inferenc:1,infil:[0,6,7,9],infin:[5,6,7,11],infinit:3,infinitesim:18,influenc:[6,10],influenti:1,inform:[0,1,3,4,6,9,11,12,13,14,16,20],infti:[3,6,13,18],ingeni:13,ingrad:2,ingredi:[0,9],inher:6,inherit:16,initi:[0,1,2,6,10,13,14,16,18],initial_epoch:[3,4],initialis:[],initialize_root:[],inject:14,inlin:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,18],inner:13,innov:20,inp:4,inplac:13,input:[0,1,3,4,5,6,7,8,9,10,12,13,14,18],input_dim:1,input_shap:[3,4],inputs:1,inputs_shuffl:[0,1],insert:[3,5,6,8,10,18],insid:[0,4,7],insight:[0,1,5,15,20],insist:[6,13],inspir:[0,1,12,20],instabl:2,instal:[0,1,5,6,9],instanc:[0,1,2,4,6,9,11,13],instanti:10,instead:[0,1,2,3,4,5,6,8,9,11,13,14,16,18],institut:1,instruct:[0,1],int32:10,int64:[],int_0:18,int_:[3,6,18],int_a:18,intak:0,integ:[1,2,13,14,16,18],integer_vector:1,integr:[3,6,18],intellig:[0,14,20],intend:10,intens:1,intention:14,interact:[0,6,9,12,15],intercept:[0,6,8,11,13],intercept_:[0,6,8,9,13],interchang:[5,12,16],interconnect:1,interest:[0,1,2,3,4,5,6,7,8,9,12,15,17,18],interfac:[0,1,16],interior:[0,9],intermedi:16,intern:[1,10,12],interpol:[1,3,4,6,12],interpr:5,interpret:[0,1,6,9,10,12,13,16,18],interv:[0,3,5,6,7,13,18],intial:13,intract:[0,4],intrins:[3,11,16,18],intro:[15,20],introduc:[0,1,5,6,8,10,12,13,16,18],introduct:[1,2,4,13,17,20],introductori:[0,4,16,20],intuit:[0,5,6,8,12,13],inv:[0,5,13],invalid:1,invalu:[0,13,15],invari:1,invd:5,inver:8,invers:[0,3,6,13],invers_period:[],inverse_transform:8,invert:[0,5,7,10],invok:[0,8],involv:[0,2,6,7,11,12],io:[0,15,17,20],iomanip:[],iostream:[],ip:[0,8,18],ipca:11,ipykernel_42331:[],ipykernel_42376:[],ipykernel_42449:[],ipykernel_42456:[],ipykernel_42530:[],ipykernel_42541:[],ipykernel_42553:[],ipykernel_42573:[],ipykernel_42580:[],ipykernel_42586:[],ipykernel_47411:[],ipykernel_47448:[],ipykernel_47647:[],ipykernel_47724:[],ipykernel_47735:[],ipynb:15,ipython:[0,1,2,3,4,5,6,7,8,9,10,11,13,14,15],iq:6,iri:[8,9],irreduc:6,irrelev:5,irrespect:0,is_integ:[],isbox:[2,13],iscomplexobj:[],isinst:[2,13],isn:5,isnul:0,isomap:11,issu:[1,3,4,9,14,16],it_arrai:13,item:[0,13],items:16,iter:[1,2,3,4,6,7,8,11,13,14,18],itr:5,its:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,18,20],itself:[5,6,12,18],ix:[],j1:16,j:[0,1,2,3,4,5,6,8,9,11,12,13,14,16,18,20],j_:6,j_lasso_sk:6,j_ridge_sk:6,j_sk:6,jackknavg:[],jackknif:[6,15],jackknstd:[],jackknvar:[],jackknvec:[],jacobian:[2,13],jacobian_shap:2,jargon:[],jason:4,jax:[3,4,14],jensen:19,jerom:20,ji:[12,16],jj:[0,5,6],jk:[0,1,6,12,16],jl:0,jm:16,joao:19,joaogca:19,job:[2,8,10],join:[0,4,6,7,9],joint:[4,5],journal:[],judg:13,judgement:6,julia:[15,16],jump:18,junk:4,jupyt:[0,15,20],just:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,18],justif:0,justifi:[3,10],jy:[],k0:7,k1:7,k:[0,1,3,5,6,7,8,9,10,11,12,13,14,15,16,18],kaggl:6,kappa_d:18,karim:[],karlsen:[],karush:8,keep:[0,1,4,5,6,11,13,14,16],keepdim:[1,6,10,16],kei:[0,1,3,6,12,20],kept:[4,6,14],kera:[0,4,15],kernel:[0,1,3,15],kernel_regular:[1,3],kernel_s:4,kernelpca:11,kev:0,kevin:20,keyboardinterrupt:[2,3,4],keyerror:[],keyword:[6,13,16],kfold:6,kg:1,ki:16,kick:[1,13],kiener:2,kilomet:6,kind:[0,2,3,4,8,12,13,14],kj:[6,12,16],kjm:15,kkt:[5,8],kktsolver:5,kl:18,km:12,kmean:14,kmeanspoint:14,kn_k:14,know:[0,1,2,5,6,8,13,15],knowledg:[0,15],known:[1,3,4,5,6,7,8,9,12,16,18,20],kondev:0,kp:18,kpca:11,kroneck:14,kuhn:8,kwarg:[0,2,3,4,13],kwd:[0,3,4],kwown:0,l0:7,l1:[0,1,3,5,7],l1_l2:[1,3],l1regl:5,l1regls_mosek2:5,l1regls_mosek:5,l2:[1,3],l:[0,1,2,3,5,6,7,8,10,11,12,13,16,18],l_1:7,l_2:[7,13],l_:16,l_j:12,la:13,la_i:12,la_k:12,lab:[15,17],label:[0,1,2,3,4,5,6,7,8,9,10,12,13,14,15,16,18],label_prob:13,labelencod:[7,10],labels:[6,8,9],labels_shuffl:[0,1],laboratori:17,lack:0,lagari:2,lagrang:[8,11],lambda:[0,1,2,3,5,6,7,8,10,12,13,18],lambda_0:11,lambda_1:[5,8,11],lambda_2:[8,11],lambda_:11,lambda_i:[8,11],lambda_iy_i:8,lambda_jy_iy_j:8,lambda_k:8,lambda_n:[5,8],lamda:1,lamdbda:5,land:[0,8],landmark:8,landscap:13,langl:[0,6,11,18],languag:[0,1,4,8,15,16,20],lapack:[5,16],laplac:5,laptop:15,larg:[0,1,2,4,5,6,8,9,10,11,13,15,16,18,20],larger:[0,3,5,6,8,10,11,13,18],largest:[4,8,11],lasso:[0,7,15,17],lasso_sk:6,last:[0,1,2,3,4,5,6,7,8,9,10,12,13,16,17,18],latent:4,latent_dim:4,latent_point:4,latent_space_value_rang:4,later:[0,1,4,6,7,8,12,13,14,15],latest:4,latest_checkpoint:4,latter:[0,3,5,6,7,8,11,13,16,17,18],lattic:12,law:0,layer:[0,4,13],lbfg:[7,9,10,11],lcc:[5,6],lda:11,ldot:[0,6,11],le:[5,7,10,13,18],lead:[0,1,3,5,6,7,8,9,10,11,12,13,16,18],leaf:9,leaki:1,leakyrelu:4,lear:13,learn:[3,4,5,6,7,8,9,10,12,16,17,20],learnabl:3,learner:10,learning_r:[3,8,10],learning_rate_init:[0,1],learning_schedul:13,least:[0,2,7,8,10,11,15,16,17,18],leat:13,leav:[0,1,3,5,6,9,11],lectur:[0,1,5,10,11,12,13,15,16,17,20],lecturenot:[15,20],lecturenovember11:[],lecturenovember12:[],lecturenovember19:[],lecturenovember25:[],lecturenovember26:[],lecturenovember4:[],lecturenovember5:[],lectureoctober14:17,lectureoctober15:17,lectureoctober1:[],lectureoctober21:[],lectureoctober22:[],lectureoctober28:[],lectureoctober29:[],lectureoctober7:[],lectureoctober8:[],lectureseptember10:[],lectureseptember16firstpart:[],lectureseptember16secondpart:[],lectureseptember17:[],lectureseptember23:[],lectureseptember24:[],lectureseptember2:[],lectureseptember30:[],lectureseptember3:[],lectureseptember9:[],lecturethursdayaugust26:[],lecturethursdayaugust27:[],left:[0,1,2,3,5,6,7,8,9,10,11,12,13,14,16,18],leftarrow:[8,12],legend:[0,2,3,4,5,6,7,8,9,10,13],len:[0,1,2,3,4,5,6,8,9,10,11,12,16],len_index:0,length:[0,1,2,3,4,8,9,13,15],length_of_sequ:4,leq:[0,5,7,8,13,14,18],less:[0,1,3,4,5,6,8,9,13,18],lessen:1,let:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,18],letter:[0,16,18],level:[0,1,5,6,9,15,16,17],li:[8,11],lib:[0,1,2,3,4,6,7,8,11,13,14],liblinear:[8,10],librari:[0,1,2,3,4,5,6,9,10,11,16,18,20],licens:[0,1,15],lie:[0,6,11,18],life:[0,1,8,12],lifetim:13,like:[0,1,2,3,4,5,6,7,9,10,11,12,13,15,16,18],likelihood:[0,1,5,9,13],lim_:18,limit:[0,5,6,7,8,11,12,16],lin_clf:8,lin_model:0,lin_reg:9,linalg:[0,2,5,6,8,11,13,16,18],line1:8,line2:8,line2d:13,line3:8,line:[0,1,3,6,8,11,13],linear:[1,3,5,6,7,9,10,11,12,15,17,18],linear_model:[0,5,6,7,8,9,10,11,13],linear_regress:6,linearli:5,linearloc:[6,13],linearregress:[0,6,7,9],linearsvc:8,liner:[1,3],linerar:10,linewidth:[0,2,4,6,8,9,10],link:[0,4,9,12,15],linlag:5,linpack:16,linreg:0,linspac:[0,2,3,4,6,8,9,10,13,16,18],linu:4,linuek:[],linux:[0,1,15],liquid:0,list:[0,1,2,3,4,5,9,15],listcomp:2,listedcolormap:[9,10],literatur:[1,7,14,20],littl:[1,3,9,12],live:8,ll:[0,18],lle:0,lloyd:[4,14],lmb:[2,5,6],lmbd:[0,1,3],lmbd_val:[0,1,3],lmbda:13,ln:[1,13],lo:5,load:[0,1,4,6,7,9,10],load_boston:0,load_breast_canc:[1,7,9,10,11],load_data:[3,4],load_digit:[1,3],load_iri:[8,9],loc:[0,3,6,7,8,9,10],local:[0,1,3,7,12,13],locat:[2,3,8],lock:[3,4],log10:[5,6],log:[0,1,2,4,5,6,7,9,10,11,13,16],log_:0,log_clf:10,logarithm:[0,5,7,16],logic:[0,1,9],logist:[0,1,2,8,9,10,11,12,15,17],logistic_predict:13,logisticregress:[7,9,10,11],logit:7,logreg:[7,9,10,11],logspac:[0,1,3,5,6],longer:[2,3,8,10,14,16,18],loocv:6,look:[0,1,2,3,4,5,6,7,8,9,10,11,13,16,18],loop:[1,4,6,10,12,14,15,16],lose:1,loss:[0,1,3,4,5,6,7,8,10,11,13,16],loss_fil:4,lossfil:4,lost:4,lot:[0,1,4,6],low:[0,6,9,10,11],lower:[0,1,3,6,9,10,16],lowercas:16,lowest:[9,13,18],lr:[1,3,4,10],lstat:0,lstm:4,lstm_2layer:4,lstsq:0,lt:6,lu:[0,5],lubksb:16,luckili:2,ludcmp:16,lux:16,lvert:1,lw:0,m:[0,1,2,3,5,6,8,9,10,11,12,13,16,17,18,19,20],m_1:14,m_:[9,12],m_h:0,m_k:14,m_l:12,m_n:0,m_p:0,m_t:13,ma:11,mac:[3,4,14],machin:[1,3,4,5,6,7,9,10,11,12,14,16,17,20],machinelearn:[6,15,17,20],mackai:20,made:[0,1,3,4,5,6,7,9,11,12],mae:0,magic:4,magnitud:[1,6,7,13],mai:[0,1,2,3,5,6,7,8,9,11,12,13,15,16,18],mail:17,main:[0,1,3,4,5,6,7,9,16,20],mainli:[0,5,6,7,9],maintain:6,major:[1,6,9,10,13,16],make:[1,2,3,4,5,6,7,8,11,12,13,15,16,18,20],make_axes_locat:6,make_moon:[8,9,10],make_pipelin:[0,6,10],make_vjp:[2,13],makedir:[0,6,7,9],makeplot:0,malcondit:16,malign:[1,7,9],mammographi:5,manag:[0,2,3,15],mani:[0,1,3,4,5,6,7,8,9,11,13,14,15,16,18,20],manifold:11,manner:3,manual:6,map:[0,1,2,6,7,8,11,12,14,18],margin:[0,5,8],mari:19,marit:0,marker:[0,7,16],markov:15,marsaglia:18,mask:[],mass:[0,1,5,13],massag:0,masses2016:0,masses2016ol:0,masses2016tre:0,masseval2016:0,master:[6,17],mat1100:15,mat1110:15,mat1120:15,mat3155:[],mat4155:[],mat:15,match:[0,1,4,5,13,14],materi:[4,5,7,13,16],math:[3,7,12,13,16,18,20],mathbb:[0,4,5,6,7,8,11,12,13,14,16,18],mathbf:[0,5,6,7,8,13,16],mathcal:[1,5,6,7,13],matheemat:3,mathemat:[0,6,11,12,13,15,16,17,18,20],mathrm:[0,1,3,4,5,6,7,8,9,10,11,12,13,14,18],matmul:[1,2,5],matmul_adjoint_1:2,matmul_vjp_0:2,matmul_vjp_1:2,matnat:[17,19,20],matplotlib:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,18],matplotlibdeprecationwarn:[6,13],matric:[0,1,3,4,6,7,8,11,13,15],matrix:[0,2,3,4,6,7,8,10,13,18],matshow:1,matter:[2,3,13],max:[0,1,2,3,4,9,10,12,13],max_depth:[0,9,10],max_diff1:2,max_diff2:2,max_diff:2,max_it:[0,1,7,8,11,13],max_iter:14,max_leaf_nod:10,max_queue_s:[3,4],max_sampl:10,maxdegre:[0,6,10],maxdepth:10,maxim:[1,4,5,7,8,11],maximum:[0,1,2,3,5,7,8,9,10,13,14],maxpolydegre:[5,6],maxpooling2d:3,mbox:[5,6],mc:[],mcculloch:12,mcint:[],mcintsqr2:[],md:11,mdoel:4,mean:[1,2,3,4,5,6,7,9,10,11,12,13,14,15,16,18],mean_absolute_error:0,mean_divisor:14,mean_i:18,mean_matrix:14,mean_squared_error:[0,4,6,7,10],mean_squared_log_error:0,mean_vector:14,mean_x:18,meaning:[0,4,7],meansquarederror:0,meant:[2,3,7,10,13],meantempvec:[],meanvec:[],measur:[0,1,2,5,6,9,11,12,14,18],mechan:[0,4,18],median:0,medicin:12,medium:[4,8,13],medv:0,meet:[0,19],mehta:0,memori:[3,4,11,12,13,16],mention:[0,12,13,18],mere:0,mersienn:[],meshgrid:[2,5,6,8,9,10,11],messag:[5,13],messi:2,met:[0,3,8],metadata:2,meteorolog:9,meter:6,method:[0,1,2,3,4,5,7,8,11,12,14,15,16,17,18,20],metion:6,metric:[0,1,3,6,7,9,10,14],metropoli:15,mev:[0,18],mgd:13,mglearn:15,mgrid:13,mhjensen:[1,2,3,4,6,7,8,11,14],mi:10,microsoft:20,mid:1,midel:4,midpoint:9,might:[0,1,2,4,6,9,13],mild:9,miller:[],millimet:6,million:0,mimic:12,min:[0,2,5,8,9],min_:[0,2,5,14],min_samples_leaf:9,mind:[0,6,13],mindboard:4,mine:15,mini:[1,11,12,13],minibatch:[1,11,13],minibathc:13,miniforge3:[0,1,2,3,4,6,7,8,11,13,14],minim:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14],minima:[0,1,7,13],minimum:[0,1,2,6,8,9,11,13],minmaxscal:0,minor:[6,13,18],minst:1,minu:7,mirror:9,misc:6,misclassif:[8,9,10],misclassifi:[8,10],miser:0,mismatch:1,miss:[0,10],mistak:4,mit:20,mix:[1,2],mixtur:13,mk:[9,16],mkdir:[0,6,7,9],ml:[0,1,10,13,16],mlab:18,mle:[5,7],mline:[],mlir:[],mlir_graph_optimization_pass:[],mlp:1,mlpclassifi:1,mlpregressor:0,mm:16,mn:[12,18,20],mnist:[1,11],mod:18,mode:17,model:[2,3,5,7,8,9,10,11,13,14,15,18,20],model_select:[0,1,3,5,6,7,9,10,11],moder:10,modern:[0,6,7,15],modif:[2,12,13],modifi:[0,1,3,5,7,8,10,12,13],modul:[0,2,3,4,5,6,7,8,9,10,11,13,16],modular:18,modulenotfounderror:[5,7,8,9],modulo:18,moe:[5,11],moment:[5,6,13,18],monitor:13,monoton:[5,12,18],mont:[0,6,15,18,20],montecarlocycl:[],moor:[5,6],more:[0,1,2,4,5,7,8,9,10,11,12,13,14,15,17,18],moreov:[0,3],morten:19,mortenimac:[],mosek:5,most:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,17,18],mostli:[1,11],motion:[0,13],motiv:[1,4],move:[0,4,5,6,7,9,12,13,14,18],mp4:17,mpl:[0,7],mpl_toolkit:[2,6,13],mplot3d:[2,6,13],mplregressor:1,mse:[0,4,5,6,9,10],mse_simpletre:10,mselassopredict:5,mselassotrain:5,mseownridgepredict:6,msepredict:5,mseridgepredict:[5,6],msetrain:5,msg:[],msle:0,mt19937_64:[],mt:[7,12],mu0:18,mu1:18,mu2:18,mu:[0,6,11,13,18],mu_:[6,18],mu_i:6,mu_n:11,mu_x:18,much:[0,1,2,3,4,5,6,8,9,10,11,12,13,16,18],mul:5,multi:[0,1,3,7,15],multiclass:[1,7],multidimension:[11,12],multilay:1,multinomi:7,multipl:[2,4,5,6,7,12,13,18],multipli:[3,5,6,11,13,16,18],multiplum:8,multivari:[0,2,10,11,15,18],multivariate_norm:[11,14],murphi:[11,20],must:[0,1,2,5,6,8,10,12,13,14,18],mutat:7,mutual:[1,3,6,13],mx_:18,myenv:[0,1,2,3,4,6,7,8,11,13,14],myriad:[0,15],mz1:18,mz2:18,n1:16,n2:16,n:[0,1,2,3,4,5,6,7,8,10,11,12,13,14,16,18],n_0:[12,18],n_:[1,2,3,8,12,18],n_b:[],n_boostrap:[6,10],n_bootstrap:6,n_categori:[1,3],n_cluster:14,n_compon:11,n_epoch:13,n_estim:10,n_examples_to_gener:4,n_featur:1,n_filter:3,n_hidden:2,n_hidden_neuron:[0,1],n_i:18,n_input:[0,1,3],n_instanc:9,n_iter_i:[7,11],n_job:10,n_k:14,n_l:[12,18],n_layer:1,n_m:9,n_neuron:1,n_neurons_connect:3,n_neurons_layer1:1,n_neurons_layer2:1,n_point:14,n_sampl:[6,8,9,10,14],n_split:6,n_step:4,n_t:2,n_x:2,nabla:[1,13],nabla_:[2,13],nabla_w:13,nag:13,naimi:0,naiv:7,naive_kmean:14,nall:0,name:[0,1,3,4,5,6,7,8,9,10,12,13,14,15,16,18,19],nameerror:[6,10],namespac:[],narrow:13,nary_f:[2,13],nary_op_arg:[2,13],nary_op_kwarg:[2,13],nary_oper:[2,13],nation:[1,5],nativ:15,natur:[0,1,4,8,9,12,13,18,20],navier:12,nb:18,nb_:16,nboot:[],nd:14,ndarrai:[2,6],ndim:2,ne:[9,10,16,18],nearest:[1,3,6,11],nearli:13,neccesari:6,necess:2,necessari:[0,1,3,4,8,14],necessarili:[0,4,11,18],necesserali:5,neck:7,need:[0,1,2,3,4,5,6,8,9,10,11,12,13,14,16,18],neg:[0,1,3,5,6,7,10,13,16,18],neg_mean_squared_error:6,neglect:18,neglig:18,neighbor:[3,6,11],neither:[4,13],neq:[13,14,18],nervou:12,nest:[2,9,12],nesterov:13,net:[2,4,12],netlib:16,network:[0,9,13,15,17,20],neural:[0,7,13,15,17,20],neural_network:[0,1,2],neuralnetwork:1,neuron:[1,2,3,4,12],neutral:0,neutron:0,never:[1,3,4,6,9,18],new_box:[2,13],new_root:[2,13],new_trac:[2,13],new_tracing_count:[3,4],newaxi:[0,3,6,9],newli:0,newton:[1,7,8,13,18],next:[0,1,2,3,4,5,6,8,9,13,14],next_guess:13,next_input:4,ng:1,ngini:[],ni:14,nian:[],nice:[0,1,5,11],nichola:[],nicholaskarlsen1102:[],niter:13,nitric:0,nlambda:[5,6],nlevel:[],nm:18,nm_n:0,nmse:6,nn:[2,5,6,12,16],nn_model:1,nnmin:2,node:[1,2,3,9,10,12],node_constructor:2,nois:[0,4,5,6,8,9,10,13],noise_dimens:4,noisi:[1,6],non:[0,1,3,4,5,6,7,9,10,11,12,13,14,16,18],none:[0,1,2,3,4,5,9,10,13,18],nonlinear:[3,6,8,9,11,12],nonneg:[6,9,13],nonparametr:6,nonsens:18,nonsingular:16,nonumb:[3,7,8,13,16],nor:[1,4,13],norm:[0,1,5,6,8,11,13],normal:[3,4,5,6,7,8,9,10,11,12,13,15,16,18],normali:16,normalize_kwarg:[],norwai:6,notat:[0,2,5,6,13,14,18],note:[0,1,2,3,4,5,6,7,8,11,12,13,14,15,16,17,18,20],notebook:[0,1,3,9,15],noth:[1,2,5,8,12,14,18],notic:[4,5,12,13,16,18],notimplementederror:[],notion:3,notrace_primit:[],novel:[3,6,10],novemb:1,now:[0,2,4,5,6,7,8,10,11,12,13,14,16,18],nowadai:[0,1,3,9,15],nox:0,np:[0,1,2,3,4,5,6,7,8,9,10,11,13,14,16,18],npr:2,nsampl:6,nspin:[],nt:2,nthi:0,ntrained_model:6,nu:18,nuclear:5,nuclei:[0,18],nucleon:0,nucleu:0,num:4,num_coordin:2,num_hidden_neuron:2,num_it:2,num_neuron:2,num_neurons_hidden:2,num_output:[3,4],num_point:2,num_tre:10,num_valu:2,number:[1,3,4,5,6,7,8,9,10,11,12,13,14,16,17,19],numberid:7,numberparamet:3,numer:[0,5,6,9,10,11,12,13,15,16,20],numpi:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,18],numpy_vjp:2,numpy_wrapp:2,nunmpi:5,nvalu:[],nx:2,nx_test:6,nx_train:6,nx_train_mean:6,ny:18,ny_pr:6,ny_train:6,ny_train_mean:6,o:[0,6,7,8,9,11,16,20],obei:[6,11,13],object:[0,1,2,4,5,6,8,10,13,16],objsens:5,obliqu:5,observ:[1,3,5,6,7,8,9,10,11,12,13,14,18],obtain:[0,1,5,6,7,8,9,10,12,13,14,16,18],obviou:[5,6,11,18],obviouli:0,obvious:[0,4,5,6,16],oc:5,occupi:0,occur:[0,6,8,9,16,18],od:0,odd:[0,3,7],odenum:2,odesi:2,oen:0,off:[1,3,4,5,9,13,18],offer:[6,11,15,16,17],offic:19,offici:17,ofil:[],ofstream:[],often:[0,1,3,4,5,6,7,8,9,10,11,12,13,14,15,16,18],ofter:16,ol:0,old:[1,5,10,13],ols_sk:6,ols_svd:6,olsbeta:5,omega:[2,3,6],omega_0:3,omit:[0,5],on_train_batch_begin:[3,4],onc:[1,6,9,11,13],one:[0,1,3,4,5,6,7,8,9,10,11,13,14,15,16,18],oneapi:[],onednn:[],onehot:1,onehot_vector:1,onehotencod:9,ones:[0,2,5,6,8,9,10,11,13,16,17],ones_lik:4,onl:3,onli:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,18],onlin:[11,17],onto:[5,11],op_nam:[3,4],open:[0,1,4,6,7,9,15,17],oper:[0,1,3,5,6,10,11,12,13,15,18],operation:18,oplu:18,opmiz:13,opportun:0,oppos:[6,13],opposit:[1,5,8],opt:[1,5],optim:[0,2,3,4,5,6,7,9,10,11,14,17],optimis:[1,3],optimizer_v2:3,option:[0,1,3,5,6,7,8,11,16],optionalxlacontext:[3,4],optmiz:[1,8,13],orang:0,order:[0,1,2,3,5,6,7,8,9,10,11,12,13,16,18],ordinari:[0,2,3,7,11,13,15,17],oreilli:20,org:[3,4,7,11,15,16,20],organ:[6,7,10,16],orient:[1,5,18],origin:[0,3,5,6,8,11,12,13,16],orthogn:5,orthogon:[0,5,6,8,11,13,16],orthonorm:5,os:[0,1,4,5,6,7,8,9],oscar:1,oscil:[3,13],oslo:[0,15,17,19],osx:[0,15],other:[0,1,2,3,5,6,7,8,10,13,14,15,17,18,20],otherwis:[0,1,4,7,13,16],ouput:[5,7,12],our:[1,2,3,6,7,8,9,10,12,14,15,16,17,18],ourmodel:0,ourselv:[0,5,6,8,11,13],out:[0,1,2,4,5,6,7,8,9,10,11,12,13,15,16,18],out_fil:9,outcom:[0,7,9,10,12,18],outdoor:9,outer:[6,12],outfil:4,outfilenam:[],outgrad:2,outlier:[0,8],outlin:[6,10,11],outlook:9,outperform:10,output:[0,1,3,4,5,6,7,8,9,10,12,13,16,18],output_bia:1,output_bias_gradi:1,output_shap:4,output_weight:1,output_weights_gradi:1,outputlayer1:12,outputlayer2:12,outsid:4,over1:13,over:[0,1,3,4,5,6,9,10,12,13,16],overal:[1,10],overcast:9,overcom:[12,13],overdetermin:0,overfit:[0,1,3,6,9,10,13],overflow:[1,5],overhead:12,overlap:[3,7,8,9],overlin:[0,5,6,9,10,11,14,16],overst:0,overtrain:4,overview:[3,20],own:[4,5,6,8,12,13,15,16],owner:0,ownridgebeta:6,oxid:0,oyvinssc:19,p0:2,p1:2,p:[0,1,2,3,4,5,6,7,8,9,10,11,13,14,16,18],p_:[2,4,8,9],p_hidden:2,p_i:[5,18],p_j:18,p_n:18,p_output:2,p_x:18,pack:0,packag:[0,1,2,3,4,5,6,7,8,11,13,14,15,18],pad:[3,4],page:[0,15],pai:[0,1,9,13],pair:[0,2,3,9,15,18],panda:[0,4,5,6,7,9,11,15],paper:1,paradigm:0,parallel:[10,16],param:[2,4],paramat:2,paramet:[0,1,2,3,4,5,6,7,8,9,10,12,13,18],parameter:[0,6,10],parametr:[0,6],paramt:[3,5],parent:2,parent_argnum:[],park:[],parser:0,part:[0,1,3,5,6,10,16,17,18,20],partial:[0,1,5,6,7,8,10,11,12,13,18],particip:[15,17],particl:[0,4,13,18],particular:[0,1,2,3,5,6,9,10,11,12,13,18,20],particularli:[5,6,8,11,13,18],partit:[1,4,9],partli:6,pass:[2,3,12,14],past:[10,18],patch:[6,18],path:[0,4,6,7,9,15],patient:7,patter:4,pattern:[0,3,4,12,17,20],pauli:0,pc:[11,15],pca:[0,7,15,17],pcolor:6,pcolormesh:6,pd:[0,4,5,6,7,9,11],pde:2,pdf:[0,3,4,5,6,9,20],pedagog:0,penal:6,penalti:[6,13],penros:[5,6],pentagon:13,peopl:[0,1,9,13,15],per:[0,1,6,17],percentag:[0,10,11],perceptron:[0,1,7],perfect:[0,1],perfectli:[4,6],perform:[0,2,3,4,5,6,8,10,11,12,13,14,15,16,18],performac:4,perhap:[0,5,13],perimet:1,period:[1,4,18],permut:11,persist:13,person:[5,6,7,17,19],perspect:20,pertin:12,petal:[8,9],peter:20,petersen:[],phantom:18,phase:[6,12],phenomena:18,phi:8,phi_k:8,philip:[],philosophi:13,phone:19,photo:4,phrase:0,physic:[0,1,4,7,12,13,18,19,20],pi:[2,3,5,6,7,9,12,13,18],pick:[1,9,10,11,13,14],pickl:1,pictur:0,pie:15,piec:[11,14],pillow:[0,15],pinv:[5,6,13],pip3:[0,1],pip:[0,1,15],pipelin:[0,6,8,10],pit:4,pitfal:6,pitt:12,pixel:[1,3,4],pixel_height:[1,3],pixel_width:[1,3],place:[0,4,6,8,13,16],plai:[0,3,4,5,6,8,11,15],plain:[8,10,12,13,14],plan:[6,9,19,20],plane:[8,9],plateau:5,platform:15,plausibl:12,pleas:[3,4,6,7,11,13,14],plenti:1,plethora:[3,12],plot:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,18],plot_confusion_matrix:[7,10],plot_count:6,plot_cumulative_gain:[7,10],plot_data:1,plot_dataset:8,plot_decision_boundari:[9,10],plot_import:10,plot_max:4,plot_min:4,plot_model:4,plot_numb:4,plot_predict:8,plot_regression_predict:9,plot_result:4,plot_roc:[7,10],plot_surfac:[2,6,13],plot_train:9,plot_tre:[9,10],plt:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,18],plu:[0,3,5,7],pm:8,pmatrix:2,pn:3,png:[0,4,6,7,9],point:[0,1,2,3,5,6,7,8,9,10,11,13,14,16,18,19],point_1:4,point_2:4,poisson:[15,18],poli:[6,8],poly100_kernel_svm_clf:8,poly3:0,poly3_plot:0,poly3dcollect:13,poly_featur:[8,9],poly_features10:9,poly_fit10:9,poly_fit:9,poly_kernel_svm_clf:8,polydegre:[0,5,6,10],polygon:13,polym:12,polynomi:[0,5,6,7,8,9,10,11],polynomial_featur:6,polynomial_svm_clf:8,polynomialfeatur:[0,6,8,9],polytrop:[0,6],pool:3,pool_siz:3,poor:[1,13],poorli:0,pop:2,popul:[0,5],popular:[0,1,3,6,7,8,9,11,12,15,16,18],popularli:0,portabl:10,portion:[11,13],pose:[0,4,5,6,11,18],posit:[0,1,2,3,5,7,8,10,11,13,14,16,18],possibl:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,15,16,18,19],posterior:5,postpon:0,postul:5,potenti:[0,3,5,6,12,13],potr:5,potrf:5,pott:12,power:[0,1,5,6,8,9,12,13],pp:[5,6],practic:[0,5,6,7,8,17,18],practition:[0,1,3],preced:[1,11,12,18],preceed:4,preceq:8,precis:[0,2,5,11,13,16,18],pred:[6,13],predicit:0,predict:[0,1,5,6,7,8,9,10,15,20],predict_prob:1,predict_proba:[7,10],predictor:[0,5,6,7,9,10,11],prefer:[0,1,6,8,9,11,15],prepar:[0,6,16],preprocess:[0,4,6,7,8,9,10,11],prerequisit:0,presenc:13,present:[0,5,6,9,12,13,16,17,18],preserv:[3,11,16],press:[13,20],pretrain:[1,4],pretti:[0,4,8,9,15],prev_centroid:14,prevent:[13,18],previou:[0,1,2,3,4,5,6,8,10,11,12,13,16,18],previous:[2,3,9,10,18],price:[0,4,9,13],primal:8,primari:[0,7],prime:18,primit:2,princip:[0,5,7,15,17],principl:[0,6,7,8,14],print:[0,1,2,3,4,5,6,7,8,9,10,11,13,14,16,18],print_funct:[8,9],printout:0,prior:[0,5,6],privat:0,prob:[1,18],probabilist:[0,20],probabl:[0,1,3,4,6,7,10,13,15,17],problem:[0,3,4,5,6,7,8,9,10,11,12,14,15,16,17,18],proce:[0,5,6,8,9,10,11,13,16],procedur:[2,4,5,6,8,10,11,13],proceed:16,process:[0,2,4,6,9,10,12,13,15,16,18,20],prod:20,prod_:[1,5,7],produc:[0,3,4,5,6,9,10,11,12,13,15,16,18],product:[0,1,3,5,6,7,8,12,13,15,16],profess:0,program:[0,1,4,5,6,8,12,14,15,16,17,18],programm:16,progress:[1,4,14],prohibit:6,project1:6,project:[0,1,2,3,5,11,13,15,17,19],project_root_dir:[0,6,7,9],promin:12,promis:8,prone:9,pronounc:[13,15],proof:[0,11,12,13],propag:[2,3,13,17],proper:[0,2,6],properli:[1,6,8,10,13],properti:[0,1,3,12,13,16],proport:[0,1,5,9,11,13,18],propos:[1,4,6,10],propto:[5,13],protect:[3,4],proton:0,prove:[3,13],provid:[0,1,3,4,5,6,8,9,10,12,13,15,16,18,20],proxi:[1,13],prune:9,pseudo:[16,18],pseudoinv:5,pseudoinvers:[5,6],pseudorandom:[6,18],psycholog:0,pt:13,ptratio:[],punish:[0,1],pure:[3,9,18],purest:9,puriti:9,purpos:[0,3,10,12,14],put:1,putarow:5,putboundslic:5,putclist:5,putobjsens:5,putqobj:5,py:[0,1,2,3,4,5,6,7,8,11,13,14],pydata:15,pydot:9,pylab:[0,7],pylint:[3,4],pypi:15,pyplot:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,18],pythagora:5,python2:0,python3:[0,1,2,3,4,6,7,8,11,13,14,15],python:[1,2,3,4,5,6,8,11,12,13,14,17,18],pytorch:[0,15],pywrap_tf:[3,4],q:[5,6,8,11,18],qp:8,qquad:[2,11,13,16],qr:[5,6,16],quad:[1,13,16],quadrat:[0,5,8,9,13],qualit:[4,9,18],qualiti:[0,9,15],quantifi:1,quantil:10,quantit:[0,6,9],quantiti:[0,2,5,6,7,9,10,11,12,14,16,18],quantum:[4,12],quartil:0,quench:5,queri:9,question:[0,5,6,9,11,12,13],qugan:4,quick:[4,18],quick_execut:[3,4],quickli:[1,3,9,11,13],quit:[1,5,6,9,10,12],quot:4,r2:[0,5,6],r2_score:0,r2score:0,r:[0,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,18],r_1:9,r_2:9,r_j:9,r_m:9,rad:0,radial:[0,8,12],radioact:18,radiu:[0,1],rag:2,rain:9,rais:[0,2,13],ramp:1,ran0:18,ran1:18,ran2:18,ran3:18,rand:[0,4,5,6,9,10,13,16],rand_max:[],randint:[6,9,13],randn:[0,1,2,5,6,9,11,13],random:[0,1,2,3,4,5,6,8,9,13,14,15,16,17],random_devic:[],random_forest_model:10,random_index:13,random_indic:[1,3],random_st:[0,7,8,9,10,11],randomforestclassifi:10,randomli:[1,6,9,13,14],randomnumbergener:[],rang:[0,1,2,3,4,5,6,7,9,10,11,12,13,14,16,18],rangl:[0,6,11,18],rangle_x:18,rank:5,rankdir:4,raphson:[1,8,13],rapidli:0,rare:[1,13],rate:[0,1,2,3,4,8,9,10,12,13],rather:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,16,18],ratio:[4,7,9,10,11],rational:0,ravel:[5,6,7,8,9,10,11,13,16],raw:3,raw_df:[],rbf:[8,11,12],rbf_kernel_svm_clf:8,rbf_pca:11,rc:[0,18],rcond:0,rcparam:[0,1,3,7,8,9,10,18],rd:[],re:[2,4,13],reach:[1,4,5,6,7,9,10,11,12,13,14],read:[0,2,3,4,5,6,7,8,11,12,16,17,18,20],read_csv:[0,6,7,9],read_fwf:0,reader:[0,6,16,18],readi:[0,1,5,6,8,10,11,12,16],readili:1,real:[0,1,2,4,7,10,11,12,13,16],real_loss:4,real_output:4,realist:8,realiti:18,realiz:[1,12],realli:[0,1],rearrang:13,reason:[0,1,3,4,10,13,20],reassign:1,rebuild:[],recal:[5,6,9,10,11,12,16,18],recalcul:[],recast:3,receiv:[1,3,10,12,18],recent:[0,2,3,4,5,6,7,8,9,10,13],recept:[3,12],receptive_field:3,recip:[0,6,7,16],reciproc:5,recogn:[0,4,5,10],recognit:[0,1,3,12,17,20],recommend:[0,2,3,4,5,6,8,13,15,16,17,20],reconsid:9,reconstruct:11,record:[10,17],recreat:[],rectangl:[9,13],rectangular:5,rectifi:[1,3,12],recur:[0,15],recurr:[0,1,15,17],recurs:[9,15,16],recycl:[],red:[0,3,4,6,8,9],redefin:[0,10],reduc:[1,3,5,6,9,10,11,13],reduct:[0,10,11,15,18],refer:[0,1,2,3,5,6,7,11,12,13,14,16,20],referenc:2,refin:12,refit:6,reflect:[0,1,4,5,18],refresh:[15,17],reg:[10,11],regard:[1,9,13],regardless:12,region:[3,4,6,9,12],regist:[6,18],reglasso:5,regr_1:[0,9],regr_2:[0,9],regr_3:[0,9],regress:[1,8,11,12,15,16,17],regressor:[0,7,10],regridg:[5,6],regular:[0,3,4,5,6,7,9,13],regularis:6,reilli:[0,20],reinforc:[0,8,15],reiter:1,rel:[0,4,6,7,9,12,13,18],relat:[0,1,3,4,5,11,13,14,16,18],relationship:[0,4,9],relativeerror:0,releas:[1,3,4,6,13,15],relev:[0,1,5,7,11,15,17,18],reli:[0,6,8],reliabl:[7,18],relu:[3,4],remain:[1,2,4,6,12,16,18],remaind:18,reman:2,remark:1,rememb:[0,8,13,16],remind:[0,5,11,13,16,18],remov:[0,4,5,6],render:0,reorder:[5,7],reorgan:0,repeat:[0,1,3,4,5,6,9,10,11,13,14,16,18],repeated:0,repeatedli:[6,10,13],repet:3,repetit:[6,17],rephras:13,replac:[0,1,3,4,5,6,10,12,13,14],replica:6,repositori:[0,4],repres:[0,1,2,3,4,5,6,7,8,9,10,12,13,18],represent:[0,1,3,6,18],representd:3,reproduc:[0,5,6,9,12,15,18],repuls:0,request:[0,13],requir:[0,1,3,4,5,6,8,9,11,12,13,16],res1:2,res2:2,res3:2,res_analyt:2,res_analytical1:2,res_analytical2:2,res_analytical3:2,resaml:6,resampl:[0,7,10,15,17],rescal:[0,11,12],rescu:5,reseach:6,research:[0,4,15,20],resembl:[6,18],reserv:[1,5,6,18],reset:[],reshap:[0,1,2,3,4,6,8,9,10,16],residenti:0,residu:[0,5,13],resiz:5,respect:[0,1,2,3,5,6,7,8,10,11,12,13,14,18],respond:12,respons:[0,7,9,12],rest:[0,5],restat:[0,12],restor:4,restored_discrimin:4,restored_gener:4,restrict:[0,3,9,12],result:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,18],result_typ:[],retail:0,retain:[5,6],return_data:14,return_kwarg:[],return_sequ:4,return_x_i:9,reus:[1,3,6],reveal:[0,12],revers:[1,16],review:[15,16,17],revisit:14,reward:[0,4],rewrit:[0,3,5,6,7,8,10,11,12,13,16,18],rewritten:[2,6,8,10,18],rewrot:13,rf:10,rgb:3,rgoj5yh7evk:15,rh:[5,6],rho:[0,10],rho_1:10,rho_2:10,rho_m:10,rich:0,ride:9,rideclass:9,ridedata:9,ridg:[0,7,11,13,15,17],ridge_sk:6,ridgebeta:5,right:[0,1,2,3,5,6,7,8,9,10,12,13,14,16,18],right_sid:2,rightarrow:[0,1,5,6,8,11,12,13,18],rigor:0,ring:6,rise:0,risk:[0,13],rival:4,river:0,rm:[0,18],rmse:0,rmsporp:13,rmsprop:[1,3,4,13],rnd_clf:10,rng:18,rnn1:4,rnn2:4,rnn:[4,12,17],rnn_2layer:4,rnn_input:4,rnn_output:4,rnn_train:4,rntrick1:18,rntrick2:18,rntrick3:18,rntrick4:18,ro:[0,13],robert:20,robust:0,robustscal:0,roc:10,role:[0,2,5,6,8,15],roll:6,room:[0,19],root:[0,5,9,13,18],rotat:[1,8,9,10],rotation_matrix:9,roughli:[1,3],round:[0,7,9,13],routin:[13,16],row:[0,1,2,5,6,7,9,11,16],rr:5,rrr:5,rthe:5,rug:13,rule:[0,1,5,6,13],run:[0,1,2,3,4,5,6,8,9,11,13,15],runtim:[1,6,14],runtimewarn:[1,6],rust:[0,15,16],rustad:19,rvert:1,rvert_2:1,s:[0,1,2,3,4,5,6,7,9,11,12,13,15,16,17,18,19],s_1:6,s_:[3,6],s_i:[6,7],s_j:6,s_k:6,saddl:13,safe:[],sai:[0,1,2,3,4,5,6,7,8,9,10,11,12,16,18],said:[6,9,13],sake:[0,5,7,11],sale:0,same:[0,1,2,3,4,5,6,8,9,11,12,13,14,16,18],samm:10,sampl:[0,1,2,3,4,5,6,7,8,9,10,13,14,15,16,18],sample_vari:14,sample_weight:[3,4],sampleexptvari:18,sastri:11,satisfactori:0,satisfi:[1,2,3,6,8,13,16,18],satur:[1,6],save:[0,4,6,7,9],save_fig:[0,6,7,9,10],savefig:[0,4,6,7,9,18],savetxt:4,saw:5,scalabl:10,scalar:[2,5,6,10,13],scale:[0,1,3,5,6,7,8,9,10,11,12,13,15,16,19],scale_mean:4,scale_std:4,scalei:[],scaler:[0,7,8,9,10,11],scalex:[],scan:[5,7],scari:5,scatter:[0,1,6,7,8,9,14],scenario:[6,13],schedul:13,scheme:[1,13],schrage:18,scienc:[0,1,10,12,13,15,17,18,20],scientif:[0,15],scientist:0,scikit:[3,5,6,7,8,9,10,13,15,16,17,20],scikitplot:[7,10],scipi:[0,3,5,6,13,15,16],scl:6,score:[0,1,3,6,7,9,10,11,19],scores_kfold:6,scratch:1,sdg:13,seaborn:[0,1,3,6,7],seamless:[0,15],search:[0,1,3,5,9,13],sec:6,second:[0,2,3,4,5,6,7,8,9,11,12,13,14,15,16,18],secondeigvector:11,secondli:12,section:[4,11,16,17,18],sector:0,see:[0,1,2,3,4,5,6,7,8,10,11,12,13,14,15,16,17,18],seed:[0,1,2,3,4,5,6,8,9,11,13,14,18],seed_imag:4,seek:[1,2,8],seem:[1,3,4],seemingli:0,seen:[0,1,3,5,10,12,18],segment:13,seismic:6,seldomli:0,select:[1,5,6,8,9,10,11,17,18,20],self:[1,3,4,5,20],sell:4,semest:[7,17],semi:[8,13],semilogx:6,send:[5,12,13,19],senior:17,sens:[0,4,6,8],sensibl:3,sensit:[0,5,6,9,13],sent:2,sentenc:[4,12],sep:[],separ:[0,1,2,4,6,8,9,12,14,15,18],sequenc:[2,3,4,7,9,10,12,13,15,16,18],sequenti:[1,3,4,10,12,18],seri:[0,1,2,3,4,5,6,10,11,12,13,16,17],serif:[0,7,18],serv:[0,1,2,3,5,7,13,20],session:[1,17],set:[1,4,5,6,7,8,10,11,13,14,15,16,18],set_major_formatt:6,set_major_loc:6,set_stream:5,set_tick:[1,8],set_ticklabel:1,set_titl:[0,1,2,3,7,12,14],set_xlabel:[0,1,2,3,7,12],set_xlim:[7,12],set_xticklabel:1,set_ylabel:[0,1,2,3,7],set_ylim:[7,12],set_ytick:7,set_yticklabel:[1,6],set_zlim:6,seth:4,setiosflag:[],setminu:6,setosa:[8,9],setosa_or_versicolor:8,setp:6,setprecis:[],setse:5,setup:[1,4,5,6,8,15],setw:[],sever:[0,3,5,6,7,8,9,11,12,13,15,16,17,18],sgd:[1,3],sgd_clf:8,sgdclassifi:8,sgdreg:13,sgdregressor:13,sgn:5,sh:[],shallow:13,shape:[0,1,2,3,4,5,6,7,8,9,10,11,13,14,16],share:[1,3],she:7,shift:[1,6,12,18],ship:3,shortcom:13,shorten:4,shorter:18,shortli:16,should:[0,2,3,5,6,8,9,11,12,13,16,18],should_sync:[3,4],show:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,18],show_shap:4,shown:[0,4,5,7,8,11,12,13,16],showpoint:[],shrink:[3,5,6,8,11],shrinkag:[5,6],shrunk:11,shuffl:[0,1,3,4,6,13],side:[0,2,5,8,12,13,16],sigh:15,sigma0:18,sigma1:18,sigma2:18,sigma:[0,1,5,6,7,10,11,12,13,16,18],sigma_0:5,sigma_1:5,sigma_2:5,sigma_:[5,16],sigma_fn:[7,12],sigma_i:[0,5],sigma_j:5,sigma_m:[6,18],sigma_n:[11,18],sigma_t:13,sigma_x:18,sigmoid:[1,2,4,7,8,10,12,13],sigmundson:[6,19],sign:[1,2,7,8,10,18],signal:[1,3,10,12],signatur:[3,4],signifi:4,signific:1,significantli:[1,13,18],sigurd:19,sim:[4,5,6,13,18],similar:[0,1,2,3,4,5,6,7,8,9,10,11,13,14,15,16],similarli:[0,1,3,5,8,10,18],simpl:[1,2,3,5,6,7,8,10,11,12,14,15,16,18],simple_rnn:4,simplepredict:10,simpler:[0,1,5,6,13,15],simplernn:4,simplest:[0,1,3,4,9,10,12,14],simpletre:10,simpli:[0,1,2,4,5,6,8,9,10,11,12,15,16,18],simplic:[2,5,6,7,8,9,10,11,12,14],simplicti:5,simplifi:[0,6,9,15],simplist:[3,6,18],simul:6,simultan:6,sin:[0,1,2,3,4,9,12,13,16],sinc:[0,1,2,3,5,6,7,8,9,10,11,13,16,18,20],sine:[3,12],singl:[0,1,2,3,5,6,7,8,9,12,13,16,18],singular:[0,6,13,16,17],sinusoid:3,site:[0,1,2,3,4,6,7,8,11,13,14,17],situat:[0,4,5,7,13,18],six:[3,18],size:[0,1,2,3,4,5,6,8,9,10,11,13,16,18],sketch:10,ski:9,skill:0,skip:[3,4,11],skiprow:[],skl:[0,6],sklearn:[0,1,3,5,6,7,8,9,10,11,13,14],skplt:[7,10],sl:6,slack:8,slice:[2,16],slide:[0,3,18],slight:[6,13],slightli:[1,2,3,5,6,7,10,18],slope:[8,11,12],slow:[0,2,8,13],slower:[5,16],slowest:16,slowli:12,slp:1,small:[0,1,2,3,5,6,8,9,10,11,12,13,15,16,18],smaller:[0,1,2,5,6,8,9,11,13,18],smallest:[0,4,14],smallest_row_index:14,smooth:[0,3,6,13],sn:[0,1,3,6,7],sne:11,so:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,18],soar:6,social:0,soft:[1,7,10,12],soften:8,softmax:[3,7],softwar:[0,8,15,16,17],sol:8,sole:[0,6],solid:[0,7],solitem:5,soltyp:5,solut:[0,1,2,3,5,6,8,10,11,13,16,18],solutionsummari:5,soluton:2,solv:[0,1,3,5,6,8,10,11,12,13,16,17],solve_expdec:2,solve_ode_deep_neural_network:2,solve_ode_neural_network:2,solve_pde_deep_neural_network:2,solveod:2,solveode_popul:2,solver:[2,5,7,8,9,10,11,16],some:[0,1,2,3,4,5,6,7,8,9,10,11,12,14,17,18],some_model:6,somehow:4,someth:[0,1,3,4,7,9,11,18],sometim:[0,1,11,12,13,14],soon:16,sophist:0,sopt:13,sort:[5,6,9,11,18],sound:[3,5],sourc:[0,1,3,6,15,16,18],space:[0,1,4,5,8,9,11,12,13,14,18],span:[0,3,5,9,11,16],spare:1,spars:[3,5,6,16],sparse_mtx:16,sparsecategoricalcrossentropi:3,sparsiti:10,spatial:[1,2,3,12],spdiag:5,speak:18,special:[6,7,10,12,13,16,18],specif:[0,1,2,3,4,5,6,7,8,9,11,12,15,16,18],specifi:[0,3,5,6,7,9,11,13,14,18],specifici:[0,10],spectacular:3,spectral:1,speech:[0,1,3,4,12],speed:[1,2,4,13],spend:18,sphere:0,spin:6,spite:0,spline:8,split:[1,3,4,5,6,8,9,10,11,14,18],splite:0,splitter:[1,10],spmatrix:5,spontan:18,spot:3,spread:[0,11,18],springer:20,spuriou:13,sqquar:5,sqrsignal:3,sqrt:[0,3,4,5,6,8,10,11,13,18],squar:[1,2,3,4,7,8,9,11,13,14,15,16,17,18],squarederror:10,squaredeuclidean:14,squash:12,squeez:2,srand:[],srtm:6,srtm_data_norway_1:6,sse4:[],stabil:5,stabl:[0,4,5,6,7,9,11,15],stack:[2,3,4],stacklevel:0,stage:[5,13],stai:[0,2,4,5,11],stand:[0,5,9,12],standard:[0,1,4,5,6,7,8,10,12,16,18],standard_basi:2,standardscal:[0,6,7,8,9,10,11],stanford:13,start:[0,1,2,3,4,5,6,8,9,10,11,12,13,14,16,17,18],start_box:[2,13],start_nod:[2,13],start_tim:14,startpoint:[],stat:6,state:[1,2,4,5,6,7,8,10,11,12,13,15,18],statement:[0,7,16],statist:[0,1,3,4,7,9,10,11,12,13,14,16,17,20],statu:[0,7,11],stavang:6,std:[0,4,6],stdev:[],stdout:5,steep:13,step:[0,1,2,3,4,6,7,9,10,11,12,13,14,16],step_fn:[7,12],step_length:13,steps_list:9,steps_per_epoch:[3,4],stereo:3,stian:19,still:[2,3,5,6,11,13,18],stimuli:12,stk2100:20,stk3155:17,stk4021:20,stk4051:20,stk4155:17,stk5000:20,stk:20,stochast:[0,1,5,6,8,11,12,17],stock:4,stoke:12,stone:[0,7],stop:[1,4,7,9,11,13,14],storag:5,store:[0,1,2,3,6,11,13,18],storehaug:19,str:[1,3,4],straight:[0,6,8,13],straightforward:[0,2,3,5,6,8,9,10,13,16],strategi:[0,1,9],stratifi:6,streamtyp:5,strength:[0,5,14],stretch:11,strict:[8,13],strictli:[8,13],stride:[4,16],strike:6,string:1,stroke:7,strong:[3,6,9,10,12,16,18],strongli:[0,8,15,16],stronli:0,structur:[0,1,2,3,6,9,10,12,15],stuck:[1,13],student:[0,17,19,20],studi:[0,3,4,5,6,7,8,11,12,13,15,20],studier:[17,20],style:[0,7,9,16],sub:[9,12],subarg:[2,13],subdivid:[0,16],subfield:0,subject:[5,6,8,18],subplot:[0,1,3,4,6,7,8,9,10,13,14],subplots_adjust:[8,18],subprogram:16,subract:0,subroutin:0,subscript:1,subsequ:[1,4,5,6,12,16,18],subset:[1,6,9,12,13,15],subspac:[0,8,11],substanti:[9,10],substep:11,substitut:[3,6,12,16],subsubset:9,subtask:6,subtl:1,subtract:[0,4,5,6,11,13,16,18],subtre:9,subval:[2,13],succeed:[0,4],success:[3,7,9,13,18],successfulli:[4,9],sucess:[],sudo:[0,15],suffer:[0,1,2,5,10],suffici:[1,6,8,11,13],suggest:[1,13,20],suit:[8,12],suitabl:[0,18],sum:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,18],sum_:[0,1,2,3,5,6,7,8,9,10,11,12,13,14,16,18],sum_i:[2,5,6,8,13],sum_j:6,sum_k:[6,8,12,16],sum_m:3,sum_n:3,sum_nx_:3,summar:[5,6,9],summari:[1,3,4,10,17],summat:3,sunni:9,superfici:3,superscript:[1,12],supervis:[0,5,6,7,9,12,15],supplement:7,support:[0,1,9,10,11,13,15,17],suppos:[0,5,6,7,8,10,11,12,13,16],suppress:[5,13],sure:[1,4,6],surf:6,surfac:[0,6],surpass:6,surpris:0,surround:[3,15],survei:[0,5,6],svc:[8,9,10],svd:[0,6,11,17],svdinv:5,svm:[8,9,10,11],svm_clf:[8,10],swap:2,swapax:2,swath:5,sy:[5,13],symbol:[1,5,11,13,15,18],symmeteri:1,symmetr:[0,5,8,11,12,13,16],symmetri:6,sympi:[0,15],synonim:18,syntax:[1,13],syntaxerror:1,syrk:5,system:[0,1,3,4,5,6,7,9,10,12,13,15,16,20],systemat:[4,6],t0:[3,6,13],t1:[2,13],t2:2,t3:2,t:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,18,19],t_0:[2,9,13],t_1:13,t_:2,t_b:10,t_i:[1,2,12],t_j:[5,12],t_k:9,tabl:[9,18,19],tabul:0,tackl:4,tag:[2,3,4,5,6,7,12,13,14,16,18],taht:0,tail:18,tailor:[2,8,11],taiwan:0,take:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,15,16,18],taken:[0,1,3,6,10,13,16],tan:3,tangent:[1,4,12,13],tanh:[1,4,7,8,12,13],tape:[3,4],target:[0,1,3,4,5,6,7,8,9,10,11,12,13],target_nam:9,task:[0,1,3,5,6,9,11,12,14],tau:[3,5,18],tax:0,taylor:[2,13],taylornr:13,tc:8,team:1,teaser:0,technic:[0,5,6,13],techniqu:[0,1,8,10,13,15,17,18,20],technolog:[0,1],tek5040:20,tell:[0,4,6,10,11,13,18],temp1:1,temp2:1,temp:1,temperatur:[0,9],temporarili:1,ten:3,tend:[3,5,6,8,9,10,12,13,14],tendenc:0,tension:6,tensor:[3,4],tensorflow:[0,2,4,8,14,15,16,17,20],term1:[5,6,11],term2:[5,6,11],term3:[5,6,11],term4:[5,6,11],term:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,18],termin:[0,4,5,9,10,13],terrain1:6,terrain:6,test:[3,4,5,6,7,8,9,10,13,14,16,18],test_acc:3,test_accuraci:[1,3],test_error:6,test_imag:[3,4],test_ind:6,test_input:4,test_label:[3,4],test_loss:3,test_pr:1,test_predict:1,test_rnn:4,test_scor:[7,10],test_siz:[0,1,3,5,6,10],test_split:9,testerror:[0,6],testi:4,testpredict:4,testx:4,text:[0,1,2,4,5,8,9,11,13,16,17,18,20],textual:9,textur:1,tf:[1,3,4,13,14],tfe_py_execut:[3,4],th:[0,1,2,5,6,7,9,12,13,14,16,18],than:[0,1,2,3,4,5,6,7,9,10,11,12,13,15,18],thank:[4,6],theano:[1,15],thei:[0,1,2,3,4,5,6,7,8,9,11,12,13,16,18],them:[0,1,3,4,6,8,9,10,11,12,13,16],theme:0,themselv:[0,18],thenc:6,theorem:[2,6,7],theoret:[0,4,10],theori:[0,1,3,8,9,12,13,15,17,20],thereaft:[0,5,6,11,12,16],therebi:[0,5,7,11],therefor:[0,1,2,3,4,6,7,8,11,13,18],therein:11,thereof:[0,6,13],theta:[1,4,13,18],theta_:[1,13],theta_i:1,theta_k:[],theta_linreg:13,theta_t:13,thi:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,20],thing:[0,1,2,4,5,7,9,18],think:[0,1,3,4,6,9,12,13,14,18],third:[0,3,6,13],thirti:7,those:[3,5,6,8,9,10,11,16,17],though:[1,2,3,4,13,16,18],thought:[6,14,18],thousand:[0,1],thread:[3,4],three:[0,1,3,5,6,8,9,12,16,17,18,19],threshold:[1,3,9,10,11,12,13],through:[0,1,2,3,4,5,6,8,11,12,13,14,15,16,18],throughout:[0,4,5,14,15,16,18],thu:[0,1,2,5,6,7,8,10,11,12,13,19],thumb:[0,6],thursdai:17,tibshirani:[6,17,20],tick_param:6,ticker:[6,13,18],tif:6,tight_layout:[1,7],tightli:11,tild:[0,5,6,11,18],till:[0,4,7,8,9,10,12,16],time:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,17,18],timefunct:[],timeit:4,timer:4,tini:1,tip:3,titl:[0,1,2,3,4,6,7,8,9,10,13,18],tmp:13,tmp_log:[3,4],tn:[2,3],to_categor:[1,3,4],to_categorical_numpi:1,to_numer:[0,6],to_str:[],todai:3,togeth:[0,3,6,8,11],toi:[13,14],told:13,toler:[2,6,14],tolist:4,tomographi:12,too:[0,2,4,5,6,9,11,13,18,20],took:8,tool:[0,1,3,6,13,15],toolbox:8,top:[0,3,5,6,9,10,15],top_node_typ:2,top_trac:2,topic:[0,5,6,7,8,15,17,20],topolog:[1,3,12],toposort:2,torkjellsdatt:19,toss:[10,18],total:[0,1,2,3,4,6,7,8,10,11,12,13,14,16,18,19],total_loss:4,totalclustervari:14,totalscatt:14,totalvari:[],toward:[1,2,7,12,13],town:0,tp:4,tpng:9,tqdm:6,trace:[2,13],trace_stack:[2,13],traceback:[0,2,3,4,5,6,7,8,9,10,13],traceback_util:[3,4],tracer:[2,13],track:[3,13,14,16],tract:0,tractabl:0,trade:[5,9],tradeoff:[0,5,17],tradit:[0,1,4,6],train:[2,3,5,6,8,9,10,11,12,13],train_accuraci:[0,1,3],train_dataset:4,train_end:[0,1],train_error:6,train_funct:[3,4],train_imag:[3,4],train_ind:6,train_label:[3,4],train_pr:1,train_siz:[0,1,3],train_step:4,train_test_split:[0,1,3,5,6,7,9,10,11],train_test_split_numpi:[0,1],trainabl:4,trainable_vari:4,trained_model:6,trainerror:0,traini:4,training_checkpoint:4,training_dataset:4,training_gradi:13,training_gradient_fun:13,training_loss:13,trainingerror:6,trainpredict:4,trainscor:4,trainx:4,trait:0,trajectori:4,tran:5,transfer:9,transform:[0,5,6,7,8,9,10,11,12,13,15,16],transit:[6,12],translat:[1,4,6,10],transpos:[1,5,11,16],travers:[0,5],treat:[0,1,3,6,12,13,18],tree:[0,1,6,15,17],tree_clf:[9,10],tree_clf_:9,tree_clf_sr:9,tree_reg1:9,tree_reg2:9,tree_reg:9,trend:18,trevor:20,tri:[2,3,4,9,13],triain:0,trial:[0,2,4,6,13,18],triangl:13,triangular:16,trick:[3,4,8,11,13,18],trickier:18,tridiagon:16,trillion:15,trivial:[0,1,5,11,18],troubl:[0,8,12],truck:3,true_beta:6,true_divid:1,true_fun:6,tucker:8,tumor:[7,9],tumour:7,tunabl:1,tune:[4,9,13,16],tup:[],tupl:[2,13],turn:[0,1,5,6,7,8,9,10,11,12,13,16,18],tutori:[1,4],tv:2,tveito:2,tweak:[1,4,10,18],twice:13,twist:11,twister:[],two:[0,1,2,4,5,6,7,9,10,11,12,13,16,17,18,20],tx:13,tx_1:13,txt:4,ty:13,type:[0,1,3,6,8,10,13,16,18],typeerror:[2,13],typic:[0,1,2,3,4,5,7,9,10,12,13,18],u:[0,2,5,6,10,11,12,16],u_:16,u_i:12,u_m:10,ua:0,ubuntu:[0,15],uci:0,uio:[17,19,20],un:14,unari:16,unary_f:[2,13],unary_oper:[2,13],unary_to_nari:2,unbalanc:[6,9],unbias:[0,5,6],uncent:6,uncertainti:[0,5],uncertitud:18,unchang:[1,3],uncorrel:[10,18],undefin:5,under:[0,1,5,6,10,13,15],underdetermin:0,underfit:[1,6],underflowproblem:5,undergo:5,undergradu:17,underli:[0,1,9,13,18],underset:[4,14],understand:[0,1,3,5,6,10,13,14,15],understood:[8,13],undesir:8,undetermin:[5,8],undo:4,unexpect:6,unexpected:18,unfair:6,unfortun:[1,8,9,10],unicode_liter:[8,9],uniform:[0,1,5,6,11,13,18],uniform_real_distribut:[],uniformli:[13,18],unifrompdf:18,unimport:13,union:[5,6],uniqu:[0,2,6,13,14,16],unique_cluster_label:14,unit:[0,1,3,4,5,10,12,18],unitari:[5,6,16],unitarili:16,uniti:18,univari:18,univers:[0,1,2,13,15,17,19],unix:1,unknow:[0,16],unknown:[0,1,3,4,5,6,8,10,16],unknowwn:12,unlabel:1,unless:[0,3,6,11,13],unlik:[1,3,8,13],unnecessarili:9,unord:3,unravel:1,unrol:[3,11],unseen:[0,7,9],unstabl:1,unsupervis:[0,1,4,12,15,17],unsymmetr:16,until:[1,2,4,9,12,13,14],untouch:0,unusu:12,up:[1,3,4,5,6,8,10,11,13,14,15,16,17,18],updat:[1,2,10,12,13,14],upload:[15,20],upon:[1,6,11,16],upper:[0,8,9,16],uppercas:16,upsampl:4,upscal:4,us:[4,5,6,8,9,10,11,12,14,16,17,18,20],usag:[0,8,15],usd10000:0,usd:0,use_bia:4,use_multiprocess:[3,4],usecol:0,useless:1,user:[0,1,2,3,4,6,7,8,11,14,15,16],userwarn:[3,4,6,14],usetex:18,usg:6,usr:18,usual:[0,3,4,7,12,13,14],ut:5,util:[0,1,3,4,6,7,10,14],ux:16,v0:18,v1:18,v2:18,v:[2,4,5,6,11,13,15],v_0:11,va:1,val:13,val_accuraci:3,val_loss:4,vale:2,valid:[0,1,4,7,9,10,13,15,17,18],validation_batch_s:[3,4],validation_data:[3,4],validation_freq:[3,4],validation_split:[3,4],validation_step:[3,4],valu:[0,1,2,3,4,6,7,8,9,10,12,13,14,15,16,17],valuat:9,valueerror:[0,2],valy:4,van:0,vandenbergh:[8,13],vandermond:0,vanilla:[0,6,11,14],vanish:[1,4,13,18],var_x:18,varabl:8,varepsilon:[5,6],varepsilon_:[5,6],varepsilon_i:[5,6],vari:[0,1,3,5,6,10],variabl:[0,1,2,3,4,5,6,7,8,10,11,12,13,14,16],varianc:[0,1,5,7,9,10,11,13,14,15,16,17,18],variance_i:[5,11],variance_x:[5,11],variant:[0,1,6,8,12,13],variat:[3,4,11],varieti:[0,3,12,15],variou:[1,3,5,6,7,8,9,11,12,13,15,16,18],vartempvec:[],varvec:[],varydimens:4,vastli:3,vaue:1,vault:0,vdot:[2,13],vec:6,vector:[0,1,2,3,4,5,6,7,9,10,11,13,14,15,17],vector_mean:14,ventur:[0,8,15],verbos:[1,3,4],veri:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,18,20],verifi:[3,11,16],versatil:8,versicolor:[8,9],version:[0,3,4,10,13,14,15,16,18],versu:1,vert:[0,1,5,6,7,8,9,11,13],vert_1:[5,6],vert_2:[5,6,11],via:[0,5,6,7,8,9,10,11,12,15,16,17,18],vidal:11,video:[0,1,12,15,17],view:[1,3,5,6,12,13,17,18,20],violat:8,virginica:9,viridi:[0,1,2,3],virtual:1,viscos:13,viscou:13,visibledeprecationwarn:2,vision:[0,3],visual:[0,3,11,12,15],visualis:1,viz:[6,8,18],vjp:[2,13],vjp_0:[],vjp_0_fun:[],vjp_1:[],vjp_1_fun:[],vjp_argnum:[],vjpfun:2,vjpmaker:[],vjpnode:[2,13],vmax:[1,6],vmc:[],vmin:[1,6],voic:3,volum:[0,3],vote:10,voting_clf:10,votingclassifi:10,votingsimpl:10,vrtx:17,vs:[0,4,6],vspace:[2,13],vstack:[5,11,16,18],vt:5,w1:8,w2:[8,11],w3:8,w:[0,1,2,3,4,5,6,7,8,10,11,12,13,14,16,18],w_1:[8,16],w_1x_1:8,w_1x_:8,w_2:[8,16],w_2x_2:8,w_2x_:8,w_3:16,w_4:16,w_:[1,12],w_hidden:2,w_i:[1,2,10],w_ix_i:12,w_j:16,w_m:16,w_output:2,w_px_:8,w_px_p:8,wa:[0,1,3,4,5,6,7,10,11,12,13,14,16],wai:[0,1,2,3,4,5,6,7,8,10,11,12,13,14,16,18],walk:9,walker:18,wang:0,want:[0,1,2,3,4,5,6,8,9,10,11,12,13,14,15,18],warn:[1,3,4,8,14],warrant:6,wast:3,watch:[3,4,15],wave:3,wavelet:8,we:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,20],weak:[9,10,14],weather:[1,12],web:[15,17],webpag:17,websit:[6,16,17],wedg:[8,18],wednesdai:17,wee:11,week:[5,6,7],weekli:[15,20],weight:[0,1,2,3,6,7,9,10,12,13,18],weigth:2,welcom:[8,15],well:[0,1,2,3,4,5,6,7,8,9,10,12,13,15,16,17,18,20],went:8,were:[0,1,3,4,5,6,7,8,10,11,12,14,18],wessel:0,westbi:19,westby:19,what:[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18],whatev:3,when:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,18],whenev:[13,18],where:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,18,19],wherea:[6,18],wherein:[1,12],whether:[0,3,5,7,9,18],which:[0,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19],whichev:[1,3],white:9,who:[0,17],whole:[1,3,4,5,9,11,13],whose:[0,6,10,18],whow:[5,11],why:[0,1,3,6,13],wide:[0,1,3,6,7,12,15,16],widehat:6,width:[0,3,8,9],wieringen:0,win:10,wind:9,wing:19,winther:2,wiothout:6,wiscons:7,wisconsin:10,wisdom:6,wise:[0,1,5,12,13],wish:[0,2,5,7,8,11,13,14,16],with_std:0,wither:6,within:[0,2,3,4,7,9,12,13,14,18,20],withinclust:14,without:[0,1,5,6,8,9,11,12,13],won:0,wonder:8,word:[0,1,3,4,5,6,14,18],work:[0,1,4,6,7,8,9,13,15,17,18],worker:[3,4],workshop:17,world:[0,8],worldwid:0,wors:[0,1,3,4,6],worst:[],worth:9,would:[0,1,3,5,6,7,8,9,10,11,12,13,16,18],wrap:[2,6,16,17],wrap_util:[2,13],wrapper:0,write:[0,1,2,3,5,6,7,8,12,13,16],written:[0,2,3,5,11,12,13,15,16,18],wrong:[1,8],wrongli:10,wrote:[5,11],wrt:[2,10,13],wth:10,www:[15,16,17,20],wx_1:8,x0:8,x1:[4,8,9,10,13],x1_exampl:8,x1d:8,x2:[8,9,10,13],x2d:[8,11],x2d_train:11,x2dsl:11,x3:8,x:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,18],x_0:[0,5,11,16],x_1:[0,2,5,6,7,8,9,10,11,13,16,18],x_2:[0,2,5,6,7,8,9,10,11,13,16,18],x_3:[8,16,18],x_4:16,x_:[0,2,3,5,6,8,10,11,13,14,16,18],x_center:11,x_data:1,x_data_ful:1,x_hidden:2,x_i:[0,1,2,5,6,7,8,9,10,11,12,13,14,16,18],x_input:2,x_ix_:0,x_iy_i:8,x_j:[0,2,8,9,12,18],x_jy_j:8,x_k:[12,14,16,18],x_l:18,x_m:[6,12,16,18],x_n:[0,2,3,6,8,11,12,13,16,18],x_new:[9,10],x_offset:6,x_output:2,x_p:[3,7,9],x_poli:9,x_poly10:9,x_pred:4,x_prev:2,x_reduc:11,x_scale:8,x_test:[0,1,3,5,6,7,9,10,11],x_test_own:6,x_test_scal:[0,6,7,9,10,11],x_tot:4,x_train:[0,1,3,4,5,6,7,9,10,11],x_train_mean:6,x_train_own:6,x_train_scal:[0,6,7,9,10,11],x_val:1,xarrai:15,xavier:1,xbnew:13,xcode:[0,15],xdclassiffierconfus:10,xdclassiffierroc:10,xg_clf:10,xgb:10,xgbclassifi:10,xgboost:9,xgboot:10,xgbregressor:10,xgparam:10,xgtree:10,xi:[8,13],xi_1:8,xi_:8,xi_i:8,xk:8,xlabel:[0,1,2,3,4,5,6,7,8,9,10,13,18],xlim:[6,10],xm:9,xmesh:13,xnew:[0,13],xp:18,xpanda:0,xpd:[5,11],xplot:0,xs:9,xscale:0,xsr:9,xt_x:13,xtest:6,xtick:[3,6,8,9],xtrain:6,xu:0,xx:[0,5,16],xy:[0,6,8,16],xytext:8,xz:16,y1:4,y2:4,y3:4,y:[0,1,3,4,5,6,7,8,9,10,11,12,13,14,16,18],y_0:[0,5,11,16],y_1:[0,5,8,9,11,13,16],y_1y_1:8,y_1y_1k:8,y_1y_2:8,y_1y_2k:8,y_1y_n:8,y_1y_nk:8,y_2:[0,5,8,9,11,16],y_2y_1:8,y_2y_1k:8,y_2y_2:8,y_2y_2k:8,y_3:[0,9,16],y_4:16,y_:[0,1,5,6,10,11,16],y_data:[0,1,5,6],y_data_ful:1,y_decis:8,y_fit:0,y_i:[0,1,5,6,7,8,9,10,11,12,13,16],y_if_:10,y_ix_:0,y_ix_i:[7,8,13],y_iy_jk:8,y_j:[6,8,12],y_k:12,y_m:16,y_model:[0,4,5,6],y_n:[8,13],y_ny_1:8,y_ny_1k:8,y_ny_2:8,y_ny_2k:8,y_ny_n:8,y_ny_nk:8,y_offset:6,y_plot:9,y_pred1:9,y_pred2:9,y_pred:[0,1,4,6,7,8,9,10],y_pred_rf:10,y_pred_tre:10,y_proba:[7,10],y_scaler:6,y_test:[0,1,3,4,5,6,7,9,10,11],y_test_onehot:1,y_test_predict:0,y_tot:4,y_train:[0,1,3,4,5,6,7,9,10,11],y_train_mean:6,y_train_onehot:1,y_train_predict:0,y_train_scal:6,y_val:1,ye:[3,6,7],year:[0,15],yet:[0,1,6,8,11,13],yi:13,yield:[0,2,5,6,8,10,12,13,14,16,18],yk:8,ylabel:[0,1,2,3,4,5,6,7,8,9,10,13,18],ylim:[3,6],ym:9,ymesh:13,yn:0,yo:[8,9,10],yoshua:[1,20],you:[0,1,2,3,4,5,6,8,9,10,11,13,15,16,18,20],young:0,your:[1,2,4,5,6,8,11,13,15,16],yourself:[11,13],youtub:15,ypred:6,ypredict2:13,ypredict:[0,13],ypredictlasso:5,ypredictol:5,ypredictown:6,ypredictownridg:6,ypredictridg:[5,6],ypredictskl:6,ys:9,ytest:6,ytick:[3,6,8,9],ytild:[0,6],ytildelasso:5,ytildenp:0,ytildeol:5,ytildeownridg:6,ytilderidg:[5,6],ytrain:6,yx:16,yy:16,yz:16,z:[0,1,2,3,4,5,6,7,8,9,11,12,13,16,18],z_0:16,z_1:16,z_2:16,z_:[1,2,12,16],z_c:1,z_h:1,z_hidden:2,z_i:[1,12],z_j:[1,12],z_k:12,z_m:1,z_mod:9,z_o:1,z_output:2,zaman:18,zaxi:6,zero:[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,18],zeros_lik:4,zfill:4,zip:[2,4,6,13],zl:5,zm_h:0,zn:0,zone:0,zx:16,zy:16,zz:16},titles:["3. Linear Regression","14. Building a Feed Forward Neural Network","15. Solving Differential Equations with Deep Learning","16. Convolutional Neural Networks","17. Recurrent neural networks: Overarching view","4. Ridge and Lasso Regression","5. Resampling Methods","6. Logistic Regression","8. Support Vector Machines, overarching aims","9. Decision trees, overarching aims","10. Ensemble Methods: From a Single Tree to Many Trees and Extreme Boosting, Meet the Jungle of Methods","11. Basic ideas of the Principal Component Analysis (PCA)","13. Neural networks","7. Optimization, the central part of any Machine Learning algortithm","12. Clustering and Unsupervised Learning","Applied Data Analysis and Machine Learning","2. Linear Algebra, Handling of Arrays and more Python Features","Teaching schedule with links to material","1. Elements of Probability Theory and Statistical Data Analysis","Teachers and Grading","Textbooks"],titleterms:{"1":[],"10":17,"11":17,"12":17,"13":[],"14":17,"15":[],"16":17,"17":17,"18":17,"19":17,"2":[0,17],"20":[],"2021":[],"2022":19,"21":17,"22":17,"23":17,"24":17,"25":17,"26":17,"27":[],"28":17,"29":17,"3":17,"30":17,"31":17,"34":17,"35":17,"36":17,"37":17,"38":17,"39":17,"4":17,"40":17,"41":17,"4155":[],"42":17,"43":17,"44":17,"45":17,"46":17,"47":17,"5":17,"6":[],"7":17,"8":[],"9":17,"case":[8,10,18],"do":1,"final":12,"function":[0,1,6,7,8,10,11,12,13,18],"import":[5,16],"new":4,A:[0,1,4,8,9],And:13,Ising:6,The:[0,1,2,3,5,6,7,8,9,11,12,15],With:4,activ:[1,12],actual:[],ad:6,adaboost:10,adam:13,adapt:10,adjust:1,adversari:4,again:[3,9],aim:[8,9],algebra:16,algorithm:[9,10,11,12],algortithm:13,all:8,an:[0,4,10],analys:5,analysi:[0,5,6,11,15,18],ani:13,anoth:9,appli:15,approach:[0,8,14],approxim:12,architectur:1,arrai:16,assist:19,august:17,autocorrel:18,autograd:[2,13],automat:13,back:[1,11,12],background:15,bag:10,base:13,basic:[0,5,7,9,10,11,16],batch:1,bay:5,befor:11,better:8,bia:6,binari:1,binomi:[],bird:10,block:[],boost:10,bootstrap:[6,10],boston:0,breast:1,bring:12,build:[1,3,9],calcul:[],cancer:[1,7,9,11],cart:9,central:[13,15,18],chain:12,chang:10,chi:0,choos:1,cifar01:3,classic:11,classif:[1,9,10],classifi:8,clip:1,cluster:14,cnn:3,code:[0,1,2,5,9,11,12,14],collect:[1,3],compar:[2,10],complex:[0,6],complic:6,compon:11,comput:9,con:9,concept:18,conjug:13,continu:[],convex:[8,13],convolut:[3,12],correl:11,cost:[1,10],cours:[15,20],covari:[5,11,18],cross:6,cumul:[],data:[0,1,3,6,7,9,11,15,18],dataset:[1,3],decai:2,decis:[9,10],decomposit:[5,11,16],deep:[1,2],defin:1,definit:[],degre:0,demonstr:[],dens:0,deriv:[5,12],descent:[2,10,13],detail:3,develop:1,deviat:[],diagon:11,dice:[],differ:8,differenti:[2,13],diffus:2,dimension:[2,3,8],disadvantag:9,discret:18,disguis:[],distribut:[5,18],domain:18,down:1,dropout:1,element:[0,18],elimin:16,ensembl:10,entropi:9,environ:0,equat:[0,2,12],error:[0,10],euler:2,evalu:1,event:[],exampl:[0,1,2,3,4,6,7,8,9,10],exercis:[0,6],expect:18,experi:18,explor:0,exponenti:2,extrapol:4,extrem:10,ey:10,fall:19,famili:1,famou:16,featur:[9,16],feed:[1,12],fine:1,first:[4,12],fit:[0,10],forc:3,forest:10,forward:[1,2,12],fourier:3,frank:6,freedom:0,frequentist:0,from:[5,10,12],full:2,further:[3,5],fy:[],gan:4,gaussian:16,gd:13,gener:[4,9],geometr:11,gini:9,good:0,grade:19,gradient:[1,2,10,13],growth:2,ha:15,handl:16,hidden:2,hous:0,how:[],hyperparamet:1,hyperplan:8,i:1,id3:9,idea:11,implement:1,implic:5,improv:1,includ:13,increment:11,index:9,inform:19,input:2,instal:15,instructor:19,interpret:[5,11],introduc:11,introduct:[0,6,15,16],invers:[5,16],iter:10,its:[],jackknif:[],jungl:10,kera:[1,3],kernel:[8,11],lagrangian:8,lasso:[5,6],later:5,layer:[1,2,3,12],learn:[0,1,2,11,13,14,15],least:[5,6],level:10,librari:15,likelihood:7,limit:[1,13,18],linear:[0,8,13,16],link:[5,11,17,20],logist:[7,13],lu:16,machin:[0,8,13,15],main:18,make:[0,9,10],mani:[10,12],materi:17,math:5,mathemat:[3,5,8],matric:[5,16],matrix:[1,5,11,12,16],matter:0,mean:0,meet:[5,10,18],mercer:8,mersenn:[],method:[6,9,10,13],mlp:12,mnist:[3,4],model:[0,1,4,6,12],moment:[],momentum:13,moon:[8,9],more:[3,6,16],multilay:12,multipl:[1,3],multipli:8,name:[],network:[1,2,3,4,7,12],neural:[1,2,3,4,12],non:8,normal:[0,1],norwai:[],notat:12,novemb:17,now:[1,9],nuclear:0,nueral:7,number:[0,2,18],numer:[2,18],numpi:16,object:3,observ:[],obtain:11,octob:17,od:2,off:6,ol:[5,6,13],one:[2,12],oper:16,optim:[1,8,13,15],ordinari:[5,6],organ:0,oslo:20,other:[4,9,11,12,16],our:[0,4,5,11,13],outcom:15,output:2,overarch:[0,4,8,9],overview:10,own:[0,10,11],packag:16,part:[13,15],partial:2,pass:1,pca:11,pdf:18,perceptron:12,perform:[1,9],period:3,perspect:1,point:4,poisson:2,polynomi:3,popul:2,practic:13,pre:[1,3],predict:4,prerequisit:[3,15],princip:11,principl:3,pro:9,probabl:[5,18],problem:[1,2,13],procedur:9,process:[1,3],program:[2,13],project:6,prop:13,propag:[1,12],properti:[5,18],pseudo:[],python:[0,9,15,16],quick:8,ran0:[],random:[10,11,18],read:9,real:6,recip:[],recurr:[4,12],reduc:0,reduct:3,reformul:2,regress:[0,5,6,7,9,10,13],regular:1,relev:20,relu:1,remark:3,remind:[6,8],requir:[2,15],resampl:6,rescal:6,resourc:2,revisit:13,ridg:[5,6],rm:13,rng:[],rule:12,s:[8,10],sampl:11,schedul:17,schemat:9,scheme:2,scikit:[0,1,11],select:[],semest:19,septemb:17,set:[0,2,3,9,12],sgd:13,should:1,simpl:[0,4,9,13],singl:10,singular:[5,11],situat:[],soft:8,softmax:1,solv:2,solver:13,some:[13,16],specifi:2,split:0,squar:[0,5,6,10],standard:13,state:0,statist:[5,6,15,18],steepest:[10,13],stk3155:[],stochast:[13,18],superposit:3,supervis:1,support:8,svd:5,systemat:3,teach:[17,19],teacher:19,techniqu:[6,11],technolog:15,tensorflow:[1,3],test:[0,1],textbook:20,theorem:[5,8,11,12,18],theori:18,three:[],tip:13,togeth:12,top:1,toss:[],toward:11,trade:6,tradeoff:6,train:[0,1,4],transform:3,tree:[9,10],tune:1,two:[3,8,15],type:[2,4,12],uncorrel:[],uniform:[],univers:[12,20],unsupervis:14,up:[0,2,9,12],us:[0,1,2,3,7,13,15],valid:6,valu:[5,11,18],variabl:18,varianc:6,variou:0,vector:[8,12,16],view:[0,4,10],visual:[1,9],vs:3,wai:9,wave:2,week:17,weekli:17,what:0,which:1,why:[],wisconsin:7,write:[4,11],xgboost:10,your:[0,10]}}) \ No newline at end of file diff --git a/doc/LectureNotes/_build/html/statistics.html b/doc/LectureNotes/_build/html/statistics.html index 39d830bd9..e4ddce8bd 100644 --- a/doc/LectureNotes/_build/html/statistics.html +++ b/doc/LectureNotes/_build/html/statistics.html @@ -313,371 +313,48 @@ const thebe_selector_output = ".output, .cell_output" 1.1. Domains and probabilities - -
  • - - 1.2. Tossing the dice - -
    • -
  • - - 1.3. Stochastic variables - -
    • -
  • - - 1.4. Stochastic variables and the main concepts, the discrete case - -
    • -
  • - - 1.5. Stochastic variables and the main concepts, the continuous case - -
    • -
  • - - 1.6. The cumulative probability - -
    • -
  • - - 1.7. Properties of PDFs - -
    • -
  • - - 1.8. Important distributions, the uniform distribution - -
    • -
  • - - 1.9. Gaussian distribution - -
    • -
  • - - 1.10. Exponential distribution - -
    • -
  • - - 1.11. Expectation values - -
    • -
  • - - 1.12. Stochastic variables and the main concepts, mean values - -
    • -
  • - - 1.13. Stochastic variables and the main concepts, central moments, the variance - -
    • -
  • - - 1.14. Probability Distribution Functions - -
    • -
  • - - 1.15. Probability Distribution Functions - -
    • -
  • - - 1.16. The three famous Probability Distribution Functions - -
    • -
  • - - 1.17. Probability Distribution Functions, the normal distribution - -
    • -
  • - - 1.18. Probability Distribution Functions, the normal distribution - -
    • -
  • - - 1.19. Probability Distribution Functions, the cumulative distribution - -
    • -
  • - - 1.20. Probability Distribution Functions, other important distribution - -
    • -
  • - - 1.21. Probability Distribution Functions, the binomial distribution - -
    • -
  • - - 1.22. Probability Distribution Functions, Poisson’s distribution - -
    • -
  • - - 1.23. Probability Distribution Functions, Poisson’s distribution - -
    • -
  • - - 1.24. Meet the covariance! - -
    • -
  • - - 1.25. Meet the covariance in matrix disguise - -
    • -
  • - - 1.26. Covariance - -
    • -
  • - - 1.27. Meet the covariance, uncorrelated events - -
    • -
  • - - 1.28. Numerical experiments and the covariance - -
    • -
  • - - 1.29. Numerical experiments and the covariance - -
    • -
  • - - 1.30. Numerical experiments and the covariance, actual situations - -
    • -
  • - - 1.31. Numerical experiments and the covariance, our observables - -
    • -
  • - - 1.32. Numerical experiments and the covariance, the sample variance - -
    • -
  • - - 1.33. Numerical experiments and the covariance, central limit theorem - -
    • -
  • - - 1.34. Definition of Correlation Functions and Standard Deviation - -
    • -
  • - - 1.35. Definition of Correlation Functions and Standard Deviation - -
    • -
  • - - 1.36. Definition of Correlation Functions and Standard Deviation - -
    • -
  • - - 1.37. Definition of Correlation Functions and Standard Deviation - -
    • -
  • - - 1.38. Definition of Correlation Functions and Standard Deviation, sample variance - -
    • -
  • - - 1.39. Definition of Correlation Functions and Standard Deviation - -
    • -
  • - - 1.40. Code to compute the Covariance matrix and the Covariance - -
    • -
  • - - 1.41. Random Numbers - -
    • -
  • - - 1.42. Random Numbers, better name: pseudo random numbers - -
    • -
  • - - 1.43. Random number generator RNG - -
    • -
  • - - 1.44. Random number generator RNG and periodic outputs - -
    • -
  • - - 1.45. Random number generator RNG and its period - -
    • -
  • - - 1.46. Random number generator RNG, other examples - -
    • -
  • - - 1.47. Random number generator RNG, other examples - -
    • -
  • - - 1.48. Random number generator RNG, RAN0 - -
    • -
  • - - 1.49. Random number generator RNG, RAN0 - -
    • -
  • - - 1.50. Random number generator RNG, RAN0 - -
    • -
  • - - 1.51. Random number generator RNG, RAN0 - -
    • -
  • - - 1.52. Random number generator RNG, RAN0 code - -
    • -
  • - - 1.53. Properties of Selected Random Number Generators - -
    • -
  • - - 1.54. Properties of Selected Random Number Generators - -
    • -
  • - - 1.55. Properties of Selected Random Number Generators - -
    • -
  • - - 1.56. Simple demonstration of RNGs using python - -
    • -
  • - - 1.57. Properties of Selected Random Number Generators - -
    • -
  • - - 1.58. Autocorrelation function - -
    • -
  • - - 1.59. Correlation function and which random number generators should I use - -
    • -
  • - - 1.60. Which RNG should I use? - -
    • -
  • - - 1.61. How to use the Mersenne generator - -
    • -
  • - - 1.62. Why blocking? - -
    • -
  • - - 1.63. Why blocking? - -
    • -
  • - - 1.64. Code to demonstrate the calculation of the autocorrelation function - -
    • -
  • - - 1.65. What is blocking? - -
    • -
  • - - 1.66. What is blocking? - -
    • -
  • - - 1.67. What is blocking? - -
    • -
  • - - 1.68. Implementation - -
    • -
  • - - 1.69. Actual implementation with code, main function - -
    • -
  • - - 1.70. The Bootstrap method - -
    • -
  • - - 1.71. Bootstrapping - -
    • -
  • - - 1.72. Bootstrapping, recipe - -
    • -
  • - - 1.73. Bootstrapping, code - -
    • -
  • - - 1.74. Jackknife, code - +
    • @@ -703,371 +380,48 @@ const thebe_selector_output = ".output, .cell_output" 1.1. Domains and probabilities - -
  • - - 1.2. Tossing the dice - -
    • -
  • - - 1.3. Stochastic variables - -
    • -
  • - - 1.4. Stochastic variables and the main concepts, the discrete case - -
    • -
  • - - 1.5. Stochastic variables and the main concepts, the continuous case - -
    • -
  • - - 1.6. The cumulative probability - -
    • -
  • - - 1.7. Properties of PDFs - -
    • -
  • - - 1.8. Important distributions, the uniform distribution - -
    • -
  • - - 1.9. Gaussian distribution - -
    • -
  • - - 1.10. Exponential distribution - -
    • -
  • - - 1.11. Expectation values - -
    • -
  • - - 1.12. Stochastic variables and the main concepts, mean values - -
    • -
  • - - 1.13. Stochastic variables and the main concepts, central moments, the variance - -
    • -
  • - - 1.14. Probability Distribution Functions - -
    • -
  • - - 1.15. Probability Distribution Functions - -
    • -
  • - - 1.16. The three famous Probability Distribution Functions - -
    • -
  • - - 1.17. Probability Distribution Functions, the normal distribution - -
    • -
  • - - 1.18. Probability Distribution Functions, the normal distribution - -
    • -
  • - - 1.19. Probability Distribution Functions, the cumulative distribution - -
    • -
  • - - 1.20. Probability Distribution Functions, other important distribution - -
    • -
  • - - 1.21. Probability Distribution Functions, the binomial distribution - -
    • -
  • - - 1.22. Probability Distribution Functions, Poisson’s distribution - -
    • -
  • - - 1.23. Probability Distribution Functions, Poisson’s distribution - -
    • -
  • - - 1.24. Meet the covariance! - -
    • -
  • - - 1.25. Meet the covariance in matrix disguise - -
    • -
  • - - 1.26. Covariance - -
    • -
  • - - 1.27. Meet the covariance, uncorrelated events - -
    • -
  • - - 1.28. Numerical experiments and the covariance - -
    • -
  • - - 1.29. Numerical experiments and the covariance - -
    • -
  • - - 1.30. Numerical experiments and the covariance, actual situations - -
    • -
  • - - 1.31. Numerical experiments and the covariance, our observables - -
    • -
  • - - 1.32. Numerical experiments and the covariance, the sample variance - -
    • -
  • - - 1.33. Numerical experiments and the covariance, central limit theorem - -
    • -
  • - - 1.34. Definition of Correlation Functions and Standard Deviation - -
    • -
  • - - 1.35. Definition of Correlation Functions and Standard Deviation - -
    • -
  • - - 1.36. Definition of Correlation Functions and Standard Deviation - -
    • -
  • - - 1.37. Definition of Correlation Functions and Standard Deviation - -
    • -
  • - - 1.38. Definition of Correlation Functions and Standard Deviation, sample variance - -
    • -
  • - - 1.39. Definition of Correlation Functions and Standard Deviation - -
    • -
  • - - 1.40. Code to compute the Covariance matrix and the Covariance - -
    • -
  • - - 1.41. Random Numbers - -
    • -
  • - - 1.42. Random Numbers, better name: pseudo random numbers - -
    • -
  • - - 1.43. Random number generator RNG - -
    • -
  • - - 1.44. Random number generator RNG and periodic outputs - -
    • -
  • - - 1.45. Random number generator RNG and its period - -
    • -
  • - - 1.46. Random number generator RNG, other examples - -
    • -
  • - - 1.47. Random number generator RNG, other examples - -
    • -
  • - - 1.48. Random number generator RNG, RAN0 - -
    • -
  • - - 1.49. Random number generator RNG, RAN0 - -
    • -
  • - - 1.50. Random number generator RNG, RAN0 - -
    • -
  • - - 1.51. Random number generator RNG, RAN0 - -
    • -
  • - - 1.52. Random number generator RNG, RAN0 code - -
    • -
  • - - 1.53. Properties of Selected Random Number Generators - -
    • -
  • - - 1.54. Properties of Selected Random Number Generators - -
    • -
  • - - 1.55. Properties of Selected Random Number Generators - -
    • -
  • - - 1.56. Simple demonstration of RNGs using python - -
    • -
  • - - 1.57. Properties of Selected Random Number Generators - -
    • -
  • - - 1.58. Autocorrelation function - -
    • -
  • - - 1.59. Correlation function and which random number generators should I use - -
    • -
  • - - 1.60. Which RNG should I use? - -
    • -
  • - - 1.61. How to use the Mersenne generator - -
    • -
  • - - 1.62. Why blocking? - -
    • -
  • - - 1.63. Why blocking? - -
    • -
  • - - 1.64. Code to demonstrate the calculation of the autocorrelation function - -
    • -
  • - - 1.65. What is blocking? - -
    • -
  • - - 1.66. What is blocking? - -
    • -
  • - - 1.67. What is blocking? - -
    • -
  • - - 1.68. Implementation - -
    • -
  • - - 1.69. Actual implementation with code, main function - -
    • -
  • - - 1.70. The Bootstrap method - -
    • -
  • - - 1.71. Bootstrapping - -
    • -
  • - - 1.72. Bootstrapping, recipe - -
    • -
  • - - 1.73. Bootstrapping, code - -
    • -
  • - - 1.74. Jackknife, code - +
    • @@ -1078,7 +432,8 @@ const thebe_selector_output = ".output, .cell_output"
    -
    +

    1. Elements of Probability Theory and Statistical Data Analysis

    1.1. Domains and probabilities

    @@ -1093,9 +448,6 @@ To this domain we have the corresponding probabilities

    \[ \{1/36,2/36/,3/36,4/36,5/36,6/36,5/36,4/36,3/36,2/36,1/36\}. \]
    -
    -
    -

    1.2. Tossing the dice

    The numbers in the domain are the outcomes of the physical process of tossing say two dice. We cannot tell beforehand whether the outcome is 3 or 5 or any other number in this domain. This defines the randomness of the outcome, or unexpectedness or any other synonimous word which @@ -1113,13 +465,9 @@ registering the sequence of outcomes, we will note that the numbers in the above \[ \{10,8,6,3,6,9,11,8,12,4,5\}. \]

    -
    -
    -

    1.3. Stochastic variables

    Random variables are characterized by a domain which contains all possible values that the random value may take. This domain has a corresponding probability distribution function(PDF).

    -
    -

    1.4. Stochastic variables and the main concepts, the discrete case

    +

    1.1.1. Stochastic variables and the main concepts, the discrete case

    There are two main concepts associated with a stochastic variable. The domain is the set \(\mathbb D = \{x\}\) of all accessible values the variable can assume, so that \(X \in \mathbb D\). An example of a @@ -1133,9 +481,6 @@ occur

    \[ p(x) = \mathrm{Prob}(X=x). \]
    -
    -
    -

    1.5. Stochastic variables and the main concepts, the continuous case

    In the continuous case, the PDF does not directly depict the actual probability. Instead we define the probability for the stochastic variable to assume any value on an infinitesimal interval @@ -1150,9 +495,6 @@ on a non-infinitesimal interval \([a,

    Qualitatively speaking, a stochastic variable represents the values of numbers chosen as if by chance from some specified PDF so that the selection of a large set of these numbers reproduces this PDF.

    -
    -
    -

    1.6. The cumulative probability

    Of interest to us is the cumulative probability distribution function (CDF), \(P(x)\), which is just the probability for a stochastic variable \(X\) to assume any value less than \(x\)

    @@ -1168,7 +510,7 @@ p(x) = \frac{d}{dx}P(x). \]
    -

    1.7. Properties of PDFs

    +

    1.1.2. Properties of PDFs

    There are two properties that all PDFs must satisfy. The first one is positivity (assuming that the PDF is normalized)

    @@ -1187,9 +529,6 @@ both discrete and continuous PDFs, this condition is

    \int_{x\in\mathbb D} p(x)\,dx & = 1. \end{align*} \end{split}\]
    -
    -
    -

    1.8. Important distributions, the uniform distribution

    The first one is the most basic PDF; namely the uniform distribution

    @@ -1209,9 +548,6 @@ p(x)dx = dx & \in [0,1]. \end{array} \]

    The latter distribution is used to generate random numbers. For other PDFs, one needs normally a mapping from this distribution to say for example the exponential distribution.

    - -
    -

    1.9. Gaussian distribution

    The second one is the Gaussian Distribution

    \[ @@ -1268,12 +604,9 @@ p(x) = \frac{1}{\sqrt{2\pi}} \exp{(-\frac{x^2}{2})},
    -_images/statistics_29_0.png +_images/statistics_32_0.png
    - -
    -

    1.10. Exponential distribution

    Another important distribution in science is the exponential distribution

    \[ @@ -1281,7 +614,7 @@ p(x) = \alpha\exp{-(\alpha x)}. \]
    -

    1.11. Expectation values

    +

    1.1.3. Expectation values

    Let \(h(x)\) be an arbitrary continuous function on the domain of the stochastic variable \(X\) whose PDF is \(p(x)\). We define the expectation value of \(h\) with respect to \(p\) as follows

    @@ -1303,9 +636,6 @@ follows

    \[ \langle x^n \rangle \equiv \int\! x^n p(x)\,dx \]
    - -
    -

    1.12. Stochastic variables and the main concepts, mean values

    The zero-th moment \(\langle 1\rangle\) is just the normalization condition of \(p\). The first moment, \(\langle x\rangle\), is called the mean of \(p\) and often denoted by the letter \(\mu\)

    @@ -1321,9 +651,6 @@ and often denoted by the letter \(\mu

    for a discrete distribution. Qualitatively it represents the centroid or the average value of the PDF and is therefore simply called the expectation value of \(p(x)\).

    -
    -
    -

    1.13. Stochastic variables and the main concepts, central moments, the variance

    A special version of the moments is the set of central moments, the n-th central moment defined as

    \[ @@ -1349,11 +676,11 @@ value of the deviation of the PDF from its mean value, interpreted qualitatively as the “spread” of \(p\) around its mean.

    -

    1.14. Probability Distribution Functions

    +

    1.1.4. Probability Distribution Functions

    The following table collects properties of probability distribution functions. In our notation we reserve the label \(p(x)\) for the probability of a certain event, while \(P(x)\) is the cumulative probability.

    - +
    @@ -1367,9 +694,6 @@ while \(P(x)\) is the cumulati
    Discrete PDF Continuous PDF
    Normalization $P_N=1$ $P(b)=1$
    -
    -
    -

    1.15. Probability Distribution Functions

    With a PDF we can compute expectation values of selected quantities such as

    \[ @@ -1382,9 +706,6 @@ while \(P(x)\) is the cumulati \]

    in the case of a continuous PDF. We have already defined the mean value \(\mu\) and the variance \(\sigma^2\).

    -
    -
    -

    1.16. The three famous Probability Distribution Functions

    There are at least three PDFs which one may encounter. These are the

    Uniform distribution

    @@ -1407,9 +728,6 @@ p(x)=\alpha \exp{(-\alpha x)}, \[ \sigma^2=\int_0^{\infty}x^2p(x)dx-\mu^2 = \frac{1}{\alpha^2}. \]
    -
    -
    -

    1.17. Probability Distribution Functions, the normal distribution

    Finally, we have the so-called univariate normal distribution, or just the normal distribution

    \[ @@ -1428,9 +746,6 @@ The mean value is

    \[ \mu =\frac{1}{b\sqrt{2\pi}}\int_{-\infty}^{\infty}b\sqrt{2}(a+b\sqrt{2}y)\exp{-y^2}dy=a. \]
    -
    -
    -

    1.18. Probability Distribution Functions, the normal distribution

    Similarly, the variance becomes

    \[ @@ -1454,9 +769,6 @@ centered around zero and with a variance -

    1.19. Probability Distribution Functions, the cumulative distribution

    The exponential and uniform distributions have simple cumulative functions, whereas the normal distribution does not, being proportional to the so-called error function \(erf(x)\), given by

    @@ -1465,9 +777,6 @@ error function \(erf(x)\), giv P(x) = \frac{1}{\sqrt{2\pi}}\int_{-\infty}^x\exp{\left(-\frac{t^2}{2}\right)}dt, \]

    which is difficult to evaluate in a quick way.

    -
    -
    -

    1.20. Probability Distribution Functions, other important distribution

    Some other PDFs which one encounters often in the natural sciences are the binomial distribution

    \[\begin{split} @@ -1481,9 +790,6 @@ in case of a random walker. Note that

    In every independent trial one registers if a specific situation happens or not, such as the jump to the left or right of a random walker.

  • The probability for every outcome in a single trial has the same value, for example the outcome of tossing (either heads or tails) a coin is always \(1/2\).

    • -
    -
    -

    1.21. Probability Distribution Functions, the binomial distribution

    In order to compute the mean and variance we need to recall Newton’s binomial formula

    @@ -1514,9 +820,6 @@ The mean value is

    \mu=ny\sum_{\nu=0}^n\left(\begin{array}{c} n-1 \\ \nu\end{array}\right)y^{\nu}(1-y)^{n-1-\nu} =ny(y+1-y)^{n-1}=ny. \end{split}\]

    The variance is slightly trickier to get. It reads \(\sigma^2=ny(1-y)\).

    -
    -
    -

    1.22. Probability Distribution Functions, Poisson’s distribution

    Another important distribution with discrete stochastic variables \(x\) is
    the Poisson model, which resembles the exponential distribution and reads

    @@ -1530,9 +833,6 @@ p(x) = \frac{\lambda^x}{x!} e^{-\lambda} \hspace{0.5cm}x=0,1,\dots,;\lambda > \frac{\lambda^{x-1}}{(x-1)!}=\lambda, \]

    and the variance is \(\sigma^2=\lambda\).

    -
    -
    -

    1.23. Probability Distribution Functions, Poisson’s distribution

    An example of applications of the Poisson distribution could be the counting of the number of \(\alpha\)-particles emitted from a radioactive source in a given time interval. In the limit of \(n\rightarrow \infty\) and for small probabilities \(y\), the binomial distribution @@ -1544,7 +844,7 @@ the binomial distribution we can show that

    \end{split}\]
    -

    1.24. Meet the covariance!

    +

    1.1.5. Meet the covariance!

    An important quantity in a statistical analysis is the so-called covariance.

    Consider the set \(\{X_i\}\) of \(n\) stochastic variables (not necessarily uncorrelated) with the @@ -1574,9 +874,6 @@ of the stochastic variables, \(X_i\)< \langle x_i\rangle = \int\cdots\int x_i P(x_1,\dots,x_n)\,dx_1\dots dx_n. \]

    -
    -
    -

    1.25. Meet the covariance in matrix disguise

    If we consider the above covariance as a matrix

    \[ @@ -1586,9 +883,6 @@ C_{ij} =\mathrm{Cov}(X_i,\,X_j), variances, \(C_{ii} = \mathrm{Cov}(X_i,\,X_i) = \mathrm{Var}(X_i)\). It turns out that all the off-diagonal elements are zero if the stochastic variables are uncorrelated.

    -
    -
    -

    1.26. Covariance

    # Importing various packages
@@ -1619,34 +913,31 @@ uncorrelated.
    -
    2.941866404575299
-[[ 2.85091337  3.42078103 10.33711888  6.57154252  3.22094791  6.36117602
-   8.03761519  5.89565264  3.47387858  5.73484667]
- [ 3.42078103  4.10455924 12.40340043  7.88512489  3.86478158  7.63270833
-   9.64425009  7.07413172  4.16827044  6.88118231]
- [10.33711888 12.40340043 37.48133064 23.82773778 11.67882608 23.06497046
-  29.1435666  21.37703055 12.59592669 20.7939646 ]
- [ 6.57154252  7.88512489 23.82773778 15.14783702  7.42449643 14.66292841
-  18.52723079 13.58986647  8.00751823 13.21919813]
- [ 3.22094791  3.86478158 11.67882608  7.42449643  3.63901111  7.18682538
-   9.08085812  6.66087937  3.92477093  6.47920156]
- [ 6.36117602  7.63270833 23.06497046 14.66292841  7.18682538 14.19354258
-  17.93414191 13.15483121  7.75118364 12.79602861]
- [ 8.03761519  9.64425009 29.1435666  18.52723079  9.08085812 17.93414191
-  22.66054752 16.62168613  9.79394867 16.16832385]
- [ 5.89565264  7.07413172 21.37703055 13.58986647  6.66087937 13.15483121
-  16.62168613 12.19213479  7.18393678 11.85959007]
- [ 3.47387858  4.16827044 12.59592669  8.00751823  3.92477093  7.75118364
-   9.79394867  7.18393678  4.23297056  6.9879924 ]
- [ 5.73484667  6.88118231 20.7939646  13.21919813  6.47920156 12.79602861
-  16.16832385 11.85959007  6.9879924  11.53611562]]
+
    0.6530742540053943
+[[14.12788968 14.881788   12.12937662 11.19379506 13.19466812  6.37692363
+   8.24924624  8.20509391 14.99435648 14.6834195 ]
+ [14.881788   15.67591616 12.77662945 11.791123   13.89876748  6.71721168
+   8.68944595  8.64293754 15.79449156 15.46696223]
+ [12.12937662 12.77662945 10.41357064  9.61033524 11.32816737  5.47485224
+   7.0823185   7.0444119  12.87327414 12.6063219 ]
+ [11.19379506 11.791123    9.61033524  8.86905621 10.4543859   5.05255759
+   6.53603432  6.50105159 11.88031314 11.63395187]
+ [13.19466812 13.89876748 11.32816737 10.4543859  12.32309075  5.95569422
+   7.70434005  7.66310422 14.00390021 13.71350226]
+ [ 6.37692363  6.71721168  5.47485224  5.05255759  5.95569422  2.87836018
+   3.72347283  3.70354373  6.76802186  6.62767384]
+ [ 8.24924624  8.68944595  7.0823185   6.53603432  7.70434005  3.72347283
+   4.81671821  4.79093776  8.75517445  8.57361898]
+ [ 8.20509391  8.64293754  7.0444119   6.50105159  7.66310422  3.70354373
+   4.79093776  4.76529528  8.70831425  8.52773051]
+ [14.99435648 15.79449156 12.87327414 11.88031314 14.00390021  6.76802186
+   8.75517445  8.70831425 15.91396388 15.58395707]
+ [14.6834195  15.46696223 12.6063219  11.63395187 13.71350226  6.62767384
+   8.57361898  8.52773051 15.58395707 15.26079358]]
    -
    -
    -

    1.27. Meet the covariance, uncorrelated events

    Consider the stochastic variables \(X_i\) and \(X_j\), (\(i\neq j\)). We have

    \[\begin{split} @@ -1670,9 +961,6 @@ Cov(X_i,\,X_j) &= \langle (x_i-\langle x_i\rangle)(x_j-\langle x_j\rangle)\r \[ Cov(X_i, X_j) = 0 \hspace{0.1cm} (i\neq j). \]
    -
    -
    -

    1.28. Numerical experiments and the covariance

    Now that we have constructed an idealized mathematical framework, let us try to apply it to empirical observations. Examples of relevant physical phenomena may be spontaneous decays of nuclei, or a purely @@ -1686,9 +974,6 @@ chain of values

    \[ \{x_1, x_2,\dots\,x_k,\dots\}. \]
    -
    -
    -

    1.29. Numerical experiments and the covariance

    We will call these values our measurements and the entire set as our measured sample. The action of measuring all the elements of a sample @@ -1701,9 +986,6 @@ stochastic variable whose PDF is \(p_ trying to determine the full distribution \(p\) we are often only interested in finding the few lowest moments, like the mean \(\mu_X^{\phantom X}\) and the variance \(\sigma_X^{\phantom X}\).

    -
    -
    -

    1.30. Numerical experiments and the covariance, actual situations

    In practical situations however, a sample is always of finite size. Let that size be \(n\). The expectation value of a sample \(\alpha\), the sample mean, is then defined as follows

    @@ -1716,9 +998,6 @@ size be \(n\). The expectation \mathrm{Var}(x) \equiv \frac{1}{n}\sum_{k=1}^n (x_{\alpha,k} - \langle x_{\alpha} \rangle)^2, \]

    with its square root being the standard deviation of the sample.

    -
    -
    -

    1.31. Numerical experiments and the covariance, our observables

    You can think of the above observables as a set of quantities which define a given experiment. This experiment is then repeated several times, say \(m\) times. The total average is then

    @@ -1747,9 +1026,6 @@ The total variance is

    \label{eq:exptvariance} \tag{7} \end{equation} \]
    -
    -
    -

    1.32. Numerical experiments and the covariance, the sample variance

    We define also the sample variance \(\sigma^2\) of all \(mn\) individual experiments as

    @@ -1767,7 +1043,7 @@ exact values for the mean value \(\mu and the covariance \(\mathrm{Cov}(X,Y)\).

    -

    1.33. Numerical experiments and the covariance, central limit theorem

    +

    1.1.6. Numerical experiments and the covariance, central limit theorem

    The central limit theorem states that the PDF \(\tilde{p}(z)\) of the average of \(m\) random values corresponding to a PDF \(p(x)\) is a normal distribution whose mean is the @@ -1781,9 +1057,6 @@ standard deviation, given by

    \frac{\sigma}{\sqrt{m}}. \]

    In many cases the above estimate for the standard deviation, in particular if correlations are strong, may be too simplistic. We need therefore a more precise defintion of the error and the variance in our results.

    -
    -
    -

    1.34. Definition of Correlation Functions and Standard Deviation

    Our estimate of the true average \(\mu_{X}\) is the sample mean \(\langle X_m \rangle\)

    \[ @@ -1801,9 +1074,6 @@ standard deviation, given by

    \]

    where the first term is the sample variance of all \(mn\) experiments divided by \(n\) and the last term is nothing but the covariance which arises when \(k\ne l\).

    -
    -
    -

    1.35. Definition of Correlation Functions and Standard Deviation

    Our estimate of the true average \(\mu_{X}\) is the sample mean \(\langle X_m \rangle\)

    If the observables are uncorrelated, then the covariance is zero and we obtain a total variance @@ -1816,9 +1086,6 @@ measurement \(x\) we receive. calculated at the end of the experiment since we need all the measurements to calculate the cross terms. Therefore, all measurements have to be stored throughout the experiment.

    -
    -
    -

    1.36. Definition of Correlation Functions and Standard Deviation

    Let us analyze the problem by splitting up the correlation term into partial sums of the form

    @@ -1832,9 +1099,6 @@ f_d = \frac{1}{nm}\sum_{\alpha=1}^m\sum_{k=1}^{n-d}(x_{\alpha,k}-\langle X_m \ra \frac{2}{mn^2}\sum_{\alpha=1}^m\sum_{k<l}^n (x_{\alpha,k}-\langle X_m \rangle)(x_{\alpha,l}-\langle X_m \rangle)= \frac{2}{n}\sum_{d=1}^{n-1} f_d \]
    -
    -
    -

    1.37. Definition of Correlation Functions and Standard Deviation

    The value of \(f_d\) reflects the correlation between measurements separated by the distance \(d\) in the samples. Notice that for \(d=0\), \(f\) is just the sample variance, \(\sigma^2\). If we divide \(f_d\) @@ -1850,9 +1114,6 @@ by \(\sigma^2\), we arrive at \]

    which gives us a useful measure of the correlation pair correlation starting always at \(1\) for \(d=0\).

    -
    -
    -

    1.38. Definition of Correlation Functions and Standard Deviation, sample variance

    The sample variance of the \(mn\) experiments can now be written in terms of the autocorrelation function

    @@ -1882,9 +1143,6 @@ correction factor the autocorrelation time

    For a correlation free experiment, \(\tau\) equals 1.

    -
    -
    -

    1.39. Definition of Correlation Functions and Standard Deviation

    From the point of view of Eq. (10) we can interpret a sequential correlation as an effective reduction of the number of measurements by @@ -1900,11 +1158,7 @@ too “good”. On the other hand, the calculation of the full autocorrelation time poses an efficiency problem if the set of measurements is very large. The solution to this problem is given by more practically oriented methods like the blocking technique.

    - -
    -
    -

    1.40. Code to compute the Covariance matrix and the Covariance

    -
    +
    # Importing various packages
 from math import exp, sqrt
@@ -1947,22 +1201,22 @@ more practically oriented methods like the blocking technique.
    -
    -0.029613363972682966
-3.7448615806559786
--0.2360682191515046
-0.8855417382991412 8.735738558766299 10.381627865854956
-2.6079169911277265 2.2886847885377843 7.0110407067093945
-[[ 0.88554174  2.60791699  2.28868479]
- [ 2.60791699  8.73573856  7.01104071]
- [ 2.28868479  7.01104071 10.38162787]]
-[17.33900713  0.09216046  2.57174058]
+
    0.12898627064868978
+4.517350858882083
+0.7469898175164704
+1.0712953943627344 9.116048442683864 13.773329728649545
+2.9898254753574576 3.041797190646905 8.203753611274545
+[[ 1.07129539  2.98982548  3.04179719]
+ [ 2.98982548  9.11604844  8.20375361]
+ [ 3.04179719  8.20375361 13.77332973]]
+[20.87647541  0.06602663  3.01817152]
    -

    1.41. Random Numbers

    +

    1.1.7. Random Numbers

    Uniform deviates are just random numbers that lie within a specified range (typically 0 to 1), with any one number in the range just as likely as any other. They are, in other words, what you probably think random numbers are. However, @@ -1973,9 +1227,6 @@ performing appropriate operations on one or more uniform deviates, as we will se in subsequent sections. So, a reliable source of random uniform deviates, the subject of this section, is an essential building block for any sort of stochastic modeling or Monte Carlo computer work.

    -
    -
    -

    1.42. Random Numbers, better name: pseudo random numbers

    A disclaimer is however appropriate. It should be fairly obvious that something as deterministic as a computer cannot generate purely random numbers.

    Numbers generated by any of the standard algorithms are in reality pseudo random @@ -1986,9 +1237,6 @@ numbers, hopefully abiding to the following criteria:

  • the period before the same sequence of random numbers is repeated is as large as possible and finally

  • the algorithm should be fast.

    • -
    -
    -

    1.43. Random number generator RNG

    The most common random number generators are based on so-called Linear congruential relations of the type

    @@ -2006,9 +1254,6 @@ and \(N_0\) is the starting value, or seed. The function \(\mathrm{MOD}\) means the remainder, that is if we were to evaluate \((13)\mathrm{MOD}(9)\), the outcome is the remainder of the division \(13/9\), namely \(4\).

    -
    -
    -

    1.44. Random number generator RNG and periodic outputs

    The problem with such generators is that their outputs are periodic; they will start to repeat themselves with a period that is at most \(M\). If however @@ -2028,9 +1273,6 @@ N_i=(27N_{i-1}+11) \mathrm{MOD} (54), \]

    which still, with \(N_0=2\), results in \(11,38,11,38,11,38,\dots\), a period of just \(2\).

    -
    -
    -

    1.45. Random number generator RNG and its period

    Typical periods for the random generators provided in the program library are of the order of \(\sim 10^9\) or larger. Other random number generators which have become increasingly popular are so-called shift-register generators. @@ -2043,9 +1285,6 @@ number is the sum of the \(l-i\) -

    -
    -

    1.46. Random number generator RNG, other examples

    Such a generator again produces a sequence of pseudorandom numbers but this time with a period much larger than \(M\). It is also possible to construct more elaborate algorithms by including @@ -2072,9 +1311,6 @@ which consists of two congruential relations

    \end{equation} \]

    which according to the authors has a period larger than \(2^{94}\).

    -
    -
    -

    1.47. Random number generator RNG, other examples

    Instead of using modular addition, we could use the bitwise exclusive-OR (\(\oplus\)) operation so that

    @@ -2090,9 +1326,6 @@ second number is \(1011\). Emp

    In Fortran90, the bitwise \(\oplus\) operation is coded through the intrinsic function \(\mathrm{IEOR}(m,n)\) where \(m\) and \(n\) are the input numbers, while in \(C\) it is given by \(m\wedge n\).

    -
    -
    -

    1.48. Random number generator RNG, RAN0

    We show here how the linear congruential algorithm can be implemented, namely

    \[ @@ -2118,9 +1351,6 @@ r = M\hspace{0.1cm}\mathrm{MOD} \hspace{0.1cm}a. \]

    where the brackets denote integer division. In the code below the numbers \(q\) and \(r\) are chosen so that \(r < q\).

    -
    -
    -

    1.49. Random number generator RNG, RAN0

    To see how this works we note first that

    @@ -2134,9 +1364,6 @@ r = M\hspace{0.1cm}\mathrm{MOD} \hspace{0.1cm}a.

    since we can add or subtract any integer multiple of \(M\) from \(aN_{i-1}\). The last term \([N_{i-1}/q]M\mathrm{MOD}(M)\) is zero since the integer division \([N_{i-1}/q]\) just yields a constant which is multiplied with \(M\).

    -
    -
    -

    1.50. Random number generator RNG, RAN0

    We can now rewrite Eq. (14) as

    @@ -2147,8 +1374,7 @@ The last term \([N_{i-1}/q]M\mathrm{M \label{eq:rntrick2} \tag{15} \end{equation} \]
    -

    which results -in

    +

    which results

    @@ -2168,9 +1394,6 @@ in

    \label{eq:rntrick4} \tag{17} \end{equation} \]
    -
    -
    -

    1.51. Random number generator RNG, RAN0

    The term \([N_{i-1}/q]r\) is always smaller or equal \(N_{i-1}(r/q)\) and with \(r < q\) we obtain always a number smaller than \(N_{i-1}\), which is smaller than \(M\). And since the number \(N_{i-1}\mathrm{MOD} (q)\) is between zero and \(q-1\) then @@ -2183,40 +1406,6 @@ the starting point for each generation of a random number. The period of \(ran0\) is \(\sim 2.1\times 10^{9}\). A special feature of this algorithm is that is should never be called with the initial seed set to \(0\).

    -
    -
    -

    1.52. Random number generator RNG, RAN0 code

    -
            /*
-         ** The function
-         **           ran0()
-         ** is an "Minimal" random number generator of Park and Miller
-         ** Set or reset the input value
-         ** idum to any integer value (except the unlikely value MASK)
-         ** to initialize the sequence; idum must not be altered between
-         ** calls for sucessive deviates in a sequence.
-         ** The function returns a uniform deviate between 0.0 and 1.0.
-         */
-    double ran0(long &idum)
-    {
-       const int a = 16807, m = 2147483647, q = 127773;
-       const int r = 2836, MASK = 123459876;
-       const double am = 1./m;
-       long     k;
-       double   ans;
-       idum ^= MASK;
-       k = (*idum)/q;
-       idum = a*(idum - k*q) - r*k;
-       // add m if negative difference
-       if(idum < 0) idum += m;
-       ans=am*(idum);
-       idum ^= MASK;
-       return ans;
-    } // End: function ran0() 
-
    -
    -
    -
    -

    1.53. Properties of Selected Random Number Generators

    As mentioned previously, the underlying PDF for the generation of random numbers is the uniform distribution, meaning that the probability for finding a number \(x\) in the interval [0,1] is \(p(x)=1\).

    @@ -2228,9 +1417,6 @@ intervals \(0.0-0.1\),

    Two additional measures are the standard deviation \(\sigma\) and the mean \(\mu=\langle x\rangle\).

    -
    -
    -

    1.54. Properties of Selected Random Number Generators

    For the uniform distribution, the mean value \(\mu\) is then

    \[ @@ -2241,12 +1427,9 @@ deviations.

    \[ \sigma=\sqrt{\langle x^2\rangle-\mu^2}=\frac{1}{\sqrt{12}}=0.2886. \]
    -
    -
    -

    1.55. Properties of Selected Random Number Generators

    The various random number generators produce results which agree rather well with these limiting values.

    - +
    @@ -2265,9 +1448,6 @@ these limiting values.

    $x$-bin ran0 ran1 ran2 ran3
    $\sigma$ 0.2882 0.2892 0.2861 0.2915
    -
    -
    -

    1.56. Simple demonstration of RNGs using python

    The following simple Python code plots the distribution of the produced random numbers using the linear congruential RNG employed by Python. The trend displayed in the previous table is seen rather clearly.

    @@ -2297,12 +1477,9 @@ these limiting values.

    -_images/statistics_178_0.png +_images/statistics_181_0.png
    -
    -
    -

    1.57. Properties of Selected Random Number Generators

    Since our random numbers, which are typically generated via a linear congruential algorithm, are never fully independent, we can then define an important test which measures the degree of correlation, namely the so-called
    @@ -2325,7 +1502,7 @@ in the evaluation of other expectation values. If they are not independent, our assumption for approximating \(\sigma_N\) is no longer valid.

    -

    1.58. Autocorrelation function

    +

    1.1.8. Autocorrelation function

    This program computes the autocorrelation function as discussed in the equation on the previous slide for random numbers generated with the normal distribution \(N(0,1)\).

    @@ -2362,412 +1539,15 @@ assumption for approximating \(\sigma
    -
    0.05290417684691035 0.9902552771282336
+
    -0.012633802944855959 1.0011828302640124
 
    
 
    
-_images/statistics_184_1.png
+_images/statistics_188_1.png

    As can be seen from the plot, the first point gives back the variance and a value of one. For the remaining values we notice that there are still non-zero values for the auto-correlation function.

    -
    -

    1.59. Correlation function and which random number generators should I use

    -

    The program here computes the correlation function for one of the standard functions included with the c++ compiler.

    -
        //  This function computes the autocorrelation function for 
-    //  the standard c++ random number generator
-    
-    #include <fstream>
-    #include <iomanip>
-    #include <iostream>
-    #include <cmath>
-    using namespace std;
-    // output file as global variable
-    ofstream ofile;  
-    
-    //     Main function begins here     
-    int main(int argc, char* argv[])
-    {
-         int n;
-         char *outfilename;
-    
-         cin >> n;
-         double MCint = 0.;      double MCintsqr2=0.;
-         double invers_period = 1./RAND_MAX; // initialise the random number generator
-         srand(time(NULL));  // This produces the so-called seed in MC jargon
-         // Compute the variance and the mean value of the uniform distribution
-         // Compute also the specific values x for each cycle in order to be able to
-         // the covariance and the correlation function  
-         // Read in output file, abort if there are too few command-line arguments
-         if( argc <= 2 ){
-           cout << "Bad Usage: " << argv[0] << 
-    	 " read also output file and number of cycles on same line" << endl;
-           exit(1);
-         }
-         else{
-           outfilename=argv[1];
-         }
-         ofile.open(outfilename); 
-         // Get  the number of Monte-Carlo samples
-         n = atoi(argv[2]);
-         double *X;  
-         X = new double[n];
-         for (int i = 0;  i < n; i++){
-               double x = double(rand())*invers_period; 
-               X[i] = x;
-               MCint += x;
-               MCintsqr2 += x*x;
-         }
-         double Mean = MCint/((double) n );
-         MCintsqr2 = MCintsqr2/((double) n );
-         double STDev = sqrt(MCintsqr2-Mean*Mean);
-         double Variance = MCintsqr2-Mean*Mean;
-    //   Write mean value and standard deviation 
-         cout << " Standard deviation= " << STDev << " Integral = " << Mean << endl;
-    
-         // Now we compute the autocorrelation function
-         double *autocor;  autocor = new double[n];
-         for (int j = 0; j < n; j++){
-           double sum = 0.0;
-           for (int k = 0; k < (n-j); k++){
-    	 sum  += (X[k]-Mean)*(X[k+j]-Mean); 
-           }
-           autocor[j] = sum/Variance/((double) n );
-           ofile << setiosflags(ios::showpoint | ios::uppercase);
-           ofile << setw(15) << setprecision(8) << j;
-           ofile << setw(15) << setprecision(8) << autocor[j] << endl;
-         }
-         ofile.close();  // close output file
-         return 0;
-    }  // end of main program 
-
    -
    -
    -
    -

    1.60. Which RNG should I use?

    -
      -

    • C++ has a class called random. The random class contains a large selection of RNGs and is highly recommended. Some of these RNGs have very large periods making it thereby very safe to use these RNGs in case one is performing large calculations. In particular, the Mersenne twister random number engine has a period of \(2^{19937}\).

      • -

    • Add RNGs in Python

      • -
    -
    -
    -

    1.61. How to use the Mersenne generator

    -

    The following part of a c++ code (from project 4) sets up the uniform distribution for \(x\in [0,1]\).

    -
        /*
-    
-    //  You need this 
-    #include <random>
-    
-    // Initialize the seed and call the Mersienne algo
-    std::random_device rd;
-    std::mt19937_64 gen(rd());
-    // Set up the uniform distribution for x \in [[0, 1]
-    std::uniform_real_distribution<double> RandomNumberGenerator(0.0,1.0);
-    
-    // Now use the RNG
-    int ix = (int) (RandomNumberGenerator(gen)*NSpins);
-
    -
    -
    -
    -

    1.62. Why blocking?

    -

    Statistical analysis.

    -
    * Monte Carlo simulations can be treated as *computer experiments*
-
-* The results can be analysed with the same statistical tools as we would use analysing experimental data.
-
-* As in all experiments, we are looking for expectation values and an estimate of how accurate they are, i.e., possible sources for errors.
-
    -
    -

    A very good article which explains blocking is H. Flyvbjerg and H. G. Petersen, Error estimates on averages of correlated data, Journal of Chemical Physics 91, 461-466 (1989).

    -
    -
    -

    1.63. Why blocking?

    -

    Statistical analysis.

    -
    * As in other experiments, Monte Carlo experiments have two classes of errors:
-
-  * Statistical errors
-
-  * Systematical errors
-
-
-* Statistical errors can be estimated using standard tools from statistics
-
-* Systematical errors are method specific and must be treated differently from case to case. (In VMC a common source is the step length or time step in importance sampling)
-
    -
    -
    -
    -

    1.64. Code to demonstrate the calculation of the autocorrelation function

    -

    The following code computes the autocorrelation function, the covariance and the standard deviation -for standard RNG. -The following file gives the code.

    -
        //  This function computes the autocorrelation function for 
-    //  the Mersenne random number generator with a uniform distribution
-    #include <iostream>
-    #include <fstream>
-    #include <iomanip>
-    #include <cstdlib>
-    #include <random>
-    #include <armadillo>
-    #include <string>
-    #include <cmath>
-    using namespace  std;
-    using namespace arma;
-    // output file
-    ofstream ofile;
-    
-    //     Main function begins here     
-    int main(int argc, char* argv[])
-    {
-      int MonteCarloCycles;
-      string filename;
-      if (argc > 1) {
-        filename=argv[1];
-        MonteCarloCycles = atoi(argv[2]);
-        string fileout = filename;
-        string argument = to_string(MonteCarloCycles);
-        fileout.append(argument);
-        ofile.open(fileout);
-      }
-    
-      // Compute the variance and the mean value of the uniform distribution
-      // Compute also the specific values x for each cycle in order to be able to
-      // compute the covariance and the correlation function  
-    
-      vec X  = zeros<vec>(MonteCarloCycles);
-      double MCint = 0.;      double MCintsqr2=0.;
-      std::random_device rd;
-      std::mt19937_64 gen(rd());
-      // Set up the uniform distribution for x \in [[0, 1]
-      std::uniform_real_distribution<double> RandomNumberGenerator(0.0,1.0);
-      for (int i = 0;  i < MonteCarloCycles; i++){
-        double x =   RandomNumberGenerator(gen); 
-        X(i) = x;
-        MCint += x;
-        MCintsqr2 += x*x;
-      }
-      double Mean = MCint/((double) MonteCarloCycles );
-      MCintsqr2 = MCintsqr2/((double) MonteCarloCycles );
-      double STDev = sqrt(MCintsqr2-Mean*Mean);
-      double Variance = MCintsqr2-Mean*Mean;
-      //   Write mean value and variance
-      cout << " Sample variance= " << Variance  << " Mean value = " << Mean << endl;
-      // Now we compute the autocorrelation function
-      vec autocorrelation = zeros<vec>(MonteCarloCycles);
-      for (int j = 0; j < MonteCarloCycles; j++){
-        double sum = 0.0;
-        for (int k = 0; k < (MonteCarloCycles-j); k++){
-          sum  += (X(k)-Mean)*(X(k+j)-Mean); 
-        }
-        autocorrelation(j) = sum/Variance/((double) MonteCarloCycles );
-        ofile << setiosflags(ios::showpoint | ios::uppercase);
-        ofile << setw(15) << setprecision(8) << j;
-        ofile << setw(15) << setprecision(8) << autocorrelation(j) << endl;
-      }
-      // Now compute the exact covariance using the autocorrelation function
-      double Covariance = 0.0;
-      for (int j = 0; j < MonteCarloCycles; j++){
-        Covariance  += autocorrelation(j);
-      }
-      Covariance *=  2.0/((double) MonteCarloCycles);
-      // Compute now the total variance, including the covariance, and obtain the standard deviation
-      double TotalVariance = (Variance/((double) MonteCarloCycles ))+Covariance;
-      cout << "Covariance =" << Covariance << "Totalvariance= " << TotalVariance << "Sample Variance/n= " << (Variance/((double) MonteCarloCycles )) << endl;
-      cout << " STD from sample variance= " << sqrt(Variance/((double) MonteCarloCycles )) << " STD with covariance = " << sqrt(TotalVariance) << endl;
-    
-      ofile.close();  // close output file
-      return 0;
-    }  // end of main program 
-
    -
    -
    -
    -

    1.65. What is blocking?

    -

    Blocking.

    -
    * Say that we have a set of samples from a Monte Carlo experiment
-
-* Assuming (wrongly) that our samples are uncorrelated our best estimate of the standard deviation of the mean $\langle \mathbf{M}\rangle$ is given by
-
    -
    -
    -\[ -\sigma=\sqrt{\frac{1}{n}\left(\langle \mathbf{M}^2\rangle-\langle \mathbf{M}\rangle^2\right)} -\]
    -
      -

    • If the samples are correlated we can rewrite our results to show that

      • -
    -
    -\[ -\sigma=\sqrt{\frac{1+2\tau/\Delta t}{n}\left(\langle \mathbf{M}^2\rangle-\langle \mathbf{M}\rangle^2\right)} -\]
    -

    where \(\tau\) is the correlation time (the time between a sample and the next uncorrelated sample) and \(\Delta t\) is time between each sample

    -
    -
    -

    1.66. What is blocking?

    -

    Blocking.

    -
    * If $\Delta t\gg\tau$ our first estimate of $\sigma$ still holds
-
-* Much more common that $\Delta t<\tau$
-
-* In the method of data blocking we divide the sequence of samples into blocks
-
-* We then take the mean $\langle \mathbf{M}_i\rangle$ of block $i=1\ldots n_{blocks}$ to calculate the total mean and variance
-
-* The size of each block must be so large that sample $j$ of block $i$ is not correlated with sample $j$ of block $i+1$
-
-* The correlation time $\tau$ would be a good choice
-
    -
    -
    -
    -

    1.67. What is blocking?

    -

    Blocking.

    -
    * Problem: We don't know $\tau$ or it is too expensive to compute
-
-* Solution: Make a plot of std. dev. as a function of blocksize
-
-* The estimate of std. dev. of correlated data is too low $\to$ the error will increase with increasing block size until the blocks are uncorrelated, where we reach a plateau
-
-* When the std. dev. stops increasing the blocks are uncorrelated
-
    -
    -
    -
    -

    1.68. Implementation

    -
    * Do a Monte Carlo simulation, storing all samples to file
-
-* Do the statistical analysis on this file, independently of your Monte Carlo program
-
-* Read the file into an array
-
-* Loop over various block sizes
-
-* For each block size $n_b$, loop over the array in steps of $n_b$ taking the mean of elements $i n_b,\ldots,(i+1) n_b$
-
-* Take the mean and variance of the resulting array
-
-* Write the results for each block size to file for later
-  analysis
-
    -
    -
    -
    -

    1.69. Actual implementation with code, main function

    -

    When the file gets large, it can be useful to write your data in binary mode instead of ascii characters. -The following python file reads data from file with the output from every Monte Carlo cycle.

    -
    -
    -
    # Blocking
-    @timeFunction
-    def blocking(self, blockSizeMax = 500):
-        blockSizeMin = 1
-
-        self.blockSizes = []
-        self.meanVec = []
-        self.varVec = []
-
-        for i in range(blockSizeMin, blockSizeMax):
-            if(len(self.data) % i != 0):
-                pass#continue
-            blockSize = i
-            meanTempVec = []
-            varTempVec = []
-            startPoint = 0
-            endPoint = blockSize
-
-            while endPoint <= len(self.data):
-                meanTempVec.append(np.average(self.data[startPoint:endPoint]))
-                startPoint = endPoint
-                endPoint += blockSize
-            mean, var = np.average(meanTempVec), np.var(meanTempVec)/len(meanTempVec)
-            self.meanVec.append(mean)
-            self.varVec.append(var)
-            self.blockSizes.append(blockSize)
-
-        self.blockingAvg = np.average(self.meanVec[-200:])
-        self.blockingVar = (np.average(self.varVec[-200:]))
-        self.blockingStd = np.sqrt(self.blockingVar)
-
    -
    -
    -
    -
      File "<ipython-input-6-2ff97f4bf03b>", line 2
-    @timeFunction
-    ^
-IndentationError: unexpected indent
-
    -
    -
    -
    -
    -
    -

    1.70. The Bootstrap method

    -

    The Bootstrap resampling method is also very popular. It is very simple:

    -
      -

    1. Start with your sample of measurements and compute the sample variance and the mean values

      2. -

    2. Then start again but pick in a random way the numbers in the sample and recalculate the mean and the sample variance.

      3. -

    3. Repeat this \(K\) times.

      4. -
    -

    It can be shown, see the article by Efron -that it produces the correct standard deviation.

    -

    This method is very useful for small ensembles of data points.

    -
    -
    -

    1.71. Bootstrapping

    -

    Given a set of \(N\) data, assume that we are interested in some -observable \(\theta\) which may be estimated from that set. This observable can also be for example the result of a fit based on all \(N\) raw data. -Let us call the value of the observable obtained from the original -data set \(\hat{\theta}\). One recreates from the sample repeatedly -other samples by choosing randomly \(N\) data out of the original set. -This costs essentially nothing, since we just recycle the original data set for the building of new sets.

    -
    -
    -

    1.72. Bootstrapping, recipe

    -

    Let us assume we have done this \(K\) times and thus have \(K\) sets of \(N\) -data values each. -Of course some values will enter more than once in the new sets. For each of these sets one computes the observable \(\theta\) resulting in values \(\theta_k\) with \(k = 1,...,K\). Then one determines

    -
    -\[ -\tilde{\theta} = \frac{1}{K} \sum_{k=1}^K \theta_k, -\]
    -

    and

    -
    -\[ -sigma^2_{\tilde{\theta}} = \frac{1}{K} \sum_{k=1}^K \left(\theta_k-\tilde{\theta}\right)^2. -\]
    -

    These are estimators for \(\angle\theta\rangle\) and its variance. They are not unbiased and therefore -\(\tilde{\theta}\neq\hat{\theta}\) for finite K.

    -

    The difference is called bias and gives an idea on how far away the result may be from -the true \(\angle\theta\rangle\). As final result for the observable one quotes \(\angle\theta\rangle = \tilde{\theta} \pm \sigma_{\tilde{\theta}}\) .

    -
    -
    -

    1.73. Bootstrapping, code

    -
        # Bootstrap
-        @timeFunction
-        def bootstrap(self, nBoots = 1000):
-            bootVec = np.zeros(nBoots)
-            for k in range(0,nBoots):
-                bootVec[k] = np.average(np.random.choice(self.data, len(self.data)))
-            self.bootAvg = np.average(bootVec)
-            self.bootVar = np.var(bootVec)
-            self.bootStd = np.std(bootVec)
-
    -
    -
    -
    -

    1.74. Jackknife, code

    -
        # Jackknife
-        @timeFunction
-        def jackknife(self):
-            jackknVec = np.zeros(len(self.data))
-            for k in range(0,len(self.data)):
-                jackknVec[k] = np.average(np.delete(self.data, k))
-            self.jackknAvg = self.avg - (len(self.data) - 1) * (np.average(jackknVec) - self.avg)
-            self.jackknVar = float(len(self.data) - 1) * np.var(jackknVec)
-            self.jackknStd = np.sqrt(self.jackknVar)
-
    -
    diff --git a/doc/LectureNotes/_build/jupyter_execute/statistics.ipynb b/doc/LectureNotes/_build/jupyter_execute/statistics.ipynb index 2dbf97f52..9632b6df2 100644 --- a/doc/LectureNotes/_build/jupyter_execute/statistics.ipynb +++ b/doc/LectureNotes/_build/jupyter_execute/statistics.ipynb @@ -2,18 +2,43 @@ "cells": [ { "cell_type": "markdown", - "metadata": {}, + "id": "1de4abfb", + "metadata": { + "editable": true + }, + "source": [ + "" + ] + }, + { + "cell_type": "markdown", + "id": "9268be0a", + "metadata": { + "editable": true + }, + "source": [ + "# Elements of Probability Theory and Statistical Data Analysis" + ] + }, + { + "cell_type": "markdown", + "id": "b24d433f", + "metadata": { + "editable": true + }, "source": [ - "# Elements of Probability Theory and Statistical Data Analysis\n", - "\n", - "\n", "## Domains and probabilities\n", + "\n", "Consider the following simple example, namely the tossing of two dice, resulting in the following possible values" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "9fa7c59b", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\{2,3,4,5,6,7,8,9,10,11,12\\}.\n", @@ -22,7 +47,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "d1c9912c", + "metadata": { + "editable": true + }, "source": [ "These values are called the *domain*. \n", "To this domain we have the corresponding *probabilities*" @@ -30,7 +58,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "6626ea89", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\{1/36,2/36/,3/36,4/36,5/36,6/36,5/36,4/36,3/36,2/36,1/36\\}.\n", @@ -39,9 +70,11 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "65c54c5b", + "metadata": { + "editable": true + }, "source": [ - "## Tossing the dice\n", "The numbers in the domain are the outcomes of the physical process of tossing say two dice.\n", "We cannot tell beforehand whether the outcome is 3 or 5 or any other number in this domain.\n", "This defines the randomness of the outcome, or unexpectedness or any other synonimous word which\n", @@ -55,7 +88,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "84e29b68", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\{2,3,4,5,6,7,8,9,10,11,12\\},\n", @@ -64,14 +100,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "8ae71934", + "metadata": { + "editable": true + }, "source": [ "appear in a random order. After 11 throws the results may look like" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "c3901951", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\{10,8,6,3,6,9,11,8,12,4,5\\}.\n", @@ -80,16 +122,23 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "9c659ce0", + "metadata": { + "editable": true + }, "source": [ - "## Stochastic variables\n", + "**Random variables are characterized by a domain which contains all possible values that the random value may take. This domain has a corresponding probability distribution function(PDF)**." + ] + }, + { + "cell_type": "markdown", + "id": "eb18a204", + "metadata": { + "editable": true + }, + "source": [ + "### Stochastic variables and the main concepts, the discrete case\n", "\n", - "**Random variables are characterized by a domain which contains all possible values that the random value may take. This domain has a corresponding probability distribution function(PDF)**.\n", - "\n", - "\n", - "\n", - "\n", - "## Stochastic variables and the main concepts, the discrete case\n", "There are two main concepts associated with a stochastic variable. The\n", "*domain* is the set $\\mathbb D = \\{x\\}$ of all accessible values\n", "the variable can assume, so that $X \\in \\mathbb D$. An example of a\n", @@ -104,7 +153,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "2589c0ee", + "metadata": { + "editable": true + }, "source": [ "$$\n", "p(x) = \\mathrm{Prob}(X=x).\n", @@ -113,9 +165,11 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "ef71f105", + "metadata": { + "editable": true + }, "source": [ - "## Stochastic variables and the main concepts, the continuous case\n", "In the continuous case, the PDF does not directly depict the\n", "actual probability. Instead we define the probability for the\n", "stochastic variable to assume any value on an infinitesimal interval\n", @@ -127,7 +181,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "9c5efaee", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\mathrm{Prob}(a\\leq X\\leq b) = \\int_a^b p(x)dx.\n", @@ -136,16 +193,15 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "cbf1d43c", + "metadata": { + "editable": true + }, "source": [ "Qualitatively speaking, a stochastic variable represents the values of\n", "numbers chosen as if by chance from some specified PDF so that the\n", "selection of a large set of these numbers reproduces this PDF.\n", "\n", - "\n", - "\n", - "\n", - "## The cumulative probability\n", "Of interest to us is the *cumulative probability\n", "distribution function* (**CDF**), $P(x)$, which is just the probability\n", "for a stochastic variable $X$ to assume any value less than $x$" @@ -153,7 +209,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "be829d81", + "metadata": { + "editable": true + }, "source": [ "$$\n", "P(x)=\\mathrm{Prob(}X\\leq x\\mathrm{)} =\n", @@ -163,14 +222,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "1f193798", + "metadata": { + "editable": true + }, "source": [ "The relation between a CDF and its corresponding PDF is then" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "3d29d961", + "metadata": { + "editable": true + }, "source": [ "$$\n", "p(x) = \\frac{d}{dx}P(x).\n", @@ -179,9 +244,12 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "fc5b2c18", + "metadata": { + "editable": true + }, "source": [ - "## Properties of PDFs\n", + "### Properties of PDFs\n", "\n", "There are two properties that all PDFs must satisfy. The first one is\n", "positivity (assuming that the PDF is normalized)" @@ -189,7 +257,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "2a465c1b", + "metadata": { + "editable": true + }, "source": [ "$$\n", "0 \\leq p(x) \\leq 1.\n", @@ -198,7 +269,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "d69e520a", + "metadata": { + "editable": true + }, "source": [ "Naturally, it would be nonsensical for any of the values of the domain\n", "to occur with a probability greater than $1$ or less than $0$. Also,\n", @@ -209,7 +283,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "2208b590", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{align*}\n", @@ -221,16 +298,21 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "f3d91dca", + "metadata": { + "editable": true + }, "source": [ - "## Important distributions, the uniform distribution\n", "The first one\n", "is the most basic PDF; namely the uniform distribution" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "0f431dd2", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -245,14 +327,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "155c6cf7", + "metadata": { + "editable": true + }, "source": [ "For $a=0$ and $b=1$ we have" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "42993dcc", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{array}{ll}\n", @@ -263,20 +351,22 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "749d5c23", + "metadata": { + "editable": true + }, "source": [ - "The latter distribution is used to generate random numbers. For other PDFs, one needs normally a mapping from this distribution to say for example the exponential distribution.\n", + "The latter distribution is used to generate random numbers. For other PDFs, one needs normally a mapping from this distribution to say for example the exponential distribution. \n", "\n", - "\n", - "\n", - "\n", - "## Gaussian distribution\n", "The second one is the Gaussian Distribution" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "6e2110dc", + "metadata": { + "editable": true + }, "source": [ "$$\n", "p(x) = \\frac{1}{\\sigma\\sqrt{2\\pi}} \\exp{(-\\frac{(x-\\mu)^2}{2\\sigma^2})},\n", @@ -285,14 +375,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "d00fdbe1", + "metadata": { + "editable": true + }, "source": [ "with mean value $\\mu$ and standard deviation $\\sigma$. If $\\mu=0$ and $\\sigma=1$, it is normally called the **standard normal distribution**" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "23c37b45", + "metadata": { + "editable": true + }, "source": [ "$$\n", "p(x) = \\frac{1}{\\sqrt{2\\pi}} \\exp{(-\\frac{x^2}{2})},\n", @@ -301,7 +397,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "07558d2c", + "metadata": { + "editable": true + }, "source": [ "The following simple Python code plots the above distribution for different values of $\\mu$ and $\\sigma$." ] @@ -309,6 +408,7 @@ { "cell_type": "code", "execution_count": 1, + "id": "b99f544d", "metadata": { "collapsed": false, "editable": true @@ -323,7 +423,7 @@ }, "metadata": { "filenames": { - "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/statistics_29_0.png" + "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/statistics_32_0.png" }, "needs_background": "light" }, @@ -374,15 +474,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "617597ef", + "metadata": { + "editable": true + }, "source": [ - "## Exponential distribution\n", "Another important distribution in science is the exponential distribution" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "18be0515", + "metadata": { + "editable": true + }, "source": [ "$$\n", "p(x) = \\alpha\\exp{-(\\alpha x)}.\n", @@ -391,9 +496,13 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "cafcff78", + "metadata": { + "editable": true + }, "source": [ - "## Expectation values\n", + "### Expectation values\n", + "\n", "Let $h(x)$ be an arbitrary continuous function on the domain of the stochastic\n", "variable $X$ whose PDF is $p(x)$. We define the *expectation value*\n", "of $h$ with respect to $p$ as follows" @@ -401,7 +510,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "f5c68a26", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -416,7 +528,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "291e3ff7", + "metadata": { + "editable": true + }, "source": [ "Whenever the PDF is known implicitly, like in this case, we will drop\n", "the index $X$ for clarity. \n", @@ -427,7 +542,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "294080d7", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\langle x^n \\rangle \\equiv \\int\\! x^n p(x)\\,dx\n", @@ -436,9 +554,11 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "9d493962", + "metadata": { + "editable": true + }, "source": [ - "## Stochastic variables and the main concepts, mean values\n", "The zero-th moment $\\langle 1\\rangle$ is just the normalization condition of\n", "$p$. The first moment, $\\langle x\\rangle$, is called the *mean* of $p$\n", "and often denoted by the letter $\\mu$" @@ -446,7 +566,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "6ca38798", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\langle x\\rangle = \\mu \\equiv \\int x p(x)dx,\n", @@ -455,14 +578,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "9a779b37", + "metadata": { + "editable": true + }, "source": [ "for a continuous distribution and" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "9fc21016", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\langle x\\rangle = \\mu \\equiv \\sum_{i=1}^N x_i p(x_i),\n", @@ -471,23 +600,24 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "a146b327", + "metadata": { + "editable": true + }, "source": [ "for a discrete distribution. \n", "Qualitatively it represents the centroid or the average value of the\n", "PDF and is therefore simply called the expectation value of $p(x)$.\n", "\n", - "\n", - "\n", - "\n", - "## Stochastic variables and the main concepts, central moments, the variance\n", - "\n", "A special version of the moments is the set of *central moments*, the n-th central moment defined as" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "0b282f14", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\langle (x-\\langle x\\rangle )^n\\rangle \\equiv \\int\\! (x-\\langle x\\rangle)^n p(x)\\,dx\n", @@ -496,7 +626,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "512e6e16", + "metadata": { + "editable": true + }, "source": [ "The zero-th and first central moments are both trivial, equal $1$ and\n", "$0$, respectively. But the second central moment, known as the\n", @@ -506,7 +639,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "e244a343", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{align*}\n", @@ -521,26 +657,31 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "c0d8e272", + "metadata": { + "editable": true + }, "source": [ "The square root of the variance, $\\sigma =\\sqrt{\\langle (x-\\langle x\\rangle)^2\\rangle}$ is called the \n", "**standard deviation** of $p$. It is the RMS (root-mean-square)\n", "value of the deviation of the PDF from its mean value, interpreted\n", - "qualitatively as the \"spread\" of $p$ around its mean.\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "## Probability Distribution Functions\n", + "qualitatively as the \"spread\" of $p$ around its mean." + ] + }, + { + "cell_type": "markdown", + "id": "12fb667e", + "metadata": { + "editable": true + }, + "source": [ + "### Probability Distribution Functions\n", "\n", "The following table collects properties of probability distribution functions.\n", "In our notation we reserve the label $p(x)$ for the probability of a certain event,\n", "while $P(x)$ is the cumulative probability. \n", "\n", - "\n", - "\n", + "
    \n", "\n", "\n", "\n", @@ -555,17 +696,15 @@ "\n", "
    Discrete PDF Continuous PDF
    \n", "\n", - "\n", - "\n", - "\n", - "\n", - "## Probability Distribution Functions\n", "With a PDF we can compute expectation values of selected quantities such as" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "7d318e93", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\langle x^k\\rangle=\\sum_{i=1}^{N}x_i^kp(x_i),\n", @@ -574,14 +713,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "6ef87b82", + "metadata": { + "editable": true + }, "source": [ "if we have a discrete PDF or" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "d11f1bdb", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\langle x^k\\rangle=\\int_a^b x^kp(x)dx,\n", @@ -590,15 +735,13 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "2a8abb3d", + "metadata": { + "editable": true + }, "source": [ "in the case of a continuous PDF. We have already defined the mean value $\\mu$\n", - "and the variance $\\sigma^2$.\n", - "\n", - "\n", - "\n", - "\n", - "## The three famous Probability Distribution Functions\n", + "and the variance $\\sigma^2$. \n", "\n", "There are at least three PDFs which one may encounter. These are the\n", "\n", @@ -607,7 +750,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "6c14f838", + "metadata": { + "editable": true + }, "source": [ "$$\n", "p(x)=\\frac{1}{b-a}\\Theta(x-a)\\Theta(b-x),\n", @@ -616,7 +762,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "62149f95", + "metadata": { + "editable": true + }, "source": [ "yielding probabilities different from zero in the interval $[a,b]$.\n", "\n", @@ -625,7 +774,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "f8f51c0a", + "metadata": { + "editable": true + }, "source": [ "$$\n", "p(x)=\\alpha \\exp{(-\\alpha x)},\n", @@ -634,14 +786,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "9a038bf3", + "metadata": { + "editable": true + }, "source": [ "yielding probabilities different from zero in the interval $[0,\\infty)$ and with mean value" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "bdaf914a", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\mu = \\int_0^{\\infty}xp(x)dx=\\int_0^{\\infty}x\\alpha \\exp{(-\\alpha x)}dx=\\frac{1}{\\alpha},\n", @@ -650,14 +808,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "74226a02", + "metadata": { + "editable": true + }, "source": [ "with variance" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "32215b82", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\sigma^2=\\int_0^{\\infty}x^2p(x)dx-\\mu^2 = \\frac{1}{\\alpha^2}.\n", @@ -666,15 +830,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "688f69a2", + "metadata": { + "editable": true + }, "source": [ - "## Probability Distribution Functions, the normal distribution\n", "Finally, we have the so-called univariate normal distribution, or just the **normal distribution**" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "082e6778", + "metadata": { + "editable": true + }, "source": [ "$$\n", "p(x)=\\frac{1}{b\\sqrt{2\\pi}}\\exp{\\left(-\\frac{(x-a)^2}{2b^2}\\right)}\n", @@ -683,7 +852,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "ba9c8bd4", + "metadata": { + "editable": true + }, "source": [ "with probabilities different from zero in the interval $(-\\infty,\\infty)$.\n", "The integral $\\int_{-\\infty}^{\\infty}\\exp{\\left(-(x^2\\right)}dx$ appears in many calculations, its value\n", @@ -693,7 +865,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "f9540188", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\mu = \\int_0^{\\infty}xp(x)dx=\\frac{1}{b\\sqrt{2\\pi}}\\int_{-\\infty}^{\\infty}x \\exp{\\left(-\\frac{(x-a)^2}{2b^2}\\right)}dx,\n", @@ -702,14 +877,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "0507fa0a", + "metadata": { + "editable": true + }, "source": [ "which becomes with a suitable change of variables" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "74ee9f1a", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\mu =\\frac{1}{b\\sqrt{2\\pi}}\\int_{-\\infty}^{\\infty}b\\sqrt{2}(a+b\\sqrt{2}y)\\exp{-y^2}dy=a.\n", @@ -718,15 +899,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "0cf42992", + "metadata": { + "editable": true + }, "source": [ - "## Probability Distribution Functions, the normal distribution\n", "Similarly, the variance becomes" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "e3554ca3", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\sigma^2 = \\frac{1}{b\\sqrt{2\\pi}}\\int_{-\\infty}^{\\infty}(x-\\mu)^2 \\exp{\\left(-\\frac{(x-a)^2}{2b^2}\\right)}dx,\n", @@ -735,14 +921,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "728ba052", + "metadata": { + "editable": true + }, "source": [ "and inserting the mean value and performing a variable change we obtain" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "60276744", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\sigma^2 = \\frac{1}{b\\sqrt{2\\pi}}\\int_{-\\infty}^{\\infty}b\\sqrt{2}(b\\sqrt{2}y)^2\\exp{\\left(-y^2\\right)}dy=\n", @@ -752,7 +944,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "c5b45c52", + "metadata": { + "editable": true + }, "source": [ "and performing a final integration by parts we obtain the well-known result $\\sigma^2=b^2$.\n", "It is useful to introduce the standard normal distribution as well, defined by $\\mu=a=0$, viz. a distribution\n", @@ -761,7 +956,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "a005fdfe", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -776,10 +974,11 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "12c13ae4", + "metadata": { + "editable": true + }, "source": [ - "## Probability Distribution Functions, the cumulative distribution\n", - "\n", "The exponential and uniform distributions have simple cumulative functions,\n", "whereas the normal distribution does not, being proportional to the so-called\n", "error function $erf(x)$, given by" @@ -787,7 +986,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "68e56af7", + "metadata": { + "editable": true + }, "source": [ "$$\n", "P(x) = \\frac{1}{\\sqrt{2\\pi}}\\int_{-\\infty}^x\\exp{\\left(-\\frac{t^2}{2}\\right)}dt,\n", @@ -796,22 +998,22 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "99071464", + "metadata": { + "editable": true + }, "source": [ - "which is difficult to evaluate in a quick way.\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "## Probability Distribution Functions, other important distribution\n", + "which is difficult to evaluate in a quick way. \n", "\n", "Some other PDFs which one encounters often in the natural sciences are the binomial distribution" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "08b3b2e3", + "metadata": { + "editable": true + }, "source": [ "$$\n", "p(x) = \\left(\\begin{array}{c} n \\\\ x\\end{array}\\right)y^x(1-y)^{n-x} \\hspace{0.5cm}x=0,1,\\dots,n,\n", @@ -820,7 +1022,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "57d2b4a7", + "metadata": { + "editable": true + }, "source": [ "where $y$ is the probability for a specific event, such as the tossing of a coin or moving left or right\n", "in case of a random walker. Note that $x$ is a discrete stochastic variable. \n", @@ -833,18 +1038,16 @@ "\n", " * The probability for every outcome in a single trial has the same value, for example the outcome of tossing (either heads or tails) a coin is always $1/2$.\n", "\n", - "\n", - "\n", - "\n", - "## Probability Distribution Functions, the binomial distribution\n", - "\n", "In order to compute the mean and variance we need to recall Newton's binomial\n", "formula" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "4ce5e39f", + "metadata": { + "editable": true + }, "source": [ "$$\n", "(a+b)^m=\\sum_{n=0}^m \\left(\\begin{array}{c} m \\\\ n\\end{array}\\right)a^nb^{m-n},\n", @@ -853,14 +1056,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "4b87bcf7", + "metadata": { + "editable": true + }, "source": [ "which can be used to show that" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "29e93603", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\sum_{x=0}^n\\left(\\begin{array}{c} n \\\\ x\\end{array}\\right)y^x(1-y)^{n-x} = (y+1-y)^n = 1,\n", @@ -869,7 +1078,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "524d4e2f", + "metadata": { + "editable": true + }, "source": [ "the PDF is normalized to one. \n", "The mean value is" @@ -877,7 +1089,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "d129841d", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\mu = \\sum_{x=0}^n x\\left(\\begin{array}{c} n \\\\ x\\end{array}\\right)y^x(1-y)^{n-x} =\n", @@ -887,14 +1102,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "f68c25df", + "metadata": { + "editable": true + }, "source": [ "resulting in" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "f19452aa", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\mu = \n", @@ -904,14 +1125,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "c75f2dc5", + "metadata": { + "editable": true + }, "source": [ "which we rewrite as" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "c7b760bf", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\mu=ny\\sum_{\\nu=0}^n\\left(\\begin{array}{c} n-1 \\\\ \\nu\\end{array}\\right)y^{\\nu}(1-y)^{n-1-\\nu} =ny(y+1-y)^{n-1}=ny.\n", @@ -920,20 +1147,23 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "6fc3e294", + "metadata": { + "editable": true + }, "source": [ "The variance is slightly trickier to get. It reads $\\sigma^2=ny(1-y)$. \n", "\n", - "\n", - "## Probability Distribution Functions, Poisson's distribution\n", - "\n", "Another important distribution with discrete stochastic variables $x$ is \n", "the Poisson model, which resembles the exponential distribution and reads" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "c67cc7c9", + "metadata": { + "editable": true + }, "source": [ "$$\n", "p(x) = \\frac{\\lambda^x}{x!} e^{-\\lambda} \\hspace{0.5cm}x=0,1,\\dots,;\\lambda > 0.\n", @@ -942,14 +1172,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "3988d52a", + "metadata": { + "editable": true + }, "source": [ "In this case both the mean value and the variance are easier to calculate," ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "9b3e4562", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\mu = \\sum_{x=0}^{\\infty} x \\frac{\\lambda^x}{x!} e^{-\\lambda} = \\lambda e^{-\\lambda}\\sum_{x=1}^{\\infty}\n", @@ -959,16 +1195,13 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "88138c21", + "metadata": { + "editable": true + }, "source": [ - "and the variance is $\\sigma^2=\\lambda$.\n", + "and the variance is $\\sigma^2=\\lambda$. \n", "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "## Probability Distribution Functions, Poisson's distribution\n", "An example of applications of the Poisson distribution could be the counting\n", "of the number of $\\alpha$-particles emitted from a radioactive source in a given time interval.\n", "In the limit of $n\\rightarrow \\infty$ and for small probabilities $y$, the binomial distribution\n", @@ -978,7 +1211,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "36ecffde", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\lim_{n\\rightarrow \\infty}\\left(\\begin{array}{c} n \\\\ x\\end{array}\\right)y^x(1-y)^{n-x} e^{-\\lambda}=\\sum_{x=1}^{\\infty}\\frac{\\lambda^x}{x!} e^{-\\lambda}.\n", @@ -987,9 +1223,13 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "be4e015c", + "metadata": { + "editable": true + }, "source": [ - "## Meet the covariance!\n", + "### Meet the covariance!\n", + "\n", "An important quantity in a statistical analysis is the so-called covariance. \n", "\n", "Consider the set $\\{X_i\\}$ of $n$\n", @@ -1000,7 +1240,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "93522854", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -1015,7 +1258,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "e667c0aa", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -1030,14 +1276,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "44f76dd9", + "metadata": { + "editable": true + }, "source": [ "with" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "1041b2af", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\langle x_i\\rangle =\n", @@ -1047,15 +1299,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "e42984a2", + "metadata": { + "editable": true + }, "source": [ - "## Meet the covariance in matrix disguise\n", "If we consider the above covariance as a matrix" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "a46f958e", + "metadata": { + "editable": true + }, "source": [ "$$\n", "C_{ij} =\\mathrm{Cov}(X_i,\\,X_j),\n", @@ -1064,22 +1321,21 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "37763243", + "metadata": { + "editable": true + }, "source": [ "then the diagonal elements are just the familiar\n", "variances, $C_{ii} = \\mathrm{Cov}(X_i,\\,X_i) = \\mathrm{Var}(X_i)$. It turns out that\n", "all the off-diagonal elements are zero if the stochastic variables are\n", - "uncorrelated.\n", - "\n", - "\n", - "\n", - "\n", - "## Covariance" + "uncorrelated." ] }, { "cell_type": "code", "execution_count": 2, + "id": "aedc8b8c", "metadata": { "collapsed": false, "editable": true @@ -1089,27 +1345,27 @@ "name": "stdout", "output_type": "stream", "text": [ - "2.941866404575299\n", - "[[ 2.85091337 3.42078103 10.33711888 6.57154252 3.22094791 6.36117602\n", - " 8.03761519 5.89565264 3.47387858 5.73484667]\n", - " [ 3.42078103 4.10455924 12.40340043 7.88512489 3.86478158 7.63270833\n", - " 9.64425009 7.07413172 4.16827044 6.88118231]\n", - " [10.33711888 12.40340043 37.48133064 23.82773778 11.67882608 23.06497046\n", - " 29.1435666 21.37703055 12.59592669 20.7939646 ]\n", - " [ 6.57154252 7.88512489 23.82773778 15.14783702 7.42449643 14.66292841\n", - " 18.52723079 13.58986647 8.00751823 13.21919813]\n", - " [ 3.22094791 3.86478158 11.67882608 7.42449643 3.63901111 7.18682538\n", - " 9.08085812 6.66087937 3.92477093 6.47920156]\n", - " [ 6.36117602 7.63270833 23.06497046 14.66292841 7.18682538 14.19354258\n", - " 17.93414191 13.15483121 7.75118364 12.79602861]\n", - " [ 8.03761519 9.64425009 29.1435666 18.52723079 9.08085812 17.93414191\n", - " 22.66054752 16.62168613 9.79394867 16.16832385]\n", - " [ 5.89565264 7.07413172 21.37703055 13.58986647 6.66087937 13.15483121\n", - " 16.62168613 12.19213479 7.18393678 11.85959007]\n", - " [ 3.47387858 4.16827044 12.59592669 8.00751823 3.92477093 7.75118364\n", - " 9.79394867 7.18393678 4.23297056 6.9879924 ]\n", - " [ 5.73484667 6.88118231 20.7939646 13.21919813 6.47920156 12.79602861\n", - " 16.16832385 11.85959007 6.9879924 11.53611562]]\n" + "0.6530742540053943\n", + "[[14.12788968 14.881788 12.12937662 11.19379506 13.19466812 6.37692363\n", + " 8.24924624 8.20509391 14.99435648 14.6834195 ]\n", + " [14.881788 15.67591616 12.77662945 11.791123 13.89876748 6.71721168\n", + " 8.68944595 8.64293754 15.79449156 15.46696223]\n", + " [12.12937662 12.77662945 10.41357064 9.61033524 11.32816737 5.47485224\n", + " 7.0823185 7.0444119 12.87327414 12.6063219 ]\n", + " [11.19379506 11.791123 9.61033524 8.86905621 10.4543859 5.05255759\n", + " 6.53603432 6.50105159 11.88031314 11.63395187]\n", + " [13.19466812 13.89876748 11.32816737 10.4543859 12.32309075 5.95569422\n", + " 7.70434005 7.66310422 14.00390021 13.71350226]\n", + " [ 6.37692363 6.71721168 5.47485224 5.05255759 5.95569422 2.87836018\n", + " 3.72347283 3.70354373 6.76802186 6.62767384]\n", + " [ 8.24924624 8.68944595 7.0823185 6.53603432 7.70434005 3.72347283\n", + " 4.81671821 4.79093776 8.75517445 8.57361898]\n", + " [ 8.20509391 8.64293754 7.0444119 6.50105159 7.66310422 3.70354373\n", + " 4.79093776 4.76529528 8.70831425 8.52773051]\n", + " [14.99435648 15.79449156 12.87327414 11.88031314 14.00390021 6.76802186\n", + " 8.75517445 8.70831425 15.91396388 15.58395707]\n", + " [14.6834195 15.46696223 12.6063219 11.63395187 13.71350226 6.62767384\n", + " 8.57361898 8.52773051 15.58395707 15.26079358]]\n" ] } ], @@ -1142,16 +1398,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "4ad05ea1", + "metadata": { + "editable": true + }, "source": [ - "## Meet the covariance, uncorrelated events\n", - "\n", "Consider the stochastic variables $X_i$ and $X_j$, ($i\\neq j$). We have" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "3f643cb0", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\begin{align*}\n", @@ -1168,14 +1428,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "bedb81e0", + "metadata": { + "editable": true + }, "source": [ "If $X_i$ and $X_j$ are independent (assuming $i \\neq j$), we have that" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "8a8f5bdd", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\langle x_i x_j\\rangle = \\langle x_i\\rangle\\langle x_j\\rangle,\n", @@ -1184,14 +1450,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "cc3e8c15", + "metadata": { + "editable": true + }, "source": [ "leading to" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "cf107fa4", + "metadata": { + "editable": true + }, "source": [ "$$\n", "Cov(X_i, X_j) = 0 \\hspace{0.1cm} (i\\neq j).\n", @@ -1200,10 +1472,11 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "c3d6eefd", + "metadata": { + "editable": true + }, "source": [ - "## Numerical experiments and the covariance\n", - "\n", "Now that we have constructed an idealized mathematical framework, let\n", "us try to apply it to empirical observations. Examples of relevant\n", "physical phenomena may be spontaneous decays of nuclei, or a purely\n", @@ -1217,7 +1490,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "11084473", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\{x_1, x_2,\\dots\\,x_k,\\dots\\}.\n", @@ -1226,9 +1502,11 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "ec3bb41f", + "metadata": { + "editable": true + }, "source": [ - "## Numerical experiments and the covariance\n", "We will call these\n", "values our *measurements* and the entire set as our measured\n", "*sample*. The action of measuring all the elements of a sample\n", @@ -1242,18 +1520,16 @@ "interested in finding the few lowest moments, like the mean\n", "$\\mu_X^{\\phantom X}$ and the variance $\\sigma_X^{\\phantom X}$.\n", "\n", - "\n", - "\n", - "\n", - "\n", - "## Numerical experiments and the covariance, actual situations\n", "In practical situations however, a sample is always of finite size. Let that\n", "size be $n$. The expectation value of a sample $\\alpha$, the **sample mean**, is then defined as follows" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "9654f276", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\langle x_{\\alpha} \\rangle \\equiv \\frac{1}{n}\\sum_{k=1}^n x_{\\alpha,k}.\n", @@ -1262,14 +1538,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "9c339ccc", + "metadata": { + "editable": true + }, "source": [ "The *sample variance* is:" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "9d523f50", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\mathrm{Var}(x) \\equiv \\frac{1}{n}\\sum_{k=1}^n (x_{\\alpha,k} - \\langle x_{\\alpha} \\rangle)^2,\n", @@ -1278,15 +1560,13 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "2793f264", + "metadata": { + "editable": true + }, "source": [ - "with its square root being the *standard deviation of the sample*.\n", + "with its square root being the *standard deviation of the sample*. \n", "\n", - "\n", - "\n", - "\n", - "\n", - "## Numerical experiments and the covariance, our observables\n", "You can think of the above observables as a set of quantities which define\n", "a given experiment. This experiment is then repeated several times, say $m$ times.\n", "The total average is then" @@ -1294,7 +1574,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "b0432b14", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -1309,7 +1592,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "c1981b2d", + "metadata": { + "editable": true + }, "source": [ "where the last sums end at $m$ and $n$.\n", "The total variance is" @@ -1317,7 +1603,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "679d42bc", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\sigma^2_m= \\frac{1}{mn^2}\\sum_{\\alpha=1}^m(\\langle x_{\\alpha} \\rangle-\\langle X_m \\rangle)^2,\n", @@ -1326,14 +1615,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "2bb94e0d", + "metadata": { + "editable": true + }, "source": [ "which we rewrite as" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "0611f240", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -1348,16 +1643,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "624f9965", + "metadata": { + "editable": true + }, "source": [ - "## Numerical experiments and the covariance, the sample variance\n", - "\n", "We define also the sample variance $\\sigma^2$ of all $mn$ individual experiments as" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "c0d8ff9e", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -1372,18 +1671,26 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "d058f856", + "metadata": { + "editable": true + }, "source": [ "These quantities, being known experimental values or the results from our calculations, \n", "may differ, in some cases\n", "significantly, from the similarly named\n", "exact values for the mean value $\\mu_X$, the variance $\\mathrm{Var}(X)$\n", - "and the covariance $\\mathrm{Cov}(X,Y)$.\n", - "\n", - "\n", - "\n", - "\n", - "## Numerical experiments and the covariance, central limit theorem\n", + "and the covariance $\\mathrm{Cov}(X,Y)$." + ] + }, + { + "cell_type": "markdown", + "id": "76e956db", + "metadata": { + "editable": true + }, + "source": [ + "### Numerical experiments and the covariance, central limit theorem\n", "\n", "The central limit theorem states that the PDF $\\tilde{p}(z)$ of\n", "the average of $m$ random values corresponding to a PDF $p(x)$ \n", @@ -1397,7 +1704,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "bd4e286e", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\sigma_m=\n", @@ -1407,20 +1717,22 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "877355c7", + "metadata": { + "editable": true + }, "source": [ "In many cases the above estimate for the standard deviation, in particular if correlations are strong, may be too simplistic. We need therefore a more precise defintion of the error and the variance in our results.\n", "\n", - "\n", - "\n", - "\n", - "## Definition of Correlation Functions and Standard Deviation\n", "Our estimate of the true average $\\mu_{X}$ is the sample mean $\\langle X_m \\rangle$" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "abc25d1f", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\mu_{X}^{\\phantom X} \\approx X_m=\\frac{1}{mn}\\sum_{\\alpha=1}^m\\sum_{k=1}^n x_{\\alpha,k}.\n", @@ -1429,14 +1741,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "29a4c67d", + "metadata": { + "editable": true + }, "source": [ "We can then use Eq. ([7](#eq:exptvariance))" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "46b6a9e4", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\sigma^2_m=\\frac{1}{mn^2}\\sum_{\\alpha=1}^m\\sum_{kl=1}^n (x_{\\alpha,k}-\\langle X_m \\rangle)(x_{\\alpha,l}-\\langle X_m \\rangle),\n", @@ -1445,14 +1763,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "350ee280", + "metadata": { + "editable": true + }, "source": [ "and rewrite it as" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "321d0a3d", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\sigma^2_m=\\frac{\\sigma^2}{n}+\\frac{2}{mn^2}\\sum_{\\alpha=1}^m\\sum_{k\n", "
    \n", @@ -1552,23 +1880,24 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "03919ae2", + "metadata": { + "editable": true + }, "source": [ "which gives us a useful measure of the correlation pair correlation\n", "starting always at $1$ for $d=0$.\n", "\n", - "\n", - "\n", - "\n", - "## Definition of Correlation Functions and Standard Deviation, sample variance\n", - "\n", "The sample variance of the $mn$ experiments can now be\n", "written in terms of the autocorrelation function" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "f4edb79b", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -1584,7 +1913,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "209c3c0a", + "metadata": { + "editable": true + }, "source": [ "and we see that $\\sigma_m$ can be expressed in terms of the\n", "uncorrelated sample variance times a correction factor $\\tau$ which\n", @@ -1594,7 +1926,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "d0c7027f", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -1609,19 +1944,16 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "8183355a", + "metadata": { + "editable": true + }, "source": [ "\n", "\n", "For a correlation free experiment, $\\tau$\n", - "equals 1.\n", + "equals 1. \n", "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "## Definition of Correlation Functions and Standard Deviation\n", "From the point of view of\n", "Eq. ([10](#eq:error_estimate_corr_time)) we can interpret a sequential\n", "correlation as an effective reduction of the number of measurements by\n", @@ -1630,7 +1962,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "04f300ab", + "metadata": { + "editable": true + }, "source": [ "$$\n", "n_\\mathrm{eff} = \\frac{n}{\\tau}\n", @@ -1639,7 +1974,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "d9bdb1eb", + "metadata": { + "editable": true + }, "source": [ "To neglect the autocorrelation time $\\tau$ will always cause our\n", "simple uncorrelated estimate of $\\sigma_m^2\\approx \\sigma^2/n$ to\n", @@ -1648,18 +1986,13 @@ "autocorrelation time poses an efficiency problem if the set of\n", "measurements is very large. The solution to this problem is given by \n", "more practically oriented methods like the blocking technique.\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "## Code to compute the Covariance matrix and the Covariance" + "" ] }, { "cell_type": "code", "execution_count": 3, + "id": "93013c0b", "metadata": { "collapsed": false, "editable": true @@ -1669,15 +2002,15 @@ "name": "stdout", "output_type": "stream", "text": [ - "-0.029613363972682966\n", - "3.7448615806559786\n", - "-0.2360682191515046\n", - "0.8855417382991412 8.735738558766299 10.381627865854956\n", - "2.6079169911277265 2.2886847885377843 7.0110407067093945\n", - "[[ 0.88554174 2.60791699 2.28868479]\n", - " [ 2.60791699 8.73573856 7.01104071]\n", - " [ 2.28868479 7.01104071 10.38162787]]\n", - "[17.33900713 0.09216046 2.57174058]\n" + "0.12898627064868978\n", + "4.517350858882083\n", + "0.7469898175164704\n", + "1.0712953943627344 9.116048442683864 13.773329728649545\n", + "2.9898254753574576 3.041797190646905 8.203753611274545\n", + "[[ 1.07129539 2.98982548 3.04179719]\n", + " [ 2.98982548 9.11604844 8.20375361]\n", + " [ 3.04179719 8.20375361 13.77332973]]\n", + "[20.87647541 0.06602663 3.01817152]\n" ] } ], @@ -1723,9 +2056,12 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "65e8cd76", + "metadata": { + "editable": true + }, "source": [ - "## Random Numbers\n", + "### Random Numbers\n", "\n", "Uniform deviates are just random numbers that lie within a specified range\n", "(typically 0 to 1), with any one number in the range just as likely as any other. They\n", @@ -1738,12 +2074,6 @@ "of this section, is an essential building block for any sort of stochastic modeling\n", "or Monte Carlo computer work.\n", "\n", - "\n", - "\n", - "\n", - "\n", - "## Random Numbers, better name: pseudo random numbers\n", - "\n", "A disclaimer is however appropriate. It should be fairly obvious that \n", "something as deterministic as a computer cannot generate purely random numbers.\n", "\n", @@ -1758,18 +2088,16 @@ "\n", " * the algorithm should be fast.\n", "\n", - "\n", - "\n", - "\n", - "\n", - "## Random number generator RNG\n", " The most common random number generators are based on so-called\n", "Linear congruential relations of the type" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "4daecdd3", + "metadata": { + "editable": true + }, "source": [ "$$\n", "N_i=(aN_{i-1}+c) \\mathrm{MOD} (M),\n", @@ -1778,14 +2106,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "fd1c5e47", + "metadata": { + "editable": true + }, "source": [ "which yield a number in the interval [0,1] through" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "d8c06853", + "metadata": { + "editable": true + }, "source": [ "$$\n", "x_i=N_i/M\n", @@ -1794,7 +2128,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "0d971106", + "metadata": { + "editable": true + }, "source": [ "The number \n", "$M$ is called the period and it should be as large as possible \n", @@ -1803,11 +2140,6 @@ "that is if we were to evaluate $(13)\\mathrm{MOD}(9)$, the outcome is the remainder\n", "of the division $13/9$, namely $4$.\n", "\n", - "\n", - "\n", - "\n", - "## Random number generator RNG and periodic outputs\n", - "\n", "The problem with such generators is that their outputs are periodic;\n", "they \n", "will start to repeat themselves with a period that is at most $M$. If however\n", @@ -1818,7 +2150,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "d12a26d7", + "metadata": { + "editable": true + }, "source": [ "$$\n", "N_i=(6N_{i-1}+7) \\mathrm{MOD} (5),\n", @@ -1827,7 +2162,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "2ee32cc9", + "metadata": { + "editable": true + }, "source": [ "with a seed $N_0=2$. This generator produces the sequence\n", "$4,1,3,0,2,4,1,3,0,2,...\\dots$, i.e., a sequence with period $5$.\n", @@ -1837,7 +2175,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "441be869", + "metadata": { + "editable": true + }, "source": [ "$$\n", "N_i=(27N_{i-1}+11) \\mathrm{MOD} (54),\n", @@ -1846,15 +2187,14 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "c0803129", + "metadata": { + "editable": true + }, "source": [ "which still, with $N_0=2$, results in $11,38,11,38,11,38,\\dots$, a period of\n", "just $2$.\n", "\n", - "\n", - "\n", - "\n", - "## Random number generator RNG and its period\n", "Typical periods for the random generators provided in the program library \n", "are of the order of $\\sim 10^9$ or larger. Other random number generators which have\n", "become increasingly popular are so-called shift-register generators.\n", @@ -1867,7 +2207,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "43aef6df", + "metadata": { + "editable": true + }, "source": [ "$$\n", "N_l=(aN_{l-i}+cN_{l-j})\\mathrm{MOD}(M).\n", @@ -1876,9 +2219,11 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "1c7bddf4", + "metadata": { + "editable": true + }, "source": [ - "## Random number generator RNG, other examples\n", "Such a generator again produces a sequence of pseudorandom numbers\n", "but this time with a period much larger than $M$.\n", "It is also possible to construct more elaborate algorithms by including\n", @@ -1889,7 +2234,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "fed2dc23", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -1904,14 +2252,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "dc9d8537", + "metadata": { + "editable": true + }, "source": [ "followed by" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "aa9465e9", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -1926,21 +2280,23 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "def99a7e", + "metadata": { + "editable": true + }, "source": [ "which according to the authors has a period larger than $2^{94}$.\n", "\n", - "\n", - "\n", - "\n", - "## Random number generator RNG, other examples\n", "Instead of using modular addition, we could use the bitwise\n", "exclusive-OR ($\\oplus$) operation so that" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "61c8f9ef", + "metadata": { + "editable": true + }, "source": [ "$$\n", "N_l=(N_{l-i})\\oplus (N_{l-j})\n", @@ -1949,7 +2305,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "c59c90b8", + "metadata": { + "editable": true + }, "source": [ "where the bitwise action of $\\oplus$ means that if $N_{l-i}=N_{l-j}$ the result is\n", "$0$ whereas if $N_{l-i}\\ne N_{l-j}$ the result is\n", @@ -1960,20 +2319,17 @@ "\n", "In Fortran90, the bitwise $\\oplus$ operation is coded through the intrinsic\n", "function $\\mathrm{IEOR}(m,n)$ where $m$ and $n$ are the input numbers, while in $C$\n", - "it is given by $m\\wedge n$.\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "## Random number generator RNG, RAN0\n", + "it is given by $m\\wedge n$. \n", "\n", "We show here how the linear congruential algorithm can be implemented, namely" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "06587f00", + "metadata": { + "editable": true + }, "source": [ "$$\n", "N_i=(aN_{i-1}) \\mathrm{MOD} (M).\n", @@ -1982,7 +2338,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "c3255e8e", + "metadata": { + "editable": true + }, "source": [ "However, since $a$ and $N_{i-1}$ are integers and their multiplication \n", "could become greater than the standard 32 bit integer, there is a trick via \n", @@ -1992,7 +2351,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "8e3ec364", + "metadata": { + "editable": true + }, "source": [ "$$\n", "M=aq+r,\n", @@ -2001,14 +2363,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "4b7c18d5", + "metadata": { + "editable": true + }, "source": [ "where we have defined" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "4e58c88d", + "metadata": { + "editable": true + }, "source": [ "$$\n", "q=[M/a],\n", @@ -2017,14 +2385,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "1fecdb9a", + "metadata": { + "editable": true + }, "source": [ "and" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "aa126ce7", + "metadata": { + "editable": true + }, "source": [ "$$\n", "r = M\\hspace{0.1cm}\\mathrm{MOD} \\hspace{0.1cm}a.\n", @@ -2033,24 +2407,23 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "20cab2fa", + "metadata": { + "editable": true + }, "source": [ "where the brackets denote integer division. In the code below the numbers \n", "$q$ and $r$ are chosen so that $r < q$.\n", "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "## Random number generator RNG, RAN0\n", - "\n", "To see how this works we note first that" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "904a6320", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -2065,23 +2438,24 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "36c6f816", + "metadata": { + "editable": true + }, "source": [ "since we can add or subtract any integer multiple of $M$ from $aN_{i-1}$.\n", "The last term $[N_{i-1}/q]M\\mathrm{MOD}(M)$ is zero since the integer division \n", - "$[N_{i-1}/q]$ just yields a constant which is multiplied with $M$.\n", + "$[N_{i-1}/q]$ just yields a constant which is multiplied with $M$. \n", "\n", - "\n", - "\n", - "\n", - "\n", - "## Random number generator RNG, RAN0\n", "We can now rewrite Eq. ([14](#eq:rntrick1)) as" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "319af13b", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -2096,15 +2470,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "0c426357", + "metadata": { + "editable": true + }, "source": [ - "which results\n", - "in" + "which results" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "52547452", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -2119,14 +2498,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "86bf958f", + "metadata": { + "editable": true + }, "source": [ "yielding" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "d2015392", + "metadata": { + "editable": true + }, "source": [ "\n", "
    \n", @@ -2141,9 +2526,11 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "83495015", + "metadata": { + "editable": true + }, "source": [ - "## Random number generator RNG, RAN0\n", "The term $[N_{i-1}/q]r$ is always smaller or equal $N_{i-1}(r/q)$ and with $r < q$ we obtain always a \n", "number smaller than $N_{i-1}$, which is smaller than $M$. \n", "And since the number $N_{i-1}\\mathrm{MOD} (q)$ is between zero and $q-1$ then\n", @@ -2155,52 +2542,7 @@ "the starting point for each generation of a random number. The period\n", "of $ran0$ is $\\sim 2.1\\times 10^{9}$. A special feature of this\n", "algorithm is that is should never be called with the initial seed \n", - "set to $0$.\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "## Random number generator RNG, RAN0 code" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - " /*\n", - " ** The function\n", - " ** ran0()\n", - " ** is an \"Minimal\" random number generator of Park and Miller\n", - " ** Set or reset the input value\n", - " ** idum to any integer value (except the unlikely value MASK)\n", - " ** to initialize the sequence; idum must not be altered between\n", - " ** calls for sucessive deviates in a sequence.\n", - " ** The function returns a uniform deviate between 0.0 and 1.0.\n", - " */\n", - " double ran0(long &idum)\n", - " {\n", - " const int a = 16807, m = 2147483647, q = 127773;\n", - " const int r = 2836, MASK = 123459876;\n", - " const double am = 1./m;\n", - " long k;\n", - " double ans;\n", - " idum ^= MASK;\n", - " k = (*idum)/q;\n", - " idum = a*(idum - k*q) - r*k;\n", - " // add m if negative difference\n", - " if(idum < 0) idum += m;\n", - " ans=am*(idum);\n", - " idum ^= MASK;\n", - " return ans;\n", - " } // End: function ran0() \n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Properties of Selected Random Number Generators\n", + "set to $0$. \n", "\n", "As mentioned previously, the underlying PDF for the generation of\n", "random numbers is the uniform distribution, meaning that the \n", @@ -2216,16 +2558,15 @@ "Two additional measures are the standard deviation $\\sigma$ and the mean\n", "$\\mu=\\langle x\\rangle$.\n", "\n", - "\n", - "\n", - "\n", - "## Properties of Selected Random Number Generators\n", "For the uniform distribution, the mean value $\\mu$ is then" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "268aac30", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\mu=\\langle x\\rangle=\\frac{1}{2}\n", @@ -2234,14 +2575,20 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "25bd4ea3", + "metadata": { + "editable": true + }, "source": [ "while the standard deviation is" ] }, { "cell_type": "markdown", - "metadata": {}, + "id": "29e8d709", + "metadata": { + "editable": true + }, "source": [ "$$\n", "\\sigma=\\sqrt{\\langle x^2\\rangle-\\mu^2}=\\frac{1}{\\sqrt{12}}=0.2886.\n", @@ -2250,13 +2597,15 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "90a07c88", + "metadata": { + "editable": true + }, "source": [ - "## Properties of Selected Random Number Generators\n", "The various random number generators produce results which agree rather well with\n", "these limiting values. \n", "\n", - "\n", + "
    \n", "\n", "\n", "\n", @@ -2276,16 +2625,13 @@ "\n", "
    $x$-bin ran0 ran1 ran2 ran3
    \n", "\n", - "\n", - "\n", - "\n", - "## Simple demonstration of RNGs using python\n", "The following simple Python code plots the distribution of the produced random numbers using the linear congruential RNG employed by Python. The trend displayed in the previous table is seen rather clearly." ] }, { "cell_type": "code", "execution_count": 4, + "id": "3c296708", "metadata": { "collapsed": false, "editable": true @@ -2293,14 +2639,14 @@ "outputs": [ { "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAESCAYAAAAMifkAAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAATYUlEQVR4nO3dwW8b553G8ee3CJBD0pSRFQQF7EPp1L7sIabkc9OEbq+LXSXF9h45vQuOfVr0UleOetnLVuofsHCs/QM24m5ylyjnsKcsJGeBAEWBRKEXSYFiD789zDueEU3ynZE0nCH5/QCCOS/JmZev5Hk47zvzjrm7AACY5G/qrgAAoPkICwBAFGEBAIgiLAAAUYQFACCKsAAARBEWODMzWzez7dy/R2a2ZmZ3zGyvxHpa56hDJ93+WddR5/rHbLNtZn0z65R4T6vEa7tm9u0Z61Z4O5gvhAXOxd1vu/uOpEeSjt19190fhOWi3jtHFe5J+tjdb59jHaeY2VGV649x92NJvQl1GiXahuk63L0n6fiM1Tu1nQL1wpwgLHAek3Y4ZXZG59kRt9x9cI73j7JS8frPYiXyfJE2zK/j5Iz1GN5OrF6YE4QFzix8Qx37XOi+6YZuqVboXumGrp1NKekSkdQK3Vdju13C893031DWkbQUttMeen0n/dab69ZJ39c1s2/Da7r5Lqawzv649Y+px1roguua2SMz+7vh9Yc6dNLHEz5jut52rjxfp0JtOKJOb6XrCJZy67mTdi/F2i2/nXy9JrTN2LbGjHF3fvg594+krqS93HJH0mZ43JK0LWlTyTd1SVrLvbYfWXdH0p3c8nZuPXsT3pevzx1J3fxzkjrh8ebwc2MeT6xHeL49Zv13cu20OeYzbg6tuzNcjzJtOKpO+dfn1tOS9Khgu/WHt1G0bUa1NT+z88ORBaryS0nfhG+f7fDzUFI/fLss2011mFs+UrLTPa+0Dt9cQD2W3P3Qk/GGUetPH4/r/vmlpP3c8mDM68q04ag6PeOhey38W3gwfYzY76hsW6NhCAtUpSXpMOysDt39lpIdxoqSb5rPDYCP654ZY+m8FfSLGYtI6/FcCAyt/yK2JZVrw7OOS0QV/F09+x1dUFujRoQFqvJI0q10IRxhrLv7wN13JW3mTsNMd2rjdkCPdPqb700NnS00xiC3jZtKAuw8zlqPIh6G9aVaY1531jZ8Tm6coqXTRwWT2m3cdqpsGzQAYYFzC4OZ70pqh8HNlieD3+kAazf32nQQ9jj3bXPbzNbHrT+sa5AOnkradvfjtIsrP0A7ZFvSe2H7J5Juh4H2/PvaSkLt3aHn1ofXP6EeXUmr6WeYtH4l3TXd4W/m7n6Ya6+Okm/l94bWl64/2oYT6pS+Zjs835W0Lun9WLsNbye/zoK/o+G2wAwxd+5nAQCYjCMLAEAUYQEAiCIsAABRhAUAIOqFuitwHq1Wy9944426q9EI33//vV566aW6q9EItEWGtsjQFpl+v/+1u79W5j0zHRavv/66Dg4O6q5GI3z22Wd666236q5GI9AWGdoiQ1tkzOx/yr6HbigAQBRhAQCIIiwAAFGEBQAgirAAAEQRFgCAKMICABA109dZAGg++43VXQVJ0qc//bTuKsw0jiwAAFGEBQAgirAAAEQRFgCAKMICABDF2VAAMCVNOTPsLDiyAABEVRIWZrZmZnsjyrpmtl62DABQr0q6odx918xup8tmthbKe2a2bmZdSa0iZe7eq6KOABZL/099/ew3P6u7GjPL3L2aFZvtufut8HhT0kN3Pwyh0JF0qUiZuz8YWu+6pHVJeu2111Y+/vjjSuo/a/588md99devaq3Dyo9Wat1+6rvvvtPLL79cax36f+rXuv3U9R9cpy2Cyy9erv3/SFNs/Gqj7+6rZd4zrQHu1tDypRJlp7j7jqQdSbp+/bpzm8TE7//199r4YqPWOvg/VvPFo6wm3D6zKd9gP/3pp7RFsHVtq/b/I7NsWmExkLR0xrKx/vJ/f6n97AL/p2bsIJGhuyFDW+CiTCss9pUdNbQl7YXlImUAgJpVdTZUV9JqbmB7V1I7HcR2917RsirqBwAop6qzoXqSXh0qSweqe2XLAAD14gpuXJi6x49SW9e26q4CMHcIi3NiBwlgETDdBwAgirAAAEQRFgCAKMICABBFWAAAoggLAEAUYQEAiCIsAABRhAUAIIqwAABEERYAgCjCAgAQRVgAAKIICwBAFGEBAIgiLAAAUYQFACCKsAAARBEWAIAowgIAEEVYAACiCAsAQBRhAQCIIiwAAFGEBQAgirAAAEQRFgCAKMICABBFWAAAol6Y1obMbE3SQFLb3XfKlAEA6jWVIwsz60o6dveepGMz64RQUCiTmXVHlU2jfgCAyabVDXUg6ZGZdZQcMRxKuinpODx/LKkzpgwAULOpdEO5+8DMtiU9ktQLxa2hl10aU3aKma1LWpek5eVlbV3butC6zqrLL16mLQLaIkNbZGiLzIY2Sr9nKmERupN67v7AzDZz4xJLQy8dVXZKGMfYkaQr7Su+8UX5Dz2Ptq5tibZI0BYZ2iJDW5zPtLqhOqHrSZLuKwmEfWVHEm1Je2PKAAA1m9bZUDuh++hYp898uhOOOlq5Qe3nygAA9ZramIVC19FQ+YPwsDepDABQLy7KAwBEERYAgCjCAgAQRVgAAKIICwBAFGEBAIgqdOqsmb2t5BqJlqSupF13/7K6agEAmqTodRYtd//SzP5b0oqSq6sBAAuiaDfU03B08djd/1eEBQAslKJhcSLp55LeN7N/UDKVOABgQRTqhnL3x5IeS5KZHYupOABgoRQ6sghdUJKeBcdKZTUCADTOxCOL0OV0S9KqmR1JsvDUkaT/rLhuAICGmBgW7v5vZtZTMq3447TczF6pvGYAgMaIjlm4+1Mzk5ndD0Um6YakX1RaMwBAYxS9zqKr0/ej6FZQFwBAQxUNi767P0kXzIzbnQLAAikaFnfNbFvSobJuqJ9UVisAQKMUDYtNd/+PdMHMblRUHwBAAxW6ziIfFMFRBXUBADRU0Vln7+cXJb0jpvwAgIVRtBvqRNJueNwWRxYAsFCKzg31UW7xiZl9U1F9AAANVLQb6hNJ3yrpgnJJ+5I+r65aAIAmOdPZUACAxVL4bCgze9/MHprZRtWVAgA0S9Epyt9Xcg/uu5IeExgAsFiKdkMd5GadfWJmE18MAJgvRcNi1cxc0kDJqbM3JDGGAQALouiYxR+V3ARpR9Itd9+qtFYAgEYpeursO5IuufvPzeyHZva2u3OnPABYEIWOLCR94+53peRmSEqu6AYALIiiYxa3zKyt5IyoJSVdUp+X2ZCZdZSMd8jdd0PZmsI4iLvvjCsDANSr6JjFR0qu3v5AUsfd751hW/dCSCyZWTuEgty9J0lm1h1VdobtAAAumLl79RsxW5fUkrTr7sehbFPSQ3c/DKHQkXRpuMzdH4xY17okLS8vr9z957uV138WXH7xsr7661d1V6MRaIsMbZGhLTIbv9rou/tqmfcU7YY6r6vh35Nwx70PlYRH3qUxZaeErqkdSbrSvuIbX3B9oCRtXdsSbZGgLTK0RYa2OJ+iA9wX4cjdB5L6So4MBkrGP/JGlQEAajYxLMzs7y9oO/u5xy0lobCv7EiiLWlvTBkAoGZjw8LMfqzk7CeZ2dtDz71ZZiNhYLuVDli7+04oa4eylrv3RpWV+jQAgEqMHbNw9ydm9gczW1KyAz9SckaUlEz38ZMyG8oNVPfKlgEA6jVxgNvdP5AkM7uRm0hQZnaj6ooBAJqj6G1VH4dpyruS9pkbCgAWC/ezAABEcT8LAEAU97MAAERxPwsAQFTh6T7CZIIfVVgXAEBDTXO6DwDAjCIsAABRRU+d5VRZAFhgRY8sjvMLZeeGAgDMtqID3B+EmxUdKpkfqvTcUACA2VU0LDbd/dl1FWb2TkX1AQA0UNFuqLaZ/YuZvWlmP5RU/b1YAQCNUTQsjtz910ru2f20ygoBAJqnaFishEHtV8ONkDrVVQkA0DRFxyx2JN1TEhKfMN0HACyWovezeCrprpn92N2fVFwnAEDDFL0o7x0zO5D0BzN7OHxPbgDAfCvaDdVy99V0gduqAsBiKTrA/W1kGQAwxyYeWZjZx0quqbhqZt9IeiqpJelI0q8rrx0AoBFi3VDb+Su3AQCLaWJYpEFhZq9IWs09dcvd71VZMQBAcxQds3gg6aqkV3M/AIAFUfRsqEdDEwn2K6oPAKCBCp86a2b/rtNTlP+isloBABqlaFi0JX2QW+5WUBcAQEMVDYt+fpoPM9urqD4AgAYqGhZ3zWxb3CkPABbSWe+Ux3QfALBACp06O+LCvKOzbjDcyzt9vGZmXTNbn1QGAKhXoSMLM7ufX5T0jqSbZTdmZl0lg+UyszVJcveema2H51rDZe7eK7sdAMDFKtoNdSJpNzxu6wxHFmbWlnScK7op6WF4fKzkxkqXRpQRFgBQs6I3P/oot/gkTCpYVjscMaTLraHnL40pOyV0T61L0vLysraucdM+Sbr84mXaIqAtMrRFhrbIbGij9HuKdkN9omRaclMyC+2+pM+LbmRMd9JA0lKBslPcfUfJbV51pX3FN74o/6Hn0da1LdEWCdoiQ1tkaIvzOdPZUGdwkhuTaJtZR0ngtMLzbUl7YXm4DABQs4lnQ5nZ/TDj7IGZvZL/KbMRdz8MRxZLygaxd5UER1fJnfh6o8rO8qEAABcrdmTxqpJv+E+VdD/dkrQp6beSSnf+5buQwvKD8LA3qQwAUK9YWHzo7k8lycx+p+SU2ZX81B8AgPk3sRvK3Z+a2Q0zO5D0tbvfJCgAYPHE7sH9OyXzQK25+5e58jfd/fNqqwYAaIpYN1RHyRhFO1xUJyWnz94R97MAgIVRZMzi8XChmZ1UVB8AQAPFxiyeC4pJ5QCA+VRo1lkAwGIjLAAAUYQFACCKsAAARBEWAIAowgIAEEVYAACiCAsAQBRhAQCIIiwAAFGEBQAgirAAAEQRFgCAKMICABBFWAAAoggLAEAUYQEAiCIsAABRhAUAIIqwAABEERYAgCjCAgAQRVgAAKIICwBAFGEBAIgiLAAAUS9MYyNm1pLUDj833f3DUL4maSCp7e4748oAAPWa1pHFe5JW3X1XksxsPYSC3L0XyrqjyqZUPwDABFMJC3ffyR0ltCUdS7oZ/lX4tzOmDABQs6l0Q6XMrC3pxN17Zvbu0NOXJLVGlA2vY13SuiQtLy9r69pWFVWdOZdfvExbBLRFhrbI0BaZDW2Ufs9Uw0LSmrvfDo8HkpaGnh9Vdko4QtmRpCvtK77xRfkPPY+2rm2JtkjQFhnaIkNbnM/UwsLM1tz9QXjckbSv7EiiLWkvLA+XAQBqNpUxizBQvWlmfTPrS1oKg93t8FzL3XujyqZRPwDAZFM5sgg7/asjyh+Eh71JZQCAenFRHgAgirAAAEQRFgCAKMICABBFWAAAoggLAEAUYQEAiCIsAABRhAUAIIqwAABEERYAgCjCAgAQRVgAAKIICwBAFGEBAIgiLAAAUYQFACCKsAAARBEWAIAowgIAEEVYAACiCAsAQBRhAQCIIiwAAFGEBQAgirAAAEQRFgCAKMICABBFWAAAoggLAEAUYQEAiHqh7goMM7M1SQNJbXffqbk6AAA17MgiBIXcvReWu/XWCAAgNSwsJN2UdBweH0vq1FgXAEBg7l53HZ4xs21J2+5+GI4qbrn7h0OvWZe0Hhb/VtJ/TbmaTbUs6eu6K9EQtEWGtsjQFpnr7v6DMm9o2pjFQNLSpBeEcYwdSTKzA3dfnUK9Go+2yNAWGdoiQ1tkzOyg7Hua1g21L6kVHrcl7dVXFQBAqlFh4e67ktqhC6qVDnQDAOrVtG4oufuD8LBIUHBqbYa2yNAWGdoiQ1tkSrdFowa4AQDN1KhuKABAMxEWAICoxo1ZjBObBmSRpgmZ9FnNrKXkTLK2pJvD16nMm6K/dzPbnOe2KPD/o6PkbyI9kWRusa/IhM96291vTXh+oAJtMRNHFrFpQBZpmpACn/U9SavpDiFcxDiXiv7eQ3l7ilWbqoLtcC/8TSyZ2cK2RVg+Ds8fhxCdW5O+GJTdb85EWCg+DcgiTRMy8bO6+07uG0I799p5FP29hx3jPLeBFGmH8IVh38za4e9jntsj9jdxIOlReqTl7ofTrFzDlNpvzkpYtIaWL5V8fp60hpZHftawkzyZ82tVWkPLo9qiPec7RyneDldD2YmZbYeuynnVGlo+1RbuPpC0LemRpJXpVKmxWkPLE/ebsxIWA02eBiT2/DwZqNhnXXP32xXXpW4DTWgLM+vOeVimBor/TRyFHWVf2dxq82igyN+EpJ67X5U0SLtiFtRAJfabsxIWsWlAFmmakOhnNbO19OLGOe+TjbXFiZl1ww6hPcdtUeT/R6qlZCcxr2Jt0cl1Pd3X4nzJHKXUfnMmwmLcNCBmtjfp+XkUa4tQvmlmfTPra47/MxT4uzgMZUt6/pB7bhT8/9FKBzDn+QygWFtI2jGz9fD8e/PcFtKz/cFq/gjqrPtNruAGAETNxJEFAKBehAUAIIqwAABEERYAgCjCAgAQRVgAAKIICwBAFGEBXDAzWzOzO2bWDheAtequE3BeM3M/C2AWhClF0ukk0ulFljTfU2xgAXAFN1ABM9uW9GGYvA+YeXRDARcodD+1lEyNvuizmmKOcGQBXKBwo6H8/TOOF+B+GlgAhAUAIIpuKABAFGEBAIgiLAAAUYQFACCKsAAARBEWAIAowgIAEPX/6wKoYq3Y9LwAAAAASUVORK5CYII=\n", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAESCAYAAAAMifkAAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAATVklEQVR4nO3dsW8bZ5rH8d9zCJAi2SwjK0hjFzvO2s0VMS3Xm03o3fZwp2SB7SNne8Gxq8M255Wj6ppb6f4Cx7o/YCPuJb1NOcVVOUjOAQG2SRT6kCywuOK5Yt7xjGiS74yk4QzJ7wcQxHmHnHn5iJqH7/vOvGPuLgAApvm7pisAAGg/kgUAIIpkAQCIIlkAAKJIFgCAKJIFACCKZIFTM7MNM9sp/D40s3Uzu21m+xW20zlDHbrZ/k+7jSa3P2GfiZkNzKxb4TWdCs/tmdn3p6xb6f1gsZAscCbufsvddyU9lHTk7nvufj8sl/XBGapwV9Kn7n7rDNs4wcwO69x+jLsfSepPqdM40Rhm23D3vqSjU1bvxH5K1AsLgmSBs5h2wKlyMDrLgbjj7sMzvH6c6zVv/zSuR9aXiWFxG8enrMfofmL1woIgWeDUwjfUietC900vdEt1QvdKL3TtbElpl4ikTui+mtjtEtb3st+hrCtpJewnGXl+N/vWW+jWyV7XM7Pvw3N6xS6msM3BpO1PqMd66ILrmdlDM/uH0e2HOnSzx1PeY7bdpFBerFOpGI6p0zvZNoKVwnZuZ91LsbgV91Os15TYTIw15oy788PPmX8k9STtF5a7krbC446kHUlbSr+pS9J64bmDyLa7km4XlncK29mf8rpifW5L6hXXSeqGx1uj6yY8nlqPsD6ZsP3bhThtTXiPWyPb7o7Wo0oMx9Wp+PzCdjqSHpaM22B0H2VjMy7W/MzPDy0L1OU3kr4L3z6T8PNA0iB8u6zaTXVQWD5UetA9q6wO351DPVbc/cDT8YZx288eT+r++Y2kR4Xl4YTnVYnhuDo956F7LfwuPZg+QexvVDXWaBmSBerSkXQQDlYH7n5T6QHjutJvmi8MgE/qnplg5awV9PMZi8jq8UISGNn+eexLqhbD045LRJX8Wz3/G51TrNEgkgXq8lDSzWwhtDA23H3o7nuStgqnYWYHtUkHoIc6+c33hkbOFppgWNjHDaUJ7CxOW48yHoTtZToTnnfaGL6gME7R0clWwbS4TdpPnbFBC5AscGZhMPN9SUkY3Ox4OvidDbD2Cs/NBmGPCt82d8xsY9L2w7aG2eCppB13P8q6uIoDtCN2JH0Q9n8s6VYYaC++LlGa1N4fWbcxuv0p9ehJWsvew7TtK+2u6Y1+M3f3g0K8ukq/ld8d2V62/WgMp9Qpe85OWN+TtCHpw1jcRvdT3GbJv9FoLDBHzJ37WQAApqNlAQCIIlkAAKJIFgCAKJIFACDqpaYrcBadTsffeuutpqvRCj/++KNeeeWVpqvRCsQiRyxyxCI3GAy+dfc3qrxmrpPFm2++qcePHzddjVb44osv9M477zRdjVYgFjlikSMWOTP7n6qvoRsKABBFsgAARJEsAABRJAsAQBTJAgAQRbIAAESRLAAAUSQLAEDUXF+UB7SV/d6aroIk6fNffN50FbAgaFkAAKJoWQCoFa2sxUDLAgAQRcsCWGCDvwz0y9//sulqYAHQsgAARNGyALAU2tDK8n/2Rvd/FrUkCzNbl3TL3W+OlA0lJe6+W6WszRi8A7AMaumGcve94nJIAHL3fljulS2ro34AgGrMvZ5mkZntZy0LM9uS9MDdD0IC6Eq6UKbM3e+PbHdD0oYkvfHGG9c//fTTWupf1uAvg0b3n7n6k6t69dVXm65GK/zwww+Nx6Itn4uLL1/UN3/7pulqtAKxyG3+dnPg7mtVXjOrMYvOyPKFCmUnhK6pXUm6evWqN32bxKb7QDOf/+JzbhkZtOH2mW35XGxf2dbmV5tNV6MViMXZzOpsqKGklVOWAQAaNquWxSPlrYZE0n5YLlMGVNKGs16ARVNLyyKMN6wVBqz3JCWhvOPu/bJlddQPAFBNLS2LcJB/faQsG6juVy0DADSLK7gBAFFcwY1z05YLFLevbDddBWDhzHWy+Ov//bU1BygAWGR0QwEAokgWAIAokgUAIGquxyyQ40I0AHWiZQEAiCJZAACiSBYAgCiSBQAgimQBAIgiWQAAokgWAIAokgUAIIpkAQCIIlkAAKJIFgCAKJIFACCKZAEAiCJZAACiSBYAgCiSBQAgimQBAIgiWQAAokgWAIAokgUAIIpkAQCIIlkAAKJIFgCAKJIFACDqpVntyMzWJQ0lJe6+W6UMANCsmbQszKwn6cjd+5KOzKwbkoJCmcysN65sFvUDAEw3q26ox5IemllXaYvhQNINSUdh/ZGk7oQyAEDDZtIN5e5DM9uR9FBSPxR3Rp52YULZCWa2IWlDklZXV7V9Zftc6zqvLr58kVgExCJHLHLEIrepzcqvmUmyCN1JfXe/b2ZbhXGJlZGnjis7IYxj7ErSpeSSb35V/U0vou0r2yIWKWKRIxY5YnE2s+qG6oauJ0m6pzQhPFLekkgk7U8oAwA0bFZnQ+2G7qMjnTzz6XZodXQKg9ovlAEAmjWzMQuFrqOR8vvhYX9aGQCgWVyUBwCIIlkAAKJIFgCAKJIFACCKZAEAiCJZAACiSp06a2bvKr1GoiOpJ2nP3b+ur1oAgDYpe51Fx92/NrP/lnRd6dXVAIAlUbYb6lloXTxx9/8VyQIAlkrZZHEs6VeSPjSzf1I6lTgAYEmU6oZy9yeSnkiSmR2JqTgAYKmUalmELihJzxPH9dpqBABonakti9DldFPSmpkdSrKw6lDSf9ZcNwBAS0xNFu7+H2bWVzqt+JOs3Mxeq71mAIDWiI5ZuPszM5OZ3QtFJumapF/XWjMAQGuUvc6ip5P3o+jVUBcAQEuVTRYDd3+aLZgZtzsFgCVSNlncMbMdSQfKu6F+XlutAACtUjZZbLn7n7MFM7tWU30AAC1U6jqLYqIIDmuoCwCgpcrOOnuvuCjpPTHlBwAsjbLdUMeS9sLjRLQsAGCplJ0b6pPC4lMz+66m+gAAWqhsN9Rnkr5X2gXlkh5J+rK+agEA2uRUZ0MBAJZL6bOhzOxDM3tgZpt1VwoA0C5lpyj/UOk9uO9IekLCAIDlUrYb6nFh1tmnZjb1yQCAxVI2WayZmUsaKj119pokxjAAYEmUHbP4d6U3QdqVdNPdt2utFQCgVcqeOvuepAvu/isz+6mZvevu3CkPAJZEqZaFpO/c/Y6U3gxJ6RXdAIAlUXbM4qaZJUrPiFpR2iX1ZZUdmVlX6XiH3H0vlK0rjIO4++6kMgBAs8qOWXyi9OrtjyR13f3uKfZ1NySJFTNLQlKQu/clycx648pOsR8AwDkzd69/J2YbkjqS9tz9KJRtSXrg7gchKXQlXRgtc/f7Y7a1IUmrq6vX7/zrndrrPw8uvnxR3/ztm6ar0QrEIkcscsQit/nbzYG7r1V5TdluqLO6HH4fhzvufaw0eRRdmFB2Quia2pWkS8kl3/yK6wMlafvKtohFiljkiEWOWJxN2QHu83Do7kNJA6Utg6HS8Y+icWUAgIZNTRZm9o/ntJ9HhccdpUnhkfKWRCJpf0IZAKBhE5OFmf1M6dlPMrN3R9a9XWUnYWC7kw1Yu/tuKEtCWcfd++PKKr0bAEAtJo5ZuPtTM/ujma0oPYAfKj0jSkqn+/h5lR0VBqr7VcsAAM2aOsDt7h9JkpldK0wkKDO7VnfFAADtUfa2qk/CNOU9SY+YGwoAlgv3swAARHE/CwBAFPezAABEcT8LAEBU6ek+wmSCn9RYFwBAS81yug8AwJwiWQAAosqeOsupsgCwxMq2LI6KC1XnhgIAzLeyA9wfhZsVHSidH6ry3FAAgPlVNllsufvz6yrM7L2a6gMAaKGy3VCJmf2bmb1tZj+VVP+9WAEArVE2WRy6+++U3rP7WZ0VAgC0T9lkcT0Mar8eboTUra9KAIC2KTtmsSvprtIk8RnTfQDAcil7P4tnku6Y2c/c/WnNdQIAtEzZi/LeM7PHkv5oZg9G78kNAFhsZbuhOu6+li1wW1UAWC5lB7i/jywDABbY1JaFmX2q9JqKy2b2naRnkjqSDiX9rvbaAQBaIdYNtVO8chsAsJymJossUZjZa5LWCqtuuvvdOisGAGiPsmMW9yVdlvR64QcAsCTKng31cGQiwUFN9QEAtFDpU2fN7E86OUX5r2urFQCgVcomi0TSR4XlXg11AQC0VNlkMShO82Fm+zXVBwDQQmWTxR0z2xF3ygOApXTaO+Ux3QcALJFSp86OuTDv8LQ7DPfyzh6vm1nPzDamlQEAmlWqZWFm94qLkt6TdKPqzsysp3SwXGa2Lknu3jezjbCuM1rm7v2q+wEAnK+y3VDHkvbC40SnaFmYWSLpqFB0Q9KD8PhI6Y2VLowpI1kAQMPK3vzok8Li0zCpYFVJaDFky52R9RcmlJ0Quqc2JGl1dVXbV7hpnyRdfPkisQiIRY5Y5IhFblOblV9TthvqM6XTkpvSWWgfSfqy7E4mdCcNJa2UKDvB3XeV3uZVl5JLvvlV9Te9iLavbItYpIhFjljkiMXZnOpsqFM4LoxJJGbWVZpwOmF9Imk/LI+WAQAaNvVsKDO7F2acfWxmrxV/quzE3Q9Cy2JF+SD2ntLE0VN6J77+uLLTvCkAwPmKtSxeV/oN/5nS7qebkrYk/Yukyp1/xS6ksHw/POxPKwMANCuWLD5292eSZGZ/UHrK7PXi1B8AgMU3tRvK3Z+Z2TUzeyzpW3e/QaIAgOUTuwf3H5TOA7Xu7l8Xyt929y/rrRoAoC1i3VBdpWMUSbioTkpPn70t7mcBAEujzJjFk9FCMzuuqT4AgBaKjVm8kCimlQMAFlOpWWcBAMuNZAEAiCJZAACiSBYAgCiSBQAgimQBAIgiWQAAokgWAIAokgUAIIpkAQCIIlkAAKJIFgCAKJIFACCKZAEAiCJZAACiSBYAgCiSBQAgimQBAIgiWQAAokgWAIAokgUAIIpkAQCIIlkAAKJIFgCAKJIFACCKZAEAiHppFjsxs46kJPzccPePQ/m6pKGkxN13J5UBAJo1q5bFB5LW3H1PksxsIyQFuXs/lPXGlc2ofgCAKWaSLNx9t9BKSCQdSboRfiv87k4oAwA0bCbdUBkzSyQdu3vfzN4fWX1BUmdM2eg2NiRtSNLq6qq2r2zXUdW5c/Hli8QiIBY5YpEjFrlNbVZ+zUyThaR1d78VHg8lrYysH1d2Qmih7ErSpeSSb35V/U0vou0r2yIWKWKRIxY5YnE2M0sWZrbu7vfD466kR8pbEomk/bA8WgYAaNhMxizCQPWWmQ3MbCBpJQx2J2Fdx93748pmUT8AwHQzaVmEg/7lMeX3w8P+tDIAQLO4KA8AEEWyAABEkSwAAFEkCwBAFMkCABBFsgAARJEsAABRJAsAQBTJAgAQRbIAAESRLAAAUSQLAEAUyQIAEEWyAABEkSwAAFEkCwBAFMkCABBFsgAARJEsAABRJAsAQBTJAgAQRbIAAESRLAAAUSQLAEAUyQIAEEWyAABEkSwAAFEkCwBAFMkCABBFsgAARJEsAABRLzVdgVFmti5pKClx992GqwMAUMtaFiFRyN37YbnXbI0AAFLLkoWkG5KOwuMjSd0G6wIACMzdm67Dc2a2I2nH3Q9Cq+Kmu3888pwNSRth8e8l/deMq9lWq5K+bboSLUEscsQiRyxyV939J1Ve0LYxi6GklWlPCOMYu5JkZo/dfW0G9Wo9YpEjFjlikSMWOTN7XPU1beuGeiSpEx4nkvabqwoAINOqZOHue5KS0AXVyQa6AQDNals3lNz9fnhYJlFwam2OWOSIRY5Y5IhFrnIsWjXADQBop1Z1QwEA2olkAQCIat2YxSSxaUCWaZqQae/VzDpKzyRLJN0YvU5l0ZT9u5vZ1iLHosT/R1fpZyI7kWRhcazIhfd6y91vTlk/VIlYzEXLIjYNyDJNE1LivX4gaS07IISLGBdS2b97KE9mWLWZKhmHu+EzsWJmSxuLsHwU1h+FJLqwpn0xqHrcnItkofg0IMs0TcjU9+ruu4VvCEnhuYso+ncPB8ZFjoEUiUP4wvDIzJLw+VjkeMQ+E48lPcxaWu5+MMvKtUyl4+a8JIvOyPKFiusXSWdkeex7DQfJ4wW/VqUzsjwuFsmCHxyleBwuh7JjM9sJXZWLqjOyfCIW7j6UtCPpoaTrs6lSa3VGlqceN+clWQw1fRqQ2PpFMlS597ru7rdqrkvThpoSCzPrLXiyzAwV/0wchgPlQPncaotoqMhnQlLf3S9LGmZdMUtqqArHzXlJFrFpQJZpmpDoezWz9ezixgXvk43F4tjMeuGAkCxwLMr8f2Q6Sg8SiyoWi26h6+meludL5jiVjptzkSwmTQNiZvvT1i+iWCxC+ZaZDcxsoAX+ZyjxuTgIZSt6scm9MEr+f3SyAcxFPgMoFgtJu2a2EdZ/sMixkJ4fD9aKLajTHje5ghsAEDUXLQsAQLNIFgCAKJIFACCKZAEAiCJZAACiSBYAgCiSBQAgimQBnDMzWzez22aWhAvAOk3XCTirubmfBTAPwpQi2XQS2fQiK1rsKTawBLiCG6iBme1I+jhM3gfMPbqhgHMUup86SqdGX/ZZTbFAaFkA5yjcaKh4/4yjJbifBpYAyQIAEEU3FAAgimQBAIgiWQAAokgWAIAokgUAIIpkAQCIIlkAAKL+HxrdslJII/kbAAAAAElFTkSuQmCC\n", "text/plain": [ "
    " ] }, "metadata": { "filenames": { - "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/statistics_178_0.png" + "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/statistics_181_0.png" }, "needs_background": "light" }, @@ -2334,9 +2680,11 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "7a028dda", + "metadata": { + "editable": true + }, "source": [ - "## Properties of Selected Random Number Generators\n", "Since our random numbers, which are typically generated via a linear congruential algorithm,\n", "are never fully independent, we can then define \n", "an important test which measures the degree of correlation, namely the so-called \n", @@ -2346,7 +2694,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "bc325e1a", + "metadata": { + "editable": true + }, "source": [ "$$\n", "C_k=\\frac{f_d}\n", @@ -2356,7 +2707,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "a7738ea8", + "metadata": { + "editable": true + }, "source": [ "with $C_0=1$. Recall that \n", "$\\sigma^2=\\langle x_i^2\\rangle-\\langle x_i\\rangle^2$ and that" @@ -2364,7 +2718,10 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "b647158e", + "metadata": { + "editable": true + }, "source": [ "$$\n", "f_d = \\frac{1}{nm}\\sum_{\\alpha=1}^m\\sum_{k=1}^{n-d}(x_{\\alpha,k}-\\langle X_m \\rangle)(x_{\\alpha,k+d}-\\langle X_m \\rangle),\n", @@ -2373,24 +2730,33 @@ }, { "cell_type": "markdown", - "metadata": {}, + "id": "273afa83", + "metadata": { + "editable": true + }, "source": [ "The non-vanishing of $C_k$ for $k\\ne 0$ means that the random\n", "numbers are not independent. The independence of the random numbers is crucial \n", "in the evaluation of other expectation values. If they are not independent, our\n", - "assumption for approximating $\\sigma_N$ is no longer valid.\n", + "assumption for approximating $\\sigma_N$ is no longer valid." + ] + }, + { + "cell_type": "markdown", + "id": "70391792", + "metadata": { + "editable": true + }, + "source": [ + "### Autocorrelation function\n", "\n", - "\n", - "\n", - "\n", - "\n", - "## Autocorrelation function\n", "This program computes the autocorrelation function as discussed in the equation on the previous slide for random numbers generated with the normal distribution $N(0,1)$." ] }, { "cell_type": "code", "execution_count": 5, + "id": "2b08e1a4", "metadata": { "collapsed": false, "editable": true @@ -2400,19 +2766,19 @@ "name": "stdout", "output_type": "stream", "text": [ - "0.05290417684691035 0.9902552771282336\n" + "-0.012633802944855959 1.0011828302640124\n" ] }, { "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAESCAYAAAAWtRmOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAlJ0lEQVR4nO3deZgU1bkG8Pcggzs2AxhAUdNg3CLiMESNcR83NCHLCPFqNJHrcCMaE424xSUkUTEu0Wi8M8aLSYxGwahRw9XBeF2jYQb33RmNK4jgKC6s890/vjrWcqq6e3q6p2tm3t/znKdrOVV1+nR3fVXnVFUbEQEREVHQgEoXgIiI0ofBgYiIHAwORETkYHAgIiIHgwMRETkYHIiIyMHgQBVnjMlUugyWMabOGNNWQL5MGctQY4xpMMY0lmHdmVKvk/omBgcqGWNMxhjTWsSiU0pemCKJyAIA7QVkdcpcSFAp0JkAbhGR6SVaX1A5y019CIMDlVIdgIwxJtvF5cqxEyy3uDJPKNG6MyLSUaJ1RZWz3NSHMDhQqTUisgPymknavOGsMabVGFPnjduAUm+MqQksU+818dQH8tbbfDZvjnxt3vS53hmNM83L2+BNmxnX5OLNqzPGzA4s45TZe20NLBdXrjpjzAde+evimo289VR75crmq7tc64vWVznLTX2QiDAxdTsByAKoAZAB0BYzvzkwPBNAXWC8NZK3BsDMwHgjgH0BNHrjGW9aXL6M3Z43PxssQ3CaNzw7uM6YstppdTZvXJmDyxVSLm94drAe4uqqgLqLXZ9XhsaY91a2cjP1rcQzByqVGhFZJNoc0hE8CyjCdACLAuNtAH4C3UFBRDpE2+Pj8tV5w9VeeYL9B9FpUwEs88qa9VKIhNv9M90of11g3G5/WYHryydufVPh1lc+PV1uSjEGByqVrG3GANCCIvoR8vRVbFDgaqq91+Ux86LTMgAWeQFjkYgcGFOm2V7zStz68pU5rlyQEvcnFLO+NJSb0o3BgbrNO/JuEpF5IjIPwOlwr4rpCLTpT0T4KNzueO0Oay60iQOB/FcD+Hzn7W0zLt+CLhR9bsw6ERhvALBM9AqmaJ5omaPr7U65onLVXZKbEf/eerLc1IsxOFC3eEfV1yLc/JD15jUGjlAbAUwJHIVPD+zwGr0dMYDPLyftsB2j0PbyvwFoDXSUdiTka/fm1wbXGTfNW952UtvO1xroWVADdMc4xs7z2CPpxsj6P18uR7lsnplevRwI4IhgR3gkz+f1E1d3udYnIoui9VXOclPfY0T4fw5ERBTGMwciInKUJTh4p7LNeebXBU9viYgoPcoSHLxOyVheW6Zt77VtwURElCKVaFaaCP966XaEr44gIqIUGFiBbWYi40OjGbzmpgYAyFRVTRgzblwPFIuIqO9obW19X0SGF7t8JYJDBwI31sQRkSYATQBQO2KEtLS09ECxiIj6DmPMv7uzfCWalRbCP3vIwrvFn4iI0qNcVyvZG47qA9Psc17mQW+oqYM+1It3YBIRpUzqb4KrHTFCWhYvrnQxiIh6FWNMq4jUFrs8b4IjIiIHgwMRETkYHIiIyMHgQEREDgYHIiJyMDgQEZEj/cEh5ZfaEhH1RekPDkRE1OMYHIiIyMHgQEREDgYHIiJyMDgQEZGDwYGIiBwMDkRE5GBwICIiB4MDERE5GByIiMiR/uDAx2cQEfW49AcHIiLqcQwORETkYHAgIiIHgwMRETkYHIiIyMHgQEREDgYHIiJyMDgQEZGDwYGIiBwMDkRE5GBwICIiR/qDA5+tRETU49IfHIiIqMcNLMdKjTH1ADoAZEWkqavziYioskp+5uDt+CEiC7zxusj8OgDt3vx2Y0xNqctARETdU45mpYkA2r3hdgDRnX8LgLleUMiKyKIylIGIiLqhHMEhExkfGhwRkQ4AjQDmApgQtwJjTIMxpsUY07Jy5coyFJGIiHIpR3DoAFCdNNNrVlogImMAdNhmqCARaRKRWhGp3WCDDcpQRCIiyqUcwWEh/LOHLIDmyPyaQFPShcgRSIiIqDJKHhxEZB6ArHeGkAl0TNsg0eQ1G9UBmMKrlYiI0sdIym8yqx0+XFqWLq10MYiIehVjTKuI1Ba7PG+CIyIiR/qDQ8rPbIiI+qL0BwciIupxDA5ERORgcCAiIgeDAxERORgciIjIweBAREQOBgciInIwOBARkYPBgYiIHAwORETkYHAgIiJH+oMDn61ERNTj0h8ciIioxzE4EBGRg8GBiIgcDA5ERORgcCAiIgeDAxERORgciIjIweBAREQOBgciInIwOBARkSP9wYGPzyAi6nHpDw5ERNTjGByIiMjB4EBERA4GByIicjA4EBGRg8GBiIgcDA5EROQYWI6VGmPqAXQAyIpIU8z8GgBZABCReeUoAxERFa/kZw5eYICILPDG62KynekFhWpjTLbUZSAiou4pR7PSRADt3nA7gJrgTGNMA4CFxpisiDSJSHt0BUREVFnlCA6ZyPjQyPgYb9pyY0yjMSaaH8aYBmNMizGmZdWqVWUoIhER5VKO4NABoDpPnjYR6QDQCqAhOtM7o6gVkdr1Bw0qfQmJiCincgSHhfDPHrIAmmPmWxloMCEiohQpeXDwOpqzXkd0JtAx3RyYn7Ed1XFXMxERUWUZSfkjsWuHDJGWDz6odDGIiHoVY0yriNQWuzxvgiMiIgeDAxERORgciIjIweBAREQOBgciInKkPzik/GoqIqK+KP3BgYiIehyDAxERORgciIjIweBAREQOBgciInIwOBARkYPBgYiIHAwORETkYHAgIiIHgwMRETnSHxz4+Awioh6X/uBAREQ9jsGBiIgcDA5ERORgcCAiIkeXg4MxZrAxZnA5CkNEROkwMF8GY8w2IvK6MeYiAJsBaPNmXVLWkhERUcXkDQ4AjPd6M4B2b7i6PMUhIqI0yBscROQ1Y8x4AKKj8hGAD8tdMCIiqpxCmpUGAxgDoAPARGNMG4Ah0EDx1/IWj4iIKqGQZqW5InKwN3xfOQtDRETpUMjVSq+VvRRERJQqhQSH040x3y57SZLw2UpERD2ukA7pDwGwb4GIqB/hHdJERORgcCAiIkdZgoMxpt4YU2eMaciTb3Y5tk9ERN1T8uBgjKkHABFZ4I3XJeSrA5At9faJiKj7ynHmMBH+YzbaAdREMxhjsoE8DmNMgzGmxRjTsmb16jIUkYiIcilHcMhExofG5MmKSGJwEJEmEakVkdqqQYNKWjgiIsqvHMGhAzkezGeMqbNNTkRElE6FPD6jqxbCP3vIAmiOzF/u9TdkAGSNMTUisqgM5SAioiKV/MxBROZBd/p1ADKBjulmb/4ib1o13CYoIiJKASMpfzxF7aabSsuKFZUuBhFRr2KMaRWR2mKX501wRETkYHAgIiIHgwMRETkYHIiIyMHgQEREDgYHIiJyMDgQEZGDwYGIiBwMDkRE5GBwICIiR/qDQ8of70FE1BelPzgQEVGPY3AgIiIHgwMRETkYHIiIyMHgQEREDgYHIiJyMDgQEZGDwYGIiBwMDkRE5GBwICIiB4MDERE50h8c+GwlIqIel/7gQEREPY7BgYiIHAwORETkYHAgIiIHgwMRETkYHIiIyMHgQEREDgYHIiJyDCzHSo0x9QA6AGRFpCkyLwMg66WJInJ6OcpARETFK/mZgxcYICILvPG6SJYpAGpFZJ43v6HUZSAiou4pR7PSRADt3nA7gJrgTBFpCpxNZAN54/HxGUREPa4cwSETGR8al8kYkwWw3J5hROY1GGNajDEta9euLUMRiYgol3IEhw4A1QXkqxeR6XEzvLOLWhGpHTiwLN0iRESUQzmCw0L4Zw9ZAM3RDMaYehG52Buuic4nIqLKKnlw8Dqas15HdCbQMd3svdYBmG2MaTXGtKKwswwiIupBRlLe4Vu70UbS8umnlS4GEVGvYoxpFZHaYpfnTXBERORgcCAiIgeDAxERORgciIjIweBAREQOBgciInKkPzik/FJbIqK+KP3BgYiIehyDAxERORgciIjIweBAREQOBgciInIwOBARkYPBgYiIHAwORETkYHAgIiIHg0NPuuMO4NZbK10KIqK8Bla6AHn1pcdnfPOb+tqX3hMR9Uk8cyAiIgeDAxEROfpPcHjvPeDjj7u/npYWYO3a7q/HuuMOwBhg2bLSrZOIqJv6T3D4wheAXXctPP8++wCnnhqe9vzzwMSJwJlnlq5cl1yir8OGAb/+denWS0TUDf0nOADAq6/mnv/ww8BJJ+nwgw8Cl10Wnv/JJ/ra3Fy6MhnjD194YeHLrV0LrF5dunIQEQX0zeAgojvOq67q2g50r710meXL4+evW6ev5WoCKqS5aulSYM0a4GtfA9Zf350/Z46e4fQVe+8N1NaWfr1XXAE89VTp10vUR/TN4HDccbrjPOkk4PLLu758e3v89M8+09cPPwxPf+aZ4vohFi8GHnrIH7fBJ8nq1cDmmwMzZgCPPx6f57jjgJ12Ck9bsAB4442ul6/U7rkHWLmya8s89BDQ2uqP338/cPfd3S/Lj38MjB/f/fUUo7U1/1lsuV15JdDWVtkyUKr13uDw2GPaJPPii+6866/3hz/6KH753/wmuXmooyN+ug0Oxmj/xRFH6I983DjgRz8CFi3q2g/u2GPD4/mCg133Pfe48w49NNxEFXTggcDOOxderu6aP18/n85O/56OJ58EDjkEOPlkHW9rA15/HXjgAeCRRwpf9/77A4cfXnzZXn0194UJRxyRXI+lUlsLbLtt7jzLl5fvfpgVK/Rz2H//8qyf+gYRSXWaAIjceqs4/uM/RACRa6915+nPStO554aniYjccosOb755/HLz5oXzi4jMmeNP22wzf/jRR8PbCy4jIjJ+vMiee7plEBHZbbfwcuut576XoDvu0Hz77usv09npvufgNtatiy9XOdntbbyxyMSJIk89JdLcrNP22Sd3eZPWlTSe5IYbRC64QIeXLxd5+21/+b32Sl5Pd+vqiSdErrwyeX5nZ/5ttLfr/MsvT17HqlXJy19+uchZZyXPf/99vww//7l+R4Luv1/kz39OXt667DKR73wnfz6qCAAt0o19b8V3/vnShOgP6YEHdLy6Wl//9Ke4WvHTrFki77wT/kF+//s6fPDB8ctdd537A7bbi6b/+7/cOzo77bXX3Pm1tcnL/u1vIkuWhNf1xz9qnm98w8//5JPuew6uZ8WKwnZ4M2Yk53n+eZ3X3Kzj776r4w88oONvvSWybJnIY4+JrF0bX09f/Wq4HF0NDv/+d3g8n2C+zTfX4TVr3O0uWxa/3DPP6E60q5LK98wzulMP7piT3Huvzt9/f3/a22+LNDbq8IUX6vwPPshdhg8/jJ//9tvhOrjhBpEjjxT59NP493DCCSJTpmj9xW1n+XKR//xP/a71FPsebr65uOUffVS/kytXlrZcKdJ/gsPy5SKnnSZy7LHhL7Y9SnvrLX096KDw/IMPDo/PmCGy9946vPPOIgsWBGvTTXfdFX92YNPtt7vTtt9eZMcdRVavjl/mww/1aK2mJn4n+emnOjxkiJ4hffaZTm9s1Onf+Y6ff8MNw2cH0R+23ZHbaWefLTJ/ftw3SZPdQaxbp+UPbveAA0Q++kjk73/3x6P1dt55yXVl0wUXJJc3qVyAyOuv588fXe7xx/OXJ2l7Y8cmr/+FF0Q22kjkpZfil7dndCIiDz+s0664QoNEcLvPPqsHDkFxweErX9Fpb70lsu22Ovzii7nf+2OPxc9vawu/zy220Nd588LLP/ts+Eznd7+L387JJ+vr1Vcn11d3TZgg0tDgj99/v27za18rbn1f/rIu/8QTpShdKvWf4HDkkfq66abuj9vurOLS4MH5dw52Jxg371vf0p1A3Lxx40T+8Ifk9S5enH/bcTuqjz5yp7e3i/zmNzo8dWr+9XR2ivzznyKvvOJPO+ccf/gvfxGZNk2PwI4/3p/+8stahmOO8ctz7bX+/O99T+SRR/zxurrwdvffv+vv2W5n3TqRpUv9b7fdqdoU3K71yScic+fG/TK6tu2k5ZKccILOv/TS+OXXrNHv1dlnazOPnf7Tn4bXHbede+7RaTb4Bg8yXn5ZDz4Akaefdst1/fXh8r/+upvnrLPCeYYN09docIj+tk44Qev7/vtFBgzwpx91lL7OmaPv2zZTdXZqk2IwULa1ueX5+GM9m4+eKVvBAAXo+u13Yfz4+GXy2WknXd6eea9bpy0M1qpV2nxcSPNaXHmTLFki8txzXV9nEbobHHpPh/TTT+vrihXuvH/+M3m5Qi5lffvt5Hm33QZ8+mn8vIEDkzuvgeKuYHrjDeDii93pra1+h/jAAp6X+LvfAXvsAcyd60/7xS/84e9+F7juOmCLLYBrr/Wnv/oqcPbZwB//qOPDhgEvv+zPv+uucJ0uWBDebtznU6gLLgCGDweeew44/3y9XDco2GF/7LHacfy972kn8mOP6a5j223L16G8apWWa+VK/+KAkSPj8zY3A9tvD/zqV8BPfuJPtzc9AsALL/jDq1cDr7yi27BXdH3yCVBfDwwa5OdbvNi/COPqq/V17VpNJ54IfP/74XK89FJ4/OOPtZ6D3n9fX0Xc93H//eHxPfcE9ttPLzaILv/JJ0BVFXDkkTr+4IPALrvoZcOAXmE3Zgxwxhn6GT3+uL6XTTbRi0h+/nPN9/DD4XK/+264DJ995v8mg9+3t94Cmpr899HZmfx9tHnWrtWnFOy5JzBqFPDaazr9zTf1qsSZM+OXv+8+rf933vGnLV6sn+OkSfGXmQPAdtvp1YQ/+lHyhRGlfAJDd3QnsvRE+vzMYcIE96huww3l86PZYo5WbbJt5/nyDR8ucthh/vi4cdqhl5Q/2AxSqRRthsuXbPNFMNk2e0Bk1Cj/yDYujR1bXDk7O0V23z13nilTkuf97/+KdHQUt20RkdbW8IUIwXnvvivy7W+LnH66Tps0ya+n667To/W2tuR+qUKS/Q4PHy5y443J+bbbzi3fiBH+0X80/fnP+nruuSItLX6Taly64Qb3dxDMn/Q7GzfOnTZ7tvYHAnrhRW2tyJe+lLsOZswInyWcfLKeVXzxi+F8S5b4v8PNNtMz7R13FNl6a51266161mXzTJ2qZzC3365JRGSHHXTeQw+F1/3Xv4rcdps/vsMOmn/xYpFf/1rPMKJnMiJ+U3BDQ3i6iPbDHXmkyIknhpe77Tad/+9/6/sUEfnXv3Te5Mki//hH+FQgeuFAHkhjsxKAegB1ABqKmR9MnweHYIemTZdc4ldksT/K4Jc5X57Ro0UOPTT8xfnJT5LzP/dc98vV3dTdwAn4p+A2XXxxct5ttiluG8EriIpJ8+eHO/27kkSS573xhj+85Zbu/F131dettqrM5ztxYunWddVV2pzS1eWCBw/dSVOmuL8n278STLZ51Sa7H7Bp9Ojc2/nsMz/PgQeG5/3yl27+YFC76CLte4l+f2w/UTA1NmrzbVI5Lr1UO8RtU/kRR7h5lizR5uEzztCr/158US+COeaYvBdMpC44eDv+em+4AUBdV+YnBoe4ZDuDDz+8NF/OfOlLX9IjZzs+dqzID36QnH/kyJ4pFxC+vDaYiu0DCKboJbe5kj166+n0la+ErzLrSoq74symfGczNhXSt8WU/hR35pwv1deLHHdccduzfanFpA031N/3nXdqx/xuu2nQOOQQkRtvlDQGh9kAarzhOgAzuzI/EDRaALTkDA5PPqmvcTuvuCtiupt23VVkk01Kv95SpGAHYalT9MwhV7JXvvS31JWjZ3ulDFPpU9JBUlfT+PFuc1YvS90NDuXokM5Exod2cT5EpElEakUk90N1NttMX6OPkrj+eu30sk48MedqEp11lnbcWhtvrB3UaRHssLUPDCyH557Ln+eHP9RX2zmZFltv3TPbee+9wvNut133t7fnnt1fR3cFL3boCUmdvID+y+J55wG//W3udUycmDzv61/3h1tbgW99yx+fMCH85IU4p5wSP32ffXIvB2hHei7RizPOPlvvcp87V397Cxbo0yDmzNGnPzz8cP5t5lOmM4e6wJnB7K7ML6hZab/99N6DJUvio+Y114iI+OM/+5mbp5Dr8UX8u6kBkVNP1fbKaL5Bg3Kvx968li9tsEHXjg7OPddv1vr97/O3tZYziYQ767ub8nVeFpqmTy/Nes48s3Tv7aST/OFg0+MppxS2vL3/opSfX7Q/5fLLRYYOzb3M00/7w7vsEp8neNZ59NHhNvjPPvNvrAsuc9hhmje6rn/8wx++8069Z2Tw4HDH7Z136vxRo7Qj+MQT/acbDBvm/56jZxizZmnH9tNPiyxcqOuaP9+f39ysHdE77+xPO/xwfX8HHKCfnYjeN2Hz7LuvyKuv6vRop/2MGXofiR3/4IPwzbo33aQXQQDaF7R0qV+ft9+e+w55sVWavmal8vc5HH+8vvu4+wEA7VwOfuFuuik8f/r08LXnSUnEf+wDoNebR284++53/bu2k9Ldd4c7spNSV3euc+b4weG669wb/vKlH/7QH66v79qycXVlv8zRH3RXdtAbbaQ/4rvvLq4cwf6n+fPD9xV0JyXd0FhMCnaoBm+EXLmysP6zXXbR+o5epDF8uH/PQTTZJtikFO34De5oAZEHH/SHBw7UR2fYO84331zkzTfddVZV6c2rV12l483NOh7chr8n89OFF4an2ZvsXnpJX7feOnmPaK8+2nZbf5q9w3/33bWOjzlGr15qa9MbUpPuJLdPF/jZz/xpq1bpsnH3mFhLl+pvy95QKqJXvD31lN5U+8IL/r1VI0aE6wLQ+1hE9AqmN97w5y1bFl5nHqkLDlomzESkPwFAc675SSk2OMyYoe8+7nEIgH9LvR3v7AzfzPPSS/4XNlcS0aOI6Jc5mOfWW/1b+YM3kwXTPffEX1JqL8W1Ke7o9PHH3Xw27+rV/qNArrtOb3izN0gFbxZMuqIqeKS6dm3xVwzdcovWS/TMZaeddHrcUWBSGj06vp6jafJk/5EfwRS8QEBEjwiLeU9x34XotLvu0h9wV9f14Yf+TuGQQ8Lb+Phjvax2/fXd5aqq9DV441dwp3/ffboDsuN2G4B+V/bZxx+Pnu1edFH4OyPi3pFuh4OP0XjlFX/nai8IuOwyLUvwzm97Zc3KleF1WmvW6GXhv/qVHvSJ+O8neLf+1VfrBQRJ7KWgwTpat05bCuwjWLoi1w1tpfDOO/p5W6++mvxYlC5KZXAoZYoNDvYUTsSdt8ce7sPoonlF9EtWyA4h7pENwTz33afTPvlEtxu3I77vPv8a+WDKZsPj996r1z4bE95m8HEZNtn3GAwO1i9/GT7SW7HCP/oC/CB02mnh7Xz968l1kXRJ7FFH+duNXudvn10Vd0d3sGklWBfZbPJnG0ynnx6fJ/g+RfwzxDFj/On22v3omVquZhQRP/DaZI/ibEAaNiz+4ojoIzw6O/WgAghfhh0UvCt/771F3nvPP0OdPNnPF2yaeP99/yh5u+20qTG6brsDnzYtXKa77vIvY62q0rydnXr1zpw58b+nqM5OkUWLkufbPPnW0x22aea3vy3P+nuR7gaH9N8hvemm7rTgHaNRu+1W2B2y+fLYzqgtt8ydz3aKb7SRrvOb33TzDBwITJ0K1NX50847Tx+9HezIHDBAl7fbtI8UF+la+c8+O9yBtckm4f+1sHU6YIA+RttqanL/CyK4vYaG8LT6+vC/1224YXwZ4/5T49BDw+OzZ+trrs826KtfjZ++6ab6P9/Rugt2CtpO6g028KeNHBnfqWwfKw6EL3yYNct/v3vsoa/77ad3vkbtsos/vHq11suOO+r45Mn6Gq279dbzh0X0zvG999a7qOfM8edtvz0wfbr+wdPQof7d82vXAtOmAX/6E3DNNX7+447Tu7D32iu8vSFDtO5nzQL+9S+dZu9itnddT53qljPIPso+l3I/Dn3kSL1Te8aM8m6nH+idwSHXVQvBH9Xtt+sf8cQZkOet33STu7440fLZ/MEyDhyoP5rg/0ecfz4wdqz+6LfYIrys3UFWVeXedi7RH2Fw3D76wBjgb3/z//NixAh/Rxi3vmuuCV9VMXcuMHq0u15Ar6ax/4k9cqT+P8GVV/rzozsZu8NOelxBUFOTf2XJO+/4/xEBaHCbMMEPxJmMvtqgPWmS/9nbnfaoUboeO33sWP3e3HablmvvvXX64MH+ds45xx+2n3VVFVBdrcMXXeTPDwY8+5luv70+nuEHP9DHPrz5Zvg92rIcdhjwP//jTx87Vnfk1nrrAf/938AOO+j48OH6Wl+vr0cfDfzXf4XXPWiQrhfQz/Tvf/eD7TnnJP8J0l/+kvwomTSxB2rULQU8pKfCRozQS8R++lN/Wq6jy+DO3B6VxUn68my8sf+MGOvRR8M7hqCNNorffvBoP9+zkOwy0eBgl4s7cyjE5Mn63KGgqVP9nfiAAfo+g+/V7tysgw4C7r1X8w4YoM/N+etfgTPPdLe3apU/HL2UbsgQ3aHbI+tocBgypPD3efzx/vDIkXpmctBB+refdqdoHXOM7ux23VV3wMOG+ZfdbrmlBumDD/bzP/GETh82DPjylwsrj30WTlWVruvee4EDDtAzkcsuS/6u2e+UPTgI2mADDR5/+IOeERQqk9G/sbVntEmGDdNy5zv4KYdp0/yzLUqt9J85GOP/mK0vfMEfju54c50R7LST/xC0pB/Pk0/qA+mC69ljj/jmllGjwmUB9EcHhE/b8/0Ak4KD3Wnbnebvf597PVG33w4cdZQ/vnYtcOON/vri6soYP0Bss40fXOwObpNN9J/e7NF0UDA4xAnuBIOf26BB8TtI6/nn3SAcLfOkSRqwot+H9dbzmzq23FJ3uva9iGjz3u67+/nHj/c/wzgXX+w+bHDNGn2tqtJ1H3ig1u2llxYf2Jub9YAoGqwLUV1d2E6/EoEB0O/xtGmV2TYVLP1nDkD4KLOxMfzkyfZ2fZLi/Pl6Kp/rC//ss/7wEUcAS5fqzWOZDHDzzdo+O3aspkLEPc117Fhtmpk40d+hBXdYjzziPnXRltnurG1wiO5YbBNJnEJOo+127NNNkwLpsmV6lD14MHDLLYWvP5vV5pglS+LnV1XpU0qHDvXPXiZM0D6COIcfrmdxO+ygT+w87bT8ZSjEzJn6xNDgTU+FiitDMDiUys47+81yRBXQO4KD3TF94xtup+jo0ZrWX1+Dw6RJha1zwAC9c3ryZG1Kqq7WpolCnHGG37Ybxx5Vf/GLGriCwSGuIzXpzME+Gtu+f2M08AQfYXzKKcDddxf+vgFtGpo1K3z3d5TtSwg2QeVzzz3ambn55sl5zjpLX994Q19z1fmdd/rDp56q7eQbb5y/HPlsv70+mrxU7JlHMcGGKKV6R3AAgA8+yL1j2G234k7hgx2qhQpeoZOL3akX2+dgm2muuEKbcyZNCl9hA+gRZlce3QDozrHQurL5CjlzGDkydz9P0FZbaeAstP6N8Ttd06a2Vv9jIPrZEPVi6e9zsDKZ0p629wS7Q8131B1tVrJHoFttpa+jR+uf71Ri52MDVa42/2Jts03l2r1LLd9nM2eO21dBlGK958yhNwp2fOZid5C2CefHP9amnxEjyla0gh19tDbBxF2dRIWL/kMbUcoxOJRTscHBmHQEBkDPHKJ/K0lEfV7vaVbqjWwzUb7gYPMF/yOZiKiCGBzKyV72OGpU7ny2w5rBgYhSgsGhnKZN07OGpLurLdusFL3/gYioQhgc0uD88/Vql3HjKl0SIiIAgJFib+/vIcaYFQBeqnQ5UmIYgJT9D2fFsC58rAsf68K3nYjEPLm0ML3haqWXJN9/SfcTxpgW1oViXfhYFz7Whc8Yk/BcmsKwWYmIiBwMDkRE5OgNwaEpf5Z+g3XhY134WBc+1oWvW3WR+g5pIiLqeb3hzIGIiHoYgwMRETlSeymrMaYeQAeArIj0i3ZEY0wGQNZLE0XkdG+6Uxf9qX6MMbP7e10YY2qg3wuIyDxvWn+ti4Led1+uC++9TReRAyPTOlCieknlmYNXeIjIAm+8rrIl6jFTANQGfvwNcXXRn+rHe29Zb7g/18WZ3vei2hiT7a914b2ndu89thtjavpjXdh9hFVoHXSlXlIZHABMBNDuDbcDqKlgWXqMiDQFInkW+t7j6qJf1I8xxtaB1S/rwhjTAGChMSbrfUf68/eiBcBceyYlIovQf+siqNA6KLhe0hocMpHxoZUoRKV4O8XlXnTPRGYPTZjWF2W9HaGViczvL3UxBvq+lhtjGr3mx0wkT7+oCxHpANAIYC6ACd7kTCRbv6iLiExkPKkO4qbFSmtw6ABQXelCVFC9iEz3hjvg1kXctD7FGFNnT30DOtAP68LT5u0YWwE0oJ/WhdcMskBExgDoCLSf97u6iOhAYXUQNy1WWjukF8KPcFkAzZUrSs8yxtSLyMXecA3i6yITM62vWe7tCDIAsv28LhbC/0FnoD/wdvTPuqixvw8AF0L76frr9yKo0DqImxYrlWcOXmdL1u4cYo4g+yTv/c42xrQaY1oBVMfVRX+oHxFZ5L2vanhf5n5cF/MAZGznodfv0C/rAkCTd6FGHYAp/bUuvPdVG+hgLqgOulIvvEOaiIgcqTxzICKiymJwICIiB4MDERE5GByIiMjB4EBERA4GB6IiGWNmVroMROXCS1mJiMjBMweiLvKeitrg3bVN1CcxOBB1XcZ77U/P7qF+hs1KREUwxswVkSMqXQ6icuGZA1EXeY/MXs5mJerLGByIitNR6QIQlROblYiIyMEzByIicjA4EBGRg8GBiIgcDA5ERORgcCAiIgeDAxERORgciIjI8f8QyxKu90rq2AAAAABJRU5ErkJggg==\n", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAESCAYAAAAWtRmOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAli0lEQVR4nO3de5hd470H8O8bSRqXsJMcqorIpkLPqcpkcqrqqDLoUcd1Qjk9Lac6UTyqWvEgCCka1ToHoZPUpS4tEkI4RROXUEJnJo2oKJoREuKSyzZEEpnM7/zxW2/X5V37vvfsNbO/n+dZz173/e53r/3+1vuud61tRARERERBA2qdACIiSh4GByIicjA4EBGRg8GBiIgcDA5ERORgcCAiIgeDA9WcMSZV6zRYxpgmY8zSAtZLVTENDcaYFmNMaxX2nar0Pql/YnCgijHGpIwxHSVsekLFE1MiEZkHoLOAVZ00FxJUCnQBgHtFZEKF9hdUzXRTP8LgQJXUBCBljEkXuV01CsFqi0vz2ArtOyUimQrtK6qa6aZ+hMGBKq0VkQLIayZZ6o2njTEdxpgmb9oGlGZjTENgm2aviac5sG6zXc+um2O9pd78mV6NxpnnrdvizZsY1+TiLWsyxkwNbOOk2XvtCGwXl64mY8xaL/1Ncc1G3n6Ge+lK58u7XPuL5lc10039kIhw4FD2ACANoAFACsDSmOVzA+MTATQFpjsi6zYAmBiYbgVwEIBWbzrlzYtbL2Xfz1ueDqYhOM8bnxrcZ0xa7bwmu25cmoPbFZIub3xqMB/i8qqAvIvdn5eG1pjPVrV0c+hfA2sOVCkNIrJQtDkkE6wFlGACgIWB6aUAfgwtoCAiGdH2+Lj1mrzx4V56gtcPovNOBLDaS2vaG0Ik3O6fKiP9TYFp+/6rC9xfPnH7OxFufuXT2+mmBGNwoEpJ22YMAO0o4TpCnmsVQwrczXDvdU3Msui8FICFXsBYKCKHxqRpqte8Ere/fGmOSxekwtcTStlfEtJNycbgQGXzzryni8gsEZkF4Hy4vWIygTb9cQifhduC1xZYM6FNHAisPw3APwpv7z3j1ptXRNJnxuwTgekWAKtFezBF14mmObrfctIVlSvvsrkH8Z+tN9NNfRiDA5XFO6uegXDzQ9pb1ho4Q20FcELgLHxCoMBr9QpiAP/oTpqxF0ah7eVzAHQELpRmsqzX6S1vDO4zbp63vb1IbS++NkBrQS3QgnF3u8xjz6RbI/v/x3Y50mXXmejly6EAxgcvhEfW+Uf+xOVdrv2JyMJoflUz3dT/GBH+nwMREYWx5kBERI6qBAevKjs3z/KmYPWWiIiSoyrBwbsoGctry7TtvbYtmIiIEqQWzUrj4PeX7kS4dwQRESXAwBq8ZyoyPSK6gtfc1AIAqUGDxu6+zz69kCwiov6jo6NjlYhsX+r2tQgOGQRurIkjItMBTAeAxh13lPb29l5IFhFR/2GMebOc7WvRrNQGv/aQhneLPxERJUe1eivZG46aA/Psc15mQW+oaYI+1It3YBIRJUzib4Jr3HFHaX/33Vong4ioTzHGdIhIY6nb8yY4IiJyMDgQEZGDwYGIiBwMDkRE5GBwICIiB4MDERE5kh8cEt7VloioP0p+cCAiol7H4EBERA4GByIicjA4EBGRg8GBiIgcDA5ERORgcCAiIgeDAxERORgciIjIweBAREQOBgciInIkPzjw2UpERL0u+cGBiIh6HYMDERE5GByIiMjB4EBERA4GByIicjA4EBGRg8GBiIgcDA5ERORgcCAiIgeDAxEROZIfHPj4DCKiXpf84EBERL1uYDV2aoxpBpABkBaR6cUuJyKi2qp4zcEr+CEi87zppsjyJgCd3vJOY0xDpdNARETlqUaz0jgAnd54J4Bo4d8OYKYXFNIisrAKaSAiojJUIzikItMjghMikgHQCmAmgLFxOzDGtBhj2o0x7Rs2bKhCEomIKJdqBIcMgOHZFnrNSvNEZHcAGdsMFSQi00WkUUQahwwZUoUkEhFRLtUIDm3waw9pAHMjyxsCTUlXIUcgISKi2qh4cBCRWQDSXg0hFbgwbYPEdK/ZqAnACeytRESUPEYSfpNZ4/bbS/sHH9Q6GUREfYoxpkNEGkvdnjfBERGRg8GBiIgcyQ8OCW/2IiLqj5IfHIiIqNcxOBARkYPBgYiIHAwORETkYHAgIiIHgwMRETkYHIiIyMHgQEREDgYHIiJyMDgQEZEj+cGBj88gIup1yQ8ORETU6xgciIjIweBAREQOBgciInIwOBARkYPBgYiIHAwORETkYHAgIiIHgwMRETkYHIiIyMHgQEREjuQHBz5biYio1yU/OBARUa9jcCAiIgeDAxERORgciIjIweBAREQOBgciInIMrMZOjTHNADIA0iIyPWZ5A4A0AIjIrGqkgYiISlfxmoMXGCAi87zpppjVLvCCwnBjTLrSaSAiovJUo1lpHIBOb7wTQENwoTGmBUCbMSYtItNFpDO6AyIiqq1qBIdUZHpEZHp3b94aY0yrMSa6PowxLcaYdmNM+8aNG6uQRCIiyqUawSEDYHiedZaKSAZAB4CW6EKvRtEoIo2fGTy48ikkIqKcqhEc2uDXHtIA5sYst1LQYEJERAlS8eDgXWhOexeiU4EL03MDy1P2QnVcbyYiIqotIwl/6mnjsGHSvnZtrZNBRNSnGGM6RKSx1O15ExwRETkYHIiIyMHgQEREDgYHIiJyMDgQEZEj+cEh4b2piIj6o+QHByIi6nUMDkRE5GBwICIiB4MDERE5GByIiMjB4EBERA4GByIicjA4EBGRg8GBiIgcDA5ERORgcCAiIkfygwOfrURE1OuSHxyIiKjXMTgQEZGDwYGIiBxFBwdjzLbGmG2rkRgiIkqGgflWMMbsJiLLjDE/B7AdgKXeomuqmjIiIqqZvMEBgPFe7wHQ6Y0Pr05yiIgoCfIGBxF5wxizLwDRSekC8GG1E0ZERLVTSLPStgB2B5ABMM4YsxTAMGiguL+6ySMioloopFlppogc7o0/Xs3EEBFRMhTSW+mNqqeCiIgSpZDgcL4x5riqpyQbPj6DiKjXFXJB+kMAvLZARFRHeIc0ERE5GByIiMhRleBgjGk2xjQZY1ryrDe1Gu9PRETlqXhwMMY0A4CIzPOmm7Ks1wQgXen3JyKi8lWj5jAO/mM2OgE0RFcwxqQD6ziMMS3GmHZjTPunmzZVIYlERJRLNYJDKjI9ImadtIhkDQ4iMl1EGkWkcfCgQRVNHBER5VeN4JBBjgfzGWOabJMTERElUyGPzyhWG/zaQxrA3MjyNd71hhSAtDGmQUQWViEdRERUoorXHERkFrTQbwKQClyYnustX+jNGw63CYqIiBLASMIfT9G47bbS3tVV62QQEfUpxpgOEWksdfvk3wSX8OBFRNQfJT84EBFRr2NwICIiB4MDERE5GByIiMjB4EBERA4GByIicjA4EBGRg8GBiIgcDA5ERORgcCAiIkfygwMfn0FE1OuSHxyIiKjXMTgQEZGDwYGIiBwMDkRE5GBwICIiB4MDERE5GByIiMjB4EBERA4GByIicjA4EBGRg8GBiIgcyQ8OfLYSEVGvS35wICKiXsfgQEREDgYHIiJyMDgQEZGDwYGIiBwMDkRE5GBwICIix8Bq7NQY0wwgAyAtItMjy1IA0t4wTkTOr0YaiIiodBWvOXiBASIyz5tuiqxyAoBGEZnlLW+pdBqIiKg81WhWGgeg0xvvBNAQXCgi0wO1iXRgXSIiSohqBIdUZHpE3ErGmDSANbaGEVnWYoxpN8a0d2/aVIUkEhFRLtUIDhkAwwtYr1lEJsQt8GoXjSLSOHBgVS6LEBFRDtUIDm3waw9pAHOjKxhjmkXkam+8IbqciIhqq+LBwbvQnPYuRKcCF6bneq9NAKYaYzqMMR0orJZBRES9yEjCH4nduNVW0v7JJ7VOBhFRn2KM6RCRxlK3501wRETkYHAgIiIHgwMRETkYHIiIyMHgQEREDgYHIiJyMDgQEZEj+cEh4fdhEBH1R8kPDkRE1OsYHIiIyMHgQEREDgYHIiJyMDgQEZGDwYGIiBwMDkly003AlCm1TgURUR/4P4ctt5T29etrnYzeYYy+Jvw7yenjj4GBA4EhQ2qdEqK6xv9zoGQZOhT4whdqnQoiKhODA1XeihW1TgERlSn5waGcJpZHHgHOPVfHn34aWLq0MmkiIurnkn/NYcgQad+wobSNbRv+hg3aBr7TTsDbb1cucQDw2mvAoEHAqFHl76vS1xx6eoCnngK+8Q1/39XWH66bEPUD9XXN4ZBDgNtuK367l17S1w8+qGhyAACjRwPpdGnbbtgAVPNi+69/rXl2//3Vew8i6pf6RnDo6QGefBJ44gng1FOL3/6TT/R1m20qm65yjRwJbLVV9fb/2mv6+tZbpe/j6aeBCy+sTHrKIQJs2lTrVBDVjb4RHKZNAw4+uPTtk1qovP9+rVOQ39e/Dlx1VeX219UFPP548dvdeCMweDDw7ruVSwsRZdU3goM9Ay5VqdcsLBEtnLq6cq+TNJVMU08P0NYGdHaWt5//+A+gqQn46U8LT9/Pfw6cdZaOv/lmee9fjL/9Dfi//8u9zoYNwPnn6/0dRP1I8oPDxo3AnXeWtw/brl9oYfTXv4YLoSeeAM48E/jRj/Rsf/Zsd5uPP9b9/+IXwKJF8fttbwdeeKGopFfE/PnAddcBy5cDDQ2lnX1v3Aj8678Cu+9eXlqeflpff/lLP9C8+Wbupq8LLvDHt9iivPcvxt57A0cemXudm24Crr5aP08lvfeeXty/++7y97VuHXDffeXvp1rWrWNPwgRKfnAAgEwm/zpLlwKPPaYF8DvvhJcVe9H3S18CdtvNn7Y1hjVrtLA47ji3FrFpk/bvnzgRGDMmfr/jxgH77Zf//Xt69AfT01NcuqNsz6EHH9TAdv31wF/+Atx+e3i95cv9z3PZZcCWW7r72rgxPH3bbcA++xSWjnvu0c8TtWqVvu62m15/KUQxweHFF4GPPip8/VKsXq2vhaSrkO+zpweYNEmvsQFAa2v+bdauzR1EfvhDoLkZWLw4/75qYepU4KtfrXUqKKJvBIeoZ57R1w8+0LN8ANhjD+Cb39QCeK+9gD//2V+/3B5Bmzfr6xZb+Gc4d98dDiCbNpXefBUtNNat04vn551X+D5aWjQ9xx0HTJigtZhoTWntWn0dPDg8f9dd/R/n5Mn6OexFfOvDD8PTp56qvcBEgOnTgREj4mtmzz0HfPvb/v0mQatWAQsWFPoJVU8PsGSJNk/lyu+eHmDfff0zfxFtJspHBHj++cLTY4PP0KG513v1VT1+4mqdQfPnA1dcAZx2mk7nqlG99ZYG9v/6L+Ckk4DXX49fzwaFal17W7tW01tqIH75ZT0Wktg0G+f116vbyzAh+mZwaGvT19Gj9Sw/6qOP9EdmFdusFNXdra9vvOHPmzAh3PT00UfAAQe42z7xhN5rkMuCBeEC6bHH9PVXvwJeeaWwNM6YoemZPVsL6wEDtCkpyBY0gwcDV16pTSJz5ui8JUvC90IsXx6+YJ7t/pD77wdOP11rVddf7y63+3joIbdweu89YP/9/elgABLR9EVrgXPmaLB5+OHcgWXZMn21zVjTpmkzUfCkIW6b007TQHniidnXC7J5nK8nXHu7vs6c6c974QX/xMOy07am1dmZvdAdOVIDuz0OsxVY9npItPZXiFWr9HpPrkB85ZXAzTfrMRi0cmXu/LaWLdPv2/7Okqy7G9hzT+Dkk2udkqrrm8HBFmL2TDjOH/7gj9uz4PXrtbDp7NR9FHrWan+wixZpIRjn1lvDhWkmo+9xyCF6E1ouBxwQrlaPHx9eFtXVpU9vffZZ4PLLNQAVIhgcLroIOOMM4Oij49ddsQL47Gf96WAhHQxYzc36oD1Am66ibKGycqXbJTZ6cXv0aD3DBrRQP+MM4PvfD69z+eX+fSuvvKJNJkuX+tsBGmSi10ZsHi1bprUJY4BLL9XrWc89p8tGjQJuuUXHg230cScVPT3hGl+2Lskiuk9bMNt9LVigTYzRnmBxNyvGNckVwwYH+/roo1qrevnl/NtOnqyBNXrd7513/GBkP9t11+kxaa9pjRkDfOUr+d/DnnTlC1533aUBsZTm1gULKnO9z+ahPYHrz0Qk0cNYv4EkPLz7rj/+/vvx62Qbfv1rfd11V5EXXxSZMUNkyhSRzZtFurv99TZvFhER+e1vi9s/ILJwYXhaJDwenM433HqrhJx+evHpAUT+6Z/09bDD8q/74IPh6QkTinsv69Zb/Xmf+1xh26ZSIl//uo6PGlX8e269dXj+Rx/54zNnZt82OD1woD9+wQXiAESOPdZf55JLdL49Zrq6RJqb/WNn2DB9/fa3dfkdd+j0SSeF9/v4427aXntN5OWX9dhcsULX27TJX/6lL+nriy+66Qzmx333uZ8zl/XrRbbaStf71a/cz3/ggTp+xhnhfY4YEX6f9euzv0cm46/3+9+Hl02aJHLzze7nePddne7pEdmwIfdnENHvpJDPmy19M2aIXHutyNy5Im+9pfsZOrT4ffUyAO0ipZe9JW/YW0PW4HDNNeGDsZiCywaH6HDmmeGCZN06zWX7AylmaGsLT8cdoMXs78kn9XXZMpHvfrf49BQ73H57edtbN91U3n5Gjix83Q8/jM/Xs87yx2fMiN923brw9KBB4WlbwHV3izzwQPw+rr1Wtzv7bP9z77FHeB0bHH7zG50+5ZTwLzouOIwbp6+HH66vmYzI+ef7y21wWLQovK+rrxb58Y/99X77Wzd/rrpK5KGHwoWwdc45/npTpui8lSs1oAW/52zfvx0/91yRjRs13cceK/KHP/jvsWhReLsf/Ujk7bfDJ2kiIm+84U93dOi8Cy/U6WiAeOUVDeg2UF97rXtcivjLc7n66nD6Xn7ZH589W4P13/4m8umnIr/4hQbybKZMEfnTn/K/Z4XUb3AIfuHFDtOmxc//7Gf1wLTTzz8vsuOOpb1H9KD68EN/fP58kZ/+tLj9/eAH+nrjjSKnnlr6Zy90+Na3ytt+7Vq3wC1l+Pzni1tfJPfybLWXaE3pM59x11m8WOSKKwpLx7nnxs8/8USRBQtEtt1Wp08/XY+Hp57StD/ySP59v/hi+ITIBoe2Ni18nnwyPh+mTctdyw76/vfDy2ztyRbIdvjTn+L3FSzcAT2Zu/FGf/rRR7UGe/zx7rb/8i9a4NrpTZvCNcjZszUt222n08uX++kOnpSdfLLmd3Dfdt2//12nb7tNp//6V13300+11n/ZZVrTuvzy8PZ33x3/ec8+W19PO80tpf/85/C6b7/trtPdLXLvvRqw1q3TdJQpkcEBQDOAJgAtpSwPDlmDw3XX5f8RZRt++cv4+aNGiXzhC6Xvt5rDpZf649EDPqlDKc1x0aHY4Py//6tBvtj3Ofro/Otss035nyf63f37v/vj552XvVYbHFpb4+c/95w/fued7vJJk3LvV0Tk9df1zD66bK+9RI44orjvITh94YUiX/1qYdtuvbUGSzu9dq3IFluE17n7bj9AtrVpeXDPPSLpdP79f/BB+Hd+xx0io0frePB9AZEf/lCbGG1+FnJyMGeOX0L39Ih85Svh5alUuBR/6im/hmzf57DD4kv87m6RG24Q+eSTvhccvIK/2RtvAdBUzPKCg0O2Ar6Q4cor4+d/8Yul77OSQ9yPOHjWlUoVtp9jjqn9Zyl32H773Mu/+c3C9rPffrX/LHFDtNlshx3yb1Po9x8dDjoo9/Lu7sKuRxUyRGsehRTawSFfzfWYY0o7CQC0mW7LLQtff889tbZXzHvce6+W0LNmZc8fEb2OEZxvayD2+1ixQk+QLrlEm61sbenCC/0o0NWltdpMRk/IJk8WmTxZkhgcpgJo8MabAEwsZnkgaLQDaM8aHC65JPeXM3589qaniy6Knz92rMjw4aUdcJUaDjrIhv3wcMMN/vjQoYXt6847RXbZpfw0NTXVNk9yDfZaTL7Bttdz0CF6LQQQWb26tJqz7TwQHIqpZfTG8PnPixx6qD8d7bSQa7j4Yr2WUOx7XnSRdkrItjxfcDv44PC1peiwzz4iX/tafNPrlltKucGhGl1ZU5HpEUUuh4hMF5FGEWkMdacMuvzy3Kk48EBgl13ilwX7mgd1d2tXtbFjc++7muxdw7nmR/vGZ7N+PfCZzxT+3r/5Tfz8YcP0LuukCN5zscMOhW2z3XaFrXfOOUUnx1FomkoxeXJp233rW/742LHxv4Gbby7tX/yGD3fnBbuSJ8EhhwB//KP/VOdCuwcfd5w+NWCEU0yFRW8aBfRmxlmzdNze8Q74fzvw3nu59/nEE3r3eDaLF2vXYft7+OIX9ebcZ56pzJMBqlRzaArUDKYWs9xpVho7trQzhRkzsvcqyTdU+qzn2WcLX3fMmHDN4YIL3HXiLpbGDR9/LPLP/6zj552Xf/3Nm+N78+yzT3xtplZDMC3BDgS5hmgzhx2Cbf5bblnY5zz5ZLcNPDjka74pZbjrLr83VqE1x+Dw2GMi06dr7ySR4s+Eg81NzzwT7s31ne9k3+6BB9ymwWCz6fjx2oMruDxbhxF7EbmUwXYBjr5XvsH2hNq8WeSQQ+Kvoz34oK6Tq2ediP62jjpK93XAASLXX68X2//4R70Ane1aEqDXqh5+WNfv7PSbn4YN0/xauVKvbwQggc1Klb3mUGpwuOMOzUw7XUyb5ymn5F9n/Xp3XrDaGj0wCn3vN94Irz9vXmmf3xYC9jpKtusswc/sH1U62Ca2GTOSGxzWrClsm+99T5sN7fT222uvn+BFXNtzJV9nh4svDnettMOzz2pf/f33z75trmabAw4ITwcvjr7+uv/9/P3v2qOnmDyLdqFcscJfdswx+btrf/JJOP+nTPGnc90D09Mj8t//7U8PGOC3w//sZ7qvaLt8sNu3HUaN0n0FmxGXL/fH7YX+YcO0vR/Q+5gAv6lWRAO73ebll3M3mTY0SCzbSeL44/X3aa1dq/ucPz+8nyVL4vcTJxgYGxu1Z+Ldd8ev+/rrTkAISlxw0DRhIiLXEwDMzbU821BwcIj+sO69V7vLAdruLpK/gLTD5Mnh6WCQAbQ7pIi7XWenf+DcdVf4x5TtvQYM8Me32y74zfrbXnVVYekODlOn6rY2KE6bpn3rs7Vzvvee+9727HjZstyfIdfwk5+Utl22wZ6l2ementwXpS+5RGtPs2eLvPOOP//ii/3Pa2sVra06HexGGTdMmhQflLq6dPvoMWvvS7n0Ur3I2NgYv99DDxX58pd1/PjjtZC0tYSVK91ff67fRrBABrR7ZtDateFjLF+XYRHtqfPCCzoevAYWvJcibjvbbt7UpL+RzZs1iG7apMs3b9Zaze23ixx5pM57/nkt/Ox+1q4NH58jR+r4ww9rL6XTTgu/p4geG3fdFd72vff0t3H77TptTwQPPDCc7qYmv3tx1DvvhIN1VLDbejA9xVi8OGfBX4hEBodKDmPHjhUZPDj+gI9+mcHpOXP8ngCjRmluBQv5++/PXqhEz2Qeeyz+oI/OW77c7yEUPDOLWzd40C9ZouPDhgW/WX9bu7yY4eqrdVv7o7/zTp2+7LL49TMZ973tYG8GtNOtrbkLhODw/PPay+Kmm7Q5LFdzzOLF8fNt4fK978Xnz5tvutvYG8fa2vxtggWiLchF/LO1Rx7Jvr/gsGSJ248f8Pum77yzTj/1lAaRDRvChfucOfH7ffBBkVWrwjeJPfqoyL/9m1+QBu2zT/x+7rxTC8HgvGA+iGhaAT3jFgnf2BY3RG3aJLLvvrrsoov0Rrps2118sY5PmuTuJx8bSLu7/Xkff+x25bT3esSlNZdMRs/Mg82T779ffDqDenp0P0ccEU53L6uP4LBypd6kIqIRO+7M7qijwtPz5vl3m+67r25raxKAfwYU3c+gQdpkFLyr9okn4g/66Lzly/Ws4bHHwstFRIYM8aftYxdsG/eqVTq9xx7BbzZ8oOcrhBcvDufBNdfodvaRGbbvdbbrH8FHHBx+uNZWbFtyNA1r14r87nf50wTo3arWp5/mPit/5RV33ooVft//U0+Nz5/Vq8PbnHWWFl6PPy4hGza4+SqiP+YFC/zpbDeKDRmid/Bm+07smZ6trQUDUFDcycY778Svm0u2rtf2jHfRIm2CuvDC+LuB33zTb1Pv6hKZOFFrUT/7mX73t9wSn1/WxIm67IorwvmRyehNgPaYs8dK9DEwhVizRk8w8gk2D5bCnu0fc0xp20e9/Xbux4b0gvoIDlFxd94GLzRdeaWeWdgbSsaP1+1sRAdEXnrJ5mB4OPts/33sXazPPKOvwQuBInq2FazaR88SguvaG6huucV/Psv22/vrXnutVrmtF17wg0xcOuOCVfBaiX0Wjn3f+fN1+qWX4rePKzy6u/1aQzAN69aFm81yDdHmkLgAYIe4wCHidyw49FB/P+ecI/Kf/6njmzeLfOMb/jbBfAyy33/0JqSorq7seRy0667hGpT12mt6zSubuO63a9bkTlMc+wiO4JBK6SNgKsU2t8SxJxr2sR0LF+pxEWWvFZTZTJJXY6M2x5Vq4cKaF+iVVJ/BIVjI2yGuzdFeNAxWqaMFSHQ/Z57pr2uDwQsv6NlpV5eexc6c6a9jm3x+8IO4b8dPj71xaf58fc7MLrv4D0IrhN1X3LN37HsE8+Daa8PbLV2q08uWFVbw5UrDpk3xd+DGDcHgIqLf3U03aeCL3v386qvx6bIXHoOBO86UKdo3PJc77tCz6VyCD7Wzw5Ah+fOlUPaRE8F7DaL5VIxgOitdAG/eXNgziChxyg0OffuR3QBwyin6GvfI5N12059MY6O7LNsjloOPAxbR10GDtC/30KH6Pw7Nzf46e++tfZij/50AAHPnAo8/ruP2sdaDBukjs996S/tQF6uhwR+3/xFgjR7tjw+IfLW77qqvwT+liW5fqOC/np10Uvb1dtrJzWdj9P8fDjvM7wNupVLx+9l5Z/2/iVx9vgH9BzWb39l85zv5/+rUflfW/PnhR4KXy/axHzlS/5t80CBgyJDS97dwoT8e98jvcgwY4B5LVBf67rc+bZr+6c/OO+t0tsI+m623jp+fLTjkctBB8T/upibg4IN13Bao0YKnWMHCfd99w8vOPRc44ggdt8Hg+uu1ILbvG9x+7Fjg2GMLf+8f/1hfjQG+9jUdP/VUDXKTJmkBavN14kT/fxeysYXOmDH671q5bh7be+/yCtByHHign5+VYD/nYYfp/1F8+ml5BfCYMfo9/8//VCR5RAD6aLNS0H33aXV69uzCqvfRNnY7bZ+1Enyqou37/eqrufdZiJ120n395S+lbR/8bNFx2xtLRD/Xo4/mbl4Ibt/drc1clTJ1qu67kF4a9mmV++/vz3vwwXBvnlqxvW/sfxPkUkpalyypfhs81TXUZbNS0HHH6RnrMccUtr4947VnaqtX67+q2X/kCj6aotCaQyHsmXu5NYeouXP1FnprwADg8MPzNy9897v6usUW7n9Kl2PiRM23YNNTNmPHAj/5if7Dl3XUUfr/0LV25JH672H5aj+l2nvvyjcBEVVQhUuqGtlzz/jxOI8+Gv47z+HDtUnk1lt1upRmpULYwrIS+wpqaip+G/u5am3AAOCaa+KX3XBD7f9TeL/9avv+RDXUP4KD9cYb+pC4XLbZJv7P4G3hHRccKnFmXW6NYeFCreXUizPPrHUKCvfQQ9r5gagf6V/BoZwfqH3668iR/rxqNCuVejY8Zkz5aaDqOPLIWqeAqOL6V3Aox2GHAQ884Pf2ASobHD73OeCVVwpriyciqjEGB8sY4Oijw/MqGRx+/3t9hv5ee5W/LyKiKuv7vZWq6Xe/A7785cpcc9hhh77Vjk5EdY3BIZfx44FFi9jlkIjqjpGkdGvMwhjzEYAKPrugT/snAFn+R7TuMC98zAsf88I3WkSG5l8tXl+45vCqiMQ8HKn+GGPamReKeeFjXviYFz5jTIkPT1NsViIiIgeDAxEROfpCcJhe6wQkCPPCx7zwMS98zAtfWXmR+AvSRETU+/pCzYGIiHoZgwMRETkS25XVGNMMIAMgLSJ10Y5ojEkBSHvDOBE535vv5EU95Y8xZmq954UxpgF6XEBEZnnz6jUvCvrc/TkvvM82QUQOjczLoEL5ksiag5d4iMg8b7qEPy3ok04A0Bj48bfE5UU95Y/32dLeeD3nxQXecTHcGJOu17zwPlOn9xk7jTEN9ZgXtoywCs2DYvIlkcEBwDgAnd54J4CGGqal14jI9EAkT0M/e1xe1EX+GGNsHlh1mRfGmBYAbcaYtHeM1PNx0Q5gpq1JichC1G9eBBWaBwXnS1KDQyoyPaIWiagVr1Bc40X3VGTxiCzz+qO0VxBaqcjyesmL3aGfa40xptVrfkxF1qmLvBCRDIBWADMBjPVmpyKr1UVeRKQi09nyIG5erKQGhwyA4bVORA01i8gEbzwDNy/i5vUrxpgmW/UNyKAO88Kz1CsYOwC0oE7zwmsGmSciuwPIBNrP6y4vIjIoLA/i5sVK6gXpNvgRLg1gbu2S0ruMMc0icrU33oD4vEjFzOtv1ngFQQpAus7zog3+DzoF/YF3oj7zosH+PgBcBb1OV6/HRVCheRA3L1Yiaw7exZa0LRxiziD7Je/zTjXGdBhjOgAMj8uLesgfEVnofa7h8A7mOs6LWQBS9uKhd92hLvMCwHSvo0YTgBPqNS+8z9UYuMBcUB4Uky+8Q5qIiByJrDkQEVFtMTgQEZGDwYGIiBwMDkRE5GBwICIiB4MDUYmMMRNrnQaiamFXViIicrDmQFQk76moLd5d20T9EoMDUfFS3ms9PbuH6gyblYhKYIyZKSLja50OomphzYGoSN4js9ewWYn6MwYHotJkap0AompisxIRETlYcyAiIgeDAxERORgciIjIweBAREQOBgciInIwOBARkYPBgYiIHP8P+Dv+/X9P+F4AAAAASUVORK5CYII=\n", "text/plain": [ "
    " ] }, "metadata": { "filenames": { - "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/statistics_184_1.png" + "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/statistics_188_1.png" }, "needs_background": "light" }, @@ -2453,534 +2819,13 @@ }, { "cell_type": "markdown", - "metadata": {}, - "source": [ - "As can be seen from the plot, the first point gives back the variance and a value of one. \n", - "For the remaining values we notice that there are still non-zero values for the auto-correlation function.\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "## Correlation function and which random number generators should I use\n", - "The program here computes the correlation function for one of the standard functions included with the c++ compiler." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - " // This function computes the autocorrelation function for \n", - " // the standard c++ random number generator\n", - " \n", - " #include \n", - " #include \n", - " #include \n", - " #include \n", - " using namespace std;\n", - " // output file as global variable\n", - " ofstream ofile; \n", - " \n", - " // Main function begins here \n", - " int main(int argc, char* argv[])\n", - " {\n", - " int n;\n", - " char *outfilename;\n", - " \n", - " cin >> n;\n", - " double MCint = 0.; double MCintsqr2=0.;\n", - " double invers_period = 1./RAND_MAX; // initialise the random number generator\n", - " srand(time(NULL)); // This produces the so-called seed in MC jargon\n", - " // Compute the variance and the mean value of the uniform distribution\n", - " // Compute also the specific values x for each cycle in order to be able to\n", - " // the covariance and the correlation function \n", - " // Read in output file, abort if there are too few command-line arguments\n", - " if( argc <= 2 ){\n", - " cout << \"Bad Usage: \" << argv[0] << \n", - " \t \" read also output file and number of cycles on same line\" << endl;\n", - " exit(1);\n", - " }\n", - " else{\n", - " outfilename=argv[1];\n", - " }\n", - " ofile.open(outfilename); \n", - " // Get the number of Monte-Carlo samples\n", - " n = atoi(argv[2]);\n", - " double *X; \n", - " X = new double[n];\n", - " for (int i = 0; i < n; i++){\n", - " double x = double(rand())*invers_period; \n", - " X[i] = x;\n", - " MCint += x;\n", - " MCintsqr2 += x*x;\n", - " }\n", - " double Mean = MCint/((double) n );\n", - " MCintsqr2 = MCintsqr2/((double) n );\n", - " double STDev = sqrt(MCintsqr2-Mean*Mean);\n", - " double Variance = MCintsqr2-Mean*Mean;\n", - " // Write mean value and standard deviation \n", - " cout << \" Standard deviation= \" << STDev << \" Integral = \" << Mean << endl;\n", - " \n", - " // Now we compute the autocorrelation function\n", - " double *autocor; autocor = new double[n];\n", - " for (int j = 0; j < n; j++){\n", - " double sum = 0.0;\n", - " for (int k = 0; k < (n-j); k++){\n", - " \t sum += (X[k]-Mean)*(X[k+j]-Mean); \n", - " }\n", - " autocor[j] = sum/Variance/((double) n );\n", - " ofile << setiosflags(ios::showpoint | ios::uppercase);\n", - " ofile << setw(15) << setprecision(8) << j;\n", - " ofile << setw(15) << setprecision(8) << autocor[j] << endl;\n", - " }\n", - " ofile.close(); // close output file\n", - " return 0;\n", - " } // end of main program \n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Which RNG should I use?\n", - "* C++ has a class called **random**. The [random class](http://www.cplusplus.com/reference/random/) contains a large selection of RNGs and is highly recommended. Some of these RNGs have very large periods making it thereby very safe to use these RNGs in case one is performing large calculations. In particular, the [Mersenne twister random number engine](http://www.cplusplus.com/reference/random/mersenne_twister_engine/) has a period of $2^{19937}$. \n", - "\n", - "* Add RNGs in Python\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "## How to use the Mersenne generator\n", - "The following part of a c++ code (from project 4) sets up the uniform distribution for $x\\in [0,1]$." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - " /*\n", - " \n", - " // You need this \n", - " #include \n", - " \n", - " // Initialize the seed and call the Mersienne algo\n", - " std::random_device rd;\n", - " std::mt19937_64 gen(rd());\n", - " // Set up the uniform distribution for x \\in [[0, 1]\n", - " std::uniform_real_distribution RandomNumberGenerator(0.0,1.0);\n", - " \n", - " // Now use the RNG\n", - " int ix = (int) (RandomNumberGenerator(gen)*NSpins);\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Why blocking?\n", - "**Statistical analysis.**\n", - "\n", - " * Monte Carlo simulations can be treated as *computer experiments*\n", - "\n", - " * The results can be analysed with the same statistical tools as we would use analysing experimental data.\n", - "\n", - " * As in all experiments, we are looking for expectation values and an estimate of how accurate they are, i.e., possible sources for errors.\n", - "\n", - "A very good article which explains blocking is H. Flyvbjerg and H. G. Petersen, *Error estimates on averages of correlated data*, [Journal of Chemical Physics 91, 461-466 (1989)](http://scitation.aip.org/content/aip/journal/jcp/91/1/10.1063/1.457480).\n", - "\n", - " \n", - "\n", - "\n", - "\n", - "\n", - "## Why blocking?\n", - "**Statistical analysis.**\n", - "\n", - " * As in other experiments, Monte Carlo experiments have two classes of errors:\n", - "\n", - " * Statistical errors\n", - "\n", - " * Systematical errors\n", - "\n", - "\n", - " * Statistical errors can be estimated using standard tools from statistics\n", - "\n", - " * Systematical errors are method specific and must be treated differently from case to case. (In VMC a common source is the step length or time step in importance sampling)\n", - "\n", - " \n", - "\n", - "\n", - "\n", - "## Code to demonstrate the calculation of the autocorrelation function\n", - "The following code computes the autocorrelation function, the covariance and the standard deviation\n", - "for standard RNG. \n", - "The [following file](https://github.com/CompPhysics/ComputationalPhysics2/tree/gh-pages/doc/Programs/LecturePrograms/programs/Blocking/autocorrelation.cpp) gives the code." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - " // This function computes the autocorrelation function for \n", - " // the Mersenne random number generator with a uniform distribution\n", - " #include \n", - " #include \n", - " #include \n", - " #include \n", - " #include \n", - " #include \n", - " #include \n", - " #include \n", - " using namespace std;\n", - " using namespace arma;\n", - " // output file\n", - " ofstream ofile;\n", - " \n", - " // Main function begins here \n", - " int main(int argc, char* argv[])\n", - " {\n", - " int MonteCarloCycles;\n", - " string filename;\n", - " if (argc > 1) {\n", - " filename=argv[1];\n", - " MonteCarloCycles = atoi(argv[2]);\n", - " string fileout = filename;\n", - " string argument = to_string(MonteCarloCycles);\n", - " fileout.append(argument);\n", - " ofile.open(fileout);\n", - " }\n", - " \n", - " // Compute the variance and the mean value of the uniform distribution\n", - " // Compute also the specific values x for each cycle in order to be able to\n", - " // compute the covariance and the correlation function \n", - " \n", - " vec X = zeros(MonteCarloCycles);\n", - " double MCint = 0.; double MCintsqr2=0.;\n", - " std::random_device rd;\n", - " std::mt19937_64 gen(rd());\n", - " // Set up the uniform distribution for x \\in [[0, 1]\n", - " std::uniform_real_distribution RandomNumberGenerator(0.0,1.0);\n", - " for (int i = 0; i < MonteCarloCycles; i++){\n", - " double x = RandomNumberGenerator(gen); \n", - " X(i) = x;\n", - " MCint += x;\n", - " MCintsqr2 += x*x;\n", - " }\n", - " double Mean = MCint/((double) MonteCarloCycles );\n", - " MCintsqr2 = MCintsqr2/((double) MonteCarloCycles );\n", - " double STDev = sqrt(MCintsqr2-Mean*Mean);\n", - " double Variance = MCintsqr2-Mean*Mean;\n", - " // Write mean value and variance\n", - " cout << \" Sample variance= \" << Variance << \" Mean value = \" << Mean << endl;\n", - " // Now we compute the autocorrelation function\n", - " vec autocorrelation = zeros(MonteCarloCycles);\n", - " for (int j = 0; j < MonteCarloCycles; j++){\n", - " double sum = 0.0;\n", - " for (int k = 0; k < (MonteCarloCycles-j); k++){\n", - " sum += (X(k)-Mean)*(X(k+j)-Mean); \n", - " }\n", - " autocorrelation(j) = sum/Variance/((double) MonteCarloCycles );\n", - " ofile << setiosflags(ios::showpoint | ios::uppercase);\n", - " ofile << setw(15) << setprecision(8) << j;\n", - " ofile << setw(15) << setprecision(8) << autocorrelation(j) << endl;\n", - " }\n", - " // Now compute the exact covariance using the autocorrelation function\n", - " double Covariance = 0.0;\n", - " for (int j = 0; j < MonteCarloCycles; j++){\n", - " Covariance += autocorrelation(j);\n", - " }\n", - " Covariance *= 2.0/((double) MonteCarloCycles);\n", - " // Compute now the total variance, including the covariance, and obtain the standard deviation\n", - " double TotalVariance = (Variance/((double) MonteCarloCycles ))+Covariance;\n", - " cout << \"Covariance =\" << Covariance << \"Totalvariance= \" << TotalVariance << \"Sample Variance/n= \" << (Variance/((double) MonteCarloCycles )) << endl;\n", - " cout << \" STD from sample variance= \" << sqrt(Variance/((double) MonteCarloCycles )) << \" STD with covariance = \" << sqrt(TotalVariance) << endl;\n", - " \n", - " ofile.close(); // close output file\n", - " return 0;\n", - " } // end of main program \n", - " \n", - " \n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## What is blocking?\n", - "**Blocking.**\n", - "\n", - " * Say that we have a set of samples from a Monte Carlo experiment\n", - "\n", - " * Assuming (wrongly) that our samples are uncorrelated our best estimate of the standard deviation of the mean $\\langle \\mathbf{M}\\rangle$ is given by" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "$$\n", - "\\sigma=\\sqrt{\\frac{1}{n}\\left(\\langle \\mathbf{M}^2\\rangle-\\langle \\mathbf{M}\\rangle^2\\right)}\n", - "$$" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "* If the samples are correlated we can rewrite our results to show that" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "$$\n", - "\\sigma=\\sqrt{\\frac{1+2\\tau/\\Delta t}{n}\\left(\\langle \\mathbf{M}^2\\rangle-\\langle \\mathbf{M}\\rangle^2\\right)}\n", - "$$" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "where $\\tau$ is the correlation time (the time between a sample and the next uncorrelated sample) and $\\Delta t$ is time between each sample\n", - "\n", - " \n", - "\n", - "\n", - "## What is blocking?\n", - "**Blocking.**\n", - "\n", - " * If $\\Delta t\\gg\\tau$ our first estimate of $\\sigma$ still holds\n", - "\n", - " * Much more common that $\\Delta t<\\tau$\n", - "\n", - " * In the method of data blocking we divide the sequence of samples into blocks\n", - "\n", - " * We then take the mean $\\langle \\mathbf{M}_i\\rangle$ of block $i=1\\ldots n_{blocks}$ to calculate the total mean and variance\n", - "\n", - " * The size of each block must be so large that sample $j$ of block $i$ is not correlated with sample $j$ of block $i+1$\n", - "\n", - " * The correlation time $\\tau$ would be a good choice\n", - "\n", - "\n", - "\n", - "\n", - "## What is blocking?\n", - "**Blocking.**\n", - "\n", - " * Problem: We don't know $\\tau$ or it is too expensive to compute\n", - "\n", - " * Solution: Make a plot of std. dev. as a function of blocksize\n", - "\n", - " * The estimate of std. dev. of correlated data is too low $\\to$ the error will increase with increasing block size until the blocks are uncorrelated, where we reach a plateau\n", - "\n", - " * When the std. dev. stops increasing the blocks are uncorrelated\n", - "\n", - "\n", - "\n", - "\n", - "## Implementation\n", - " * Do a Monte Carlo simulation, storing all samples to file\n", - "\n", - " * Do the statistical analysis on this file, independently of your Monte Carlo program\n", - "\n", - " * Read the file into an array\n", - "\n", - " * Loop over various block sizes\n", - "\n", - " * For each block size $n_b$, loop over the array in steps of $n_b$ taking the mean of elements $i n_b,\\ldots,(i+1) n_b$\n", - "\n", - " * Take the mean and variance of the resulting array\n", - "\n", - " * Write the results for each block size to file for later\n", - " analysis\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "## Actual implementation with code, main function\n", - "When the file gets large, it can be useful to write your data in binary mode instead of ascii characters.\n", - "The [following python file](https://github.com/CompPhysics/MachineLearning/blob/master/doc/Programs/Sampling/analysis.py) reads data from file with the output from every Monte Carlo cycle." - ] - }, - { - "cell_type": "code", - "execution_count": 6, + "id": "5848dd32", "metadata": { - "collapsed": false, "editable": true }, - "outputs": [ - { - "ename": "IndentationError", - "evalue": "unexpected indent (, line 2)", - "output_type": "error", - "traceback": [ - "\u001b[0;36m File \u001b[0;32m\"\"\u001b[0;36m, line \u001b[0;32m2\u001b[0m\n\u001b[0;31m @timeFunction\u001b[0m\n\u001b[0m ^\u001b[0m\n\u001b[0;31mIndentationError\u001b[0m\u001b[0;31m:\u001b[0m unexpected indent\n" - ] - } - ], "source": [ - "# Blocking\n", - " @timeFunction\n", - " def blocking(self, blockSizeMax = 500):\n", - " blockSizeMin = 1\n", - "\n", - " self.blockSizes = []\n", - " self.meanVec = []\n", - " self.varVec = []\n", - "\n", - " for i in range(blockSizeMin, blockSizeMax):\n", - " if(len(self.data) % i != 0):\n", - " pass#continue\n", - " blockSize = i\n", - " meanTempVec = []\n", - " varTempVec = []\n", - " startPoint = 0\n", - " endPoint = blockSize\n", - "\n", - " while endPoint <= len(self.data):\n", - " meanTempVec.append(np.average(self.data[startPoint:endPoint]))\n", - " startPoint = endPoint\n", - " endPoint += blockSize\n", - " mean, var = np.average(meanTempVec), np.var(meanTempVec)/len(meanTempVec)\n", - " self.meanVec.append(mean)\n", - " self.varVec.append(var)\n", - " self.blockSizes.append(blockSize)\n", - "\n", - " self.blockingAvg = np.average(self.meanVec[-200:])\n", - " self.blockingVar = (np.average(self.varVec[-200:]))\n", - " self.blockingStd = np.sqrt(self.blockingVar)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## The Bootstrap method\n", - "\n", - "The Bootstrap resampling method is also very popular. It is very simple:\n", - "\n", - "1. Start with your sample of measurements and compute the sample variance and the mean values\n", - "\n", - "2. Then start again but pick in a random way the numbers in the sample and recalculate the mean and the sample variance.\n", - "\n", - "3. Repeat this $K$ times.\n", - "\n", - "It can be shown, see the article by [Efron](https://projecteuclid.org/download/pdf_1/euclid.aos/1176344552)\n", - "that it produces the correct standard deviation.\n", - "\n", - "This method is very useful for small ensembles of data points. \n", - "\n", - "\n", - "## Bootstrapping\n", - "Given a set of $N$ data, assume that we are interested in some \n", - "observable $\\theta$ which may be estimated from that set. This observable can also be for example the result of a fit based on all $N$ raw data. \n", - "Let us call the value of the observable obtained from the original \n", - "data set $\\hat{\\theta}$. One recreates from the sample repeatedly \n", - "other samples by choosing randomly $N$ data out of the original set. \n", - "This costs essentially nothing, since we just recycle the original data set for the building of new sets. \n", - "\n", - "\n", - "## Bootstrapping, recipe\n", - "Let us assume we have done this $K$ times and thus have $K$ sets of $N$ \n", - "data values each. \n", - "Of course some values will enter more than once in the new sets. For each of these sets one computes the observable $\\theta$ resulting in values $\\theta_k$ with $k = 1,...,K$. Then one determines" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "$$\n", - "\\tilde{\\theta} = \\frac{1}{K} \\sum_{k=1}^K \\theta_k,\n", - "$$" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "and" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "$$\n", - "sigma^2_{\\tilde{\\theta}} = \\frac{1}{K} \\sum_{k=1}^K \\left(\\theta_k-\\tilde{\\theta}\\right)^2.\n", - "$$" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "These are estimators for $\\angle\\theta\\rangle$ and its variance. They are not unbiased and therefore \n", - "$\\tilde{\\theta}\\neq\\hat{\\theta}$ for finite K. \n", - "\n", - "The difference is called bias and gives an idea on how far away the result may be from \n", - "the true $\\angle\\theta\\rangle$. As final result for the observable one quotes $\\angle\\theta\\rangle = \\tilde{\\theta} \\pm \\sigma_{\\tilde{\\theta}}$ .\n", - "\n", - "\n", - "\n", - "## Bootstrapping, [code](https://github.com/CompPhysics/MachineLearning/blob/master/doc/Programs/Sampling/analysis.py)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - " # Bootstrap\n", - " @timeFunction\n", - " def bootstrap(self, nBoots = 1000):\n", - " bootVec = np.zeros(nBoots)\n", - " for k in range(0,nBoots):\n", - " bootVec[k] = np.average(np.random.choice(self.data, len(self.data)))\n", - " self.bootAvg = np.average(bootVec)\n", - " self.bootVar = np.var(bootVec)\n", - " self.bootStd = np.std(bootVec)\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Jackknife, [code](https://github.com/CompPhysics/MachineLearning/blob/master/doc/Programs/Sampling/analysis.py)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - " # Jackknife\n", - " @timeFunction\n", - " def jackknife(self):\n", - " jackknVec = np.zeros(len(self.data))\n", - " for k in range(0,len(self.data)):\n", - " jackknVec[k] = np.average(np.delete(self.data, k))\n", - " self.jackknAvg = self.avg - (len(self.data) - 1) * (np.average(jackknVec) - self.avg)\n", - " self.jackknVar = float(len(self.data) - 1) * np.var(jackknVec)\n", - " self.jackknStd = np.sqrt(self.jackknVar)\n" + "As can be seen from the plot, the first point gives back the variance and a value of one. \n", + "For the remaining values we notice that there are still non-zero values for the auto-correlation function." ] } ], @@ -2999,5 +2844,5 @@ } }, "nbformat": 4, - "nbformat_minor": 4 + "nbformat_minor": 5 } \ No newline at end of file diff --git a/doc/LectureNotes/_build/jupyter_execute/statistics.py b/doc/LectureNotes/_build/jupyter_execute/statistics.py index d46d27034..9720a012f 100644 --- a/doc/LectureNotes/_build/jupyter_execute/statistics.py +++ b/doc/LectureNotes/_build/jupyter_execute/statistics.py @@ -1,10 +1,13 @@ #!/usr/bin/env python # coding: utf-8 +# + # # Elements of Probability Theory and Statistical Data Analysis -# -# + # ## Domains and probabilities +# # Consider the following simple example, namely the tossing of two dice, resulting in the following possible values # $$ @@ -18,7 +21,6 @@ # \{1/36,2/36/,3/36,4/36,5/36,6/36,5/36,4/36,3/36,2/36,1/36\}. # $$ -# ## Tossing the dice # The numbers in the domain are the outcomes of the physical process of tossing say two dice. # We cannot tell beforehand whether the outcome is 3 or 5 or any other number in this domain. # This defines the randomness of the outcome, or unexpectedness or any other synonimous word which @@ -39,14 +41,10 @@ # \{10,8,6,3,6,9,11,8,12,4,5\}. # $$ -# ## Stochastic variables -# # **Random variables are characterized by a domain which contains all possible values that the random value may take. This domain has a corresponding probability distribution function(PDF)**. + +# ### Stochastic variables and the main concepts, the discrete case # -# -# -# -# ## Stochastic variables and the main concepts, the discrete case # There are two main concepts associated with a stochastic variable. The # *domain* is the set $\mathbb D = \{x\}$ of all accessible values # the variable can assume, so that $X \in \mathbb D$. An example of a @@ -62,7 +60,6 @@ # p(x) = \mathrm{Prob}(X=x). # $$ -# ## Stochastic variables and the main concepts, the continuous case # In the continuous case, the PDF does not directly depict the # actual probability. Instead we define the probability for the # stochastic variable to assume any value on an infinitesimal interval @@ -79,10 +76,6 @@ # numbers chosen as if by chance from some specified PDF so that the # selection of a large set of these numbers reproduces this PDF. # -# -# -# -# ## The cumulative probability # Of interest to us is the *cumulative probability # distribution function* (**CDF**), $P(x)$, which is just the probability # for a stochastic variable $X$ to assume any value less than $x$ @@ -98,7 +91,7 @@ # p(x) = \frac{d}{dx}P(x). # $$ -# ## Properties of PDFs +# ### Properties of PDFs # # There are two properties that all PDFs must satisfy. The first one is # positivity (assuming that the PDF is normalized) @@ -120,7 +113,6 @@ # \end{align*} # $$ -# ## Important distributions, the uniform distribution # The first one # is the most basic PDF; namely the uniform distribution @@ -142,12 +134,8 @@ # \end{array} # $$ -# The latter distribution is used to generate random numbers. For other PDFs, one needs normally a mapping from this distribution to say for example the exponential distribution. +# The latter distribution is used to generate random numbers. For other PDFs, one needs normally a mapping from this distribution to say for example the exponential distribution. # -# -# -# -# ## Gaussian distribution # The second one is the Gaussian Distribution # $$ @@ -206,14 +194,14 @@ plt.savefig('gaussian.pdf', format='pdf') plt.show() -# ## Exponential distribution # Another important distribution in science is the exponential distribution # $$ # p(x) = \alpha\exp{-(\alpha x)}. # $$ -# ## Expectation values +# ### Expectation values +# # Let $h(x)$ be an arbitrary continuous function on the domain of the stochastic # variable $X$ whose PDF is $p(x)$. We define the *expectation value* # of $h$ with respect to $p$ as follows @@ -238,7 +226,6 @@ plt.show() # \langle x^n \rangle \equiv \int\! x^n p(x)\,dx # $$ -# ## Stochastic variables and the main concepts, mean values # The zero-th moment $\langle 1\rangle$ is just the normalization condition of # $p$. The first moment, $\langle x\rangle$, is called the *mean* of $p$ # and often denoted by the letter $\mu$ @@ -257,11 +244,6 @@ plt.show() # Qualitatively it represents the centroid or the average value of the # PDF and is therefore simply called the expectation value of $p(x)$. # -# -# -# -# ## Stochastic variables and the main concepts, central moments, the variance -# # A special version of the moments is the set of *central moments*, the n-th central moment defined as # $$ @@ -287,20 +269,14 @@ plt.show() # **standard deviation** of $p$. It is the RMS (root-mean-square) # value of the deviation of the PDF from its mean value, interpreted # qualitatively as the "spread" of $p$ around its mean. -# -# -# -# -# -# -# ## Probability Distribution Functions + +# ### Probability Distribution Functions # # The following table collects properties of probability distribution functions. # In our notation we reserve the label $p(x)$ for the probability of a certain event, # while $P(x)$ is the cumulative probability. # -# -# +#
    # # # @@ -315,11 +291,6 @@ plt.show() # #
    Discrete PDF Continuous PDF
    # -# -# -# -# -# ## Probability Distribution Functions # With a PDF we can compute expectation values of selected quantities such as # $$ @@ -333,12 +304,7 @@ plt.show() # $$ # in the case of a continuous PDF. We have already defined the mean value $\mu$ -# and the variance $\sigma^2$. -# -# -# -# -# ## The three famous Probability Distribution Functions +# and the variance $\sigma^2$. # # There are at least three PDFs which one may encounter. These are the # @@ -368,7 +334,6 @@ plt.show() # \sigma^2=\int_0^{\infty}x^2p(x)dx-\mu^2 = \frac{1}{\alpha^2}. # $$ -# ## Probability Distribution Functions, the normal distribution # Finally, we have the so-called univariate normal distribution, or just the **normal distribution** # $$ @@ -390,7 +355,6 @@ plt.show() # \mu =\frac{1}{b\sqrt{2\pi}}\int_{-\infty}^{\infty}b\sqrt{2}(a+b\sqrt{2}y)\exp{-y^2}dy=a. # $$ -# ## Probability Distribution Functions, the normal distribution # Similarly, the variance becomes # $$ @@ -418,8 +382,6 @@ plt.show() # \end{equation} # $$ -# ## Probability Distribution Functions, the cumulative distribution -# # The exponential and uniform distributions have simple cumulative functions, # whereas the normal distribution does not, being proportional to the so-called # error function $erf(x)$, given by @@ -428,13 +390,7 @@ plt.show() # P(x) = \frac{1}{\sqrt{2\pi}}\int_{-\infty}^x\exp{\left(-\frac{t^2}{2}\right)}dt, # $$ -# which is difficult to evaluate in a quick way. -# -# -# -# -# -# ## Probability Distribution Functions, other important distribution +# which is difficult to evaluate in a quick way. # # Some other PDFs which one encounters often in the natural sciences are the binomial distribution @@ -453,11 +409,6 @@ plt.show() # # * The probability for every outcome in a single trial has the same value, for example the outcome of tossing (either heads or tails) a coin is always $1/2$. # -# -# -# -# ## Probability Distribution Functions, the binomial distribution -# # In order to compute the mean and variance we need to recall Newton's binomial # formula @@ -494,9 +445,6 @@ plt.show() # The variance is slightly trickier to get. It reads $\sigma^2=ny(1-y)$. # -# -# ## Probability Distribution Functions, Poisson's distribution -# # Another important distribution with discrete stochastic variables $x$ is # the Poisson model, which resembles the exponential distribution and reads @@ -511,14 +459,8 @@ plt.show() # \frac{\lambda^{x-1}}{(x-1)!}=\lambda, # $$ -# and the variance is $\sigma^2=\lambda$. +# and the variance is $\sigma^2=\lambda$. # -# -# -# -# -# -# ## Probability Distribution Functions, Poisson's distribution # An example of applications of the Poisson distribution could be the counting # of the number of $\alpha$-particles emitted from a radioactive source in a given time interval. # In the limit of $n\rightarrow \infty$ and for small probabilities $y$, the binomial distribution @@ -529,7 +471,8 @@ plt.show() # \lim_{n\rightarrow \infty}\left(\begin{array}{c} n \\ x\end{array}\right)y^x(1-y)^{n-x} e^{-\lambda}=\sum_{x=1}^{\infty}\frac{\lambda^x}{x!} e^{-\lambda}. # $$ -# ## Meet the covariance! +# ### Meet the covariance! +# # An important quantity in a statistical analysis is the so-called covariance. # # Consider the set $\{X_i\}$ of $n$ @@ -564,7 +507,6 @@ plt.show() # \int\cdots\int x_i P(x_1,\dots,x_n)\,dx_1\dots dx_n. # $$ -# ## Meet the covariance in matrix disguise # If we consider the above covariance as a matrix # $$ @@ -575,11 +517,6 @@ plt.show() # variances, $C_{ii} = \mathrm{Cov}(X_i,\,X_i) = \mathrm{Var}(X_i)$. It turns out that # all the off-diagonal elements are zero if the stochastic variables are # uncorrelated. -# -# -# -# -# ## Covariance # In[2]: @@ -610,8 +547,6 @@ c = np.cov(z.T) print(c) -# ## Meet the covariance, uncorrelated events -# # Consider the stochastic variables $X_i$ and $X_j$, ($i\neq j$). We have # $$ @@ -638,8 +573,6 @@ print(c) # Cov(X_i, X_j) = 0 \hspace{0.1cm} (i\neq j). # $$ -# ## Numerical experiments and the covariance -# # Now that we have constructed an idealized mathematical framework, let # us try to apply it to empirical observations. Examples of relevant # physical phenomena may be spontaneous decays of nuclei, or a purely @@ -654,7 +587,6 @@ print(c) # \{x_1, x_2,\dots\,x_k,\dots\}. # $$ -# ## Numerical experiments and the covariance # We will call these # values our *measurements* and the entire set as our measured # *sample*. The action of measuring all the elements of a sample @@ -668,11 +600,6 @@ print(c) # interested in finding the few lowest moments, like the mean # $\mu_X^{\phantom X}$ and the variance $\sigma_X^{\phantom X}$. # -# -# -# -# -# ## Numerical experiments and the covariance, actual situations # In practical situations however, a sample is always of finite size. Let that # size be $n$. The expectation value of a sample $\alpha$, the **sample mean**, is then defined as follows @@ -686,13 +613,8 @@ print(c) # \mathrm{Var}(x) \equiv \frac{1}{n}\sum_{k=1}^n (x_{\alpha,k} - \langle x_{\alpha} \rangle)^2, # $$ -# with its square root being the *standard deviation of the sample*. +# with its square root being the *standard deviation of the sample*. # -# -# -# -# -# ## Numerical experiments and the covariance, our observables # You can think of the above observables as a set of quantities which define # a given experiment. This experiment is then repeated several times, say $m$ times. # The total average is then @@ -726,8 +648,6 @@ print(c) # \end{equation} # $$ -# ## Numerical experiments and the covariance, the sample variance -# # We define also the sample variance $\sigma^2$ of all $mn$ individual experiments as # @@ -745,11 +665,8 @@ print(c) # significantly, from the similarly named # exact values for the mean value $\mu_X$, the variance $\mathrm{Var}(X)$ # and the covariance $\mathrm{Cov}(X,Y)$. -# -# -# -# -# ## Numerical experiments and the covariance, central limit theorem + +# ### Numerical experiments and the covariance, central limit theorem # # The central limit theorem states that the PDF $\tilde{p}(z)$ of # the average of $m$ random values corresponding to a PDF $p(x)$ @@ -767,10 +684,6 @@ print(c) # In many cases the above estimate for the standard deviation, in particular if correlations are strong, may be too simplistic. We need therefore a more precise defintion of the error and the variance in our results. # -# -# -# -# ## Definition of Correlation Functions and Standard Deviation # Our estimate of the true average $\mu_{X}$ is the sample mean $\langle X_m \rangle$ # $$ @@ -790,16 +703,8 @@ print(c) # $$ # where the first term is the sample variance of all $mn$ experiments divided by $n$ -# and the last term is nothing but the covariance which arises when $k\ne l$. +# and the last term is nothing but the covariance which arises when $k\ne l$. # -# -# -# -# -# -# -# -# ## Definition of Correlation Functions and Standard Deviation # Our estimate of the true average $\mu_{X}$ is the sample mean $\langle X_m \rangle$ # # If the @@ -814,11 +719,6 @@ print(c) # measurements to calculate the cross terms. Therefore, all measurements # have to be stored throughout the experiment. # -# -# -# -# ## Definition of Correlation Functions and Standard Deviation -# # Let us analyze the problem by splitting up the correlation term into # partial sums of the form @@ -834,7 +734,6 @@ print(c) # \frac{2}{n}\sum_{d=1}^{n-1} f_d # $$ -# ## Definition of Correlation Functions and Standard Deviation # The value of $f_d$ reflects the correlation between measurements # separated by the distance $d$ in the samples. Notice that for # $d=0$, $f$ is just the sample variance, $\sigma^2$. If we divide $f_d$ @@ -853,11 +752,6 @@ print(c) # which gives us a useful measure of the correlation pair correlation # starting always at $1$ for $d=0$. # -# -# -# -# ## Definition of Correlation Functions and Standard Deviation, sample variance -# # The sample variance of the $mn$ experiments can now be # written in terms of the autocorrelation function @@ -890,14 +784,8 @@ print(c) # # # For a correlation free experiment, $\tau$ -# equals 1. +# equals 1. # -# -# -# -# -# -# ## Definition of Correlation Functions and Standard Deviation # From the point of view of # Eq. ([10](#eq:error_estimate_corr_time)) we can interpret a sequential # correlation as an effective reduction of the number of measurements by @@ -915,12 +803,6 @@ print(c) # measurements is very large. The solution to this problem is given by # more practically oriented methods like the blocking technique. # -# -# -# -# -# -# ## Code to compute the Covariance matrix and the Covariance # In[3]: @@ -964,7 +846,7 @@ Eigvals, Eigvecs = np.linalg.eig(c) print(Eigvals) -# ## Random Numbers +# ### Random Numbers # # Uniform deviates are just random numbers that lie within a specified range # (typically 0 to 1), with any one number in the range just as likely as any other. They @@ -977,12 +859,6 @@ print(Eigvals) # of this section, is an essential building block for any sort of stochastic modeling # or Monte Carlo computer work. # -# -# -# -# -# ## Random Numbers, better name: pseudo random numbers -# # A disclaimer is however appropriate. It should be fairly obvious that # something as deterministic as a computer cannot generate purely random numbers. # @@ -997,11 +873,6 @@ print(Eigvals) # # * the algorithm should be fast. # -# -# -# -# -# ## Random number generator RNG # The most common random number generators are based on so-called # Linear congruential relations of the type @@ -1022,11 +893,6 @@ print(Eigvals) # that is if we were to evaluate $(13)\mathrm{MOD}(9)$, the outcome is the remainder # of the division $13/9$, namely $4$. # -# -# -# -# ## Random number generator RNG and periodic outputs -# # The problem with such generators is that their outputs are periodic; # they # will start to repeat themselves with a period that is at most $M$. If however @@ -1050,10 +916,6 @@ print(Eigvals) # which still, with $N_0=2$, results in $11,38,11,38,11,38,\dots$, a period of # just $2$. # -# -# -# -# ## Random number generator RNG and its period # Typical periods for the random generators provided in the program library # are of the order of $\sim 10^9$ or larger. Other random number generators which have # become increasingly popular are so-called shift-register generators. @@ -1067,7 +929,6 @@ print(Eigvals) # N_l=(aN_{l-i}+cN_{l-j})\mathrm{MOD}(M). # $$ -# ## Random number generator RNG, other examples # Such a generator again produces a sequence of pseudorandom numbers # but this time with a period much larger than $M$. # It is also possible to construct more elaborate algorithms by including @@ -1099,10 +960,6 @@ print(Eigvals) # which according to the authors has a period larger than $2^{94}$. # -# -# -# -# ## Random number generator RNG, other examples # Instead of using modular addition, we could use the bitwise # exclusive-OR ($\oplus$) operation so that @@ -1119,13 +976,7 @@ print(Eigvals) # # In Fortran90, the bitwise $\oplus$ operation is coded through the intrinsic # function $\mathrm{IEOR}(m,n)$ where $m$ and $n$ are the input numbers, while in $C$ -# it is given by $m\wedge n$. -# -# -# -# -# -# ## Random number generator RNG, RAN0 +# it is given by $m\wedge n$. # # We show here how the linear congruential algorithm can be implemented, namely @@ -1157,13 +1008,6 @@ print(Eigvals) # where the brackets denote integer division. In the code below the numbers # $q$ and $r$ are chosen so that $r < q$. # -# -# -# -# -# -# ## Random number generator RNG, RAN0 -# # To see how this works we note first that # @@ -1178,13 +1022,8 @@ print(Eigvals) # since we can add or subtract any integer multiple of $M$ from $aN_{i-1}$. # The last term $[N_{i-1}/q]M\mathrm{MOD}(M)$ is zero since the integer division -# $[N_{i-1}/q]$ just yields a constant which is multiplied with $M$. +# $[N_{i-1}/q]$ just yields a constant which is multiplied with $M$. # -# -# -# -# -# ## Random number generator RNG, RAN0 # We can now rewrite Eq. ([14](#eq:rntrick1)) as # @@ -1198,7 +1037,6 @@ print(Eigvals) # $$ # which results -# in # #
    @@ -1222,7 +1060,6 @@ print(Eigvals) # \end{equation} # $$ -# ## Random number generator RNG, RAN0 # The term $[N_{i-1}/q]r$ is always smaller or equal $N_{i-1}(r/q)$ and with $r < q$ we obtain always a # number smaller than $N_{i-1}$, which is smaller than $M$. # And since the number $N_{i-1}\mathrm{MOD} (q)$ is between zero and $q-1$ then @@ -1234,43 +1071,7 @@ print(Eigvals) # the starting point for each generation of a random number. The period # of $ran0$ is $\sim 2.1\times 10^{9}$. A special feature of this # algorithm is that is should never be called with the initial seed -# set to $0$. -# -# -# -# -# -# ## Random number generator RNG, RAN0 code - -# /* -# ** The function -# ** ran0() -# ** is an "Minimal" random number generator of Park and Miller -# ** Set or reset the input value -# ** idum to any integer value (except the unlikely value MASK) -# ** to initialize the sequence; idum must not be altered between -# ** calls for sucessive deviates in a sequence. -# ** The function returns a uniform deviate between 0.0 and 1.0. -# */ -# double ran0(long &idum) -# { -# const int a = 16807, m = 2147483647, q = 127773; -# const int r = 2836, MASK = 123459876; -# const double am = 1./m; -# long k; -# double ans; -# idum ^= MASK; -# k = (*idum)/q; -# idum = a*(idum - k*q) - r*k; -# // add m if negative difference -# if(idum < 0) idum += m; -# ans=am*(idum); -# idum ^= MASK; -# return ans; -# } // End: function ran0() -# - -# ## Properties of Selected Random Number Generators +# set to $0$. # # As mentioned previously, the underlying PDF for the generation of # random numbers is the uniform distribution, meaning that the @@ -1286,10 +1087,6 @@ print(Eigvals) # Two additional measures are the standard deviation $\sigma$ and the mean # $\mu=\langle x\rangle$. # -# -# -# -# ## Properties of Selected Random Number Generators # For the uniform distribution, the mean value $\mu$ is then # $$ @@ -1302,11 +1099,10 @@ print(Eigvals) # \sigma=\sqrt{\langle x^2\rangle-\mu^2}=\frac{1}{\sqrt{12}}=0.2886. # $$ -# ## Properties of Selected Random Number Generators # The various random number generators produce results which agree rather well with # these limiting values. # -# +#
    # # # @@ -1326,10 +1122,6 @@ print(Eigvals) # #
    $x$-bin ran0 ran1 ran2 ran3
    # -# -# -# -# ## Simple demonstration of RNGs using python # The following simple Python code plots the distribution of the produced random numbers using the linear congruential RNG employed by Python. The trend displayed in the previous table is seen rather clearly. # In[4]: @@ -1359,7 +1151,6 @@ plt.grid(True) plt.show() -# ## Properties of Selected Random Number Generators # Since our random numbers, which are typically generated via a linear congruential algorithm, # are never fully independent, we can then define # an important test which measures the degree of correlation, namely the so-called @@ -1382,12 +1173,9 @@ plt.show() # numbers are not independent. The independence of the random numbers is crucial # in the evaluation of other expectation values. If they are not independent, our # assumption for approximating $\sigma_N$ is no longer valid. + +# ### Autocorrelation function # -# -# -# -# -# ## Autocorrelation function # This program computes the autocorrelation function as discussed in the equation on the previous slide for random numbers generated with the normal distribution $N(0,1)$. # In[5]: @@ -1426,427 +1214,3 @@ plt.show() # As can be seen from the plot, the first point gives back the variance and a value of one. # For the remaining values we notice that there are still non-zero values for the auto-correlation function. -# -# -# -# -# -# -# -# -# -# -# -# -# -# -# -# -# -# ## Correlation function and which random number generators should I use -# The program here computes the correlation function for one of the standard functions included with the c++ compiler. - -# // This function computes the autocorrelation function for -# // the standard c++ random number generator -# -# #include -# #include -# #include -# #include -# using namespace std; -# // output file as global variable -# ofstream ofile; -# -# // Main function begins here -# int main(int argc, char* argv[]) -# { -# int n; -# char *outfilename; -# -# cin >> n; -# double MCint = 0.; double MCintsqr2=0.; -# double invers_period = 1./RAND_MAX; // initialise the random number generator -# srand(time(NULL)); // This produces the so-called seed in MC jargon -# // Compute the variance and the mean value of the uniform distribution -# // Compute also the specific values x for each cycle in order to be able to -# // the covariance and the correlation function -# // Read in output file, abort if there are too few command-line arguments -# if( argc <= 2 ){ -# cout << "Bad Usage: " << argv[0] << -# " read also output file and number of cycles on same line" << endl; -# exit(1); -# } -# else{ -# outfilename=argv[1]; -# } -# ofile.open(outfilename); -# // Get the number of Monte-Carlo samples -# n = atoi(argv[2]); -# double *X; -# X = new double[n]; -# for (int i = 0; i < n; i++){ -# double x = double(rand())*invers_period; -# X[i] = x; -# MCint += x; -# MCintsqr2 += x*x; -# } -# double Mean = MCint/((double) n ); -# MCintsqr2 = MCintsqr2/((double) n ); -# double STDev = sqrt(MCintsqr2-Mean*Mean); -# double Variance = MCintsqr2-Mean*Mean; -# // Write mean value and standard deviation -# cout << " Standard deviation= " << STDev << " Integral = " << Mean << endl; -# -# // Now we compute the autocorrelation function -# double *autocor; autocor = new double[n]; -# for (int j = 0; j < n; j++){ -# double sum = 0.0; -# for (int k = 0; k < (n-j); k++){ -# sum += (X[k]-Mean)*(X[k+j]-Mean); -# } -# autocor[j] = sum/Variance/((double) n ); -# ofile << setiosflags(ios::showpoint | ios::uppercase); -# ofile << setw(15) << setprecision(8) << j; -# ofile << setw(15) << setprecision(8) << autocor[j] << endl; -# } -# ofile.close(); // close output file -# return 0; -# } // end of main program -# - -# ## Which RNG should I use? -# * C++ has a class called **random**. The [random class](http://www.cplusplus.com/reference/random/) contains a large selection of RNGs and is highly recommended. Some of these RNGs have very large periods making it thereby very safe to use these RNGs in case one is performing large calculations. In particular, the [Mersenne twister random number engine](http://www.cplusplus.com/reference/random/mersenne_twister_engine/) has a period of $2^{19937}$. -# -# * Add RNGs in Python -# -# -# -# -# -# ## How to use the Mersenne generator -# The following part of a c++ code (from project 4) sets up the uniform distribution for $x\in [0,1]$. - -# /* -# -# // You need this -# #include -# -# // Initialize the seed and call the Mersienne algo -# std::random_device rd; -# std::mt19937_64 gen(rd()); -# // Set up the uniform distribution for x \in [[0, 1] -# std::uniform_real_distribution RandomNumberGenerator(0.0,1.0); -# -# // Now use the RNG -# int ix = (int) (RandomNumberGenerator(gen)*NSpins); -# - -# ## Why blocking? -# **Statistical analysis.** -# -# * Monte Carlo simulations can be treated as *computer experiments* -# -# * The results can be analysed with the same statistical tools as we would use analysing experimental data. -# -# * As in all experiments, we are looking for expectation values and an estimate of how accurate they are, i.e., possible sources for errors. -# -# A very good article which explains blocking is H. Flyvbjerg and H. G. Petersen, *Error estimates on averages of correlated data*, [Journal of Chemical Physics 91, 461-466 (1989)](http://scitation.aip.org/content/aip/journal/jcp/91/1/10.1063/1.457480). -# -# -# -# -# -# -# ## Why blocking? -# **Statistical analysis.** -# -# * As in other experiments, Monte Carlo experiments have two classes of errors: -# -# * Statistical errors -# -# * Systematical errors -# -# -# * Statistical errors can be estimated using standard tools from statistics -# -# * Systematical errors are method specific and must be treated differently from case to case. (In VMC a common source is the step length or time step in importance sampling) -# -# -# -# -# -# ## Code to demonstrate the calculation of the autocorrelation function -# The following code computes the autocorrelation function, the covariance and the standard deviation -# for standard RNG. -# The [following file](https://github.com/CompPhysics/ComputationalPhysics2/tree/gh-pages/doc/Programs/LecturePrograms/programs/Blocking/autocorrelation.cpp) gives the code. - -# // This function computes the autocorrelation function for -# // the Mersenne random number generator with a uniform distribution -# #include -# #include -# #include -# #include -# #include -# #include -# #include -# #include -# using namespace std; -# using namespace arma; -# // output file -# ofstream ofile; -# -# // Main function begins here -# int main(int argc, char* argv[]) -# { -# int MonteCarloCycles; -# string filename; -# if (argc > 1) { -# filename=argv[1]; -# MonteCarloCycles = atoi(argv[2]); -# string fileout = filename; -# string argument = to_string(MonteCarloCycles); -# fileout.append(argument); -# ofile.open(fileout); -# } -# -# // Compute the variance and the mean value of the uniform distribution -# // Compute also the specific values x for each cycle in order to be able to -# // compute the covariance and the correlation function -# -# vec X = zeros(MonteCarloCycles); -# double MCint = 0.; double MCintsqr2=0.; -# std::random_device rd; -# std::mt19937_64 gen(rd()); -# // Set up the uniform distribution for x \in [[0, 1] -# std::uniform_real_distribution RandomNumberGenerator(0.0,1.0); -# for (int i = 0; i < MonteCarloCycles; i++){ -# double x = RandomNumberGenerator(gen); -# X(i) = x; -# MCint += x; -# MCintsqr2 += x*x; -# } -# double Mean = MCint/((double) MonteCarloCycles ); -# MCintsqr2 = MCintsqr2/((double) MonteCarloCycles ); -# double STDev = sqrt(MCintsqr2-Mean*Mean); -# double Variance = MCintsqr2-Mean*Mean; -# // Write mean value and variance -# cout << " Sample variance= " << Variance << " Mean value = " << Mean << endl; -# // Now we compute the autocorrelation function -# vec autocorrelation = zeros(MonteCarloCycles); -# for (int j = 0; j < MonteCarloCycles; j++){ -# double sum = 0.0; -# for (int k = 0; k < (MonteCarloCycles-j); k++){ -# sum += (X(k)-Mean)*(X(k+j)-Mean); -# } -# autocorrelation(j) = sum/Variance/((double) MonteCarloCycles ); -# ofile << setiosflags(ios::showpoint | ios::uppercase); -# ofile << setw(15) << setprecision(8) << j; -# ofile << setw(15) << setprecision(8) << autocorrelation(j) << endl; -# } -# // Now compute the exact covariance using the autocorrelation function -# double Covariance = 0.0; -# for (int j = 0; j < MonteCarloCycles; j++){ -# Covariance += autocorrelation(j); -# } -# Covariance *= 2.0/((double) MonteCarloCycles); -# // Compute now the total variance, including the covariance, and obtain the standard deviation -# double TotalVariance = (Variance/((double) MonteCarloCycles ))+Covariance; -# cout << "Covariance =" << Covariance << "Totalvariance= " << TotalVariance << "Sample Variance/n= " << (Variance/((double) MonteCarloCycles )) << endl; -# cout << " STD from sample variance= " << sqrt(Variance/((double) MonteCarloCycles )) << " STD with covariance = " << sqrt(TotalVariance) << endl; -# -# ofile.close(); // close output file -# return 0; -# } // end of main program -# -# -# - -# ## What is blocking? -# **Blocking.** -# -# * Say that we have a set of samples from a Monte Carlo experiment -# -# * Assuming (wrongly) that our samples are uncorrelated our best estimate of the standard deviation of the mean $\langle \mathbf{M}\rangle$ is given by - -# $$ -# \sigma=\sqrt{\frac{1}{n}\left(\langle \mathbf{M}^2\rangle-\langle \mathbf{M}\rangle^2\right)} -# $$ - -# * If the samples are correlated we can rewrite our results to show that - -# $$ -# \sigma=\sqrt{\frac{1+2\tau/\Delta t}{n}\left(\langle \mathbf{M}^2\rangle-\langle \mathbf{M}\rangle^2\right)} -# $$ - -# where $\tau$ is the correlation time (the time between a sample and the next uncorrelated sample) and $\Delta t$ is time between each sample -# -# -# -# -# ## What is blocking? -# **Blocking.** -# -# * If $\Delta t\gg\tau$ our first estimate of $\sigma$ still holds -# -# * Much more common that $\Delta t<\tau$ -# -# * In the method of data blocking we divide the sequence of samples into blocks -# -# * We then take the mean $\langle \mathbf{M}_i\rangle$ of block $i=1\ldots n_{blocks}$ to calculate the total mean and variance -# -# * The size of each block must be so large that sample $j$ of block $i$ is not correlated with sample $j$ of block $i+1$ -# -# * The correlation time $\tau$ would be a good choice -# -# -# -# -# ## What is blocking? -# **Blocking.** -# -# * Problem: We don't know $\tau$ or it is too expensive to compute -# -# * Solution: Make a plot of std. dev. as a function of blocksize -# -# * The estimate of std. dev. of correlated data is too low $\to$ the error will increase with increasing block size until the blocks are uncorrelated, where we reach a plateau -# -# * When the std. dev. stops increasing the blocks are uncorrelated -# -# -# -# -# ## Implementation -# * Do a Monte Carlo simulation, storing all samples to file -# -# * Do the statistical analysis on this file, independently of your Monte Carlo program -# -# * Read the file into an array -# -# * Loop over various block sizes -# -# * For each block size $n_b$, loop over the array in steps of $n_b$ taking the mean of elements $i n_b,\ldots,(i+1) n_b$ -# -# * Take the mean and variance of the resulting array -# -# * Write the results for each block size to file for later -# analysis -# -# -# -# -# -# -# -# -# ## Actual implementation with code, main function -# When the file gets large, it can be useful to write your data in binary mode instead of ascii characters. -# The [following python file](https://github.com/CompPhysics/MachineLearning/blob/master/doc/Programs/Sampling/analysis.py) reads data from file with the output from every Monte Carlo cycle. - -# In[6]: - - -# Blocking - @timeFunction - def blocking(self, blockSizeMax = 500): - blockSizeMin = 1 - - self.blockSizes = [] - self.meanVec = [] - self.varVec = [] - - for i in range(blockSizeMin, blockSizeMax): - if(len(self.data) % i != 0): - pass#continue - blockSize = i - meanTempVec = [] - varTempVec = [] - startPoint = 0 - endPoint = blockSize - - while endPoint <= len(self.data): - meanTempVec.append(np.average(self.data[startPoint:endPoint])) - startPoint = endPoint - endPoint += blockSize - mean, var = np.average(meanTempVec), np.var(meanTempVec)/len(meanTempVec) - self.meanVec.append(mean) - self.varVec.append(var) - self.blockSizes.append(blockSize) - - self.blockingAvg = np.average(self.meanVec[-200:]) - self.blockingVar = (np.average(self.varVec[-200:])) - self.blockingStd = np.sqrt(self.blockingVar) - - -# ## The Bootstrap method -# -# The Bootstrap resampling method is also very popular. It is very simple: -# -# 1. Start with your sample of measurements and compute the sample variance and the mean values -# -# 2. Then start again but pick in a random way the numbers in the sample and recalculate the mean and the sample variance. -# -# 3. Repeat this $K$ times. -# -# It can be shown, see the article by [Efron](https://projecteuclid.org/download/pdf_1/euclid.aos/1176344552) -# that it produces the correct standard deviation. -# -# This method is very useful for small ensembles of data points. -# -# -# ## Bootstrapping -# Given a set of $N$ data, assume that we are interested in some -# observable $\theta$ which may be estimated from that set. This observable can also be for example the result of a fit based on all $N$ raw data. -# Let us call the value of the observable obtained from the original -# data set $\hat{\theta}$. One recreates from the sample repeatedly -# other samples by choosing randomly $N$ data out of the original set. -# This costs essentially nothing, since we just recycle the original data set for the building of new sets. -# -# -# ## Bootstrapping, recipe -# Let us assume we have done this $K$ times and thus have $K$ sets of $N$ -# data values each. -# Of course some values will enter more than once in the new sets. For each of these sets one computes the observable $\theta$ resulting in values $\theta_k$ with $k = 1,...,K$. Then one determines - -# $$ -# \tilde{\theta} = \frac{1}{K} \sum_{k=1}^K \theta_k, -# $$ - -# and - -# $$ -# sigma^2_{\tilde{\theta}} = \frac{1}{K} \sum_{k=1}^K \left(\theta_k-\tilde{\theta}\right)^2. -# $$ - -# These are estimators for $\angle\theta\rangle$ and its variance. They are not unbiased and therefore -# $\tilde{\theta}\neq\hat{\theta}$ for finite K. -# -# The difference is called bias and gives an idea on how far away the result may be from -# the true $\angle\theta\rangle$. As final result for the observable one quotes $\angle\theta\rangle = \tilde{\theta} \pm \sigma_{\tilde{\theta}}$ . -# -# -# -# ## Bootstrapping, [code](https://github.com/CompPhysics/MachineLearning/blob/master/doc/Programs/Sampling/analysis.py) - -# # Bootstrap -# @timeFunction -# def bootstrap(self, nBoots = 1000): -# bootVec = np.zeros(nBoots) -# for k in range(0,nBoots): -# bootVec[k] = np.average(np.random.choice(self.data, len(self.data))) -# self.bootAvg = np.average(bootVec) -# self.bootVar = np.var(bootVec) -# self.bootStd = np.std(bootVec) -# - -# ## Jackknife, [code](https://github.com/CompPhysics/MachineLearning/blob/master/doc/Programs/Sampling/analysis.py) - -# # Jackknife -# @timeFunction -# def jackknife(self): -# jackknVec = np.zeros(len(self.data)) -# for k in range(0,len(self.data)): -# jackknVec[k] = np.average(np.delete(self.data, k)) -# self.jackknAvg = self.avg - (len(self.data) - 1) * (np.average(jackknVec) - self.avg) -# self.jackknVar = float(len(self.data) - 1) * np.var(jackknVec) -# self.jackknStd = np.sqrt(self.jackknVar) -# diff --git a/doc/LectureNotes/_build/jupyter_execute/statistics_181_0.png b/doc/LectureNotes/_build/jupyter_execute/statistics_181_0.png new file mode 100644 index 000000000..ec260ba1d Binary files /dev/null and b/doc/LectureNotes/_build/jupyter_execute/statistics_181_0.png differ diff --git a/doc/LectureNotes/_build/jupyter_execute/statistics_188_1.png b/doc/LectureNotes/_build/jupyter_execute/statistics_188_1.png new file mode 100644 index 000000000..b13e4947f Binary files /dev/null and b/doc/LectureNotes/_build/jupyter_execute/statistics_188_1.png differ diff --git a/doc/LectureNotes/_build/jupyter_execute/statistics_32_0.png b/doc/LectureNotes/_build/jupyter_execute/statistics_32_0.png new file mode 100644 index 000000000..5adbf9cac Binary files /dev/null and b/doc/LectureNotes/_build/jupyter_execute/statistics_32_0.png differ diff --git a/doc/LectureNotes/gaussian.pdf b/doc/LectureNotes/gaussian.pdf index f3ca2ac24..868ff6e2e 100644 Binary files a/doc/LectureNotes/gaussian.pdf and b/doc/LectureNotes/gaussian.pdf differ diff --git a/doc/LectureNotes/statistics.ipynb b/doc/LectureNotes/statistics.ipynb index 4ea8e3730..abcd4c7f8 100644 --- a/doc/LectureNotes/statistics.ipynb +++ b/doc/LectureNotes/statistics.ipynb @@ -2,7 +2,7 @@ "cells": [ { "cell_type": "markdown", - "id": "1de4abfb", + "id": "70706709", "metadata": { "editable": true }, @@ -13,7 +13,7 @@ }, { "cell_type": "markdown", - "id": "9268be0a", + "id": "85b1eb1e", "metadata": { "editable": true }, @@ -23,7 +23,7 @@ }, { "cell_type": "markdown", - "id": "b24d433f", + "id": "f338b4e0", "metadata": { "editable": true }, @@ -35,7 +35,7 @@ }, { "cell_type": "markdown", - "id": "9fa7c59b", + "id": "596202aa", "metadata": { "editable": true }, @@ -47,7 +47,7 @@ }, { "cell_type": "markdown", - "id": "d1c9912c", + "id": "d81eea8e", "metadata": { "editable": true }, @@ -58,7 +58,7 @@ }, { "cell_type": "markdown", - "id": "6626ea89", + "id": "432953c1", "metadata": { "editable": true }, @@ -70,7 +70,7 @@ }, { "cell_type": "markdown", - "id": "65c54c5b", + "id": "1abe1fd4", "metadata": { "editable": true }, @@ -88,7 +88,7 @@ }, { "cell_type": "markdown", - "id": "84e29b68", + "id": "9320be8c", "metadata": { "editable": true }, @@ -100,7 +100,7 @@ }, { "cell_type": "markdown", - "id": "8ae71934", + "id": "7ac13e6b", "metadata": { "editable": true }, @@ -110,7 +110,7 @@ }, { "cell_type": "markdown", - "id": "c3901951", + "id": "4ad0a660", "metadata": { "editable": true }, @@ -122,7 +122,7 @@ }, { "cell_type": "markdown", - "id": "9c659ce0", + "id": "402e3e2c", "metadata": { "editable": true }, @@ -132,7 +132,7 @@ }, { "cell_type": "markdown", - "id": "eb18a204", + "id": "3969e72a", "metadata": { "editable": true }, @@ -153,7 +153,7 @@ }, { "cell_type": "markdown", - "id": "2589c0ee", + "id": "cbaeacff", "metadata": { "editable": true }, @@ -165,7 +165,7 @@ }, { "cell_type": "markdown", - "id": "ef71f105", + "id": "058b8fd1", "metadata": { "editable": true }, @@ -181,7 +181,7 @@ }, { "cell_type": "markdown", - "id": "9c5efaee", + "id": "8afee525", "metadata": { "editable": true }, @@ -193,7 +193,7 @@ }, { "cell_type": "markdown", - "id": "cbf1d43c", + "id": "004446a9", "metadata": { "editable": true }, @@ -209,7 +209,7 @@ }, { "cell_type": "markdown", - "id": "be829d81", + "id": "f090295d", "metadata": { "editable": true }, @@ -222,7 +222,7 @@ }, { "cell_type": "markdown", - "id": "1f193798", + "id": "a14393c7", "metadata": { "editable": true }, @@ -232,7 +232,7 @@ }, { "cell_type": "markdown", - "id": "3d29d961", + "id": "b8a627c3", "metadata": { "editable": true }, @@ -244,7 +244,7 @@ }, { "cell_type": "markdown", - "id": "fc5b2c18", + "id": "73831b21", "metadata": { "editable": true }, @@ -257,7 +257,7 @@ }, { "cell_type": "markdown", - "id": "2a465c1b", + "id": "c42aff41", "metadata": { "editable": true }, @@ -269,7 +269,7 @@ }, { "cell_type": "markdown", - "id": "d69e520a", + "id": "221a3766", "metadata": { "editable": true }, @@ -283,7 +283,7 @@ }, { "cell_type": "markdown", - "id": "2208b590", + "id": "d7bf2e2c", "metadata": { "editable": true }, @@ -298,7 +298,7 @@ }, { "cell_type": "markdown", - "id": "f3d91dca", + "id": "b52f3a4a", "metadata": { "editable": true }, @@ -309,7 +309,7 @@ }, { "cell_type": "markdown", - "id": "0f431dd2", + "id": "bb00c079", "metadata": { "editable": true }, @@ -327,7 +327,7 @@ }, { "cell_type": "markdown", - "id": "155c6cf7", + "id": "024255b1", "metadata": { "editable": true }, @@ -337,7 +337,7 @@ }, { "cell_type": "markdown", - "id": "42993dcc", + "id": "2c417fdd", "metadata": { "editable": true }, @@ -351,7 +351,7 @@ }, { "cell_type": "markdown", - "id": "749d5c23", + "id": "8edaa55e", "metadata": { "editable": true }, @@ -363,7 +363,7 @@ }, { "cell_type": "markdown", - "id": "6e2110dc", + "id": "23da1cfa", "metadata": { "editable": true }, @@ -375,7 +375,7 @@ }, { "cell_type": "markdown", - "id": "d00fdbe1", + "id": "66d5445e", "metadata": { "editable": true }, @@ -385,7 +385,7 @@ }, { "cell_type": "markdown", - "id": "23c37b45", + "id": "fc8027ff", "metadata": { "editable": true }, @@ -397,7 +397,7 @@ }, { "cell_type": "markdown", - "id": "07558d2c", + "id": "351fa52b", "metadata": { "editable": true }, @@ -408,7 +408,7 @@ { "cell_type": "code", "execution_count": 1, - "id": "b99f544d", + "id": "1eac297f", "metadata": { "collapsed": false, "editable": true @@ -458,7 +458,7 @@ }, { "cell_type": "markdown", - "id": "617597ef", + "id": "689c8867", "metadata": { "editable": true }, @@ -468,7 +468,7 @@ }, { "cell_type": "markdown", - "id": "18be0515", + "id": "c13c84f0", "metadata": { "editable": true }, @@ -480,7 +480,7 @@ }, { "cell_type": "markdown", - "id": "cafcff78", + "id": "44d6f4d6", "metadata": { "editable": true }, @@ -494,7 +494,7 @@ }, { "cell_type": "markdown", - "id": "f5c68a26", + "id": "b721f71c", "metadata": { "editable": true }, @@ -512,7 +512,7 @@ }, { "cell_type": "markdown", - "id": "291e3ff7", + "id": "5bf7a516", "metadata": { "editable": true }, @@ -526,7 +526,7 @@ }, { "cell_type": "markdown", - "id": "294080d7", + "id": "fa031db8", "metadata": { "editable": true }, @@ -538,7 +538,7 @@ }, { "cell_type": "markdown", - "id": "9d493962", + "id": "3c0c268c", "metadata": { "editable": true }, @@ -550,7 +550,7 @@ }, { "cell_type": "markdown", - "id": "6ca38798", + "id": "96f430cf", "metadata": { "editable": true }, @@ -562,7 +562,7 @@ }, { "cell_type": "markdown", - "id": "9a779b37", + "id": "cd05ac98", "metadata": { "editable": true }, @@ -572,7 +572,7 @@ }, { "cell_type": "markdown", - "id": "9fc21016", + "id": "bcfe0cef", "metadata": { "editable": true }, @@ -584,7 +584,7 @@ }, { "cell_type": "markdown", - "id": "a146b327", + "id": "d78ce192", "metadata": { "editable": true }, @@ -598,7 +598,7 @@ }, { "cell_type": "markdown", - "id": "0b282f14", + "id": "bab3a068", "metadata": { "editable": true }, @@ -610,7 +610,7 @@ }, { "cell_type": "markdown", - "id": "512e6e16", + "id": "d8e5ddc2", "metadata": { "editable": true }, @@ -623,7 +623,7 @@ }, { "cell_type": "markdown", - "id": "e244a343", + "id": "336072c9", "metadata": { "editable": true }, @@ -641,7 +641,7 @@ }, { "cell_type": "markdown", - "id": "c0d8e272", + "id": "59df87e7", "metadata": { "editable": true }, @@ -654,7 +654,7 @@ }, { "cell_type": "markdown", - "id": "12fb667e", + "id": "9fabf8a8", "metadata": { "editable": true }, @@ -685,7 +685,7 @@ }, { "cell_type": "markdown", - "id": "7d318e93", + "id": "cddc38d1", "metadata": { "editable": true }, @@ -697,7 +697,7 @@ }, { "cell_type": "markdown", - "id": "6ef87b82", + "id": "62d0c741", "metadata": { "editable": true }, @@ -707,7 +707,7 @@ }, { "cell_type": "markdown", - "id": "d11f1bdb", + "id": "5ac68815", "metadata": { "editable": true }, @@ -719,7 +719,7 @@ }, { "cell_type": "markdown", - "id": "2a8abb3d", + "id": "c0ecfbcb", "metadata": { "editable": true }, @@ -734,7 +734,7 @@ }, { "cell_type": "markdown", - "id": "6c14f838", + "id": "fe23b255", "metadata": { "editable": true }, @@ -746,7 +746,7 @@ }, { "cell_type": "markdown", - "id": "62149f95", + "id": "09ebf8ca", "metadata": { "editable": true }, @@ -758,7 +758,7 @@ }, { "cell_type": "markdown", - "id": "f8f51c0a", + "id": "401e693f", "metadata": { "editable": true }, @@ -770,7 +770,7 @@ }, { "cell_type": "markdown", - "id": "9a038bf3", + "id": "6c7d8d10", "metadata": { "editable": true }, @@ -780,7 +780,7 @@ }, { "cell_type": "markdown", - "id": "bdaf914a", + "id": "1bfafdfd", "metadata": { "editable": true }, @@ -792,7 +792,7 @@ }, { "cell_type": "markdown", - "id": "74226a02", + "id": "5596ac72", "metadata": { "editable": true }, @@ -802,7 +802,7 @@ }, { "cell_type": "markdown", - "id": "32215b82", + "id": "f59c48fc", "metadata": { "editable": true }, @@ -814,7 +814,7 @@ }, { "cell_type": "markdown", - "id": "688f69a2", + "id": "d7e68ca7", "metadata": { "editable": true }, @@ -824,7 +824,7 @@ }, { "cell_type": "markdown", - "id": "082e6778", + "id": "a6f6eebb", "metadata": { "editable": true }, @@ -836,7 +836,7 @@ }, { "cell_type": "markdown", - "id": "ba9c8bd4", + "id": "58f2e885", "metadata": { "editable": true }, @@ -849,7 +849,7 @@ }, { "cell_type": "markdown", - "id": "f9540188", + "id": "ec46859a", "metadata": { "editable": true }, @@ -861,7 +861,7 @@ }, { "cell_type": "markdown", - "id": "0507fa0a", + "id": "b65e2164", "metadata": { "editable": true }, @@ -871,7 +871,7 @@ }, { "cell_type": "markdown", - "id": "74ee9f1a", + "id": "f7874346", "metadata": { "editable": true }, @@ -883,7 +883,7 @@ }, { "cell_type": "markdown", - "id": "0cf42992", + "id": "63d8f111", "metadata": { "editable": true }, @@ -893,7 +893,7 @@ }, { "cell_type": "markdown", - "id": "e3554ca3", + "id": "ed985f65", "metadata": { "editable": true }, @@ -905,7 +905,7 @@ }, { "cell_type": "markdown", - "id": "728ba052", + "id": "4e2f6eb1", "metadata": { "editable": true }, @@ -915,7 +915,7 @@ }, { "cell_type": "markdown", - "id": "60276744", + "id": "1a073de2", "metadata": { "editable": true }, @@ -928,7 +928,7 @@ }, { "cell_type": "markdown", - "id": "c5b45c52", + "id": "cef07ac6", "metadata": { "editable": true }, @@ -940,7 +940,7 @@ }, { "cell_type": "markdown", - "id": "a005fdfe", + "id": "7fb0d04c", "metadata": { "editable": true }, @@ -958,7 +958,7 @@ }, { "cell_type": "markdown", - "id": "12c13ae4", + "id": "4404131d", "metadata": { "editable": true }, @@ -970,7 +970,7 @@ }, { "cell_type": "markdown", - "id": "68e56af7", + "id": "40d36fc7", "metadata": { "editable": true }, @@ -982,7 +982,7 @@ }, { "cell_type": "markdown", - "id": "99071464", + "id": "3c6be327", "metadata": { "editable": true }, @@ -994,7 +994,7 @@ }, { "cell_type": "markdown", - "id": "08b3b2e3", + "id": "33d98314", "metadata": { "editable": true }, @@ -1006,7 +1006,7 @@ }, { "cell_type": "markdown", - "id": "57d2b4a7", + "id": "182fc903", "metadata": { "editable": true }, @@ -1028,7 +1028,7 @@ }, { "cell_type": "markdown", - "id": "4ce5e39f", + "id": "ad6f2431", "metadata": { "editable": true }, @@ -1040,7 +1040,7 @@ }, { "cell_type": "markdown", - "id": "4b87bcf7", + "id": "0a7d17bf", "metadata": { "editable": true }, @@ -1050,7 +1050,7 @@ }, { "cell_type": "markdown", - "id": "29e93603", + "id": "1a31e460", "metadata": { "editable": true }, @@ -1062,7 +1062,7 @@ }, { "cell_type": "markdown", - "id": "524d4e2f", + "id": "f605d561", "metadata": { "editable": true }, @@ -1073,7 +1073,7 @@ }, { "cell_type": "markdown", - "id": "d129841d", + "id": "62d83d95", "metadata": { "editable": true }, @@ -1086,7 +1086,7 @@ }, { "cell_type": "markdown", - "id": "f68c25df", + "id": "800314ec", "metadata": { "editable": true }, @@ -1096,7 +1096,7 @@ }, { "cell_type": "markdown", - "id": "f19452aa", + "id": "418db0a7", "metadata": { "editable": true }, @@ -1109,7 +1109,7 @@ }, { "cell_type": "markdown", - "id": "c75f2dc5", + "id": "de9eb757", "metadata": { "editable": true }, @@ -1119,7 +1119,7 @@ }, { "cell_type": "markdown", - "id": "c7b760bf", + "id": "1bc554b4", "metadata": { "editable": true }, @@ -1131,7 +1131,7 @@ }, { "cell_type": "markdown", - "id": "6fc3e294", + "id": "9f92c026", "metadata": { "editable": true }, @@ -1144,7 +1144,7 @@ }, { "cell_type": "markdown", - "id": "c67cc7c9", + "id": "a79037de", "metadata": { "editable": true }, @@ -1156,7 +1156,7 @@ }, { "cell_type": "markdown", - "id": "3988d52a", + "id": "59153bee", "metadata": { "editable": true }, @@ -1166,7 +1166,7 @@ }, { "cell_type": "markdown", - "id": "9b3e4562", + "id": "57b60d32", "metadata": { "editable": true }, @@ -1179,7 +1179,7 @@ }, { "cell_type": "markdown", - "id": "88138c21", + "id": "9ee3d242", "metadata": { "editable": true }, @@ -1195,7 +1195,7 @@ }, { "cell_type": "markdown", - "id": "36ecffde", + "id": "47897048", "metadata": { "editable": true }, @@ -1207,7 +1207,7 @@ }, { "cell_type": "markdown", - "id": "be4e015c", + "id": "9ad853ba", "metadata": { "editable": true }, @@ -1224,7 +1224,7 @@ }, { "cell_type": "markdown", - "id": "93522854", + "id": "e44ea56e", "metadata": { "editable": true }, @@ -1242,7 +1242,7 @@ }, { "cell_type": "markdown", - "id": "e667c0aa", + "id": "c2c32ffc", "metadata": { "editable": true }, @@ -1260,7 +1260,7 @@ }, { "cell_type": "markdown", - "id": "44f76dd9", + "id": "87496c64", "metadata": { "editable": true }, @@ -1270,7 +1270,7 @@ }, { "cell_type": "markdown", - "id": "1041b2af", + "id": "841b5856", "metadata": { "editable": true }, @@ -1283,7 +1283,7 @@ }, { "cell_type": "markdown", - "id": "e42984a2", + "id": "50ede176", "metadata": { "editable": true }, @@ -1293,7 +1293,7 @@ }, { "cell_type": "markdown", - "id": "a46f958e", + "id": "64687e2f", "metadata": { "editable": true }, @@ -1305,7 +1305,7 @@ }, { "cell_type": "markdown", - "id": "37763243", + "id": "ab77c972", "metadata": { "editable": true }, @@ -1319,7 +1319,7 @@ { "cell_type": "code", "execution_count": 2, - "id": "aedc8b8c", + "id": "1ffa428b", "metadata": { "collapsed": false, "editable": true @@ -1354,7 +1354,7 @@ }, { "cell_type": "markdown", - "id": "4ad05ea1", + "id": "7a9a1bee", "metadata": { "editable": true }, @@ -1364,7 +1364,7 @@ }, { "cell_type": "markdown", - "id": "3f643cb0", + "id": "825a940b", "metadata": { "editable": true }, @@ -1384,7 +1384,7 @@ }, { "cell_type": "markdown", - "id": "bedb81e0", + "id": "dbb414d5", "metadata": { "editable": true }, @@ -1394,7 +1394,7 @@ }, { "cell_type": "markdown", - "id": "8a8f5bdd", + "id": "a617dca7", "metadata": { "editable": true }, @@ -1406,7 +1406,7 @@ }, { "cell_type": "markdown", - "id": "cc3e8c15", + "id": "7d3f260d", "metadata": { "editable": true }, @@ -1416,7 +1416,7 @@ }, { "cell_type": "markdown", - "id": "cf107fa4", + "id": "996e13c9", "metadata": { "editable": true }, @@ -1428,7 +1428,7 @@ }, { "cell_type": "markdown", - "id": "c3d6eefd", + "id": "5a9b492d", "metadata": { "editable": true }, @@ -1446,7 +1446,7 @@ }, { "cell_type": "markdown", - "id": "11084473", + "id": "43f5dd21", "metadata": { "editable": true }, @@ -1458,7 +1458,7 @@ }, { "cell_type": "markdown", - "id": "ec3bb41f", + "id": "9ede60b9", "metadata": { "editable": true }, @@ -1482,7 +1482,7 @@ }, { "cell_type": "markdown", - "id": "9654f276", + "id": "e65f6a16", "metadata": { "editable": true }, @@ -1494,7 +1494,7 @@ }, { "cell_type": "markdown", - "id": "9c339ccc", + "id": "4625dd90", "metadata": { "editable": true }, @@ -1504,7 +1504,7 @@ }, { "cell_type": "markdown", - "id": "9d523f50", + "id": "9fe0ba5b", "metadata": { "editable": true }, @@ -1516,7 +1516,7 @@ }, { "cell_type": "markdown", - "id": "2793f264", + "id": "02018277", "metadata": { "editable": true }, @@ -1530,7 +1530,7 @@ }, { "cell_type": "markdown", - "id": "b0432b14", + "id": "b70c018b", "metadata": { "editable": true }, @@ -1548,7 +1548,7 @@ }, { "cell_type": "markdown", - "id": "c1981b2d", + "id": "b5e3445e", "metadata": { "editable": true }, @@ -1559,7 +1559,7 @@ }, { "cell_type": "markdown", - "id": "679d42bc", + "id": "f21c34e2", "metadata": { "editable": true }, @@ -1571,7 +1571,7 @@ }, { "cell_type": "markdown", - "id": "2bb94e0d", + "id": "b6e821b1", "metadata": { "editable": true }, @@ -1581,7 +1581,7 @@ }, { "cell_type": "markdown", - "id": "0611f240", + "id": "1b841b98", "metadata": { "editable": true }, @@ -1599,7 +1599,7 @@ }, { "cell_type": "markdown", - "id": "624f9965", + "id": "3eaf58b1", "metadata": { "editable": true }, @@ -1609,7 +1609,7 @@ }, { "cell_type": "markdown", - "id": "c0d8ff9e", + "id": "8111dbd1", "metadata": { "editable": true }, @@ -1627,7 +1627,7 @@ }, { "cell_type": "markdown", - "id": "d058f856", + "id": "769561de", "metadata": { "editable": true }, @@ -1641,7 +1641,7 @@ }, { "cell_type": "markdown", - "id": "76e956db", + "id": "707a2251", "metadata": { "editable": true }, @@ -1660,7 +1660,7 @@ }, { "cell_type": "markdown", - "id": "bd4e286e", + "id": "40849ecf", "metadata": { "editable": true }, @@ -1673,7 +1673,7 @@ }, { "cell_type": "markdown", - "id": "877355c7", + "id": "b3afcf6c", "metadata": { "editable": true }, @@ -1685,7 +1685,7 @@ }, { "cell_type": "markdown", - "id": "abc25d1f", + "id": "e6a97eec", "metadata": { "editable": true }, @@ -1697,7 +1697,7 @@ }, { "cell_type": "markdown", - "id": "29a4c67d", + "id": "f47332cb", "metadata": { "editable": true }, @@ -1707,7 +1707,7 @@ }, { "cell_type": "markdown", - "id": "46b6a9e4", + "id": "b397eb5b", "metadata": { "editable": true }, @@ -1719,7 +1719,7 @@ }, { "cell_type": "markdown", - "id": "350ee280", + "id": "2336c2fc", "metadata": { "editable": true }, @@ -1729,7 +1729,7 @@ }, { "cell_type": "markdown", - "id": "321d0a3d", + "id": "1e5e68d4", "metadata": { "editable": true }, @@ -1741,7 +1741,7 @@ }, { "cell_type": "markdown", - "id": "a6cec686", + "id": "d230496b", "metadata": { "editable": true }, @@ -1769,7 +1769,7 @@ }, { "cell_type": "markdown", - "id": "64525de2", + "id": "3a344727", "metadata": { "editable": true }, @@ -1781,7 +1781,7 @@ }, { "cell_type": "markdown", - "id": "a8e20afa", + "id": "1146f588", "metadata": { "editable": true }, @@ -1792,7 +1792,7 @@ }, { "cell_type": "markdown", - "id": "51ec2352", + "id": "c7e8ee38", "metadata": { "editable": true }, @@ -1805,7 +1805,7 @@ }, { "cell_type": "markdown", - "id": "b445a98b", + "id": "0da99fb8", "metadata": { "editable": true }, @@ -1818,7 +1818,7 @@ }, { "cell_type": "markdown", - "id": "fd16dccb", + "id": "59f94571", "metadata": { "editable": true }, @@ -1836,7 +1836,7 @@ }, { "cell_type": "markdown", - "id": "03919ae2", + "id": "5711d982", "metadata": { "editable": true }, @@ -1850,7 +1850,7 @@ }, { "cell_type": "markdown", - "id": "f4edb79b", + "id": "f43f4e99", "metadata": { "editable": true }, @@ -1869,7 +1869,7 @@ }, { "cell_type": "markdown", - "id": "209c3c0a", + "id": "f3e40026", "metadata": { "editable": true }, @@ -1882,7 +1882,7 @@ }, { "cell_type": "markdown", - "id": "d0c7027f", + "id": "5e80ce67", "metadata": { "editable": true }, @@ -1900,7 +1900,7 @@ }, { "cell_type": "markdown", - "id": "8183355a", + "id": "77e3a4ed", "metadata": { "editable": true }, @@ -1918,7 +1918,7 @@ }, { "cell_type": "markdown", - "id": "04f300ab", + "id": "5bc1a688", "metadata": { "editable": true }, @@ -1930,7 +1930,7 @@ }, { "cell_type": "markdown", - "id": "d9bdb1eb", + "id": "a17a0e31", "metadata": { "editable": true }, @@ -1948,7 +1948,7 @@ { "cell_type": "code", "execution_count": 3, - "id": "93013c0b", + "id": "3c348b11", "metadata": { "collapsed": false, "editable": true @@ -1996,7 +1996,7 @@ }, { "cell_type": "markdown", - "id": "65e8cd76", + "id": "ecaeea61", "metadata": { "editable": true }, @@ -2034,7 +2034,7 @@ }, { "cell_type": "markdown", - "id": "4daecdd3", + "id": "8e731bf2", "metadata": { "editable": true }, @@ -2046,7 +2046,7 @@ }, { "cell_type": "markdown", - "id": "fd1c5e47", + "id": "d0341530", "metadata": { "editable": true }, @@ -2056,7 +2056,7 @@ }, { "cell_type": "markdown", - "id": "d8c06853", + "id": "4d3b7ea7", "metadata": { "editable": true }, @@ -2068,7 +2068,7 @@ }, { "cell_type": "markdown", - "id": "0d971106", + "id": "4b36e97b", "metadata": { "editable": true }, @@ -2090,7 +2090,7 @@ }, { "cell_type": "markdown", - "id": "d12a26d7", + "id": "e35bec75", "metadata": { "editable": true }, @@ -2102,7 +2102,7 @@ }, { "cell_type": "markdown", - "id": "2ee32cc9", + "id": "e1793cdf", "metadata": { "editable": true }, @@ -2115,7 +2115,7 @@ }, { "cell_type": "markdown", - "id": "441be869", + "id": "d6fba025", "metadata": { "editable": true }, @@ -2127,7 +2127,7 @@ }, { "cell_type": "markdown", - "id": "c0803129", + "id": "6ffd5658", "metadata": { "editable": true }, @@ -2147,7 +2147,7 @@ }, { "cell_type": "markdown", - "id": "43aef6df", + "id": "c5da7f78", "metadata": { "editable": true }, @@ -2159,7 +2159,7 @@ }, { "cell_type": "markdown", - "id": "1c7bddf4", + "id": "2c25b836", "metadata": { "editable": true }, @@ -2174,7 +2174,7 @@ }, { "cell_type": "markdown", - "id": "fed2dc23", + "id": "e4a135e8", "metadata": { "editable": true }, @@ -2192,7 +2192,7 @@ }, { "cell_type": "markdown", - "id": "dc9d8537", + "id": "b37cd0b7", "metadata": { "editable": true }, @@ -2202,7 +2202,7 @@ }, { "cell_type": "markdown", - "id": "aa9465e9", + "id": "da41fe10", "metadata": { "editable": true }, @@ -2220,7 +2220,7 @@ }, { "cell_type": "markdown", - "id": "def99a7e", + "id": "9a489418", "metadata": { "editable": true }, @@ -2233,7 +2233,7 @@ }, { "cell_type": "markdown", - "id": "61c8f9ef", + "id": "4294433a", "metadata": { "editable": true }, @@ -2245,7 +2245,7 @@ }, { "cell_type": "markdown", - "id": "c59c90b8", + "id": "9bfc4598", "metadata": { "editable": true }, @@ -2266,7 +2266,7 @@ }, { "cell_type": "markdown", - "id": "06587f00", + "id": "a4a8c15a", "metadata": { "editable": true }, @@ -2278,7 +2278,7 @@ }, { "cell_type": "markdown", - "id": "c3255e8e", + "id": "5ac68b4d", "metadata": { "editable": true }, @@ -2291,7 +2291,7 @@ }, { "cell_type": "markdown", - "id": "8e3ec364", + "id": "d47724cd", "metadata": { "editable": true }, @@ -2303,7 +2303,7 @@ }, { "cell_type": "markdown", - "id": "4b7c18d5", + "id": "f3ca32a6", "metadata": { "editable": true }, @@ -2313,7 +2313,7 @@ }, { "cell_type": "markdown", - "id": "4e58c88d", + "id": "26cb7a8c", "metadata": { "editable": true }, @@ -2325,7 +2325,7 @@ }, { "cell_type": "markdown", - "id": "1fecdb9a", + "id": "8c88d4ec", "metadata": { "editable": true }, @@ -2335,7 +2335,7 @@ }, { "cell_type": "markdown", - "id": "aa126ce7", + "id": "7e2a9bfc", "metadata": { "editable": true }, @@ -2347,7 +2347,7 @@ }, { "cell_type": "markdown", - "id": "20cab2fa", + "id": "6f25911e", "metadata": { "editable": true }, @@ -2360,7 +2360,7 @@ }, { "cell_type": "markdown", - "id": "904a6320", + "id": "e3363165", "metadata": { "editable": true }, @@ -2378,7 +2378,7 @@ }, { "cell_type": "markdown", - "id": "36c6f816", + "id": "caa9b226", "metadata": { "editable": true }, @@ -2392,7 +2392,7 @@ }, { "cell_type": "markdown", - "id": "319af13b", + "id": "807586a1", "metadata": { "editable": true }, @@ -2410,7 +2410,7 @@ }, { "cell_type": "markdown", - "id": "0c426357", + "id": "22f852c7", "metadata": { "editable": true }, @@ -2420,7 +2420,7 @@ }, { "cell_type": "markdown", - "id": "52547452", + "id": "0bb72673", "metadata": { "editable": true }, @@ -2438,7 +2438,7 @@ }, { "cell_type": "markdown", - "id": "86bf958f", + "id": "5e06548e", "metadata": { "editable": true }, @@ -2448,7 +2448,7 @@ }, { "cell_type": "markdown", - "id": "d2015392", + "id": "1227c261", "metadata": { "editable": true }, @@ -2466,7 +2466,7 @@ }, { "cell_type": "markdown", - "id": "83495015", + "id": "c1b82a5c", "metadata": { "editable": true }, @@ -2503,7 +2503,7 @@ }, { "cell_type": "markdown", - "id": "268aac30", + "id": "dacf04eb", "metadata": { "editable": true }, @@ -2515,7 +2515,7 @@ }, { "cell_type": "markdown", - "id": "25bd4ea3", + "id": "44216fdd", "metadata": { "editable": true }, @@ -2525,7 +2525,7 @@ }, { "cell_type": "markdown", - "id": "29e8d709", + "id": "bb4297ee", "metadata": { "editable": true }, @@ -2537,7 +2537,7 @@ }, { "cell_type": "markdown", - "id": "90a07c88", + "id": "bed1a0da", "metadata": { "editable": true }, @@ -2571,7 +2571,7 @@ { "cell_type": "code", "execution_count": 4, - "id": "3c296708", + "id": "dd0f1669", "metadata": { "collapsed": false, "editable": true @@ -2604,7 +2604,7 @@ }, { "cell_type": "markdown", - "id": "7a028dda", + "id": "3b439592", "metadata": { "editable": true }, @@ -2618,7 +2618,7 @@ }, { "cell_type": "markdown", - "id": "bc325e1a", + "id": "65c84eb8", "metadata": { "editable": true }, @@ -2631,7 +2631,7 @@ }, { "cell_type": "markdown", - "id": "a7738ea8", + "id": "5bdabd24", "metadata": { "editable": true }, @@ -2642,7 +2642,7 @@ }, { "cell_type": "markdown", - "id": "b647158e", + "id": "9965ab90", "metadata": { "editable": true }, @@ -2654,7 +2654,7 @@ }, { "cell_type": "markdown", - "id": "273afa83", + "id": "23e0c2ff", "metadata": { "editable": true }, @@ -2667,7 +2667,7 @@ }, { "cell_type": "markdown", - "id": "70391792", + "id": "f740a54c", "metadata": { "editable": true }, @@ -2680,7 +2680,7 @@ { "cell_type": "code", "execution_count": 5, - "id": "2b08e1a4", + "id": "4acc6d55", "metadata": { "collapsed": false, "editable": true @@ -2720,7 +2720,7 @@ }, { "cell_type": "markdown", - "id": "5848dd32", + "id": "2b9b8e55", "metadata": { "editable": true },