diff --git a/doc/src/LectureNotes/_build/.doctrees/chapter2.doctree b/doc/src/LectureNotes/_build/.doctrees/chapter2.doctree index 3610d2792..60b977c26 100644 Binary files a/doc/src/LectureNotes/_build/.doctrees/chapter2.doctree and b/doc/src/LectureNotes/_build/.doctrees/chapter2.doctree differ diff --git a/doc/src/LectureNotes/_build/.doctrees/environment.pickle b/doc/src/LectureNotes/_build/.doctrees/environment.pickle index da58c25c9..25b1690a3 100644 Binary files a/doc/src/LectureNotes/_build/.doctrees/environment.pickle and b/doc/src/LectureNotes/_build/.doctrees/environment.pickle differ diff --git a/doc/src/LectureNotes/_build/html/_images/chapter2_178_0.png b/doc/src/LectureNotes/_build/html/_images/chapter2_178_0.png index f26cbb55e..c5a3d04f2 100644 Binary files a/doc/src/LectureNotes/_build/html/_images/chapter2_178_0.png and b/doc/src/LectureNotes/_build/html/_images/chapter2_178_0.png differ diff --git a/doc/src/LectureNotes/_build/html/_images/chapter2_184_1.png b/doc/src/LectureNotes/_build/html/_images/chapter2_184_1.png index 23b842d84..babfb1319 100644 Binary files a/doc/src/LectureNotes/_build/html/_images/chapter2_184_1.png and b/doc/src/LectureNotes/_build/html/_images/chapter2_184_1.png differ diff --git a/doc/src/LectureNotes/_build/html/_sources/chapter2.ipynb b/doc/src/LectureNotes/_build/html/_sources/chapter2.ipynb index 010c93ced..be58c6c16 100644 --- a/doc/src/LectureNotes/_build/html/_sources/chapter2.ipynb +++ b/doc/src/LectureNotes/_build/html/_sources/chapter2.ipynb @@ -1669,7 +1669,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Random Numbers\n", + "## Random Numbers\n", "\n", "Uniform deviates are just random numbers that lie within a specified range\n", "(typically 0 to 1), with any one number in the range just as likely as any other. They\n", @@ -1686,7 +1686,7 @@ "\n", "\n", "\n", - "# Random Numbers, better name: pseudo random numbers\n", + "## Random Numbers, better name: pseudo random numbers\n", "\n", "A disclaimer is however appropriate. It should be fairly obvious that \n", "something as deterministic as a computer cannot generate purely random numbers.\n", @@ -1706,7 +1706,7 @@ "\n", "\n", "\n", - "# Random number generator RNG\n", + "## Random number generator RNG\n", " The most common random number generators are based on so-called\n", "Linear congruential relations of the type" ] @@ -1750,7 +1750,7 @@ "\n", "\n", "\n", - "# Random number generator RNG and periodic outputs\n", + "## Random number generator RNG and periodic outputs\n", "\n", "The problem with such generators is that their outputs are periodic;\n", "they \n", @@ -1798,7 +1798,7 @@ "\n", "\n", "\n", - "# Random number generator RNG and its period\n", + "## Random number generator RNG and its period\n", "Typical periods for the random generators provided in the program library \n", "are of the order of $\\sim 10^9$ or larger. Other random number generators which have\n", "become increasingly popular are so-called shift-register generators.\n", @@ -1822,7 +1822,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Random number generator RNG, other examples\n", + "## Random number generator RNG, other examples\n", "Such a generator again produces a sequence of pseudorandom numbers\n", "but this time with a period much larger than $M$.\n", "It is also possible to construct more elaborate algorithms by including\n", @@ -1877,7 +1877,7 @@ "\n", "\n", "\n", - "# Random number generator RNG, other examples\n", + "## Random number generator RNG, other examples\n", "Instead of using modular addition, we could use the bitwise\n", "exclusive-OR ($\\oplus$) operation so that" ] @@ -1910,7 +1910,7 @@ "\n", "\n", "\n", - "# Random number generator RNG, RAN0\n", + "## Random number generator RNG, RAN0\n", "\n", "We show here how the linear congruential algorithm can be implemented, namely" ] @@ -1987,7 +1987,7 @@ "\n", "\n", "\n", - "# Random number generator RNG, RAN0\n", + "## Random number generator RNG, RAN0\n", "\n", "To see how this works we note first that" ] @@ -2019,7 +2019,7 @@ "\n", "\n", "\n", - "# Random number generator RNG, RAN0\n", + "## Random number generator RNG, RAN0\n", "We can now rewrite Eq. ([14](#eq:rntrick1)) as" ] }, @@ -2087,7 +2087,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Random number generator RNG, RAN0\n", + "## Random number generator RNG, RAN0\n", "The term $[N_{i-1}/q]r$ is always smaller or equal $N_{i-1}(r/q)$ and with $r < q$ we obtain always a \n", "number smaller than $N_{i-1}$, which is smaller than $M$. \n", "And since the number $N_{i-1}\\mathrm{MOD} (q)$ is between zero and $q-1$ then\n", @@ -2105,7 +2105,7 @@ "\n", "\n", "\n", - "# Random number generator RNG, RAN0 code" + "## Random number generator RNG, RAN0 code" ] }, { @@ -2457,7 +2457,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Which RNG should I use?\n", + "## Which RNG should I use?\n", "* C++ has a class called **random**. The [random class](http://www.cplusplus.com/reference/random/) contains a large selection of RNGs and is highly recommended. Some of these RNGs have very large periods making it thereby very safe to use these RNGs in case one is performing large calculations. In particular, the [Mersenne twister random number engine](http://www.cplusplus.com/reference/random/mersenne_twister_engine/) has a period of $2^{19937}$. \n", "\n", "* Add RNGs in Python\n", diff --git a/doc/src/LectureNotes/_build/html/chapter1.html b/doc/src/LectureNotes/_build/html/chapter1.html index de400ade1..46441256c 100644 --- a/doc/src/LectureNotes/_build/html/chapter1.html +++ b/doc/src/LectureNotes/_build/html/chapter1.html @@ -89,99 +89,34 @@ 1. Elements of Probability Theory and Statistical Data Analysis -
  • - - 2. Random Numbers - -
    • -
  • - - 3. Random Numbers, better name: pseudo random numbers - -
    • -
  • - - 4. Random number generator RNG - -
    • -
  • - - 5. Random number generator RNG and periodic outputs - -
    • -
  • - - 6. Random number generator RNG and its period - -
    • -
  • - - 7. Random number generator RNG, other examples - -
    • -
  • - - 8. Random number generator RNG, other examples - -
    • -
  • - - 9. Random number generator RNG, RAN0 - -
    • -
  • - - 10. Random number generator RNG, RAN0 - -
    • -
  • - - 11. Random number generator RNG, RAN0 - -
    • -
  • - - 12. Random number generator RNG, RAN0 - -
    • -
  • - - 13. Random number generator RNG, RAN0 code - -
    • -
  • - - 14. Which RNG should I use? - -
  • - 15. Getting started, our first data and Machine Learning encounters + 2. Getting started, our first data and Machine Learning encounters
  • - 16. Linear Regression and more Advanced Regression Analysis + 3. Linear Regression and more Advanced Regression Analysis
  • - 17. Logistic Regression + 4. Logistic Regression
  • - 18. Neural networks, from the simple perceptron to deep learning + 5. Neural networks, from the simple perceptron to deep learning
  • - 19. Support Vector Machines, overarching aims + 6. Support Vector Machines, overarching aims
  • - 20. Dimensionality Reduction + 7. Dimensionality Reduction
    • diff --git a/doc/src/LectureNotes/_build/html/chapter2.html b/doc/src/LectureNotes/_build/html/chapter2.html index c699338e4..ac06b5ffb 100644 --- a/doc/src/LectureNotes/_build/html/chapter2.html +++ b/doc/src/LectureNotes/_build/html/chapter2.html @@ -38,7 +38,7 @@ - + @@ -90,99 +90,34 @@ 1. Elements of Probability Theory and Statistical Data Analysis -
  • - - 2. Random Numbers - -
    • -
  • - - 3. Random Numbers, better name: pseudo random numbers - -
    • -
  • - - 4. Random number generator RNG - -
    • -
  • - - 5. Random number generator RNG and periodic outputs - -
    • -
  • - - 6. Random number generator RNG and its period - -
    • -
  • - - 7. Random number generator RNG, other examples - -
    • -
  • - - 8. Random number generator RNG, other examples - -
    • -
  • - - 9. Random number generator RNG, RAN0 - -
    • -
  • - - 10. Random number generator RNG, RAN0 - -
    • -
  • - - 11. Random number generator RNG, RAN0 - -
    • -
  • - - 12. Random number generator RNG, RAN0 - -
    • -
  • - - 13. Random number generator RNG, RAN0 code - -
    • -
  • - - 14. Which RNG should I use? - -
  • - 15. Getting started, our first data and Machine Learning encounters + 2. Getting started, our first data and Machine Learning encounters
  • - 16. Linear Regression and more Advanced Regression Analysis + 3. Linear Regression and more Advanced Regression Analysis
  • - 17. Logistic Regression + 4. Logistic Regression
  • - 18. Neural networks, from the simple perceptron to deep learning + 5. Neural networks, from the simple perceptron to deep learning
  • - 19. Support Vector Machines, overarching aims + 6. Support Vector Machines, overarching aims
  • - 20. Dimensionality Reduction + 7. Dimensionality Reduction
    • @@ -455,6 +390,176 @@ 1.40. Code to compute the Covariance matrix and the Covariance +
  • + + 1.41. Random Numbers + +
    • +
  • + + 1.42. Random Numbers, better name: pseudo random numbers + +
    • +
  • + + 1.43. Random number generator RNG + +
    • +
  • + + 1.44. Random number generator RNG and periodic outputs + +
    • +
  • + + 1.45. Random number generator RNG and its period + +
    • +
  • + + 1.46. Random number generator RNG, other examples + +
    • +
  • + + 1.47. Random number generator RNG, other examples + +
    • +
  • + + 1.48. Random number generator RNG, RAN0 + +
    • +
  • + + 1.49. Random number generator RNG, RAN0 + +
    • +
  • + + 1.50. Random number generator RNG, RAN0 + +
    • +
  • + + 1.51. Random number generator RNG, RAN0 + +
    • +
  • + + 1.52. Random number generator RNG, RAN0 code + +
    • +
  • + + 1.53. Properties of Selected Random Number Generators + +
    • +
  • + + 1.54. Properties of Selected Random Number Generators + +
    • +
  • + + 1.55. Properties of Selected Random Number Generators + +
    • +
  • + + 1.56. Simple demonstration of RNGs using python + +
    • +
  • + + 1.57. Properties of Selected Random Number Generators + +
    • +
  • + + 1.58. Autocorrelation function + +
    • +
  • + + 1.59. Correlation function and which random number generators should I use + +
    • +
  • + + 1.60. Which RNG should I use? + +
    • +
  • + + 1.61. How to use the Mersenne generator + +
    • +
  • + + 1.62. Why blocking? + +
    • +
  • + + 1.63. Why blocking? + +
    • +
  • + + 1.64. Code to demonstrate the calculation of the autocorrelation function + +
    • +
  • + + 1.65. What is blocking? + +
    • +
  • + + 1.66. What is blocking? + +
    • +
  • + + 1.67. What is blocking? + +
    • +
  • + + 1.68. Implementation + +
    • +
  • + + 1.69. Actual implementation with code, main function + +
    • +
  • + + 1.70. The Bootstrap method + +
    • +
  • + + 1.71. Bootstrapping + +
    • +
  • + + 1.72. Bootstrapping, recipe + +
    • +
  • + + 1.73. Bootstrapping, code + +
    • +
  • + + 1.74. Jackknife, code + +
    • @@ -1011,27 +1116,37 @@ uncorrelated.

    -
    9.106279149747735
-[[  1.87336856   1.77787047   1.44771535   6.44982327  11.43684153
-    5.83694909   1.84228806   7.37557482  -2.58314202   6.48643762]
- [  1.77787047   1.68724056   1.37391564   6.12103277  10.85382947
-    5.53940088   1.74837435   6.9995926   -2.45146205   6.15578065]
- [  1.44771535   1.37391564   1.11877597   4.98434123   8.83824539
-    4.51071987   1.42369673   5.69975023  -1.99621924   5.01263633]
- [  6.44982327   6.12103277   4.98434123  22.2061057   39.37591798
-   20.09604032   6.34281617  25.39337703  -8.89350334  22.33216531]
- [ 11.43684153  10.85382947   8.83824539  39.37591798  69.82146882
-   35.63434516  11.2470963   45.02759486 -15.76998069  39.59944717]
- [  5.83694909   5.53940088   4.51071987  20.09604032  35.63434516
-   18.18647726   5.74011     22.9804512   -8.04842615  20.21012151]
- [  1.84228806   1.74837435   1.42369673   6.34281617  11.2470963
-    5.74011      1.8117232    7.25320885  -2.54028588   6.37882306]
- [  7.37557482   6.9995926    5.69975023  25.39337703  45.02759486
-   22.9804512    7.25320885  29.03812156 -10.16999948  25.53753014]
- [ -2.58314202  -2.45146205  -1.99621924  -8.89350334 -15.76998069
-   -8.04842615  -2.54028588 -10.16999948   3.56183127  -8.94398998]
- [  6.48643762   6.15578065   5.01263633  22.33216531  39.59944717
-   20.21012151   6.37882306  25.53753014  -8.94398998  22.45894054]]
+
    2.56416115970942
+[[ 5.90838144e+00  6.98010863e+00 -1.93947492e-01  1.33851317e+00
+   8.51355225e+00  8.12403403e+00  4.82725395e+00  7.54223480e+00
+   9.38189254e+00  5.58473257e+00]
+ [ 6.98010863e+00  8.24623749e+00 -2.29127820e-01  1.58130740e+00
+   1.00578340e+01  9.59766065e+00  5.70287435e+00  8.91032829e+00
+   1.10836834e+01  6.59775278e+00]
+ [-1.93947492e-01 -2.29127820e-01  6.36648631e-03 -4.39377982e-02
+  -2.79464371e-01 -2.66678115e-01 -1.58458590e-01 -2.47580075e-01
+  -3.07968357e-01 -1.83323451e-01]
+ [ 1.33851317e+00  1.58130740e+00 -4.39377982e-02  3.03233215e-01
+   1.92870110e+00  1.84045777e+00  1.09358934e+00  1.70865417e+00
+   2.12541909e+00  1.26519220e+00]
+ [ 8.51355225e+00  1.00578340e+01 -2.79464371e-01  1.92870110e+00
+   1.22674158e+01  1.17061481e+01  6.95572537e+00  1.08678173e+01
+   1.35186316e+01  8.04719753e+00]
+ [ 8.12403403e+00  9.59766065e+00 -2.66678115e-01  1.84045777e+00
+   1.17061481e+01  1.11705599e+01  6.63748198e+00  1.03705850e+01
+   1.29001174e+01  7.67901631e+00]
+ [ 4.82725395e+00  5.70287435e+00 -1.58458590e-01  1.09358934e+00
+   6.95572537e+00  6.63748198e+00  3.94395333e+00  6.16214154e+00
+   7.66517501e+00  4.56282700e+00]
+ [ 7.54223480e+00  8.91032829e+00 -2.47580075e-01  1.70865417e+00
+   1.08678173e+01  1.03705850e+01  6.16214154e+00  9.62790003e+00
+   1.19762810e+01  7.12908684e+00]
+ [ 9.38189254e+00  1.10836834e+01 -3.07968357e-01  2.12541909e+00
+   1.35186316e+01  1.29001174e+01  7.66517501e+00  1.19762810e+01
+   1.48974653e+01  8.86797194e+00]
+ [ 5.58473257e+00  6.59775278e+00 -1.83323451e-01  1.26519220e+00
+   8.04719753e+00  7.67901631e+00  4.56282700e+00  7.12908684e+00
+   8.86797194e+00  5.27881252e+00]]
    @@ -1339,23 +1454,22 @@ more practically oriented methods like the blocking technique.

    -
    0.06894012083823547
-4.139781119573823
-0.04321525078901231
-1.0006918520539008 10.50426403458282 18.88724031810869
-3.0717579594084814 3.4645963550106305 10.40024628605229
-[[ 1.00069185  3.07175796  3.46459636]
- [ 3.07175796 10.50426403 10.40024629]
- [ 3.46459636 10.40024629 18.88724032]]
-[26.72833966  0.07676079  3.58709575]
+
    -0.06895120441963641
+3.74382855355186
+-0.2186383556244991
+0.8186111112149481 8.041491608761895 5.824398367359107
+2.4292738814902837 1.6663000899456695 4.915994880963659
+[[0.81861111 2.42927388 1.66630009]
+ [2.42927388 8.04149161 4.91599488]
+ [1.66630009 4.91599488 5.82439837]]
+[12.69915666  0.06805927  1.91728516]
    -
    -

    2. Random Numbers

    +

    1.41. Random Numbers

    Uniform deviates are just random numbers that lie within a specified range (typically 0 to 1), with any one number in the range just as likely as any other. They are, in other words, what you probably think random numbers are. However, @@ -1368,7 +1482,7 @@ of this section, is an essential building block for any sort of stochastic model or Monte Carlo computer work.

    -

    3. Random Numbers, better name: pseudo random numbers

    +

    1.42. Random Numbers, better name: pseudo random numbers

    A disclaimer is however appropriate. It should be fairly obvious that something as deterministic as a computer cannot generate purely random numbers.

    Numbers generated by any of the standard algorithms are in reality pseudo random @@ -1381,7 +1495,7 @@ numbers, hopefully abiding to the following criteria:

    -

    4. Random number generator RNG

    +

    1.43. Random number generator RNG

    The most common random number generators are based on so-called Linear congruential relations of the type

    @@ -1401,7 +1515,7 @@ that is if we were to evaluate \((13) of the division \(13/9\), namely \(4\).

    -

    5. Random number generator RNG and periodic outputs

    +

    1.44. Random number generator RNG and periodic outputs

    The problem with such generators is that their outputs are periodic; they will start to repeat themselves with a period that is at most \(M\). If however @@ -1423,7 +1537,7 @@ N_i=(27N_{i-1}+11) \mathrm{MOD} (54), just \(2\).

    -

    6. Random number generator RNG and its period

    +

    1.45. Random number generator RNG and its period

    Typical periods for the random generators provided in the program library are of the order of \(\sim 10^9\) or larger. Other random number generators which have become increasingly popular are so-called shift-register generators. @@ -1438,7 +1552,7 @@ N_l=(aN_{l-i}+cN_{l-j})\mathrm{MOD}(M). \]

    -

    7. Random number generator RNG, other examples

    +

    1.46. Random number generator RNG, other examples

    Such a generator again produces a sequence of pseudorandom numbers but this time with a period much larger than \(M\). It is also possible to construct more elaborate algorithms by including @@ -1467,7 +1581,7 @@ which consists of two congruential relations

    which according to the authors has a period larger than \(2^{94}\).

    -

    8. Random number generator RNG, other examples

    +

    1.47. Random number generator RNG, other examples

    Instead of using modular addition, we could use the bitwise exclusive-OR (\(\oplus\)) operation so that

    @@ -1485,7 +1599,7 @@ function \(\mathrm{IEOR}(m,n)\)\(m\wedge n\).

    -

    9. Random number generator RNG, RAN0

    +

    1.48. Random number generator RNG, RAN0

    We show here how the linear congruential algorithm can be implemented, namely

    \[ @@ -1513,7 +1627,7 @@ r = M\hspace{0.1cm}\mathrm{MOD} \hspace{0.1cm}a. \(q\) and \(r\) are chosen so that \(r < q\).

    -

    10. Random number generator RNG, RAN0

    +

    1.49. Random number generator RNG, RAN0

    To see how this works we note first that

    @@ -1529,7 +1643,7 @@ The last term \([N_{i-1}/q]M\mathrm{M \([N_{i-1}/q]\) just yields a constant which is multiplied with \(M\).

    -

    11. Random number generator RNG, RAN0

    +

    1.50. Random number generator RNG, RAN0

    We can now rewrite Eq. (14) as

    @@ -1563,7 +1677,7 @@ in

    \]
    -

    12. Random number generator RNG, RAN0

    +

    1.51. Random number generator RNG, RAN0

    The term \([N_{i-1}/q]r\) is always smaller or equal \(N_{i-1}(r/q)\) and with \(r < q\) we obtain always a number smaller than \(N_{i-1}\), which is smaller than \(M\). And since the number \(N_{i-1}\mathrm{MOD} (q)\) is between zero and \(q-1\) then @@ -1578,7 +1692,7 @@ algorithm is that is should never be called with the initial seed set to \(0\).

    -

    13. Random number generator RNG, RAN0 code

    +

    1.52. Random number generator RNG, RAN0 code

            /*
          ** The function
          **           ran0()
@@ -1607,8 +1721,9 @@ set to \(0\).
    
     } // End: function ran0()
    +
    -

    13.1. Properties of Selected Random Number Generators

    +

    1.53. Properties of Selected Random Number Generators

    As mentioned previously, the underlying PDF for the generation of random numbers is the uniform distribution, meaning that the probability for finding a number \(x\) in the interval [0,1] is \(p(x)=1\).

    @@ -1622,7 +1737,7 @@ deviations.

    \(\mu=\langle x\rangle\).

    -

    13.2. Properties of Selected Random Number Generators

    +

    1.54. Properties of Selected Random Number Generators

    For the uniform distribution, the mean value \(\mu\) is then

    \[ @@ -1635,7 +1750,7 @@ deviations.

    \]
    -

    13.3. Properties of Selected Random Number Generators

    +

    1.55. Properties of Selected Random Number Generators

    The various random number generators produce results which agree rather well with these limiting values.

    @@ -1659,7 +1774,7 @@ these limiting values.

    -

    13.4. Simple demonstration of RNGs using python

    +

    1.56. Simple demonstration of RNGs using python

    The following simple Python code plots the distribution of the produced random numbers using the linear congruential RNG employed by Python. The trend displayed in the previous table is seen rather clearly.

    @@ -1694,7 +1809,7 @@ these limiting values.

    -

    13.5. Properties of Selected Random Number Generators

    +

    1.57. Properties of Selected Random Number Generators

    Since our random numbers, which are typically generated via a linear congruential algorithm, are never fully independent, we can then define an important test which measures the degree of correlation, namely the so-called
    @@ -1717,7 +1832,7 @@ in the evaluation of other expectation values. If they are not independent, our assumption for approximating \(\sigma_N\) is no longer valid.

    -

    13.6. Autocorrelation function

    +

    1.58. Autocorrelation function

    This program computes the autocorrelation function as discussed in the equation on the previous slide for random numbers generated with the normal distribution \(N(0,1)\).

    @@ -1754,7 +1869,7 @@ assumption for approximating \(\sigma
    -
    -0.027481252820017347 1.0170183171235068
+
    0.052276839521249285 0.9675166677604984
 
    
 
    
 _images/chapter2_184_1.png
@@ -1764,7 +1879,7 @@ assumption for approximating \(\sigma
 For the remaining values we notice that there are still non-zero values for the auto-correlation function.
    -

    13.7. Correlation function and which random number generators should I use

    +

    1.59. Correlation function and which random number generators should I use

    The program here computes the correlation function for one of the standard functions included with the c++ compiler.

        //  This function computes the autocorrelation function for 
     //  the standard c++ random number generator
@@ -1835,15 +1950,15 @@ For the remaining values we notice that there are still non-zero values for the
    -
    -

    14. Which RNG should I use?

    +

    1.60. Which RNG should I use?

    • C++ has a class called random. The random class contains a large selection of RNGs and is highly recommended. Some of these RNGs have very large periods making it thereby very safe to use these RNGs in case one is performing large calculations. In particular, the Mersenne twister random number engine has a period of \(2^{19937}\).

    • Add RNGs in Python

    +
    -

    14.1. How to use the Mersenne generator

    +

    1.61. How to use the Mersenne generator

    The following part of a c++ code (from project 4) sets up the uniform distribution for \(x\in [0,1]\).

        /*
     
@@ -1862,7 +1977,7 @@ For the remaining values we notice that there are still non-zero values for the
    -

    14.2. Why blocking?

    +

    1.62. Why blocking?

    Statistical analysis.

    * Monte Carlo simulations can be treated as *computer experiments*
 
@@ -1874,7 +1989,7 @@ For the remaining values we notice that there are still non-zero values for the
 
    A very good article which explains blocking is H. Flyvbjerg and H. G. Petersen, Error estimates on averages of correlated data,  Journal of Chemical Physics 91, 461-466 (1989).
    -

    14.3. Why blocking?

    +

    1.63. Why blocking?

    Statistical analysis.

    * As in other experiments, Monte Carlo experiments have two classes of errors:
 
@@ -1890,7 +2005,7 @@ For the remaining values we notice that there are still non-zero values for the
    -

    14.4. Code to demonstrate the calculation of the autocorrelation function

    +

    1.64. Code to demonstrate the calculation of the autocorrelation function

    The following code computes the autocorrelation function, the covariance and the standard deviation for standard RNG. The following file gives the code.

    @@ -1975,7 +2090,7 @@ The -

    14.5. What is blocking?

    +

    1.65. What is blocking?

    Blocking.

    * Say that we have a set of samples from a Monte Carlo experiment
 
@@ -1996,7 +2111,7 @@ The \(\tau\) is the correlation time (the time between a sample and the next uncorrelated sample) and \(\Delta t\) is time between each sample
    -

    14.6. What is blocking?

    +

    1.66. What is blocking?

    Blocking.

    * If $\Delta t\gg\tau$ our first estimate of $\sigma$ still holds
 
@@ -2013,7 +2128,7 @@ The 
-
    14.7. What is blocking?
    
+
    1.67. What is blocking?
    
 
    Blocking.
    
 
    * Problem: We don't know $\tau$ or it is too expensive to compute
 
@@ -2026,7 +2141,7 @@ The 
-
    14.8. Implementation
    
+
    1.68. Implementation
    
 
    * Do a Monte Carlo simulation, storing all samples to file
 
 * Do the statistical analysis on this file, independently of your Monte Carlo program
@@ -2045,7 +2160,7 @@ The 
-
    14.9. Actual implementation with code, main function
    
+
    1.69. Actual implementation with code, main function
    
 
    When the file gets large, it can be useful to write your data in binary mode instead of ascii characters.
 The following python file   reads data from file with the output from every Monte Carlo cycle.
    
 
    
@@ -2094,7 +2209,7 @@ The 
-
    14.10. The Bootstrap method
    
+
    1.70. The Bootstrap method
    
 
    The Bootstrap  resampling method is also very popular. It is very simple:
    
 
      
 
    1. Start with your sample of measurements and compute the sample variance and the mean values
      2. 
@@ -2106,7 +2221,7 @@ that it produces the correct standard deviation.
      
 
      This method is very useful for small ensembles of data points.
      
 
    
 
    
-
    14.11. Bootstrapping
    
+
    1.71. Bootstrapping
    
 
    Given a set of \(N\) data, assume that we are interested in some
 observable \(\theta\) which may be estimated from that set. This observable can also be for example the result of a fit based on all \(N\) raw data.
 Let us call the value of the observable obtained from the original
@@ -2115,7 +2230,7 @@ other samples by choosing randomly \(
 This costs essentially nothing, since we just recycle the original data set for the building of new sets.
    
 
    
 
    
-
    14.12. Bootstrapping, recipe
    
+
    1.72. Bootstrapping, recipe
    
 
    Let us assume we have done this \(K\) times and thus have \(K\) sets of \(N\)
 data values each.
 Of course some values will enter more than once in the new sets. For each of these sets one computes the observable \(\theta\) resulting in values \(\theta_k\) with \(k = 1,...,K\). Then one determines
    
@@ -2134,7 +2249,7 @@ sigma^2_{\tilde{\theta}} = \frac{1}{K} \sum_{k=1}^K \left(\theta_k-\tilde{\theta
 the true \(\angle\theta\rangle\). As final result for the observable one quotes \(\angle\theta\rangle = \tilde{\theta} \pm \sigma_{\tilde{\theta}}\) .
    
 
    
 
    
-
    14.13. Bootstrapping, code
    
+
    1.73. Bootstrapping, code
    
 
        # Bootstrap
         @timeFunction
         def bootstrap(self, nBoots = 1000):
@@ -2148,7 +2263,7 @@ the true \(\angle\theta\rangle\)
 
    
 
    
-
    14.14. Jackknife, code
    
+
    1.74. Jackknife, code
    
 
        # Jackknife
         @timeFunction
         def jackknife(self):
@@ -2192,7 +2307,7 @@ the true \(\angle\theta\rangle\)
         
     Introduction to Applied Data Analysis and Machine Learning
-    15. Getting started, our first data and Machine Learning encounters
+    2. Getting started, our first data and Machine Learning encounters