From 37174bf0c6fabd8e45295fb0eaa43bf6095ad167 Mon Sep 17 00:00:00 2001 From: Morten Hjorth-Jensen Date: Thu, 9 Sep 2021 08:16:28 +0200 Subject: [PATCH] adding bayes --- doc/pub/week36/html/week36-bs.html | 64 ++++++++------- doc/pub/week36/html/week36-reveal.html | 62 ++++++++++++++- doc/pub/week36/html/week36-solarized.html | 61 +++++++++++++- doc/pub/week36/html/week36.html | 61 +++++++++++++- doc/pub/week36/ipynb/ipynb-week36-src.tar.gz | Bin 191 -> 192 bytes doc/pub/week36/ipynb/week36.ipynb | 79 ++++++++++++++++++- doc/src/week36/week36.do.txt | 50 +++++++++++- 7 files changed, 338 insertions(+), 39 deletions(-) diff --git a/doc/pub/week36/html/week36-bs.html b/doc/pub/week36/html/week36-bs.html index d7cdc9232..a1cd812e4 100644 --- a/doc/pub/week36/html/week36-bs.html +++ b/doc/pub/week36/html/week36-bs.html @@ -129,6 +129,9 @@ Automatically generated HTML file from DocOnce source 2, None, 'more-basic-statistics-and-bayes-theorem'), + ('Marginal Probability', 2, None, 'marginal-probability'), + ('Conditional Probability', 2, None, 'conditional-probability'), + ("Bayes' Theorem", 2, None, 'bayes-theorem'), ('Frliday September 10', 2, None, 'frliday-september-10'), ('Why resampling methods', 2, None, 'why-resampling-methods'), ('Resampling methods', 2, None, 'resampling-methods'), @@ -287,34 +290,37 @@ MathJax.Hub.Config({
  • Maximum Likelihood Estimation (MLE)
  • A new Cost Function
  • More basic Statistics and Bayes' theorem
    • -
  • Frliday September 10
    • -
  • Why resampling methods
    • -
  • Resampling methods
    • -
  • Resampling approaches can be computationally expensive
    • -
  • Why resampling methods ?
    • -
  • Statistical analysis
    • -
  • Resampling methods
    • -
  • Resampling methods: Jackknife and Bootstrap
    • -
  • Resampling methods: Jackknife
    • -
  • Jackknife code example
    • -
  • Resampling methods: Bootstrap
    • -
  • Resampling methods: Bootstrap background
    • -
  • Resampling methods: More Bootstrap background
    • -
  • Resampling methods: Bootstrap approach
    • -
  • Resampling methods: Bootstrap steps
    • -
  • Code example for the Bootstrap method
    • -
  • Various steps in cross-validation
    • -
  • How to set up the cross-validation for Ridge and/or Lasso
    • -
  • Cross-validation in brief
    • -
  • Code Example for Cross-validation and \( k \)-fold Cross-validation
    • -
  • The bias-variance tradeoff
    • -
  • Example code for Bias-Variance tradeoff
    • -
  • Understanding what happens
    • -
  • Summing up
    • -
  • Another Example from Scikit-Learn's Repository
    • -
  • More examples on bootstrap and cross-validation and errors
    • -
  • The same example but now with cross-validation
    • -
  • Cross-validation with Ridge
    • +
  • Marginal Probability
    • +
  • Conditional Probability
    • +
  • Bayes' Theorem
    • +
  • Frliday September 10
    • +
  • Why resampling methods
    • +
  • Resampling methods
    • +
  • Resampling approaches can be computationally expensive
    • +
  • Why resampling methods ?
    • +
  • Statistical analysis
    • +
  • Resampling methods
    • +
  • Resampling methods: Jackknife and Bootstrap
    • +
  • Resampling methods: Jackknife
    • +
  • Jackknife code example
    • +
  • Resampling methods: Bootstrap
    • +
  • Resampling methods: Bootstrap background
    • +
  • Resampling methods: More Bootstrap background
    • +
  • Resampling methods: Bootstrap approach
    • +
  • Resampling methods: Bootstrap steps
    • +
  • Code example for the Bootstrap method
    • +
  • Various steps in cross-validation
    • +
  • How to set up the cross-validation for Ridge and/or Lasso
    • +
  • Cross-validation in brief
    • +
  • Code Example for Cross-validation and \( k \)-fold Cross-validation
    • +
  • The bias-variance tradeoff
    • +
  • Example code for Bias-Variance tradeoff
    • +
  • Understanding what happens
    • +
  • Summing up
    • +
  • Another Example from Scikit-Learn's Repository
    • +
  • More examples on bootstrap and cross-validation and errors
    • +
  • The same example but now with cross-validation
    • +
  • Cross-validation with Ridge
    • @@ -373,7 +379,7 @@ MathJax.Hub.Config({
  • 9
  • 10
  • ...
    • -
  • 61
    • +
  • 64
  • »
    • diff --git a/doc/pub/week36/html/week36-reveal.html b/doc/pub/week36/html/week36-reveal.html index 2e790d0d5..a242e608c 100644 --- a/doc/pub/week36/html/week36-reveal.html +++ b/doc/pub/week36/html/week36-reveal.html @@ -1211,15 +1211,73 @@ $$

    -The product rule is given by +The product rule (aka joint probability) is given by

     
    $$ -p(X \cup Y)= p(X,y)= p(X\vert Y)p(Y)=p(Y\vert X)p(x), +p(X \cup Y)= p(X,Y)= p(X\vert Y)p(Y)=p(Y\vert X)p(x), $$

     
    where we read \( p(X\vert Y) \) as the likelihood of obtaining \( X \) given \( Y \).

    + +

    +If we have independent events then \( p(X,Y)=p(X)p(Y) \). + + + +

    +

    Marginal Probability

    + +

    +The marginal probability is defined in terms of only of the set of variables \( X,Y \). For a discrete probability we have +

    +Discrete Probability +

     
    +$$ +p(X)=\sum_{i=0}^{n-1}p(X,Y=y_i)=\sum_{i=0}^{n-1}p(X\vert Y=y_i)p(Y=y_i)=\sum_{i=0}^{n-1}p(X\vert y_i)p(y_i). +$$ +

     
    +

    +
    + + +
    +

    Conditional Probability

    + +

    +The conditional probability, if \( p(Y) > 0 \), is +

    + +

     
    +$$ +p(X\vert Y)= \frac{p(X,Y)}{p(Y)}=\frac{p(X,Y)}{\sum_{i=0}^{n-1}p(Y\vert X=x_i)p(x_i)}. +$$ +

     
    +

    +
    + + +
    +

    Bayes' Theorem

    + +

    +If we combine the conditional probability with the marginal probability and the standard product rule, we have +

     
    +$$ +p(X\vert Y)= \frac{p(X,Y)}{p(Y)}, +$$ +

     
    + +we can rewrite this as + +

     
    +$$ +p(X\vert Y)= \frac{p(X,Y)}{\sum_{i=0}^{n-1}p(Y\vert X=x_i)p(x_i)}=\frac{p(Y\vert X)p(X)}{\sum_{i=0}^{n-1}p(Y\vert X=x_i)p(x_i)}, +$$ +

     
    + +Which is Bayes' theorem. More text to be added here by Friday September 10.

    diff --git a/doc/pub/week36/html/week36-solarized.html b/doc/pub/week36/html/week36-solarized.html index 278882d0d..3d592906c 100644 --- a/doc/pub/week36/html/week36-solarized.html +++ b/doc/pub/week36/html/week36-solarized.html @@ -149,6 +149,9 @@ div { text-align: justify; text-justify: inter-word; } 2, None, 'more-basic-statistics-and-bayes-theorem'), + ('Marginal Probability', 2, None, 'marginal-probability'), + ('Conditional Probability', 2, None, 'conditional-probability'), + ("Bayes' Theorem", 2, None, 'bayes-theorem'), ('Frliday September 10', 2, None, 'frliday-september-10'), ('Why resampling methods', 2, None, 'why-resampling-methods'), ('Resampling methods', 2, None, 'resampling-methods'), @@ -1188,16 +1191,70 @@ $$

    -The product rule is given by +The product rule (aka joint probability) is given by

    $$ -p(X \cup Y)= p(X,y)= p(X\vert Y)p(Y)=p(Y\vert X)p(x), +p(X \cup Y)= p(X,Y)= p(X\vert Y)p(Y)=p(Y\vert X)p(x), $$ where we read \( p(X\vert Y) \) as the likelihood of obtaining \( X \) given \( Y \).

    +

    +If we have independent events then \( p(X,Y)=p(X)p(Y) \). + +

    +









    + +

    Marginal Probability

    + +

    +The marginal probability is defined in terms of only of the set of variables \( X,Y \). For a discrete probability we have +

    +Discrete Probability +

    +$$ +p(X)=\sum_{i=0}^{n-1}p(X,Y=y_i)=\sum_{i=0}^{n-1}p(X\vert Y=y_i)p(Y=y_i)=\sum_{i=0}^{n-1}p(X\vert y_i)p(y_i). +$$ +

    + + +

    +









    + +

    Conditional Probability

    + +

    +The conditional probability, if \( p(Y) > 0 \), is +

    + +

    +$$ +p(X\vert Y)= \frac{p(X,Y)}{p(Y)}=\frac{p(X,Y)}{\sum_{i=0}^{n-1}p(Y\vert X=x_i)p(x_i)}. +$$ +

    + + +

    +









    + +

    Bayes' Theorem

    + +

    +If we combine the conditional probability with the marginal probability and the standard product rule, we have +$$ +p(X\vert Y)= \frac{p(X,Y)}{p(Y)}, +$$ + +we can rewrite this as + +$$ +p(X\vert Y)= \frac{p(X,Y)}{\sum_{i=0}^{n-1}p(Y\vert X=x_i)p(x_i)}=\frac{p(Y\vert X)p(X)}{\sum_{i=0}^{n-1}p(Y\vert X=x_i)p(x_i)}, +$$ + +Which is Bayes' theorem. More text to be added here by Friday September 10. +











    diff --git a/doc/pub/week36/html/week36.html b/doc/pub/week36/html/week36.html index d44b27577..18219fc4a 100644 --- a/doc/pub/week36/html/week36.html +++ b/doc/pub/week36/html/week36.html @@ -154,6 +154,9 @@ div { text-align: justify; text-justify: inter-word; } 2, None, 'more-basic-statistics-and-bayes-theorem'), + ('Marginal Probability', 2, None, 'marginal-probability'), + ('Conditional Probability', 2, None, 'conditional-probability'), + ("Bayes' Theorem", 2, None, 'bayes-theorem'), ('Frliday September 10', 2, None, 'frliday-september-10'), ('Why resampling methods', 2, None, 'why-resampling-methods'), ('Resampling methods', 2, None, 'resampling-methods'), @@ -1193,16 +1196,70 @@ $$

    -The product rule is given by +The product rule (aka joint probability) is given by

    $$ -p(X \cup Y)= p(X,y)= p(X\vert Y)p(Y)=p(Y\vert X)p(x), +p(X \cup Y)= p(X,Y)= p(X\vert Y)p(Y)=p(Y\vert X)p(x), $$ where we read \( p(X\vert Y) \) as the likelihood of obtaining \( X \) given \( Y \).

    +

    +If we have independent events then \( p(X,Y)=p(X)p(Y) \). + +

    +









    + +

    Marginal Probability

    + +

    +The marginal probability is defined in terms of only of the set of variables \( X,Y \). For a discrete probability we have +

    +Discrete Probability +

    +$$ +p(X)=\sum_{i=0}^{n-1}p(X,Y=y_i)=\sum_{i=0}^{n-1}p(X\vert Y=y_i)p(Y=y_i)=\sum_{i=0}^{n-1}p(X\vert y_i)p(y_i). +$$ +

    + + +

    +









    + +

    Conditional Probability

    + +

    +The conditional probability, if \( p(Y) > 0 \), is +

    + +

    +$$ +p(X\vert Y)= \frac{p(X,Y)}{p(Y)}=\frac{p(X,Y)}{\sum_{i=0}^{n-1}p(Y\vert X=x_i)p(x_i)}. +$$ +

    + + +

    +









    + +

    Bayes' Theorem

    + +

    +If we combine the conditional probability with the marginal probability and the standard product rule, we have +$$ +p(X\vert Y)= \frac{p(X,Y)}{p(Y)}, +$$ + +we can rewrite this as + +$$ +p(X\vert Y)= \frac{p(X,Y)}{\sum_{i=0}^{n-1}p(Y\vert X=x_i)p(x_i)}=\frac{p(Y\vert X)p(X)}{\sum_{i=0}^{n-1}p(Y\vert X=x_i)p(x_i)}, +$$ + +Which is Bayes' theorem. More text to be added here by Friday September 10. +











    diff --git a/doc/pub/week36/ipynb/ipynb-week36-src.tar.gz b/doc/pub/week36/ipynb/ipynb-week36-src.tar.gz index 89a5af289fa95ad824c554339356ce2b8061d263..d05ae0b2c7be3fd26f804f66f7f83069501de60d 100644 GIT binary patch literal 192 zcmV;x06+g9iwFRXra55%1MSaC3c@fD2H>uHia9~aqpfSfE?fvAULd7Y8*7uAq-bw% zAD}D6O%WmA=AV#Zm^ozY%`S`F-35ywgrso{W3JL u(T-nW@Y+d(AXE>cD5R5GiA&fTeKMl4QTXd)JkRsIuRQ=mljf8F2mk<51X!s6 literal 191 zcmV;w06_mAiwFP~qB&sz1MSbv3c@f92k@Qu6nTQt?4vsidhj5K_yQfvT%ESscIe*T zeSn@6FGGa>E`LHoNOr5maOOVA!dLnB2QvuF$ac8hIs&N zzLQ>B>xAh~IoC&0mejkseyplK?3rGHXa0$OEiJ6KovXD1r5zPoH*h0r5+2D*rwS-F t*`Wo7uB|i#LUka@QaY)XIE5FZ4@R7A6#n`d&+|O*YY$F{qg((8001d!RsR40 diff --git a/doc/pub/week36/ipynb/week36.ipynb b/doc/pub/week36/ipynb/week36.ipynb index bdb0e54f0..41f8d94e4 100644 --- a/doc/pub/week36/ipynb/week36.ipynb +++ b/doc/pub/week36/ipynb/week36.ipynb @@ -1547,7 +1547,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "**The product rule is given by.**" + "**The product rule (aka joint probability) is given by.**" ] }, { @@ -1555,7 +1555,7 @@ "metadata": {}, "source": [ "$$\n", - "p(X \\cup Y)= p(X,y)= p(X\\vert Y)p(Y)=p(Y\\vert X)p(x),\n", + "p(X \\cup Y)= p(X,Y)= p(X\\vert Y)p(Y)=p(Y\\vert X)p(x),\n", "$$" ] }, @@ -1567,6 +1567,81 @@ "\n", "\n", "\n", + "If we have independent events then $p(X,Y)=p(X)p(Y)$.\n", + "\n", + "\n", + "## Marginal Probability\n", + "\n", + "The marginal probability is defined in terms of only of the set of variables $X,Y$. For a discrete probability we have\n", + "**Discrete Probability.**" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "p(X)=\\sum_{i=0}^{n-1}p(X,Y=y_i)=\\sum_{i=0}^{n-1}p(X\\vert Y=y_i)p(Y=y_i)=\\sum_{i=0}^{n-1}p(X\\vert y_i)p(y_i).\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Conditional Probability\n", + "\n", + "The conditional probability, if $p(Y) > 0$, is" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "p(X\\vert Y)= \\frac{p(X,Y)}{p(Y)}=\\frac{p(X,Y)}{\\sum_{i=0}^{n-1}p(Y\\vert X=x_i)p(x_i)}.\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Bayes' Theorem\n", + "\n", + "If we combine the conditional probability with the marginal probability and the standard product rule, we have" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "p(X\\vert Y)= \\frac{p(X,Y)}{p(Y)},\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "we can rewrite this as" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "p(X\\vert Y)= \\frac{p(X,Y)}{\\sum_{i=0}^{n-1}p(Y\\vert X=x_i)p(x_i)}=\\frac{p(Y\\vert X)p(X)}{\\sum_{i=0}^{n-1}p(Y\\vert X=x_i)p(x_i)},\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Which is Bayes' theorem. More text to be added here by Friday September 10.\n", "\n", "\n", "## Frliday September 10\n", diff --git a/doc/src/week36/week36.do.txt b/doc/src/week36/week36.do.txt index d5a3d8dc7..a3e874169 100644 --- a/doc/src/week36/week36.do.txt +++ b/doc/src/week36/week36.do.txt @@ -883,15 +883,61 @@ p(X \cup Y)= p(X)+p(Y)-p(X \cap Y). !eblock -!bblock The product rule is given by +!bblock The product rule (aka joint probability) is given by !bt \[ -p(X \cup Y)= p(X,y)= p(X\vert Y)p(Y)=p(Y\vert X)p(x), +p(X \cup Y)= p(X,Y)= p(X\vert Y)p(Y)=p(Y\vert X)p(x), \] !et where we read $p(X\vert Y)$ as the likelihood of obtaining $X$ given $Y$. !eblock +If we have independent events then $p(X,Y)=p(X)p(Y)$. + + +!split +===== Marginal Probability ===== + +The marginal probability is defined in terms of only of the set of variables $X,Y$. For a discrete probability we have +!bblock Discrete Probability +!bt +\[ +p(X)=\sum_{i=0}^{n-1}p(X,Y=y_i)=\sum_{i=0}^{n-1}p(X\vert Y=y_i)p(Y=y_i)=\sum_{i=0}^{n-1}p(X\vert y_i)p(y_i). +\] +!et +!eblock + + +!split +===== Conditional Probability ===== + +The conditional probability, if $p(Y) > 0$, is +!bblock +!bt +\[ +p(X\vert Y)= \frac{p(X,Y)}{p(Y)}=\frac{p(X,Y)}{\sum_{i=0}^{n-1}p(Y\vert X=x_i)p(x_i)}. +\] +!et +!eblock + + +!split +===== Bayes' Theorem ===== + +If we combine the conditional probability with the marginal probability and the standard product rule, we have +!bt +\[ +p(X\vert Y)= \frac{p(X,Y)}{p(Y)}, +\] +!et +we can rewrite this as + +!bt +\[ +p(X\vert Y)= \frac{p(X,Y)}{\sum_{i=0}^{n-1}p(Y\vert X=x_i)p(x_i)}=\frac{p(Y\vert X)p(X)}{\sum_{i=0}^{n-1}p(Y\vert X=x_i)p(x_i)}, +\] +!et +Which is Bayes' theorem. More text to be added here by Friday September 10. !split