lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

adding bayes

Browse Source
This commit is contained in:
Morten Hjorth-Jensen
2021-09-09 08:16:28 +02:00
parent 0650414b41
commit 37174bf0c6
7 changed files with 338 additions and 39 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/pub/week36/html/week36-bs.html
+35 -29
View File
@@ -129,6 +129,9 @@ Automatically generated HTML file from DocOnce source
2,
None,
'more-basic-statistics-and-bayes-theorem'),
('Marginal Probability', 2, None, 'marginal-probability'),
('Conditional Probability', 2, None, 'conditional-probability'),
("Bayes' Theorem", 2, None, 'bayes-theorem'),
('Frliday September 10', 2, None, 'frliday-september-10'),
('Why resampling methods', 2, None, 'why-resampling-methods'),
('Resampling methods', 2, None, 'resampling-methods'),
@@ -287,34 +290,37 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._week36-bs030.html#maximum-likelihood-estimation-mle" style="font-size: 80%;">Maximum Likelihood Estimation (MLE)</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs031.html#a-new-cost-function" style="font-size: 80%;">A new Cost Function</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs032.html#more-basic-statistics-and-bayes-theorem" style="font-size: 80%;">More basic Statistics and Bayes' theorem</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs033.html#frliday-september-10" style="font-size: 80%;">Frliday September 10</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs037.html#why-resampling-methods" style="font-size: 80%;">Why resampling methods</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs039.html#resampling-methods" style="font-size: 80%;">Resampling methods</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs036.html#resampling-approaches-can-be-computationally-expensive" style="font-size: 80%;">Resampling approaches can be computationally expensive</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs037.html#why-resampling-methods" style="font-size: 80%;">Why resampling methods ?</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs038.html#statistical-analysis" style="font-size: 80%;">Statistical analysis</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs039.html#resampling-methods" style="font-size: 80%;">Resampling methods</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs040.html#resampling-methods-jackknife-and-bootstrap" style="font-size: 80%;">Resampling methods: Jackknife and Bootstrap</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs041.html#resampling-methods-jackknife" style="font-size: 80%;">Resampling methods: Jackknife</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs042.html#jackknife-code-example" style="font-size: 80%;">Jackknife code example</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs043.html#resampling-methods-bootstrap" style="font-size: 80%;">Resampling methods: Bootstrap</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs044.html#resampling-methods-bootstrap-background" style="font-size: 80%;">Resampling methods: Bootstrap background</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs045.html#resampling-methods-more-bootstrap-background" style="font-size: 80%;">Resampling methods: More Bootstrap background</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs046.html#resampling-methods-bootstrap-approach" style="font-size: 80%;">Resampling methods: Bootstrap approach</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs047.html#resampling-methods-bootstrap-steps" style="font-size: 80%;">Resampling methods: Bootstrap steps</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs048.html#code-example-for-the-bootstrap-method" style="font-size: 80%;">Code example for the Bootstrap method</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs049.html#various-steps-in-cross-validation" style="font-size: 80%;">Various steps in cross-validation</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs050.html#how-to-set-up-the-cross-validation-for-ridge-and-or-lasso" style="font-size: 80%;">How to set up the cross-validation for Ridge and/or Lasso</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs051.html#cross-validation-in-brief" style="font-size: 80%;">Cross-validation in brief</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs052.html#code-example-for-cross-validation-and-k-fold-cross-validation" style="font-size: 80%;">Code Example for Cross-validation and \( k \)-fold Cross-validation</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs053.html#the-bias-variance-tradeoff" style="font-size: 80%;">The bias-variance tradeoff</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs054.html#example-code-for-bias-variance-tradeoff" style="font-size: 80%;">Example code for Bias-Variance tradeoff</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs055.html#understanding-what-happens" style="font-size: 80%;">Understanding what happens</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs056.html#summing-up" style="font-size: 80%;">Summing up</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs057.html#another-example-from-scikit-learn-s-repository" style="font-size: 80%;">Another Example from Scikit-Learn's Repository</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs058.html#more-examples-on-bootstrap-and-cross-validation-and-errors" style="font-size: 80%;">More examples on bootstrap and cross-validation and errors</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs059.html#the-same-example-but-now-with-cross-validation" style="font-size: 80%;">The same example but now with cross-validation</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs060.html#cross-validation-with-ridge" style="font-size: 80%;">Cross-validation with Ridge</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs033.html#marginal-probability" style="font-size: 80%;">Marginal Probability</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs034.html#conditional-probability" style="font-size: 80%;">Conditional Probability</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs035.html#bayes-theorem" style="font-size: 80%;">Bayes' Theorem</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs036.html#frliday-september-10" style="font-size: 80%;">Frliday September 10</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs040.html#why-resampling-methods" style="font-size: 80%;">Why resampling methods</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs042.html#resampling-methods" style="font-size: 80%;">Resampling methods</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs039.html#resampling-approaches-can-be-computationally-expensive" style="font-size: 80%;">Resampling approaches can be computationally expensive</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs040.html#why-resampling-methods" style="font-size: 80%;">Why resampling methods ?</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs041.html#statistical-analysis" style="font-size: 80%;">Statistical analysis</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs042.html#resampling-methods" style="font-size: 80%;">Resampling methods</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs043.html#resampling-methods-jackknife-and-bootstrap" style="font-size: 80%;">Resampling methods: Jackknife and Bootstrap</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs044.html#resampling-methods-jackknife" style="font-size: 80%;">Resampling methods: Jackknife</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs045.html#jackknife-code-example" style="font-size: 80%;">Jackknife code example</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs046.html#resampling-methods-bootstrap" style="font-size: 80%;">Resampling methods: Bootstrap</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs047.html#resampling-methods-bootstrap-background" style="font-size: 80%;">Resampling methods: Bootstrap background</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs048.html#resampling-methods-more-bootstrap-background" style="font-size: 80%;">Resampling methods: More Bootstrap background</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs049.html#resampling-methods-bootstrap-approach" style="font-size: 80%;">Resampling methods: Bootstrap approach</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs050.html#resampling-methods-bootstrap-steps" style="font-size: 80%;">Resampling methods: Bootstrap steps</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs051.html#code-example-for-the-bootstrap-method" style="font-size: 80%;">Code example for the Bootstrap method</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs052.html#various-steps-in-cross-validation" style="font-size: 80%;">Various steps in cross-validation</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs053.html#how-to-set-up-the-cross-validation-for-ridge-and-or-lasso" style="font-size: 80%;">How to set up the cross-validation for Ridge and/or Lasso</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs054.html#cross-validation-in-brief" style="font-size: 80%;">Cross-validation in brief</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs055.html#code-example-for-cross-validation-and-k-fold-cross-validation" style="font-size: 80%;">Code Example for Cross-validation and \( k \)-fold Cross-validation</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs056.html#the-bias-variance-tradeoff" style="font-size: 80%;">The bias-variance tradeoff</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs057.html#example-code-for-bias-variance-tradeoff" style="font-size: 80%;">Example code for Bias-Variance tradeoff</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs058.html#understanding-what-happens" style="font-size: 80%;">Understanding what happens</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs059.html#summing-up" style="font-size: 80%;">Summing up</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs060.html#another-example-from-scikit-learn-s-repository" style="font-size: 80%;">Another Example from Scikit-Learn's Repository</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs061.html#more-examples-on-bootstrap-and-cross-validation-and-errors" style="font-size: 80%;">More examples on bootstrap and cross-validation and errors</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs062.html#the-same-example-but-now-with-cross-validation" style="font-size: 80%;">The same example but now with cross-validation</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs063.html#cross-validation-with-ridge" style="font-size: 80%;">Cross-validation with Ridge</a></li>
</ul>
</li>
@@ -373,7 +379,7 @@ MathJax.Hub.Config({
<li><a href="._week36-bs008.html">9</a></li>
<li><a href="._week36-bs009.html">10</a></li>
<li><a href="">...</a></li>
<li><a href="._week36-bs060.html">61</a></li>
<li><a href="._week36-bs063.html">64</a></li>
<li><a href="._week36-bs001.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/week36/html/week36-reveal.html
+60 -2
View File
@@ -1211,15 +1211,73 @@ $$
<p>
<div class="alert alert-block alert-block alert-text-normal">
<b>The product rule is given by</b>
<b>The product rule (aka joint probability) is given by</b>
<p>&nbsp;<br>
$$
p(X \cup Y)= p(X,y)= p(X\vert Y)p(Y)=p(Y\vert X)p(x),
p(X \cup Y)= p(X,Y)= p(X\vert Y)p(Y)=p(Y\vert X)p(x),
$$
<p>&nbsp;<br>
where we read \( p(X\vert Y) \) as the likelihood of obtaining \( X \) given \( Y \).
</div>
<p>
If we have independent events then \( p(X,Y)=p(X)p(Y) \).
</section>
<section>
<h2 id="marginal-probability">Marginal Probability </h2>
<p>
The marginal probability is defined in terms of only of the set of variables \( X,Y \). For a discrete probability we have
<div class="alert alert-block alert-block alert-text-normal">
<b>Discrete Probability</b>
<p>&nbsp;<br>
$$
p(X)=\sum_{i=0}^{n-1}p(X,Y=y_i)=\sum_{i=0}^{n-1}p(X\vert Y=y_i)p(Y=y_i)=\sum_{i=0}^{n-1}p(X\vert y_i)p(y_i).
$$
<p>&nbsp;<br>
</div>
</section>
<section>
<h2 id="conditional-probability">Conditional Probability </h2>
<p>
The conditional probability, if \( p(Y) > 0 \), is
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>&nbsp;<br>
$$
p(X\vert Y)= \frac{p(X,Y)}{p(Y)}=\frac{p(X,Y)}{\sum_{i=0}^{n-1}p(Y\vert X=x_i)p(x_i)}.
$$
<p>&nbsp;<br>
</div>
</section>
<section>
<h2 id="bayes-theorem">Bayes' Theorem </h2>
<p>
If we combine the conditional probability with the marginal probability and the standard product rule, we have
<p>&nbsp;<br>
$$
p(X\vert Y)= \frac{p(X,Y)}{p(Y)},
$$
<p>&nbsp;<br>
we can rewrite this as
<p>&nbsp;<br>
$$
p(X\vert Y)= \frac{p(X,Y)}{\sum_{i=0}^{n-1}p(Y\vert X=x_i)p(x_i)}=\frac{p(Y\vert X)p(X)}{\sum_{i=0}^{n-1}p(Y\vert X=x_i)p(x_i)},
$$
<p>&nbsp;<br>
Which is Bayes' theorem. More text to be added here by Friday September 10.
</section>
doc/pub/week36/html/week36-solarized.html
+59 -2
View File
@@ -149,6 +149,9 @@ div { text-align: justify; text-justify: inter-word; }
2,
None,
'more-basic-statistics-and-bayes-theorem'),
('Marginal Probability', 2, None, 'marginal-probability'),
('Conditional Probability', 2, None, 'conditional-probability'),
("Bayes' Theorem", 2, None, 'bayes-theorem'),
('Frliday September 10', 2, None, 'frliday-september-10'),
('Why resampling methods', 2, None, 'why-resampling-methods'),
('Resampling methods', 2, None, 'resampling-methods'),
@@ -1188,16 +1191,70 @@ $$
<p>
<div class="alert alert-block alert-block alert-text-normal">
<b>The product rule is given by</b>
<b>The product rule (aka joint probability) is given by</b>
<p>
$$
p(X \cup Y)= p(X,y)= p(X\vert Y)p(Y)=p(Y\vert X)p(x),
p(X \cup Y)= p(X,Y)= p(X\vert Y)p(Y)=p(Y\vert X)p(x),
$$
where we read \( p(X\vert Y) \) as the likelihood of obtaining \( X \) given \( Y \).
</div>
<p>
If we have independent events then \( p(X,Y)=p(X)p(Y) \).
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="marginal-probability">Marginal Probability </h2>
<p>
The marginal probability is defined in terms of only of the set of variables \( X,Y \). For a discrete probability we have
<div class="alert alert-block alert-block alert-text-normal">
<b>Discrete Probability</b>
<p>
$$
p(X)=\sum_{i=0}^{n-1}p(X,Y=y_i)=\sum_{i=0}^{n-1}p(X\vert Y=y_i)p(Y=y_i)=\sum_{i=0}^{n-1}p(X\vert y_i)p(y_i).
$$
</div>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="conditional-probability">Conditional Probability </h2>
<p>
The conditional probability, if \( p(Y) > 0 \), is
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
$$
p(X\vert Y)= \frac{p(X,Y)}{p(Y)}=\frac{p(X,Y)}{\sum_{i=0}^{n-1}p(Y\vert X=x_i)p(x_i)}.
$$
</div>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="bayes-theorem">Bayes' Theorem </h2>
<p>
If we combine the conditional probability with the marginal probability and the standard product rule, we have
$$
p(X\vert Y)= \frac{p(X,Y)}{p(Y)},
$$
we can rewrite this as
$$
p(X\vert Y)= \frac{p(X,Y)}{\sum_{i=0}^{n-1}p(Y\vert X=x_i)p(x_i)}=\frac{p(Y\vert X)p(X)}{\sum_{i=0}^{n-1}p(Y\vert X=x_i)p(x_i)},
$$
Which is Bayes' theorem. More text to be added here by Friday September 10.
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
doc/pub/week36/html/week36.html
+59 -2
View File
@@ -154,6 +154,9 @@ div { text-align: justify; text-justify: inter-word; }
2,
None,
'more-basic-statistics-and-bayes-theorem'),
('Marginal Probability', 2, None, 'marginal-probability'),
('Conditional Probability', 2, None, 'conditional-probability'),
("Bayes' Theorem", 2, None, 'bayes-theorem'),
('Frliday September 10', 2, None, 'frliday-september-10'),
('Why resampling methods', 2, None, 'why-resampling-methods'),
('Resampling methods', 2, None, 'resampling-methods'),
@@ -1193,16 +1196,70 @@ $$
<p>
<div class="alert alert-block alert-block alert-text-normal">
<b>The product rule is given by</b>
<b>The product rule (aka joint probability) is given by</b>
<p>
$$
p(X \cup Y)= p(X,y)= p(X\vert Y)p(Y)=p(Y\vert X)p(x),
p(X \cup Y)= p(X,Y)= p(X\vert Y)p(Y)=p(Y\vert X)p(x),
$$
where we read \( p(X\vert Y) \) as the likelihood of obtaining \( X \) given \( Y \).
</div>
<p>
If we have independent events then \( p(X,Y)=p(X)p(Y) \).
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="marginal-probability">Marginal Probability </h2>
<p>
The marginal probability is defined in terms of only of the set of variables \( X,Y \). For a discrete probability we have
<div class="alert alert-block alert-block alert-text-normal">
<b>Discrete Probability</b>
<p>
$$
p(X)=\sum_{i=0}^{n-1}p(X,Y=y_i)=\sum_{i=0}^{n-1}p(X\vert Y=y_i)p(Y=y_i)=\sum_{i=0}^{n-1}p(X\vert y_i)p(y_i).
$$
</div>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="conditional-probability">Conditional Probability </h2>
<p>
The conditional probability, if \( p(Y) > 0 \), is
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
$$
p(X\vert Y)= \frac{p(X,Y)}{p(Y)}=\frac{p(X,Y)}{\sum_{i=0}^{n-1}p(Y\vert X=x_i)p(x_i)}.
$$
</div>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="bayes-theorem">Bayes' Theorem </h2>
<p>
If we combine the conditional probability with the marginal probability and the standard product rule, we have
$$
p(X\vert Y)= \frac{p(X,Y)}{p(Y)},
$$
we can rewrite this as
$$
p(X\vert Y)= \frac{p(X,Y)}{\sum_{i=0}^{n-1}p(Y\vert X=x_i)p(x_i)}=\frac{p(Y\vert X)p(X)}{\sum_{i=0}^{n-1}p(Y\vert X=x_i)p(x_i)},
$$
Which is Bayes' theorem. More text to be added here by Friday September 10.
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
doc/pub/week36/ipynb/ipynb-week36-src.tar.gz
BIN
View File
Binary file not shown.
doc/pub/week36/ipynb/week36.ipynb
+77 -2
View File
@@ -1547,7 +1547,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"**The product rule is given by.**"
"**The product rule (aka joint probability) is given by.**"
]
},
{
@@ -1555,7 +1555,7 @@
"metadata": {},
"source": [
"$$\n",
"p(X \\cup Y)= p(X,y)= p(X\\vert Y)p(Y)=p(Y\\vert X)p(x),\n",
"p(X \\cup Y)= p(X,Y)= p(X\\vert Y)p(Y)=p(Y\\vert X)p(x),\n",
"$$"
]
},
@@ -1567,6 +1567,81 @@
"\n",
"\n",
"\n",
"If we have independent events then $p(X,Y)=p(X)p(Y)$.\n",
"\n",
"\n",
"## Marginal Probability\n",
"\n",
"The marginal probability is defined in terms of only of the set of variables $X,Y$. For a discrete probability we have\n",
"**Discrete Probability.**"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$$\n",
"p(X)=\\sum_{i=0}^{n-1}p(X,Y=y_i)=\\sum_{i=0}^{n-1}p(X\\vert Y=y_i)p(Y=y_i)=\\sum_{i=0}^{n-1}p(X\\vert y_i)p(y_i).\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Conditional Probability\n",
"\n",
"The conditional probability, if $p(Y) > 0$, is"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$$\n",
"p(X\\vert Y)= \\frac{p(X,Y)}{p(Y)}=\\frac{p(X,Y)}{\\sum_{i=0}^{n-1}p(Y\\vert X=x_i)p(x_i)}.\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Bayes' Theorem\n",
"\n",
"If we combine the conditional probability with the marginal probability and the standard product rule, we have"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$$\n",
"p(X\\vert Y)= \\frac{p(X,Y)}{p(Y)},\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"we can rewrite this as"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$$\n",
"p(X\\vert Y)= \\frac{p(X,Y)}{\\sum_{i=0}^{n-1}p(Y\\vert X=x_i)p(x_i)}=\\frac{p(Y\\vert X)p(X)}{\\sum_{i=0}^{n-1}p(Y\\vert X=x_i)p(x_i)},\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Which is Bayes' theorem. More text to be added here by Friday September 10.\n",
"\n",
"\n",
"## Frliday September 10\n",
doc/src/week36/week36.do.txt
+48 -2
View File
@@ -883,15 +883,61 @@ p(X \cup Y)= p(X)+p(Y)-p(X \cap Y).
!eblock
!bblock The product rule is given by
!bblock The product rule (aka joint probability) is given by
!bt
\[
p(X \cup Y)= p(X,y)= p(X\vert Y)p(Y)=p(Y\vert X)p(x),
p(X \cup Y)= p(X,Y)= p(X\vert Y)p(Y)=p(Y\vert X)p(x),
\]
!et
where we read $p(X\vert Y)$ as the likelihood of obtaining $X$ given $Y$.
!eblock
If we have independent events then $p(X,Y)=p(X)p(Y)$.
!split
===== Marginal Probability =====
The marginal probability is defined in terms of only of the set of variables $X,Y$. For a discrete probability we have
!bblock Discrete Probability
!bt
\[
p(X)=\sum_{i=0}^{n-1}p(X,Y=y_i)=\sum_{i=0}^{n-1}p(X\vert Y=y_i)p(Y=y_i)=\sum_{i=0}^{n-1}p(X\vert y_i)p(y_i).
\]
!et
!eblock
!split
===== Conditional Probability =====
The conditional probability, if $p(Y) > 0$, is
!bblock
!bt
\[
p(X\vert Y)= \frac{p(X,Y)}{p(Y)}=\frac{p(X,Y)}{\sum_{i=0}^{n-1}p(Y\vert X=x_i)p(x_i)}.
\]
!et
!eblock
!split
===== Bayes' Theorem =====
If we combine the conditional probability with the marginal probability and the standard product rule, we have
!bt
\[
p(X\vert Y)= \frac{p(X,Y)}{p(Y)},
\]
!et
we can rewrite this as
!bt
\[
p(X\vert Y)= \frac{p(X,Y)}{\sum_{i=0}^{n-1}p(Y\vert X=x_i)p(x_i)}=\frac{p(Y\vert X)p(X)}{\sum_{i=0}^{n-1}p(Y\vert X=x_i)p(x_i)},
\]
!et
Which is Bayes' theorem. More text to be added here by Friday September 10.
!split