lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

updating

Browse Source
This commit is contained in:
Morten Hjorth-Jensen
2021-09-09 21:25:22 +02:00
parent 125a2094ae
commit 544ecb70b5
7 changed files with 441 additions and 39 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/pub/week36/html/week36-bs.html
+40 -28
View File
@@ -146,6 +146,15 @@ Automatically generated HTML file from DocOnce source
('Marginal Probability', 2, None, 'marginal-probability'),
('Conditional Probability', 2, None, 'conditional-probability'),
("Bayes' Theorem", 2, None, 'bayes-theorem'),
("Interpretations of Bayes' Theorem",
2,
None,
'interpretations-of-bayes-theorem'),
("Example of Usage of Bayes' theorem",
2,
None,
'example-of-usage-of-bayes-theorem'),
('Doing it correctly', 2, None, 'doing-it-correctly'),
('Why resampling methods', 2, None, 'why-resampling-methods'),
('Resampling methods', 2, None, 'resampling-methods'),
('Resampling approaches can be computationally expensive',
@@ -311,33 +320,36 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._week36-bs038.html#marginal-probability" style="font-size: 80%;">Marginal Probability</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs039.html#conditional-probability" style="font-size: 80%;">Conditional Probability</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs040.html#bayes-theorem" style="font-size: 80%;">Bayes' Theorem</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs044.html#why-resampling-methods" style="font-size: 80%;">Why resampling methods</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs046.html#resampling-methods" style="font-size: 80%;">Resampling methods</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs043.html#resampling-approaches-can-be-computationally-expensive" style="font-size: 80%;">Resampling approaches can be computationally expensive</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs044.html#why-resampling-methods" style="font-size: 80%;">Why resampling methods ?</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs045.html#statistical-analysis" style="font-size: 80%;">Statistical analysis</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs046.html#resampling-methods" style="font-size: 80%;">Resampling methods</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs047.html#resampling-methods-jackknife-and-bootstrap" style="font-size: 80%;">Resampling methods: Jackknife and Bootstrap</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs048.html#resampling-methods-jackknife" style="font-size: 80%;">Resampling methods: Jackknife</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs049.html#jackknife-code-example" style="font-size: 80%;">Jackknife code example</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs050.html#resampling-methods-bootstrap" style="font-size: 80%;">Resampling methods: Bootstrap</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs051.html#resampling-methods-bootstrap-background" style="font-size: 80%;">Resampling methods: Bootstrap background</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs052.html#resampling-methods-more-bootstrap-background" style="font-size: 80%;">Resampling methods: More Bootstrap background</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs053.html#resampling-methods-bootstrap-approach" style="font-size: 80%;">Resampling methods: Bootstrap approach</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs054.html#resampling-methods-bootstrap-steps" style="font-size: 80%;">Resampling methods: Bootstrap steps</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs055.html#code-example-for-the-bootstrap-method" style="font-size: 80%;">Code example for the Bootstrap method</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs056.html#various-steps-in-cross-validation" style="font-size: 80%;">Various steps in cross-validation</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs057.html#how-to-set-up-the-cross-validation-for-ridge-and-or-lasso" style="font-size: 80%;">How to set up the cross-validation for Ridge and/or Lasso</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs058.html#cross-validation-in-brief" style="font-size: 80%;">Cross-validation in brief</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs059.html#code-example-for-cross-validation-and-k-fold-cross-validation" style="font-size: 80%;">Code Example for Cross-validation and \( k \)-fold Cross-validation</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs060.html#the-bias-variance-tradeoff" style="font-size: 80%;">The bias-variance tradeoff</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs061.html#example-code-for-bias-variance-tradeoff" style="font-size: 80%;">Example code for Bias-Variance tradeoff</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs062.html#understanding-what-happens" style="font-size: 80%;">Understanding what happens</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs063.html#summing-up" style="font-size: 80%;">Summing up</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs064.html#another-example-from-scikit-learn-s-repository" style="font-size: 80%;">Another Example from Scikit-Learn's Repository</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs065.html#more-examples-on-bootstrap-and-cross-validation-and-errors" style="font-size: 80%;">More examples on bootstrap and cross-validation and errors</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs066.html#the-same-example-but-now-with-cross-validation" style="font-size: 80%;">The same example but now with cross-validation</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs067.html#cross-validation-with-ridge" style="font-size: 80%;">Cross-validation with Ridge</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs041.html#interpretations-of-bayes-theorem" style="font-size: 80%;">Interpretations of Bayes' Theorem</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs042.html#example-of-usage-of-bayes-theorem" style="font-size: 80%;">Example of Usage of Bayes' theorem</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs043.html#doing-it-correctly" style="font-size: 80%;">Doing it correctly</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs047.html#why-resampling-methods" style="font-size: 80%;">Why resampling methods</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs049.html#resampling-methods" style="font-size: 80%;">Resampling methods</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs046.html#resampling-approaches-can-be-computationally-expensive" style="font-size: 80%;">Resampling approaches can be computationally expensive</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs047.html#why-resampling-methods" style="font-size: 80%;">Why resampling methods ?</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs048.html#statistical-analysis" style="font-size: 80%;">Statistical analysis</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs049.html#resampling-methods" style="font-size: 80%;">Resampling methods</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs050.html#resampling-methods-jackknife-and-bootstrap" style="font-size: 80%;">Resampling methods: Jackknife and Bootstrap</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs051.html#resampling-methods-jackknife" style="font-size: 80%;">Resampling methods: Jackknife</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs052.html#jackknife-code-example" style="font-size: 80%;">Jackknife code example</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs053.html#resampling-methods-bootstrap" style="font-size: 80%;">Resampling methods: Bootstrap</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs054.html#resampling-methods-bootstrap-background" style="font-size: 80%;">Resampling methods: Bootstrap background</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs055.html#resampling-methods-more-bootstrap-background" style="font-size: 80%;">Resampling methods: More Bootstrap background</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs056.html#resampling-methods-bootstrap-approach" style="font-size: 80%;">Resampling methods: Bootstrap approach</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs057.html#resampling-methods-bootstrap-steps" style="font-size: 80%;">Resampling methods: Bootstrap steps</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs058.html#code-example-for-the-bootstrap-method" style="font-size: 80%;">Code example for the Bootstrap method</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs059.html#various-steps-in-cross-validation" style="font-size: 80%;">Various steps in cross-validation</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs060.html#how-to-set-up-the-cross-validation-for-ridge-and-or-lasso" style="font-size: 80%;">How to set up the cross-validation for Ridge and/or Lasso</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs061.html#cross-validation-in-brief" style="font-size: 80%;">Cross-validation in brief</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs062.html#code-example-for-cross-validation-and-k-fold-cross-validation" style="font-size: 80%;">Code Example for Cross-validation and \( k \)-fold Cross-validation</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs063.html#the-bias-variance-tradeoff" style="font-size: 80%;">The bias-variance tradeoff</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs064.html#example-code-for-bias-variance-tradeoff" style="font-size: 80%;">Example code for Bias-Variance tradeoff</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs065.html#understanding-what-happens" style="font-size: 80%;">Understanding what happens</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs066.html#summing-up" style="font-size: 80%;">Summing up</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs067.html#another-example-from-scikit-learn-s-repository" style="font-size: 80%;">Another Example from Scikit-Learn's Repository</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs068.html#more-examples-on-bootstrap-and-cross-validation-and-errors" style="font-size: 80%;">More examples on bootstrap and cross-validation and errors</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs069.html#the-same-example-but-now-with-cross-validation" style="font-size: 80%;">The same example but now with cross-validation</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs070.html#cross-validation-with-ridge" style="font-size: 80%;">Cross-validation with Ridge</a></li>
</ul>
</li>
@@ -396,7 +408,7 @@ MathJax.Hub.Config({
<li><a href="._week36-bs008.html">9</a></li>
<li><a href="._week36-bs009.html">10</a></li>
<li><a href="">...</a></li>
<li><a href="._week36-bs067.html">68</a></li>
<li><a href="._week36-bs070.html">71</a></li>
<li><a href="._week36-bs001.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/week36/html/week36-reveal.html
+82 -2
View File
@@ -1567,7 +1567,7 @@ If we have independent events then \( p(X,Y)=p(X)p(Y) \).
<p>
The marginal probability is defined in terms of only of the set of variables \( X,Y \). For a discrete probability we have
<div class="alert alert-block alert-block alert-text-normal">
<b>Discrete Probability</b>
<b></b>
<p>&nbsp;<br>
$$
p(X)=\sum_{i=0}^{n-1}p(X,Y=y_i)=\sum_{i=0}^{n-1}p(X\vert Y=y_i)p(Y=y_i)=\sum_{i=0}^{n-1}p(X\vert y_i)p(y_i).
@@ -1612,7 +1612,87 @@ p(X\vert Y)= \frac{p(X,Y)}{\sum_{i=0}^{n-1}p(Y\vert X=x_i)p(x_i)}=\frac{p(Y\vert
$$
<p>&nbsp;<br>
which is Bayes' theorem. More text to be added here by Friday September 10.
which is Bayes' theorem. It allows us to evaluate the uncertainty in in \( X \) after we have observed \( Y \). We can easily interchange \( X \) with \( Y \).
</section>
<section>
<h2 id="interpretations-of-bayes-theorem">Interpretations of Bayes' Theorem </h2>
<p>
The quantity \( p(Y\vert X) \) on the right-hand side of the theorem is
evaluated for the observed data \( Y \) and can be viewed as a function of
the parameter space represented by \( X \). This function is not
necesseraly normalized and is normally called the likelihood function.
<p>
The function \( p(X) \) on the right hand side is called the prior while the function on the left hand side is the called the posterior probability. The denominator on the right hand side serves as a normalization factor for the posterior distribution.
<p>
Let us try to illustrate Bayes' theorem through an example.
</section>
<section>
<h2 id="example-of-usage-of-bayes-theorem">Example of Usage of Bayes' theorem </h2>
<p>
Let us suppose that you are undergoing a series of mammography scan in
order to rule out possible breast cancer cases. We define the
sensitivity for a positive event by the variable \( X \) (it takes binary
values with \( X=1 \) representing a positive event and \( X=0 \) being a
negative event. We reserve \( Y \) as a classification parameter for
either a negative or a postive breast cancer confirmation.
<p>
We let \( Y=1 \) represent the the case of having breast cancer and \( Y=0 \) as not.
<p>
Let us assume that if you have breast cancer, the test will be positive with a probability of \( 0.8 \), that is we have
<p>&nbsp;<br>
$$
p(X=1\vert Y=1) =0.8.
$$
<p>&nbsp;<br>
<p>
This obviously sounds scaring since many would conclude that if the test is positive, there is a likelihood of \( 80\% \) for having cancer.
It is however not correct, as the following Bayesian analysis shows.
</section>
<section>
<h2 id="doing-it-correctly">Doing it correctly </h2>
<p>
If we look at various national surveys on breast cancer, the general likelihood of developing breast cancer is a very small number.
Let us assume that the prior probability in the population as a whole is
<p>&nbsp;<br>
$$
p(Y=1) =0.004.
$$
<p>&nbsp;<br>
<p>
We need also to account for the fact that the test may produce a false positive result (false alarm). Let us here assume that we have
<p>&nbsp;<br>
$$
p(X=1\vert Y=0) =0.1.
$$
<p>&nbsp;<br>
<p>
Using Bayes' theorem we can then find the posterior probability that the person has breast cancer in case of a positive test, that is we can compute
<p>&nbsp;<br>
$$
p(Y=1\vert X=1)=\frac{p(X=1\vert Y=1)p(Y=1)}{p(X=1\vert Y=1)p(Y=1)+p(X=1\vert Y=0)p(Y=0)}=\frac{0.8\times 0.004}{0.8\times 0.004+0.1\times 0.996}=0.031.
$$
<p>&nbsp;<br>
That is, in case of a positive test, there is only a \( 3\% \) chance of having cancer!
</section>
doc/pub/week36/html/week36-solarized.html
+83 -2
View File
@@ -166,6 +166,15 @@ div { text-align: justify; text-justify: inter-word; }
('Marginal Probability', 2, None, 'marginal-probability'),
('Conditional Probability', 2, None, 'conditional-probability'),
("Bayes' Theorem", 2, None, 'bayes-theorem'),
("Interpretations of Bayes' Theorem",
2,
None,
'interpretations-of-bayes-theorem'),
("Example of Usage of Bayes' theorem",
2,
None,
'example-of-usage-of-bayes-theorem'),
('Doing it correctly', 2, None, 'doing-it-correctly'),
('Why resampling methods', 2, None, 'why-resampling-methods'),
('Resampling methods', 2, None, 'resampling-methods'),
('Resampling approaches can be computationally expensive',
@@ -1554,7 +1563,7 @@ If we have independent events then \( p(X,Y)=p(X)p(Y) \).
<p>
The marginal probability is defined in terms of only of the set of variables \( X,Y \). For a discrete probability we have
<div class="alert alert-block alert-block alert-text-normal">
<b>Discrete Probability</b>
<b></b>
<p>
$$
p(X)=\sum_{i=0}^{n-1}p(X,Y=y_i)=\sum_{i=0}^{n-1}p(X\vert Y=y_i)p(Y=y_i)=\sum_{i=0}^{n-1}p(X\vert y_i)p(y_i).
@@ -1595,7 +1604,79 @@ $$
p(X\vert Y)= \frac{p(X,Y)}{\sum_{i=0}^{n-1}p(Y\vert X=x_i)p(x_i)}=\frac{p(Y\vert X)p(X)}{\sum_{i=0}^{n-1}p(Y\vert X=x_i)p(x_i)},
$$
which is Bayes' theorem. More text to be added here by Friday September 10.
which is Bayes' theorem. It allows us to evaluate the uncertainty in in \( X \) after we have observed \( Y \). We can easily interchange \( X \) with \( Y \).
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="interpretations-of-bayes-theorem">Interpretations of Bayes' Theorem </h2>
<p>
The quantity \( p(Y\vert X) \) on the right-hand side of the theorem is
evaluated for the observed data \( Y \) and can be viewed as a function of
the parameter space represented by \( X \). This function is not
necesseraly normalized and is normally called the likelihood function.
<p>
The function \( p(X) \) on the right hand side is called the prior while the function on the left hand side is the called the posterior probability. The denominator on the right hand side serves as a normalization factor for the posterior distribution.
<p>
Let us try to illustrate Bayes' theorem through an example.
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="example-of-usage-of-bayes-theorem">Example of Usage of Bayes' theorem </h2>
<p>
Let us suppose that you are undergoing a series of mammography scan in
order to rule out possible breast cancer cases. We define the
sensitivity for a positive event by the variable \( X \) (it takes binary
values with \( X=1 \) representing a positive event and \( X=0 \) being a
negative event. We reserve \( Y \) as a classification parameter for
either a negative or a postive breast cancer confirmation.
<p>
We let \( Y=1 \) represent the the case of having breast cancer and \( Y=0 \) as not.
<p>
Let us assume that if you have breast cancer, the test will be positive with a probability of \( 0.8 \), that is we have
$$
p(X=1\vert Y=1) =0.8.
$$
<p>
This obviously sounds scaring since many would conclude that if the test is positive, there is a likelihood of \( 80\% \) for having cancer.
It is however not correct, as the following Bayesian analysis shows.
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="doing-it-correctly">Doing it correctly </h2>
<p>
If we look at various national surveys on breast cancer, the general likelihood of developing breast cancer is a very small number.
Let us assume that the prior probability in the population as a whole is
$$
p(Y=1) =0.004.
$$
<p>
We need also to account for the fact that the test may produce a false positive result (false alarm). Let us here assume that we have
$$
p(X=1\vert Y=0) =0.1.
$$
<p>
Using Bayes' theorem we can then find the posterior probability that the person has breast cancer in case of a positive test, that is we can compute
$$
p(Y=1\vert X=1)=\frac{p(X=1\vert Y=1)p(Y=1)}{p(X=1\vert Y=1)p(Y=1)+p(X=1\vert Y=0)p(Y=0)}=\frac{0.8\times 0.004}{0.8\times 0.004+0.1\times 0.996}=0.031.
$$
That is, in case of a positive test, there is only a \( 3\% \) chance of having cancer!
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
doc/pub/week36/html/week36.html
+83 -2
View File
@@ -171,6 +171,15 @@ div { text-align: justify; text-justify: inter-word; }
('Marginal Probability', 2, None, 'marginal-probability'),
('Conditional Probability', 2, None, 'conditional-probability'),
("Bayes' Theorem", 2, None, 'bayes-theorem'),
("Interpretations of Bayes' Theorem",
2,
None,
'interpretations-of-bayes-theorem'),
("Example of Usage of Bayes' theorem",
2,
None,
'example-of-usage-of-bayes-theorem'),
('Doing it correctly', 2, None, 'doing-it-correctly'),
('Why resampling methods', 2, None, 'why-resampling-methods'),
('Resampling methods', 2, None, 'resampling-methods'),
('Resampling approaches can be computationally expensive',
@@ -1559,7 +1568,7 @@ If we have independent events then \( p(X,Y)=p(X)p(Y) \).
<p>
The marginal probability is defined in terms of only of the set of variables \( X,Y \). For a discrete probability we have
<div class="alert alert-block alert-block alert-text-normal">
<b>Discrete Probability</b>
<b></b>
<p>
$$
p(X)=\sum_{i=0}^{n-1}p(X,Y=y_i)=\sum_{i=0}^{n-1}p(X\vert Y=y_i)p(Y=y_i)=\sum_{i=0}^{n-1}p(X\vert y_i)p(y_i).
@@ -1600,7 +1609,79 @@ $$
p(X\vert Y)= \frac{p(X,Y)}{\sum_{i=0}^{n-1}p(Y\vert X=x_i)p(x_i)}=\frac{p(Y\vert X)p(X)}{\sum_{i=0}^{n-1}p(Y\vert X=x_i)p(x_i)},
$$
which is Bayes' theorem. More text to be added here by Friday September 10.
which is Bayes' theorem. It allows us to evaluate the uncertainty in in \( X \) after we have observed \( Y \). We can easily interchange \( X \) with \( Y \).
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="interpretations-of-bayes-theorem">Interpretations of Bayes' Theorem </h2>
<p>
The quantity \( p(Y\vert X) \) on the right-hand side of the theorem is
evaluated for the observed data \( Y \) and can be viewed as a function of
the parameter space represented by \( X \). This function is not
necesseraly normalized and is normally called the likelihood function.
<p>
The function \( p(X) \) on the right hand side is called the prior while the function on the left hand side is the called the posterior probability. The denominator on the right hand side serves as a normalization factor for the posterior distribution.
<p>
Let us try to illustrate Bayes' theorem through an example.
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="example-of-usage-of-bayes-theorem">Example of Usage of Bayes' theorem </h2>
<p>
Let us suppose that you are undergoing a series of mammography scan in
order to rule out possible breast cancer cases. We define the
sensitivity for a positive event by the variable \( X \) (it takes binary
values with \( X=1 \) representing a positive event and \( X=0 \) being a
negative event. We reserve \( Y \) as a classification parameter for
either a negative or a postive breast cancer confirmation.
<p>
We let \( Y=1 \) represent the the case of having breast cancer and \( Y=0 \) as not.
<p>
Let us assume that if you have breast cancer, the test will be positive with a probability of \( 0.8 \), that is we have
$$
p(X=1\vert Y=1) =0.8.
$$
<p>
This obviously sounds scaring since many would conclude that if the test is positive, there is a likelihood of \( 80\% \) for having cancer.
It is however not correct, as the following Bayesian analysis shows.
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="doing-it-correctly">Doing it correctly </h2>
<p>
If we look at various national surveys on breast cancer, the general likelihood of developing breast cancer is a very small number.
Let us assume that the prior probability in the population as a whole is
$$
p(Y=1) =0.004.
$$
<p>
We need also to account for the fact that the test may produce a false positive result (false alarm). Let us here assume that we have
$$
p(X=1\vert Y=0) =0.1.
$$
<p>
Using Bayes' theorem we can then find the posterior probability that the person has breast cancer in case of a positive test, that is we can compute
$$
p(Y=1\vert X=1)=\frac{p(X=1\vert Y=1)p(Y=1)}{p(X=1\vert Y=1)p(Y=1)+p(X=1\vert Y=0)p(Y=0)}=\frac{0.8\times 0.004}{0.8\times 0.004+0.1\times 0.996}=0.031.
$$
That is, in case of a positive test, there is only a \( 3\% \) chance of having cancer!
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
doc/pub/week36/ipynb/ipynb-week36-src.tar.gz
BIN
View File
Binary file not shown.
doc/pub/week36/ipynb/week36.ipynb
+92 -3
View File
@@ -1969,8 +1969,7 @@
"\n",
"## Marginal Probability\n",
"\n",
"The marginal probability is defined in terms of only of the set of variables $X,Y$. For a discrete probability we have\n",
"**Discrete Probability.**"
"The marginal probability is defined in terms of only of the set of variables $X,Y$. For a discrete probability we have"
]
},
{
@@ -2038,11 +2037,101 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"which is Bayes' theorem. More text to be added here by Friday September 10.\n",
"which is Bayes' theorem. It allows us to evaluate the uncertainty in in $X$ after we have observed $Y$. We can easily interchange $X$ with $Y$. \n",
"\n",
"## Interpretations of Bayes' Theorem\n",
"\n",
"The quantity $p(Y\\vert X)$ on the right-hand side of the theorem is\n",
"evaluated for the observed data $Y$ and can be viewed as a function of\n",
"the parameter space represented by $X$. This function is not\n",
"necesseraly normalized and is normally called the likelihood function.\n",
"\n",
"The function $p(X)$ on the right hand side is called the prior while the function on the left hand side is the called the posterior probability. The denominator on the right hand side serves as a normalization factor for the posterior distribution.\n",
"\n",
"Let us try to illustrate Bayes' theorem through an example.\n",
"\n",
"## Example of Usage of Bayes' theorem\n",
"\n",
"Let us suppose that you are undergoing a series of mammography scan in\n",
"order to rule out possible breast cancer cases. We define the\n",
"sensitivity for a positive event by the variable $X$ (it takes binary\n",
"values with $X=1$ representing a positive event and $X=0$ being a\n",
"negative event. We reserve $Y$ as a classification parameter for\n",
"either a negative or a postive breast cancer confirmation.\n",
"\n",
"We let $Y=1$ represent the the case of having breast cancer and $Y=0$ as not.\n",
"\n",
"Let us assume that if you have breast cancer, the test will be positive with a probability of $0.8$, that is we have"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$$\n",
"p(X=1\\vert Y=1) =0.8.\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This obviously sounds scaring since many would conclude that if the test is positive, there is a likelihood of $80\\%$ for having cancer.\n",
"It is however not correct, as the following Bayesian analysis shows.\n",
"\n",
"## Doing it correctly\n",
"\n",
"If we look at various national surveys on breast cancer, the general likelihood of developing breast cancer is a very small number.\n",
"Let us assume that the prior probability in the population as a whole is"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$$\n",
"p(Y=1) =0.004.\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We need also to account for the fact that the test may produce a false positive result (false alarm). Let us here assume that we have"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$$\n",
"p(X=1\\vert Y=0) =0.1.\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Using Bayes' theorem we can then find the posterior probability that the person has breast cancer in case of a positive test, that is we can compute"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$$\n",
"p(Y=1\\vert X=1)=\\frac{p(X=1\\vert Y=1)p(Y=1)}{p(X=1\\vert Y=1)p(Y=1)+p(X=1\\vert Y=0)p(Y=0)}=\\frac{0.8\\times 0.004}{0.8\\times 0.004+0.1\\times 0.996}=0.031.\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"That is, in case of a positive test, there is only a $3\\%$ chance of having cancer!\n",
"\n",
"## Why resampling methods\n",
"\n",
doc/src/week36/week36.do.txt
+61 -2
View File
@@ -1215,7 +1215,7 @@ If we have independent events then $p(X,Y)=p(X)p(Y)$.
===== Marginal Probability =====
The marginal probability is defined in terms of only of the set of variables $X,Y$. For a discrete probability we have
!bblock Discrete Probability
!bblock
!bt
\[
p(X)=\sum_{i=0}^{n-1}p(X,Y=y_i)=\sum_{i=0}^{n-1}p(X\vert Y=y_i)p(Y=y_i)=\sum_{i=0}^{n-1}p(X\vert y_i)p(y_i).
@@ -1253,11 +1253,70 @@ which we can rewrite as
p(X\vert Y)= \frac{p(X,Y)}{\sum_{i=0}^{n-1}p(Y\vert X=x_i)p(x_i)}=\frac{p(Y\vert X)p(X)}{\sum_{i=0}^{n-1}p(Y\vert X=x_i)p(x_i)},
\]
!et
which is Bayes' theorem. More text to be added here by Friday September 10.
which is Bayes' theorem. It allows us to evaluate the uncertainty in in $X$ after we have observed $Y$. We can easily interchange $X$ with $Y$.
!split
===== Interpretations of Bayes' Theorem =====
The quantity $p(Y\vert X)$ on the right-hand side of the theorem is
evaluated for the observed data $Y$ and can be viewed as a function of
the parameter space represented by $X$. This function is not
necesseraly normalized and is normally called the likelihood function.
The function $p(X)$ on the right hand side is called the prior while the function on the left hand side is the called the posterior probability. The denominator on the right hand side serves as a normalization factor for the posterior distribution.
Let us try to illustrate Bayes' theorem through an example.
!split
===== Example of Usage of Bayes' theorem =====
Let us suppose that you are undergoing a series of mammography scan in
order to rule out possible breast cancer cases. We define the
sensitivity for a positive event by the variable $X$ (it takes binary
values with $X=1$ representing a positive event and $X=0$ being a
negative event. We reserve $Y$ as a classification parameter for
either a negative or a postive breast cancer confirmation.
We let $Y=1$ represent the the case of having breast cancer and $Y=0$ as not.
Let us assume that if you have breast cancer, the test will be positive with a probability of $0.8$, that is we have
!bt
\[
p(X=1\vert Y=1) =0.8.
\]
!et
This obviously sounds scaring since many would conclude that if the test is positive, there is a likelihood of $80\%$ for having cancer.
It is however not correct, as the following Bayesian analysis shows.
!split
===== Doing it correctly =====
If we look at various national surveys on breast cancer, the general likelihood of developing breast cancer is a very small number.
Let us assume that the prior probability in the population as a whole is
!bt
\[
p(Y=1) =0.004.
\]
!et
We need also to account for the fact that the test may produce a false positive result (false alarm). Let us here assume that we have
!bt
\[
p(X=1\vert Y=0) =0.1.
\]
!et
Using Bayes' theorem we can then find the posterior probability that the person has breast cancer in case of a positive test, that is we can compute
!bt
\[
p(Y=1\vert X=1)=\frac{p(X=1\vert Y=1)p(Y=1)}{p(X=1\vert Y=1)p(Y=1)+p(X=1\vert Y=0)p(Y=0)}=\frac{0.8\times 0.004}{0.8\times 0.004+0.1\times 0.996}=0.031.
\]
!et
That is, in case of a positive test, there is only a $3\%$ chance of having cancer!
!split
===== Why resampling methods =====