added Bayes stuff

This commit is contained in:
Morten Hjorth-Jensen
2021-09-09 21:46:16 +02:00
parent 544ecb70b5
commit 146e70d762
7 changed files with 123 additions and 43 deletions
+33 -28
View File
@@ -155,6 +155,10 @@ Automatically generated HTML file from DocOnce source
None,
'example-of-usage-of-bayes-theorem'),
('Doing it correctly', 2, None, 'doing-it-correctly'),
("Bayes' Theorem and Ridge and Lasso Regression",
2,
None,
'bayes-theorem-and-ridge-and-lasso-regression'),
('Why resampling methods', 2, None, 'why-resampling-methods'),
('Resampling methods', 2, None, 'resampling-methods'),
('Resampling approaches can be computationally expensive',
@@ -323,33 +327,34 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._week36-bs041.html#interpretations-of-bayes-theorem" style="font-size: 80%;">Interpretations of Bayes' Theorem</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs042.html#example-of-usage-of-bayes-theorem" style="font-size: 80%;">Example of Usage of Bayes' theorem</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs043.html#doing-it-correctly" style="font-size: 80%;">Doing it correctly</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs047.html#why-resampling-methods" style="font-size: 80%;">Why resampling methods</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs049.html#resampling-methods" style="font-size: 80%;">Resampling methods</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs046.html#resampling-approaches-can-be-computationally-expensive" style="font-size: 80%;">Resampling approaches can be computationally expensive</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs047.html#why-resampling-methods" style="font-size: 80%;">Why resampling methods ?</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs048.html#statistical-analysis" style="font-size: 80%;">Statistical analysis</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs049.html#resampling-methods" style="font-size: 80%;">Resampling methods</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs050.html#resampling-methods-jackknife-and-bootstrap" style="font-size: 80%;">Resampling methods: Jackknife and Bootstrap</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs051.html#resampling-methods-jackknife" style="font-size: 80%;">Resampling methods: Jackknife</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs052.html#jackknife-code-example" style="font-size: 80%;">Jackknife code example</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs053.html#resampling-methods-bootstrap" style="font-size: 80%;">Resampling methods: Bootstrap</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs054.html#resampling-methods-bootstrap-background" style="font-size: 80%;">Resampling methods: Bootstrap background</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs055.html#resampling-methods-more-bootstrap-background" style="font-size: 80%;">Resampling methods: More Bootstrap background</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs056.html#resampling-methods-bootstrap-approach" style="font-size: 80%;">Resampling methods: Bootstrap approach</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs057.html#resampling-methods-bootstrap-steps" style="font-size: 80%;">Resampling methods: Bootstrap steps</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs058.html#code-example-for-the-bootstrap-method" style="font-size: 80%;">Code example for the Bootstrap method</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs059.html#various-steps-in-cross-validation" style="font-size: 80%;">Various steps in cross-validation</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs060.html#how-to-set-up-the-cross-validation-for-ridge-and-or-lasso" style="font-size: 80%;">How to set up the cross-validation for Ridge and/or Lasso</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs061.html#cross-validation-in-brief" style="font-size: 80%;">Cross-validation in brief</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs062.html#code-example-for-cross-validation-and-k-fold-cross-validation" style="font-size: 80%;">Code Example for Cross-validation and \( k \)-fold Cross-validation</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs063.html#the-bias-variance-tradeoff" style="font-size: 80%;">The bias-variance tradeoff</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs064.html#example-code-for-bias-variance-tradeoff" style="font-size: 80%;">Example code for Bias-Variance tradeoff</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs065.html#understanding-what-happens" style="font-size: 80%;">Understanding what happens</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs066.html#summing-up" style="font-size: 80%;">Summing up</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs067.html#another-example-from-scikit-learn-s-repository" style="font-size: 80%;">Another Example from Scikit-Learn's Repository</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs068.html#more-examples-on-bootstrap-and-cross-validation-and-errors" style="font-size: 80%;">More examples on bootstrap and cross-validation and errors</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs069.html#the-same-example-but-now-with-cross-validation" style="font-size: 80%;">The same example but now with cross-validation</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs070.html#cross-validation-with-ridge" style="font-size: 80%;">Cross-validation with Ridge</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs044.html#bayes-theorem-and-ridge-and-lasso-regression" style="font-size: 80%;">Bayes' Theorem and Ridge and Lasso Regression</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs048.html#why-resampling-methods" style="font-size: 80%;">Why resampling methods</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs050.html#resampling-methods" style="font-size: 80%;">Resampling methods</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs047.html#resampling-approaches-can-be-computationally-expensive" style="font-size: 80%;">Resampling approaches can be computationally expensive</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs048.html#why-resampling-methods" style="font-size: 80%;">Why resampling methods ?</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs049.html#statistical-analysis" style="font-size: 80%;">Statistical analysis</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs050.html#resampling-methods" style="font-size: 80%;">Resampling methods</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs051.html#resampling-methods-jackknife-and-bootstrap" style="font-size: 80%;">Resampling methods: Jackknife and Bootstrap</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs052.html#resampling-methods-jackknife" style="font-size: 80%;">Resampling methods: Jackknife</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs053.html#jackknife-code-example" style="font-size: 80%;">Jackknife code example</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs054.html#resampling-methods-bootstrap" style="font-size: 80%;">Resampling methods: Bootstrap</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs055.html#resampling-methods-bootstrap-background" style="font-size: 80%;">Resampling methods: Bootstrap background</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs056.html#resampling-methods-more-bootstrap-background" style="font-size: 80%;">Resampling methods: More Bootstrap background</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs057.html#resampling-methods-bootstrap-approach" style="font-size: 80%;">Resampling methods: Bootstrap approach</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs058.html#resampling-methods-bootstrap-steps" style="font-size: 80%;">Resampling methods: Bootstrap steps</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs059.html#code-example-for-the-bootstrap-method" style="font-size: 80%;">Code example for the Bootstrap method</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs060.html#various-steps-in-cross-validation" style="font-size: 80%;">Various steps in cross-validation</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs061.html#how-to-set-up-the-cross-validation-for-ridge-and-or-lasso" style="font-size: 80%;">How to set up the cross-validation for Ridge and/or Lasso</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs062.html#cross-validation-in-brief" style="font-size: 80%;">Cross-validation in brief</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs063.html#code-example-for-cross-validation-and-k-fold-cross-validation" style="font-size: 80%;">Code Example for Cross-validation and \( k \)-fold Cross-validation</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs064.html#the-bias-variance-tradeoff" style="font-size: 80%;">The bias-variance tradeoff</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs065.html#example-code-for-bias-variance-tradeoff" style="font-size: 80%;">Example code for Bias-Variance tradeoff</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs066.html#understanding-what-happens" style="font-size: 80%;">Understanding what happens</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs067.html#summing-up" style="font-size: 80%;">Summing up</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs068.html#another-example-from-scikit-learn-s-repository" style="font-size: 80%;">Another Example from Scikit-Learn's Repository</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs069.html#more-examples-on-bootstrap-and-cross-validation-and-errors" style="font-size: 80%;">More examples on bootstrap and cross-validation and errors</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs070.html#the-same-example-but-now-with-cross-validation" style="font-size: 80%;">The same example but now with cross-validation</a></li>
<!-- navigation toc: --> <li><a href="._week36-bs071.html#cross-validation-with-ridge" style="font-size: 80%;">Cross-validation with Ridge</a></li>
</ul>
</li>
@@ -408,7 +413,7 @@ MathJax.Hub.Config({
<li><a href="._week36-bs008.html">9</a></li>
<li><a href="._week36-bs009.html">10</a></li>
<li><a href="">...</a></li>
<li><a href="._week36-bs070.html">71</a></li>
<li><a href="._week36-bs071.html">72</a></li>
<li><a href="._week36-bs001.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
+17 -3
View File
@@ -1637,9 +1637,9 @@ Let us try to illustrate Bayes' theorem through an example.
<h2 id="example-of-usage-of-bayes-theorem">Example of Usage of Bayes' theorem </h2>
<p>
Let us suppose that you are undergoing a series of mammography scan in
Let us suppose that you are undergoing a series of mammography scans in
order to rule out possible breast cancer cases. We define the
sensitivity for a positive event by the variable \( X \) (it takes binary
sensitivity for a positive event by the variable \( X \). It takes binary
values with \( X=1 \) representing a positive event and \( X=0 \) being a
negative event. We reserve \( Y \) as a classification parameter for
either a negative or a postive breast cancer confirmation.
@@ -1657,7 +1657,7 @@ $$
<p>&nbsp;<br>
<p>
This obviously sounds scaring since many would conclude that if the test is positive, there is a likelihood of \( 80\% \) for having cancer.
This obviously sounds scary since many would conclude that if the test is positive, there is a likelihood of \( 80\% \) for having cancer.
It is however not correct, as the following Bayesian analysis shows.
</section>
@@ -1696,6 +1696,20 @@ That is, in case of a positive test, there is only a \( 3\% \) chance of having
</section>
<section>
<h2 id="bayes-theorem-and-ridge-and-lasso-regression">Bayes' Theorem and Ridge and Lasso Regression </h2>
<p>
Hitherto we have discussed Ridge and Lasso regression in terms of a
linear analysis. This may to many of you feel rather technical and
perhaps not that intuitive. The question is whether we can develop a
more intuitive way of understanding what Ridge and Lasso express.
<p>
Before we proceed let us perform a Ridge and OLS analysis of a polynomial fit.
</section>
<section>
<h2 id="why-resampling-methods">Why resampling methods </h2>
+21 -3
View File
@@ -175,6 +175,10 @@ div { text-align: justify; text-justify: inter-word; }
None,
'example-of-usage-of-bayes-theorem'),
('Doing it correctly', 2, None, 'doing-it-correctly'),
("Bayes' Theorem and Ridge and Lasso Regression",
2,
None,
'bayes-theorem-and-ridge-and-lasso-regression'),
('Why resampling methods', 2, None, 'why-resampling-methods'),
('Resampling methods', 2, None, 'resampling-methods'),
('Resampling approaches can be computationally expensive',
@@ -1629,9 +1633,9 @@ Let us try to illustrate Bayes' theorem through an example.
<h2 id="example-of-usage-of-bayes-theorem">Example of Usage of Bayes' theorem </h2>
<p>
Let us suppose that you are undergoing a series of mammography scan in
Let us suppose that you are undergoing a series of mammography scans in
order to rule out possible breast cancer cases. We define the
sensitivity for a positive event by the variable \( X \) (it takes binary
sensitivity for a positive event by the variable \( X \). It takes binary
values with \( X=1 \) representing a positive event and \( X=0 \) being a
negative event. We reserve \( Y \) as a classification parameter for
either a negative or a postive breast cancer confirmation.
@@ -1647,7 +1651,7 @@ p(X=1\vert Y=1) =0.8.
$$
<p>
This obviously sounds scaring since many would conclude that if the test is positive, there is a likelihood of \( 80\% \) for having cancer.
This obviously sounds scary since many would conclude that if the test is positive, there is a likelihood of \( 80\% \) for having cancer.
It is however not correct, as the following Bayesian analysis shows.
<p>
@@ -1681,6 +1685,20 @@ That is, in case of a positive test, there is only a \( 3\% \) chance of having
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="bayes-theorem-and-ridge-and-lasso-regression">Bayes' Theorem and Ridge and Lasso Regression </h2>
<p>
Hitherto we have discussed Ridge and Lasso regression in terms of a
linear analysis. This may to many of you feel rather technical and
perhaps not that intuitive. The question is whether we can develop a
more intuitive way of understanding what Ridge and Lasso express.
<p>
Before we proceed let us perform a Ridge and OLS analysis of a polynomial fit.
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="why-resampling-methods">Why resampling methods </h2>
<p>
+21 -3
View File
@@ -180,6 +180,10 @@ div { text-align: justify; text-justify: inter-word; }
None,
'example-of-usage-of-bayes-theorem'),
('Doing it correctly', 2, None, 'doing-it-correctly'),
("Bayes' Theorem and Ridge and Lasso Regression",
2,
None,
'bayes-theorem-and-ridge-and-lasso-regression'),
('Why resampling methods', 2, None, 'why-resampling-methods'),
('Resampling methods', 2, None, 'resampling-methods'),
('Resampling approaches can be computationally expensive',
@@ -1634,9 +1638,9 @@ Let us try to illustrate Bayes' theorem through an example.
<h2 id="example-of-usage-of-bayes-theorem">Example of Usage of Bayes' theorem </h2>
<p>
Let us suppose that you are undergoing a series of mammography scan in
Let us suppose that you are undergoing a series of mammography scans in
order to rule out possible breast cancer cases. We define the
sensitivity for a positive event by the variable \( X \) (it takes binary
sensitivity for a positive event by the variable \( X \). It takes binary
values with \( X=1 \) representing a positive event and \( X=0 \) being a
negative event. We reserve \( Y \) as a classification parameter for
either a negative or a postive breast cancer confirmation.
@@ -1652,7 +1656,7 @@ p(X=1\vert Y=1) =0.8.
$$
<p>
This obviously sounds scaring since many would conclude that if the test is positive, there is a likelihood of \( 80\% \) for having cancer.
This obviously sounds scary since many would conclude that if the test is positive, there is a likelihood of \( 80\% \) for having cancer.
It is however not correct, as the following Bayesian analysis shows.
<p>
@@ -1686,6 +1690,20 @@ That is, in case of a positive test, there is only a \( 3\% \) chance of having
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="bayes-theorem-and-ridge-and-lasso-regression">Bayes' Theorem and Ridge and Lasso Regression </h2>
<p>
Hitherto we have discussed Ridge and Lasso regression in terms of a
linear analysis. This may to many of you feel rather technical and
perhaps not that intuitive. The question is whether we can develop a
more intuitive way of understanding what Ridge and Lasso express.
<p>
Before we proceed let us perform a Ridge and OLS analysis of a polynomial fit.
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="why-resampling-methods">Why resampling methods </h2>
<p>
Binary file not shown.
+15 -3
View File
@@ -2052,9 +2052,9 @@
"\n",
"## Example of Usage of Bayes' theorem\n",
"\n",
"Let us suppose that you are undergoing a series of mammography scan in\n",
"Let us suppose that you are undergoing a series of mammography scans in\n",
"order to rule out possible breast cancer cases. We define the\n",
"sensitivity for a positive event by the variable $X$ (it takes binary\n",
"sensitivity for a positive event by the variable $X$. It takes binary\n",
"values with $X=1$ representing a positive event and $X=0$ being a\n",
"negative event. We reserve $Y$ as a classification parameter for\n",
"either a negative or a postive breast cancer confirmation.\n",
@@ -2077,7 +2077,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"This obviously sounds scaring since many would conclude that if the test is positive, there is a likelihood of $80\\%$ for having cancer.\n",
"This obviously sounds scary since many would conclude that if the test is positive, there is a likelihood of $80\\%$ for having cancer.\n",
"It is however not correct, as the following Bayesian analysis shows.\n",
"\n",
"## Doing it correctly\n",
@@ -2133,6 +2133,18 @@
"source": [
"That is, in case of a positive test, there is only a $3\\%$ chance of having cancer!\n",
"\n",
"\n",
"## Bayes' Theorem and Ridge and Lasso Regression\n",
"\n",
"Hitherto we have discussed Ridge and Lasso regression in terms of a\n",
"linear analysis. This may to many of you feel rather technical and\n",
"perhaps not that intuitive. The question is whether we can develop a\n",
"more intuitive way of understanding what Ridge and Lasso express.\n",
"\n",
"Before we proceed let us perform a Ridge and OLS analysis of a polynomial fit. \n",
"\n",
"\n",
"\n",
"## Why resampling methods\n",
"\n",
"Before we proceed, we need to rethink what we have been doing. In our\n",
+16 -3
View File
@@ -1270,9 +1270,9 @@ Let us try to illustrate Bayes' theorem through an example.
!split
===== Example of Usage of Bayes' theorem =====
Let us suppose that you are undergoing a series of mammography scan in
Let us suppose that you are undergoing a series of mammography scans in
order to rule out possible breast cancer cases. We define the
sensitivity for a positive event by the variable $X$ (it takes binary
sensitivity for a positive event by the variable $X$. It takes binary
values with $X=1$ representing a positive event and $X=0$ being a
negative event. We reserve $Y$ as a classification parameter for
either a negative or a postive breast cancer confirmation.
@@ -1287,7 +1287,7 @@ p(X=1\vert Y=1) =0.8.
\]
!et
This obviously sounds scaring since many would conclude that if the test is positive, there is a likelihood of $80\%$ for having cancer.
This obviously sounds scary since many would conclude that if the test is positive, there is a likelihood of $80\%$ for having cancer.
It is however not correct, as the following Bayesian analysis shows.
!split
@@ -1318,6 +1318,19 @@ p(Y=1\vert X=1)=\frac{p(X=1\vert Y=1)p(Y=1)}{p(X=1\vert Y=1)p(Y=1)+p(X=1\vert Y=
!et
That is, in case of a positive test, there is only a $3\%$ chance of having cancer!
!split
===== Bayes' Theorem and Ridge and Lasso Regression =====
Hitherto we have discussed Ridge and Lasso regression in terms of a
linear analysis. This may to many of you feel rather technical and
perhaps not that intuitive. The question is whether we can develop a
more intuitive way of understanding what Ridge and Lasso express.
Before we proceed let us perform a Ridge and OLS analysis of a polynomial fit.
!split
===== Why resampling methods =====