lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update book

Browse Source
This commit is contained in:
Morten Hjorth-Jensen
2022-08-23 11:19:05 +02:00
parent 602b1c6ee0
commit 398fba5b97
233 changed files with 5964 additions and 6430 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/LectureNotes/_build/html/chapter3.html
+338 -249
View File
@@ -7,8 +7,8 @@
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>5. Resampling Methods &#8212; Applied Data Analysis and Machine Learning</title>
<link href="_static/css/theme.css" rel="stylesheet" />
<link href="_static/css/index.c5995385ac14fb8791e8eb36b4908be2.css" rel="stylesheet" />
<link href="_static/css/theme.css" rel="stylesheet">
<link href="_static/css/index.ff1ffe594081f20da1ef19478df9384b.css" rel="stylesheet">
<link rel="stylesheet"
@@ -31,7 +31,7 @@
<link rel="stylesheet" type="text/css" href="_static/panels-main.c949a650a448cc0ae9fd3441c0e17fb0.css" />
<link rel="stylesheet" type="text/css" href="_static/panels-variables.06eb56fa6e07937060861dad626602ad.css" />
<link rel="preload" as="script" href="_static/js/index.1c5a1a01449ed65a7b51.js">
<link rel="preload" as="script" href="_static/js/index.be7d3bbb2ef33a8344ce.js">
<script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
<script src="_static/jquery.js"></script>
@@ -41,22 +41,24 @@
<script src="_static/clipboard.min.js"></script>
<script src="_static/copybutton.js"></script>
<script>var togglebuttonSelector = '.toggle, .admonition.dropdown, .tag_hide_input div.cell_input, .tag_hide-input div.cell_input, .tag_hide_output div.cell_output, .tag_hide-output div.cell_output, .tag_hide_cell.cell, .tag_hide-cell.cell';</script>
<script src="_static/sphinx-book-theme.12a9622fbb08dcb3a2a40b2c02b83a57.js"></script>
<script async="async" src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
<script>window.MathJax = {"options": {"processHtmlClass": "tex2jax_process|mathjax_process|math|output_area"}}</script>
<script async="async" src="https://unpkg.com/thebe@0.5.1/lib/index.js"></script>
<script>
const thebe_selector = ".thebe"
const thebe_selector_input = "pre"
const thebe_selector_output = ".output"
</script>
<script src="_static/sphinx-book-theme.d59cb220de22ca1c485ebbdc042f0030.js"></script>
<script>const THEBE_JS_URL = "https://unpkg.com/thebe@0.8.2/lib/index.js"
const thebe_selector = ".thebe,.cell"
const thebe_selector_input = "pre"
const thebe_selector_output = ".output, .cell_output"
</script>
<script async="async" src="_static/sphinx-thebe.js"></script>
<script defer="defer" src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
<script>window.MathJax = {"options": {"processHtmlClass": "tex2jax_process|mathjax_process|math|output_area"}}</script>
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="6. Logistic Regression" href="chapter4.html" />
<link rel="prev" title="4. Ridge and Lasso Regression" href="chapter2.html" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<meta name="docsearch:language" content="en" />
<meta name="docsearch:language" content="None">
<!-- Google Analytics -->
</head>
<body data-spy="scroll" data-target="#bd-toc-nav" data-offset="80">
@@ -91,11 +93,11 @@
<ul class="nav bd-sidenav">
<li class="toctree-l1">
<a class="reference internal" href="intro.html">
Applied Data Analysis and Machine Learning, FYS-STK3155/4155 at the University of Oslo, Norway
Applied Data Analysis and Machine Learning
</a>
</li>
</ul>
<p class="caption" role="heading">
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
About the course
</span>
@@ -117,7 +119,7 @@
</a>
</li>
</ul>
<p class="caption" role="heading">
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
Review of Statistics with Resampling Techniques and Linear Algebra
</span>
@@ -134,7 +136,7 @@
</a>
</li>
</ul>
<p class="caption" role="heading">
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
From Regression to Support Vector Machines
</span>
@@ -171,7 +173,7 @@
</a>
</li>
</ul>
<p class="caption" role="heading">
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
Decision Trees, Ensemble Methods and Boosting
</span>
@@ -188,7 +190,7 @@
</a>
</li>
</ul>
<p class="caption" role="heading">
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
Dimensionality Reduction
</span>
@@ -199,8 +201,13 @@
11. Basic ideas of the Principal Component Analysis (PCA)
</a>
</li>
<li class="toctree-l1">
<a class="reference internal" href="clustering.html">
12. Clustering and Unsupervised Learning
</a>
</li>
</ul>
<p class="caption" role="heading">
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
Deep Learning Methods
</span>
@@ -208,17 +215,27 @@
<ul class="nav bd-sidenav">
<li class="toctree-l1">
<a class="reference internal" href="chapter9.html">
12. Neural networks
13. Neural networks
</a>
</li>
<li class="toctree-l1">
<a class="reference internal" href="chapter10.html">
13. Building a Feed Forward Neural Network
14. Building a Feed Forward Neural Network
</a>
</li>
<li class="toctree-l1">
<a class="reference internal" href="chapter11.html">
14. Solving Differential Equations with Deep Learning
15. Solving Differential Equations with Deep Learning
</a>
</li>
<li class="toctree-l1">
<a class="reference internal" href="chapter12.html">
16. Convolutional Neural Networks
</a>
</li>
<li class="toctree-l1">
<a class="reference internal" href="chapter13.html">
17. Recurrent neural networks: Overarching view
</a>
</li>
</ul>
@@ -267,7 +284,7 @@
data-placement="left">.ipynb</button></a>
<!-- Download PDF via print -->
<button type="button" id="download-print" class="btn btn-secondary topbarbtn" title="Print to PDF"
onClick="window.print()" data-toggle="tooltip" data-placement="left">.pdf</button>
onclick="printPdf(this)" data-toggle="tooltip" data-placement="left">.pdf</button>
</div>
</div>
@@ -285,7 +302,7 @@
</div>
<!-- Table of contents -->
<div class="d-none d-md-block col-md-2 bd-toc show">
<div class="d-none d-md-block col-md-2 bd-toc show noprint">
<div class="tocsection onthispage pt-5 pb-3">
<i class="fas fa-list"></i> Contents
@@ -379,7 +396,104 @@
</div>
<div id="main-content" class="row">
<div class="col-12 col-md-9 pl-md-3 pr-md-0">
<!-- Table of contents that is only displayed when printing the page -->
<div id="jb-print-docs-body" class="onlyprint">
<h1>Resampling Methods</h1>
<!-- Table of contents -->
<div id="print-main-content">
<div id="jb-print-toc">
<div>
<h2> Contents </h2>
</div>
<nav aria-label="Page">
<ul class="visible nav section-nav flex-column">
<li class="toc-h2 nav-item toc-entry">
<a class="reference internal nav-link" href="#introduction">
5.1. Introduction
</a>
</li>
<li class="toc-h2 nav-item toc-entry">
<a class="reference internal nav-link" href="#reminder-on-statistics">
5.2. Reminder on Statistics
</a>
</li>
<li class="toc-h2 nav-item toc-entry">
<a class="reference internal nav-link" href="#id1">
5.3. Resampling methods
</a>
<ul class="nav section-nav flex-column">
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#bootstrap">
5.3.1. Bootstrap
</a>
</li>
</ul>
</li>
<li class="toc-h2 nav-item toc-entry">
<a class="reference internal nav-link" href="#the-bias-variance-tradeoff">
5.4. The bias-variance tradeoff
</a>
</li>
<li class="toc-h2 nav-item toc-entry">
<a class="reference internal nav-link" href="#cross-validation">
5.5. Cross-validation
</a>
</li>
<li class="toc-h2 nav-item toc-entry">
<a class="reference internal nav-link" href="#more-on-rescaling-data">
5.6. More on Rescaling data
</a>
</li>
<li class="toc-h2 nav-item toc-entry">
<a class="reference internal nav-link" href="#more-complicated-example-the-ising-model">
5.7. More complicated Example: The Ising model
</a>
</li>
<li class="toc-h2 nav-item toc-entry">
<a class="reference internal nav-link" href="#exercises-and-projects">
5.8. Exercises and Projects
</a>
<ul class="nav section-nav flex-column">
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#exercise-ordinary-least-square-ols-on-the-franke-function">
5.8.1. Exercise: Ordinary Least Square (OLS) on the Franke function
</a>
</li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#exercise-bias-variance-trade-off-and-resampling-techniques">
5.8.2. Exercise: Bias-variance trade-off and resampling techniques
</a>
</li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#exercise-cross-validation-as-resampling-techniques-adding-more-complexity">
5.8.3. Exercise: Cross-validation as resampling techniques, adding more complexity
</a>
</li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#exercise-ridge-regression-on-the-franke-function-with-resampling">
5.8.4. Exercise: Ridge Regression on the Franke function with resampling
</a>
</li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#exercise-lasso-regression-on-the-franke-function-with-resampling">
5.8.5. Exercise: Lasso Regression on the Franke function with resampling
</a>
</li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#exercise-analysis-of-real-data">
5.8.6. Exercise: Analysis of real data
</a>
</li>
</ul>
</li>
</ul>
</nav>
</div>
</div>
</div>
<div>
<div class="tex2jax_ignore mathjax_ignore section" id="resampling-methods">
@@ -643,10 +757,10 @@ number <span class="math notranslate nohighlight">\(i\)</span> is left out. Usin
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Runtime: 0.184895 sec
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Runtime: 0.0896981 sec
Jackknife Statistics :
original bias std. error
100.109 100.099 0.148768
100.213 100.203 0.148564
</pre></div>
</div>
</div>
@@ -865,7 +979,7 @@ theorem.</p>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Bootstrap Statistics :
original bias std. error
99.7134 15.0572 99.7151 0.151986
99.8879 15.0782 99.8894 0.149213
</pre></div>
</div>
</div>
@@ -1001,10 +1115,10 @@ We use a more compact notation in terms of the expectation value</p>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Error: 0.013121573975499602
Bias^2: 0.012073649439965807
Var: 0.0010479245355337968
0.013121573975499602 &gt;= 0.012073649439965807 + 0.0010479245355337968 = 0.013121573975499604
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Error: 0.013121574061370796
Bias^2: 0.012073649472576395
Var: 0.0010479245887943952
0.013121574061370796 &gt;= 0.012073649472576395 + 0.0010479245887943952 = 0.01312157406137079
</pre></div>
</div>
<img alt="_images/chapter3_61_1.png" src="_images/chapter3_61_1.png" />
@@ -1063,86 +1177,83 @@ Var: 0.0010479245355337968
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree: 0
Error: 0.3214960170351912
Error: 0.32149601703519115
Bias^2: 0.3123314713548606
Var: 0.009164545680330616
0.3214960170351912 &gt;= 0.3123314713548606 + 0.009164545680330616 = 0.3214960170351912
0.32149601703519115 &gt;= 0.3123314713548606 + 0.009164545680330616 = 0.3214960170351912
Polynomial degree: 1
Error: 0.08426840630693411
Bias^2: 0.07968918676726029
Var: 0.004579219539673836
0.08426840630693411 &gt;= 0.07968918676726029 + 0.004579219539673836 = 0.08426840630693413
Error: 0.08426840630693412
Bias^2: 0.0796891867672603
Var: 0.004579219539673834
0.08426840630693412 &gt;= 0.0796891867672603 + 0.004579219539673834 = 0.08426840630693413
Polynomial degree: 2
Error: 0.10398646080125035
Error: 0.10398646080125037
Bias^2: 0.10077114273548984
Var: 0.003215318065760509
0.10398646080125035 &gt;= 0.10077114273548984 + 0.003215318065760509 = 0.10398646080125035
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree: 3
Error: 0.06547790180152357
Bias^2: 0.06208238634231953
Var: 0.0033955154592040944
0.06547790180152357 &gt;= 0.06208238634231953 + 0.0033955154592040944 = 0.06547790180152363
Var: 0.0032153180657605125
0.10398646080125037 &gt;= 0.10077114273548984 + 0.0032153180657605125 = 0.10398646080125036
Polynomial degree: 3
Error: 0.06547790180152352
Bias^2: 0.06208238634231944
Var: 0.003395515459204093
0.06547790180152352 &gt;= 0.06208238634231944 + 0.003395515459204093 = 0.06547790180152353
Polynomial degree: 4
Error: 0.06844519414009438
Bias^2: 0.06453579006728315
Var: 0.003909404072811231
0.06844519414009438 &gt;= 0.06453579006728315 + 0.003909404072811231 = 0.06844519414009438
Error: 0.06844519414009442
Bias^2: 0.06453579006728322
Var: 0.003909404072811217
0.06844519414009442 &gt;= 0.06453579006728322 + 0.003909404072811217 = 0.06844519414009444
Polynomial degree: 5
Error: 0.05227921801205692
Bias^2: 0.04818727730430296
Var: 0.0040919407077539514
0.05227921801205692 &gt;= 0.04818727730430296 + 0.0040919407077539514 = 0.05227921801205691
Error: 0.052279218012057004
Bias^2: 0.048187277304303056
Var: 0.004091940707753948
0.052279218012057004 &gt;= 0.048187277304303056 + 0.004091940707753948 = 0.052279218012057004
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree: 6
Error: 0.03781367141738885
Bias^2: 0.033657685071527485
Var: 0.004155986345861374
0.03781367141738885 &gt;= 0.033657685071527485 + 0.004155986345861374 = 0.03781367141738886
Error: 0.037813671417388985
Bias^2: 0.033657685071527624
Var: 0.004155986345861364
0.037813671417388985 &gt;= 0.033657685071527624 + 0.004155986345861364 = 0.03781367141738899
Polynomial degree: 7
Error: 0.027609773491022314
Bias^2: 0.02299949826036602
Var: 0.004610275230656294
0.027609773491022314 &gt;= 0.02299949826036602 + 0.004610275230656294 = 0.027609773491022314
Error: 0.027609773491022407
Bias^2: 0.0229994982603662
Var: 0.004610275230656187
0.027609773491022407 &gt;= 0.0229994982603662 + 0.004610275230656187 = 0.027609773491022387
Polynomial degree: 8
Error: 0.017355848195591845
Bias^2: 0.01033172130665515
Var: 0.007024126888936694
0.017355848195591845 &gt;= 0.01033172130665515 + 0.007024126888936694 = 0.01735584819559184
Error: 0.017355848195593354
Bias^2: 0.010331721306655144
Var: 0.0070241268889382116
0.017355848195593354 &gt;= 0.010331721306655144 + 0.0070241268889382116 = 0.017355848195593354
Polynomial degree:
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree: 9
Error: 0.026605727637176654
Bias^2: 0.010018312644139347
Var: 0.016587414993037307
0.026605727637176654 &gt;= 0.010018312644139347 + 0.016587414993037307 = 0.026605727637176654
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 9
Error: 0.026605727637184613
Bias^2: 0.010018312644139205
Var: 0.016587414993045405
0.026605727637184613 &gt;= 0.010018312644139205 + 0.016587414993045405 = 0.02660572763718461
Polynomial degree: 10
Error: 0.02159270458799264
Bias^2: 0.010516485576652856
Var: 0.011076219011339788
0.02159270458799264 &gt;= 0.010516485576652856 + 0.011076219011339788 = 0.021592704587992645
Error: 0.021592704588021167
Bias^2: 0.010516485576646513
Var: 0.01107621901137465
0.021592704588021167 &gt;= 0.010516485576646513 + 0.01107621901137465 = 0.021592704588021164
Polynomial degree: 11
Error: 0.07160048164248561
Bias^2: 0.014436800088969727
Var: 0.05716368155351588
0.07160048164248561 &gt;= 0.014436800088969727 + 0.05716368155351588 = 0.07160048164248561
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree: 12
Error: 0.11547777218940905
Bias^2: 0.016285782696075054
Var: 0.099191989493334
0.11547777218940905 &gt;= 0.016285782696075054 + 0.099191989493334 = 0.11547777218940906
Error: 0.07160048164232467
Bias^2: 0.014436800088896274
Var: 0.057163681553428394
0.07160048164232467 &gt;= 0.014436800088896274 + 0.057163681553428394 = 0.07160048164232467
Polynomial degree: 12
Error: 0.11547777218875695
Bias^2: 0.016285782696017055
Var: 0.0991919894927399
0.11547777218875695 &gt;= 0.016285782696017055 + 0.0991919894927399 = 0.11547777218875696
Polynomial degree: 13
Error: 0.22842468702288576
Bias^2: 0.01975416527179247
Var: 0.20867052175109335
0.22842468702288576 &gt;= 0.01975416527179247 + 0.20867052175109335 = 0.22842468702288582
Error: 0.2284246870217459
Bias^2: 0.019754165271682844
Var: 0.20867052175006306
0.2284246870217459 &gt;= 0.019754165271682844 + 0.20867052175006306 = 0.2284246870217459
</pre></div>
</div>
<img alt="_images/chapter3_62_5.png" src="_images/chapter3_62_5.png" />
<img alt="_images/chapter3_62_3.png" src="_images/chapter3_62_3.png" />
</div>
</div>
<p>The bias-variance tradeoff summarizes the fundamental tension in
@@ -1360,17 +1471,13 @@ Mean squared error on test data: 481979.17861098
Degree of polynomial: 2
Mean squared error on training data: 115822.95008046
Mean squared error on test data: 123711.53703498
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 3
Degree of polynomial: 3
Mean squared error on training data: 9011.85263220
Mean squared error on test data: 10913.84780262
Degree of polynomial: 4
Mean squared error on training data: 303.47610036
Mean squared error on test data: 426.30787294
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 5
Degree of polynomial: 5
Mean squared error on training data: 3.80354994
Mean squared error on test data: 5.98822371
Degree of polynomial: 6
@@ -1384,19 +1491,15 @@ Mean squared error on test data: 2.00607783
Degree of polynomial: 8
Mean squared error on training data: 0.04912436
Mean squared error on test data: 0.21596432
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 9
Degree of polynomial: 9
Mean squared error on training data: 0.02522069
Mean squared error on test data: 0.08576932
Degree of polynomial: 10
Mean squared error on training data: 0.02511518
Mean squared error on test data: 1.20015436
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 11
Degree of polynomial: 11
Mean squared error on training data: 0.01640891
Mean squared error on test data: 1.35533774
Mean squared error on test data: 1.35533773
Degree of polynomial: 12
Mean squared error on training data: 0.00813803
Mean squared error on test data: 0.17446471
@@ -1407,77 +1510,65 @@ Mean squared error on training data: 0.00759119
Mean squared error on test data: 1.08131003
Degree of polynomial: 14
Mean squared error on training data: 0.00472199
Mean squared error on test data: 0.81333805
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 15
Mean squared error on test data: 0.81333804
Degree of polynomial: 15
Mean squared error on training data: 0.00410478
Mean squared error on test data: 92.09149881
Mean squared error on test data: 92.09172408
Degree of polynomial: 16
Mean squared error on training data: 0.00315593
Mean squared error on test data: 234.39095416
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 17
Mean squared error on test data: 234.38533184
Degree of polynomial: 17
Mean squared error on training data: 0.00242999
Mean squared error on test data: 1270.94548496
Mean squared error on test data: 1271.35771842
Degree of polynomial: 18
Mean squared error on training data: 0.00228741
Mean squared error on test data: 108.28590743
Mean squared error on test data: 108.27092897
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 19
Mean squared error on training data: 0.00156379
Mean squared error on test data: 1378.43761347
Mean squared error on training data: 0.00156382
Mean squared error on test data: 1371.99049330
Degree of polynomial: 20
Mean squared error on training data: 0.00137835
Mean squared error on test data: 1954.37992857
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 21
Mean squared error on training data: 0.00118527
Mean squared error on test data: 14818.20320502
Mean squared error on training data: 0.00137823
Mean squared error on test data: 1887.85953586
Degree of polynomial: 21
Mean squared error on training data: 0.00118504
Mean squared error on test data: 14859.70127680
Degree of polynomial: 22
Mean squared error on training data: 0.00092646
Mean squared error on test data: 871.17339342
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 23
Mean squared error on training data: 0.00085884
Mean squared error on test data: 5566.16660817
Mean squared error on training data: 0.00092645
Mean squared error on test data: 876.51214899
Degree of polynomial: 23
Mean squared error on training data: 0.00085883
Mean squared error on test data: 5594.60685864
Degree of polynomial: 24
Mean squared error on training data: 0.00084705
Mean squared error on test data: 1314.42631342
Mean squared error on training data: 0.00084711
Mean squared error on test data: 1277.60619654
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 25
Mean squared error on training data: 0.00079123
Mean squared error on test data: 127043.53189647
Mean squared error on training data: 0.00079130
Mean squared error on test data: 128664.09744272
Degree of polynomial: 26
Mean squared error on training data: 0.00076925
Mean squared error on test data: 18526.05756733
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 27
Mean squared error on training data: 0.00069103
Mean squared error on test data: 2470.53697476
Mean squared error on training data: 0.00076919
Mean squared error on test data: 19003.95079764
Degree of polynomial: 27
Mean squared error on training data: 0.00068941
Mean squared error on test data: 2379.66226149
Degree of polynomial: 28
Mean squared error on training data: 0.00062595
Mean squared error on test data: 4022.12945452
Mean squared error on training data: 0.00062582
Mean squared error on test data: 4082.19994371
Degree of polynomial: 29
Mean squared error on training data: 0.00060708
Mean squared error on test data: 3250.24770094
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 29
Mean squared error on training data: 0.00060705
Mean squared error on test data: 3384.63675140
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/jy/g42mrgv128v34gnnhxwk9nrc0000gp/T/ipykernel_42456/626635268.py:73: RuntimeWarning: divide by zero encountered in log10
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;ipython-input-7-40a38ad763f1&gt;:73: RuntimeWarning: divide by zero encountered in log10
plt.plot(polynomial, np.log10(trainingerror), label=&#39;Training Error&#39;)
/var/folders/jy/g42mrgv128v34gnnhxwk9nrc0000gp/T/ipykernel_42456/626635268.py:74: RuntimeWarning: divide by zero encountered in log10
&lt;ipython-input-7-40a38ad763f1&gt;:74: RuntimeWarning: divide by zero encountered in log10
plt.plot(polynomial, np.log10(testerror), label=&#39;Test Error&#39;)
</pre></div>
</div>
<img alt="_images/chapter3_65_16.png" src="_images/chapter3_65_16.png" />
<img alt="_images/chapter3_65_6.png" src="_images/chapter3_65_6.png" />
</div>
</div>
</div>
@@ -1707,7 +1798,7 @@ cross-validation (LOOCV).</p>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/jy/g42mrgv128v34gnnhxwk9nrc0000gp/T/ipykernel_42456/3817475779.py:63: RuntimeWarning: divide by zero encountered in log10
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;ipython-input-9-6e75736fdab1&gt;:63: RuntimeWarning: divide by zero encountered in log10
plt.plot(polynomial, np.log10(estimated_mse_sklearn), label=&#39;Test Error&#39;)
</pre></div>
</div>
@@ -1971,17 +2062,17 @@ Note also that we do not split the data into training and test.</p>
Fitted beta: [2.08376632 0.19569961 3.97898392]
Sklearn fitted beta: [2.08376632 0.19569961 3.97898392]
MSE with intercept column
0.0041136346174431284
0.00411363461744314
MSE with intercept column from SKL
0.004113634617443141
Manual intercept: 2.0837663229239016
0.004113634617443116
Manual intercept: 2.083766322923899
Fitted beta (wiothout intercept): [0.19569961 3.97898392]
Sklearn intercept: 2.0837663229239025
Sklearn intercept: 2.0837663229239043
Sklearn fitted beta (without intercept): [0.19569961 3.97898392]
MSE with Manual intercept
0.00411363461744314
MSE with Sklearn intercept
0.004113634617443135
0.004113634617443131
</pre></div>
</div>
<img alt="_images/chapter3_107_1.png" src="_images/chapter3_107_1.png" />
@@ -2092,19 +2183,19 @@ intercept.</p>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Beta values for own Ridge implementation
[ 1.03032441e+00 6.28336218e-02 -6.24175744e-01 5.21169159e-02
2.80847477e-01 2.12552073e-01 8.13220608e-02 -1.69634577e-02
-6.50846112e-02 -7.38962192e-02 -5.94226022e-02 -3.50227564e-02
-9.80609615e-03 1.08299273e-02 2.41882037e-02 2.93492130e-02
2.64742912e-02 1.63249532e-02 -5.01831130e-05 -2.15098090e-02]
-6.50846111e-02 -7.38962192e-02 -5.94226022e-02 -3.50227564e-02
-9.80609616e-03 1.08299273e-02 2.41882036e-02 2.93492130e-02
2.64742912e-02 1.63249532e-02 -5.01831036e-05 -2.15098090e-02]
Beta values for Scikit-Learn Ridge implementation
[ 1.03032441e+00 6.28336218e-02 -6.24175744e-01 5.21169159e-02
2.80847477e-01 2.12552073e-01 8.13220608e-02 -1.69634577e-02
-6.50846112e-02 -7.38962192e-02 -5.94226022e-02 -3.50227564e-02
-9.80609615e-03 1.08299273e-02 2.41882037e-02 2.93492130e-02
2.64742912e-02 1.63249532e-02 -5.01831200e-05 -2.15098090e-02]
2.64742912e-02 1.63249532e-02 -5.01831207e-05 -2.15098090e-02]
MSE values for own Ridge implementation
4.3632959170548605e-07
4.363295924430451e-07
MSE values for Scikit-Learn Ridge implementation
4.363295916414895e-07
4.363295916323784e-07
Beta values for own Ridge implementation
[ 1.03630548 -0.01963611 -0.37900111 -0.07062318 0.12182967 0.16343471
0.13003291 0.07490892 0.02365049 -0.01449782 -0.03814292 -0.04909093
@@ -2116,9 +2207,9 @@ Beta values for Scikit-Learn Ridge implementation
-0.05009826 -0.04389027 -0.03279636 -0.01866537 -0.00289724 0.01348565
0.02976145 0.04543942]
MSE values for own Ridge implementation
5.194042826653172e-06
5.19404282648955e-06
MSE values for Scikit-Learn Ridge implementation
5.194042826815498e-06
5.1940428268204826e-06
Beta values for own Ridge implementation
[ 1.04220758 -0.10931453 -0.17641709 -0.06020587 0.02208512 0.05789007
0.06491736 0.05785343 0.04537385 0.03196357 0.01969145 0.00934499
@@ -2130,9 +2221,9 @@ Beta values for Scikit-Learn Ridge implementation
0.00107405 -0.00526348 -0.00992331 -0.01318643 -0.01531845 -0.01655318
-0.01708852 -0.01708781]
MSE values for own Ridge implementation
2.0940821989673748e-05
2.094082198966615e-05
MSE values for Scikit-Learn Ridge implementation
2.0940821989624095e-05
2.094082198961999e-05
Beta values for own Ridge implementation
[ 1.01219292 -0.06043581 -0.10391807 -0.05651951 -0.01898855 0.00312361
0.01463049 0.01975848 0.02123176 0.02068067 0.01905883 0.01691985
@@ -2144,9 +2235,9 @@ Beta values for Scikit-Learn Ridge implementation
0.01458337 0.01223198 0.00996754 0.00784393 0.00588657 0.00410387
0.00249435 0.00105081]
MSE values for own Ridge implementation
0.0003153514830958126
0.0003153514830958235
MSE values for Scikit-Learn Ridge implementation
0.0003153514830958081
0.00031535148309580783
Beta values for own Ridge implementation
[ 8.38916861e-01 1.31276579e-01 8.97497404e-03 -1.72271878e-02
-2.11744554e-02 -1.91492986e-02 -1.57201944e-02 -1.23002365e-02
@@ -2160,9 +2251,9 @@ Beta values for Scikit-Learn Ridge implementation
-1.84923989e-03 -8.13661243e-04 7.46984697e-06 6.56636616e-04
1.16805821e-03 1.56912044e-03 1.88168312e-03 2.12318726e-03]
MSE values for own Ridge implementation
0.015072388895177109
0.015072388895177239
MSE values for Scikit-Learn Ridge implementation
0.015072388895177088
0.0150723888951771
Beta values for own Ridge implementation
[0.37396662 0.14174745 0.0764924 0.04892055 0.03447512 0.02586427
0.02024962 0.01633913 0.01347916 0.0113104 0.0096208 0.00827728
@@ -2174,9 +2265,9 @@ Beta values for Scikit-Learn Ridge implementation
0.00719176 0.00630331 0.00556826 0.0049544 0.00443743 0.0039987
0.0036237 0.003301 ]
MSE values for own Ridge implementation
0.2640931530791005
0.26409315307910053
MSE values for Scikit-Learn Ridge implementation
0.2640931530791003
0.26409315307910025
</pre></div>
</div>
<img alt="_images/chapter3_115_1.png" src="_images/chapter3_115_1.png" />
@@ -2272,7 +2363,7 @@ Let us see how we can change this code by zero centering.</p>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Beta values for own Ridge implementation
[ 3.43579948e-02 -5.43330971e-01 -3.10141413e-03 2.47116868e-01
2.18613217e-01 1.02054837e-01 -4.25617657e-04 -5.90475506e-02
2.18613217e-01 1.02054837e-01 -4.25617658e-04 -5.90475506e-02
-7.68534263e-02 -6.68929213e-02 -4.24906604e-02 -1.40927184e-02
1.11482289e-02 2.88529063e-02 3.67047975e-02 3.38135733e-02
2.02198702e-02 -3.46383925e-03 -3.63025821e-02]
@@ -2283,13 +2374,13 @@ Beta values for Scikit-Learn Ridge implementation
1.11482289e-02 2.88529063e-02 3.67047975e-02 3.38135733e-02
2.02198702e-02 -3.46383925e-03 -3.63025821e-02]
Intercept from own implementation:
1.0330308045180234
1.0330308045190182
Intercept from Scikit-Learn Ridge implementation
1.0330308045183163
1.0330308045183219
MSE values for own Ridge implementation
3.1392559581788775e-06
3.13925595925919e-06
MSE values for Scikit-Learn Ridge implementation
3.1392559584983597e-06
3.1392559585048734e-06
Beta values for own Ridge implementation
[-0.05807125 -0.29822833 -0.08551306 0.08156108 0.13679863 0.12333649
0.08251519 0.03815288 0.00111756 -0.02498832 -0.04010697 -0.04566964
@@ -2301,13 +2392,13 @@ Beta values for Scikit-Learn Ridge implementation
-0.04355837 -0.03562355 -0.02348765 -0.00848904 0.00831018 0.0260906
0.04423486]
Intercept from own implementation:
1.0411487294305746
1.041148729430502
Intercept from Scikit-Learn Ridge implementation
1.0411487294305246
1.041148729430523
MSE values for own Ridge implementation
1.9601304850213484e-05
1.9601304850018328e-05
MSE values for Scikit-Learn Ridge implementation
1.960130485007934e-05
1.960130485007504e-05
Beta values for own Ridge implementation
[-0.1416398 -0.14021063 -0.05383795 0.01367553 0.04784395 0.05796251
0.05447415 0.044613 0.03267527 0.02098261 0.01066519 0.00217499
@@ -2319,13 +2410,13 @@ Beta values for Scikit-Learn Ridge implementation
-0.00440346 -0.00917248 -0.01231917 -0.01405935 -0.0146081 -0.01416528
-0.01290947]
Intercept from own implementation:
1.0495569966278315
1.0495569966278238
Intercept from Scikit-Learn Ridge implementation
1.0495569966278269
MSE values for own Ridge implementation
5.495916150938325e-05
5.4959161509356135e-05
MSE values for Scikit-Learn Ridge implementation
5.495916150936654e-05
5.495916150936645e-05
Beta values for own Ridge implementation
[-0.13535942 -0.08593216 -0.03568439 -0.0036367 0.01397146 0.02229529
0.02503753 0.0245528 0.02228115 0.01908936 0.01549377 0.01179792
@@ -2341,9 +2432,9 @@ Intercept from own implementation:
Intercept from Scikit-Learn Ridge implementation
1.0399676689527975
MSE values for own Ridge implementation
7.571105947979439e-05
7.571105947979336e-05
MSE values for Scikit-Learn Ridge implementation
7.571105947979395e-05
7.571105947979394e-05
Beta values for own Ridge implementation
[-0.05100875 -0.04063602 -0.02723445 -0.01713366 -0.0100706 -0.00517114
-0.00174276 0.00068734 0.00243186 0.00369758 0.00462287 0.0053018
@@ -2359,9 +2450,9 @@ Intercept from own implementation:
Intercept from Scikit-Learn Ridge implementation
0.999955585168597
MSE values for own Ridge implementation
0.0007698473260556339
0.0007698473260556344
MSE values for Scikit-Learn Ridge implementation
0.0007698473260556334
0.0007698473260556325
Beta values for own Ridge implementation
[-0.00834567 -0.00803064 -0.00673407 -0.00554552 -0.00458878 -0.0038335
-0.00323332 -0.00274989 -0.0023548 -0.00202756 -0.00175331 -0.00152117
@@ -2379,7 +2470,7 @@ Intercept from Scikit-Learn Ridge implementation
MSE values for own Ridge implementation
0.0023813163025848865
MSE values for Scikit-Learn Ridge implementation
0.002381316302584885
0.002381316302584886
</pre></div>
</div>
<img alt="_images/chapter3_117_1.png" src="_images/chapter3_117_1.png" />
@@ -2596,7 +2687,9 @@ linear system as an equation would reduce this down to
</div>
</div>
<div class="cell_output docutils container">
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/jy/g42mrgv128v34gnnhxwk9nrc0000gp/T/ipykernel_42456/4162706317.py:7: UserWarning: FixedFormatter should only be used together with FixedLocator
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;ipython-input-20-6f7a6bd7d79f&gt;:6: MatplotlibDeprecationWarning: Auto-removal of grids by pcolor() and pcolormesh() is deprecated since 3.5 and will be removed two minor releases later; please call grid(False) first.
cb = fig.colorbar(im)
&lt;ipython-input-20-6f7a6bd7d79f&gt;:7: UserWarning: FixedFormatter should only be used together with FixedLocator
cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)
</pre></div>
</div>
@@ -2740,7 +2833,9 @@ with the form utilized in linear regression, viz.</p>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/jy/g42mrgv128v34gnnhxwk9nrc0000gp/T/ipykernel_42456/3777801602.py:7: UserWarning: FixedFormatter should only be used together with FixedLocator
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;ipython-input-25-5dd54edf2138&gt;:6: MatplotlibDeprecationWarning: Auto-removal of grids by pcolor() and pcolormesh() is deprecated since 3.5 and will be removed two minor releases later; please call grid(False) first.
cb = fig.colorbar(im)
&lt;ipython-input-25-5dd54edf2138&gt;:7: UserWarning: FixedFormatter should only be used together with FixedLocator
cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)
</pre></div>
</div>
@@ -2788,7 +2883,9 @@ K</p>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/jy/g42mrgv128v34gnnhxwk9nrc0000gp/T/ipykernel_42456/438060758.py:10: UserWarning: FixedFormatter should only be used together with FixedLocator
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;ipython-input-26-fe5b9d300cc0&gt;:9: MatplotlibDeprecationWarning: Auto-removal of grids by pcolor() and pcolormesh() is deprecated since 3.5 and will be removed two minor releases later; please call grid(False) first.
cb = fig.colorbar(im)
&lt;ipython-input-26-fe5b9d300cc0&gt;:10: UserWarning: FixedFormatter should only be used together with FixedLocator
cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)
</pre></div>
</div>
@@ -2823,7 +2920,9 @@ K</p>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/jy/g42mrgv128v34gnnhxwk9nrc0000gp/T/ipykernel_42456/3544313922.py:9: UserWarning: FixedFormatter should only be used together with FixedLocator
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;ipython-input-27-25845e8df859&gt;:8: MatplotlibDeprecationWarning: Auto-removal of grids by pcolor() and pcolormesh() is deprecated since 3.5 and will be removed two minor releases later; please call grid(False) first.
cb = fig.colorbar(im)
&lt;ipython-input-27-25845e8df859&gt;:9: UserWarning: FixedFormatter should only be used together with FixedLocator
cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)
</pre></div>
</div>
@@ -2876,43 +2975,43 @@ constant as opposed to ridge and OLS. We get a sparse solution with
</div>
</div>
<div class="cell_output docutils container">
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 0%| | 0/10 [00:00&lt;?, ?it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 0%| | 0/10 [00:00&lt;?, ?it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/MortenImac/anaconda3/lib/python3.8/site-packages/sklearn/linear_model/_coordinate_descent.py:645: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations, check the scale of the features or consider increasing regularisation. Duality gap: 3.924e+00, tolerance: 1.797e+00
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/linear_model/_coordinate_descent.py:647: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations, check the scale of the features or consider increasing regularisation. Duality gap: 3.924e+00, tolerance: 1.797e+00
model = cd_fast.enet_coordinate_descent(
10%|█████████████▍ | 1/10 [00:00&lt;00:04, 1.95it/s]
10%|█ | 1/10 [00:00&lt;00:06, 1.50it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 20%|██████████████████████████▊ | 2/10 [00:00&lt;00:03, 2.63it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 20%|██ | 2/10 [00:01&lt;00:03, 2.11it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 30%|████████████████████████████████████████▏ | 3/10 [00:00&lt;00:01, 3.57it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 30%|███ | 3/10 [00:01&lt;00:02, 3.06it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 40%|█████████████████████████████████████████████████████▌ | 4/10 [00:01&lt;00:01, 4.54it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 40%|████ | 4/10 [00:01&lt;00:01, 4.00it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 50%|███████████████████████████████████████████████████████████████████ | 5/10 [00:01&lt;00:00, 5.36it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 50%|█████ | 5/10 [00:01&lt;00:01, 4.87it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 60%|████████████████████████████████████████████████████████████████████████████████▍ | 6/10 [00:01&lt;00:00, 6.04it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 60%|██████ | 6/10 [00:01&lt;00:00, 5.73it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 70%|█████████████████████████████████████████████████████████████████████████████████████████████▊ | 7/10 [00:01&lt;00:00, 5.69it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 70%|███████ | 7/10 [00:01&lt;00:00, 6.34it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 80%|███████████████████████████████████████████████████████████████████████████████████████████████████████████▏ | 8/10 [00:01&lt;00:00, 6.18it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 80%|████████ | 8/10 [00:01&lt;00:00, 6.22it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 90%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▌ | 9/10 [00:01&lt;00:00, 6.57it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 90%|█████████ | 9/10 [00:01&lt;00:00, 6.41it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:01&lt;00:00, 6.75it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>100%|██████████| 10/10 [00:02&lt;00:00, 6.14it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:01&lt;00:00, 5.15it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>100%|██████████| 10/10 [00:02&lt;00:00, 4.66it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>
@@ -3059,8 +3158,10 @@ which polynomial fits the data best.</p>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/jy/g42mrgv128v34gnnhxwk9nrc0000gp/T/ipykernel_42456/3980313467.py:9: MatplotlibDeprecationWarning: Calling gca() with keyword arguments was deprecated in Matplotlib 3.4. Starting two minor releases later, gca() will take no keyword arguments. The gca() function should only be used to get the current axes, or if no axes exist, create new axes with default keyword arguments. To create a new axes with non-default arguments, use plt.axes() or plt.subplot().
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;ipython-input-30-bc298b802fe2&gt;:9: MatplotlibDeprecationWarning: Calling gca() with keyword arguments was deprecated in Matplotlib 3.4. Starting two minor releases later, gca() will take no keyword arguments. The gca() function should only be used to get the current axes, or if no axes exist, create new axes with default keyword arguments. To create a new axes with non-default arguments, use plt.axes() or plt.subplot().
ax = fig.gca(projection=&#39;3d&#39;)
&lt;ipython-input-30-bc298b802fe2&gt;:37: MatplotlibDeprecationWarning: Auto-removal of grids by pcolor() and pcolormesh() is deprecated since 3.5 and will be removed two minor releases later; please call grid(False) first.
fig.colorbar(surf, shrink=0.5, aspect=5)
</pre></div>
</div>
<img alt="_images/chapter3_181_1.png" src="_images/chapter3_181_1.png" />
@@ -3220,7 +3321,7 @@ Python program using</p>
<div class="cell_output docutils container">
<div class="output traceback highlight-ipythontb notranslate"><div class="highlight"><pre><span></span><span class="gt">---------------------------------------------------------------------------</span>
<span class="ne">NameError</span><span class="g g-Whitespace"> </span>Traceback (most recent call last)
<span class="o">/</span><span class="n">var</span><span class="o">/</span><span class="n">folders</span><span class="o">/</span><span class="n">jy</span><span class="o">/</span><span class="n">g42mrgv128v34gnnhxwk9nrc0000gp</span><span class="o">/</span><span class="n">T</span><span class="o">/</span><span class="n">ipykernel_42456</span><span class="o">/</span><span class="mf">1950915150.</span><span class="n">py</span> <span class="ow">in</span> <span class="o">&lt;</span><span class="n">module</span><span class="o">&gt;</span>
<span class="o">&lt;</span><span class="n">ipython</span><span class="o">-</span><span class="nb">input</span><span class="o">-</span><span class="mi">31</span><span class="o">-</span><span class="n">d985fb40c43d</span><span class="o">&gt;</span> <span class="ow">in</span> <span class="o">&lt;</span><span class="n">module</span><span class="o">&gt;</span>
<span class="ne">----&gt; </span><span class="mi">1</span> <span class="n">scipy</span><span class="o">.</span><span class="n">misc</span><span class="o">.</span><span class="n">imread</span>
<span class="ne">NameError</span>: name &#39;scipy&#39; is not defined
@@ -3292,54 +3393,42 @@ of data presented here (either the terrain data we propose or other data sets).<
</div>
<div class='prev-next-bottom'>
<div id="prev">
<a class="left-prev" href="chapter2.html" title="previous page">
<i class="prevnext-label fas fa-angle-left"></i>
<div class="prevnext-info">
<p class="prevnext-label">previous</p>
<p class="prevnext-title"><span class="section-number">4. </span>Ridge and Lasso Regression</p>
</div>
</a>
<!-- Previous / next buttons -->
<div class='prev-next-area'>
<a class='left-prev' id="prev-link" href="chapter2.html" title="previous page">
<i class="fas fa-angle-left"></i>
<div class="prev-next-info">
<p class="prev-next-subtitle">previous</p>
<p class="prev-next-title"><span class="section-number">4. </span>Ridge and Lasso Regression</p>
</div>
</a>
<a class='right-next' id="next-link" href="chapter4.html" title="next page">
<div class="prev-next-info">
<p class="prev-next-subtitle">next</p>
<p class="prev-next-title"><span class="section-number">6. </span>Logistic Regression</p>
</div>
<div id="next">
<a class="right-next" href="chapter4.html" title="next page">
<div class="prevnext-info">
<p class="prevnext-label">next</p>
<p class="prevnext-title"><span class="section-number">6. </span>Logistic Regression</p>
</div>
<i class="prevnext-label fas fa-angle-right"></i>
</a>
</div>
<i class="fas fa-angle-right"></i>
</a>
</div>
</div>
</div>
<footer class="footer">
<div class="container">
<p>
By Morten Hjorth-Jensen<br/>
&copy; Copyright 2021.<br/>
</p>
</div>
</footer>
<p>
By Morten Hjorth-Jensen<br/>
&copy; Copyright 2021.<br/>
</p>
</footer>
</main>
</div>
</div>
<script src="_static/js/index.1c5a1a01449ed65a7b51.js"></script>
<script src="_static/js/index.be7d3bbb2ef33a8344ce.js"></script>
</body>
</html>