update
This commit is contained in:
Morten Hjorth-Jensen
@@ -55,7 +55,6 @@ const thebe_selector_output = ".output, .cell_output"
|
||||
<script defer="defer" src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
|
||||
<link rel="index" title="Index" href="genindex.html" />
|
||||
<link rel="search" title="Search" href="search.html" />
|
||||
<link rel="next" title="Exercises week 36" href="exercisesweek36.html" />
|
||||
<link rel="prev" title="Exercises week 35" href="exercisesweek35.html" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
||||
<meta name="docsearch:language" content="None">
|
||||
@@ -268,123 +267,6 @@ const thebe_selector_output = ".output, .cell_output"
|
||||
Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="exercisesweek36.html">
|
||||
Exercises week 36
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week36.html">
|
||||
Week 36: Statistical interpretation of Linear Regression and Resampling techniques
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="exercisesweek37.html">
|
||||
Exercises week 37
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week37.html">
|
||||
Week 37: Statistical interpretations and Resampling Methods
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="exercisesweek38.html">
|
||||
Exercises week 38
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week38.html">
|
||||
Week 38: Logistic Regression and Optimization
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="exercisesweek39.html">
|
||||
Exercises week 39
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week39.html">
|
||||
Week 39: Optimization and Gradient Methods
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week40.html">
|
||||
Week 40: Gradient descent methods (continued) and start Neural networks
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="exercisesweek41.html">
|
||||
Exercises week 41
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week41.html">
|
||||
Week 41 Neural networks and constructing a neural network code
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="exercisesweek42.html">
|
||||
Exercises week 42
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week42.html">
|
||||
Week 42 Constructing a Neural Network code with introduction to Tensor flow
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="exercisesweek43.html">
|
||||
Exercises weeks 43 and 44
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week43.html">
|
||||
Week 43: Deep Learning: Constructing a Neural Network code and solving differential equations
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week44.html">
|
||||
Week 44, Convolutional Neural Networks (CNN)
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week45.html">
|
||||
Week 45, Recurrent Neural Networks
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week46.html">
|
||||
Week 46: Decision Trees, Ensemble methods and Random Forests
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week47.html">
|
||||
Week 47: From Decision Trees to Ensemble Methods, Random Forests and Boosting Methods and Summary of Course
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="exercisesweek47.html">
|
||||
Exercise week 47
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading">
|
||||
<span class="caption-text">
|
||||
Projects
|
||||
</span>
|
||||
</p>
|
||||
<ul class="nav bd-sidenav">
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="project1.html">
|
||||
Project 1 on Machine Learning, deadline October 9 (midnight), 2023
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="project2.html">
|
||||
Project 2 on Machine Learning, deadline November 17 (Midnight)
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
@@ -1240,7 +1122,7 @@ doconce format html week35.do.txt --no_mako -->
|
||||
<!-- dom:TITLE: Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression --><div class="tex2jax_ignore mathjax_ignore section" id="week-35-from-ordinary-linear-regression-to-ridge-and-lasso-regression">
|
||||
<h1>Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression<a class="headerlink" href="#week-35-from-ordinary-linear-regression-to-ridge-and-lasso-regression" title="Permalink to this headline">¶</a></h1>
|
||||
<p><strong>Morten Hjorth-Jensen</strong>, Department of Physics, University of Oslo and Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</p>
|
||||
<p>Date: <strong>August 28-September 1</strong></p>
|
||||
<p>Date: <strong>August 26-30</strong></p>
|
||||
<div class="section" id="plans-for-week-35">
|
||||
<h2>Plans for week 35<a class="headerlink" href="#plans-for-week-35" title="Permalink to this headline">¶</a></h2>
|
||||
<p>The main topics are:</p>
|
||||
@@ -1248,17 +1130,14 @@ doconce format html week35.do.txt --no_mako -->
|
||||
<li><p>Brief repetition from last week</p></li>
|
||||
<li><p>Derivation of the equations for ordinary least squares</p></li>
|
||||
<li><p>Discussion on how to prepare data and examples of applications of linear regression</p></li>
|
||||
<li><p>Material for the lecture on Thursday: Mathematical interpretations of linear regression</p></li>
|
||||
<li><p>Thursday: Ridge and Lasso regression and Singular Value Decomposition</p></li>
|
||||
<li><p><a class="reference external" href="https://youtu.be/qBNm-HGSxL4">Video of lecture</a></p></li>
|
||||
<li><p><a class="reference external" href="https://github.com/CompPhysics/MachineLearning/blob/master/doc/HandWrittenNotes/2023/NotesAug31.pdf">Whiteboard notes</a></p></li>
|
||||
<li><p>Material for the lecture on Monday: Mathematical interpretations of linear regression</p></li>
|
||||
<li><p>Monday: Ridge and Lasso regression and Singular Value Decomposition</p></li>
|
||||
</ol>
|
||||
<div class="section" id="reading-recommendations">
|
||||
<h3>Reading recommendations:<a class="headerlink" href="#reading-recommendations" title="Permalink to this headline">¶</a></h3>
|
||||
<ol class="simple">
|
||||
<li><p>See lecture notes for week 35 at <a class="reference external" href="https://compphysics.github.io/MachineLearning/doc/web/course.html">https://compphysics.github.io/MachineLearning/doc/web/course.html</a></p></li>
|
||||
<li><p>Goodfellow, Bengio and Courville, Deep Learning, chapter 2 on linear algebra and sections 3.1-3.10 on elements of statistics (background)</p></li>
|
||||
<li><p>Hastie, Tibshirani and Friedman, The elements of statistical learning, sections 3.1-3.4 (on relevance for the discussion of linear regression).</p></li>
|
||||
</ol>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1731,7 +1610,7 @@ Since we are not using <strong>Scikit-Learn</strong> here we can define our own
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.9958946686888259
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.9952638231265687
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1748,7 +1627,7 @@ Since we are not using <strong>Scikit-Learn</strong> here we can define our own
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.008142188979400687
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.011761161707539526
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1763,23 +1642,23 @@ Since we are not using <strong>Scikit-Learn</strong> here we can define our own
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[0.0476021 0.02689869 0.01088331 0.01783105 0.00544013 0.05110385
|
||||
0.02900389 0.01629703 0.05594058 0.02527366 0.00657884 0.04127087
|
||||
0.01925607 0.02221978 0.01212083 0.04919181 0.00745959 0.03110176
|
||||
0.010203 0.0076995 0.00298213 0.01702968 0.04557362 0.03192124
|
||||
0.06668218 0.0178392 0.00706728 0.0095239 0.00784983 0.05197707
|
||||
0.01519861 0.0134093 0.00291822 0.00311528 0.02036289 0.01136976
|
||||
0.0189559 0.04908155 0.01384493 0.01715895 0.01262581 0.00756465
|
||||
0.00473818 0.00224783 0.01773579 0.03804636 0.03945128 0.01662346
|
||||
0.05137822 0.00206124 0.06090176 0.01632212 0.01220987 0.06361921
|
||||
0.00318122 0.00362359 0.03177421 0.06554078 0.00123144 0.01091059
|
||||
0.04958045 0.00291334 0.01541622 0.00607264 0.05274561 0.007352
|
||||
0.06263415 0.01593612 0.00853836 0.01006042 0.00223784 0.02106518
|
||||
0.02410507 0.08294341 0.0043675 0.06502562 0.03422156 0.00213264
|
||||
0.02365779 0.01883403 0.00683222 0.01848399 0.02930957 0.02161016
|
||||
0.02746315 0.02774744 0.03591454 0.04814746 0.00568413 0.00215333
|
||||
0.03631783 0.02866734 0.01684326 0.00953152 0.01001378 0.00119895
|
||||
0.02603725 0.00127672 0.04770636 0.028797 ]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[0.00035753 0.04937621 0.02268114 0.03112297 0.01856502 0.05322899
|
||||
0.01397927 0.03999935 0.02508836 0.01834042 0.0652633 0.00887114
|
||||
0.01956921 0.01987248 0.06294313 0.01152476 0.00328431 0.03564082
|
||||
0.02337045 0.01743586 0.01278035 0.02400968 0.08388138 0.03270341
|
||||
0.00335956 0.01031903 0.09575469 0.00956774 0.00900879 0.01867985
|
||||
0.01191227 0.02090296 0.04045671 0.03582958 0.06692165 0.06615863
|
||||
0.06594872 0.03756493 0.00488992 0.01405237 0.00117071 0.0017567
|
||||
0.05006306 0.02545639 0.03456485 0.00373984 0.02576476 0.03530741
|
||||
0.00092902 0.0275742 0.05948007 0.01321652 0.18500705 0.00382166
|
||||
0.00327313 0.01853877 0.01771317 0.05293662 0.07199977 0.00836148
|
||||
0.01541649 0.00343257 0.00626797 0.05350297 0.01548272 0.05235058
|
||||
0.04310698 0.00225189 0.02356396 0.01690512 0.03467756 0.00064364
|
||||
0.02593596 0.00019607 0.0029508 0.0180194 0.06825695 0.01659559
|
||||
0.01971341 0.02012338 0.02241311 0.00135736 0.0095653 0.05695438
|
||||
0.00395659 0.07068033 0.02699873 0.00919237 0.02493299 0.00803115
|
||||
0.0293055 0.02178063 0.00594353 0.04081883 0.01325225 0.0386443
|
||||
0.01889695 0.02810253 0.0181166 0.01135924]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1848,15 +1727,15 @@ but now splitting the data into a training set and a test set.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[ 2.02283241 0.15972118 3.84256187 1.89005305 -0.93145755]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[ 2.00507876 0.39026883 3.20764972 2.73806921 -1.39609089]
|
||||
Training R2
|
||||
0.9959044445566834
|
||||
0.9972493421341901
|
||||
Training MSE
|
||||
0.010349061754867921
|
||||
0.007021475481484069
|
||||
Test R2
|
||||
0.9961996615568259
|
||||
0.9976639055733267
|
||||
Test MSE
|
||||
0.008771887357306985
|
||||
0.00575158411598985
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -2903,45 +2782,73 @@ the house using the features (predictors) listed here.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function load_boston is deprecated; `load_boston` is deprecated in 1.0 and will be removed in 1.2.
|
||||
<div class="output traceback highlight-ipythontb notranslate"><div class="highlight"><pre><span></span>---------------------------------------------------------------------------
|
||||
ImportError Traceback (most recent call last)
|
||||
Cell In[16], line 1
|
||||
----> 1 from sklearn.datasets import load_boston
|
||||
3 boston_dataset = load_boston()
|
||||
5 # boston_dataset is a dictionary
|
||||
6 # let's check what it contains
|
||||
|
||||
The Boston housing prices dataset has an ethical problem. You can refer to
|
||||
the documentation of this function for further details.
|
||||
File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/datasets/__init__.py:157, in __getattr__(name)
|
||||
108 if name == "load_boston":
|
||||
109 msg = textwrap.dedent("""
|
||||
110 `load_boston` has been removed from scikit-learn since version 1.2.
|
||||
111
|
||||
(...)
|
||||
155 <https://www.researchgate.net/publication/4974606_Hedonic_housing_prices_and_the_demand_for_clean_air>
|
||||
156 """)
|
||||
--> 157 raise ImportError(msg)
|
||||
158 try:
|
||||
159 return globals()[name]
|
||||
|
||||
The scikit-learn maintainers therefore strongly discourage the use of this
|
||||
dataset unless the purpose of the code is to study and educate about
|
||||
ethical issues in data science and machine learning.
|
||||
ImportError:
|
||||
`load_boston` has been removed from scikit-learn since version 1.2.
|
||||
|
||||
In this special case, you can fetch the dataset from the original
|
||||
source::
|
||||
The Boston housing prices dataset has an ethical problem: as
|
||||
investigated in [1], the authors of this dataset engineered a
|
||||
non-invertible variable "B" assuming that racial self-segregation had a
|
||||
positive impact on house prices [2]. Furthermore the goal of the
|
||||
research that led to the creation of this dataset was to study the
|
||||
impact of air quality but it did not give adequate demonstration of the
|
||||
validity of this assumption.
|
||||
|
||||
import pandas as pd
|
||||
import numpy as np
|
||||
The scikit-learn maintainers therefore strongly discourage the use of
|
||||
this dataset unless the purpose of the code is to study and educate
|
||||
about ethical issues in data science and machine learning.
|
||||
|
||||
In this special case, you can fetch the dataset from the original
|
||||
source::
|
||||
|
||||
data_url = "http://lib.stat.cmu.edu/datasets/boston"
|
||||
raw_df = pd.read_csv(data_url, sep="\s+", skiprows=22, header=None)
|
||||
data = np.hstack([raw_df.values[::2, :], raw_df.values[1::2, :2]])
|
||||
target = raw_df.values[1::2, 2]
|
||||
import pandas as pd
|
||||
import numpy as np
|
||||
|
||||
Alternative datasets include the California housing dataset (i.e.
|
||||
:func:`~sklearn.datasets.fetch_california_housing`) and the Ames housing
|
||||
dataset. You can load the datasets as follows::
|
||||
data_url = "http://lib.stat.cmu.edu/datasets/boston"
|
||||
raw_df = pd.read_csv(data_url, sep="\s+", skiprows=22, header=None)
|
||||
data = np.hstack([raw_df.values[::2, :], raw_df.values[1::2, :2]])
|
||||
target = raw_df.values[1::2, 2]
|
||||
|
||||
from sklearn.datasets import fetch_california_housing
|
||||
housing = fetch_california_housing()
|
||||
Alternative datasets include the California housing dataset and the
|
||||
Ames housing dataset. You can load the datasets as follows::
|
||||
|
||||
for the California housing dataset and::
|
||||
from sklearn.datasets import fetch_california_housing
|
||||
housing = fetch_california_housing()
|
||||
|
||||
from sklearn.datasets import fetch_openml
|
||||
housing = fetch_openml(name="house_prices", as_frame=True)
|
||||
for the California housing dataset and::
|
||||
|
||||
for the Ames housing dataset.
|
||||
|
||||
warnings.warn(msg, category=FutureWarning)
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>dict_keys(['data', 'target', 'feature_names', 'DESCR', 'filename', 'data_module'])
|
||||
from sklearn.datasets import fetch_openml
|
||||
housing = fetch_openml(name="house_prices", as_frame=True)
|
||||
|
||||
for the Ames housing dataset.
|
||||
|
||||
[1] M Carlisle.
|
||||
"Racist data destruction?"
|
||||
<https://medium.com/@docintangible/racist-data-destruction-113e3eff54a8>
|
||||
|
||||
[2] Harrison Jr, David, and Daniel L. Rubinfeld.
|
||||
"Hedonic housing prices and the demand for clean air."
|
||||
Journal of environmental economics and management 5.1 (1978): 81-102.
|
||||
<https://www.researchgate.net/publication/4974606_Hedonic_housing_prices_and_the_demand_for_clean_air>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -2964,25 +2871,6 @@ the house using the features (predictors) listed here.</p>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>CRIM 0
|
||||
ZN 0
|
||||
INDUS 0
|
||||
CHAS 0
|
||||
NOX 0
|
||||
RM 0
|
||||
AGE 0
|
||||
DIS 0
|
||||
RAD 0
|
||||
TAX 0
|
||||
PTRATIO 0
|
||||
B 0
|
||||
LSTAT 0
|
||||
MEDV 0
|
||||
dtype: int64
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<p>We can then visualize the data</p>
|
||||
<div class="cell docutils container">
|
||||
@@ -2996,13 +2884,6 @@ dtype: int64
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/seaborn/distributions.py:2619: FutureWarning: `distplot` is a deprecated function and will be removed in a future version. Please adapt your code to use either `displot` (a figure-level function with similar flexibility) or `histplot` (an axes-level function for histograms).
|
||||
warnings.warn(msg, FutureWarning)
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/week35_199_1.png" src="_images/week35_199_1.png" />
|
||||
</div>
|
||||
</div>
|
||||
<p>It is now useful to look at the correlation matrix</p>
|
||||
<div class="cell docutils container">
|
||||
@@ -3015,12 +2896,6 @@ dtype: int64
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span><AxesSubplot:>
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/week35_201_1.png" src="_images/week35_201_1.png" />
|
||||
</div>
|
||||
</div>
|
||||
<p>From the above coorelation plot we can see that <strong>MEDV</strong> is strongly correlated to <strong>LSTAT</strong> and <strong>RM</strong>. We see also that <strong>RAD</strong> and <strong>TAX</strong> are stronly correlated, but we don’t include this in our features together to avoid multi-colinearity</p>
|
||||
<div class="cell docutils container">
|
||||
@@ -3041,9 +2916,6 @@ dtype: int64
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<img alt="_images/week35_203_0.png" src="_images/week35_203_0.png" />
|
||||
</div>
|
||||
</div>
|
||||
<p>Now we start training our model</p>
|
||||
<div class="cell docutils container">
|
||||
@@ -3069,14 +2941,6 @@ dtype: int64
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>(404, 2)
|
||||
(102, 2)
|
||||
(404,)
|
||||
(102,)
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<p>Then we use the linear regression functionality from <strong>Scikit-Learn</strong></p>
|
||||
<div class="cell docutils container">
|
||||
@@ -3115,20 +2979,6 @@ dtype: int64
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>The model performance for training set
|
||||
--------------------------------------
|
||||
RMSE is 5.637129335071195
|
||||
R2 score is 0.6300745149331701
|
||||
|
||||
|
||||
The model performance for testing set
|
||||
--------------------------------------
|
||||
RMSE is 5.137400784702911
|
||||
R2 score is 0.6628996975186953
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
@@ -3139,9 +2989,6 @@ R2 score is 0.6628996975186953
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<img alt="_images/week35_210_0.png" src="_images/week35_210_0.png" />
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="section" id="material-for-lecture-thursday-august-31">
|
||||
@@ -3421,25 +3268,6 @@ In general the economy-size SVD leads to less FLOPS and still conserving the des
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[ 1. -1.]
|
||||
[ 1. -1.]]
|
||||
test U
|
||||
[[0. 0.]
|
||||
[0. 0.]]
|
||||
test VT
|
||||
[[0. 0.]
|
||||
[0. 0.]]
|
||||
[[-0.70710678 -0.70710678]
|
||||
[-0.70710678 0.70710678]]
|
||||
[2.00000000e+00 3.35470445e-17]
|
||||
[[-0.70710678 0.70710678]
|
||||
[ 0.70710678 0.70710678]]
|
||||
[[-3.33066907e-16 4.44089210e-16]
|
||||
[ 0.00000000e+00 2.22044605e-16]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<p>The matrix <span class="math notranslate nohighlight">\(\boldsymbol{X}\)</span> has columns that are linearly dependent. The first
|
||||
column is the row-wise sum of the other two columns. The rank of a
|
||||
@@ -3795,14 +3623,6 @@ covariance matrix through the <strong>np.linalg.eig()</strong> function.</p>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.10790125813226321
|
||||
4.340071371496255
|
||||
[[ 1.04193203 3.08165104]
|
||||
[ 3.08165104 10.18383522]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="section" id="correlation-matrix">
|
||||
@@ -3838,14 +3658,6 @@ a more brute force way. Here we scale the mean values for each column of the des
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.08881497884574564
|
||||
1.7086067479626619
|
||||
[[1. 0.66080313]
|
||||
[0.66080313 1. ]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<p>We see that the matrix elements along the diagonal are one as they
|
||||
should be and that the matrix is symmetric. Furthermore, diagonalizing
|
||||
@@ -3874,34 +3686,6 @@ this matrix we easily see that it is a positive definite matrix.</p>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[-0.40620066 -2.01265755]
|
||||
[ 0.01458611 0.37737221]
|
||||
[-1.0895387 -3.65442354]
|
||||
[ 0.2338675 1.12044974]
|
||||
[ 0.4676059 1.54393936]
|
||||
[-0.65891389 -3.16304863]
|
||||
[-0.1715252 0.39197698]
|
||||
[ 0.71142161 2.95511792]
|
||||
[ 0.39214397 0.13069442]
|
||||
[ 0.50655336 2.3105791 ]]
|
||||
0 1
|
||||
0 -0.406201 -2.012658
|
||||
1 0.014586 0.377372
|
||||
2 -1.089539 -3.654424
|
||||
3 0.233868 1.120450
|
||||
4 0.467606 1.543939
|
||||
5 -0.658914 -3.163049
|
||||
6 -0.171525 0.391977
|
||||
7 0.711422 2.955118
|
||||
8 0.392144 0.130694
|
||||
9 0.506553 2.310579
|
||||
0 1
|
||||
0 1.000000 0.952387
|
||||
1 0.952387 1.000000
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<p>We expand this model to the Franke function discussed above.</p>
|
||||
</div>
|
||||
@@ -3955,43 +3739,6 @@ this matrix we easily see that it is a positive definite matrix.</p>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 0 1 2 3 4 5 6 7 \
|
||||
0 0.0 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
|
||||
1 0.0 0.078974 0.081276 0.075889 0.077168 0.078540 0.065410 0.066474
|
||||
2 0.0 0.081276 0.084076 0.078336 0.079946 0.081655 0.067707 0.069028
|
||||
3 0.0 0.075889 0.078336 0.077460 0.078986 0.080616 0.069452 0.070737
|
||||
4 0.0 0.077168 0.079946 0.078986 0.080764 0.082653 0.070986 0.072476
|
||||
5 0.0 0.078540 0.081655 0.080616 0.082653 0.084809 0.072621 0.074323
|
||||
6 0.0 0.065410 0.067707 0.069452 0.070986 0.072621 0.064074 0.065378
|
||||
7 0.0 0.066474 0.069028 0.070737 0.072476 0.074323 0.065378 0.066854
|
||||
8 0.0 0.067637 0.070457 0.072132 0.074084 0.076150 0.066787 0.068441
|
||||
9 0.0 0.068906 0.072000 0.073644 0.075816 0.078110 0.068307 0.070146
|
||||
10 0.0 0.055734 0.057835 0.060872 0.062337 0.063894 0.057393 0.058645
|
||||
11 0.0 0.056683 0.058996 0.062016 0.063653 0.065390 0.058552 0.059951
|
||||
12 0.0 0.057722 0.060254 0.063260 0.065077 0.066999 0.059807 0.061359
|
||||
13 0.0 0.058854 0.061614 0.064609 0.066612 0.068727 0.061163 0.062874
|
||||
14 0.0 0.060083 0.063080 0.066066 0.068264 0.070582 0.062624 0.064501
|
||||
|
||||
8 9 10 11 12 13 14
|
||||
0 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
|
||||
1 0.067637 0.068906 0.055734 0.056683 0.057722 0.058854 0.060083
|
||||
2 0.070457 0.072000 0.057835 0.058996 0.060254 0.061614 0.063080
|
||||
3 0.072132 0.073644 0.060872 0.062016 0.063260 0.064609 0.066066
|
||||
4 0.074084 0.075816 0.062337 0.063653 0.065077 0.066612 0.068264
|
||||
5 0.076150 0.078110 0.063894 0.065390 0.066999 0.068727 0.070582
|
||||
6 0.066787 0.068307 0.057393 0.058552 0.059807 0.061163 0.062624
|
||||
7 0.068441 0.070146 0.058645 0.059951 0.061359 0.062874 0.064501
|
||||
8 0.070213 0.072111 0.059993 0.061452 0.063019 0.064699 0.066500
|
||||
9 0.072111 0.074210 0.061443 0.063061 0.064793 0.066647 0.068629
|
||||
10 0.059993 0.061443 0.052305 0.053417 0.054617 0.055910 0.057300
|
||||
11 0.061452 0.063061 0.053417 0.054655 0.055987 0.057418 0.058952
|
||||
12 0.063019 0.064793 0.054617 0.055987 0.057457 0.059031 0.060716
|
||||
13 0.064699 0.066647 0.055910 0.057418 0.059031 0.060756 0.062599
|
||||
14 0.066500 0.068629 0.057300 0.058952 0.060716 0.062599 0.064606
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<p>We note here that the covariance is zero for the first rows and
|
||||
columns since all matrix elements in the design matrix were set to one
|
||||
@@ -4331,13 +4078,6 @@ C(\boldsymbol{X},\boldsymbol{\beta})=\frac{1}{n}\left\{(\boldsymbol{y}-\boldsymb
|
||||
<p class="prev-next-title">Exercises week 35</p>
|
||||
</div>
|
||||
</a>
|
||||
<a class='right-next' id="next-link" href="exercisesweek36.html" title="next page">
|
||||
<div class="prev-next-info">
|
||||
<p class="prev-next-subtitle">next</p>
|
||||
<p class="prev-next-title">Exercises week 36</p>
|
||||
</div>
|
||||
<i class="fas fa-angle-right"></i>
|
||||
</a>
|
||||
</div>
|
||||
|
||||
</div>
|
||||
|
||||
Reference in New Issue
Block a user