update week34
This commit is contained in:
Morten Hjorth-Jensen
@@ -43,3 +43,12 @@ Translating doconce text in chapter1.do.txt to ipynb
|
||||
Failed to remove ans_at_end environment
|
||||
Failed to remove sol_at_end environment
|
||||
output in chapter1.ipynb
|
||||
Translating doconce text in chapter1.do.txt to ipynb
|
||||
*** replacing \bm{...} by \boldsymbol{...} (\bm is not supported by MathJax)
|
||||
|
||||
*** warning: latex envir \begin{bmatrix} does not work well in Markdown. Stick to \[ ... \], equation, equation*, align, or align* environments in math environments.
|
||||
|
||||
*** warning: latex envir \begin{bmatrix} does not work well in Markdown. Stick to \[ ... \], equation, equation*, align, or align* environments in math environments.
|
||||
Failed to remove ans_at_end environment
|
||||
Failed to remove sol_at_end environment
|
||||
output in chapter1.ipynb
|
||||
|
||||
@@ -495,11 +495,13 @@ ways of dealing with outliers.
|
||||
The Huber cost function is defined as
|
||||
!bt
|
||||
\[
|
||||
H_{\delta}(a)=\left\{\begin{array}\frac{1}{2}a^{2}&{\text{for }}|a|\leq \delta ,\\ \delta (|a|-\frac {1}{2}\del\
|
||||
ta ),&{\text{otherwise.}\end{array}\right.
|
||||
H_{\delta}(\bm{a})=\left\{\begin{array}{cc}\frac{1}{2} \bm{a}^{2}& \text{for }|\bm{a}|\leq \delta\\ \delta (|\b\
|
||||
m{a}|-\frac{1}{2}\delta ),&\text{otherwise}.\end{array}\right.
|
||||
\]
|
||||
!et
|
||||
Here $a=\bm{y} - \bm{\tilde{y}}$.
|
||||
Here $\bm{a}=\bm{y} - \bm{\tilde{y}}$.
|
||||
|
||||
|
||||
We will discuss in more
|
||||
detail these and other functions in the various lectures. We conclude this part with another example. Instead of
|
||||
a linear $x$-dependence we study now a cubic polynomial and use the polynomial regression analysis tools of scikit-learn.
|
||||
@@ -2181,7 +2183,7 @@ It also common to split the data in a _training_ set and a _testing_ set. A typi
|
||||
for testing. This can be done as follows with our design matrix $\bm{X}$ and data $\bm{y}$ (remember to import _scikit-learn_)
|
||||
!bc pycod
|
||||
# split in training and test data
|
||||
X_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.2)
|
||||
# X_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.2)
|
||||
!ec
|
||||
Then we can use the standard scaler to scale our data as
|
||||
!bc pycod
|
||||
@@ -2211,16 +2213,27 @@ x = np.linspace(-3, 3, n).reshape(-1, 1)
|
||||
y = np.exp(-x**2) + 1.5 * np.exp(-(x-2)**2)+ np.random.normal(0, 0.1, x.shape)
|
||||
!ec
|
||||
where $y$ is the function we want to fit with a given polynomial.
|
||||
!bsubex
|
||||
Write a first code which sets up a design matrix $X$ defined by a fifth-order polynomial. Scale your data and split it in training and test data.
|
||||
!esubex
|
||||
|
||||
!bsubex
|
||||
Perform an ordinary least squares and compute the means squared error and the $R2$ factor for the training data and the test data, with and without scaling.
|
||||
!esubex
|
||||
|
||||
!bsubex
|
||||
Add now a model which allows you to make polynomials up to degree $15$. Perform a standard OLS fitting of the training data and compute the MSE and $R2$ for the training and test data and plot both test and training data MSE and $R2$ as functions of the polynomial degree. Compare what you see with Figure 2.11 of Hastie et al. Comment your results. For which polynomial degree do you find an optimal MSE (smallest value)?
|
||||
Write a first code which sets up a design matrix $X$ defined by a
|
||||
fifth-order polynomial. Scale your data and split it in training and
|
||||
test data.
|
||||
|
||||
|
||||
|
||||
Perform an ordinary least squares and compute the means squared error
|
||||
and the $R2$ factor for the training data and the test data, with and
|
||||
without scaling.
|
||||
|
||||
|
||||
|
||||
Add now a model which allows you to make polynomials up to degree
|
||||
$15$. Perform a standard OLS fitting of the training data and compute
|
||||
the MSE and $R2$ for the training and test data and plot both test and
|
||||
training data MSE and $R2$ as functions of the polynomial
|
||||
degree. Compare what you see with Figure 2.11 of Hastie et al. Comment
|
||||
your results. For which polynomial degree do you find an optimal MSE
|
||||
(smallest value)?
|
||||
|
||||
|
||||
!esubex
|
||||
|
||||
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
|
Before Width: | Height: | Size: 6.5 KiB After Width: | Height: | Size: 6.4 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 9.2 KiB After Width: | Height: | Size: 9.1 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 13 KiB After Width: | Height: | Size: 14 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 9.4 KiB After Width: | Height: | Size: 9.7 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 26 KiB After Width: | Height: | Size: 25 KiB |
@@ -629,8 +629,8 @@
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"H_{\\delta}(a)=\\left\\{\\begin{array}\\frac{1}{2}a^{2}&{\\text{for }}|a|\\leq \\delta ,\\\\ \\delta (|a|-\\frac {1}{2}\\del\\\n",
|
||||
"ta ),&{\\text{otherwise.}\\end{array}\\right.\n",
|
||||
"H_{\\delta}(\\boldsymbol{a})=\\left\\{\\begin{array}{cc}\\frac{1}{2} \\boldsymbol{a}^{2}& \\text{for }|\\boldsymbol{a}|\\leq \\delta\\\\ \\delta (|\\b\\\n",
|
||||
"m{a}|-\\frac{1}{2}\\delta ),&\\text{otherwise}.\\end{array}\\right.\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
@@ -638,7 +638,9 @@
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Here $a=\\boldsymbol{y} - \\boldsymbol{\\tilde{y}}$.\n",
|
||||
"Here $\\boldsymbol{a}=\\boldsymbol{y} - \\boldsymbol{\\tilde{y}}$.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"We will discuss in more\n",
|
||||
"detail these and other functions in the various lectures. We conclude this part with another example. Instead of \n",
|
||||
"a linear $x$-dependence we study now a cubic polynomial and use the polynomial regression analysis tools of scikit-learn."
|
||||
@@ -3412,7 +3414,7 @@
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# split in training and test data\n",
|
||||
"X_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.2)"
|
||||
"# X_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.2)"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -3475,18 +3477,27 @@
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"where $y$ is the function we want to fit with a given polynomial.\n",
|
||||
"!bsubex\n",
|
||||
"Write a first code which sets up a design matrix $X$ defined by a fifth-order polynomial. Scale your data and split it in training and test data. \n",
|
||||
"!esubex\n",
|
||||
"\n",
|
||||
"!bsubex\n",
|
||||
"Perform an ordinary least squares and compute the means squared error and the $R2$ factor for the training data and the test data, with and without scaling.\n",
|
||||
"!esubex\n",
|
||||
"\n",
|
||||
"!bsubex\n",
|
||||
"Add now a model which allows you to make polynomials up to degree $15$. Perform a standard OLS fitting of the training data and compute the MSE and $R2$ for the training and test data and plot both test and training data MSE and $R2$ as functions of the polynomial degree. Compare what you see with Figure 2.11 of Hastie et al. Comment your results. For which polynomial degree do you find an optimal MSE (smallest value)?\n",
|
||||
"Write a first code which sets up a design matrix $X$ defined by a\n",
|
||||
"fifth-order polynomial. Scale your data and split it in training and\n",
|
||||
"test data.\n",
|
||||
"\n",
|
||||
"!esubex"
|
||||
"\n",
|
||||
"\n",
|
||||
"Perform an ordinary least squares and compute the means squared error\n",
|
||||
"and the $R2$ factor for the training data and the test data, with and\n",
|
||||
"without scaling.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"Add now a model which allows you to make polynomials up to degree\n",
|
||||
"$15$. Perform a standard OLS fitting of the training data and compute\n",
|
||||
"the MSE and $R2$ for the training and test data and plot both test and\n",
|
||||
"training data MSE and $R2$ as functions of the polynomial\n",
|
||||
"degree. Compare what you see with Figure 2.11 of Hastie et al. Comment\n",
|
||||
"your results. For which polynomial degree do you find an optimal MSE\n",
|
||||
"(smallest value)?"
|
||||
]
|
||||
}
|
||||
],
|
||||
|
||||
@@ -781,13 +781,13 @@ example of the functionality of <strong>Scikit-Learn</strong>.</p>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>The intercept alpha:
|
||||
[2.15024669]
|
||||
[1.99723611]
|
||||
Coefficient beta :
|
||||
[[4.89975818]]
|
||||
Mean squared error: 0.26
|
||||
[[5.1007125]]
|
||||
Mean squared error: 0.22
|
||||
Variance score: 0.90
|
||||
Mean squared log error: 0.01
|
||||
Mean absolute error: 0.42
|
||||
Mean absolute error: 0.36
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapter1_13_1.png" src="_images/chapter1_13_1.png" />
|
||||
@@ -844,11 +844,11 @@ ways of dealing with outliers.</p>
|
||||
<p>The Huber cost function is defined as</p>
|
||||
<div class="math notranslate nohighlight">
|
||||
\[\begin{split}
|
||||
H_{\delta}(a)=\left\{\begin{array}\frac{1}{2}a^{2}&{\text{for }}|a|\leq \delta ,\\ \delta (|a|-\frac {1}{2}\del\
|
||||
ta ),&{\text{otherwise.}\end{array}\right.
|
||||
H_{\delta}(\boldsymbol{a})=\left\{\begin{array}{cc}\frac{1}{2} \boldsymbol{a}^{2}& \text{for }|\boldsymbol{a}|\leq \delta\\ \delta (|\b\
|
||||
m{a}|-\frac{1}{2}\delta ),&\text{otherwise}.\end{array}\right.
|
||||
\end{split}\]</div>
|
||||
<p>Here <span class="math notranslate nohighlight">\(a=\boldsymbol{y} - \boldsymbol{\tilde{y}}\)</span>.
|
||||
We will discuss in more
|
||||
<p>Here <span class="math notranslate nohighlight">\(\boldsymbol{a}=\boldsymbol{y} - \boldsymbol{\tilde{y}}\)</span>.</p>
|
||||
<p>We will discuss in more
|
||||
detail these and other functions in the various lectures. We conclude this part with another example. Instead of
|
||||
a linear <span class="math notranslate nohighlight">\(x\)</span>-dependence we study now a cubic polynomial and use the polynomial regression analysis tools of scikit-learn.</p>
|
||||
<div class="cell docutils container">
|
||||
@@ -888,7 +888,7 @@ a linear <span class="math notranslate nohighlight">\(x\)</span>-dependence we s
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<img alt="_images/chapter1_27_0.png" src="_images/chapter1_27_0.png" />
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.0050000000000000044
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.005000000000000007
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1283,6 +1283,38 @@ functionality.</p>
|
||||
warnings.warn(
|
||||
/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
@@ -1317,42 +1349,12 @@ functionality.</p>
|
||||
warnings.warn(
|
||||
/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapter1_61_10.png" src="_images/chapter1_61_10.png" />
|
||||
@@ -2330,13 +2332,13 @@ but now splitting the data into a training set and a test set.</p>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Training R2
|
||||
0.9999868619217517
|
||||
0.9999878260589065
|
||||
Training MSE
|
||||
5.965885569080809
|
||||
5.415634483874318
|
||||
Test R2
|
||||
0.9999794306626945
|
||||
0.999974281880048
|
||||
Test MSE
|
||||
8.300162456113691
|
||||
11.653095390463358
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -2747,31 +2749,31 @@ techniques.</p>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>MSE before scaling: 0.00
|
||||
R2 score before scaling 1.00
|
||||
R2 score before scaling 0.99
|
||||
Feature min values before scaling:
|
||||
[1.00000000e+00 2.54152940e-03 1.38207279e-03 6.45937170e-06
|
||||
3.51257863e-06 1.91012519e-06 1.64166831e-08 8.92732185e-09
|
||||
4.85463934e-09 2.63993205e-09 4.17234827e-11 2.26890510e-11
|
||||
1.23382086e-11 6.70946493e-12 3.64857826e-12 1.06041458e-13
|
||||
5.76648901e-14 3.13579199e-14 1.70523024e-14 9.27296891e-15
|
||||
5.04260073e-15]
|
||||
[1.00000000e+00 2.31109204e-04 5.13266452e-03 5.34114641e-08
|
||||
1.18620601e-06 2.63442451e-05 1.23438810e-11 2.74143127e-10
|
||||
6.08839750e-09 1.35216172e-07 2.85278450e-15 6.33569998e-14
|
||||
1.40708470e-12 3.12497018e-11 6.94019247e-10 6.59304755e-19
|
||||
1.46423858e-17 3.25190225e-16 7.22209371e-15 1.60394236e-13
|
||||
3.56216797e-12]
|
||||
Feature max values before scaling:
|
||||
[1. 0.99817842 0.99945628 0.99636015 0.99763569 0.99891285
|
||||
0.9945452 0.99581841 0.99709325 0.99836973 0.99273355 0.99400444
|
||||
0.99527696 0.99655111 0.99782689 0.9909252 0.99219378 0.99346398
|
||||
0.99473581 0.99600927 0.99728435]
|
||||
[1. 0.99997737 0.9950799 0.99995475 0.99505739 0.99018401
|
||||
0.99993212 0.99503487 0.99016161 0.98531221 0.9999095 0.99501236
|
||||
0.99013921 0.98528992 0.98046438 0.99988687 0.99498985 0.9901168
|
||||
0.98526763 0.9804422 0.9756404 ]
|
||||
Feature min values after scaling:
|
||||
[ 0. -1.63437572 -1.76504618 -1.03871832 -1.08415761 -1.13457922
|
||||
-0.81620806 -0.83935285 -0.86436607 -0.8914984 -0.69347005 -0.70790937
|
||||
-0.72312577 -0.73921714 -0.75629493 -0.61234223 -0.6226921 -0.63339159
|
||||
-0.64447921 -0.65599927 -0.66800261]
|
||||
[ 0. -1.7621419 -1.74382593 -1.13803928 -1.14662074 -1.15475232
|
||||
-0.89942598 -0.90825063 -0.9171356 -0.92605247 -0.76570177 -0.77317984
|
||||
-0.78080633 -0.78857629 -0.79648291 -0.67708423 -0.68316185 -0.68937695
|
||||
-0.69573183 -0.7022283 -0.70886748]
|
||||
Feature max values after scaling:
|
||||
[0. 1.86574276 1.72218808 2.41770932 2.31730641 2.21347282
|
||||
2.88297395 2.79328828 2.70234019 2.60999846 3.291614 3.20772452
|
||||
3.12283463 3.03697069 2.95014575 3.65766387 3.57871326 3.49865673
|
||||
3.41754964 3.33544681 3.25240108]
|
||||
[0. 1.71669651 1.63315151 2.21623344 2.15324631 2.088871
|
||||
2.62856593 2.5755035 2.52158335 2.46675058 2.987902 2.94034531
|
||||
2.8923 2.84372853 2.79459035 3.30968181 3.2653678 3.22076362
|
||||
3.1758506 3.13060772 3.08501147]
|
||||
MSE after scaling: 0.00
|
||||
R2 score for scaled data: 1.00
|
||||
R2 score for scaled data: 0.99
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -2907,39 +2909,7 @@ for testing. This can be done as follows with our design matrix <span class="mat
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="c1"># split in training and test data</span>
|
||||
<span class="n">X_train</span><span class="p">,</span> <span class="n">X_test</span><span class="p">,</span> <span class="n">y_train</span><span class="p">,</span> <span class="n">y_test</span> <span class="o">=</span> <span class="n">train_test_split</span><span class="p">(</span><span class="n">X</span><span class="p">,</span><span class="n">y</span><span class="p">,</span><span class="n">test_size</span><span class="o">=</span><span class="mf">0.2</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output traceback highlight-ipythontb notranslate"><div class="highlight"><pre><span></span><span class="gt">---------------------------------------------------------------------------</span>
|
||||
<span class="ne">ValueError</span><span class="g g-Whitespace"> </span>Traceback (most recent call last)
|
||||
<span class="o"><</span><span class="n">ipython</span><span class="o">-</span><span class="nb">input</span><span class="o">-</span><span class="mi">38</span><span class="o">-</span><span class="mi">9</span><span class="n">b9cf4fa1a95</span><span class="o">></span> <span class="ow">in</span> <span class="o"><</span><span class="n">module</span><span class="o">></span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">1</span> <span class="c1"># split in training and test data</span>
|
||||
<span class="ne">----> </span><span class="mi">2</span> <span class="n">X_train</span><span class="p">,</span> <span class="n">X_test</span><span class="p">,</span> <span class="n">y_train</span><span class="p">,</span> <span class="n">y_test</span> <span class="o">=</span> <span class="n">train_test_split</span><span class="p">(</span><span class="n">X</span><span class="p">,</span><span class="n">y</span><span class="p">,</span><span class="n">test_size</span><span class="o">=</span><span class="mf">0.2</span><span class="p">)</span>
|
||||
|
||||
<span class="nn">~/opt/anaconda3/lib/python3.8/site-packages/sklearn/model_selection/_split.py</span> in <span class="ni">train_test_split</span><span class="nt">(*arrays, **options)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">2125</span> <span class="k">raise</span> <span class="ne">TypeError</span><span class="p">(</span><span class="s2">"Invalid parameters passed: </span><span class="si">%s</span><span class="s2">"</span> <span class="o">%</span> <span class="nb">str</span><span class="p">(</span><span class="n">options</span><span class="p">))</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">2126</span>
|
||||
<span class="ne">-> </span><span class="mi">2127</span> <span class="n">arrays</span> <span class="o">=</span> <span class="n">indexable</span><span class="p">(</span><span class="o">*</span><span class="n">arrays</span><span class="p">)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">2128</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">2129</span> <span class="n">n_samples</span> <span class="o">=</span> <span class="n">_num_samples</span><span class="p">(</span><span class="n">arrays</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span>
|
||||
|
||||
<span class="nn">~/opt/anaconda3/lib/python3.8/site-packages/sklearn/utils/validation.py</span> in <span class="ni">indexable</span><span class="nt">(*iterables)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">290</span> <span class="s2">"""</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">291</span><span class="s2"> result = [_make_indexable(X) for X in iterables]</span>
|
||||
<span class="ne">--> </span><span class="mi">292</span><span class="s2"> check_consistent_length(*result)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">293</span><span class="s2"> return result</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">294</span><span class="s2"> </span>
|
||||
|
||||
<span class="nn">~/opt/anaconda3/lib/python3.8/site-packages/sklearn/utils/validation.py</span> in <span class="ni">check_consistent_length</span><span class="nt">(*arrays)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">253</span><span class="s2"> uniques = np.unique(lengths)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">254</span><span class="s2"> if len(uniques) > 1:</span>
|
||||
<span class="ne">--> </span><span class="mi">255</span><span class="s2"> raise ValueError("Found input variables with inconsistent numbers of"</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">256</span><span class="s2"> " samples: </span><span class="si">%r</span><span class="s2">" % [int(l) for l in lengths])</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">257</span><span class="s2"> </span>
|
||||
|
||||
<span class="ne">ValueError</span>: Found input variables with inconsistent numbers of samples: [1000, 100]
|
||||
<span class="c1"># X_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.2)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -2973,16 +2943,20 @@ the aims is to reproduce Figure 2.11 of <a class="reference external" href="http
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<p>where <span class="math notranslate nohighlight">\(y\)</span> is the function we want to fit with a given polynomial.
|
||||
!bsubex
|
||||
Write a first code which sets up a design matrix <span class="math notranslate nohighlight">\(X\)</span> defined by a fifth-order polynomial. Scale your data and split it in training and test data.
|
||||
!esubex</p>
|
||||
<p>!bsubex
|
||||
Perform an ordinary least squares and compute the means squared error and the <span class="math notranslate nohighlight">\(R2\)</span> factor for the training data and the test data, with and without scaling.
|
||||
!esubex</p>
|
||||
<p>!bsubex
|
||||
Add now a model which allows you to make polynomials up to degree <span class="math notranslate nohighlight">\(15\)</span>. Perform a standard OLS fitting of the training data and compute the MSE and <span class="math notranslate nohighlight">\(R2\)</span> for the training and test data and plot both test and training data MSE and <span class="math notranslate nohighlight">\(R2\)</span> as functions of the polynomial degree. Compare what you see with Figure 2.11 of Hastie et al. Comment your results. For which polynomial degree do you find an optimal MSE (smallest value)?</p>
|
||||
<p>!esubex</p>
|
||||
<p>where <span class="math notranslate nohighlight">\(y\)</span> is the function we want to fit with a given polynomial.</p>
|
||||
<p>Write a first code which sets up a design matrix <span class="math notranslate nohighlight">\(X\)</span> defined by a
|
||||
fifth-order polynomial. Scale your data and split it in training and
|
||||
test data.</p>
|
||||
<p>Perform an ordinary least squares and compute the means squared error
|
||||
and the <span class="math notranslate nohighlight">\(R2\)</span> factor for the training data and the test data, with and
|
||||
without scaling.</p>
|
||||
<p>Add now a model which allows you to make polynomials up to degree
|
||||
<span class="math notranslate nohighlight">\(15\)</span>. Perform a standard OLS fitting of the training data and compute
|
||||
the MSE and <span class="math notranslate nohighlight">\(R2\)</span> for the training and test data and plot both test and
|
||||
training data MSE and <span class="math notranslate nohighlight">\(R2\)</span> as functions of the polynomial
|
||||
degree. Compare what you see with Figure 2.11 of Hastie et al. Comment
|
||||
your results. For which polynomial degree do you find an optimal MSE
|
||||
(smallest value)?</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -483,11 +483,13 @@ ways of dealing with outliers.
|
||||
The Huber cost function is defined as
|
||||
|
||||
$$
|
||||
H_{\delta}(a)=\left\{\begin{array}\frac{1}{2}a^{2}&{\text{for }}|a|\leq \delta ,\\ \delta (|a|-\frac {1}{2}\del\
|
||||
ta ),&{\text{otherwise.}\end{array}\right.
|
||||
H_{\delta}(\boldsymbol{a})=\left\{\begin{array}{cc}\frac{1}{2} \boldsymbol{a}^{2}& \text{for }|\boldsymbol{a}|\leq \delta\\ \delta (|\b\
|
||||
m{a}|-\frac{1}{2}\delta ),&\text{otherwise}.\end{array}\right.
|
||||
$$
|
||||
|
||||
Here $a=\boldsymbol{y} - \boldsymbol{\tilde{y}}$.
|
||||
Here $\boldsymbol{a}=\boldsymbol{y} - \boldsymbol{\tilde{y}}$.
|
||||
|
||||
|
||||
We will discuss in more
|
||||
detail these and other functions in the various lectures. We conclude this part with another example. Instead of
|
||||
a linear $x$-dependence we study now a cubic polynomial and use the polynomial regression analysis tools of scikit-learn.
|
||||
@@ -2146,7 +2148,7 @@ It also common to split the data in a **training** set and a **testing** set. A
|
||||
for testing. This can be done as follows with our design matrix $\boldsymbol{X}$ and data $\boldsymbol{y}$ (remember to import **scikit-learn**)
|
||||
|
||||
# split in training and test data
|
||||
X_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.2)
|
||||
# X_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.2)
|
||||
|
||||
Then we can use the standard scaler to scale our data as
|
||||
|
||||
@@ -2174,15 +2176,24 @@ x = np.linspace(-3, 3, n).reshape(-1, 1)
|
||||
y = np.exp(-x**2) + 1.5 * np.exp(-(x-2)**2)+ np.random.normal(0, 0.1, x.shape)
|
||||
|
||||
where $y$ is the function we want to fit with a given polynomial.
|
||||
!bsubex
|
||||
Write a first code which sets up a design matrix $X$ defined by a fifth-order polynomial. Scale your data and split it in training and test data.
|
||||
!esubex
|
||||
|
||||
!bsubex
|
||||
Perform an ordinary least squares and compute the means squared error and the $R2$ factor for the training data and the test data, with and without scaling.
|
||||
!esubex
|
||||
|
||||
!bsubex
|
||||
Add now a model which allows you to make polynomials up to degree $15$. Perform a standard OLS fitting of the training data and compute the MSE and $R2$ for the training and test data and plot both test and training data MSE and $R2$ as functions of the polynomial degree. Compare what you see with Figure 2.11 of Hastie et al. Comment your results. For which polynomial degree do you find an optimal MSE (smallest value)?
|
||||
Write a first code which sets up a design matrix $X$ defined by a
|
||||
fifth-order polynomial. Scale your data and split it in training and
|
||||
test data.
|
||||
|
||||
!esubex
|
||||
|
||||
|
||||
Perform an ordinary least squares and compute the means squared error
|
||||
and the $R2$ factor for the training data and the test data, with and
|
||||
without scaling.
|
||||
|
||||
|
||||
|
||||
Add now a model which allows you to make polynomials up to degree
|
||||
$15$. Perform a standard OLS fitting of the training data and compute
|
||||
the MSE and $R2$ for the training and test data and plot both test and
|
||||
training data MSE and $R2$ as functions of the polynomial
|
||||
degree. Compare what you see with Figure 2.11 of Hastie et al. Comment
|
||||
your results. For which polynomial degree do you find an optimal MSE
|
||||
(smallest value)?
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 6.5 KiB After Width: | Height: | Size: 6.4 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 9.2 KiB After Width: | Height: | Size: 9.1 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 13 KiB After Width: | Height: | Size: 14 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 9.4 KiB After Width: | Height: | Size: 9.7 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 26 KiB After Width: | Height: | Size: 25 KiB |
@@ -629,8 +629,8 @@
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"H_{\\delta}(a)=\\left\\{\\begin{array}\\frac{1}{2}a^{2}&{\\text{for }}|a|\\leq \\delta ,\\\\ \\delta (|a|-\\frac {1}{2}\\del\\\n",
|
||||
"ta ),&{\\text{otherwise.}\\end{array}\\right.\n",
|
||||
"H_{\\delta}(\\boldsymbol{a})=\\left\\{\\begin{array}{cc}\\frac{1}{2} \\boldsymbol{a}^{2}& \\text{for }|\\boldsymbol{a}|\\leq \\delta\\\\ \\delta (|\\b\\\n",
|
||||
"m{a}|-\\frac{1}{2}\\delta ),&\\text{otherwise}.\\end{array}\\right.\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
@@ -638,7 +638,9 @@
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Here $a=\\boldsymbol{y} - \\boldsymbol{\\tilde{y}}$.\n",
|
||||
"Here $\\boldsymbol{a}=\\boldsymbol{y} - \\boldsymbol{\\tilde{y}}$.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"We will discuss in more\n",
|
||||
"detail these and other functions in the various lectures. We conclude this part with another example. Instead of \n",
|
||||
"a linear $x$-dependence we study now a cubic polynomial and use the polynomial regression analysis tools of scikit-learn."
|
||||
@@ -3412,7 +3414,7 @@
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# split in training and test data\n",
|
||||
"X_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.2)"
|
||||
"# X_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.2)"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -3475,18 +3477,27 @@
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"where $y$ is the function we want to fit with a given polynomial.\n",
|
||||
"!bsubex\n",
|
||||
"Write a first code which sets up a design matrix $X$ defined by a fifth-order polynomial. Scale your data and split it in training and test data. \n",
|
||||
"!esubex\n",
|
||||
"\n",
|
||||
"!bsubex\n",
|
||||
"Perform an ordinary least squares and compute the means squared error and the $R2$ factor for the training data and the test data, with and without scaling.\n",
|
||||
"!esubex\n",
|
||||
"\n",
|
||||
"!bsubex\n",
|
||||
"Add now a model which allows you to make polynomials up to degree $15$. Perform a standard OLS fitting of the training data and compute the MSE and $R2$ for the training and test data and plot both test and training data MSE and $R2$ as functions of the polynomial degree. Compare what you see with Figure 2.11 of Hastie et al. Comment your results. For which polynomial degree do you find an optimal MSE (smallest value)?\n",
|
||||
"Write a first code which sets up a design matrix $X$ defined by a\n",
|
||||
"fifth-order polynomial. Scale your data and split it in training and\n",
|
||||
"test data.\n",
|
||||
"\n",
|
||||
"!esubex"
|
||||
"\n",
|
||||
"\n",
|
||||
"Perform an ordinary least squares and compute the means squared error\n",
|
||||
"and the $R2$ factor for the training data and the test data, with and\n",
|
||||
"without scaling.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"Add now a model which allows you to make polynomials up to degree\n",
|
||||
"$15$. Perform a standard OLS fitting of the training data and compute\n",
|
||||
"the MSE and $R2$ for the training and test data and plot both test and\n",
|
||||
"training data MSE and $R2$ as functions of the polynomial\n",
|
||||
"degree. Compare what you see with Figure 2.11 of Hastie et al. Comment\n",
|
||||
"your results. For which polynomial degree do you find an optimal MSE\n",
|
||||
"(smallest value)?"
|
||||
]
|
||||
}
|
||||
],
|
||||
|
||||
@@ -640,13 +640,16 @@ ways of dealing with outliers.
|
||||
<p>
|
||||
The Huber cost function is defined as
|
||||
$$
|
||||
H_{\delta}(a)=\left\{\begin{array}\frac{1}{2}a^{2}&{\text{for }}|a|\leq \delta ,\\ \delta (|a|-\frac {1}{2}\delta ),&{\text{otherwise.}\end{array}\right.
|
||||
H_{\delta}(\boldsymbol{a})=\left\{\begin{array}{cc}\frac{1}{2} \boldsymbol{a}^{2}& \text{for }|\boldsymbol{a}|\leq \delta\\ \delta (|\boldsymbol{a}|-\frac{1}{2}\delta ),&\text{otherwise}.\end{array}\right.
|
||||
$$
|
||||
|
||||
Here \( a=\boldsymbol{y} - \boldsymbol{\tilde{y}} \).
|
||||
We will discuss in more
|
||||
detail these and other functions in the various lectures. We conclude this part with another example. Instead of
|
||||
a linear \( x \)-dependence we study now a cubic polynomial and use the polynomial regression analysis tools of scikit-learn.
|
||||
Here \( \boldsymbol{a}=\boldsymbol{y} - \boldsymbol{\tilde{y}} \).
|
||||
|
||||
<p>
|
||||
We will discuss in more detail these and other functions in the
|
||||
various lectures. We conclude this part with another example. Instead
|
||||
of a linear \( x \)-dependence we study now a cubic polynomial and use the
|
||||
polynomial regression analysis tools of scikit-learn.
|
||||
|
||||
<p>
|
||||
|
||||
|
||||
@@ -656,27 +656,6 @@ Perform an ordinary least squares and compute the means squared error and the \(
|
||||
<b>c)</b>
|
||||
Add now a model which allows you to make polynomials up to degree \( 15 \). Perform a standard OLS fitting of the training data and compute the MSE and \( R2 \) for the training and test data and plot both test and training data MSE and \( R2 \) as functions of the polynomial degree. Compare what you see with Figure 2.11 of Hastie et al. Comment your results. For which polynomial degree do you find an optimal MSE (smallest value)?
|
||||
|
||||
<p>
|
||||
<!-- --- begin solution of exercise --- -->
|
||||
|
||||
<p>
|
||||
<a class="glyphicon glyphicon-hand-right showdetails" data-toggle="collapse"
|
||||
data-target="#exer_3_1" style="font-size: 80%;"></a>
|
||||
<a href="#exer_3_1" data-toggle="collapse">
|
||||
<b>Solution.</b>
|
||||
</a>
|
||||
<div class="collapse-group">
|
||||
<p><div class="collapse" id="exer_3_1">
|
||||
|
||||
<p>
|
||||
Here you simply need to change the degree of the polynomial in the above code to \( n=15 \).
|
||||
</div></p>
|
||||
</div>
|
||||
</p>
|
||||
|
||||
<p>
|
||||
<!-- --- end solution of exercise --- -->
|
||||
|
||||
<p>
|
||||
<!-- --- end exercise --- -->
|
||||
|
||||
|
||||
@@ -1743,14 +1743,17 @@ ways of dealing with outliers.
|
||||
The Huber cost function is defined as
|
||||
<p> <br>
|
||||
$$
|
||||
H_{\delta}(a)=\left\{\begin{array}\frac{1}{2}a^{2}&{\text{for }}|a|\leq \delta ,\\ \delta (|a|-\frac {1}{2}\delta ),&{\text{otherwise.}\end{array}\right.
|
||||
H_{\delta}(\boldsymbol{a})=\left\{\begin{array}{cc}\frac{1}{2} \boldsymbol{a}^{2}& \text{for }|\boldsymbol{a}|\leq \delta\\ \delta (|\boldsymbol{a}|-\frac{1}{2}\delta ),&\text{otherwise}.\end{array}\right.
|
||||
$$
|
||||
<p> <br>
|
||||
|
||||
Here \( a=\boldsymbol{y} - \boldsymbol{\tilde{y}} \).
|
||||
We will discuss in more
|
||||
detail these and other functions in the various lectures. We conclude this part with another example. Instead of
|
||||
a linear \( x \)-dependence we study now a cubic polynomial and use the polynomial regression analysis tools of scikit-learn.
|
||||
Here \( \boldsymbol{a}=\boldsymbol{y} - \boldsymbol{\tilde{y}} \).
|
||||
|
||||
<p>
|
||||
We will discuss in more detail these and other functions in the
|
||||
various lectures. We conclude this part with another example. Instead
|
||||
of a linear \( x \)-dependence we study now a cubic polynomial and use the
|
||||
polynomial regression analysis tools of scikit-learn.
|
||||
|
||||
<p>
|
||||
|
||||
@@ -3579,12 +3582,6 @@ Perform an ordinary least squares and compute the means squared error and the \(
|
||||
<b>c)</b>
|
||||
Add now a model which allows you to make polynomials up to degree \( 15 \). Perform a standard OLS fitting of the training data and compute the MSE and \( R2 \) for the training and test data and plot both test and training data MSE and \( R2 \) as functions of the polynomial degree. Compare what you see with Figure 2.11 of Hastie et al. Comment your results. For which polynomial degree do you find an optimal MSE (smallest value)?
|
||||
|
||||
<p>
|
||||
<!-- --- begin solution of exercise --- -->
|
||||
<b>Solution.</b>
|
||||
Here you simply need to change the degree of the polynomial in the above code to \( n=15 \).
|
||||
<!-- --- end solution of exercise --- -->
|
||||
|
||||
<p>
|
||||
<!-- --- end exercise --- -->
|
||||
</section>
|
||||
|
||||
@@ -1826,13 +1826,16 @@ ways of dealing with outliers.
|
||||
<p>
|
||||
The Huber cost function is defined as
|
||||
$$
|
||||
H_{\delta}(a)=\left\{\begin{array}\frac{1}{2}a^{2}&{\text{for }}|a|\leq \delta ,\\ \delta (|a|-\frac {1}{2}\delta ),&{\text{otherwise.}\end{array}\right.
|
||||
H_{\delta}(\boldsymbol{a})=\left\{\begin{array}{cc}\frac{1}{2} \boldsymbol{a}^{2}& \text{for }|\boldsymbol{a}|\leq \delta\\ \delta (|\boldsymbol{a}|-\frac{1}{2}\delta ),&\text{otherwise}.\end{array}\right.
|
||||
$$
|
||||
|
||||
Here \( a=\boldsymbol{y} - \boldsymbol{\tilde{y}} \).
|
||||
We will discuss in more
|
||||
detail these and other functions in the various lectures. We conclude this part with another example. Instead of
|
||||
a linear \( x \)-dependence we study now a cubic polynomial and use the polynomial regression analysis tools of scikit-learn.
|
||||
Here \( \boldsymbol{a}=\boldsymbol{y} - \boldsymbol{\tilde{y}} \).
|
||||
|
||||
<p>
|
||||
We will discuss in more detail these and other functions in the
|
||||
various lectures. We conclude this part with another example. Instead
|
||||
of a linear \( x \)-dependence we study now a cubic polynomial and use the
|
||||
polynomial regression analysis tools of scikit-learn.
|
||||
|
||||
<p>
|
||||
|
||||
@@ -3562,12 +3565,6 @@ Perform an ordinary least squares and compute the means squared error and the \(
|
||||
<b>c)</b>
|
||||
Add now a model which allows you to make polynomials up to degree \( 15 \). Perform a standard OLS fitting of the training data and compute the MSE and \( R2 \) for the training and test data and plot both test and training data MSE and \( R2 \) as functions of the polynomial degree. Compare what you see with Figure 2.11 of Hastie et al. Comment your results. For which polynomial degree do you find an optimal MSE (smallest value)?
|
||||
|
||||
<p>
|
||||
<!-- --- begin solution of exercise --- -->
|
||||
<b>Solution.</b>
|
||||
Here you simply need to change the degree of the polynomial in the above code to \( n=15 \).
|
||||
<!-- --- end solution of exercise --- -->
|
||||
|
||||
<p>
|
||||
<!-- --- end exercise --- -->
|
||||
|
||||
|
||||
@@ -1831,13 +1831,16 @@ ways of dealing with outliers.
|
||||
<p>
|
||||
The Huber cost function is defined as
|
||||
$$
|
||||
H_{\delta}(a)=\left\{\begin{array}\frac{1}{2}a^{2}&{\text{for }}|a|\leq \delta ,\\ \delta (|a|-\frac {1}{2}\delta ),&{\text{otherwise.}\end{array}\right.
|
||||
H_{\delta}(\boldsymbol{a})=\left\{\begin{array}{cc}\frac{1}{2} \boldsymbol{a}^{2}& \text{for }|\boldsymbol{a}|\leq \delta\\ \delta (|\boldsymbol{a}|-\frac{1}{2}\delta ),&\text{otherwise}.\end{array}\right.
|
||||
$$
|
||||
|
||||
Here \( a=\boldsymbol{y} - \boldsymbol{\tilde{y}} \).
|
||||
We will discuss in more
|
||||
detail these and other functions in the various lectures. We conclude this part with another example. Instead of
|
||||
a linear \( x \)-dependence we study now a cubic polynomial and use the polynomial regression analysis tools of scikit-learn.
|
||||
Here \( \boldsymbol{a}=\boldsymbol{y} - \boldsymbol{\tilde{y}} \).
|
||||
|
||||
<p>
|
||||
We will discuss in more detail these and other functions in the
|
||||
various lectures. We conclude this part with another example. Instead
|
||||
of a linear \( x \)-dependence we study now a cubic polynomial and use the
|
||||
polynomial regression analysis tools of scikit-learn.
|
||||
|
||||
<p>
|
||||
|
||||
@@ -3567,12 +3570,6 @@ Perform an ordinary least squares and compute the means squared error and the \(
|
||||
<b>c)</b>
|
||||
Add now a model which allows you to make polynomials up to degree \( 15 \). Perform a standard OLS fitting of the training data and compute the MSE and \( R2 \) for the training and test data and plot both test and training data MSE and \( R2 \) as functions of the polynomial degree. Compare what you see with Figure 2.11 of Hastie et al. Comment your results. For which polynomial degree do you find an optimal MSE (smallest value)?
|
||||
|
||||
<p>
|
||||
<!-- --- begin solution of exercise --- -->
|
||||
<b>Solution.</b>
|
||||
Here you simply need to change the degree of the polynomial in the above code to \( n=15 \).
|
||||
<!-- --- end solution of exercise --- -->
|
||||
|
||||
<p>
|
||||
<!-- --- end exercise --- -->
|
||||
|
||||
|
||||
Binary file not shown.
@@ -1860,7 +1860,7 @@
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"H_{\\delta}(a)=\\left\\{\\begin{array}\\frac{1}{2}a^{2}&{\\text{for }}|a|\\leq \\delta ,\\\\ \\delta (|a|-\\frac {1}{2}\\delta ),&{\\text{otherwise.}\\end{array}\\right.\n",
|
||||
"H_{\\delta}(\\boldsymbol{a})=\\left\\{\\begin{array}{cc}\\frac{1}{2} \\boldsymbol{a}^{2}& \\text{for }|\\boldsymbol{a}|\\leq \\delta\\\\ \\delta (|\\boldsymbol{a}|-\\frac{1}{2}\\delta ),&\\text{otherwise}.\\end{array}\\right.\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
@@ -1868,10 +1868,13 @@
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Here $a=\\boldsymbol{y} - \\boldsymbol{\\tilde{y}}$.\n",
|
||||
"We will discuss in more\n",
|
||||
"detail these and other functions in the various lectures. We conclude this part with another example. Instead of \n",
|
||||
"a linear $x$-dependence we study now a cubic polynomial and use the polynomial regression analysis tools of scikit-learn."
|
||||
"Here $\\boldsymbol{a}=\\boldsymbol{y} - \\boldsymbol{\\tilde{y}}$.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"We will discuss in more detail these and other functions in the\n",
|
||||
"various lectures. We conclude this part with another example. Instead\n",
|
||||
"of a linear $x$-dependence we study now a cubic polynomial and use the\n",
|
||||
"polynomial regression analysis tools of scikit-learn."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -4396,12 +4399,6 @@
|
||||
"Add now a model which allows you to make polynomials up to degree $15$. Perform a standard OLS fitting of the training data and compute the MSE and $R2$ for the training and test data and plot both test and training data MSE and $R2$ as functions of the polynomial degree. Compare what you see with Figure 2.11 of Hastie et al. Comment your results. For which polynomial degree do you find an optimal MSE (smallest value)?\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"<!-- --- begin solution of exercise --- -->\n",
|
||||
"**Solution.**\n",
|
||||
"Here you simply need to change the degree of the polynomial in the above code to $n=15$.\n",
|
||||
"<!-- --- end solution of exercise --- -->\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"<!-- --- end exercise --- -->"
|
||||
]
|
||||
}
|
||||
|
||||
@@ -1285,13 +1285,16 @@ ways of dealing with outliers.
|
||||
The Huber cost function is defined as
|
||||
!bt
|
||||
\[
|
||||
H_{\delta}(a)=\left\{\begin{array}\frac{1}{2}a^{2}&{\text{for }}|a|\leq \delta ,\\ \delta (|a|-\frac {1}{2}\delta ),&{\text{otherwise.}\end{array}\right.
|
||||
H_{\delta}(\bm{a})=\left\{\begin{array}{cc}\frac{1}{2} \bm{a}^{2}& \text{for }|\bm{a}|\leq \delta\\ \delta (|\bm{a}|-\frac{1}{2}\delta ),&\text{otherwise}.\end{array}\right.
|
||||
\]
|
||||
!et
|
||||
Here $a=\bm{y} - \bm{\tilde{y}}$.
|
||||
We will discuss in more
|
||||
detail these and other functions in the various lectures. We conclude this part with another example. Instead of
|
||||
a linear $x$-dependence we study now a cubic polynomial and use the polynomial regression analysis tools of scikit-learn.
|
||||
Here $\bm{a}=\bm{y} - \bm{\tilde{y}}$.
|
||||
|
||||
|
||||
We will discuss in more detail these and other functions in the
|
||||
various lectures. We conclude this part with another example. Instead
|
||||
of a linear $x$-dependence we study now a cubic polynomial and use the
|
||||
polynomial regression analysis tools of scikit-learn.
|
||||
|
||||
!bc pycod
|
||||
import matplotlib.pyplot as plt
|
||||
@@ -2872,9 +2875,7 @@ Perform an ordinary least squares and compute the means squared error and the $R
|
||||
|
||||
!bsubex
|
||||
Add now a model which allows you to make polynomials up to degree $15$. Perform a standard OLS fitting of the training data and compute the MSE and $R2$ for the training and test data and plot both test and training data MSE and $R2$ as functions of the polynomial degree. Compare what you see with Figure 2.11 of Hastie et al. Comment your results. For which polynomial degree do you find an optimal MSE (smallest value)?
|
||||
!bsol
|
||||
Here you simply need to change the degree of the polynomial in the above code to $n=15$.
|
||||
!esol
|
||||
|
||||
!esubex
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user