tensorflow problems with chapter 12
This commit is contained in:
Morten Hjorth-Jensen
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -690,7 +690,7 @@ or a maximum amount of iterations.</p>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Converged at iteration 5
|
||||
Runtime: 0.4831998348236084 seconds
|
||||
Runtime: 0.4887218475341797 seconds
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -840,7 +840,7 @@ two improvements.</p>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Converged at iteration: 5
|
||||
Runtime: 0.42905116081237793 seconds
|
||||
Runtime: 0.432811975479126 seconds
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -866,7 +866,7 @@ i.e. the loop over all the samples. Nonetheless, let us do some profiling!</p>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Converged at iteration: 11
|
||||
Runtime: 0.8538670539855957 seconds
|
||||
Runtime: 0.8653810024261475 seconds
|
||||
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -977,7 +977,7 @@ fruits of our labor.</p>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Converged at iteration: 5
|
||||
Runtime: 0.004904031753540039 seconds
|
||||
Runtime: 0.0036406517028808594 seconds
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 6.3 KiB After Width: | Height: | Size: 6.2 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 9.2 KiB After Width: | Height: | Size: 9.6 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 14 KiB After Width: | Height: | Size: 14 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 9.5 KiB After Width: | Height: | Size: 10 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 25 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 15 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 11 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 13 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 20 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 31 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 30 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 31 KiB After Width: | Height: | Size: 20 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 193 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 71 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 28 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 11 KiB After Width: | Height: | Size: 11 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 16 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 28 KiB After Width: | Height: | Size: 31 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 21 KiB After Width: | Height: | Size: 20 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 31 KiB After Width: | Height: | Size: 29 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 12 KiB After Width: | Height: | Size: 12 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 10 KiB After Width: | Height: | Size: 11 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 11 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 5.0 KiB After Width: | Height: | Size: 5.0 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 9.5 KiB After Width: | Height: | Size: 9.4 KiB |
@@ -3411,16 +3411,13 @@
|
||||
"TrainError = np.zeros(maxdegree)\n",
|
||||
"polydegree = np.zeros(maxdegree)\n",
|
||||
"x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.2)\n",
|
||||
"scaler = StandardScaler()\n",
|
||||
"scaler.fit(x_train)\n",
|
||||
"x_train_scaled = scaler.transform(x_train)\n",
|
||||
"x_test_scaled = scaler.transform(x_test)\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"for degree in range(maxdegree):\n",
|
||||
" model = make_pipeline(PolynomialFeatures(degree=degree), LinearRegression(fit_intercept=False))\n",
|
||||
" clf = model.fit(x_train_scaled,y_train)\n",
|
||||
" y_fit = clf.predict(x_train_scaled)\n",
|
||||
" y_pred = clf.predict(x_test_scaled) \n",
|
||||
" clf = model.fit(x_train,y_train)\n",
|
||||
" y_fit = clf.predict(x_train)\n",
|
||||
" y_pred = clf.predict(x_test) \n",
|
||||
" polydegree[degree] = degree\n",
|
||||
" TestError[degree] = np.mean( np.mean((y_test - y_pred)**2) )\n",
|
||||
" TrainError[degree] = np.mean( np.mean((y_train - y_fit)**2) )\n",
|
||||
|
||||
@@ -1708,42 +1708,6 @@
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"We assume \n",
|
||||
"that every column of $\\boldsymbol{X}$ is centered, which we can do by subtracting the mean,"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": false,
|
||||
"editable": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"X = X - np.mean(X,axis=0)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"This means that we need to rewrite $X_{ij}$ as $\\tilde{X}_{ij}=X_{ij}-\\mu_j$, where"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"\\mu_j = \\frac{1}{n}\\sum_{i=0}^{n-1}X_{ij}.\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
@@ -1765,7 +1729,7 @@
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Assuming that the matrix elements $X_{i1}$ are centered, what we have is"
|
||||
"We obtain then"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -1773,7 +1737,7 @@
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"\\beta_0 = \\frac{1}{n}\\sum_{i=0}^{n-1}y_i - \\beta_1\\frac{1}{n}\\sum_{i=0}^{n-1} \\left(X_{i1}-\\mu_{1}\\right),\n",
|
||||
"\\beta_0 = \\frac{1}{n}\\sum_{i=0}^{n-1}y_i - \\beta_1\\frac{1}{n}\\sum_{i=0}^{n-1} X_{i1}.\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
@@ -1781,7 +1745,7 @@
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"where"
|
||||
"If we define"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -1821,7 +1785,7 @@
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"\\beta_0 = \\mu_y - \\beta_1\\frac{1}{n}\\sum_{i=0}^{n-1} (X_{i1}-\\mu_{1}),\n",
|
||||
"\\beta_0 = \\mu_y - \\beta_1\\mu_{1}.\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
@@ -1829,7 +1793,7 @@
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"and it is easy to see that the last sum equals zero! This means that we have"
|
||||
"In the general case withmore parameters than $\\beta_0$ and $\\beta_1$, we have"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -1837,7 +1801,7 @@
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"\\beta_0 = \\mu_y,\n",
|
||||
"\\beta_0 = \\frac{1}{n}\\sum_{i=0}^{n-1}y_i - \\frac{1}{n}\\sum_{i=0}^{n-1}\\sum_{j=1}^{p-1} X_{ij}\\beta_j.\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
@@ -1845,26 +1809,7 @@
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"if the columns of the design matrix are centered. It is straight forward to generalize this results to more values of $\\beta$.\n",
|
||||
"We have thus"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"\\beta_0 = \\frac{1}{n}\\sum_{i=0}^{n-1} y_i = \\overline{\\boldsymbol{y}},\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"the average value of $\\boldsymbol{y}$.\n",
|
||||
"\n",
|
||||
"Replacing $y_i$ with $y_i - \\beta_0 = y_i - \\overline{\\boldsymbol{y}}$ and centering also our design matrix results in a cost function (in vector-matrix disguise)"
|
||||
"Replacing $y_i$ with $y_i - y_i - \\overline{\\boldsymbol{y}}$ and centering also our design matrix results in a cost function (in vector-matrix disguise)"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -2238,8 +2183,6 @@
|
||||
" OwnRidgeBeta = np.linalg.pinv(X_train_scaled.T @ X_train_scaled+lmb*I) @ X_train_scaled.T @ (y_train_scaled)\n",
|
||||
" intercept_ = y_scaler - X_train_mean@OwnRidgeBeta #The intercept can be shifted so the model can predict on uncentered data\n",
|
||||
" #Add intercept to prediction\n",
|
||||
" ypredictOwnRidge = X_test @ OwnRidgeBeta + intercept_ \n",
|
||||
" #Add intercept to prediction\n",
|
||||
" ypredictOwnRidge = X_test_scaled @ OwnRidgeBeta + y_scaler \n",
|
||||
" RegRidge = linear_model.Ridge(lmb)\n",
|
||||
" RegRidge.fit(X_train,y_train)\n",
|
||||
@@ -2925,8 +2868,8 @@
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"6\n",
|
||||
"0\n",
|
||||
"5\n",
|
||||
"8\n",
|
||||
" \n",
|
||||
"<\n",
|
||||
"<\n",
|
||||
|
||||
@@ -124,7 +124,9 @@ For the reading assignments we use the following abbreviations:
|
||||
### Week 42 October 18-22
|
||||
- Lab Wednesday: Work on project 2
|
||||
- Lecture Thursday: Solving differential equations with neural networks and start Convolutional Neural Networks and classification problems
|
||||
- Video of Lecture at https://www.uio.no/studier/emner/matnat/fys/FYS-STK3155/h21/forelesningsvideoer/LectureOctober21.mp4?vrtx=view-as-webpage
|
||||
- Lecture Friday: Convolutional Neural Networks and classification problems
|
||||
- Video of Lecture at https://www.uio.no/studier/emner/matnat/fys/FYS-STK4155/h21/forelesningsvideoer/LectureOctober22.mp4?vrtx=view-as-webpage
|
||||
- Reading recommendations:
|
||||
- See lecture notes for week 42 at https://compphysics.github.io/MachineLearning/doc/web/course.html.
|
||||
- For neural networks we recommend Goodfellow et al chapters 6 and 7. For CNNs, see Goodfellow et al chapter 9. See also chapter 11 and 12 on practicalities and applications
|
||||
|
||||
@@ -799,13 +799,13 @@ example of the functionality of <strong>Scikit-Learn</strong>.</p>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>The intercept alpha:
|
||||
[1.9082084]
|
||||
[2.08534155]
|
||||
Coefficient beta :
|
||||
[[5.307514]]
|
||||
Mean squared error: 0.21
|
||||
Variance score: 0.91
|
||||
[[4.76326745]]
|
||||
Mean squared error: 0.26
|
||||
Variance score: 0.88
|
||||
Mean squared log error: 0.01
|
||||
Mean absolute error: 0.39
|
||||
Mean absolute error: 0.42
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapter1_13_1.png" src="_images/chapter1_13_1.png" />
|
||||
@@ -905,7 +905,7 @@ a linear <span class="math notranslate nohighlight">\(x\)</span>-dependence we s
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<img alt="_images/chapter1_27_0.png" src="_images/chapter1_27_0.png" />
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.004999999999999996
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.0050000000000000044
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1243,7 +1243,7 @@ A
|
||||
270 3344 160 110 270 Ds 7.253775 7.253775
|
||||
|
||||
[267 rows x 6 columns]
|
||||
0.009883615646716184
|
||||
0.009883615646716186
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1300,8 +1300,6 @@ functionality.</p>
|
||||
warnings.warn(
|
||||
/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
@@ -1318,8 +1316,6 @@ functionality.</p>
|
||||
warnings.warn(
|
||||
/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
@@ -1328,6 +1324,12 @@ functionality.</p>
|
||||
warnings.warn(
|
||||
/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
</pre></div>
|
||||
@@ -1356,8 +1358,18 @@ functionality.</p>
|
||||
warnings.warn(
|
||||
/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
@@ -1372,15 +1384,7 @@ functionality.</p>
|
||||
warnings.warn(
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/neural_network/_multilayer_perceptron.py:582: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.
|
||||
warnings.warn(
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapter1_61_10.png" src="_images/chapter1_61_10.png" />
|
||||
<img alt="_images/chapter1_61_12.png" src="_images/chapter1_61_12.png" />
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -2326,15 +2330,15 @@ the fit becomes better or worse.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[ 2.00396825 -0.03005885 5.03970472]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[2.00240127 0.09885613 4.86421839]
|
||||
Training R2
|
||||
0.9951332163662493
|
||||
0.994788006691541
|
||||
Training MSE
|
||||
0.010400010476420727
|
||||
0.011239358265567037
|
||||
Test R2
|
||||
0.9922167804917393
|
||||
0.9940257354319456
|
||||
Test MSE
|
||||
0.01533265828497683
|
||||
0.010708859852916969
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -2437,13 +2441,13 @@ but now splitting the data into a training set and a test set.</p>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Training R2
|
||||
0.9999871853340275
|
||||
0.9999940124132914
|
||||
Training MSE
|
||||
6.459553321949357
|
||||
2.1043064188572242
|
||||
Test R2
|
||||
0.999960835374545
|
||||
0.9999292683946882
|
||||
Test MSE
|
||||
6.844139253527833
|
||||
56.241106846210215
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -3189,16 +3193,13 @@ ourmodel (here in terms of the polynomial degree of the model).</p>
|
||||
<span class="n">TrainError</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">zeros</span><span class="p">(</span><span class="n">maxdegree</span><span class="p">)</span>
|
||||
<span class="n">polydegree</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">zeros</span><span class="p">(</span><span class="n">maxdegree</span><span class="p">)</span>
|
||||
<span class="n">x_train</span><span class="p">,</span> <span class="n">x_test</span><span class="p">,</span> <span class="n">y_train</span><span class="p">,</span> <span class="n">y_test</span> <span class="o">=</span> <span class="n">train_test_split</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">,</span> <span class="n">test_size</span><span class="o">=</span><span class="mf">0.2</span><span class="p">)</span>
|
||||
<span class="n">scaler</span> <span class="o">=</span> <span class="n">StandardScaler</span><span class="p">()</span>
|
||||
<span class="n">scaler</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">x_train</span><span class="p">)</span>
|
||||
<span class="n">x_train_scaled</span> <span class="o">=</span> <span class="n">scaler</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">x_train</span><span class="p">)</span>
|
||||
<span class="n">x_test_scaled</span> <span class="o">=</span> <span class="n">scaler</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">x_test</span><span class="p">)</span>
|
||||
|
||||
|
||||
<span class="k">for</span> <span class="n">degree</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">maxdegree</span><span class="p">):</span>
|
||||
<span class="n">model</span> <span class="o">=</span> <span class="n">make_pipeline</span><span class="p">(</span><span class="n">PolynomialFeatures</span><span class="p">(</span><span class="n">degree</span><span class="o">=</span><span class="n">degree</span><span class="p">),</span> <span class="n">LinearRegression</span><span class="p">(</span><span class="n">fit_intercept</span><span class="o">=</span><span class="kc">False</span><span class="p">))</span>
|
||||
<span class="n">clf</span> <span class="o">=</span> <span class="n">model</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">x_train_scaled</span><span class="p">,</span><span class="n">y_train</span><span class="p">)</span>
|
||||
<span class="n">y_fit</span> <span class="o">=</span> <span class="n">clf</span><span class="o">.</span><span class="n">predict</span><span class="p">(</span><span class="n">x_train_scaled</span><span class="p">)</span>
|
||||
<span class="n">y_pred</span> <span class="o">=</span> <span class="n">clf</span><span class="o">.</span><span class="n">predict</span><span class="p">(</span><span class="n">x_test_scaled</span><span class="p">)</span>
|
||||
<span class="n">clf</span> <span class="o">=</span> <span class="n">model</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">x_train</span><span class="p">,</span><span class="n">y_train</span><span class="p">)</span>
|
||||
<span class="n">y_fit</span> <span class="o">=</span> <span class="n">clf</span><span class="o">.</span><span class="n">predict</span><span class="p">(</span><span class="n">x_train</span><span class="p">)</span>
|
||||
<span class="n">y_pred</span> <span class="o">=</span> <span class="n">clf</span><span class="o">.</span><span class="n">predict</span><span class="p">(</span><span class="n">x_test</span><span class="p">)</span>
|
||||
<span class="n">polydegree</span><span class="p">[</span><span class="n">degree</span><span class="p">]</span> <span class="o">=</span> <span class="n">degree</span>
|
||||
<span class="n">TestError</span><span class="p">[</span><span class="n">degree</span><span class="p">]</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span> <span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">((</span><span class="n">y_test</span> <span class="o">-</span> <span class="n">y_pred</span><span class="p">)</span><span class="o">**</span><span class="mi">2</span><span class="p">)</span> <span class="p">)</span>
|
||||
<span class="n">TrainError</span><span class="p">[</span><span class="n">degree</span><span class="p">]</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span> <span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">((</span><span class="n">y_train</span> <span class="o">-</span> <span class="n">y_fit</span><span class="p">)</span><span class="o">**</span><span class="mi">2</span><span class="p">)</span> <span class="p">)</span>
|
||||
|
||||
@@ -1886,12 +1886,11 @@ Accuracy score on test set: 0.8666666666666667
|
||||
Learning rate = 1.0
|
||||
Lambda = 1e-05
|
||||
Accuracy score on test set: 0.09166666666666666
|
||||
|
||||
Learning rate = 1.0
|
||||
Lambda = 0.0001
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Accuracy score on test set: 0.11944444444444445
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 1.0
|
||||
Lambda = 0.0001
|
||||
Accuracy score on test set: 0.11944444444444445
|
||||
|
||||
Learning rate = 1.0
|
||||
Lambda = 0.001
|
||||
@@ -1927,12 +1926,11 @@ Accuracy score on test set: 0.08611111111111111
|
||||
Learning rate = 10.0
|
||||
Lambda = 0.001
|
||||
Accuracy score on test set: 0.08888888888888889
|
||||
|
||||
Learning rate = 10.0
|
||||
Lambda = 0.01
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Accuracy score on test set: 0.08888888888888889
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 10.0
|
||||
Lambda = 0.01
|
||||
Accuracy score on test set: 0.08888888888888889
|
||||
|
||||
Learning rate = 10.0
|
||||
Lambda = 0.1
|
||||
|
||||
@@ -1111,10 +1111,10 @@ covariance matrix through the <strong>np.linalg.eig()</strong> function.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.05805002932374468
|
||||
4.309519578637819
|
||||
[[ 1.23033954 3.6804336 ]
|
||||
[ 3.6804336 11.92527504]]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.0119839668275544
|
||||
3.9026821944532157
|
||||
[[ 1.21805483 3.6417002 ]
|
||||
[ 3.6417002 11.67950066]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1151,10 +1151,10 @@ a more brute force way. Here we scale the mean values for each column of the des
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.09198004868226574
|
||||
1.9493620821393187
|
||||
[[1. 0.68904673]
|
||||
[0.68904673 1. ]]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.08283349271892
|
||||
1.6319972672199556
|
||||
[[1. 0.65062105]
|
||||
[0.65062105 1. ]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1184,30 +1184,30 @@ this matrix we easily see that it is a positive definite matrix.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[-1.13352411e+00 -4.86145508e+00]
|
||||
[ 5.06793578e-01 2.24339370e+00]
|
||||
[ 4.08359441e-03 -1.44109702e+00]
|
||||
[-2.02972824e-01 -1.17393096e+00]
|
||||
[ 9.99703985e-01 5.00745587e+00]
|
||||
[-1.05430325e-01 3.31446832e-02]
|
||||
[ 4.31082669e-01 6.38783434e-01]
|
||||
[ 3.40259257e-02 5.84099240e-01]
|
||||
[-8.44862840e-01 -2.71546272e+00]
|
||||
[ 3.11100342e-01 1.68506886e+00]]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[ 0.30199745 0.72164987]
|
||||
[ 1.04858244 2.95539603]
|
||||
[ 0.46840512 1.52940309]
|
||||
[ 0.24117177 1.78848481]
|
||||
[-1.38690487 -4.17705119]
|
||||
[-0.65735283 -2.46141305]
|
||||
[ 0.94916469 2.81774574]
|
||||
[-0.52449732 -2.22140879]
|
||||
[-0.66999836 -2.66511337]
|
||||
[ 0.22943192 1.71230687]]
|
||||
0 1
|
||||
0 -1.133524 -4.861455
|
||||
1 0.506794 2.243394
|
||||
2 0.004084 -1.441097
|
||||
3 -0.202973 -1.173931
|
||||
4 0.999704 5.007456
|
||||
5 -0.105430 0.033145
|
||||
6 0.431083 0.638783
|
||||
7 0.034026 0.584099
|
||||
8 -0.844863 -2.715463
|
||||
9 0.311100 1.685069
|
||||
0 0.301997 0.721650
|
||||
1 1.048582 2.955396
|
||||
2 0.468405 1.529403
|
||||
3 0.241172 1.788485
|
||||
4 -1.386905 -4.177051
|
||||
5 -0.657353 -2.461413
|
||||
6 0.949165 2.817746
|
||||
7 -0.524497 -2.221409
|
||||
8 -0.669998 -2.665113
|
||||
9 0.229432 1.712307
|
||||
0 1
|
||||
0 1.000000 0.958527
|
||||
1 0.958527 1.000000
|
||||
0 1.000000 0.975127
|
||||
1 0.975127 1.000000
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1264,37 +1264,37 @@ this matrix we easily see that it is a positive definite matrix.</p>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 0 1 2 3 4 5 6 7 \
|
||||
0 0.0 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
|
||||
1 0.0 0.082212 0.080493 0.081616 0.077560 0.073611 0.073557 0.069528
|
||||
2 0.0 0.080493 0.080583 0.082413 0.079267 0.075998 0.075624 0.072071
|
||||
3 0.0 0.081616 0.082413 0.087107 0.084071 0.080840 0.082136 0.078414
|
||||
4 0.0 0.077560 0.079267 0.084071 0.081732 0.079085 0.080069 0.076853
|
||||
5 0.0 0.073611 0.075998 0.080840 0.079085 0.076946 0.077662 0.074900
|
||||
6 0.0 0.073557 0.075624 0.082136 0.080069 0.077662 0.079815 0.076731
|
||||
7 0.0 0.069528 0.072071 0.078414 0.076853 0.074900 0.076731 0.074079
|
||||
8 0.0 0.065794 0.068700 0.074858 0.073728 0.072170 0.073713 0.071447
|
||||
9 0.0 0.062352 0.065534 0.071499 0.070739 0.069526 0.070812 0.068888
|
||||
10 0.0 0.065582 0.068203 0.075503 0.074125 0.072352 0.074961 0.072447
|
||||
11 0.0 0.062022 0.064909 0.071917 0.070915 0.069494 0.071782 0.069625
|
||||
12 0.0 0.058765 0.061857 0.068581 0.067904 0.066793 0.068790 0.066951
|
||||
13 0.0 0.055788 0.059038 0.065492 0.065096 0.064256 0.065991 0.064434
|
||||
14 0.0 0.053069 0.056441 0.062636 0.062486 0.061885 0.063380 0.062076
|
||||
1 0.0 0.096033 0.089499 0.100065 0.095636 0.091356 0.093135 0.089366
|
||||
2 0.0 0.089499 0.084679 0.095430 0.091860 0.088348 0.090343 0.087109
|
||||
3 0.0 0.100065 0.095430 0.109996 0.106272 0.102568 0.105983 0.102429
|
||||
4 0.0 0.095636 0.091860 0.106272 0.103072 0.099840 0.103254 0.100074
|
||||
5 0.0 0.091356 0.088348 0.102568 0.099840 0.097041 0.100443 0.097606
|
||||
6 0.0 0.093135 0.090343 0.105983 0.103254 0.100443 0.104606 0.101680
|
||||
7 0.0 0.089366 0.087109 0.102429 0.100074 0.097606 0.101680 0.099049
|
||||
8 0.0 0.085816 0.084032 0.099028 0.097007 0.094850 0.098834 0.096471
|
||||
9 0.0 0.082468 0.081101 0.095771 0.094052 0.092175 0.096070 0.093951
|
||||
10 0.0 0.085269 0.083748 0.099374 0.097448 0.095375 0.099827 0.097488
|
||||
11 0.0 0.082003 0.080845 0.096087 0.094440 0.092628 0.096955 0.094852
|
||||
12 0.0 0.078938 0.078099 0.092968 0.091572 0.089996 0.094201 0.092313
|
||||
13 0.0 0.076058 0.075503 0.090008 0.088838 0.087475 0.091561 0.089869
|
||||
14 0.0 0.073350 0.073045 0.087198 0.086231 0.085061 0.089033 0.087519
|
||||
|
||||
8 9 10 11 12 13 14
|
||||
0 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
|
||||
1 0.065794 0.062352 0.065582 0.062022 0.058765 0.055788 0.053069
|
||||
2 0.068700 0.065534 0.068203 0.064909 0.061857 0.059038 0.056441
|
||||
3 0.074858 0.071499 0.075503 0.071917 0.068581 0.065492 0.062636
|
||||
4 0.073728 0.070739 0.074125 0.070915 0.067904 0.065096 0.062486
|
||||
5 0.072170 0.069526 0.072352 0.069494 0.066793 0.064256 0.061885
|
||||
6 0.073713 0.070812 0.074961 0.071782 0.068790 0.065991 0.063380
|
||||
7 0.071447 0.068888 0.072447 0.069625 0.066951 0.064434 0.062076
|
||||
8 0.069161 0.066912 0.069940 0.067442 0.065059 0.062803 0.060678
|
||||
9 0.066912 0.064945 0.067492 0.065289 0.063171 0.061154 0.059245
|
||||
10 0.069940 0.067492 0.071541 0.068800 0.066196 0.063738 0.061430
|
||||
11 0.067442 0.065289 0.068800 0.066372 0.064050 0.061846 0.059767
|
||||
12 0.065059 0.063171 0.066196 0.064050 0.061983 0.060011 0.058141
|
||||
13 0.062803 0.061154 0.063738 0.061846 0.060011 0.058249 0.056570
|
||||
14 0.060678 0.059245 0.061430 0.059767 0.058141 0.056570 0.055065
|
||||
1 0.085816 0.082468 0.085269 0.082003 0.078938 0.076058 0.073350
|
||||
2 0.084032 0.081101 0.083748 0.080845 0.078099 0.075503 0.073045
|
||||
3 0.099028 0.095771 0.099374 0.096087 0.092968 0.090008 0.087198
|
||||
4 0.097007 0.094052 0.097448 0.094440 0.091572 0.088838 0.086231
|
||||
5 0.094850 0.092175 0.095375 0.092628 0.089996 0.087475 0.085061
|
||||
6 0.098834 0.096070 0.099827 0.096955 0.094201 0.091561 0.089033
|
||||
7 0.096471 0.093951 0.097488 0.094852 0.092313 0.089869 0.087519
|
||||
8 0.094138 0.091843 0.095173 0.092754 0.090414 0.088152 0.085968
|
||||
9 0.091843 0.089754 0.092891 0.090673 0.088516 0.086423 0.084394
|
||||
10 0.095173 0.092891 0.096568 0.094135 0.091779 0.089500 0.087297
|
||||
11 0.092754 0.090673 0.094135 0.091901 0.089727 0.087616 0.085567
|
||||
12 0.090414 0.088516 0.091779 0.089727 0.087721 0.085765 0.083859
|
||||
13 0.088152 0.086423 0.089500 0.087616 0.085765 0.083951 0.082179
|
||||
14 0.085968 0.084394 0.087297 0.085567 0.083859 0.082179 0.080529
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -2150,15 +2150,13 @@ set of <span class="math notranslate nohighlight">\(\lambda\)</span> values.</p>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[ 2.03099776 -0.17917768 5.18029127]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Training MSE for OLS
|
||||
Training MSE for OLS
|
||||
0.009163470508352228
|
||||
Test MSE OLS
|
||||
0.008675369724976777
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapter2_249_2.png" src="_images/chapter2_249_2.png" />
|
||||
<img alt="_images/chapter2_249_1.png" src="_images/chapter2_249_1.png" />
|
||||
</div>
|
||||
</div>
|
||||
<p>Both these example send a clear message. The addition of a
|
||||
|
||||
@@ -645,10 +645,10 @@ number <span class="math notranslate nohighlight">\(i\)</span> is left out. Usin
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Runtime: 0.135707 sec
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Runtime: 0.135976 sec
|
||||
Jackknife Statistics :
|
||||
original bias std. error
|
||||
100.099 100.089 0.150795
|
||||
99.655 99.645 0.148675
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -867,7 +867,7 @@ theorem.</p>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Bootstrap Statistics :
|
||||
original bias std. error
|
||||
100.186 15.0063 100.185 0.148455
|
||||
100.098 15.248 100.098 0.153966
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1069,9 +1069,10 @@ Error: 0.32149601703519126
|
||||
Bias^2: 0.3123314713548606
|
||||
Var: 0.009164545680330616
|
||||
0.32149601703519126 >= 0.3123314713548606 + 0.009164545680330616 = 0.3214960170351912
|
||||
Polynomial degree:
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree: 1
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 1
|
||||
Error: 0.08426840630693411
|
||||
Bias^2: 0.07968918676726028
|
||||
Var: 0.004579219539673833
|
||||
@@ -1113,14 +1114,14 @@ Error: 0.017355848195591973
|
||||
Bias^2: 0.010331721306655588
|
||||
Var: 0.007024126888936384
|
||||
0.017355848195591973 >= 0.010331721306655588 + 0.007024126888936384 = 0.017355848195591973
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree: 9
|
||||
Polynomial degree: 9
|
||||
Error: 0.026605727637189085
|
||||
Bias^2: 0.010018312644140933
|
||||
Var: 0.016587414993048166
|
||||
0.026605727637189085 >= 0.010018312644140933 + 0.016587414993048166 = 0.0266057276371891
|
||||
Polynomial degree: 10
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree: 10
|
||||
Error: 0.021592704588043153
|
||||
Bias^2: 0.010516485576652981
|
||||
Var: 0.011076219011390184
|
||||
@@ -1135,16 +1136,14 @@ Error: 0.1154777721897675
|
||||
Bias^2: 0.01628578269590588
|
||||
Var: 0.09919198949386163
|
||||
0.1154777721897675 >= 0.01628578269590588 + 0.09919198949386163 = 0.11547777218976751
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree: 13
|
||||
Polynomial degree: 13
|
||||
Error: 0.22842468702166951
|
||||
Bias^2: 0.01975416527163567
|
||||
Var: 0.20867052175003387
|
||||
0.22842468702166951 >= 0.01975416527163567 + 0.20867052175003387 = 0.22842468702166954
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapter3_62_5.png" src="_images/chapter3_62_5.png" />
|
||||
<img alt="_images/chapter3_62_4.png" src="_images/chapter3_62_4.png" />
|
||||
</div>
|
||||
</div>
|
||||
<p>The bias-variance tradeoff summarizes the fundamental tension in
|
||||
@@ -1437,12 +1436,12 @@ Mean squared error on test data: 873.95463048
|
||||
Degree of polynomial: 23
|
||||
Mean squared error on training data: 0.00085890
|
||||
Mean squared error on test data: 5535.20053452
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 24
|
||||
Degree of polynomial: 24
|
||||
Mean squared error on training data: 0.00084714
|
||||
Mean squared error on test data: 1289.22422186
|
||||
Degree of polynomial: 25
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 25
|
||||
Mean squared error on training data: 0.00079022
|
||||
Mean squared error on test data: 136582.88824397
|
||||
Degree of polynomial: 26
|
||||
@@ -1801,32 +1800,18 @@ When we take the derivative with respect to <span class="math notranslate nohigh
|
||||
\[
|
||||
\sum_{i=0}^{n-1} \beta_0 = \sum_{i=0}^{n-1}y_i - \sum_{i=0}^{n-1} \sum_{j=1}^{p-1} X_{ij} \beta_j.
|
||||
\]</div>
|
||||
<p>We assume
|
||||
that every column of <span class="math notranslate nohighlight">\(\boldsymbol{X}\)</span> is centered, which we can do by subtracting the mean,</p>
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="n">X</span> <span class="o">=</span> <span class="n">X</span> <span class="o">-</span> <span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">X</span><span class="p">,</span><span class="n">axis</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<p>This means that we need to rewrite <span class="math notranslate nohighlight">\(X_{ij}\)</span> as <span class="math notranslate nohighlight">\(\tilde{X}_{ij}=X_{ij}-\mu_j\)</span>, where</p>
|
||||
<div class="math notranslate nohighlight">
|
||||
\[
|
||||
\mu_j = \frac{1}{n}\sum_{i=0}^{n-1}X_{ij}.
|
||||
\]</div>
|
||||
<p>Let us special first to the case where we have only two parameters <span class="math notranslate nohighlight">\(\beta_0\)</span> and <span class="math notranslate nohighlight">\(\beta_1\)</span>.
|
||||
Our result for <span class="math notranslate nohighlight">\(\beta_0\)</span> simplifies then to</p>
|
||||
<div class="math notranslate nohighlight">
|
||||
\[
|
||||
n\beta_0 = \sum_{i=0}^{n-1}y_i - \sum_{i=0}^{n-1} X_{i1} \beta_1.
|
||||
\]</div>
|
||||
<p>Assuming that the matrix elements <span class="math notranslate nohighlight">\(X_{i1}\)</span> are centered, what we have is</p>
|
||||
<p>We obtain then</p>
|
||||
<div class="math notranslate nohighlight">
|
||||
\[
|
||||
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} \left(X_{i1}-\mu_{1}\right),
|
||||
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} X_{i1}.
|
||||
\]</div>
|
||||
<p>where</p>
|
||||
<p>If we define</p>
|
||||
<div class="math notranslate nohighlight">
|
||||
\[
|
||||
\mu_1=\frac{1}{n}\sum_{i=0}^{n-1} (X_{i1},
|
||||
@@ -1839,21 +1824,14 @@ n\beta_0 = \sum_{i=0}^{n-1}y_i - \sum_{i=0}^{n-1} X_{i1} \beta_1.
|
||||
<p>we have</p>
|
||||
<div class="math notranslate nohighlight">
|
||||
\[
|
||||
\beta_0 = \mu_y - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} (X_{i1}-\mu_{1}),
|
||||
\beta_0 = \mu_y - \beta_1\mu_{1}.
|
||||
\]</div>
|
||||
<p>and it is easy to see that the last sum equals zero! This means that we have</p>
|
||||
<p>In the general case withmore parameters than <span class="math notranslate nohighlight">\(\beta_0\)</span> and <span class="math notranslate nohighlight">\(\beta_1\)</span>, we have</p>
|
||||
<div class="math notranslate nohighlight">
|
||||
\[
|
||||
\beta_0 = \mu_y,
|
||||
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \frac{1}{n}\sum_{i=0}^{n-1}\sum_{j=1}^{p-1} X_{ij}\beta_j.
|
||||
\]</div>
|
||||
<p>if the columns of the design matrix are centered. It is straight forward to generalize this results to more values of <span class="math notranslate nohighlight">\(\beta\)</span>.
|
||||
We have thus</p>
|
||||
<div class="math notranslate nohighlight">
|
||||
\[
|
||||
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1} y_i = \overline{\boldsymbol{y}},
|
||||
\]</div>
|
||||
<p>the average value of <span class="math notranslate nohighlight">\(\boldsymbol{y}\)</span>.</p>
|
||||
<p>Replacing <span class="math notranslate nohighlight">\(y_i\)</span> with <span class="math notranslate nohighlight">\(y_i - \beta_0 = y_i - \overline{\boldsymbol{y}}\)</span> and centering also our design matrix results in a cost function (in vector-matrix disguise)</p>
|
||||
<p>Replacing <span class="math notranslate nohighlight">\(y_i\)</span> with <span class="math notranslate nohighlight">\(y_i - y_i - \overline{\boldsymbol{y}}\)</span> and centering also our design matrix results in a cost function (in vector-matrix disguise)</p>
|
||||
<div class="math notranslate nohighlight">
|
||||
\[
|
||||
C(\boldsymbol{\beta}) = (\boldsymbol{\tilde{y}} - \tilde{X}\boldsymbol{\beta})^T(\boldsymbol{\tilde{y}} - \tilde{X}\boldsymbol{\beta}).
|
||||
@@ -1982,7 +1960,7 @@ MSE with Sklearn intercept
|
||||
0.004113634617443135
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapter3_109_1.png" src="_images/chapter3_109_1.png" />
|
||||
<img alt="_images/chapter3_103_1.png" src="_images/chapter3_103_1.png" />
|
||||
</div>
|
||||
</div>
|
||||
<p>The intercept is the value of our output/target variable
|
||||
@@ -2177,7 +2155,7 @@ MSE values for Scikit-Learn Ridge implementation
|
||||
0.26409315307910036
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapter3_117_1.png" src="_images/chapter3_117_1.png" />
|
||||
<img alt="_images/chapter3_111_1.png" src="_images/chapter3_111_1.png" />
|
||||
</div>
|
||||
</div>
|
||||
<p>The results here agree when we force <strong>Scikit-Learn</strong>’s Ridge function to include the first column in our design matrix.
|
||||
@@ -2236,8 +2214,6 @@ Let us see how we can change this code by zero centering.</p>
|
||||
<span class="n">OwnRidgeBeta</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">linalg</span><span class="o">.</span><span class="n">pinv</span><span class="p">(</span><span class="n">X_train_scaled</span><span class="o">.</span><span class="n">T</span> <span class="o">@</span> <span class="n">X_train_scaled</span><span class="o">+</span><span class="n">lmb</span><span class="o">*</span><span class="n">I</span><span class="p">)</span> <span class="o">@</span> <span class="n">X_train_scaled</span><span class="o">.</span><span class="n">T</span> <span class="o">@</span> <span class="p">(</span><span class="n">y_train_scaled</span><span class="p">)</span>
|
||||
<span class="n">intercept_</span> <span class="o">=</span> <span class="n">y_scaler</span> <span class="o">-</span> <span class="n">X_train_mean</span><span class="nd">@OwnRidgeBeta</span> <span class="c1">#The intercept can be shifted so the model can predict on uncentered data</span>
|
||||
<span class="c1">#Add intercept to prediction</span>
|
||||
<span class="n">ypredictOwnRidge</span> <span class="o">=</span> <span class="n">X_test</span> <span class="o">@</span> <span class="n">OwnRidgeBeta</span> <span class="o">+</span> <span class="n">intercept_</span>
|
||||
<span class="c1">#Add intercept to prediction</span>
|
||||
<span class="n">ypredictOwnRidge</span> <span class="o">=</span> <span class="n">X_test_scaled</span> <span class="o">@</span> <span class="n">OwnRidgeBeta</span> <span class="o">+</span> <span class="n">y_scaler</span>
|
||||
<span class="n">RegRidge</span> <span class="o">=</span> <span class="n">linear_model</span><span class="o">.</span><span class="n">Ridge</span><span class="p">(</span><span class="n">lmb</span><span class="p">)</span>
|
||||
<span class="n">RegRidge</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">X_train</span><span class="p">,</span><span class="n">y_train</span><span class="p">)</span>
|
||||
@@ -2382,7 +2358,7 @@ MSE values for Scikit-Learn Ridge implementation
|
||||
0.002381316302584886
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapter3_119_1.png" src="_images/chapter3_119_1.png" />
|
||||
<img alt="_images/chapter3_113_1.png" src="_images/chapter3_113_1.png" />
|
||||
</div>
|
||||
</div>
|
||||
<p>We see here, when compared to the code which includes explicitely the
|
||||
@@ -2596,11 +2572,11 @@ linear system as an equation would reduce this down to
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span><ipython-input-21-6f7a6bd7d79f>:7: UserWarning: FixedFormatter should only be used together with FixedLocator
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span><ipython-input-20-6f7a6bd7d79f>:7: UserWarning: FixedFormatter should only be used together with FixedLocator
|
||||
cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapter3_150_1.png" src="_images/chapter3_150_1.png" />
|
||||
<img alt="_images/chapter3_144_1.png" src="_images/chapter3_144_1.png" />
|
||||
</div>
|
||||
</div>
|
||||
<p>It is interesting to note that OLS
|
||||
@@ -2740,11 +2716,11 @@ with the form utilized in linear regression, viz.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span><ipython-input-26-5dd54edf2138>:7: UserWarning: FixedFormatter should only be used together with FixedLocator
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span><ipython-input-25-5dd54edf2138>:7: UserWarning: FixedFormatter should only be used together with FixedLocator
|
||||
cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapter3_168_1.png" src="_images/chapter3_168_1.png" />
|
||||
<img alt="_images/chapter3_162_1.png" src="_images/chapter3_162_1.png" />
|
||||
</div>
|
||||
</div>
|
||||
<p>The results agree perfectly with our previous discussion where we used our own code.</p>
|
||||
@@ -2753,8 +2729,8 @@ regression. In ridge regression we include a <strong>regularizer</strong>. This
|
||||
involves a new cost function which leads to a new estimate for the
|
||||
weights <span class="math notranslate nohighlight">\(\boldsymbol{\beta}\)</span>. This results in a penalized regression problem. The
|
||||
cost function is given by</p>
|
||||
<p>6
|
||||
0</p>
|
||||
<p>5
|
||||
8</p>
|
||||
<p><
|
||||
<
|
||||
<
|
||||
@@ -2788,11 +2764,11 @@ K</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span><ipython-input-27-fe5b9d300cc0>:10: UserWarning: FixedFormatter should only be used together with FixedLocator
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span><ipython-input-26-fe5b9d300cc0>:10: UserWarning: FixedFormatter should only be used together with FixedLocator
|
||||
cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapter3_171_1.png" src="_images/chapter3_171_1.png" />
|
||||
<img alt="_images/chapter3_165_1.png" src="_images/chapter3_165_1.png" />
|
||||
</div>
|
||||
</div>
|
||||
<p>In the <strong>Least Absolute Shrinkage and Selection Operator</strong> (LASSO)-method we get a third cost function.</p>
|
||||
@@ -2823,11 +2799,11 @@ K</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span><ipython-input-28-25845e8df859>:9: UserWarning: FixedFormatter should only be used together with FixedLocator
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span><ipython-input-27-25845e8df859>:9: UserWarning: FixedFormatter should only be used together with FixedLocator
|
||||
cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapter3_175_1.png" src="_images/chapter3_175_1.png" />
|
||||
<img alt="_images/chapter3_169_1.png" src="_images/chapter3_169_1.png" />
|
||||
</div>
|
||||
</div>
|
||||
<p>It is quite striking how LASSO breaks the symmetry of the coupling
|
||||
@@ -2885,40 +2861,40 @@ constant as opposed to ridge and OLS. We get a sparse solution with
|
||||
10%|█ | 1/10 [00:00<00:04, 2.02it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 20%|██ | 2/10 [00:00<00:03, 2.38it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 20%|██ | 2/10 [00:00<00:03, 2.25it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 30%|███ | 3/10 [00:00<00:02, 2.99it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 30%|███ | 3/10 [00:00<00:02, 2.82it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 40%|████ | 4/10 [00:00<00:01, 3.72it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 40%|████ | 4/10 [00:01<00:01, 3.33it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 50%|█████ | 5/10 [00:01<00:01, 4.40it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 50%|█████ | 5/10 [00:01<00:01, 3.94it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 60%|██████ | 6/10 [00:01<00:00, 5.16it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 60%|██████ | 6/10 [00:01<00:00, 4.61it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 70%|███████ | 7/10 [00:01<00:00, 5.95it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 70%|███████ | 7/10 [00:01<00:00, 5.17it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 80%|████████ | 8/10 [00:01<00:00, 6.55it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 80%|████████ | 8/10 [00:01<00:00, 5.73it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 90%|█████████ | 9/10 [00:01<00:00, 6.99it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 90%|█████████ | 9/10 [00:01<00:00, 6.46it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>100%|██████████| 10/10 [00:01<00:00, 7.35it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>100%|██████████| 10/10 [00:01<00:00, 6.87it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>100%|██████████| 10/10 [00:01<00:00, 5.87it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>100%|██████████| 10/10 [00:01<00:00, 5.22it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapter3_177_13.png" src="_images/chapter3_177_13.png" />
|
||||
<img alt="_images/chapter3_171_13.png" src="_images/chapter3_171_13.png" />
|
||||
</div>
|
||||
</div>
|
||||
<p>We see that LASSO reaches a good solution for low
|
||||
@@ -2967,7 +2943,7 @@ testing set that is close to the accuracy of the training set.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<img alt="_images/chapter3_179_0.png" src="_images/chapter3_179_0.png" />
|
||||
<img alt="_images/chapter3_173_0.png" src="_images/chapter3_173_0.png" />
|
||||
</div>
|
||||
</div>
|
||||
<p>From the above figure we can see that LASSO with <span class="math notranslate nohighlight">\(\lambda = 10^{-2}\)</span>
|
||||
@@ -3059,7 +3035,7 @@ which polynomial fits the data best.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<img alt="_images/chapter3_183_0.png" src="_images/chapter3_183_0.png" />
|
||||
<img alt="_images/chapter3_177_0.png" src="_images/chapter3_177_0.png" />
|
||||
</div>
|
||||
</div>
|
||||
<div class="section" id="exercise-ordinary-least-square-ols-on-the-franke-function">
|
||||
@@ -3216,7 +3192,7 @@ Python program using</p>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output traceback highlight-ipythontb notranslate"><div class="highlight"><pre><span></span><span class="gt">---------------------------------------------------------------------------</span>
|
||||
<span class="ne">NameError</span><span class="g g-Whitespace"> </span>Traceback (most recent call last)
|
||||
<span class="o"><</span><span class="n">ipython</span><span class="o">-</span><span class="nb">input</span><span class="o">-</span><span class="mi">32</span><span class="o">-</span><span class="n">d985fb40c43d</span><span class="o">></span> <span class="ow">in</span> <span class="o"><</span><span class="n">module</span><span class="o">></span>
|
||||
<span class="o"><</span><span class="n">ipython</span><span class="o">-</span><span class="nb">input</span><span class="o">-</span><span class="mi">31</span><span class="o">-</span><span class="n">d985fb40c43d</span><span class="o">></span> <span class="ow">in</span> <span class="o"><</span><span class="n">module</span><span class="o">></span>
|
||||
<span class="ne">----> </span><span class="mi">1</span> <span class="n">scipy</span><span class="o">.</span><span class="n">misc</span><span class="o">.</span><span class="n">imread</span>
|
||||
|
||||
<span class="ne">NameError</span>: name 'scipy' is not defined
|
||||
|
||||
@@ -552,9 +552,9 @@ predicting the target features of query instances is as follows:</p>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>2nd degree coefficients:
|
||||
zero power: 1.055455110765643
|
||||
first power: 0.022116476923538613
|
||||
second power: -0.00019348106804731915
|
||||
zero power: -2.575591916051299
|
||||
first power: -0.07310737041838755
|
||||
second power: 0.0002644543695187602
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapter6_1_1.png" src="_images/chapter6_1_1.png" />
|
||||
|
||||
@@ -532,10 +532,10 @@ covariance matrix through the <strong>np.linalg.eig()</strong> function.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>-0.1014841451296029
|
||||
3.744674093727182
|
||||
[[ 1.04621283 3.2509618 ]
|
||||
[ 3.2509618 11.28244928]]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>-0.13740265686642364
|
||||
3.647867896223553
|
||||
[[0.88732076 2.58645106]
|
||||
[2.58645106 8.40379027]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -575,10 +575,10 @@ a more brute force way. Here we scale the mean values for each column of the des
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.08899392007617646
|
||||
1.960919045835183
|
||||
[[1. 0.64606848]
|
||||
[0.64606848 1. ]]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.09282061240353208
|
||||
2.0679953259809554
|
||||
[[1. 0.65276752]
|
||||
[0.65276752 1. ]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -607,30 +607,30 @@ this matrix we easily see that it is a positive definite matrix.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[-0.26338419 -0.43128913]
|
||||
[-0.40052348 -0.40977892]
|
||||
[-1.21495309 -3.03292103]
|
||||
[-0.68288031 -2.86961626]
|
||||
[ 1.847187 7.221629 ]
|
||||
[-0.01324363 0.06365882]
|
||||
[-0.08321393 -1.11171059]
|
||||
[ 0.6497097 1.58382502]
|
||||
[-0.3196681 -1.46807752]
|
||||
[ 0.48097003 0.45428062]]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[ 0.27416797 2.69235507]
|
||||
[-0.30895532 -1.36796587]
|
||||
[ 1.03285324 2.17106672]
|
||||
[-0.35304998 -2.85655181]
|
||||
[-0.79084976 -1.67951873]
|
||||
[ 0.19247017 0.868215 ]
|
||||
[-0.45745139 -2.25170803]
|
||||
[-0.63165748 -1.85258411]
|
||||
[-0.19636844 0.14649905]
|
||||
[ 1.238841 4.13019269]]
|
||||
0 1
|
||||
0 -0.263384 -0.431289
|
||||
1 -0.400523 -0.409779
|
||||
2 -1.214953 -3.032921
|
||||
3 -0.682880 -2.869616
|
||||
4 1.847187 7.221629
|
||||
5 -0.013244 0.063659
|
||||
6 -0.083214 -1.111711
|
||||
7 0.649710 1.583825
|
||||
8 -0.319668 -1.468078
|
||||
9 0.480970 0.454281
|
||||
0 0.274168 2.692355
|
||||
1 -0.308955 -1.367966
|
||||
2 1.032853 2.171067
|
||||
3 -0.353050 -2.856552
|
||||
4 -0.790850 -1.679519
|
||||
5 0.192470 0.868215
|
||||
6 -0.457451 -2.251708
|
||||
7 -0.631657 -1.852584
|
||||
8 -0.196368 0.146499
|
||||
9 1.238841 4.130193
|
||||
0 1
|
||||
0 1.000000 0.959043
|
||||
1 0.959043 1.000000
|
||||
0 1.000000 0.899606
|
||||
1 0.899606 1.000000
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -687,37 +687,37 @@ this matrix we easily see that it is a positive definite matrix.</p>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 0 1 2 3 4 5 6 7 \
|
||||
0 0.0 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
|
||||
1 0.0 0.077527 0.078731 0.077076 0.078129 0.079210 0.068328 0.069362
|
||||
2 0.0 0.078731 0.081031 0.079460 0.081106 0.082747 0.071284 0.072687
|
||||
3 0.0 0.077076 0.079460 0.081529 0.083368 0.085189 0.075205 0.076784
|
||||
4 0.0 0.078129 0.081106 0.083368 0.085569 0.087738 0.077404 0.079228
|
||||
5 0.0 0.079210 0.082747 0.085189 0.087738 0.090239 0.079562 0.081623
|
||||
6 0.0 0.068328 0.071284 0.075205 0.077404 0.079562 0.071309 0.073110
|
||||
7 0.0 0.069362 0.072687 0.076784 0.079228 0.081623 0.073110 0.075086
|
||||
8 0.0 0.070455 0.074134 0.078402 0.081085 0.083711 0.074934 0.077084
|
||||
9 0.0 0.071609 0.075631 0.080066 0.082982 0.085835 0.076789 0.079111
|
||||
10 0.0 0.059604 0.062733 0.067433 0.069731 0.071979 0.065235 0.067080
|
||||
11 0.0 0.060603 0.063986 0.068834 0.071311 0.073734 0.066778 0.068758
|
||||
12 0.0 0.061660 0.065292 0.070286 0.072940 0.075537 0.068363 0.070479
|
||||
13 0.0 0.062777 0.066653 0.071794 0.074624 0.077395 0.069997 0.072248
|
||||
14 0.0 0.063954 0.068071 0.073361 0.076367 0.079312 0.071682 0.074070
|
||||
1 0.0 0.090368 0.089828 0.084745 0.089694 0.094385 0.072722 0.077607
|
||||
2 0.0 0.089828 0.091016 0.082203 0.087699 0.093293 0.069684 0.074647
|
||||
3 0.0 0.084745 0.082203 0.084433 0.088351 0.091653 0.075222 0.079774
|
||||
4 0.0 0.089694 0.087699 0.088351 0.092763 0.096661 0.078226 0.083104
|
||||
5 0.0 0.094385 0.093293 0.091653 0.096661 0.101332 0.080548 0.085760
|
||||
6 0.0 0.072722 0.069684 0.075222 0.078226 0.080548 0.068780 0.072668
|
||||
7 0.0 0.077607 0.074647 0.079774 0.083104 0.085760 0.072668 0.076854
|
||||
8 0.0 0.082726 0.079963 0.084408 0.088121 0.091191 0.076564 0.081069
|
||||
9 0.0 0.088020 0.085632 0.089011 0.093180 0.096774 0.080344 0.085192
|
||||
10 0.0 0.062054 0.059107 0.065855 0.068244 0.069993 0.061366 0.064683
|
||||
11 0.0 0.066283 0.063260 0.070074 0.072690 0.074644 0.065135 0.068701
|
||||
12 0.0 0.070793 0.067729 0.074516 0.077390 0.079587 0.069075 0.072910
|
||||
13 0.0 0.075587 0.072539 0.079162 0.082333 0.084822 0.073160 0.077286
|
||||
14 0.0 0.080656 0.077711 0.083974 0.087490 0.090338 0.077343 0.081784
|
||||
|
||||
8 9 10 11 12 13 14
|
||||
0 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
|
||||
1 0.070455 0.071609 0.059604 0.060603 0.061660 0.062777 0.063954
|
||||
2 0.074134 0.075631 0.062733 0.063986 0.065292 0.066653 0.068071
|
||||
3 0.078402 0.080066 0.067433 0.068834 0.070286 0.071794 0.073361
|
||||
4 0.081085 0.082982 0.069731 0.071311 0.072940 0.074624 0.076367
|
||||
5 0.083711 0.085835 0.071979 0.073734 0.075537 0.077395 0.079312
|
||||
6 0.074934 0.076789 0.065235 0.066778 0.068363 0.069997 0.071682
|
||||
7 0.077084 0.079111 0.067080 0.068758 0.070479 0.072248 0.074070
|
||||
8 0.079253 0.081450 0.068941 0.070753 0.072608 0.074513 0.076473
|
||||
9 0.081450 0.083817 0.070824 0.072770 0.074761 0.076803 0.078901
|
||||
10 0.068941 0.070824 0.060591 0.062149 0.063743 0.065378 0.067060
|
||||
11 0.070753 0.072770 0.062149 0.063814 0.065517 0.067262 0.069055
|
||||
12 0.072608 0.074761 0.063743 0.065517 0.067329 0.069185 0.071092
|
||||
13 0.074513 0.076803 0.065378 0.067262 0.069185 0.071155 0.073178
|
||||
14 0.076473 0.078901 0.067060 0.069055 0.071092 0.073178 0.075318
|
||||
1 0.082726 0.088020 0.062054 0.066283 0.070793 0.075587 0.080656
|
||||
2 0.079963 0.085632 0.059107 0.063260 0.067729 0.072539 0.077711
|
||||
3 0.084408 0.089011 0.065855 0.070074 0.074516 0.079162 0.083974
|
||||
4 0.088121 0.093180 0.068244 0.072690 0.077390 0.082333 0.087490
|
||||
5 0.091191 0.096774 0.069993 0.074644 0.079587 0.084822 0.090338
|
||||
6 0.076564 0.080344 0.061366 0.065135 0.069075 0.073160 0.077343
|
||||
7 0.081069 0.085192 0.064683 0.068701 0.072910 0.077286 0.081784
|
||||
8 0.085636 0.090149 0.067977 0.072252 0.076743 0.081428 0.086266
|
||||
9 0.090149 0.095107 0.071137 0.075674 0.080455 0.085465 0.090671
|
||||
10 0.067977 0.071137 0.055542 0.058856 0.062305 0.065864 0.069488
|
||||
11 0.072252 0.075674 0.058856 0.062396 0.066086 0.069899 0.073789
|
||||
12 0.076743 0.080455 0.062305 0.066086 0.070032 0.074118 0.078298
|
||||
13 0.081428 0.085465 0.065864 0.069899 0.074118 0.078497 0.082989
|
||||
14 0.086266 0.090671 0.069488 0.073789 0.078298 0.082989 0.087822
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -906,10 +906,10 @@ We can write our own code or simply use either the functionaly of <strong>numpy<
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 0 1
|
||||
0 3.982025 2.012742
|
||||
1 2.012742 2.042269
|
||||
[[3.98202489 2.01274179]
|
||||
[2.01274179 2.04226949]]
|
||||
0 3.900266 1.942180
|
||||
1 1.942180 1.965724
|
||||
[[3.90026646 1.9421795 ]
|
||||
[1.9421795 1.96572363]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -936,8 +936,8 @@ Our own code here is not very elegant and asks for obvious improvements. It is t
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Centered covariance using own code
|
||||
[[3.98202489 2.01274179]
|
||||
[2.01274179 2.04226949]]
|
||||
[[3.90026646 1.9421795 ]
|
||||
[1.9421795 1.96572363]]
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapter8_65_1.png" src="_images/chapter8_65_1.png" />
|
||||
@@ -997,16 +997,16 @@ questions.</p>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Eigenvalues of Covariance matrix
|
||||
5.246379112732898
|
||||
0.7779152676639822
|
||||
5.102712819944882
|
||||
0.763277270613554
|
||||
First eigenvector
|
||||
[0.84678774 0.53193095]
|
||||
[0.85023678 0.52640043]
|
||||
Second eigenvector
|
||||
[-0.53193095 0.84678774]
|
||||
[-0.52640043 0.85023678]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Eigenvector of largest eigenvalue
|
||||
[0.84678774 0.53193095]
|
||||
[-0.85023678 -0.52640043]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@@ -752,7 +752,7 @@ which equals</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span><mpl_toolkits.mplot3d.art3d.Poly3DCollection at 0x7fc6b1bd1790>
|
||||
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span><mpl_toolkits.mplot3d.art3d.Poly3DCollection at 0x7f9f087db790>
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapteroptimization_56_1.png" src="_images/chapteroptimization_56_1.png" />
|
||||
@@ -810,7 +810,7 @@ which equals</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[<matplotlib.lines.Line2D at 0x7fc6a0739340>]
|
||||
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[<matplotlib.lines.Line2D at 0x7f9f1a921340>]
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapteroptimization_64_1.png" src="_images/chapteroptimization_64_1.png" />
|
||||
@@ -1067,11 +1067,11 @@ when <span class="math notranslate nohighlight">\(||\nabla_\beta C(\beta_k) || \
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[0.30875823 4.41879302]
|
||||
[[3.96285336]
|
||||
[3.28329188]]
|
||||
[[3.96285336]
|
||||
[3.28329188]]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[0.30862336 4.94226154]
|
||||
[[4.09553421]
|
||||
[2.99517651]]
|
||||
[[4.09553421]
|
||||
[2.99517651]]
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapteroptimization_118_1.png" src="_images/chapteroptimization_118_1.png" />
|
||||
@@ -1100,9 +1100,9 @@ when <span class="math notranslate nohighlight">\(||\nabla_\beta C(\beta_k) || \
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[3.75582035]
|
||||
[3.15167261]]
|
||||
[3.7821634] [3.18303503]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[4.1224808 ]
|
||||
[2.97290346]]
|
||||
[4.09600813] [2.93854356]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1173,10 +1173,10 @@ C_{\text{ridge}}(\beta) = \frac{1}{n}||X\beta -\mathbf{y}||^2 + \lambda ||\beta|
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[4.05636001]
|
||||
[2.75240368]]
|
||||
[[3.99394477]
|
||||
[2.80463702]]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[3.95338481]
|
||||
[2.95611545]]
|
||||
[[3.99545147]
|
||||
[2.91812296]]
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapteroptimization_127_1.png" src="_images/chapteroptimization_127_1.png" />
|
||||
@@ -1401,21 +1401,19 @@ function.</p>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
|
||||
[[3.88168518]
|
||||
[3.04348504]]
|
||||
[[3.87218485]
|
||||
[3.01403105]]
|
||||
sgdreg from scikit
|
||||
[3.92115534] [3.13964362]
|
||||
[3.9030809] [3.07154525]
|
||||
theta from own gd
|
||||
[[3.88168518]
|
||||
[3.04348504]]
|
||||
[[3.87218485]
|
||||
[3.01403105]]
|
||||
theta from own sdg
|
||||
[[3.86110774]
|
||||
[3.00070278]]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own sdg
|
||||
[[3.92555588]
|
||||
[3.0163616 ]]
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapteroptimization_141_2.png" src="_images/chapteroptimization_141_2.png" />
|
||||
<img alt="_images/chapteroptimization_141_1.png" src="_images/chapteroptimization_141_1.png" />
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@@ -455,8 +455,8 @@ matrices and vectors.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[-0.98226332 0.21126669 0.23494196 1.04532409 -0.994015 -0.37831577
|
||||
-0.22534931 0.39445208 -1.97921062 1.042552 ]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[-0.62370636 0.34673123 0.80408254 2.08638654 1.16894795 1.8464107
|
||||
0.13620973 -1.33881021 -0.05404886 -0.99995891]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@@ -585,8 +585,16 @@
|
||||
<h3>Week 42 October 18-22<a class="headerlink" href="#week-42-october-18-22" title="Permalink to this headline">¶</a></h3>
|
||||
<ul class="simple">
|
||||
<li><p>Lab Wednesday: Work on project 2</p></li>
|
||||
<li><p>Lecture Thursday: Solving differential equations with neural networks and start Convolutional Neural Networks and classification problems</p></li>
|
||||
<li><p>Lecture Friday: Convolutional Neural Networks and classification problems</p></li>
|
||||
<li><p>Lecture Thursday: Solving differential equations with neural networks and start Convolutional Neural Networks and classification problems</p>
|
||||
<ul>
|
||||
<li><p>Video of Lecture at <a class="reference external" href="https://www.uio.no/studier/emner/matnat/fys/FYS-STK3155/h21/forelesningsvideoer/LectureOctober21.mp4?vrtx=view-as-webpage">https://www.uio.no/studier/emner/matnat/fys/FYS-STK3155/h21/forelesningsvideoer/LectureOctober21.mp4?vrtx=view-as-webpage</a></p></li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><p>Lecture Friday: Convolutional Neural Networks and classification problems</p>
|
||||
<ul>
|
||||
<li><p>Video of Lecture at <a class="reference external" href="https://www.uio.no/studier/emner/matnat/fys/FYS-STK4155/h21/forelesningsvideoer/LectureOctober22.mp4?vrtx=view-as-webpage">https://www.uio.no/studier/emner/matnat/fys/FYS-STK4155/h21/forelesningsvideoer/LectureOctober22.mp4?vrtx=view-as-webpage</a></p></li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><p>Reading recommendations:</p>
|
||||
<ul>
|
||||
<li><p>See lecture notes for week 42 at <a class="reference external" href="https://compphysics.github.io/MachineLearning/doc/web/course.html">https://compphysics.github.io/MachineLearning/doc/web/course.html</a>.</p></li>
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -1216,27 +1216,27 @@ uncorrelated.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.9524796542295857
|
||||
[[12.32978852 17.93351637 8.83074569 6.12730791 9.17878199 10.22862364
|
||||
14.56202678 10.49110643 8.12113897 14.46581332]
|
||||
[17.93351637 26.08406534 12.84420428 8.91208933 13.35041853 14.87739949
|
||||
21.18027777 15.25917728 11.81209057 21.04033655]
|
||||
[ 8.83074569 12.84420428 6.32468833 4.38845304 6.57395619 7.32586565
|
||||
10.4295021 7.51385904 5.81645929 10.36059285]
|
||||
[ 6.12730791 8.91208933 4.38845304 3.04497536 4.56141023 5.0831307
|
||||
7.23662224 5.21357193 4.03581286 7.18880881]
|
||||
[ 9.17878199 13.35041853 6.57395619 4.56141023 6.83304817 7.61459179
|
||||
10.8405484 7.80999435 6.04569689 10.7689233 ]
|
||||
[10.22862364 14.87739949 7.32586565 5.0831307 7.61459179 8.48552604
|
||||
12.08045792 8.70327816 6.73718563 12.00064055]
|
||||
[14.56202678 21.18027777 10.4295021 7.23662224 10.8405484 12.08045792
|
||||
17.19839911 12.39046173 9.59142511 17.08476676]
|
||||
[10.49110643 15.25917728 7.51385904 5.21357193 7.80999435 8.70327816
|
||||
12.39046173 8.92661816 6.91007256 12.30859612]
|
||||
[ 8.12113897 11.81209057 5.81645929 4.03581286 6.04569689 6.73718563
|
||||
9.59142511 6.91007256 5.3490697 9.52805315]
|
||||
[14.46581332 21.04033655 10.36059285 7.18880881 10.7689233 12.00064055
|
||||
17.08476676 12.30859612 9.52805315 16.97188519]]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>2.4890704720929593
|
||||
[[11.60990408 7.82544198 14.11622885 1.78979284 6.76376813 14.09060397
|
||||
14.04624055 3.77253612 5.11082097 14.99821162]
|
||||
[ 7.82544198 5.27459502 9.51478402 1.20637689 4.55899331 9.49751204
|
||||
9.4676097 2.54280848 3.44485473 10.10926826]
|
||||
[14.11622885 9.51478402 17.16361442 2.17617003 8.22391797 17.13245767
|
||||
17.07851716 4.58694429 6.21413561 18.23599801]
|
||||
[ 1.78979284 1.20637689 2.17617003 0.27591601 1.04270833 2.17221967
|
||||
2.16538057 0.58157742 0.78788857 2.31213725]
|
||||
[ 6.76376813 4.55899331 8.22391797 1.04270833 3.94047694 8.20898927
|
||||
8.18314376 2.19782691 2.97749299 8.73774883]
|
||||
[14.09060397 9.49751204 17.13245767 2.17221967 8.20898927 17.10135749
|
||||
17.04751489 4.57861771 6.20285522 18.20289459]
|
||||
[14.04624055 9.4676097 17.07851716 2.16538057 8.18314376 17.04751489
|
||||
16.99384182 4.56420221 6.18332591 18.14558387]
|
||||
[ 3.77253612 2.54280848 4.58694429 0.58157742 2.19782691 4.57861771
|
||||
4.56420221 1.2258524 1.66071628 4.87353683]
|
||||
[ 5.11082097 3.44485473 6.21413561 0.78788857 2.97749299 6.20285522
|
||||
6.18332591 1.66071628 2.24984554 6.60239515]
|
||||
[14.99821162 10.10926826 18.23599801 2.31213725 8.73774883 18.20289459
|
||||
18.14558387 4.87353683 6.60239515 19.37538418]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1544,15 +1544,15 @@ more practically oriented methods like the blocking technique.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.0553457008138535
|
||||
4.04568240116212
|
||||
0.3078370846883027
|
||||
0.8876822883855056 9.132825151794417 26.525617578564635
|
||||
2.6555465443226955 3.4427165765504326 10.884978392264447
|
||||
[[ 0.88768229 2.65554654 3.44271658]
|
||||
[ 2.65554654 9.13282515 10.88497839]
|
||||
[ 3.44271658 10.88497839 26.52561758]]
|
||||
[32.33737942 0.10238075 4.10636484]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>-0.000171990965637209
|
||||
3.967581388294481
|
||||
-0.025521898567525465
|
||||
0.853884005749657 8.208733310166176 9.244798847617261
|
||||
2.497355838775297 2.088848443911507 6.174325810031482
|
||||
[[0.85388401 2.49735584 2.08884844]
|
||||
[2.49735584 8.20873331 6.17432581]
|
||||
[2.08884844 6.17432581 9.24479885]]
|
||||
[15.62869983 0.07851446 2.60020187]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1959,7 +1959,7 @@ assumption for approximating <span class="math notranslate nohighlight">\(\sigma
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>-0.00532744652682568 1.0584909962471034
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.005481494389472717 1.029117514654528
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/statistics_184_1.png" src="_images/statistics_184_1.png" />
|
||||
|
||||
@@ -475,7 +475,7 @@
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Converged at iteration 5\n",
|
||||
"Runtime: 0.4697279930114746 seconds\n"
|
||||
"Runtime: 0.4887218475341797 seconds\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
@@ -604,7 +604,7 @@
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Converged at iteration: 5\n",
|
||||
"Runtime: 0.41312265396118164 seconds\n"
|
||||
"Runtime: 0.432811975479126 seconds\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
@@ -745,7 +745,7 @@
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Converged at iteration: 11\n",
|
||||
"Runtime: 0.8252480030059814 seconds\n",
|
||||
"Runtime: 0.8653810024261475 seconds\n",
|
||||
" "
|
||||
]
|
||||
}
|
||||
@@ -877,7 +877,7 @@
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Converged at iteration: 5\n",
|
||||
"Runtime: 0.0037932395935058594 seconds\n"
|
||||
"Runtime: 0.0036406517028808594 seconds\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -2336,38 +2336,34 @@
|
||||
"Learning rate = 1.0\n",
|
||||
"Lambda = 1e-05\n",
|
||||
"Accuracy score on test set: 0.09166666666666666\n",
|
||||
"\n",
|
||||
"\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Learning rate = 1.0\n",
|
||||
"Lambda = 0.0001\n",
|
||||
"Accuracy score on test set: 0.11944444444444445\n",
|
||||
"\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"\n",
|
||||
"Learning rate = 1.0\n",
|
||||
"Lambda = 0.001\n",
|
||||
"Accuracy score on test set: 0.1361111111111111\n",
|
||||
"\n",
|
||||
"\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Learning rate = 1.0\n",
|
||||
"Lambda = 0.01\n",
|
||||
"Accuracy score on test set: 0.1527777777777778\n",
|
||||
"\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"\n",
|
||||
"Learning rate = 1.0\n",
|
||||
"Lambda = 0.1\n",
|
||||
"Accuracy score on test set: 0.16666666666666666\n",
|
||||
"\n",
|
||||
"Learning rate = 1.0\n",
|
||||
"Lambda = 1.0\n",
|
||||
"Accuracy score on test set: 0.1111111111111111\n",
|
||||
"\n"
|
||||
]
|
||||
},
|
||||
@@ -2375,10 +2371,20 @@
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Learning rate = 1.0\n",
|
||||
"Lambda = 1.0\n",
|
||||
"Accuracy score on test set: 0.1111111111111111\n",
|
||||
"\n",
|
||||
"Learning rate = 1.0\n",
|
||||
"Lambda = 10.0\n",
|
||||
"Accuracy score on test set: 0.05\n",
|
||||
"\n",
|
||||
"\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Learning rate = 10.0\n",
|
||||
"Lambda = 1e-05\n",
|
||||
"Accuracy score on test set: 0.08888888888888889\n",
|
||||
@@ -2388,15 +2394,15 @@
|
||||
"Accuracy score on test set: 0.08611111111111111\n",
|
||||
"\n",
|
||||
"Learning rate = 10.0\n",
|
||||
"Lambda = 0.001\n"
|
||||
"Lambda = 0.001\n",
|
||||
"Accuracy score on test set: 0.08888888888888889\n",
|
||||
"\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Accuracy score on test set: 0.08888888888888889\n",
|
||||
"\n",
|
||||
"Learning rate = 10.0\n",
|
||||
"Lambda = 0.01\n",
|
||||
"Accuracy score on test set: 0.08888888888888889\n",
|
||||
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 6.2 KiB After Width: | Height: | Size: 6.2 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 9.4 KiB After Width: | Height: | Size: 9.6 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 14 KiB After Width: | Height: | Size: 14 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 10 KiB After Width: | Height: | Size: 10 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 26 KiB After Width: | Height: | Size: 25 KiB |
@@ -1470,10 +1470,10 @@
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"0.05805002932374468\n",
|
||||
"4.309519578637819\n",
|
||||
"[[ 1.23033954 3.6804336 ]\n",
|
||||
" [ 3.6804336 11.92527504]]\n"
|
||||
"0.0119839668275544\n",
|
||||
"3.9026821944532157\n",
|
||||
"[[ 1.21805483 3.6417002 ]\n",
|
||||
" [ 3.6417002 11.67950066]]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
@@ -1513,10 +1513,10 @@
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"0.09198004868226574\n",
|
||||
"1.9493620821393187\n",
|
||||
"[[1. 0.68904673]\n",
|
||||
" [0.68904673 1. ]]\n"
|
||||
"0.08283349271892\n",
|
||||
"1.6319972672199556\n",
|
||||
"[[1. 0.65062105]\n",
|
||||
" [0.65062105 1. ]]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
@@ -1571,30 +1571,30 @@
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"[[-1.13352411e+00 -4.86145508e+00]\n",
|
||||
" [ 5.06793578e-01 2.24339370e+00]\n",
|
||||
" [ 4.08359441e-03 -1.44109702e+00]\n",
|
||||
" [-2.02972824e-01 -1.17393096e+00]\n",
|
||||
" [ 9.99703985e-01 5.00745587e+00]\n",
|
||||
" [-1.05430325e-01 3.31446832e-02]\n",
|
||||
" [ 4.31082669e-01 6.38783434e-01]\n",
|
||||
" [ 3.40259257e-02 5.84099240e-01]\n",
|
||||
" [-8.44862840e-01 -2.71546272e+00]\n",
|
||||
" [ 3.11100342e-01 1.68506886e+00]]\n",
|
||||
"[[ 0.30199745 0.72164987]\n",
|
||||
" [ 1.04858244 2.95539603]\n",
|
||||
" [ 0.46840512 1.52940309]\n",
|
||||
" [ 0.24117177 1.78848481]\n",
|
||||
" [-1.38690487 -4.17705119]\n",
|
||||
" [-0.65735283 -2.46141305]\n",
|
||||
" [ 0.94916469 2.81774574]\n",
|
||||
" [-0.52449732 -2.22140879]\n",
|
||||
" [-0.66999836 -2.66511337]\n",
|
||||
" [ 0.22943192 1.71230687]]\n",
|
||||
" 0 1\n",
|
||||
"0 -1.133524 -4.861455\n",
|
||||
"1 0.506794 2.243394\n",
|
||||
"2 0.004084 -1.441097\n",
|
||||
"3 -0.202973 -1.173931\n",
|
||||
"4 0.999704 5.007456\n",
|
||||
"5 -0.105430 0.033145\n",
|
||||
"6 0.431083 0.638783\n",
|
||||
"7 0.034026 0.584099\n",
|
||||
"8 -0.844863 -2.715463\n",
|
||||
"9 0.311100 1.685069\n",
|
||||
"0 0.301997 0.721650\n",
|
||||
"1 1.048582 2.955396\n",
|
||||
"2 0.468405 1.529403\n",
|
||||
"3 0.241172 1.788485\n",
|
||||
"4 -1.386905 -4.177051\n",
|
||||
"5 -0.657353 -2.461413\n",
|
||||
"6 0.949165 2.817746\n",
|
||||
"7 -0.524497 -2.221409\n",
|
||||
"8 -0.669998 -2.665113\n",
|
||||
"9 0.229432 1.712307\n",
|
||||
" 0 1\n",
|
||||
"0 1.000000 0.958527\n",
|
||||
"1 0.958527 1.000000\n"
|
||||
"0 1.000000 0.975127\n",
|
||||
"1 0.975127 1.000000\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
@@ -1636,37 +1636,37 @@
|
||||
"text": [
|
||||
" 0 1 2 3 4 5 6 7 \\\n",
|
||||
"0 0.0 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 \n",
|
||||
"1 0.0 0.082212 0.080493 0.081616 0.077560 0.073611 0.073557 0.069528 \n",
|
||||
"2 0.0 0.080493 0.080583 0.082413 0.079267 0.075998 0.075624 0.072071 \n",
|
||||
"3 0.0 0.081616 0.082413 0.087107 0.084071 0.080840 0.082136 0.078414 \n",
|
||||
"4 0.0 0.077560 0.079267 0.084071 0.081732 0.079085 0.080069 0.076853 \n",
|
||||
"5 0.0 0.073611 0.075998 0.080840 0.079085 0.076946 0.077662 0.074900 \n",
|
||||
"6 0.0 0.073557 0.075624 0.082136 0.080069 0.077662 0.079815 0.076731 \n",
|
||||
"7 0.0 0.069528 0.072071 0.078414 0.076853 0.074900 0.076731 0.074079 \n",
|
||||
"8 0.0 0.065794 0.068700 0.074858 0.073728 0.072170 0.073713 0.071447 \n",
|
||||
"9 0.0 0.062352 0.065534 0.071499 0.070739 0.069526 0.070812 0.068888 \n",
|
||||
"10 0.0 0.065582 0.068203 0.075503 0.074125 0.072352 0.074961 0.072447 \n",
|
||||
"11 0.0 0.062022 0.064909 0.071917 0.070915 0.069494 0.071782 0.069625 \n",
|
||||
"12 0.0 0.058765 0.061857 0.068581 0.067904 0.066793 0.068790 0.066951 \n",
|
||||
"13 0.0 0.055788 0.059038 0.065492 0.065096 0.064256 0.065991 0.064434 \n",
|
||||
"14 0.0 0.053069 0.056441 0.062636 0.062486 0.061885 0.063380 0.062076 \n",
|
||||
"1 0.0 0.096033 0.089499 0.100065 0.095636 0.091356 0.093135 0.089366 \n",
|
||||
"2 0.0 0.089499 0.084679 0.095430 0.091860 0.088348 0.090343 0.087109 \n",
|
||||
"3 0.0 0.100065 0.095430 0.109996 0.106272 0.102568 0.105983 0.102429 \n",
|
||||
"4 0.0 0.095636 0.091860 0.106272 0.103072 0.099840 0.103254 0.100074 \n",
|
||||
"5 0.0 0.091356 0.088348 0.102568 0.099840 0.097041 0.100443 0.097606 \n",
|
||||
"6 0.0 0.093135 0.090343 0.105983 0.103254 0.100443 0.104606 0.101680 \n",
|
||||
"7 0.0 0.089366 0.087109 0.102429 0.100074 0.097606 0.101680 0.099049 \n",
|
||||
"8 0.0 0.085816 0.084032 0.099028 0.097007 0.094850 0.098834 0.096471 \n",
|
||||
"9 0.0 0.082468 0.081101 0.095771 0.094052 0.092175 0.096070 0.093951 \n",
|
||||
"10 0.0 0.085269 0.083748 0.099374 0.097448 0.095375 0.099827 0.097488 \n",
|
||||
"11 0.0 0.082003 0.080845 0.096087 0.094440 0.092628 0.096955 0.094852 \n",
|
||||
"12 0.0 0.078938 0.078099 0.092968 0.091572 0.089996 0.094201 0.092313 \n",
|
||||
"13 0.0 0.076058 0.075503 0.090008 0.088838 0.087475 0.091561 0.089869 \n",
|
||||
"14 0.0 0.073350 0.073045 0.087198 0.086231 0.085061 0.089033 0.087519 \n",
|
||||
"\n",
|
||||
" 8 9 10 11 12 13 14 \n",
|
||||
"0 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 \n",
|
||||
"1 0.065794 0.062352 0.065582 0.062022 0.058765 0.055788 0.053069 \n",
|
||||
"2 0.068700 0.065534 0.068203 0.064909 0.061857 0.059038 0.056441 \n",
|
||||
"3 0.074858 0.071499 0.075503 0.071917 0.068581 0.065492 0.062636 \n",
|
||||
"4 0.073728 0.070739 0.074125 0.070915 0.067904 0.065096 0.062486 \n",
|
||||
"5 0.072170 0.069526 0.072352 0.069494 0.066793 0.064256 0.061885 \n",
|
||||
"6 0.073713 0.070812 0.074961 0.071782 0.068790 0.065991 0.063380 \n",
|
||||
"7 0.071447 0.068888 0.072447 0.069625 0.066951 0.064434 0.062076 \n",
|
||||
"8 0.069161 0.066912 0.069940 0.067442 0.065059 0.062803 0.060678 \n",
|
||||
"9 0.066912 0.064945 0.067492 0.065289 0.063171 0.061154 0.059245 \n",
|
||||
"10 0.069940 0.067492 0.071541 0.068800 0.066196 0.063738 0.061430 \n",
|
||||
"11 0.067442 0.065289 0.068800 0.066372 0.064050 0.061846 0.059767 \n",
|
||||
"12 0.065059 0.063171 0.066196 0.064050 0.061983 0.060011 0.058141 \n",
|
||||
"13 0.062803 0.061154 0.063738 0.061846 0.060011 0.058249 0.056570 \n",
|
||||
"14 0.060678 0.059245 0.061430 0.059767 0.058141 0.056570 0.055065 \n"
|
||||
"1 0.085816 0.082468 0.085269 0.082003 0.078938 0.076058 0.073350 \n",
|
||||
"2 0.084032 0.081101 0.083748 0.080845 0.078099 0.075503 0.073045 \n",
|
||||
"3 0.099028 0.095771 0.099374 0.096087 0.092968 0.090008 0.087198 \n",
|
||||
"4 0.097007 0.094052 0.097448 0.094440 0.091572 0.088838 0.086231 \n",
|
||||
"5 0.094850 0.092175 0.095375 0.092628 0.089996 0.087475 0.085061 \n",
|
||||
"6 0.098834 0.096070 0.099827 0.096955 0.094201 0.091561 0.089033 \n",
|
||||
"7 0.096471 0.093951 0.097488 0.094852 0.092313 0.089869 0.087519 \n",
|
||||
"8 0.094138 0.091843 0.095173 0.092754 0.090414 0.088152 0.085968 \n",
|
||||
"9 0.091843 0.089754 0.092891 0.090673 0.088516 0.086423 0.084394 \n",
|
||||
"10 0.095173 0.092891 0.096568 0.094135 0.091779 0.089500 0.087297 \n",
|
||||
"11 0.092754 0.090673 0.094135 0.091901 0.089727 0.087616 0.085567 \n",
|
||||
"12 0.090414 0.088516 0.091779 0.089727 0.087721 0.085765 0.083859 \n",
|
||||
"13 0.088152 0.086423 0.089500 0.087616 0.085765 0.083951 0.082179 \n",
|
||||
"14 0.085968 0.084394 0.087297 0.085567 0.083859 0.082179 0.080529 \n"
|
||||
]
|
||||
}
|
||||
],
|
||||
@@ -3273,14 +3273,7 @@
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"[ 2.03099776 -0.17917768 5.18029127]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"\n",
|
||||
"[ 2.03099776 -0.17917768 5.18029127]\n",
|
||||
"Training MSE for OLS\n",
|
||||
"0.009163470508352228\n",
|
||||
"Test MSE OLS\n",
|
||||
@@ -3296,7 +3289,7 @@
|
||||
},
|
||||
"metadata": {
|
||||
"filenames": {
|
||||
"image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter2_249_2.png"
|
||||
"image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter2_249_1.png"
|
||||
},
|
||||
"needs_background": "light"
|
||||
},
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -1236,17 +1236,6 @@ $$
|
||||
\sum_{i=0}^{n-1} \beta_0 = \sum_{i=0}^{n-1}y_i - \sum_{i=0}^{n-1} \sum_{j=1}^{p-1} X_{ij} \beta_j.
|
||||
$$
|
||||
|
||||
We assume
|
||||
that every column of $\boldsymbol{X}$ is centered, which we can do by subtracting the mean,
|
||||
|
||||
X = X - np.mean(X,axis=0)
|
||||
|
||||
This means that we need to rewrite $X_{ij}$ as $\tilde{X}_{ij}=X_{ij}-\mu_j$, where
|
||||
|
||||
$$
|
||||
\mu_j = \frac{1}{n}\sum_{i=0}^{n-1}X_{ij}.
|
||||
$$
|
||||
|
||||
Let us special first to the case where we have only two parameters $\beta_0$ and $\beta_1$.
|
||||
Our result for $\beta_0$ simplifies then to
|
||||
|
||||
@@ -1254,13 +1243,13 @@ $$
|
||||
n\beta_0 = \sum_{i=0}^{n-1}y_i - \sum_{i=0}^{n-1} X_{i1} \beta_1.
|
||||
$$
|
||||
|
||||
Assuming that the matrix elements $X_{i1}$ are centered, what we have is
|
||||
We obtain then
|
||||
|
||||
$$
|
||||
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} \left(X_{i1}-\mu_{1}\right),
|
||||
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} X_{i1}.
|
||||
$$
|
||||
|
||||
where
|
||||
If we define
|
||||
|
||||
$$
|
||||
\mu_1=\frac{1}{n}\sum_{i=0}^{n-1} (X_{i1},
|
||||
@@ -1275,25 +1264,16 @@ $$
|
||||
we have
|
||||
|
||||
$$
|
||||
\beta_0 = \mu_y - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} (X_{i1}-\mu_{1}),
|
||||
\beta_0 = \mu_y - \beta_1\mu_{1}.
|
||||
$$
|
||||
|
||||
and it is easy to see that the last sum equals zero! This means that we have
|
||||
In the general case withmore parameters than $\beta_0$ and $\beta_1$, we have
|
||||
|
||||
$$
|
||||
\beta_0 = \mu_y,
|
||||
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \frac{1}{n}\sum_{i=0}^{n-1}\sum_{j=1}^{p-1} X_{ij}\beta_j.
|
||||
$$
|
||||
|
||||
if the columns of the design matrix are centered. It is straight forward to generalize this results to more values of $\beta$.
|
||||
We have thus
|
||||
|
||||
$$
|
||||
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1} y_i = \overline{\boldsymbol{y}},
|
||||
$$
|
||||
|
||||
the average value of $\boldsymbol{y}$.
|
||||
|
||||
Replacing $y_i$ with $y_i - \beta_0 = y_i - \overline{\boldsymbol{y}}$ and centering also our design matrix results in a cost function (in vector-matrix disguise)
|
||||
Replacing $y_i$ with $y_i - y_i - \overline{\boldsymbol{y}}$ and centering also our design matrix results in a cost function (in vector-matrix disguise)
|
||||
|
||||
$$
|
||||
C(\boldsymbol{\beta}) = (\boldsymbol{\tilde{y}} - \tilde{X}\boldsymbol{\beta})^T(\boldsymbol{\tilde{y}} - \tilde{X}\boldsymbol{\beta}).
|
||||
@@ -1567,8 +1547,6 @@ for i in range(nlambdas):
|
||||
OwnRidgeBeta = np.linalg.pinv(X_train_scaled.T @ X_train_scaled+lmb*I) @ X_train_scaled.T @ (y_train_scaled)
|
||||
intercept_ = y_scaler - X_train_mean@OwnRidgeBeta #The intercept can be shifted so the model can predict on uncentered data
|
||||
#Add intercept to prediction
|
||||
ypredictOwnRidge = X_test @ OwnRidgeBeta + intercept_
|
||||
#Add intercept to prediction
|
||||
ypredictOwnRidge = X_test_scaled @ OwnRidgeBeta + y_scaler
|
||||
RegRidge = linear_model.Ridge(lmb)
|
||||
RegRidge.fit(X_train,y_train)
|
||||
@@ -1939,8 +1917,8 @@ involves a new cost function which leads to a new estimate for the
|
||||
weights $\boldsymbol{\beta}$. This results in a penalized regression problem. The
|
||||
cost function is given by
|
||||
|
||||
6
|
||||
0
|
||||
5
|
||||
8
|
||||
|
||||
<
|
||||
<
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 15 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 11 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 13 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 20 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 31 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 30 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 31 KiB After Width: | Height: | Size: 20 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 193 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 71 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 28 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 11 KiB After Width: | Height: | Size: 11 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 16 KiB |
File diff suppressed because one or more lines are too long
Binary file not shown.
|
Before Width: | Height: | Size: 28 KiB After Width: | Height: | Size: 31 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 21 KiB After Width: | Height: | Size: 20 KiB |
File diff suppressed because one or more lines are too long
Binary file not shown.
|
Before Width: | Height: | Size: 31 KiB After Width: | Height: | Size: 29 KiB |
File diff suppressed because one or more lines are too long
Binary file not shown.
|
Before Width: | Height: | Size: 12 KiB After Width: | Height: | Size: 12 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 10 KiB After Width: | Height: | Size: 11 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 11 KiB |
@@ -159,8 +159,8 @@
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"[-0.98226332 0.21126669 0.23494196 1.04532409 -0.994015 -0.37831577\n",
|
||||
" -0.22534931 0.39445208 -1.97921062 1.042552 ]\n"
|
||||
"[-0.62370636 0.34673123 0.80408254 2.08638654 1.16894795 1.8464107\n",
|
||||
" 0.13620973 -1.33881021 -0.05404886 -0.99995891]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
|
||||
File diff suppressed because one or more lines are too long
Binary file not shown.
|
Before Width: | Height: | Size: 5.0 KiB After Width: | Height: | Size: 5.0 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 9.5 KiB After Width: | Height: | Size: 9.4 KiB |
Binary file not shown.
Reference in New Issue
Block a user