lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update

Browse Source
This commit is contained in:
Morten Hjorth-Jensen
2023-08-28 10:43:56 +02:00
parent 707c8e08f3
commit d595a91b18
29 changed files with 3297 additions and 1826 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/LectureNotes/_build/.doctrees/environment.pickle
BIN
View File
Binary file not shown.
doc/LectureNotes/_build/.doctrees/week35.doctree
BIN
View File
Binary file not shown.
doc/LectureNotes/_build/html/_images/week35_146_0.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 23 KiB

doc/LectureNotes/_build/html/_images/week35_181_1.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 27 KiB

doc/LectureNotes/_build/html/_images/week35_199_1.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 40 KiB

doc/LectureNotes/_build/html/_images/week35_201_1.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 165 KiB

doc/LectureNotes/_build/html/_images/week35_203_0.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 58 KiB

doc/LectureNotes/_build/html/_images/week35_210_0.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 19 KiB

doc/LectureNotes/_build/html/_sources/week35.ipynb
+434 -433
View File
File diff suppressed because it is too large Load Diff
doc/LectureNotes/_build/html/searchindex.js
+1 -1
View File
File diff suppressed because one or more lines are too long
doc/LectureNotes/_build/html/week35.html
+648 -18
View File
@@ -1583,22 +1583,11 @@ We assume our data can represented by a fourth-order polynomial. For the <span c
<p>Alternatively, you can use the least squares functionality in <strong>Numpy</strong> as</p>
<div class="cell docutils container">
<div class="cell_input docutils container">
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="n">fit</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">linalg</span><span class="o">.</span><span class="n">lstsq</span><span class="p">(</span><span class="n">X</span><span class="p">,</span> <span class="n">Energies</span><span class="p">,</span> <span class="n">rcond</span> <span class="o">=</span><span class="kc">None</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span>
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="n">fit</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">linalg</span><span class="o">.</span><span class="n">lstsq</span><span class="p">(</span><span class="n">X</span><span class="p">,</span> <span class="n">y</span><span class="p">,</span> <span class="n">rcond</span> <span class="o">=</span><span class="kc">None</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span>
<span class="n">ytildenp</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">dot</span><span class="p">(</span><span class="n">fit</span><span class="p">,</span><span class="n">X</span><span class="o">.</span><span class="n">T</span><span class="p">)</span>
</pre></div>
</div>
</div>
<div class="cell_output docutils container">
<div class="output traceback highlight-ipythontb notranslate"><div class="highlight"><pre><span></span><span class="gt">---------------------------------------------------------------------------</span>
<span class="ne">NameError</span><span class="g g-Whitespace"> </span>Traceback (most recent call last)
<span class="nn">Input In [2],</span> in <span class="ni">&lt;cell line: 1&gt;</span><span class="nt">()</span>
<span class="ne">----&gt; </span><span class="mi">1</span> <span class="n">fit</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">linalg</span><span class="o">.</span><span class="n">lstsq</span><span class="p">(</span><span class="n">X</span><span class="p">,</span> <span class="n">Energies</span><span class="p">,</span> <span class="n">rcond</span> <span class="o">=</span><span class="kc">None</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span>
<span class="g g-Whitespace"> </span><span class="mi">2</span> <span class="n">ytildenp</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">dot</span><span class="p">(</span><span class="n">fit</span><span class="p">,</span><span class="n">X</span><span class="o">.</span><span class="n">T</span><span class="p">)</span>
<span class="ne">NameError</span>: name &#39;Energies&#39; is not defined
</pre></div>
</div>
</div>
</div>
</div>
<div class="section" id="adding-error-analysis-and-training-set-up">
@@ -1616,7 +1605,12 @@ Since we are not using <strong>Scikit-Learn</strong> here we can define our own
<p>and we would be using it as</p>
<div class="cell docutils container">
<div class="cell_input docutils container">
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="nb">print</span><span class="p">(</span><span class="n">R2</span><span class="p">(</span><span class="n">Energies</span><span class="p">,</span><span class="n">ytilde</span><span class="p">))</span>
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="nb">print</span><span class="p">(</span><span class="n">R2</span><span class="p">(</span><span class="n">y</span><span class="p">,</span><span class="n">ytilde</span><span class="p">))</span>
</pre></div>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.9952495547161103
</pre></div>
</div>
</div>
@@ -1628,7 +1622,12 @@ Since we are not using <strong>Scikit-Learn</strong> here we can define our own
<span class="n">n</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">size</span><span class="p">(</span><span class="n">y_model</span><span class="p">)</span>
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">sum</span><span class="p">((</span><span class="n">y_data</span><span class="o">-</span><span class="n">y_model</span><span class="p">)</span><span class="o">**</span><span class="mi">2</span><span class="p">)</span><span class="o">/</span><span class="n">n</span>
<span class="nb">print</span><span class="p">(</span><span class="n">MSE</span><span class="p">(</span><span class="n">Energies</span><span class="p">,</span><span class="n">ytilde</span><span class="p">))</span>
<span class="nb">print</span><span class="p">(</span><span class="n">MSE</span><span class="p">(</span><span class="n">y</span><span class="p">,</span><span class="n">ytilde</span><span class="p">))</span>
</pre></div>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.00950269957492606
</pre></div>
</div>
</div>
@@ -1638,7 +1637,28 @@ Since we are not using <strong>Scikit-Learn</strong> here we can define our own
<div class="cell_input docutils container">
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">RelativeError</span><span class="p">(</span><span class="n">y_data</span><span class="p">,</span><span class="n">y_model</span><span class="p">):</span>
<span class="k">return</span> <span class="nb">abs</span><span class="p">((</span><span class="n">y_data</span><span class="o">-</span><span class="n">y_model</span><span class="p">)</span><span class="o">/</span><span class="n">y_data</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">RelativeError</span><span class="p">(</span><span class="n">Energies</span><span class="p">,</span> <span class="n">ytilde</span><span class="p">))</span>
<span class="nb">print</span><span class="p">(</span><span class="n">RelativeError</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="n">ytilde</span><span class="p">))</span>
</pre></div>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[0.0193453 0.01242273 0.01256189 0.01871582 0.02840224 0.05004617
0.03536673 0.00706459 0.01326608 0.00817056 0.0443659 0.02090061
0.03724642 0.02376036 0.01943082 0.11352543 0.03437582 0.04426816
0.0055043 0.00779804 0.00415473 0.01761413 0.01206708 0.0234505
0.02928941 0.03893106 0.04678736 0.08370888 0.0739975 0.02035804
0.04028979 0.03739417 0.04670006 0.02383034 0.00416461 0.03232719
0.0156548 0.00393128 0.0957156 0.01406736 0.02551463 0.00388494
0.06432223 0.03509912 0.01422494 0.04787099 0.01323884 0.02803041
0.00075315 0.01037947 0.03255258 0.01631138 0.02129803 0.00715835
0.00427039 0.02298589 0.00128634 0.00877989 0.00444561 0.0459682
0.01751529 0.02917931 0.00791291 0.01207435 0.01960258 0.00431511
0.00407056 0.00220229 0.00267599 0.03371334 0.01654471 0.10880921
0.07557024 0.01944669 0.01542944 0.07143876 0.0136671 0.00695357
0.0021877 0.08084392 0.02862254 0.02701924 0.0499586 0.00202464
0.0113975 0.03423556 0.09652672 0.05150928 0.02979908 0.00569317
0.00112431 0.00780259 0.01200342 0.00397836 0.06287717 0.00403785
0.0068094 0.02086933 0.0165866 0.0066916 ]
</pre></div>
</div>
</div>
@@ -1706,6 +1726,19 @@ but now splitting the data into a training set and a test set.</p>
</pre></div>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[ 1.96769675 -0.05402365 6.44997121 -3.20038136 1.84614066]
Training R2
0.996694948530336
Training MSE
0.00740173378493795
Test R2
0.9920099299023244
Test MSE
0.013537225644261203
</pre></div>
</div>
</div>
</div>
</div>
<div class="section" id="making-your-own-test-train-splitting">
@@ -1844,6 +1877,346 @@ represent a specific feature whose mean value is subracted.</p>
</pre></div>
</div>
</div>
<div class="cell_output docutils container">
<div class="output text_html"><div>
<style scoped>
.dataframe tbody tr th:only-of-type {
vertical-align: middle;
}
.dataframe tbody tr th {
vertical-align: top;
}
.dataframe thead th {
text-align: right;
}
</style>
<table border="1" class="dataframe">
<thead>
<tr style="text-align: right;">
<th></th>
<th>0</th>
<th>1</th>
<th>2</th>
<th>3</th>
<th>4</th>
</tr>
</thead>
<tbody>
<tr>
<th>0</th>
<td>-1.749765</td>
<td>0.342680</td>
<td>1.153036</td>
<td>-0.252436</td>
<td>0.981321</td>
</tr>
<tr>
<th>1</th>
<td>0.514219</td>
<td>0.221180</td>
<td>-1.070043</td>
<td>-0.189496</td>
<td>0.255001</td>
</tr>
<tr>
<th>2</th>
<td>-0.458027</td>
<td>0.435163</td>
<td>-0.583595</td>
<td>0.816847</td>
<td>0.672721</td>
</tr>
<tr>
<th>3</th>
<td>-0.104411</td>
<td>-0.531280</td>
<td>1.029733</td>
<td>-0.438136</td>
<td>-1.118318</td>
</tr>
<tr>
<th>4</th>
<td>1.618982</td>
<td>1.541605</td>
<td>-0.251879</td>
<td>-0.842436</td>
<td>0.184519</td>
</tr>
<tr>
<th>5</th>
<td>0.937082</td>
<td>0.731000</td>
<td>1.361556</td>
<td>-0.326238</td>
<td>0.055676</td>
</tr>
<tr>
<th>6</th>
<td>0.222400</td>
<td>-1.443217</td>
<td>-0.756352</td>
<td>0.816454</td>
<td>0.750445</td>
</tr>
<tr>
<th>7</th>
<td>-0.455947</td>
<td>1.189622</td>
<td>-1.690617</td>
<td>-1.356399</td>
<td>-1.232435</td>
</tr>
<tr>
<th>8</th>
<td>-0.544439</td>
<td>-0.668172</td>
<td>0.007315</td>
<td>-0.612939</td>
<td>1.299748</td>
</tr>
<tr>
<th>9</th>
<td>-1.733096</td>
<td>-0.983310</td>
<td>0.357508</td>
<td>-1.613579</td>
<td>1.470714</td>
</tr>
</tbody>
</table>
</div></div><div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0 -0.175300
1 0.083527
2 -0.044334
3 -0.399836
4 0.331939
dtype: float64
0 1.069584
1 0.965548
2 1.018232
3 0.793167
4 0.918992
dtype: float64
</pre></div>
</div>
<div class="output text_html"><div>
<style scoped>
.dataframe tbody tr th:only-of-type {
vertical-align: middle;
}
.dataframe tbody tr th {
vertical-align: top;
}
.dataframe thead th {
text-align: right;
}
</style>
<table border="1" class="dataframe">
<thead>
<tr style="text-align: right;">
<th></th>
<th>0</th>
<th>1</th>
<th>2</th>
<th>3</th>
<th>4</th>
</tr>
</thead>
<tbody>
<tr>
<th>0</th>
<td>-1.574465</td>
<td>0.259153</td>
<td>1.197370</td>
<td>0.147400</td>
<td>0.649382</td>
</tr>
<tr>
<th>1</th>
<td>0.689519</td>
<td>0.137652</td>
<td>-1.025709</td>
<td>0.210340</td>
<td>-0.076938</td>
</tr>
<tr>
<th>2</th>
<td>-0.282727</td>
<td>0.351636</td>
<td>-0.539261</td>
<td>1.216683</td>
<td>0.340782</td>
</tr>
<tr>
<th>3</th>
<td>0.070889</td>
<td>-0.614808</td>
<td>1.074067</td>
<td>-0.038300</td>
<td>-1.450257</td>
</tr>
<tr>
<th>4</th>
<td>1.794282</td>
<td>1.458078</td>
<td>-0.207545</td>
<td>-0.442600</td>
<td>-0.147420</td>
</tr>
<tr>
<th>5</th>
<td>1.112383</td>
<td>0.647473</td>
<td>1.405890</td>
<td>0.073598</td>
<td>-0.276263</td>
</tr>
<tr>
<th>6</th>
<td>0.397700</td>
<td>-1.526744</td>
<td>-0.712018</td>
<td>1.216290</td>
<td>0.418506</td>
</tr>
<tr>
<th>7</th>
<td>-0.280647</td>
<td>1.106095</td>
<td>-1.646283</td>
<td>-0.956563</td>
<td>-1.564374</td>
</tr>
<tr>
<th>8</th>
<td>-0.369139</td>
<td>-0.751699</td>
<td>0.051649</td>
<td>-0.213103</td>
<td>0.967809</td>
</tr>
<tr>
<th>9</th>
<td>-1.557795</td>
<td>-1.066837</td>
<td>0.401842</td>
<td>-1.213743</td>
<td>1.138775</td>
</tr>
</tbody>
</table>
</div></div><div class="output text_html"><div>
<style scoped>
.dataframe tbody tr th:only-of-type {
vertical-align: middle;
}
.dataframe tbody tr th {
vertical-align: top;
}
.dataframe thead th {
text-align: right;
}
</style>
<table border="1" class="dataframe">
<thead>
<tr style="text-align: right;">
<th></th>
<th>0</th>
<th>1</th>
<th>2</th>
<th>3</th>
<th>4</th>
</tr>
</thead>
<tbody>
<tr>
<th>0</th>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
</tr>
<tr>
<th>1</th>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
</tr>
<tr>
<th>2</th>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
</tr>
<tr>
<th>3</th>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
</tr>
<tr>
<th>4</th>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
</tr>
<tr>
<th>5</th>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
</tr>
<tr>
<th>6</th>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
</tr>
<tr>
<th>7</th>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
</tr>
<tr>
<th>8</th>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
</tr>
<tr>
<th>9</th>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
</tr>
</tbody>
</table>
</div></div></div>
</div>
<p>Small exercise: perform the standard scaling by including the standard deviation and compare with what Scikit-Learn gives.</p>
</div>
@@ -1919,6 +2292,9 @@ the aims is to reproduce Figure 2.11 of <a class="reference external" href="http
</pre></div>
</div>
</div>
<div class="cell_output docutils container">
<img alt="_images/week35_146_0.png" src="_images/week35_146_0.png" />
</div>
</div>
</div>
<div class="section" id="more-preprocessing-examples-two-dimensional-example-the-franke-function">
@@ -2020,6 +2396,36 @@ the aims is to reproduce Figure 2.11 of <a class="reference external" href="http
</pre></div>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>MSE before scaling: 0.00
R2 score before scaling 1.00
Feature min values before scaling:
[1.00000000e+00 6.97906022e-03 2.43639284e-03 4.87072815e-05
1.70037324e-05 5.93601008e-06 3.39931051e-07 1.18670072e-07
4.14277718e-08 1.44624525e-08 2.37239927e-09 8.28205578e-10
2.89126914e-10 1.00934327e-10 3.52362157e-11 1.65571174e-11
5.78009660e-12 2.01783414e-12 7.04426744e-13 2.45915671e-13
8.58492636e-14]
Feature max values before scaling:
[1. 0.99970894 0.99978365 0.99941797 0.99949266 0.99956735
0.99912709 0.99920175 0.99927642 0.9993511 0.99883628 0.99891093
0.99898558 0.99906023 0.99913489 0.99854557 0.99862019 0.99869482
0.99876945 0.99884409 0.99891873]
Feature min values after scaling:
[ 0. -1.71761101 -1.75770568 -1.12330033 -1.12591227 -1.12871842
-0.88613493 -0.884669 -0.88323026 -0.88182591 -0.75269037 -0.75050135
-0.74829661 -0.74607851 -0.74384949 -0.6652177 -0.66294408 -0.66064822
-0.65833132 -0.65599456 -0.6536392 ]
Feature max values after scaling:
[0. 1.71737253 1.75576555 2.20916295 2.23971032 2.26995402
2.60543038 2.63162342 2.65743689 2.68286725 2.94273542 2.96631321
2.98947894 3.01222822 3.034557 3.24159785 3.26297455 3.28391978
3.30442964 3.32450054 3.34412923]
MSE after scaling: 0.00
R2 score for scaled data: 1.00
</pre></div>
</div>
</div>
</div>
</div>
<div class="section" id="to-think-about-first-part">
@@ -2067,7 +2473,8 @@ This can clearly lead to problems in evaluating the cost/loss functions.</p>
<div class="section" id="still-thinking">
<h2>Still thinking<a class="headerlink" href="#still-thinking" title="Permalink to this headline"></a></h2>
<p>Keep in mind that when you transform your data set before training a model, the same transformation needs to be done
on your eventual new data set before making a prediction. If we translate this into a Python code, it would could be implemented as follows</p>
on your eventual new data set before making a prediction. If we translate this into a Python code, it would could be implemented as follows
(note that the lines are commented since the model function has not been defined)</p>
<div class="cell docutils container">
<div class="cell_input docutils container">
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="c1">#Model training, we compute the mean value of y and X</span>
@@ -2077,12 +2484,12 @@ on your eventual new data set before making a prediction. If we translate this
<span class="n">y_train</span> <span class="o">=</span> <span class="n">y_train</span> <span class="o">-</span> <span class="n">y_train_mean</span>
<span class="c1"># The we fit our model with the training data</span>
<span class="n">trained_model</span> <span class="o">=</span> <span class="n">some_model</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">X_train</span><span class="p">,</span><span class="n">y_train</span><span class="p">)</span>
<span class="c1">#trained_model = some_model.fit(X_train,y_train)</span>
<span class="c1">#Model prediction, we need also to transform our data set used for the prediction.</span>
<span class="n">X_test</span> <span class="o">=</span> <span class="n">X_test</span> <span class="o">-</span> <span class="n">X_train_mean</span> <span class="c1">#Use mean from training data</span>
<span class="n">y_pred</span> <span class="o">=</span> <span class="n">trained_model</span><span class="p">(</span><span class="n">X_test</span><span class="p">)</span>
<span class="c1">#y_pred = trained_model(X_test)</span>
<span class="n">y_pred</span> <span class="o">=</span> <span class="n">y_pred</span> <span class="o">+</span> <span class="n">y_train_mean</span>
</pre></div>
</div>
@@ -2269,6 +2676,26 @@ Note also that we do not split the data into training and test.</p>
</pre></div>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>True beta: [2, 0.5, 3.7]
Fitted beta: [2.08376632 0.19569961 3.97898392]
Sklearn fitted beta: [2.08376632 0.19569961 3.97898392]
MSE with intercept column
0.004113634617443139
MSE with intercept column from SKL
0.004113634617443147
Manual intercept: 2.083766322923899
Fitted beta (wiothout intercept): [0.19569961 3.97898392]
Sklearn intercept: 2.0837663229239043
Sklearn fitted beta (without intercept): [0.19569961 3.97898392]
MSE with Manual intercept
0.00411363461744314
MSE with Sklearn intercept
0.004113634617443131
</pre></div>
</div>
<img alt="_images/week35_181_1.png" src="_images/week35_181_1.png" />
</div>
</div>
<p>The intercept is the value of our output/target variable
when all our features are zero and our function crosses the <span class="math notranslate nohighlight">\(y\)</span>-axis (for a one-dimensional case).</p>
@@ -2354,6 +2781,49 @@ the house using the features (predictors) listed here.</p>
</pre></div>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function load_boston is deprecated; `load_boston` is deprecated in 1.0 and will be removed in 1.2.
The Boston housing prices dataset has an ethical problem. You can refer to
the documentation of this function for further details.
The scikit-learn maintainers therefore strongly discourage the use of this
dataset unless the purpose of the code is to study and educate about
ethical issues in data science and machine learning.
In this special case, you can fetch the dataset from the original
source::
import pandas as pd
import numpy as np
data_url = &quot;http://lib.stat.cmu.edu/datasets/boston&quot;
raw_df = pd.read_csv(data_url, sep=&quot;\s+&quot;, skiprows=22, header=None)
data = np.hstack([raw_df.values[::2, :], raw_df.values[1::2, :2]])
target = raw_df.values[1::2, 2]
Alternative datasets include the California housing dataset (i.e.
:func:`~sklearn.datasets.fetch_california_housing`) and the Ames housing
dataset. You can load the datasets as follows::
from sklearn.datasets import fetch_california_housing
housing = fetch_california_housing()
for the California housing dataset and::
from sklearn.datasets import fetch_openml
housing = fetch_openml(name=&quot;house_prices&quot;, as_frame=True)
for the Ames housing dataset.
warnings.warn(msg, category=FutureWarning)
</pre></div>
</div>
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>dict_keys([&#39;data&#39;, &#39;target&#39;, &#39;feature_names&#39;, &#39;DESCR&#39;, &#39;filename&#39;, &#39;data_module&#39;])
</pre></div>
</div>
</div>
</div>
<p>Then we invoke Pandas</p>
<div class="cell docutils container">
@@ -2373,6 +2843,25 @@ the house using the features (predictors) listed here.</p>
</pre></div>
</div>
</div>
<div class="cell_output docutils container">
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>CRIM 0
ZN 0
INDUS 0
CHAS 0
NOX 0
RM 0
AGE 0
DIS 0
RAD 0
TAX 0
PTRATIO 0
B 0
LSTAT 0
MEDV 0
dtype: int64
</pre></div>
</div>
</div>
</div>
<p>We can then visualize the data</p>
<div class="cell docutils container">
@@ -2386,6 +2875,13 @@ the house using the features (predictors) listed here.</p>
</pre></div>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/seaborn/distributions.py:2619: FutureWarning: `distplot` is a deprecated function and will be removed in a future version. Please adapt your code to use either `displot` (a figure-level function with similar flexibility) or `histplot` (an axes-level function for histograms).
warnings.warn(msg, FutureWarning)
</pre></div>
</div>
<img alt="_images/week35_199_1.png" src="_images/week35_199_1.png" />
</div>
</div>
<p>It is now useful to look at the correlation matrix</p>
<div class="cell docutils container">
@@ -2398,6 +2894,12 @@ the house using the features (predictors) listed here.</p>
</pre></div>
</div>
</div>
<div class="cell_output docutils container">
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;AxesSubplot:&gt;
</pre></div>
</div>
<img alt="_images/week35_201_1.png" src="_images/week35_201_1.png" />
</div>
</div>
<p>From the above coorelation plot we can see that <strong>MEDV</strong> is strongly correlated to <strong>LSTAT</strong> and <strong>RM</strong>. We see also that <strong>RAD</strong> and <strong>TAX</strong> are stronly correlated, but we don’t include this in our features together to avoid multi-colinearity</p>
<div class="cell docutils container">
@@ -2418,6 +2920,9 @@ the house using the features (predictors) listed here.</p>
</pre></div>
</div>
</div>
<div class="cell_output docutils container">
<img alt="_images/week35_203_0.png" src="_images/week35_203_0.png" />
</div>
</div>
<p>Now we start training our model</p>
<div class="cell docutils container">
@@ -2443,6 +2948,14 @@ the house using the features (predictors) listed here.</p>
</pre></div>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>(404, 2)
(102, 2)
(404,)
(102,)
</pre></div>
</div>
</div>
</div>
<p>Then we use the linear regression functionality from <strong>Scikit-Learn</strong></p>
<div class="cell docutils container">
@@ -2481,6 +2994,20 @@ the house using the features (predictors) listed here.</p>
</pre></div>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>The model performance for training set
--------------------------------------
RMSE is 5.637129335071195
R2 score is 0.6300745149331701
The model performance for testing set
--------------------------------------
RMSE is 5.137400784702911
R2 score is 0.6628996975186953
</pre></div>
</div>
</div>
</div>
<div class="cell docutils container">
<div class="cell_input docutils container">
@@ -2491,6 +3018,9 @@ the house using the features (predictors) listed here.</p>
</pre></div>
</div>
</div>
<div class="cell_output docutils container">
<img alt="_images/week35_210_0.png" src="_images/week35_210_0.png" />
</div>
</div>
</div>
<div class="section" id="material-for-lecture-thursday-august-31">
@@ -2770,6 +3300,25 @@ In general the economy-size SVD leads to less FLOPS and still conserving the des
</pre></div>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[ 1. -1.]
[ 1. -1.]]
test U
[[0. 0.]
[0. 0.]]
test VT
[[0. 0.]
[0. 0.]]
[[-0.70710678 -0.70710678]
[-0.70710678 0.70710678]]
[2.00000000e+00 3.35470445e-17]
[[-0.70710678 0.70710678]
[ 0.70710678 0.70710678]]
[[-3.33066907e-16 4.44089210e-16]
[ 0.00000000e+00 2.22044605e-16]]
</pre></div>
</div>
</div>
</div>
<p>The matrix <span class="math notranslate nohighlight">\(\boldsymbol{X}\)</span> has columns that are linearly dependent. The first
column is the row-wise sum of the other two columns. The rank of a
@@ -3123,6 +3672,14 @@ covariance matrix through the <strong>np.linalg.eig()</strong> function.</p>
</pre></div>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.10790125813226321
4.340071371496255
[[ 1.04193203 3.08165104]
[ 3.08165104 10.18383522]]
</pre></div>
</div>
</div>
</div>
</div>
<div class="section" id="correlation-matrix">
@@ -3158,6 +3715,14 @@ a more brute force way. Here we scale the mean values for each column of the des
</pre></div>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.08881497884574564
1.7086067479626619
[[1. 0.66080313]
[0.66080313 1. ]]
</pre></div>
</div>
</div>
</div>
<p>We see that the matrix elements along the diagonal are one as they
should be and that the matrix is symmetric. Furthermore, diagonalizing
@@ -3186,6 +3751,34 @@ this matrix we easily see that it is a positive definite matrix.</p>
</pre></div>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[-0.40620066 -2.01265755]
[ 0.01458611 0.37737221]
[-1.0895387 -3.65442354]
[ 0.2338675 1.12044974]
[ 0.4676059 1.54393936]
[-0.65891389 -3.16304863]
[-0.1715252 0.39197698]
[ 0.71142161 2.95511792]
[ 0.39214397 0.13069442]
[ 0.50655336 2.3105791 ]]
0 1
0 -0.406201 -2.012658
1 0.014586 0.377372
2 -1.089539 -3.654424
3 0.233868 1.120450
4 0.467606 1.543939
5 -0.658914 -3.163049
6 -0.171525 0.391977
7 0.711422 2.955118
8 0.392144 0.130694
9 0.506553 2.310579
0 1
0 1.000000 0.952387
1 0.952387 1.000000
</pre></div>
</div>
</div>
</div>
<p>We expand this model to the Franke function discussed above.</p>
</div>
@@ -3239,6 +3832,43 @@ this matrix we easily see that it is a positive definite matrix.</p>
</pre></div>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 0 1 2 3 4 5 6 7 \
0 0.0 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
1 0.0 0.078974 0.081276 0.075889 0.077168 0.078540 0.065410 0.066474
2 0.0 0.081276 0.084076 0.078336 0.079946 0.081655 0.067707 0.069028
3 0.0 0.075889 0.078336 0.077460 0.078986 0.080616 0.069452 0.070737
4 0.0 0.077168 0.079946 0.078986 0.080764 0.082653 0.070986 0.072476
5 0.0 0.078540 0.081655 0.080616 0.082653 0.084809 0.072621 0.074323
6 0.0 0.065410 0.067707 0.069452 0.070986 0.072621 0.064074 0.065378
7 0.0 0.066474 0.069028 0.070737 0.072476 0.074323 0.065378 0.066854
8 0.0 0.067637 0.070457 0.072132 0.074084 0.076150 0.066787 0.068441
9 0.0 0.068906 0.072000 0.073644 0.075816 0.078110 0.068307 0.070146
10 0.0 0.055734 0.057835 0.060872 0.062337 0.063894 0.057393 0.058645
11 0.0 0.056683 0.058996 0.062016 0.063653 0.065390 0.058552 0.059951
12 0.0 0.057722 0.060254 0.063260 0.065077 0.066999 0.059807 0.061359
13 0.0 0.058854 0.061614 0.064609 0.066612 0.068727 0.061163 0.062874
14 0.0 0.060083 0.063080 0.066066 0.068264 0.070582 0.062624 0.064501
8 9 10 11 12 13 14
0 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
1 0.067637 0.068906 0.055734 0.056683 0.057722 0.058854 0.060083
2 0.070457 0.072000 0.057835 0.058996 0.060254 0.061614 0.063080
3 0.072132 0.073644 0.060872 0.062016 0.063260 0.064609 0.066066
4 0.074084 0.075816 0.062337 0.063653 0.065077 0.066612 0.068264
5 0.076150 0.078110 0.063894 0.065390 0.066999 0.068727 0.070582
6 0.066787 0.068307 0.057393 0.058552 0.059807 0.061163 0.062624
7 0.068441 0.070146 0.058645 0.059951 0.061359 0.062874 0.064501
8 0.070213 0.072111 0.059993 0.061452 0.063019 0.064699 0.066500
9 0.072111 0.074210 0.061443 0.063061 0.064793 0.066647 0.068629
10 0.059993 0.061443 0.052305 0.053417 0.054617 0.055910 0.057300
11 0.061452 0.063061 0.053417 0.054655 0.055987 0.057418 0.058952
12 0.063019 0.064793 0.054617 0.055987 0.057457 0.059031 0.060716
13 0.064699 0.066647 0.055910 0.057418 0.059031 0.060756 0.062599
14 0.066500 0.068629 0.057300 0.058952 0.060716 0.062599 0.064606
</pre></div>
</div>
</div>
</div>
<p>We note here that the covariance is zero for the first rows and
columns since all matrix elements in the design matrix were set to one
doc/LectureNotes/_build/jupyter_execute/week35.ipynb
+1304 -467
View File
File diff suppressed because one or more lines are too long
doc/LectureNotes/_build/jupyter_execute/week35.py
+7 -6
View File
@@ -534,7 +534,7 @@ ytilde = X @ beta
# In[2]:
fit = np.linalg.lstsq(X, Energies, rcond =None)[0]
fit = np.linalg.lstsq(X, y, rcond =None)[0]
ytildenp = np.dot(fit,X.T)
@@ -555,7 +555,7 @@ def R2(y_data, y_model):
# In[4]:
print(R2(Energies,ytilde))
print(R2(y,ytilde))
# We can easily add our **MSE** score as
@@ -567,7 +567,7 @@ def MSE(y_data,y_model):
n = np.size(y_model)
return np.sum((y_data-y_model)**2)/n
print(MSE(Energies,ytilde))
print(MSE(y,ytilde))
# and finally the relative error as
@@ -577,7 +577,7 @@ print(MSE(Energies,ytilde))
def RelativeError(y_data,y_model):
return abs((y_data-y_model)/y_data)
print(RelativeError(Energies, ytilde))
print(RelativeError(y, ytilde))
# ## Splitting our Data in Training and Test data
@@ -1011,6 +1011,7 @@ print("R2 score for scaled data: {:.2f}".format(clf.score(X_test_scaled,y_test)
#
# Keep in mind that when you transform your data set before training a model, the same transformation needs to be done
# on your eventual new data set before making a prediction. If we translate this into a Python code, it would could be implemented as follows
# (note that the lines are commented since the model function has not been defined)
# In[13]:
@@ -1022,12 +1023,12 @@ X_train = X_train - X_train_mean
y_train = y_train - y_train_mean
# The we fit our model with the training data
trained_model = some_model.fit(X_train,y_train)
#trained_model = some_model.fit(X_train,y_train)
#Model prediction, we need also to transform our data set used for the prediction.
X_test = X_test - X_train_mean #Use mean from training data
y_pred = trained_model(X_test)
#y_pred = trained_model(X_test)
y_pred = y_pred + y_train_mean
doc/LectureNotes/_build/jupyter_execute/week35_146_0.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 23 KiB

doc/LectureNotes/_build/jupyter_execute/week35_181_1.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 27 KiB

doc/LectureNotes/_build/jupyter_execute/week35_199_1.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 40 KiB

doc/LectureNotes/_build/jupyter_execute/week35_201_1.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 165 KiB

doc/LectureNotes/_build/jupyter_execute/week35_203_0.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 58 KiB

doc/LectureNotes/_build/jupyter_execute/week35_210_0.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 19 KiB

doc/LectureNotes/week35.ipynb
+434 -433
View File
File diff suppressed because it is too large Load Diff
doc/pub/week35/html/._week35-bs018.html
+1 -1
View File
@@ -416,7 +416,7 @@ ytilde <span style="color: #666666">=</span> X <span style="color: #666666">@</s
<div class="inner_cell">
<div class="input_area">
<div class="highlight" style="background: #f8f8f8">
<pre style="line-height: 125%;">fit <span style="color: #666666">=</span> np<span style="color: #666666">.</span>linalg<span style="color: #666666">.</span>lstsq(X, Energies, rcond <span style="color: #666666">=</span><span style="color: #008000; font-weight: bold">None</span>)[<span style="color: #666666">0</span>]
<pre style="line-height: 125%;">fit <span style="color: #666666">=</span> np<span style="color: #666666">.</span>linalg<span style="color: #666666">.</span>lstsq(X, y, rcond <span style="color: #666666">=</span><span style="color: #008000; font-weight: bold">None</span>)[<span style="color: #666666">0</span>]
ytildenp <span style="color: #666666">=</span> np<span style="color: #666666">.</span>dot(fit,X<span style="color: #666666">.</span>T)
</pre>
</div>
doc/pub/week35/html/._week35-bs019.html
+3 -3
View File
@@ -404,7 +404,7 @@ Since we are not using <b>Scikit-Learn</b> here we can define our own \( R2 \) f
<div class="inner_cell">
<div class="input_area">
<div class="highlight" style="background: #f8f8f8">
<pre style="line-height: 125%;"><span style="color: #008000">print</span>(R2(Energies,ytilde))
<pre style="line-height: 125%;"><span style="color: #008000">print</span>(R2(y,ytilde))
</pre>
</div>
</div>
@@ -432,7 +432,7 @@ Since we are not using <b>Scikit-Learn</b> here we can define our own \( R2 \) f
n <span style="color: #666666">=</span> np<span style="color: #666666">.</span>size(y_model)
<span style="color: #008000; font-weight: bold">return</span> np<span style="color: #666666">.</span>sum((y_data<span style="color: #666666">-</span>y_model)<span style="color: #666666">**2</span>)<span style="color: #666666">/</span>n
<span style="color: #008000">print</span>(MSE(Energies,ytilde))
<span style="color: #008000">print</span>(MSE(y,ytilde))
</pre>
</div>
</div>
@@ -458,7 +458,7 @@ Since we are not using <b>Scikit-Learn</b> here we can define our own \( R2 \) f
<div class="highlight" style="background: #f8f8f8">
<pre style="line-height: 125%;"><span style="color: #008000; font-weight: bold">def</span> <span style="color: #0000FF">RelativeError</span>(y_data,y_model):
<span style="color: #008000; font-weight: bold">return</span> <span style="color: #008000">abs</span>((y_data<span style="color: #666666">-</span>y_model)<span style="color: #666666">/</span>y_data)
<span style="color: #008000">print</span>(RelativeError(Energies, ytilde))
<span style="color: #008000">print</span>(RelativeError(y, ytilde))
</pre>
</div>
</div>
doc/pub/week35/html/._week35-bs034.html
+3 -3
View File
@@ -371,9 +371,9 @@ MathJax.Hub.Config({
<p>Keep in mind that when you transform your data set before training a model, the same transformation needs to be done
on your eventual new data set before making a prediction. If we translate this into a Python code, it would could be implemented as follows
(note that the lines are commented since the model function has not been defined)
</p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
@@ -387,12 +387,12 @@ X_train <span style="color: #666666">=</span> X_train <span style="color: #66666
y_train <span style="color: #666666">=</span> y_train <span style="color: #666666">-</span> y_train_mean
<span style="color: #408080; font-style: italic"># The we fit our model with the training data</span>
trained_model <span style="color: #666666">=</span> some_model<span style="color: #666666">.</span>fit(X_train,y_train)
<span style="color: #408080; font-style: italic">#trained_model = some_model.fit(X_train,y_train)</span>
<span style="color: #408080; font-style: italic">#Model prediction, we need also to transform our data set used for the prediction.</span>
X_test <span style="color: #666666">=</span> X_test <span style="color: #666666">-</span> X_train_mean <span style="color: #408080; font-style: italic">#Use mean from training data</span>
y_pred <span style="color: #666666">=</span> trained_model(X_test)
<span style="color: #408080; font-style: italic">#y_pred = trained_model(X_test)</span>
y_pred <span style="color: #666666">=</span> y_pred <span style="color: #666666">+</span> y_train_mean
</pre>
</div>
doc/pub/week35/html/week35-reveal.html
+7 -7
View File
@@ -866,7 +866,7 @@ ytilde = X @ beta
<div class="inner_cell">
<div class="input_area">
<div class="highlight" style="background: #eeeedd">
<pre style="font-size: 80%; line-height: 125%;">fit = np.linalg.lstsq(X, Energies, rcond =<span style="color: #8B008B; font-weight: bold">None</span>)[<span style="color: #B452CD">0</span>]
<pre style="font-size: 80%; line-height: 125%;">fit = np.linalg.lstsq(X, y, rcond =<span style="color: #8B008B; font-weight: bold">None</span>)[<span style="color: #B452CD">0</span>]
ytildenp = np.dot(fit,X.T)
</pre>
</div>
@@ -922,7 +922,7 @@ Since we are not using <b>Scikit-Learn</b> here we can define our own \( R2 \) f
<div class="inner_cell">
<div class="input_area">
<div class="highlight" style="background: #eeeedd">
<pre style="font-size: 80%; line-height: 125%;"><span style="color: #658b00">print</span>(R2(Energies,ytilde))
<pre style="font-size: 80%; line-height: 125%;"><span style="color: #658b00">print</span>(R2(y,ytilde))
</pre>
</div>
</div>
@@ -950,7 +950,7 @@ Since we are not using <b>Scikit-Learn</b> here we can define our own \( R2 \) f
n = np.size(y_model)
<span style="color: #8B008B; font-weight: bold">return</span> np.sum((y_data-y_model)**<span style="color: #B452CD">2</span>)/n
<span style="color: #658b00">print</span>(MSE(Energies,ytilde))
<span style="color: #658b00">print</span>(MSE(y,ytilde))
</pre>
</div>
</div>
@@ -976,7 +976,7 @@ Since we are not using <b>Scikit-Learn</b> here we can define our own \( R2 \) f
<div class="highlight" style="background: #eeeedd">
<pre style="font-size: 80%; line-height: 125%;"><span style="color: #8B008B; font-weight: bold">def</span> <span style="color: #008b45">RelativeError</span>(y_data,y_model):
<span style="color: #8B008B; font-weight: bold">return</span> <span style="color: #658b00">abs</span>((y_data-y_model)/y_data)
<span style="color: #658b00">print</span>(RelativeError(Energies, ytilde))
<span style="color: #658b00">print</span>(RelativeError(y, ytilde))
</pre>
</div>
</div>
@@ -1594,9 +1594,9 @@ This can clearly lead to problems in evaluating the cost/loss functions.
<p>Keep in mind that when you transform your data set before training a model, the same transformation needs to be done
on your eventual new data set before making a prediction. If we translate this into a Python code, it would could be implemented as follows
(note that the lines are commented since the model function has not been defined)
</p>
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
@@ -1610,12 +1610,12 @@ X_train = X_train - X_train_mean
y_train = y_train - y_train_mean
<span style="color: #228B22"># The we fit our model with the training data</span>
trained_model = some_model.fit(X_train,y_train)
<span style="color: #228B22">#trained_model = some_model.fit(X_train,y_train)</span>
<span style="color: #228B22">#Model prediction, we need also to transform our data set used for the prediction.</span>
X_test = X_test - X_train_mean <span style="color: #228B22">#Use mean from training data</span>
y_pred = trained_model(X_test)
<span style="color: #228B22">#y_pred = trained_model(X_test)</span>
y_pred = y_pred + y_train_mean
</pre>
</div>
doc/pub/week35/html/week35-solarized.html
+7 -7
View File
@@ -876,7 +876,7 @@ ytilde = X @ beta
<div class="inner_cell">
<div class="input_area">
<div class="highlight" style="background: #eeeedd">
<pre style="line-height: 125%;">fit = np.linalg.lstsq(X, Energies, rcond =<span style="color: #8B008B; font-weight: bold">None</span>)[<span style="color: #B452CD">0</span>]
<pre style="line-height: 125%;">fit = np.linalg.lstsq(X, y, rcond =<span style="color: #8B008B; font-weight: bold">None</span>)[<span style="color: #B452CD">0</span>]
ytildenp = np.dot(fit,X.T)
</pre>
</div>
@@ -932,7 +932,7 @@ Since we are not using <b>Scikit-Learn</b> here we can define our own \( R2 \) f
<div class="inner_cell">
<div class="input_area">
<div class="highlight" style="background: #eeeedd">
<pre style="line-height: 125%;"><span style="color: #658b00">print</span>(R2(Energies,ytilde))
<pre style="line-height: 125%;"><span style="color: #658b00">print</span>(R2(y,ytilde))
</pre>
</div>
</div>
@@ -960,7 +960,7 @@ Since we are not using <b>Scikit-Learn</b> here we can define our own \( R2 \) f
n = np.size(y_model)
<span style="color: #8B008B; font-weight: bold">return</span> np.sum((y_data-y_model)**<span style="color: #B452CD">2</span>)/n
<span style="color: #658b00">print</span>(MSE(Energies,ytilde))
<span style="color: #658b00">print</span>(MSE(y,ytilde))
</pre>
</div>
</div>
@@ -986,7 +986,7 @@ Since we are not using <b>Scikit-Learn</b> here we can define our own \( R2 \) f
<div class="highlight" style="background: #eeeedd">
<pre style="line-height: 125%;"><span style="color: #8B008B; font-weight: bold">def</span> <span style="color: #008b45">RelativeError</span>(y_data,y_model):
<span style="color: #8B008B; font-weight: bold">return</span> <span style="color: #658b00">abs</span>((y_data-y_model)/y_data)
<span style="color: #658b00">print</span>(RelativeError(Energies, ytilde))
<span style="color: #658b00">print</span>(RelativeError(y, ytilde))
</pre>
</div>
</div>
@@ -1593,9 +1593,9 @@ This can clearly lead to problems in evaluating the cost/loss functions.
<p>Keep in mind that when you transform your data set before training a model, the same transformation needs to be done
on your eventual new data set before making a prediction. If we translate this into a Python code, it would could be implemented as follows
(note that the lines are commented since the model function has not been defined)
</p>
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
@@ -1609,12 +1609,12 @@ X_train = X_train - X_train_mean
y_train = y_train - y_train_mean
<span style="color: #228B22"># The we fit our model with the training data</span>
trained_model = some_model.fit(X_train,y_train)
<span style="color: #228B22">#trained_model = some_model.fit(X_train,y_train)</span>
<span style="color: #228B22">#Model prediction, we need also to transform our data set used for the prediction.</span>
X_test = X_test - X_train_mean <span style="color: #228B22">#Use mean from training data</span>
y_pred = trained_model(X_test)
<span style="color: #228B22">#y_pred = trained_model(X_test)</span>
y_pred = y_pred + y_train_mean
</pre>
</div>
doc/pub/week35/html/week35.html
+7 -7
View File
@@ -953,7 +953,7 @@ ytilde <span style="color: #666666">=</span> X <span style="color: #666666">@</s
<div class="inner_cell">
<div class="input_area">
<div class="highlight" style="background: #f8f8f8">
<pre style="line-height: 125%;">fit <span style="color: #666666">=</span> np<span style="color: #666666">.</span>linalg<span style="color: #666666">.</span>lstsq(X, Energies, rcond <span style="color: #666666">=</span><span style="color: #008000; font-weight: bold">None</span>)[<span style="color: #666666">0</span>]
<pre style="line-height: 125%;">fit <span style="color: #666666">=</span> np<span style="color: #666666">.</span>linalg<span style="color: #666666">.</span>lstsq(X, y, rcond <span style="color: #666666">=</span><span style="color: #008000; font-weight: bold">None</span>)[<span style="color: #666666">0</span>]
ytildenp <span style="color: #666666">=</span> np<span style="color: #666666">.</span>dot(fit,X<span style="color: #666666">.</span>T)
</pre>
</div>
@@ -1009,7 +1009,7 @@ Since we are not using <b>Scikit-Learn</b> here we can define our own \( R2 \) f
<div class="inner_cell">
<div class="input_area">
<div class="highlight" style="background: #f8f8f8">
<pre style="line-height: 125%;"><span style="color: #008000">print</span>(R2(Energies,ytilde))
<pre style="line-height: 125%;"><span style="color: #008000">print</span>(R2(y,ytilde))
</pre>
</div>
</div>
@@ -1037,7 +1037,7 @@ Since we are not using <b>Scikit-Learn</b> here we can define our own \( R2 \) f
n <span style="color: #666666">=</span> np<span style="color: #666666">.</span>size(y_model)
<span style="color: #008000; font-weight: bold">return</span> np<span style="color: #666666">.</span>sum((y_data<span style="color: #666666">-</span>y_model)<span style="color: #666666">**2</span>)<span style="color: #666666">/</span>n
<span style="color: #008000">print</span>(MSE(Energies,ytilde))
<span style="color: #008000">print</span>(MSE(y,ytilde))
</pre>
</div>
</div>
@@ -1063,7 +1063,7 @@ Since we are not using <b>Scikit-Learn</b> here we can define our own \( R2 \) f
<div class="highlight" style="background: #f8f8f8">
<pre style="line-height: 125%;"><span style="color: #008000; font-weight: bold">def</span> <span style="color: #0000FF">RelativeError</span>(y_data,y_model):
<span style="color: #008000; font-weight: bold">return</span> <span style="color: #008000">abs</span>((y_data<span style="color: #666666">-</span>y_model)<span style="color: #666666">/</span>y_data)
<span style="color: #008000">print</span>(RelativeError(Energies, ytilde))
<span style="color: #008000">print</span>(RelativeError(y, ytilde))
</pre>
</div>
</div>
@@ -1670,9 +1670,9 @@ This can clearly lead to problems in evaluating the cost/loss functions.
<p>Keep in mind that when you transform your data set before training a model, the same transformation needs to be done
on your eventual new data set before making a prediction. If we translate this into a Python code, it would could be implemented as follows
(note that the lines are commented since the model function has not been defined)
</p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
@@ -1686,12 +1686,12 @@ X_train <span style="color: #666666">=</span> X_train <span style="color: #66666
y_train <span style="color: #666666">=</span> y_train <span style="color: #666666">-</span> y_train_mean
<span style="color: #408080; font-style: italic"># The we fit our model with the training data</span>
trained_model <span style="color: #666666">=</span> some_model<span style="color: #666666">.</span>fit(X_train,y_train)
<span style="color: #408080; font-style: italic">#trained_model = some_model.fit(X_train,y_train)</span>
<span style="color: #408080; font-style: italic">#Model prediction, we need also to transform our data set used for the prediction.</span>
X_test <span style="color: #666666">=</span> X_test <span style="color: #666666">-</span> X_train_mean <span style="color: #408080; font-style: italic">#Use mean from training data</span>
y_pred <span style="color: #666666">=</span> trained_model(X_test)
<span style="color: #408080; font-style: italic">#y_pred = trained_model(X_test)</span>
y_pred <span style="color: #666666">=</span> y_pred <span style="color: #666666">+</span> y_train_mean
</pre>
</div>
doc/pub/week35/ipynb/ipynb-week35-src.tar.gz
BIN
View File
Binary file not shown.
doc/pub/week35/ipynb/week35.ipynb
+434 -433
View File
File diff suppressed because it is too large Load Diff
doc/src/week35/week35.do.txt
+7 -7
View File
@@ -601,7 +601,7 @@ ytilde = X @ beta
!ec
Alternatively, you can use the least squares functionality in _Numpy_ as
!bc pycod
fit = np.linalg.lstsq(X, Energies, rcond =None)[0]
fit = np.linalg.lstsq(X, y, rcond =None)[0]
ytildenp = np.dot(fit,X.T)
!ec
@@ -617,7 +617,7 @@ def R2(y_data, y_model):
!ec
and we would be using it as
!bc pycod
print(R2(Energies,ytilde))
print(R2(y,ytilde))
!ec
We can easily add our _MSE_ score as
@@ -626,13 +626,13 @@ def MSE(y_data,y_model):
n = np.size(y_model)
return np.sum((y_data-y_model)**2)/n
print(MSE(Energies,ytilde))
print(MSE(y,ytilde))
!ec
and finally the relative error as
!bc pycod
def RelativeError(y_data,y_model):
return abs((y_data-y_model)/y_data)
print(RelativeError(Energies, ytilde))
print(RelativeError(y, ytilde))
!ec
@@ -1105,7 +1105,7 @@ This can clearly lead to problems in evaluating the cost/loss functions.
Keep in mind that when you transform your data set before training a model, the same transformation needs to be done
on your eventual new data set before making a prediction. If we translate this into a Python code, it would could be implemented as follows
(note that the lines are commented since the model function has not been defined)
!bc pycod
#Model training, we compute the mean value of y and X
y_train_mean = np.mean(y_train)
@@ -1114,12 +1114,12 @@ X_train = X_train - X_train_mean
y_train = y_train - y_train_mean
# The we fit our model with the training data
trained_model = some_model.fit(X_train,y_train)
#trained_model = some_model.fit(X_train,y_train)
#Model prediction, we need also to transform our data set used for the prediction.
X_test = X_test - X_train_mean #Use mean from training data
y_pred = trained_model(X_test)
#y_pred = trained_model(X_test)
y_pred = y_pred + y_train_mean
!ec