small update
This commit is contained in:
Morten Hjorth-Jensen
@@ -10,13 +10,13 @@ edge [fontname="helvetica"] ;
|
||||
2 -> 3 ;
|
||||
4 [label="gini = 0.0\nsamples = 239\nvalue = [[239, 0]\n[0, 239]]", fillcolor="#e58139"] ;
|
||||
3 -> 4 ;
|
||||
5 [label="worst compactness <= 0.085\ngini = 0.444\nsamples = 3\nvalue = [[2, 1]\n[1, 2]]", fillcolor="#fdf6f0"] ;
|
||||
5 [label="mean radius <= 12.265\ngini = 0.444\nsamples = 3\nvalue = [[2, 1]\n[1, 2]]", fillcolor="#fdf6f0"] ;
|
||||
3 -> 5 ;
|
||||
6 [label="gini = 0.0\nsamples = 1\nvalue = [[0, 1]\n[1, 0]]", fillcolor="#e58139"] ;
|
||||
5 -> 6 ;
|
||||
7 [label="gini = 0.0\nsamples = 2\nvalue = [[2, 0]\n[0, 2]]", fillcolor="#e58139"] ;
|
||||
5 -> 7 ;
|
||||
8 [label="worst texture <= 29.455\ngini = 0.397\nsamples = 11\nvalue = [[8, 3]\n[3, 8]]", fillcolor="#fae9dd"] ;
|
||||
8 [label="mean texture <= 20.84\ngini = 0.397\nsamples = 11\nvalue = [[8, 3]\n[3, 8]]", fillcolor="#fae9dd"] ;
|
||||
2 -> 8 ;
|
||||
9 [label="gini = 0.0\nsamples = 8\nvalue = [[8, 0]\n[0, 8]]", fillcolor="#e58139"] ;
|
||||
8 -> 9 ;
|
||||
@@ -30,11 +30,11 @@ edge [fontname="helvetica"] ;
|
||||
11 -> 13 ;
|
||||
14 [label="worst texture <= 20.645\ngini = 0.202\nsamples = 167\nvalue = [[19, 148]\n[148, 19]]", fillcolor="#f0b68c"] ;
|
||||
0 -> 14 [labeldistance=2.5, labelangle=-45, headlabel="False"] ;
|
||||
15 [label="worst radius <= 17.74\ngini = 0.375\nsamples = 16\nvalue = [[12, 4]\n[4, 12]]", fillcolor="#f9e3d4"] ;
|
||||
15 [label="worst area <= 964.4\ngini = 0.375\nsamples = 16\nvalue = [[12, 4]\n[4, 12]]", fillcolor="#f9e3d4"] ;
|
||||
14 -> 15 ;
|
||||
16 [label="gini = 0.0\nsamples = 11\nvalue = [[11, 0]\n[0, 11]]", fillcolor="#e58139"] ;
|
||||
15 -> 16 ;
|
||||
17 [label="worst smoothness <= 0.106\ngini = 0.32\nsamples = 5\nvalue = [[1, 4]\n[4, 1]]", fillcolor="#f6d5bd"] ;
|
||||
17 [label="concavity error <= 0.016\ngini = 0.32\nsamples = 5\nvalue = [[1, 4]\n[4, 1]]", fillcolor="#f6d5bd"] ;
|
||||
15 -> 17 ;
|
||||
18 [label="gini = 0.0\nsamples = 1\nvalue = [[1, 0]\n[0, 1]]", fillcolor="#e58139"] ;
|
||||
17 -> 18 ;
|
||||
@@ -42,7 +42,7 @@ edge [fontname="helvetica"] ;
|
||||
17 -> 19 ;
|
||||
20 [label="mean concave points <= 0.049\ngini = 0.088\nsamples = 151\nvalue = [[7, 144]\n[144, 7]]", fillcolor="#ea985d"] ;
|
||||
14 -> 20 ;
|
||||
21 [label="compactness error <= 0.016\ngini = 0.48\nsamples = 15\nvalue = [[6, 9]\n[9, 6]]", fillcolor="#ffffff"] ;
|
||||
21 [label="concave points error <= 0.01\ngini = 0.48\nsamples = 15\nvalue = [[6, 9]\n[9, 6]]", fillcolor="#ffffff"] ;
|
||||
20 -> 21 ;
|
||||
22 [label="gini = 0.0\nsamples = 9\nvalue = [[0, 9]\n[9, 0]]", fillcolor="#e58139"] ;
|
||||
21 -> 22 ;
|
||||
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 244 KiB After Width: | Height: | Size: 241 KiB |
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
|
Before Width: | Height: | Size: 14 KiB After Width: | Height: | Size: 13 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 18 KiB After Width: | Height: | Size: 18 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 26 KiB After Width: | Height: | Size: 25 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 18 KiB After Width: | Height: | Size: 17 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 22 KiB After Width: | Height: | Size: 22 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 52 KiB After Width: | Height: | Size: 46 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 37 KiB After Width: | Height: | Size: 39 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 36 KiB After Width: | Height: | Size: 40 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 23 KiB After Width: | Height: | Size: 23 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 21 KiB After Width: | Height: | Size: 22 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 22 KiB After Width: | Height: | Size: 21 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 10 KiB After Width: | Height: | Size: 9.9 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 18 KiB After Width: | Height: | Size: 19 KiB |
@@ -0,0 +1,393 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "50b2fbbe",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)\n",
|
||||
"doconce format html exercisesweek36.do.txt -->\n",
|
||||
"<!-- dom:TITLE: Exercises week 36 -->"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "da488c0c",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"# Exercises week 36\n",
|
||||
"**September 4-8, 2023**\n",
|
||||
"\n",
|
||||
"Date: **Deadline is Sunday September 10 at midnight**"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "b84d6b13",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"## Overarching aims of the exercises this week\n",
|
||||
"\n",
|
||||
"This set of exercises form an important part of the first project. The\n",
|
||||
"analytical exercises deal with the material covered last week on the\n",
|
||||
"mathematical interpretations of ordinary least squares and of Ridge\n",
|
||||
"regression. The numerical exercises can be seen as a continuation of\n",
|
||||
"exercise 3 from week 35, with the inclusion of Ridge regression. This\n",
|
||||
"material enters also the discussions of the first project."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "96c9c28e",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"## Exercise 1: Analytical exercises\n",
|
||||
"\n",
|
||||
"The aim here is to derive the expression for the optimal parameters\n",
|
||||
"using Ridge regression. Furthermore, using the singular value\n",
|
||||
"decomposition, we will analyze the difference between the ordinary\n",
|
||||
"least squares approach and Ridge regression.\n",
|
||||
"\n",
|
||||
"The expression for the standard Mean Squared Error (MSE) which we used to define our cost function and the equations for the ordinary least squares (OLS) method, was given by the\n",
|
||||
"optimization problem"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "439f1456",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"{\\displaystyle \\min_{\\boldsymbol{\\beta}\\in {\\mathbb{R}}^{p}}}\\frac{1}{n}\\left\\{\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\right)^T\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\right)\\right\\}.\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "b51e09f7",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"which we can also write as"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "02c45981",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"{\\displaystyle \\min_{\\boldsymbol{\\beta}\\in\n",
|
||||
"{\\mathbb{R}}^{p}}}\\frac{1}{n}\\sum_{i=0}^{n-1}\\left(y_i-\\tilde{y}_i\\right)^2=\\frac{1}{n}\\vert\\vert \\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\vert\\vert_2^2,\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "cc0e91ea",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"where we have used the definition of a norm-2 vector, that is"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "b5805f35",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"\\vert\\vert \\boldsymbol{x}\\vert\\vert_2 = \\sqrt{\\sum_i x_i^2}.\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "6e3095bf",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"By minimizing the above equation with respect to the parameters\n",
|
||||
"$\\boldsymbol{\\beta}$ we could then obtain an analytical expression for the\n",
|
||||
"parameters $\\boldsymbol{\\beta}$.\n",
|
||||
"\n",
|
||||
"We can add a regularization parameter $\\lambda$ by\n",
|
||||
"defining a new cost function to be optimized, that is"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "da90fe04",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"{\\displaystyle \\min_{\\boldsymbol{\\beta}\\in\n",
|
||||
"{\\mathbb{R}}^{p}}}\\frac{1}{n}\\vert\\vert \\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\vert\\vert_2^2+\\lambda\\vert\\vert \\boldsymbol{\\beta}\\vert\\vert_2^2\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "1a106e07",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"which leads to the Ridge regression minimization problem. One can require as part of the optimization problem \n",
|
||||
"that $\\vert\\vert \\boldsymbol{\\beta}\\vert\\vert_2^2\\le t$, where $t$ is\n",
|
||||
"a finite number larger than zero. We will not implement that here."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "3917877b",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"### a) Expression for Ridge regression\n",
|
||||
"\n",
|
||||
"Show that the optimal parameters"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "78226f28",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"\\hat{\\boldsymbol{\\beta}}_{\\mathrm{Ridge}} = \\left(\\boldsymbol{X}^T\\boldsymbol{X}+\\lambda\\boldsymbol{I}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y},\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "951dfffa",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"with $\\boldsymbol{I}$ being a $p\\times p$ identity matrix with the constraint that"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "21d2770e",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"\\sum_{i=0}^{p-1} \\beta_i^2 \\leq t,\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "ec212498",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"with $t$ a finite positive number. \n",
|
||||
"\n",
|
||||
"The ordinary least squares result is"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "4ffabf6c",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"\\hat{\\boldsymbol{\\beta}}_{\\mathrm{OLS}} = \\left(\\boldsymbol{X}^T\\boldsymbol{X}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y},\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "f97a6f45",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"### b) The singular value decomposition\n",
|
||||
"\n",
|
||||
"Use the singular value decomposition of an n\\times p$ matrix $\\boldsymbol{X}$ (our design matrix)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "8761ed23",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"\\boldsymbol{X}=\\boldsymbol{U}\\boldsymbol{\\Sigma}\\boldsymbol{V}^T,\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "92f8479e",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"where $\\boldsymbol{U}$ and $\\boldsymbol{V}$ are orthogonal matrices of dimensions\n",
|
||||
"$n\\times n$ and $p\\times p$, respectively, and $\\boldsymbol{\\Sigma}$ is an\n",
|
||||
"$n\\times p$ matrix which contains the ingular values only. This material was discussed during the lectures of week 35.\n",
|
||||
"\n",
|
||||
"Show that you can write the \n",
|
||||
"OLS solutions in terms of the eigenvectors (the columns) of the orthogonal matrix $\\boldsymbol{U}$ as"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "9df91bda",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"\\tilde{\\boldsymbol{y}}_{\\mathrm{OLS}}=\\boldsymbol{X}\\boldsymbol{\\beta} = \\sum_{j=0}^{p-1}\\boldsymbol{u}_j\\boldsymbol{u}_j^T\\boldsymbol{y}.\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "e6de0312",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"For Ridge regression, show that the corresponding equation is"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "8e09d132",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"\\tilde{\\boldsymbol{y}}_{\\mathrm{Ridge}}=\\boldsymbol{X}\\boldsymbol{\\beta}_{\\mathrm{Ridge}} = \\boldsymbol{U\\Sigma V^T}\\left(\\boldsymbol{V}\\boldsymbol{\\Sigma}^2\\boldsymbol{V}^T+\\lambda\\boldsymbol{I} \\right)^{-1}(\\boldsymbol{U\\Sigma V^T})^T\\boldsymbol{y}=\\sum_{j=0}^{p-1}\\boldsymbol{u}_j\\boldsymbol{u}_j^T\\frac{\\sigma_j^2}{\\sigma_j^2+\\lambda}\\boldsymbol{y},\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "a9c924ab",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"with the vectors $\\boldsymbol{u}_j$ being the columns of $\\boldsymbol{U}$ from the SVD of the matrix $\\boldsymbol{X}$. \n",
|
||||
"\n",
|
||||
"Give an interpretation of the results. [Section 3.4 of Hastie et al's textbook gives a good discussion of the above results](https://link.springer.com/book/10.1007/978-0-387-84858-7)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "3b9328a1",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"## Exercise 2: Adding Ridge Regression\n",
|
||||
"\n",
|
||||
"This exercise is a continuation of exercise 3 from week 35, see <https://compphysics.github.io/MachineLearning/doc/LectureNotes/_build/html/exercisesweek35.html>. We will use the same function to\n",
|
||||
"generate our data set, still staying with a simple function $y(x)$\n",
|
||||
"which we want to fit using linear regression, but now extending the\n",
|
||||
"analysis to include the Ridge regression method.\n",
|
||||
"\n",
|
||||
"In this exercise you need to include the same elements from last week, that is\n",
|
||||
"1. scale your data by subtracting the mean value from each column in the design matrix.\n",
|
||||
"\n",
|
||||
"2. perform a split of the data in a training set and a test set.\n",
|
||||
"\n",
|
||||
"The addition to the analysis this time is the introduction of the hyperparameter $\\lambda$ when introducing Ridge regression.\n",
|
||||
"\n",
|
||||
"Extend the code from exercise 3 from [week 35](https://compphysics.github.io/MachineLearning/doc/LectureNotes/_build/html/exercisesweek35.html) to include Ridge regression with the hyperparameter $\\lambda$. The optimal parameters $\\hat{\\beta}$ for Ridge regression can be obtained by matrix inversion in a similar way as done for ordinary least squares. You need to add to your code the following equations"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "5b54b7b4",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"\\hat{\\boldsymbol{\\beta}}_{\\mathrm{Ridge}} = \\left(\\boldsymbol{X}^T\\boldsymbol{X}+\\lambda\\boldsymbol{I}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y}.\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "a7ebe31b",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"The ordinary least squares result you encoded last week is given by"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "874e0dd3",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"\\hat{\\boldsymbol{\\beta}}_{\\mathrm{OLS}} = \\left(\\boldsymbol{X}^T\\boldsymbol{X}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y},\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "9adcc39f",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"Use these results to compute the mean squared error for ordinary least\n",
|
||||
"squares and Ridge regression first for a polynomial of degree five\n",
|
||||
"with $n=100$ data points and five selected values of\n",
|
||||
"$\\lambda=[0.0001,0.001, 0.01,0.1,1.0]$. Compute thereafter the mean\n",
|
||||
"squared error for the same values of $\\lambda$ for polynomials of degree ten\n",
|
||||
"and $15$. Discuss your results for the training MSE and test MSE with\n",
|
||||
"Ridge regression and ordinary least squares."
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -268,6 +268,16 @@ const thebe_selector_output = ".output, .cell_output"
|
||||
Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="exercisesweek36.html">
|
||||
Exercises week 36
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week36.html">
|
||||
Week 36: Statistical interpretation of Linear Regression and Resampling techniques
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
@@ -984,13 +994,13 @@ example of the functionality of <strong>Scikit-Learn</strong>.</p>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>The intercept alpha:
|
||||
[2.11366595]
|
||||
[2.05177695]
|
||||
Coefficient beta :
|
||||
[[4.95136998]]
|
||||
[[5.05971574]]
|
||||
Mean squared error: 0.28
|
||||
Variance score: 0.89
|
||||
Variance score: 0.88
|
||||
Mean squared log error: 0.01
|
||||
Mean absolute error: 0.41
|
||||
Mean absolute error: 0.45
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapter1_19_1.png" src="_images/chapter1_19_1.png" />
|
||||
@@ -1090,7 +1100,7 @@ a linear <span class="math notranslate nohighlight">\(x\)</span>-dependence we s
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<img alt="_images/chapter1_33_0.png" src="_images/chapter1_33_0.png" />
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.004999999999999996
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.004999999999999997
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@@ -268,6 +268,16 @@ const thebe_selector_output = ".output, .cell_output"
|
||||
Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="exercisesweek36.html">
|
||||
Exercises week 36
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week36.html">
|
||||
Week 36: Statistical interpretation of Linear Regression and Resampling techniques
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
@@ -1301,7 +1311,7 @@ the <em>Hadamard product</em>, meaning element-wise multiplication.</p>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Old accuracy on training data: 0.1440501043841336
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -1635,7 +1645,7 @@ Lambda = 10.0
|
||||
Accuracy score on test set: 0.19166666666666668
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -1644,7 +1654,7 @@ Lambda = 1e-05
|
||||
Accuracy score on test set: 0.10555555555555556
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -1653,7 +1663,7 @@ Lambda = 0.0001
|
||||
Accuracy score on test set: 0.08611111111111111
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -1662,7 +1672,7 @@ Lambda = 0.001
|
||||
Accuracy score on test set: 0.10555555555555556
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -1671,7 +1681,7 @@ Lambda = 0.01
|
||||
Accuracy score on test set: 0.08888888888888889
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -1680,7 +1690,7 @@ Lambda = 0.1
|
||||
Accuracy score on test set: 0.08611111111111111
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -1689,7 +1699,7 @@ Lambda = 1.0
|
||||
Accuracy score on test set: 0.08888888888888889
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -1698,11 +1708,11 @@ Lambda = 10.0
|
||||
Accuracy score on test set: 0.09166666666666666
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
exp_term = np.exp(self.z_o)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -1711,11 +1721,11 @@ Lambda = 1e-05
|
||||
Accuracy score on test set: 0.07777777777777778
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
exp_term = np.exp(self.z_o)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -1724,11 +1734,11 @@ Lambda = 0.0001
|
||||
Accuracy score on test set: 0.07777777777777778
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
exp_term = np.exp(self.z_o)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -1737,11 +1747,11 @@ Lambda = 0.001
|
||||
Accuracy score on test set: 0.07777777777777778
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
exp_term = np.exp(self.z_o)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -1750,11 +1760,11 @@ Lambda = 0.01
|
||||
Accuracy score on test set: 0.07777777777777778
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
exp_term = np.exp(self.z_o)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -1763,7 +1773,7 @@ Lambda = 0.1
|
||||
Accuracy score on test set: 0.07777777777777778
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -1772,11 +1782,11 @@ Lambda = 1.0
|
||||
Accuracy score on test set: 0.10555555555555556
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
exp_term = np.exp(self.z_o)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -1785,11 +1795,11 @@ Lambda = 10.0
|
||||
Accuracy score on test set: 0.07777777777777778
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
exp_term = np.exp(self.z_o)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -1798,11 +1808,11 @@ Lambda = 1e-05
|
||||
Accuracy score on test set: 0.07777777777777778
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
exp_term = np.exp(self.z_o)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -1811,11 +1821,11 @@ Lambda = 0.0001
|
||||
Accuracy score on test set: 0.07777777777777778
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
exp_term = np.exp(self.z_o)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -1824,11 +1834,11 @@ Lambda = 0.001
|
||||
Accuracy score on test set: 0.07777777777777778
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
exp_term = np.exp(self.z_o)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -1837,11 +1847,11 @@ Lambda = 0.01
|
||||
Accuracy score on test set: 0.07777777777777778
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
exp_term = np.exp(self.z_o)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -1850,11 +1860,11 @@ Lambda = 0.1
|
||||
Accuracy score on test set: 0.07777777777777778
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
exp_term = np.exp(self.z_o)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -1863,11 +1873,11 @@ Lambda = 1.0
|
||||
Accuracy score on test set: 0.07777777777777778
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:43: RuntimeWarning: overflow encountered in exp
|
||||
exp_term = np.exp(self.z_o)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide
|
||||
self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -1920,15 +1930,15 @@ Accuracy score on test set: 0.07777777777777778
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp
|
||||
return 1/(1 + np.exp(-x))
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -2187,27 +2197,26 @@ Accuracy score on test set: 0.9861111111111112
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.01
|
||||
Lambda = 0.1
|
||||
Accuracy score on test set: 0.9888888888888889
|
||||
|
||||
Learning rate = 0.01
|
||||
Lambda = 1.0
|
||||
Accuracy score on test set: 0.9722222222222222
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.01
|
||||
Lambda = 1.0
|
||||
Accuracy score on test set: 0.9722222222222222
|
||||
|
||||
Learning rate = 0.01
|
||||
Lambda = 10.0
|
||||
Accuracy score on test set: 0.9527777777777777
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.1
|
||||
Lambda = 1e-05
|
||||
Accuracy score on test set: 0.9027777777777778
|
||||
|
||||
Learning rate = 0.1
|
||||
Lambda = 0.0001
|
||||
Accuracy score on test set: 0.8583333333333333
|
||||
Lambda = 1e-05
|
||||
Accuracy score on test set: 0.9027777777777778
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.1
|
||||
Lambda = 0.0001
|
||||
Accuracy score on test set: 0.8583333333333333
|
||||
|
||||
Learning rate = 0.1
|
||||
Lambda = 0.001
|
||||
Accuracy score on test set: 0.8722222222222222
|
||||
</pre></div>
|
||||
@@ -2219,31 +2228,30 @@ Accuracy score on test set: 0.9055555555555556
|
||||
Learning rate = 0.1
|
||||
Lambda = 0.1
|
||||
Accuracy score on test set: 0.8805555555555555
|
||||
|
||||
Learning rate = 0.1
|
||||
Lambda = 1.0
|
||||
Accuracy score on test set: 0.8722222222222222
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 0.1
|
||||
Lambda = 1.0
|
||||
Accuracy score on test set: 0.8722222222222222
|
||||
|
||||
Learning rate = 0.1
|
||||
Lambda = 10.0
|
||||
Accuracy score on test set: 0.8666666666666667
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 1.0
|
||||
|
||||
Learning rate = 1.0
|
||||
Lambda = 1e-05
|
||||
Accuracy score on test set: 0.08611111111111111
|
||||
|
||||
Learning rate = 1.0
|
||||
Lambda = 0.0001
|
||||
Accuracy score on test set: 0.10555555555555556
|
||||
|
||||
Learning rate = 1.0
|
||||
Lambda = 0.001
|
||||
Accuracy score on test set: 0.10555555555555556
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 1.0
|
||||
Lambda = 0.001
|
||||
Accuracy score on test set: 0.10555555555555556
|
||||
|
||||
Learning rate = 1.0
|
||||
Lambda = 0.01
|
||||
Accuracy score on test set: 0.17777777777777778
|
||||
|
||||
@@ -2263,13 +2271,13 @@ Accuracy score on test set: 0.09444444444444444
|
||||
Learning rate = 10.0
|
||||
Lambda = 1e-05
|
||||
Accuracy score on test set: 0.17222222222222222
|
||||
|
||||
Learning rate = 10.0
|
||||
Lambda = 0.0001
|
||||
Accuracy score on test set: 0.11666666666666667
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Learning rate = 10.0
|
||||
Lambda = 0.0001
|
||||
Accuracy score on test set: 0.11666666666666667
|
||||
|
||||
Learning rate = 10.0
|
||||
Lambda = 0.001
|
||||
Accuracy score on test set: 0.10555555555555556
|
||||
|
||||
|
||||
@@ -268,6 +268,16 @@ const thebe_selector_output = ".output, .cell_output"
|
||||
Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="exercisesweek36.html">
|
||||
Exercises week 36
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week36.html">
|
||||
Week 36: Statistical interpretation of Linear Regression and Resampling techniques
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
@@ -2610,43 +2620,6 @@ Using TensorFlow results in a much better execution time. Try it!</p>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">19</span> <span class="n">x</span> <span class="o">=</span> <span class="nb">tuple</span><span class="p">(</span><span class="n">args</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">argnum</span><span class="p">)</span>
|
||||
<span class="ne">---> </span><span class="mi">20</span> <span class="k">return</span> <span class="n">unary_operator</span><span class="p">(</span><span class="n">unary_f</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="o">*</span><span class="n">nary_op_args</span><span class="p">,</span> <span class="o">**</span><span class="n">nary_op_kwargs</span><span class="p">)</span>
|
||||
|
||||
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/differential_operators.py:57,</span> in <span class="ni">jacobian</span><span class="nt">(fun, x)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">47</span> <span class="nd">@unary_to_nary</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">48</span> <span class="k">def</span> <span class="nf">jacobian</span><span class="p">(</span><span class="n">fun</span><span class="p">,</span> <span class="n">x</span><span class="p">):</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">49</span> <span class="sd">"""</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">50</span><span class="sd"> Returns a function which computes the Jacobian of `fun` with respect to</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">51</span><span class="sd"> positional argument number `argnum`, which must be a scalar or array. Unlike</span>
|
||||
<span class="sd"> (...)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">55</span><span class="sd"> (out1, out2, ...) then the Jacobian has shape (out1, out2, ..., in1, in2, ...).</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">56</span><span class="sd"> """</span>
|
||||
<span class="ne">---> </span><span class="mi">57</span> <span class="n">vjp</span><span class="p">,</span> <span class="n">ans</span> <span class="o">=</span> <span class="n">_make_vjp</span><span class="p">(</span><span class="n">fun</span><span class="p">,</span> <span class="n">x</span><span class="p">)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">58</span> <span class="n">ans_vspace</span> <span class="o">=</span> <span class="n">vspace</span><span class="p">(</span><span class="n">ans</span><span class="p">)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">59</span> <span class="n">jacobian_shape</span> <span class="o">=</span> <span class="n">ans_vspace</span><span class="o">.</span><span class="n">shape</span> <span class="o">+</span> <span class="n">vspace</span><span class="p">(</span><span class="n">x</span><span class="p">)</span><span class="o">.</span><span class="n">shape</span>
|
||||
|
||||
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/core.py:10,</span> in <span class="ni">make_vjp</span><span class="nt">(fun, x)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">8</span> <span class="k">def</span> <span class="nf">make_vjp</span><span class="p">(</span><span class="n">fun</span><span class="p">,</span> <span class="n">x</span><span class="p">):</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">9</span> <span class="n">start_node</span> <span class="o">=</span> <span class="n">VJPNode</span><span class="o">.</span><span class="n">new_root</span><span class="p">()</span>
|
||||
<span class="ne">---> </span><span class="mi">10</span> <span class="n">end_value</span><span class="p">,</span> <span class="n">end_node</span> <span class="o">=</span> <span class="n">trace</span><span class="p">(</span><span class="n">start_node</span><span class="p">,</span> <span class="n">fun</span><span class="p">,</span> <span class="n">x</span><span class="p">)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">11</span> <span class="k">if</span> <span class="n">end_node</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">12</span> <span class="k">def</span> <span class="nf">vjp</span><span class="p">(</span><span class="n">g</span><span class="p">):</span> <span class="k">return</span> <span class="n">vspace</span><span class="p">(</span><span class="n">x</span><span class="p">)</span><span class="o">.</span><span class="n">zeros</span><span class="p">()</span>
|
||||
|
||||
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/tracer.py:10,</span> in <span class="ni">trace</span><span class="nt">(start_node, fun, x)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">8</span> <span class="k">with</span> <span class="n">trace_stack</span><span class="o">.</span><span class="n">new_trace</span><span class="p">()</span> <span class="k">as</span> <span class="n">t</span><span class="p">:</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">9</span> <span class="n">start_box</span> <span class="o">=</span> <span class="n">new_box</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">t</span><span class="p">,</span> <span class="n">start_node</span><span class="p">)</span>
|
||||
<span class="ne">---> </span><span class="mi">10</span> <span class="n">end_box</span> <span class="o">=</span> <span class="n">fun</span><span class="p">(</span><span class="n">start_box</span><span class="p">)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">11</span> <span class="k">if</span> <span class="n">isbox</span><span class="p">(</span><span class="n">end_box</span><span class="p">)</span> <span class="ow">and</span> <span class="n">end_box</span><span class="o">.</span><span class="n">_trace</span> <span class="o">==</span> <span class="n">start_box</span><span class="o">.</span><span class="n">_trace</span><span class="p">:</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">12</span> <span class="k">return</span> <span class="n">end_box</span><span class="o">.</span><span class="n">_value</span><span class="p">,</span> <span class="n">end_box</span><span class="o">.</span><span class="n">_node</span>
|
||||
|
||||
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/wrap_util.py:15,</span> in <span class="ni">unary_to_nary.<locals>.nary_operator.<locals>.nary_f.<locals>.unary_f</span><span class="nt">(x)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">13</span> <span class="k">else</span><span class="p">:</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">14</span> <span class="n">subargs</span> <span class="o">=</span> <span class="n">subvals</span><span class="p">(</span><span class="n">args</span><span class="p">,</span> <span class="nb">zip</span><span class="p">(</span><span class="n">argnum</span><span class="p">,</span> <span class="n">x</span><span class="p">))</span>
|
||||
<span class="ne">---> </span><span class="mi">15</span> <span class="k">return</span> <span class="n">fun</span><span class="p">(</span><span class="o">*</span><span class="n">subargs</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
|
||||
|
||||
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/wrap_util.py:20,</span> in <span class="ni">unary_to_nary.<locals>.nary_operator.<locals>.nary_f</span><span class="nt">(*args, **kwargs)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">18</span> <span class="k">else</span><span class="p">:</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">19</span> <span class="n">x</span> <span class="o">=</span> <span class="nb">tuple</span><span class="p">(</span><span class="n">args</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">argnum</span><span class="p">)</span>
|
||||
<span class="ne">---> </span><span class="mi">20</span> <span class="k">return</span> <span class="n">unary_operator</span><span class="p">(</span><span class="n">unary_f</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="o">*</span><span class="n">nary_op_args</span><span class="p">,</span> <span class="o">**</span><span class="n">nary_op_kwargs</span><span class="p">)</span>
|
||||
|
||||
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/differential_operators.py:61,</span> in <span class="ni">jacobian</span><span class="nt">(fun, x)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">59</span> <span class="n">jacobian_shape</span> <span class="o">=</span> <span class="n">ans_vspace</span><span class="o">.</span><span class="n">shape</span> <span class="o">+</span> <span class="n">vspace</span><span class="p">(</span><span class="n">x</span><span class="p">)</span><span class="o">.</span><span class="n">shape</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">60</span> <span class="n">grads</span> <span class="o">=</span> <span class="nb">map</span><span class="p">(</span><span class="n">vjp</span><span class="p">,</span> <span class="n">ans_vspace</span><span class="o">.</span><span class="n">standard_basis</span><span class="p">())</span>
|
||||
@@ -2680,23 +2653,31 @@ Using TensorFlow results in a much better execution time. Try it!</p>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">22</span> <span class="k">for</span> <span class="n">parent</span><span class="p">,</span> <span class="n">ingrad</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">parents</span><span class="p">,</span> <span class="n">ingrads</span><span class="p">):</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">23</span> <span class="n">outgrads</span><span class="p">[</span><span class="n">parent</span><span class="p">]</span> <span class="o">=</span> <span class="n">add_outgrads</span><span class="p">(</span><span class="n">outgrads</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">parent</span><span class="p">),</span> <span class="n">ingrad</span><span class="p">)</span>
|
||||
|
||||
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/core.py:67,</span> in <span class="ni">defvjp.<locals>.vjp_argnums.<locals>.<lambda></span><span class="nt">(g)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">64</span> <span class="k">raise</span> <span class="ne">NotImplementedError</span><span class="p">(</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">65</span> <span class="s2">"VJP of </span><span class="si">{}</span><span class="s2"> wrt argnum 0 not defined"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">fun</span><span class="o">.</span><span class="vm">__name__</span><span class="p">))</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">66</span> <span class="n">vjp</span> <span class="o">=</span> <span class="n">vjpfun</span><span class="p">(</span><span class="n">ans</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
|
||||
<span class="ne">---> </span><span class="mi">67</span> <span class="k">return</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="p">(</span><span class="n">vjp</span><span class="p">(</span><span class="n">g</span><span class="p">),)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">68</span> <span class="k">elif</span> <span class="n">L</span> <span class="o">==</span> <span class="mi">2</span><span class="p">:</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">69</span> <span class="n">argnum_0</span><span class="p">,</span> <span class="n">argnum_1</span> <span class="o">=</span> <span class="n">argnums</span>
|
||||
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/core.py:78,</span> in <span class="ni">defvjp.<locals>.vjp_argnums.<locals>.<lambda></span><span class="nt">(g)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">76</span> <span class="n">vjp_0</span> <span class="o">=</span> <span class="n">vjp_0_fun</span><span class="p">(</span><span class="n">ans</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">77</span> <span class="n">vjp_1</span> <span class="o">=</span> <span class="n">vjp_1_fun</span><span class="p">(</span><span class="n">ans</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
|
||||
<span class="ne">---> </span><span class="mi">78</span> <span class="k">return</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="p">(</span><span class="n">vjp_0</span><span class="p">(</span><span class="n">g</span><span class="p">),</span> <span class="n">vjp_1</span><span class="p">(</span><span class="n">g</span><span class="p">))</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">79</span> <span class="k">else</span><span class="p">:</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">80</span> <span class="n">vjps</span> <span class="o">=</span> <span class="p">[</span><span class="n">vjps_dict</span><span class="p">[</span><span class="n">argnum</span><span class="p">](</span><span class="n">ans</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span> <span class="k">for</span> <span class="n">argnum</span> <span class="ow">in</span> <span class="n">argnums</span><span class="p">]</span>
|
||||
|
||||
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/numpy/numpy_vjps.py:82,</span> in <span class="ni"><lambda></span><span class="nt">(g)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">80</span> <span class="n">defvjp</span><span class="p">(</span><span class="n">anp</span><span class="o">.</span><span class="n">log10</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span> <span class="p">:</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="n">g</span> <span class="o">/</span> <span class="n">x</span> <span class="o">/</span> <span class="n">anp</span><span class="o">.</span><span class="n">log</span><span class="p">(</span><span class="mi">10</span><span class="p">))</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">81</span> <span class="n">defvjp</span><span class="p">(</span><span class="n">anp</span><span class="o">.</span><span class="n">log1p</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span> <span class="p">:</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="n">g</span> <span class="o">/</span> <span class="p">(</span><span class="n">x</span> <span class="o">+</span> <span class="mi">1</span><span class="p">))</span>
|
||||
<span class="ne">---> </span><span class="mi">82</span> <span class="n">defvjp</span><span class="p">(</span><span class="n">anp</span><span class="o">.</span><span class="n">sin</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span> <span class="p">:</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="n">g</span> <span class="o">*</span> <span class="n">anp</span><span class="o">.</span><span class="n">cos</span><span class="p">(</span><span class="n">x</span><span class="p">))</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">83</span> <span class="n">defvjp</span><span class="p">(</span><span class="n">anp</span><span class="o">.</span><span class="n">cos</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span> <span class="p">:</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="o">-</span> <span class="n">g</span> <span class="o">*</span> <span class="n">anp</span><span class="o">.</span><span class="n">sin</span><span class="p">(</span><span class="n">x</span><span class="p">))</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">84</span> <span class="n">defvjp</span><span class="p">(</span><span class="n">anp</span><span class="o">.</span><span class="n">tan</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span> <span class="p">:</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="n">g</span> <span class="o">/</span> <span class="n">anp</span><span class="o">.</span><span class="n">cos</span><span class="p">(</span><span class="n">x</span><span class="p">)</span> <span class="o">**</span><span class="mi">2</span><span class="p">)</span>
|
||||
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/numpy/numpy_vjps.py:660,</span> in <span class="ni">unbroadcast_f.<locals>.<lambda></span><span class="nt">(g)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">658</span> <span class="k">def</span> <span class="nf">unbroadcast_f</span><span class="p">(</span><span class="n">target</span><span class="p">,</span> <span class="n">f</span><span class="p">):</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">659</span> <span class="n">target_meta</span> <span class="o">=</span> <span class="n">anp</span><span class="o">.</span><span class="n">metadata</span><span class="p">(</span><span class="n">target</span><span class="p">)</span>
|
||||
<span class="ne">--> </span><span class="mi">660</span> <span class="k">return</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="n">unbroadcast</span><span class="p">(</span><span class="n">f</span><span class="p">(</span><span class="n">g</span><span class="p">),</span> <span class="n">target_meta</span><span class="p">)</span>
|
||||
|
||||
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/numpy/numpy_boxes.py:27,</span> in <span class="ni">ArrayBox.__mul__</span><span class="nt">(self, other)</span>
|
||||
<span class="ne">---> </span><span class="mi">27</span> <span class="k">def</span> <span class="fm">__mul__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">other</span><span class="p">):</span> <span class="k">return</span> <span class="n">anp</span><span class="o">.</span><span class="n">multiply</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">other</span><span class="p">)</span>
|
||||
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/numpy/numpy_vjps.py:35,</span> in <span class="ni"><lambda></span><span class="nt">(g)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">30</span> <span class="c1"># ----- Binary ufuncs -----</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">32</span> <span class="n">defvjp</span><span class="p">(</span><span class="n">anp</span><span class="o">.</span><span class="n">add</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="p">:</span> <span class="n">unbroadcast_f</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="n">g</span><span class="p">),</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">33</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="p">:</span> <span class="n">unbroadcast_f</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="n">g</span><span class="p">))</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">34</span> <span class="n">defvjp</span><span class="p">(</span><span class="n">anp</span><span class="o">.</span><span class="n">multiply</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="p">:</span> <span class="n">unbroadcast_f</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="n">y</span> <span class="o">*</span> <span class="n">g</span><span class="p">),</span>
|
||||
<span class="ne">---> </span><span class="mi">35</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="p">:</span> <span class="n">unbroadcast_f</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="n">x</span> <span class="o">*</span> <span class="n">g</span><span class="p">))</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">36</span> <span class="n">defvjp</span><span class="p">(</span><span class="n">anp</span><span class="o">.</span><span class="n">subtract</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="p">:</span> <span class="n">unbroadcast_f</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="n">g</span><span class="p">),</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">37</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="p">:</span> <span class="n">unbroadcast_f</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="o">-</span><span class="n">g</span><span class="p">))</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">38</span> <span class="n">defvjp</span><span class="p">(</span><span class="n">anp</span><span class="o">.</span><span class="n">divide</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="p">:</span> <span class="n">unbroadcast_f</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="n">g</span> <span class="o">/</span> <span class="n">y</span><span class="p">),</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">39</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="p">:</span> <span class="n">unbroadcast_f</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="o">-</span> <span class="n">g</span> <span class="o">*</span> <span class="n">x</span> <span class="o">/</span> <span class="n">y</span><span class="o">**</span><span class="mi">2</span><span class="p">))</span>
|
||||
|
||||
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/numpy/numpy_boxes.py:35,</span> in <span class="ni">ArrayBox.__rmul__</span><span class="nt">(self, other)</span>
|
||||
<span class="ne">---> </span><span class="mi">35</span> <span class="k">def</span> <span class="fm">__rmul__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">other</span><span class="p">):</span> <span class="k">return</span> <span class="n">anp</span><span class="o">.</span><span class="n">multiply</span><span class="p">(</span><span class="n">other</span><span class="p">,</span> <span class="bp">self</span><span class="p">)</span>
|
||||
|
||||
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/tracer.py:45,</span> in <span class="ni">primitive.<locals>.f_wrapped</span><span class="nt">(*args, **kwargs)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">43</span> <span class="n">argnums</span> <span class="o">=</span> <span class="nb">tuple</span><span class="p">(</span><span class="n">argnum</span> <span class="k">for</span> <span class="n">argnum</span><span class="p">,</span> <span class="n">_</span> <span class="ow">in</span> <span class="n">boxed_args</span><span class="p">)</span>
|
||||
@@ -2711,29 +2692,13 @@ Using TensorFlow results in a much better execution time. Try it!</p>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">35</span> <span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">fun_name</span><span class="p">,</span> <span class="n">parent_argnums</span><span class="p">))</span>
|
||||
<span class="ne">---> </span><span class="mi">36</span> <span class="bp">self</span><span class="o">.</span><span class="n">vjp</span> <span class="o">=</span> <span class="n">vjpmaker</span><span class="p">(</span><span class="n">parent_argnums</span><span class="p">,</span> <span class="n">value</span><span class="p">,</span> <span class="n">args</span><span class="p">,</span> <span class="n">kwargs</span><span class="p">)</span>
|
||||
|
||||
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/core.py:76,</span> in <span class="ni">defvjp.<locals>.vjp_argnums</span><span class="nt">(argnums, ans, args, kwargs)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">73</span> <span class="k">except</span> <span class="ne">KeyError</span><span class="p">:</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">74</span> <span class="k">raise</span> <span class="ne">NotImplementedError</span><span class="p">(</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">75</span> <span class="s2">"VJP of </span><span class="si">{}</span><span class="s2"> wrt argnums 0, 1 not defined"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">fun</span><span class="o">.</span><span class="vm">__name__</span><span class="p">))</span>
|
||||
<span class="ne">---> </span><span class="mi">76</span> <span class="n">vjp_0</span> <span class="o">=</span> <span class="n">vjp_0_fun</span><span class="p">(</span><span class="n">ans</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">77</span> <span class="n">vjp_1</span> <span class="o">=</span> <span class="n">vjp_1_fun</span><span class="p">(</span><span class="n">ans</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">78</span> <span class="k">return</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="p">(</span><span class="n">vjp_0</span><span class="p">(</span><span class="n">g</span><span class="p">),</span> <span class="n">vjp_1</span><span class="p">(</span><span class="n">g</span><span class="p">))</span>
|
||||
|
||||
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/numpy/numpy_vjps.py:34,</span> in <span class="ni"><lambda></span><span class="nt">(ans, x, y)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">30</span> <span class="c1"># ----- Binary ufuncs -----</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">32</span> <span class="n">defvjp</span><span class="p">(</span><span class="n">anp</span><span class="o">.</span><span class="n">add</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="p">:</span> <span class="n">unbroadcast_f</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="n">g</span><span class="p">),</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">33</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="p">:</span> <span class="n">unbroadcast_f</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="n">g</span><span class="p">))</span>
|
||||
<span class="ne">---> </span><span class="mi">34</span> <span class="n">defvjp</span><span class="p">(</span><span class="n">anp</span><span class="o">.</span><span class="n">multiply</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="p">:</span> <span class="n">unbroadcast_f</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="n">y</span> <span class="o">*</span> <span class="n">g</span><span class="p">),</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">35</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="p">:</span> <span class="n">unbroadcast_f</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="n">x</span> <span class="o">*</span> <span class="n">g</span><span class="p">))</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">36</span> <span class="n">defvjp</span><span class="p">(</span><span class="n">anp</span><span class="o">.</span><span class="n">subtract</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="p">:</span> <span class="n">unbroadcast_f</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="n">g</span><span class="p">),</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">37</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="p">:</span> <span class="n">unbroadcast_f</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="o">-</span><span class="n">g</span><span class="p">))</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">38</span> <span class="n">defvjp</span><span class="p">(</span><span class="n">anp</span><span class="o">.</span><span class="n">divide</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="p">:</span> <span class="n">unbroadcast_f</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="n">g</span> <span class="o">/</span> <span class="n">y</span><span class="p">),</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">39</span> <span class="k">lambda</span> <span class="n">ans</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="p">:</span> <span class="n">unbroadcast_f</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="o">-</span> <span class="n">g</span> <span class="o">*</span> <span class="n">x</span> <span class="o">/</span> <span class="n">y</span><span class="o">**</span><span class="mi">2</span><span class="p">))</span>
|
||||
|
||||
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/numpy/numpy_vjps.py:659,</span> in <span class="ni">unbroadcast_f</span><span class="nt">(target, f)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">658</span> <span class="k">def</span> <span class="nf">unbroadcast_f</span><span class="p">(</span><span class="n">target</span><span class="p">,</span> <span class="n">f</span><span class="p">):</span>
|
||||
<span class="ne">--> </span><span class="mi">659</span> <span class="n">target_meta</span> <span class="o">=</span> <span class="n">anp</span><span class="o">.</span><span class="n">metadata</span><span class="p">(</span><span class="n">target</span><span class="p">)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">660</span> <span class="k">return</span> <span class="k">lambda</span> <span class="n">g</span><span class="p">:</span> <span class="n">unbroadcast</span><span class="p">(</span><span class="n">f</span><span class="p">(</span><span class="n">g</span><span class="p">),</span> <span class="n">target_meta</span><span class="p">)</span>
|
||||
<span class="nn">File ~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/core.py:56,</span> in <span class="ni">defvjp.<locals>.vjp_argnums</span><span class="nt">(argnums, ans, args, kwargs)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">53</span> <span class="n">argnums</span> <span class="o">=</span> <span class="n">kwargs</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'argnums'</span><span class="p">,</span> <span class="n">count</span><span class="p">())</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">54</span> <span class="n">vjps_dict</span> <span class="o">=</span> <span class="p">{</span><span class="n">argnum</span> <span class="p">:</span> <span class="n">translate_vjp</span><span class="p">(</span><span class="n">vjpmaker</span><span class="p">,</span> <span class="n">fun</span><span class="p">,</span> <span class="n">argnum</span><span class="p">)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">55</span> <span class="k">for</span> <span class="n">argnum</span><span class="p">,</span> <span class="n">vjpmaker</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">argnums</span><span class="p">,</span> <span class="n">vjpmakers</span><span class="p">)}</span>
|
||||
<span class="ne">---> </span><span class="mi">56</span> <span class="k">def</span> <span class="nf">vjp_argnums</span><span class="p">(</span><span class="n">argnums</span><span class="p">,</span> <span class="n">ans</span><span class="p">,</span> <span class="n">args</span><span class="p">,</span> <span class="n">kwargs</span><span class="p">):</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">57</span> <span class="n">L</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">argnums</span><span class="p">)</span>
|
||||
<span class="g g-Whitespace"> </span><span class="mi">58</span> <span class="c1"># These first two cases are just optimizations</span>
|
||||
|
||||
<span class="ne">KeyboardInterrupt</span>:
|
||||
</pre></div>
|
||||
|
||||
@@ -268,6 +268,16 @@ const thebe_selector_output = ".output, .cell_output"
|
||||
Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="exercisesweek36.html">
|
||||
Exercises week 36
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week36.html">
|
||||
Week 36: Statistical interpretation of Linear Regression and Resampling techniques
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
|
||||
@@ -268,6 +268,16 @@ const thebe_selector_output = ".output, .cell_output"
|
||||
Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="exercisesweek36.html">
|
||||
Exercises week 36
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week36.html">
|
||||
Week 36: Statistical interpretation of Linear Regression and Resampling techniques
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
|
||||
@@ -268,6 +268,16 @@ const thebe_selector_output = ".output, .cell_output"
|
||||
Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="exercisesweek36.html">
|
||||
Exercises week 36
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week36.html">
|
||||
Week 36: Statistical interpretation of Linear Regression and Resampling techniques
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
@@ -1233,10 +1243,10 @@ covariance matrix through the <strong>np.linalg.eig()</strong> function.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.008947823579448178
|
||||
4.14822021080294
|
||||
[[0.73947737 2.16006961]
|
||||
[2.16006961 7.24782786]]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.01354598394614281
|
||||
4.037503978471253
|
||||
[[0.95927495 2.85834701]
|
||||
[2.85834701 9.70233292]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1273,10 +1283,10 @@ a more brute force way. Here we scale the mean values for each column of the des
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.08657894597048958
|
||||
2.219222942590059
|
||||
[[1. 0.6343356]
|
||||
[0.6343356 1. ]]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.09309965420717024
|
||||
1.6546329613199204
|
||||
[[1. 0.57867898]
|
||||
[0.57867898 1. ]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1306,30 +1316,30 @@ this matrix we easily see that it is a positive definite matrix.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[ 0.29724806 0.26804287]
|
||||
[-0.15626984 -1.36853738]
|
||||
[-0.77070756 -2.13536532]
|
||||
[-0.45372697 -3.1582408 ]
|
||||
[ 0.52580392 2.72567956]
|
||||
[-0.86515815 -1.35704388]
|
||||
[-0.73738602 -2.12933164]
|
||||
[-0.10486183 1.06292011]
|
||||
[ 1.75670484 5.27381733]
|
||||
[ 0.50835355 0.81805914]]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[-0.49352496 -2.38394242]
|
||||
[ 0.18849928 0.73454039]
|
||||
[-1.58104393 -5.16350368]
|
||||
[ 0.34695383 0.23472346]
|
||||
[ 0.95953339 2.95819409]
|
||||
[ 1.31331481 3.59914165]
|
||||
[ 0.14846308 1.1180677 ]
|
||||
[ 0.26022531 0.23496851]
|
||||
[-0.12178678 -0.11868087]
|
||||
[-1.02063403 -1.21350883]]
|
||||
0 1
|
||||
0 0.297248 0.268043
|
||||
1 -0.156270 -1.368537
|
||||
2 -0.770708 -2.135365
|
||||
3 -0.453727 -3.158241
|
||||
4 0.525804 2.725680
|
||||
5 -0.865158 -1.357044
|
||||
6 -0.737386 -2.129332
|
||||
7 -0.104862 1.062920
|
||||
8 1.756705 5.273817
|
||||
9 0.508354 0.818059
|
||||
0 -0.493525 -2.383942
|
||||
1 0.188499 0.734540
|
||||
2 -1.581044 -5.163504
|
||||
3 0.346954 0.234723
|
||||
4 0.959533 2.958194
|
||||
5 1.313315 3.599142
|
||||
6 0.148463 1.118068
|
||||
7 0.260225 0.234969
|
||||
8 -0.121787 -0.118681
|
||||
9 -1.020634 -1.213509
|
||||
0 1
|
||||
0 1.000000 0.915549
|
||||
1 0.915549 1.000000
|
||||
0 1.000000 0.949087
|
||||
1 0.949087 1.000000
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1386,37 +1396,37 @@ this matrix we easily see that it is a positive definite matrix.</p>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 0 1 2 3 4 5 6 7 \
|
||||
0 0.0 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
|
||||
1 0.0 0.086925 0.087377 0.087520 0.087878 0.088164 0.079680 0.079580
|
||||
2 0.0 0.087377 0.089240 0.088074 0.089063 0.089924 0.079827 0.080060
|
||||
3 0.0 0.087520 0.088074 0.094227 0.094252 0.094139 0.089441 0.088974
|
||||
4 0.0 0.087878 0.089063 0.094252 0.094610 0.094812 0.089040 0.088778
|
||||
5 0.0 0.088164 0.089924 0.094139 0.094812 0.095315 0.088488 0.088425
|
||||
6 0.0 0.079680 0.079827 0.089441 0.089040 0.088488 0.087315 0.086524
|
||||
7 0.0 0.079580 0.080060 0.088974 0.088778 0.088425 0.086524 0.085876
|
||||
8 0.0 0.079499 0.080295 0.088502 0.088506 0.088348 0.085716 0.085210
|
||||
9 0.0 0.079448 0.080548 0.088037 0.088238 0.088275 0.084906 0.084541
|
||||
10 0.0 0.071751 0.071438 0.082839 0.082089 0.081194 0.082509 0.081481
|
||||
11 0.0 0.071392 0.071284 0.082139 0.081533 0.080780 0.081559 0.080641
|
||||
12 0.0 0.071071 0.071164 0.081471 0.081007 0.080395 0.080635 0.079827
|
||||
13 0.0 0.070794 0.071084 0.080839 0.080518 0.080048 0.079741 0.079043
|
||||
14 0.0 0.070562 0.071051 0.080249 0.080070 0.079745 0.078880 0.078293
|
||||
1 0.0 0.072254 0.074732 0.075050 0.075400 0.075727 0.068297 0.068215
|
||||
2 0.0 0.074732 0.078265 0.076259 0.077107 0.077954 0.068418 0.068622
|
||||
3 0.0 0.075050 0.076259 0.082604 0.082322 0.081988 0.078133 0.077663
|
||||
4 0.0 0.075400 0.077107 0.082322 0.082315 0.082269 0.077387 0.077091
|
||||
5 0.0 0.075727 0.077954 0.081988 0.082269 0.082525 0.076570 0.076454
|
||||
6 0.0 0.068297 0.068418 0.078133 0.077387 0.076570 0.075952 0.075224
|
||||
7 0.0 0.068215 0.068622 0.077663 0.077091 0.076454 0.075224 0.074613
|
||||
8 0.0 0.068168 0.068876 0.077210 0.076818 0.076371 0.074495 0.074004
|
||||
9 0.0 0.068163 0.069190 0.076775 0.076573 0.076324 0.073764 0.073398
|
||||
10 0.0 0.060924 0.060401 0.071600 0.070601 0.069522 0.071006 0.070141
|
||||
11 0.0 0.060711 0.060369 0.071122 0.070241 0.069283 0.070364 0.069583
|
||||
12 0.0 0.060534 0.060381 0.070671 0.069911 0.069080 0.069738 0.069042
|
||||
13 0.0 0.060394 0.060441 0.070245 0.069612 0.068912 0.069125 0.068517
|
||||
14 0.0 0.060291 0.060550 0.069845 0.069343 0.068782 0.068524 0.068007
|
||||
|
||||
8 9 10 11 12 13 14
|
||||
0 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
|
||||
1 0.079499 0.079448 0.071751 0.071392 0.071071 0.070794 0.070562
|
||||
2 0.080295 0.080548 0.071438 0.071284 0.071164 0.071084 0.071051
|
||||
3 0.088502 0.088037 0.082839 0.082139 0.081471 0.080839 0.080249
|
||||
4 0.088506 0.088238 0.082089 0.081533 0.081007 0.080518 0.080070
|
||||
5 0.088348 0.088275 0.081194 0.080780 0.080395 0.080048 0.079745
|
||||
6 0.085716 0.084906 0.082509 0.081559 0.080635 0.079741 0.078880
|
||||
7 0.085210 0.084541 0.081481 0.080641 0.079827 0.079043 0.078293
|
||||
8 0.084685 0.084157 0.080434 0.079704 0.078999 0.078326 0.077688
|
||||
9 0.084157 0.083772 0.079379 0.078759 0.078165 0.077604 0.077079
|
||||
10 0.080434 0.079379 0.079152 0.078033 0.076935 0.075863 0.074818
|
||||
11 0.079704 0.078759 0.078033 0.077004 0.075996 0.075014 0.074061
|
||||
12 0.078999 0.078165 0.076935 0.075996 0.075079 0.074187 0.073325
|
||||
13 0.078326 0.077604 0.075863 0.075014 0.074187 0.073388 0.072618
|
||||
14 0.077688 0.077079 0.074818 0.074061 0.073325 0.072618 0.071942
|
||||
1 0.068168 0.068163 0.060924 0.060711 0.060534 0.060394 0.060291
|
||||
2 0.068876 0.069190 0.060401 0.060369 0.060381 0.060441 0.060550
|
||||
3 0.077210 0.076775 0.071600 0.071122 0.070671 0.070245 0.069845
|
||||
4 0.076818 0.076573 0.070601 0.070241 0.069911 0.069612 0.069343
|
||||
5 0.076371 0.076324 0.069522 0.069283 0.069080 0.068912 0.068782
|
||||
6 0.074495 0.073764 0.071006 0.070364 0.069738 0.069125 0.068524
|
||||
7 0.074004 0.073398 0.070141 0.069583 0.069042 0.068517 0.068007
|
||||
8 0.073520 0.073044 0.069265 0.068792 0.068339 0.067905 0.067489
|
||||
9 0.073044 0.072705 0.068375 0.067990 0.067628 0.067288 0.066969
|
||||
10 0.069265 0.068375 0.067400 0.066672 0.065952 0.065237 0.064526
|
||||
11 0.068792 0.067990 0.066672 0.066006 0.065350 0.064701 0.064057
|
||||
12 0.068339 0.067628 0.065952 0.065350 0.064759 0.064176 0.063600
|
||||
13 0.067905 0.067288 0.065237 0.064701 0.064176 0.063661 0.063155
|
||||
14 0.067489 0.066969 0.064526 0.064057 0.063600 0.063155 0.062721
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@@ -268,6 +268,16 @@ const thebe_selector_output = ".output, .cell_output"
|
||||
Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="exercisesweek36.html">
|
||||
Exercises week 36
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week36.html">
|
||||
Week 36: Statistical interpretation of Linear Regression and Resampling techniques
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
@@ -787,10 +797,10 @@ number <span class="math notranslate nohighlight">\(i\)</span> is left out. Usin
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Runtime: 0.0907831 sec
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Runtime: 0.0907788 sec
|
||||
Jackknife Statistics :
|
||||
original bias std. error
|
||||
100.142 100.132 0.149864
|
||||
100.022 100.012 0.148734
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1009,7 +1019,7 @@ theorem.</p>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Bootstrap Statistics :
|
||||
original bias std. error
|
||||
100.033 14.9292 100.032 0.149452
|
||||
99.7522 14.9594 99.7525 0.149991
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1591,9 +1601,9 @@ Mean squared error on training data: 0.00060705
|
||||
Mean squared error on test data: 3250.17647619
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20614/626635268.py:73: RuntimeWarning: divide by zero encountered in log10
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16213/626635268.py:73: RuntimeWarning: divide by zero encountered in log10
|
||||
plt.plot(polynomial, np.log10(trainingerror), label='Training Error')
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20614/626635268.py:74: RuntimeWarning: divide by zero encountered in log10
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16213/626635268.py:74: RuntimeWarning: divide by zero encountered in log10
|
||||
plt.plot(polynomial, np.log10(testerror), label='Test Error')
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -1827,7 +1837,7 @@ cross-validation (LOOCV).</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20614/3817475779.py:63: RuntimeWarning: divide by zero encountered in log10
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16213/3817475779.py:63: RuntimeWarning: divide by zero encountered in log10
|
||||
plt.plot(polynomial, np.log10(estimated_mse_sklearn), label='Test Error')
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -2716,9 +2726,9 @@ linear system as an equation would reduce this down to
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20614/4162706317.py:6: MatplotlibDeprecationWarning: Auto-removal of grids by pcolor() and pcolormesh() is deprecated since 3.5 and will be removed two minor releases later; please call grid(False) first.
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16213/4162706317.py:6: MatplotlibDeprecationWarning: Auto-removal of grids by pcolor() and pcolormesh() is deprecated since 3.5 and will be removed two minor releases later; please call grid(False) first.
|
||||
cb = fig.colorbar(im)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20614/4162706317.py:7: UserWarning: FixedFormatter should only be used together with FixedLocator
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16213/4162706317.py:7: UserWarning: FixedFormatter should only be used together with FixedLocator
|
||||
cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -2862,9 +2872,9 @@ with the form utilized in linear regression, viz.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20614/3777801602.py:6: MatplotlibDeprecationWarning: Auto-removal of grids by pcolor() and pcolormesh() is deprecated since 3.5 and will be removed two minor releases later; please call grid(False) first.
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16213/3777801602.py:6: MatplotlibDeprecationWarning: Auto-removal of grids by pcolor() and pcolormesh() is deprecated since 3.5 and will be removed two minor releases later; please call grid(False) first.
|
||||
cb = fig.colorbar(im)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20614/3777801602.py:7: UserWarning: FixedFormatter should only be used together with FixedLocator
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16213/3777801602.py:7: UserWarning: FixedFormatter should only be used together with FixedLocator
|
||||
cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -2904,9 +2914,9 @@ cost function is given by</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20614/438060758.py:9: MatplotlibDeprecationWarning: Auto-removal of grids by pcolor() and pcolormesh() is deprecated since 3.5 and will be removed two minor releases later; please call grid(False) first.
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16213/438060758.py:9: MatplotlibDeprecationWarning: Auto-removal of grids by pcolor() and pcolormesh() is deprecated since 3.5 and will be removed two minor releases later; please call grid(False) first.
|
||||
cb = fig.colorbar(im)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20614/438060758.py:10: UserWarning: FixedFormatter should only be used together with FixedLocator
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16213/438060758.py:10: UserWarning: FixedFormatter should only be used together with FixedLocator
|
||||
cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -2941,9 +2951,9 @@ cost function is given by</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20614/3544313922.py:8: MatplotlibDeprecationWarning: Auto-removal of grids by pcolor() and pcolormesh() is deprecated since 3.5 and will be removed two minor releases later; please call grid(False) first.
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16213/3544313922.py:8: MatplotlibDeprecationWarning: Auto-removal of grids by pcolor() and pcolormesh() is deprecated since 3.5 and will be removed two minor releases later; please call grid(False) first.
|
||||
cb = fig.colorbar(im)
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20614/3544313922.py:9: UserWarning: FixedFormatter should only be used together with FixedLocator
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16213/3544313922.py:9: UserWarning: FixedFormatter should only be used together with FixedLocator
|
||||
cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -3005,34 +3015,34 @@ constant as opposed to ridge and OLS. We get a sparse solution with
|
||||
10%|██████████████▋ | 1/10 [00:00<00:05, 1.53it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 20%|█████████████████████████████▍ | 2/10 [00:00<00:03, 2.15it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 20%|█████████████████████████████▍ | 2/10 [00:00<00:03, 2.12it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 30%|████████████████████████████████████████████ | 3/10 [00:01<00:02, 3.06it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 30%|████████████████████████████████████████████ | 3/10 [00:01<00:02, 3.10it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 40%|██████████████████████████████████████████████████████████▊ | 4/10 [00:01<00:01, 3.41it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 40%|██████████████████████████████████████████████████████████▊ | 4/10 [00:01<00:01, 3.80it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 50%|█████████████████████████████████████████████████████████████████████████▌ | 5/10 [00:01<00:01, 4.28it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 50%|█████████████████████████████████████████████████████████████████████████▌ | 5/10 [00:01<00:01, 4.70it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 60%|████████████████████████████████████████████████████████████████████████████████████████▏ | 6/10 [00:01<00:00, 5.00it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 60%|████████████████████████████████████████████████████████████████████████████████████████▏ | 6/10 [00:01<00:00, 5.50it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 70%|██████████████████████████████████████████████████████████████████████████████████████████████████████▉ | 7/10 [00:01<00:00, 5.70it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 70%|██████████████████████████████████████████████████████████████████████████████████████████████████████▉ | 7/10 [00:01<00:00, 6.18it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 80%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▌ | 8/10 [00:01<00:00, 6.29it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 80%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▌ | 8/10 [00:01<00:00, 6.77it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 90%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▎ | 9/10 [00:02<00:00, 6.85it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 90%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▎ | 9/10 [00:01<00:00, 7.22it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:02<00:00, 7.30it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:02<00:00, 7.32it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:02<00:00, 4.68it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:02<00:00, 4.88it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>
|
||||
@@ -3179,9 +3189,9 @@ which polynomial fits the data best.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20614/3980313467.py:9: MatplotlibDeprecationWarning: Calling gca() with keyword arguments was deprecated in Matplotlib 3.4. Starting two minor releases later, gca() will take no keyword arguments. The gca() function should only be used to get the current axes, or if no axes exist, create new axes with default keyword arguments. To create a new axes with non-default arguments, use plt.axes() or plt.subplot().
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16213/3980313467.py:9: MatplotlibDeprecationWarning: Calling gca() with keyword arguments was deprecated in Matplotlib 3.4. Starting two minor releases later, gca() will take no keyword arguments. The gca() function should only be used to get the current axes, or if no axes exist, create new axes with default keyword arguments. To create a new axes with non-default arguments, use plt.axes() or plt.subplot().
|
||||
ax = fig.gca(projection='3d')
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20614/3980313467.py:37: MatplotlibDeprecationWarning: Auto-removal of grids by pcolor() and pcolormesh() is deprecated since 3.5 and will be removed two minor releases later; please call grid(False) first.
|
||||
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16213/3980313467.py:37: MatplotlibDeprecationWarning: Auto-removal of grids by pcolor() and pcolormesh() is deprecated since 3.5 and will be removed two minor releases later; please call grid(False) first.
|
||||
fig.colorbar(surf, shrink=0.5, aspect=5)
|
||||
</pre></div>
|
||||
</div>
|
||||
|
||||
@@ -268,6 +268,16 @@ const thebe_selector_output = ".output, .cell_output"
|
||||
Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="exercisesweek36.html">
|
||||
Exercises week 36
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week36.html">
|
||||
Week 36: Statistical interpretation of Linear Regression and Resampling techniques
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
|
||||
@@ -268,6 +268,16 @@ const thebe_selector_output = ".output, .cell_output"
|
||||
Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="exercisesweek36.html">
|
||||
Exercises week 36
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week36.html">
|
||||
Week 36: Statistical interpretation of Linear Regression and Resampling techniques
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
|
||||
@@ -268,6 +268,16 @@ const thebe_selector_output = ".output, .cell_output"
|
||||
Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="exercisesweek36.html">
|
||||
Exercises week 36
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week36.html">
|
||||
Week 36: Statistical interpretation of Linear Regression and Resampling techniques
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
@@ -715,9 +725,9 @@ predicting the target features of query instances is as follows:</p>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>2nd degree coefficients:
|
||||
zero power: -4.653578701904388
|
||||
first power: 0.17297886491529482
|
||||
second power: -0.0007790285013223805
|
||||
zero power: -6.548110376991839
|
||||
first power: 0.2232822462117919
|
||||
second power: -0.0007480407244119591
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapter6_1_1.png" src="_images/chapter6_1_1.png" />
|
||||
|
||||
@@ -268,6 +268,16 @@ const thebe_selector_output = ".output, .cell_output"
|
||||
Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="exercisesweek36.html">
|
||||
Exercises week 36
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week36.html">
|
||||
Week 36: Statistical interpretation of Linear Regression and Resampling techniques
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
|
||||
@@ -268,6 +268,16 @@ const thebe_selector_output = ".output, .cell_output"
|
||||
Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="exercisesweek36.html">
|
||||
Exercises week 36
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week36.html">
|
||||
Week 36: Statistical interpretation of Linear Regression and Resampling techniques
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
@@ -669,10 +679,10 @@ covariance matrix through the <strong>np.linalg.eig()</strong> function.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.264540221699101
|
||||
4.673457751724773
|
||||
[[0.83632853 2.54078623]
|
||||
[2.54078623 8.44021223]]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.26662339374864535
|
||||
4.736115211426478
|
||||
[[ 1.20561803 3.63264564]
|
||||
[ 3.63264564 12.10207647]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -712,10 +722,10 @@ a more brute force way. Here we scale the mean values for each column of the des
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.08374032367704139
|
||||
1.7696835316227453
|
||||
[[1. 0.66443521]
|
||||
[0.66443521 1. ]]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.09318696260700278
|
||||
1.9096305360355206
|
||||
[[1. 0.65907898]
|
||||
[0.65907898 1. ]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -744,30 +754,30 @@ this matrix we easily see that it is a positive definite matrix.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[ 1.20708879 4.33201844]
|
||||
[-0.3838783 -1.24125217]
|
||||
[ 0.74722409 1.60194224]
|
||||
[-0.04086326 0.37419664]
|
||||
[ 0.62422045 2.05587489]
|
||||
[ 1.75357263 5.63710438]
|
||||
[-2.53968309 -7.28219089]
|
||||
[-1.42054337 -4.90629812]
|
||||
[-0.13019959 -0.83320794]
|
||||
[ 0.18306165 0.26181254]]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[ 0.32360769 2.53264317]
|
||||
[ 0.04531375 -0.70569833]
|
||||
[ 0.17001895 -0.49570819]
|
||||
[ 1.60938882 4.69896355]
|
||||
[ 0.14052537 1.37535105]
|
||||
[-0.12909917 1.25781559]
|
||||
[-0.03016916 0.01780471]
|
||||
[-0.38816656 -0.82894017]
|
||||
[-0.34591885 -3.1893772 ]
|
||||
[-1.39550083 -4.66285417]]
|
||||
0 1
|
||||
0 1.207089 4.332018
|
||||
1 -0.383878 -1.241252
|
||||
2 0.747224 1.601942
|
||||
3 -0.040863 0.374197
|
||||
4 0.624220 2.055875
|
||||
5 1.753573 5.637104
|
||||
6 -2.539683 -7.282191
|
||||
7 -1.420543 -4.906298
|
||||
8 -0.130200 -0.833208
|
||||
9 0.183062 0.261813
|
||||
0 0.323608 2.532643
|
||||
1 0.045314 -0.705698
|
||||
2 0.170019 -0.495708
|
||||
3 1.609389 4.698964
|
||||
4 0.140525 1.375351
|
||||
5 -0.129099 1.257816
|
||||
6 -0.030169 0.017805
|
||||
7 -0.388167 -0.828940
|
||||
8 -0.345919 -3.189377
|
||||
9 -1.395501 -4.662854
|
||||
0 1
|
||||
0 1.000000 0.992504
|
||||
1 0.992504 1.000000
|
||||
0 1.000000 0.899734
|
||||
1 0.899734 1.000000
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -824,37 +834,37 @@ this matrix we easily see that it is a positive definite matrix.</p>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 0 1 2 3 4 5 6 7 \
|
||||
0 0.0 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
|
||||
1 0.0 0.092290 0.091376 0.091772 0.091659 0.091579 0.083691 0.083321
|
||||
2 0.0 0.091376 0.091209 0.090620 0.090923 0.091266 0.082166 0.082105
|
||||
3 0.0 0.091772 0.090620 0.098069 0.097435 0.096845 0.093556 0.092724
|
||||
4 0.0 0.091659 0.090923 0.097435 0.097111 0.096833 0.092478 0.091896
|
||||
5 0.0 0.091579 0.091266 0.096845 0.096833 0.096872 0.091445 0.091113
|
||||
6 0.0 0.083691 0.082166 0.093556 0.092478 0.091445 0.092007 0.090828
|
||||
7 0.0 0.083321 0.082105 0.092724 0.091896 0.091113 0.090828 0.089857
|
||||
8 0.0 0.083051 0.082145 0.091996 0.091418 0.090888 0.089744 0.088982
|
||||
9 0.0 0.082877 0.082285 0.091369 0.091044 0.090768 0.088754 0.088201
|
||||
10 0.0 0.075829 0.073985 0.087419 0.086021 0.084663 0.087860 0.086440
|
||||
11 0.0 0.075277 0.073684 0.086460 0.085272 0.084124 0.086624 0.085384
|
||||
12 0.0 0.074819 0.073477 0.085603 0.084624 0.083685 0.085484 0.084422
|
||||
13 0.0 0.074453 0.073363 0.084846 0.084076 0.083348 0.084438 0.083555
|
||||
14 0.0 0.074180 0.073344 0.084187 0.083627 0.083111 0.083485 0.082779
|
||||
1 0.0 0.086358 0.084977 0.084028 0.085456 0.086705 0.073624 0.075225
|
||||
2 0.0 0.084977 0.085778 0.080455 0.082791 0.085242 0.069320 0.071314
|
||||
3 0.0 0.084028 0.080455 0.086647 0.086999 0.086904 0.078715 0.079856
|
||||
4 0.0 0.085456 0.082791 0.086999 0.087848 0.088361 0.078426 0.079839
|
||||
5 0.0 0.086705 0.085242 0.086904 0.088361 0.089641 0.077606 0.079333
|
||||
6 0.0 0.073624 0.069320 0.078715 0.078426 0.077606 0.073298 0.074046
|
||||
7 0.0 0.075225 0.071314 0.079856 0.079839 0.079333 0.074046 0.074971
|
||||
8 0.0 0.076867 0.073465 0.080914 0.081223 0.081100 0.074653 0.075779
|
||||
9 0.0 0.078521 0.075782 0.081827 0.082527 0.082877 0.075047 0.076403
|
||||
10 0.0 0.063766 0.059453 0.069855 0.069291 0.068201 0.066209 0.066728
|
||||
11 0.0 0.065175 0.061037 0.071105 0.070700 0.069782 0.067233 0.067873
|
||||
12 0.0 0.066656 0.062742 0.072370 0.072150 0.071437 0.068239 0.069013
|
||||
13 0.0 0.068206 0.064577 0.073634 0.073630 0.073163 0.069203 0.070127
|
||||
14 0.0 0.069816 0.066556 0.074868 0.075121 0.074951 0.070092 0.071185
|
||||
|
||||
8 9 10 11 12 13 14
|
||||
0 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
|
||||
1 0.083051 0.082877 0.075829 0.075277 0.074819 0.074453 0.074180
|
||||
2 0.082145 0.082285 0.073985 0.073684 0.073477 0.073363 0.073344
|
||||
3 0.091996 0.091369 0.087419 0.086460 0.085603 0.084846 0.084187
|
||||
4 0.091418 0.091044 0.086021 0.085272 0.084624 0.084076 0.083627
|
||||
5 0.090888 0.090768 0.084663 0.084124 0.083685 0.083348 0.083111
|
||||
6 0.089744 0.088754 0.087860 0.086624 0.085484 0.084438 0.083485
|
||||
7 0.088982 0.088201 0.086440 0.085384 0.084422 0.083555 0.082779
|
||||
8 0.088317 0.087746 0.085108 0.084230 0.083446 0.082756 0.082158
|
||||
9 0.087746 0.087386 0.083859 0.083159 0.082553 0.082040 0.081620
|
||||
10 0.085108 0.083859 0.085252 0.083832 0.082501 0.081258 0.080098
|
||||
11 0.084230 0.083159 0.083832 0.082569 0.081395 0.080305 0.079298
|
||||
12 0.083446 0.082553 0.082501 0.081395 0.080374 0.079437 0.078582
|
||||
13 0.082756 0.082040 0.081258 0.080305 0.079437 0.078652 0.077949
|
||||
14 0.082158 0.081620 0.080098 0.079298 0.078582 0.077949 0.077397
|
||||
1 0.076867 0.078521 0.063766 0.065175 0.066656 0.068206 0.069816
|
||||
2 0.073465 0.075782 0.059453 0.061037 0.062742 0.064577 0.066556
|
||||
3 0.080914 0.081827 0.069855 0.071105 0.072370 0.073634 0.074868
|
||||
4 0.081223 0.082527 0.069291 0.070700 0.072150 0.073630 0.075121
|
||||
5 0.081100 0.082877 0.068201 0.069782 0.071437 0.073163 0.074951
|
||||
6 0.074653 0.075047 0.066209 0.067233 0.068239 0.069203 0.070092
|
||||
7 0.075779 0.076403 0.066728 0.067873 0.069013 0.070127 0.071185
|
||||
8 0.076819 0.077713 0.067089 0.068365 0.069654 0.070936 0.072186
|
||||
9 0.077713 0.078926 0.067224 0.068643 0.070096 0.071567 0.073038
|
||||
10 0.067089 0.067224 0.060603 0.061468 0.062298 0.063070 0.063750
|
||||
11 0.068365 0.068643 0.061468 0.062424 0.063353 0.064232 0.065030
|
||||
12 0.069654 0.070096 0.062298 0.063353 0.064390 0.065389 0.066317
|
||||
13 0.070936 0.071567 0.063070 0.064232 0.065389 0.066519 0.067593
|
||||
14 0.072186 0.073038 0.063750 0.065030 0.066317 0.067593 0.068832
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1043,10 +1053,10 @@ We can write our own code or simply use either the functionaly of <strong>numpy<
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 0 1
|
||||
0 3.956454 1.972286
|
||||
1 1.972286 1.977089
|
||||
[[3.95645365 1.97228638]
|
||||
[1.97228638 1.97708897]]
|
||||
0 4.059118 2.009163
|
||||
1 2.009163 2.004788
|
||||
[[4.05911793 2.00916336]
|
||||
[2.00916336 2.00478786]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1073,8 +1083,8 @@ Our own code here is not very elegant and asks for obvious improvements. It is t
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Centered covariance using own code
|
||||
[[3.95645365 1.97228638]
|
||||
[1.97228638 1.97708897]]
|
||||
[[4.05911793 2.00916336]
|
||||
[2.00916336 2.00478786]]
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapter8_65_1.png" src="_images/chapter8_65_1.png" />
|
||||
@@ -1134,16 +1144,16 @@ questions.</p>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Eigenvalues of Covariance matrix
|
||||
5.173439546289586
|
||||
0.7601030735620569
|
||||
5.288455813429108
|
||||
0.7754499790100834
|
||||
First eigenvector
|
||||
[0.85102768 0.52512084]
|
||||
[0.85299536 0.52191849]
|
||||
Second eigenvector
|
||||
[-0.52512084 0.85102768]
|
||||
[-0.52191849 0.85299536]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Eigenvector of largest eigenvalue
|
||||
[-0.85102768 -0.52512084]
|
||||
[0.85299536 0.52191849]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@@ -268,6 +268,16 @@ const thebe_selector_output = ".output, .cell_output"
|
||||
Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="exercisesweek36.html">
|
||||
Exercises week 36
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week36.html">
|
||||
Week 36: Statistical interpretation of Linear Regression and Resampling techniques
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
|
||||
@@ -268,6 +268,16 @@ const thebe_selector_output = ".output, .cell_output"
|
||||
Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="exercisesweek36.html">
|
||||
Exercises week 36
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week36.html">
|
||||
Week 36: Statistical interpretation of Linear Regression and Resampling techniques
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
@@ -965,11 +975,11 @@ which equals</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20669/483257001.py:18: MatplotlibDeprecationWarning: Calling gca() with keyword arguments was deprecated in Matplotlib 3.4. Starting two minor releases later, gca() will take no keyword arguments. The gca() function should only be used to get the current axes, or if no axes exist, create new axes with default keyword arguments. To create a new axes with non-default arguments, use plt.axes() or plt.subplot().
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16261/483257001.py:18: MatplotlibDeprecationWarning: Calling gca() with keyword arguments was deprecated in Matplotlib 3.4. Starting two minor releases later, gca() will take no keyword arguments. The gca() function should only be used to get the current axes, or if no axes exist, create new axes with default keyword arguments. To create a new axes with non-default arguments, use plt.axes() or plt.subplot().
|
||||
ax = fig.gca(projection="3d")
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span><mpl_toolkits.mplot3d.art3d.Poly3DCollection at 0x1336a6040>
|
||||
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span><mpl_toolkits.mplot3d.art3d.Poly3DCollection at 0x138098070>
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapteroptimization_61_2.png" src="_images/chapteroptimization_61_2.png" />
|
||||
@@ -1027,7 +1037,7 @@ which equals</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[<matplotlib.lines.Line2D at 0x133c2e1c0>]
|
||||
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[<matplotlib.lines.Line2D at 0x1387541c0>]
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapteroptimization_69_1.png" src="_images/chapteroptimization_69_1.png" />
|
||||
@@ -1284,11 +1294,11 @@ when <span class="math notranslate nohighlight">\(||\nabla_\beta C(\beta_k) || \
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[0.29972182 4.52744746]
|
||||
[[4.01247056]
|
||||
[2.97656972]]
|
||||
[[4.01247056]
|
||||
[2.97656972]]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[0.28836053 4.52113415]
|
||||
[[4.19528375]
|
||||
[2.90383424]]
|
||||
[[4.19528375]
|
||||
[2.90383424]]
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapteroptimization_123_1.png" src="_images/chapteroptimization_123_1.png" />
|
||||
@@ -1317,9 +1327,9 @@ when <span class="math notranslate nohighlight">\(||\nabla_\beta C(\beta_k) || \
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[4.02158709]
|
||||
[2.93023603]]
|
||||
[4.00882596] [2.93522293]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[4.16575256]
|
||||
[2.8620652 ]]
|
||||
[4.11520281] [2.85097049]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1390,10 +1400,10 @@ C_{\text{ridge}}(\beta) = \frac{1}{n}||X\beta -\mathbf{y}||^2 + \lambda ||\beta|
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[4.09781386]
|
||||
[2.97980332]]
|
||||
[[4.0527437 ]
|
||||
[3.01510929]]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[4.20793824]
|
||||
[2.75460639]]
|
||||
[[4.106971 ]
|
||||
[2.83724637]]
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapteroptimization_132_1.png" src="_images/chapteroptimization_132_1.png" />
|
||||
@@ -1643,15 +1653,15 @@ function.</p>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
|
||||
[[3.87618586]
|
||||
[3.13847924]]
|
||||
Eigenvalues of Hessian Matrix:[0.3313155 4.62759057]
|
||||
[[4.31347523]
|
||||
[2.69915639]]
|
||||
Eigenvalues of Hessian Matrix:[0.28457442 4.43693489]
|
||||
theta from own gd
|
||||
[[3.87618586]
|
||||
[3.13847924]]
|
||||
[[4.31347523]
|
||||
[2.69915639]]
|
||||
theta from own sdg
|
||||
[[3.87379129]
|
||||
[3.15508406]]
|
||||
[[4.2891298 ]
|
||||
[2.67783138]]
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapteroptimization_148_1.png" src="_images/chapteroptimization_148_1.png" />
|
||||
|
||||
@@ -268,6 +268,16 @@ const thebe_selector_output = ".output, .cell_output"
|
||||
Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="exercisesweek36.html">
|
||||
Exercises week 36
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week36.html">
|
||||
Week 36: Statistical interpretation of Linear Regression and Resampling techniques
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
|
||||
@@ -268,6 +268,16 @@ const thebe_selector_output = ".output, .cell_output"
|
||||
Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="exercisesweek36.html">
|
||||
Exercises week 36
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week36.html">
|
||||
Week 36: Statistical interpretation of Linear Regression and Resampling techniques
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
|
||||
@@ -268,6 +268,16 @@ const thebe_selector_output = ".output, .cell_output"
|
||||
Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="exercisesweek36.html">
|
||||
Exercises week 36
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week36.html">
|
||||
Week 36: Statistical interpretation of Linear Regression and Resampling techniques
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
|
||||
@@ -0,0 +1,622 @@
|
||||
|
||||
<!DOCTYPE html>
|
||||
|
||||
<html>
|
||||
<head>
|
||||
<meta charset="utf-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>Exercises week 36 — Applied Data Analysis and Machine Learning</title>
|
||||
|
||||
<link href="_static/css/theme.css" rel="stylesheet">
|
||||
<link href="_static/css/index.ff1ffe594081f20da1ef19478df9384b.css" rel="stylesheet">
|
||||
|
||||
|
||||
<link rel="stylesheet"
|
||||
href="_static/vendor/fontawesome/5.13.0/css/all.min.css">
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin
|
||||
href="_static/vendor/fontawesome/5.13.0/webfonts/fa-solid-900.woff2">
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin
|
||||
href="_static/vendor/fontawesome/5.13.0/webfonts/fa-brands-400.woff2">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/sphinx-book-theme.css?digest=c3fdc42140077d1ad13ad2f1588a4309" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/copybutton.css" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/mystnb.css" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/sphinx-thebe.css" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/panels-main.c949a650a448cc0ae9fd3441c0e17fb0.css" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/panels-variables.06eb56fa6e07937060861dad626602ad.css" />
|
||||
|
||||
<link rel="preload" as="script" href="_static/js/index.be7d3bbb2ef33a8344ce.js">
|
||||
|
||||
<script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
|
||||
<script src="_static/jquery.js"></script>
|
||||
<script src="_static/underscore.js"></script>
|
||||
<script src="_static/doctools.js"></script>
|
||||
<script src="_static/clipboard.min.js"></script>
|
||||
<script src="_static/copybutton.js"></script>
|
||||
<script>let toggleHintShow = 'Click to show';</script>
|
||||
<script>let toggleHintHide = 'Click to hide';</script>
|
||||
<script>let toggleOpenOnPrint = 'true';</script>
|
||||
<script src="_static/togglebutton.js"></script>
|
||||
<script>var togglebuttonSelector = '.toggle, .admonition.dropdown, .tag_hide_input div.cell_input, .tag_hide-input div.cell_input, .tag_hide_output div.cell_output, .tag_hide-output div.cell_output, .tag_hide_cell.cell, .tag_hide-cell.cell';</script>
|
||||
<script src="_static/sphinx-book-theme.d59cb220de22ca1c485ebbdc042f0030.js"></script>
|
||||
<script>const THEBE_JS_URL = "https://unpkg.com/thebe@0.8.2/lib/index.js"
|
||||
const thebe_selector = ".thebe,.cell"
|
||||
const thebe_selector_input = "pre"
|
||||
const thebe_selector_output = ".output, .cell_output"
|
||||
</script>
|
||||
<script async="async" src="_static/sphinx-thebe.js"></script>
|
||||
<script>window.MathJax = {"options": {"processHtmlClass": "tex2jax_process|mathjax_process|math|output_area"}}</script>
|
||||
<script defer="defer" src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
|
||||
<link rel="index" title="Index" href="genindex.html" />
|
||||
<link rel="search" title="Search" href="search.html" />
|
||||
<link rel="next" title="Week 36: Statistical interpretation of Linear Regression and Resampling techniques" href="week36.html" />
|
||||
<link rel="prev" title="Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression" href="week35.html" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
||||
<meta name="docsearch:language" content="None">
|
||||
|
||||
|
||||
<!-- Google Analytics -->
|
||||
|
||||
</head>
|
||||
<body data-spy="scroll" data-target="#bd-toc-nav" data-offset="80">
|
||||
|
||||
<div class="container-fluid" id="banner"></div>
|
||||
|
||||
|
||||
|
||||
<div class="container-xl">
|
||||
<div class="row">
|
||||
|
||||
<div class="col-12 col-md-3 bd-sidebar site-navigation show" id="site-navigation">
|
||||
|
||||
<div class="navbar-brand-box">
|
||||
<a class="navbar-brand text-wrap" href="index.html">
|
||||
|
||||
<!-- `logo` is deprecated in Sphinx 4.0, so remove this when we stop supporting 3 -->
|
||||
|
||||
|
||||
|
||||
<img src="_static/logo.png" class="logo" alt="logo">
|
||||
|
||||
|
||||
<h1 class="site-logo" id="site-title">Applied Data Analysis and Machine Learning</h1>
|
||||
|
||||
</a>
|
||||
</div><form class="bd-search d-flex align-items-center" action="search.html" method="get">
|
||||
<i class="icon fas fa-search"></i>
|
||||
<input type="search" class="form-control" name="q" id="search-input" placeholder="Search this book..." aria-label="Search this book..." autocomplete="off" >
|
||||
</form><nav class="bd-links" id="bd-docs-nav" aria-label="Main">
|
||||
<div class="bd-toc-item active">
|
||||
<ul class="nav bd-sidenav">
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="intro.html">
|
||||
Applied Data Analysis and Machine Learning
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading">
|
||||
<span class="caption-text">
|
||||
About the course
|
||||
</span>
|
||||
</p>
|
||||
<ul class="nav bd-sidenav">
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="schedule.html">
|
||||
Teaching schedule with links to material
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="teachers.html">
|
||||
Teachers and Grading
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="textbooks.html">
|
||||
Textbooks
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading">
|
||||
<span class="caption-text">
|
||||
Review of Statistics with Resampling Techniques and Linear Algebra
|
||||
</span>
|
||||
</p>
|
||||
<ul class="nav bd-sidenav">
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="statistics.html">
|
||||
1. Elements of Probability Theory and Statistical Data Analysis
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="linalg.html">
|
||||
2. Linear Algebra, Handling of Arrays and more Python Features
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading">
|
||||
<span class="caption-text">
|
||||
From Regression to Support Vector Machines
|
||||
</span>
|
||||
</p>
|
||||
<ul class="nav bd-sidenav">
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="chapter1.html">
|
||||
3. Linear Regression
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="chapter2.html">
|
||||
4. Ridge and Lasso Regression
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="chapter3.html">
|
||||
5. Resampling Methods
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="chapter4.html">
|
||||
6. Logistic Regression
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="chapteroptimization.html">
|
||||
7. Optimization, the central part of any Machine Learning algortithm
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="chapter5.html">
|
||||
8. Support Vector Machines, overarching aims
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading">
|
||||
<span class="caption-text">
|
||||
Decision Trees, Ensemble Methods and Boosting
|
||||
</span>
|
||||
</p>
|
||||
<ul class="nav bd-sidenav">
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="chapter6.html">
|
||||
9. Decision trees, overarching aims
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="chapter7.html">
|
||||
10. Ensemble Methods: From a Single Tree to Many Trees and Extreme Boosting, Meet the Jungle of Methods
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading">
|
||||
<span class="caption-text">
|
||||
Dimensionality Reduction
|
||||
</span>
|
||||
</p>
|
||||
<ul class="nav bd-sidenav">
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="chapter8.html">
|
||||
11. Basic ideas of the Principal Component Analysis (PCA)
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="clustering.html">
|
||||
12. Clustering and Unsupervised Learning
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading">
|
||||
<span class="caption-text">
|
||||
Deep Learning Methods
|
||||
</span>
|
||||
</p>
|
||||
<ul class="nav bd-sidenav">
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="chapter9.html">
|
||||
13. Neural networks
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="chapter10.html">
|
||||
14. Building a Feed Forward Neural Network
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="chapter11.html">
|
||||
15. Solving Differential Equations with Deep Learning
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="chapter12.html">
|
||||
16. Convolutional Neural Networks
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="chapter13.html">
|
||||
17. Recurrent neural networks: Overarching view
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading">
|
||||
<span class="caption-text">
|
||||
Weekly material, notes and exercises
|
||||
</span>
|
||||
</p>
|
||||
<ul class="current nav bd-sidenav">
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="exercisesweek34.html">
|
||||
Exercises week 34
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week34.html">
|
||||
Week 34: Introduction to the course, Logistics and Practicalities
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="exercisesweek35.html">
|
||||
Exercises week 35
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week35.html">
|
||||
Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1 current active">
|
||||
<a class="current reference internal" href="#">
|
||||
Exercises week 36
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week36.html">
|
||||
Week 36: Statistical interpretation of Linear Regression and Resampling techniques
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
</nav> <!-- To handle the deprecated key -->
|
||||
|
||||
<div class="navbar_extra_footer">
|
||||
Powered by <a href="https://jupyterbook.org">Jupyter Book</a>
|
||||
</div>
|
||||
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<main class="col py-md-3 pl-md-4 bd-content overflow-auto" role="main">
|
||||
|
||||
<div class="topbar container-xl fixed-top">
|
||||
<div class="topbar-contents row">
|
||||
<div class="col-12 col-md-3 bd-topbar-whitespace site-navigation show"></div>
|
||||
<div class="col pl-md-4 topbar-main">
|
||||
|
||||
<button id="navbar-toggler" class="navbar-toggler ml-0" type="button" data-toggle="collapse"
|
||||
data-toggle="tooltip" data-placement="bottom" data-target=".site-navigation" aria-controls="navbar-menu"
|
||||
aria-expanded="true" aria-label="Toggle navigation" aria-controls="site-navigation"
|
||||
title="Toggle navigation" data-toggle="tooltip" data-placement="left">
|
||||
<i class="fas fa-bars"></i>
|
||||
<i class="fas fa-arrow-left"></i>
|
||||
<i class="fas fa-arrow-up"></i>
|
||||
</button>
|
||||
|
||||
|
||||
<div class="dropdown-buttons-trigger">
|
||||
<button id="dropdown-buttons-trigger" class="btn btn-secondary topbarbtn" aria-label="Download this page"><i
|
||||
class="fas fa-download"></i></button>
|
||||
|
||||
<div class="dropdown-buttons">
|
||||
<!-- ipynb file if we had a myst markdown file -->
|
||||
|
||||
<!-- Download raw file -->
|
||||
<a class="dropdown-buttons" href="_sources/exercisesweek36.ipynb"><button type="button"
|
||||
class="btn btn-secondary topbarbtn" title="Download source file" data-toggle="tooltip"
|
||||
data-placement="left">.ipynb</button></a>
|
||||
<!-- Download PDF via print -->
|
||||
<button type="button" id="download-print" class="btn btn-secondary topbarbtn" title="Print to PDF"
|
||||
onclick="printPdf(this)" data-toggle="tooltip" data-placement="left">.pdf</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Source interaction buttons -->
|
||||
|
||||
<!-- Full screen (wrap in <a> to have style consistency -->
|
||||
|
||||
<a class="full-screen-button"><button type="button" class="btn btn-secondary topbarbtn" data-toggle="tooltip"
|
||||
data-placement="bottom" onclick="toggleFullScreen()" aria-label="Fullscreen mode"
|
||||
title="Fullscreen mode"><i
|
||||
class="fas fa-expand"></i></button></a>
|
||||
|
||||
<!-- Launch buttons -->
|
||||
|
||||
</div>
|
||||
|
||||
<!-- Table of contents -->
|
||||
<div class="d-none d-md-block col-md-2 bd-toc show noprint">
|
||||
|
||||
<div class="tocsection onthispage pt-5 pb-3">
|
||||
<i class="fas fa-list"></i> Contents
|
||||
</div>
|
||||
<nav id="bd-toc-nav" aria-label="Page">
|
||||
<ul class="visible nav section-nav flex-column">
|
||||
<li class="toc-h2 nav-item toc-entry">
|
||||
<a class="reference internal nav-link" href="#overarching-aims-of-the-exercises-this-week">
|
||||
Overarching aims of the exercises this week
|
||||
</a>
|
||||
</li>
|
||||
<li class="toc-h2 nav-item toc-entry">
|
||||
<a class="reference internal nav-link" href="#exercise-1-analytical-exercises">
|
||||
Exercise 1: Analytical exercises
|
||||
</a>
|
||||
<ul class="nav section-nav flex-column">
|
||||
<li class="toc-h3 nav-item toc-entry">
|
||||
<a class="reference internal nav-link" href="#a-expression-for-ridge-regression">
|
||||
a) Expression for Ridge regression
|
||||
</a>
|
||||
</li>
|
||||
<li class="toc-h3 nav-item toc-entry">
|
||||
<a class="reference internal nav-link" href="#b-the-singular-value-decomposition">
|
||||
b) The singular value decomposition
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li class="toc-h2 nav-item toc-entry">
|
||||
<a class="reference internal nav-link" href="#exercise-2-adding-ridge-regression">
|
||||
Exercise 2: Adding Ridge Regression
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</nav>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div id="main-content" class="row">
|
||||
<div class="col-12 col-md-9 pl-md-3 pr-md-0">
|
||||
<!-- Table of contents that is only displayed when printing the page -->
|
||||
<div id="jb-print-docs-body" class="onlyprint">
|
||||
<h1>Exercises week 36</h1>
|
||||
<!-- Table of contents -->
|
||||
<div id="print-main-content">
|
||||
<div id="jb-print-toc">
|
||||
|
||||
<div>
|
||||
<h2> Contents </h2>
|
||||
</div>
|
||||
<nav aria-label="Page">
|
||||
<ul class="visible nav section-nav flex-column">
|
||||
<li class="toc-h2 nav-item toc-entry">
|
||||
<a class="reference internal nav-link" href="#overarching-aims-of-the-exercises-this-week">
|
||||
Overarching aims of the exercises this week
|
||||
</a>
|
||||
</li>
|
||||
<li class="toc-h2 nav-item toc-entry">
|
||||
<a class="reference internal nav-link" href="#exercise-1-analytical-exercises">
|
||||
Exercise 1: Analytical exercises
|
||||
</a>
|
||||
<ul class="nav section-nav flex-column">
|
||||
<li class="toc-h3 nav-item toc-entry">
|
||||
<a class="reference internal nav-link" href="#a-expression-for-ridge-regression">
|
||||
a) Expression for Ridge regression
|
||||
</a>
|
||||
</li>
|
||||
<li class="toc-h3 nav-item toc-entry">
|
||||
<a class="reference internal nav-link" href="#b-the-singular-value-decomposition">
|
||||
b) The singular value decomposition
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li class="toc-h2 nav-item toc-entry">
|
||||
<a class="reference internal nav-link" href="#exercise-2-adding-ridge-regression">
|
||||
Exercise 2: Adding Ridge Regression
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</nav>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div>
|
||||
|
||||
<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)
|
||||
doconce format html exercisesweek36.do.txt -->
|
||||
<!-- dom:TITLE: Exercises week 36 --><div class="tex2jax_ignore mathjax_ignore section" id="exercises-week-36">
|
||||
<h1>Exercises week 36<a class="headerlink" href="#exercises-week-36" title="Permalink to this headline">¶</a></h1>
|
||||
<p><strong>September 4-8, 2023</strong></p>
|
||||
<p>Date: <strong>Deadline is Sunday September 10 at midnight</strong></p>
|
||||
<div class="section" id="overarching-aims-of-the-exercises-this-week">
|
||||
<h2>Overarching aims of the exercises this week<a class="headerlink" href="#overarching-aims-of-the-exercises-this-week" title="Permalink to this headline">¶</a></h2>
|
||||
<p>This set of exercises form an important part of the first project. The
|
||||
analytical exercises deal with the material covered last week on the
|
||||
mathematical interpretations of ordinary least squares and of Ridge
|
||||
regression. The numerical exercises can be seen as a continuation of
|
||||
exercise 3 from week 35, with the inclusion of Ridge regression. This
|
||||
material enters also the discussions of the first project.</p>
|
||||
</div>
|
||||
<div class="section" id="exercise-1-analytical-exercises">
|
||||
<h2>Exercise 1: Analytical exercises<a class="headerlink" href="#exercise-1-analytical-exercises" title="Permalink to this headline">¶</a></h2>
|
||||
<p>The aim here is to derive the expression for the optimal parameters
|
||||
using Ridge regression. Furthermore, using the singular value
|
||||
decomposition, we will analyze the difference between the ordinary
|
||||
least squares approach and Ridge regression.</p>
|
||||
<p>The expression for the standard Mean Squared Error (MSE) which we used to define our cost function and the equations for the ordinary least squares (OLS) method, was given by the
|
||||
optimization problem</p>
|
||||
<div class="math notranslate nohighlight">
|
||||
\[
|
||||
{\displaystyle \min_{\boldsymbol{\beta}\in {\mathbb{R}}^{p}}}\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)\right\}.
|
||||
\]</div>
|
||||
<p>which we can also write as</p>
|
||||
<div class="math notranslate nohighlight">
|
||||
\[
|
||||
{\displaystyle \min_{\boldsymbol{\beta}\in
|
||||
{\mathbb{R}}^{p}}}\frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2=\frac{1}{n}\vert\vert \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\vert\vert_2^2,
|
||||
\]</div>
|
||||
<p>where we have used the definition of a norm-2 vector, that is</p>
|
||||
<div class="math notranslate nohighlight">
|
||||
\[
|
||||
\vert\vert \boldsymbol{x}\vert\vert_2 = \sqrt{\sum_i x_i^2}.
|
||||
\]</div>
|
||||
<p>By minimizing the above equation with respect to the parameters
|
||||
<span class="math notranslate nohighlight">\(\boldsymbol{\beta}\)</span> we could then obtain an analytical expression for the
|
||||
parameters <span class="math notranslate nohighlight">\(\boldsymbol{\beta}\)</span>.</p>
|
||||
<p>We can add a regularization parameter <span class="math notranslate nohighlight">\(\lambda\)</span> by
|
||||
defining a new cost function to be optimized, that is</p>
|
||||
<div class="math notranslate nohighlight">
|
||||
\[
|
||||
{\displaystyle \min_{\boldsymbol{\beta}\in
|
||||
{\mathbb{R}}^{p}}}\frac{1}{n}\vert\vert \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\vert\vert_2^2+\lambda\vert\vert \boldsymbol{\beta}\vert\vert_2^2
|
||||
\]</div>
|
||||
<p>which leads to the Ridge regression minimization problem. One can require as part of the optimization problem
|
||||
that <span class="math notranslate nohighlight">\(\vert\vert \boldsymbol{\beta}\vert\vert_2^2\le t\)</span>, where <span class="math notranslate nohighlight">\(t\)</span> is
|
||||
a finite number larger than zero. We will not implement that here.</p>
|
||||
<div class="section" id="a-expression-for-ridge-regression">
|
||||
<h3>a) Expression for Ridge regression<a class="headerlink" href="#a-expression-for-ridge-regression" title="Permalink to this headline">¶</a></h3>
|
||||
<p>Show that the optimal parameters</p>
|
||||
<div class="math notranslate nohighlight">
|
||||
\[
|
||||
\hat{\boldsymbol{\beta}}_{\mathrm{Ridge}} = \left(\boldsymbol{X}^T\boldsymbol{X}+\lambda\boldsymbol{I}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y},
|
||||
\]</div>
|
||||
<p>with <span class="math notranslate nohighlight">\(\boldsymbol{I}\)</span> being a <span class="math notranslate nohighlight">\(p\times p\)</span> identity matrix with the constraint that</p>
|
||||
<div class="math notranslate nohighlight">
|
||||
\[
|
||||
\sum_{i=0}^{p-1} \beta_i^2 \leq t,
|
||||
\]</div>
|
||||
<p>with <span class="math notranslate nohighlight">\(t\)</span> a finite positive number.</p>
|
||||
<p>The ordinary least squares result is</p>
|
||||
<div class="math notranslate nohighlight">
|
||||
\[
|
||||
\hat{\boldsymbol{\beta}}_{\mathrm{OLS}} = \left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y},
|
||||
\]</div>
|
||||
</div>
|
||||
<div class="section" id="b-the-singular-value-decomposition">
|
||||
<h3>b) The singular value decomposition<a class="headerlink" href="#b-the-singular-value-decomposition" title="Permalink to this headline">¶</a></h3>
|
||||
<p>Use the singular value decomposition of an n\times p<span class="math notranslate nohighlight">\( matrix \)</span>\boldsymbol{X}$ (our design matrix)</p>
|
||||
<div class="math notranslate nohighlight">
|
||||
\[
|
||||
\boldsymbol{X}=\boldsymbol{U}\boldsymbol{\Sigma}\boldsymbol{V}^T,
|
||||
\]</div>
|
||||
<p>where <span class="math notranslate nohighlight">\(\boldsymbol{U}\)</span> and <span class="math notranslate nohighlight">\(\boldsymbol{V}\)</span> are orthogonal matrices of dimensions
|
||||
<span class="math notranslate nohighlight">\(n\times n\)</span> and <span class="math notranslate nohighlight">\(p\times p\)</span>, respectively, and <span class="math notranslate nohighlight">\(\boldsymbol{\Sigma}\)</span> is an
|
||||
<span class="math notranslate nohighlight">\(n\times p\)</span> matrix which contains the ingular values only. This material was discussed during the lectures of week 35.</p>
|
||||
<p>Show that you can write the
|
||||
OLS solutions in terms of the eigenvectors (the columns) of the orthogonal matrix <span class="math notranslate nohighlight">\(\boldsymbol{U}\)</span> as</p>
|
||||
<div class="math notranslate nohighlight">
|
||||
\[
|
||||
\tilde{\boldsymbol{y}}_{\mathrm{OLS}}=\boldsymbol{X}\boldsymbol{\beta} = \sum_{j=0}^{p-1}\boldsymbol{u}_j\boldsymbol{u}_j^T\boldsymbol{y}.
|
||||
\]</div>
|
||||
<p>For Ridge regression, show that the corresponding equation is</p>
|
||||
<div class="math notranslate nohighlight">
|
||||
\[
|
||||
\tilde{\boldsymbol{y}}_{\mathrm{Ridge}}=\boldsymbol{X}\boldsymbol{\beta}_{\mathrm{Ridge}} = \boldsymbol{U\Sigma V^T}\left(\boldsymbol{V}\boldsymbol{\Sigma}^2\boldsymbol{V}^T+\lambda\boldsymbol{I} \right)^{-1}(\boldsymbol{U\Sigma V^T})^T\boldsymbol{y}=\sum_{j=0}^{p-1}\boldsymbol{u}_j\boldsymbol{u}_j^T\frac{\sigma_j^2}{\sigma_j^2+\lambda}\boldsymbol{y},
|
||||
\]</div>
|
||||
<p>with the vectors <span class="math notranslate nohighlight">\(\boldsymbol{u}_j\)</span> being the columns of <span class="math notranslate nohighlight">\(\boldsymbol{U}\)</span> from the SVD of the matrix <span class="math notranslate nohighlight">\(\boldsymbol{X}\)</span>.</p>
|
||||
<p>Give an interpretation of the results. <a class="reference external" href="https://link.springer.com/book/10.1007/978-0-387-84858-7">Section 3.4 of Hastie et al’s textbook gives a good discussion of the above results</a>.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="section" id="exercise-2-adding-ridge-regression">
|
||||
<h2>Exercise 2: Adding Ridge Regression<a class="headerlink" href="#exercise-2-adding-ridge-regression" title="Permalink to this headline">¶</a></h2>
|
||||
<p>This exercise is a continuation of exercise 3 from week 35, see <a class="reference external" href="https://compphysics.github.io/MachineLearning/doc/LectureNotes/_build/html/exercisesweek35.html">https://compphysics.github.io/MachineLearning/doc/LectureNotes/_build/html/exercisesweek35.html</a>. We will use the same function to
|
||||
generate our data set, still staying with a simple function <span class="math notranslate nohighlight">\(y(x)\)</span>
|
||||
which we want to fit using linear regression, but now extending the
|
||||
analysis to include the Ridge regression method.</p>
|
||||
<p>In this exercise you need to include the same elements from last week, that is</p>
|
||||
<ol class="simple">
|
||||
<li><p>scale your data by subtracting the mean value from each column in the design matrix.</p></li>
|
||||
<li><p>perform a split of the data in a training set and a test set.</p></li>
|
||||
</ol>
|
||||
<p>The addition to the analysis this time is the introduction of the hyperparameter <span class="math notranslate nohighlight">\(\lambda\)</span> when introducing Ridge regression.</p>
|
||||
<p>Extend the code from exercise 3 from <a class="reference external" href="https://compphysics.github.io/MachineLearning/doc/LectureNotes/_build/html/exercisesweek35.html">week 35</a> to include Ridge regression with the hyperparameter <span class="math notranslate nohighlight">\(\lambda\)</span>. The optimal parameters <span class="math notranslate nohighlight">\(\hat{\beta}\)</span> for Ridge regression can be obtained by matrix inversion in a similar way as done for ordinary least squares. You need to add to your code the following equations</p>
|
||||
<div class="math notranslate nohighlight">
|
||||
\[
|
||||
\hat{\boldsymbol{\beta}}_{\mathrm{Ridge}} = \left(\boldsymbol{X}^T\boldsymbol{X}+\lambda\boldsymbol{I}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
|
||||
\]</div>
|
||||
<p>The ordinary least squares result you encoded last week is given by</p>
|
||||
<div class="math notranslate nohighlight">
|
||||
\[
|
||||
\hat{\boldsymbol{\beta}}_{\mathrm{OLS}} = \left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y},
|
||||
\]</div>
|
||||
<p>Use these results to compute the mean squared error for ordinary least
|
||||
squares and Ridge regression first for a polynomial of degree five
|
||||
with <span class="math notranslate nohighlight">\(n=100\)</span> data points and five selected values of
|
||||
<span class="math notranslate nohighlight">\(\lambda=[0.0001,0.001, 0.01,0.1,1.0]\)</span>. Compute thereafter the mean
|
||||
squared error for the same values of <span class="math notranslate nohighlight">\(\lambda\)</span> for polynomials of degree ten
|
||||
and <span class="math notranslate nohighlight">\(15\)</span>. Discuss your results for the training MSE and test MSE with
|
||||
Ridge regression and ordinary least squares.</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script type="text/x-thebe-config">
|
||||
{
|
||||
requestKernel: true,
|
||||
binderOptions: {
|
||||
repo: "binder-examples/jupyter-stacks-datascience",
|
||||
ref: "master",
|
||||
},
|
||||
codeMirrorConfig: {
|
||||
theme: "abcdef",
|
||||
mode: "python"
|
||||
},
|
||||
kernelOptions: {
|
||||
kernelName: "python3",
|
||||
path: "./."
|
||||
},
|
||||
predefinedOutput: true
|
||||
}
|
||||
</script>
|
||||
<script>kernelName = 'python3'</script>
|
||||
|
||||
</div>
|
||||
|
||||
|
||||
<!-- Previous / next buttons -->
|
||||
<div class='prev-next-area'>
|
||||
<a class='left-prev' id="prev-link" href="week35.html" title="previous page">
|
||||
<i class="fas fa-angle-left"></i>
|
||||
<div class="prev-next-info">
|
||||
<p class="prev-next-subtitle">previous</p>
|
||||
<p class="prev-next-title">Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression</p>
|
||||
</div>
|
||||
</a>
|
||||
<a class='right-next' id="next-link" href="week36.html" title="next page">
|
||||
<div class="prev-next-info">
|
||||
<p class="prev-next-subtitle">next</p>
|
||||
<p class="prev-next-title">Week 36: Statistical interpretation of Linear Regression and Resampling techniques</p>
|
||||
</div>
|
||||
<i class="fas fa-angle-right"></i>
|
||||
</a>
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
<footer class="footer">
|
||||
<p>
|
||||
|
||||
By Morten Hjorth-Jensen<br/>
|
||||
|
||||
© Copyright 2021.<br/>
|
||||
</p>
|
||||
</footer>
|
||||
</main>
|
||||
|
||||
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script src="_static/js/index.be7d3bbb2ef33a8344ce.js"></script>
|
||||
|
||||
</body>
|
||||
</html>
|
||||
@@ -264,6 +264,16 @@ const thebe_selector_output = ".output, .cell_output"
|
||||
Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="exercisesweek36.html">
|
||||
Exercises week 36
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week36.html">
|
||||
Week 36: Statistical interpretation of Linear Regression and Resampling techniques
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
|
||||
@@ -265,6 +265,16 @@ const thebe_selector_output = ".output, .cell_output"
|
||||
Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="exercisesweek36.html">
|
||||
Exercises week 36
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week36.html">
|
||||
Week 36: Statistical interpretation of Linear Regression and Resampling techniques
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
|
||||
@@ -268,6 +268,16 @@ const thebe_selector_output = ".output, .cell_output"
|
||||
Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="exercisesweek36.html">
|
||||
Exercises week 36
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week36.html">
|
||||
Week 36: Statistical interpretation of Linear Regression and Resampling techniques
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
@@ -571,8 +581,8 @@ matrices and vectors.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[-0.38763091 -2.70501534 -0.3581571 -0.96251494 -1.26223899 0.35309734
|
||||
2.28186376 -1.85104809 -0.37114298 -1.20893188]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[-0.87136737 1.4300745 -0.3322326 -0.66758934 -1.00283636 -0.27625974
|
||||
1.89249454 -0.25006757 0.73565195 0.33697405]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -793,26 +803,26 @@ as (recall that we user lowercase letters for vectors and uppercase letters for
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[0.78459267 0.75453081 0.05363779 0.57350724 0.69764852 0.65795279
|
||||
0.51507839 0.20461136 0.38788697 0.96496641]
|
||||
[0.25028968 0.96081861 0.18931988 0.51108791 0.30337713 0.43036842
|
||||
0.52839842 0.15321987 0.78561443 0.09030825]
|
||||
[0.10097956 0.50584526 0.34989509 0.55626454 0.69154964 0.2895238
|
||||
0.13393141 0.15503141 0.26015755 0.42902155]
|
||||
[0.25789255 0.9492866 0.90252116 0.904221 0.51933924 0.14432948
|
||||
0.54445121 0.02699523 0.18657863 0.971688 ]
|
||||
[0.13392097 0.27801122 0.50931378 0.04234339 0.22442417 0.44065609
|
||||
0.74943449 0.42451192 0.33736485 0.97952271]
|
||||
[0.95824108 0.59950055 0.91346044 0.58042237 0.13228567 0.31519573
|
||||
0.12427889 0.64736858 0.60236782 0.18036103]
|
||||
[0.95911004 0.82027884 0.27547877 0.84317815 0.89842298 0.68322599
|
||||
0.02377668 0.39943328 0.00162091 0.0525221 ]
|
||||
[0.94076632 0.88335933 0.75492292 0.7860324 0.41923956 0.86181269
|
||||
0.45979894 0.44190304 0.07829878 0.00458014]
|
||||
[0.42915006 0.68127341 0.23875722 0.31988705 0.54956992 0.24801014
|
||||
0.65335632 0.97713364 0.05635863 0.12160173]
|
||||
[0.93386901 0.74935095 0.96534137 0.98400474 0.98581925 0.30313128
|
||||
0.41386599 0.88450476 0.87099757 0.22566121]]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[0.52428467 0.89421873 0.57286194 0.35473061 0.34626037 0.77640601
|
||||
0.60707801 0.60515083 0.41335011 0.16061448]
|
||||
[0.51508374 0.7651899 0.07657669 0.14045566 0.92863147 0.32541271
|
||||
0.62761922 0.20965182 0.55051175 0.14072037]
|
||||
[0.70311568 0.03617972 0.22335823 0.45797073 0.2223808 0.86208133
|
||||
0.64452882 0.0770789 0.7866024 0.45739087]
|
||||
[0.38469441 0.16105443 0.09873132 0.21913656 0.34561469 0.521986
|
||||
0.89261458 0.75628425 0.25070907 0.96504592]
|
||||
[0.40506137 0.48241947 0.32158804 0.45112054 0.5783575 0.3267061
|
||||
0.96261297 0.25216021 0.94271589 0.60651016]
|
||||
[0.91410101 0.48217977 0.39116042 0.99397049 0.80768431 0.68650491
|
||||
0.04184636 0.5174581 0.86144422 0.46794929]
|
||||
[0.62405149 0.27498198 0.63550707 0.85902603 0.38644889 0.86858926
|
||||
0.68282653 0.65554133 0.81051173 0.00281835]
|
||||
[0.66001778 0.64858537 0.90034533 0.62389761 0.5333734 0.75390337
|
||||
0.97926642 0.9893405 0.61605739 0.51905011]
|
||||
[0.59277468 0.52678301 0.68347072 0.76707201 0.08821204 0.55220861
|
||||
0.14648477 0.17606773 0.59609892 0.83723539]
|
||||
[0.40961677 0.07325301 0.34652523 0.72201591 0.66250644 0.6367311
|
||||
0.79577619 0.85212606 0.86811137 0.43001767]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -872,13 +882,13 @@ covariance matrix through the <strong>np.linalg.eig()</strong> function.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.0626202708457115
|
||||
4.327883798133277
|
||||
0.3178411477108273
|
||||
[[ 1.01422853 3.21909039 2.82750687]
|
||||
[ 3.21909039 11.42694395 9.22887861]
|
||||
[ 2.82750687 9.22887861 17.2086376 ]]
|
||||
[24.72855434 0.09533574 4.82592 ]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.03499693245669077
|
||||
4.187268182169147
|
||||
-0.19627430151896047
|
||||
[[ 1.0622197 3.16232481 2.98018358]
|
||||
[ 3.16232481 10.48442771 8.58039072]
|
||||
[ 2.98018358 8.58039072 13.66351927]]
|
||||
[21.70244042 0.07978179 3.42794448]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
Binary file not shown.
@@ -266,6 +266,16 @@ const thebe_selector_output = ".output, .cell_output"
|
||||
Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="exercisesweek36.html">
|
||||
Exercises week 36
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week36.html">
|
||||
Week 36: Statistical interpretation of Linear Regression and Resampling techniques
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
|
||||
@@ -270,6 +270,16 @@ const thebe_selector_output = ".output, .cell_output"
|
||||
Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="exercisesweek36.html">
|
||||
Exercises week 36
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week36.html">
|
||||
Week 36: Statistical interpretation of Linear Regression and Resampling techniques
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -268,6 +268,16 @@ const thebe_selector_output = ".output, .cell_output"
|
||||
Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="exercisesweek36.html">
|
||||
Exercises week 36
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week36.html">
|
||||
Week 36: Statistical interpretation of Linear Regression and Resampling techniques
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
@@ -943,37 +953,27 @@ uncorrelated.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>3.0818000034712947
|
||||
[[1.82949718e-02 3.31658671e-01 3.68886089e-01 6.49666527e-01
|
||||
1.49052317e-01 5.97305528e-01 4.57810898e-01 2.51578522e-01
|
||||
2.89396164e-01 5.90342093e-01]
|
||||
[3.31658671e-01 6.01244295e+00 6.68731669e+00 1.17774184e+01
|
||||
2.70208087e+00 1.08281969e+01 8.29938171e+00 4.56071752e+00
|
||||
5.24629107e+00 1.07019610e+01]
|
||||
[3.68886089e-01 6.68731669e+00 7.43794243e+00 1.30993886e+01
|
||||
3.00537912e+00 1.20436207e+01 9.23095558e+00 5.07264063e+00
|
||||
5.83516719e+00 1.19032152e+01]
|
||||
[6.49666527e-01 1.17774184e+01 1.30993886e+01 2.30700873e+01
|
||||
5.29294618e+00 2.12107137e+01 1.62571673e+01 8.93371943e+00
|
||||
1.02766489e+01 2.09634375e+01]
|
||||
[1.49052317e-01 2.70208087e+00 3.00537912e+00 5.29294618e+00
|
||||
1.21435514e+00 4.86635201e+00 3.72986500e+00 2.04965397e+00
|
||||
2.35776087e+00 4.80961969e+00]
|
||||
[5.97305528e-01 1.08281969e+01 1.20436207e+01 2.12107137e+01
|
||||
4.86635201e+00 1.95011994e+01 1.49468927e+01 8.21369083e+00
|
||||
9.44838453e+00 1.92738529e+01]
|
||||
[4.57810898e-01 8.29938171e+00 9.23095558e+00 1.62571673e+01
|
||||
3.72986500e+00 1.49468927e+01 1.14561980e+01 6.29546690e+00
|
||||
7.24181044e+00 1.47726407e+01]
|
||||
[2.51578522e-01 4.56071752e+00 5.07264063e+00 8.93371943e+00
|
||||
2.04965397e+00 8.21369083e+00 6.29546690e+00 3.45951629e+00
|
||||
3.97955570e+00 8.11793498e+00]
|
||||
[2.89396164e-01 5.24629107e+00 5.83516719e+00 1.02766489e+01
|
||||
2.35776087e+00 9.44838453e+00 7.24181044e+00 3.97955570e+00
|
||||
4.57776818e+00 9.33823452e+00]
|
||||
[5.90342093e-01 1.07019610e+01 1.19032152e+01 2.09634375e+01
|
||||
4.80961969e+00 1.92738529e+01 1.47726407e+01 8.11793498e+00
|
||||
9.33823452e+00 1.90491568e+01]]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>1.9474044318317307
|
||||
[[16.27404277 4.07430684 4.90189742 9.80462258 9.27412395 4.40264255
|
||||
5.15060311 17.7265799 -0.62769765 9.00658717]
|
||||
[ 4.07430684 1.02002781 1.22722021 2.45464765 2.32183405 1.10222868
|
||||
1.28948521 4.43795846 -0.15714797 2.25485457]
|
||||
[ 4.90189742 1.22722021 1.47649841 2.95324615 2.79345488 1.32611807
|
||||
1.55141095 5.33941551 -0.18906854 2.71287025]
|
||||
[ 9.80462258 2.45464765 2.95324615 5.90699099 5.58738148 2.65246006
|
||||
3.10308387 10.67973263 -0.37816901 5.42619859]
|
||||
[ 9.27412395 2.32183405 2.79345488 5.58738148 5.28506507 2.50894343
|
||||
2.93518534 10.10188442 -0.35770742 5.13260331]
|
||||
[ 4.40264255 1.10222868 1.32611807 2.65246006 2.50894343 1.19105386
|
||||
1.39340081 4.79559972 -0.16981204 2.43656628]
|
||||
[ 5.15060311 1.28948521 1.55141095 3.10308387 2.93518534 1.39340081
|
||||
1.63012429 5.61031939 -0.19866124 2.85051211]
|
||||
[17.7265799 4.43795846 5.33941551 10.67973263 10.10188442 4.79559972
|
||||
5.61031939 19.308763 -0.68372271 9.8104687 ]
|
||||
[-0.62769765 -0.15714797 -0.18906854 -0.37816901 -0.35770742 -0.16981204
|
||||
-0.19866124 -0.68372271 0.0242106 -0.3473884 ]
|
||||
[ 9.00658717 2.25485457 2.71287025 5.42619859 5.13260331 2.43656628
|
||||
2.85051211 9.8104687 -0.3473884 4.98453971]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1241,15 +1241,15 @@ more practically oriented methods like the blocking technique.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>-0.054244842835462305
|
||||
4.000854409696581
|
||||
0.13083543199018746
|
||||
0.8437169762110144 8.948675162607389 10.317825933186352
|
||||
2.601583341274718 2.1245596497124075 6.443538568902246
|
||||
[[ 0.84371698 2.60158334 2.12455965]
|
||||
[ 2.60158334 8.94867516 6.44353857]
|
||||
[ 2.12455965 6.44353857 10.31782593]]
|
||||
[16.80422999 0.07128434 3.23470374]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.08549632935144091
|
||||
4.438111004052204
|
||||
0.15485374225770068
|
||||
0.8225325933960339 7.887392472315955 7.062734874360835
|
||||
2.364244035362507 1.814860456269878 5.4421034379901165
|
||||
[[0.82253259 2.36424404 1.81486046]
|
||||
[2.36424404 7.88739247 5.44210344]
|
||||
[1.81486046 5.44210344 7.06273487]]
|
||||
[13.62123034 0.09662862 2.05480098]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1579,7 +1579,7 @@ assumption for approximating <span class="math notranslate nohighlight">\(\sigma
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>-0.048547423739546604 0.9959293935368551
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.029574060388349064 0.9577775794806141
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/statistics_188_1.png" src="_images/statistics_188_1.png" />
|
||||
|
||||
@@ -266,6 +266,16 @@ const thebe_selector_output = ".output, .cell_output"
|
||||
Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="exercisesweek36.html">
|
||||
Exercises week 36
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week36.html">
|
||||
Week 36: Statistical interpretation of Linear Regression and Resampling techniques
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
|
||||
@@ -266,6 +266,16 @@ const thebe_selector_output = ".output, .cell_output"
|
||||
Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="exercisesweek36.html">
|
||||
Exercises week 36
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week36.html">
|
||||
Week 36: Statistical interpretation of Linear Regression and Resampling techniques
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
|
||||
@@ -268,6 +268,16 @@ const thebe_selector_output = ".output, .cell_output"
|
||||
Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="exercisesweek36.html">
|
||||
Exercises week 36
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week36.html">
|
||||
Week 36: Statistical interpretation of Linear Regression and Resampling techniques
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
@@ -1733,8 +1743,8 @@ developed in the 1970s, namely EISPACK and LINPACK. We describe them shortly he
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[-0.03483677 -1.06136773 0.56481272 -0.35947075 -1.64972151 -1.58590795
|
||||
-1.17428293 0.7225597 -1.14061742 -1.22893751]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[-0.75408649 -0.05177092 -0.07339251 1.49372893 -1.21467644 0.24712854
|
||||
0.67554302 -0.45143018 0.34212496 0.63262164]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1959,26 +1969,26 @@ lowercase letters for vectors and uppercase letters for matrices)</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[0.24279008 0.63112036 0.80947943 0.97509292 0.19425617 0.3957482
|
||||
0.1655226 0.83760781 0.07995375 0.76400155]
|
||||
[0.75621895 0.20893514 0.93082503 0.79419162 0.02783644 0.21296315
|
||||
0.64298419 0.34578026 0.60975366 0.46369869]
|
||||
[0.77859437 0.23477043 0.35438626 0.63115792 0.2460037 0.35568525
|
||||
0.0825971 0.94117118 0.14900336 0.30035718]
|
||||
[0.36692356 0.78972773 0.67655635 0.67160204 0.80108096 0.31507591
|
||||
0.21328866 0.41340248 0.3005849 0.40672425]
|
||||
[0.21922061 0.88274486 0.86572911 0.06486061 0.07565581 0.26678445
|
||||
0.03265139 0.22090974 0.33135331 0.66973261]
|
||||
[0.7221662 0.96941962 0.39707147 0.24929083 0.31531613 0.33079801
|
||||
0.06538944 0.42352791 0.94227931 0.27809912]
|
||||
[0.07195822 0.31719317 0.47248297 0.18264218 0.64033527 0.51146442
|
||||
0.49545491 0.91936525 0.81656508 0.78329097]
|
||||
[0.58666142 0.01646892 0.11029323 0.67442363 0.6914791 0.87902877
|
||||
0.98950411 0.27090196 0.08732305 0.89543736]
|
||||
[0.25253892 0.57505712 0.24848907 0.9631064 0.46312791 0.96431281
|
||||
0.28744729 0.09772449 0.17676228 0.51656406]
|
||||
[0.68664664 0.63281351 0.62806444 0.36809474 0.98129668 0.62914124
|
||||
0.37885034 0.6577093 0.57947706 0.24688732]]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[0.11167579 0.88778091 0.24908246 0.32566092 0.37585988 0.61893735
|
||||
0.58574143 0.60081665 0.81065392 0.00509527]
|
||||
[0.02840884 0.39600607 0.96810393 0.28213741 0.19237495 0.54219245
|
||||
0.85581869 0.17928538 0.43185254 0.37624303]
|
||||
[0.82045496 0.22828694 0.52773891 0.54526769 0.84708672 0.59118929
|
||||
0.18221254 0.94640767 0.25328924 0.03192893]
|
||||
[0.98328759 0.68996179 0.81168086 0.08086066 0.10659682 0.66599246
|
||||
0.78035464 0.57403561 0.96995249 0.78990036]
|
||||
[0.9909606 0.39920034 0.59813773 0.85982436 0.66665241 0.60522226
|
||||
0.08889302 0.74433499 0.85630009 0.2745333 ]
|
||||
[0.60826013 0.94413158 0.32305813 0.27011455 0.38636467 0.02568505
|
||||
0.0610602 0.92275042 0.09189927 0.67177934]
|
||||
[0.72097126 0.35819312 0.98060869 0.46573851 0.70482138 0.51749473
|
||||
0.17012048 0.53752634 0.66726905 0.08874264]
|
||||
[0.79974359 0.05023564 0.84150451 0.05491301 0.90643349 0.75505597
|
||||
0.7300372 0.57445493 0.39626502 0.22016672]
|
||||
[0.44463741 0.01448144 0.01167264 0.85595734 0.09989861 0.81007682
|
||||
0.30844205 0.68075712 0.9067362 0.96317459]
|
||||
[0.87327422 0.12086283 0.50504743 0.60715594 0.9440887 0.54041662
|
||||
0.7316348 0.42976761 0.63933182 0.22772039]]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -2033,13 +2043,13 @@ covariance matrix through the <strong>np.linalg.eig()</strong> function.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>-0.19542686939553625
|
||||
3.3929303193461995
|
||||
-0.3886211842840971
|
||||
[[ 0.96552318 2.67354673 2.55640013]
|
||||
[ 2.67354673 8.3318361 7.41646135]
|
||||
[ 2.55640013 7.41646135 11.19890453]]
|
||||
[18.10686259 0.09289204 2.29650918]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>-0.05279147850537411
|
||||
3.9698377974429513
|
||||
0.22403397554517984
|
||||
[[ 1.03173163 3.03859372 3.15084741]
|
||||
[ 3.03859372 9.96514849 9.33589475]
|
||||
[ 3.15084741 9.33589475 15.3141149 ]]
|
||||
[23.20313883 0.08502083 3.02283536]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -2262,7 +2272,7 @@ Name: Aragorn, dtype: object
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20702/1326197715.py:6: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16293/1326197715.py:6: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
|
||||
data_pandas=data_pandas.append(pd.DataFrame(new_hobbit, index=['Pippin']))
|
||||
</pre></div>
|
||||
</div>
|
||||
|
||||
@@ -55,6 +55,7 @@ const thebe_selector_output = ".output, .cell_output"
|
||||
<script defer="defer" src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
|
||||
<link rel="index" title="Index" href="genindex.html" />
|
||||
<link rel="search" title="Search" href="search.html" />
|
||||
<link rel="next" title="Exercises week 36" href="exercisesweek36.html" />
|
||||
<link rel="prev" title="Exercises week 35" href="exercisesweek35.html" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
||||
<meta name="docsearch:language" content="None">
|
||||
@@ -267,6 +268,16 @@ const thebe_selector_output = ".output, .cell_output"
|
||||
Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="exercisesweek36.html">
|
||||
Exercises week 36
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week36.html">
|
||||
Week 36: Statistical interpretation of Linear Regression and Resampling techniques
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</div>
|
||||
@@ -1613,7 +1624,7 @@ Since we are not using <strong>Scikit-Learn</strong> here we can define our own
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.9949347794612255
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.9959033816551833
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1630,7 +1641,7 @@ Since we are not using <strong>Scikit-Learn</strong> here we can define our own
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.01028995427580139
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.009290411029763584
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1645,31 +1656,23 @@ Since we are not using <strong>Scikit-Learn</strong> here we can define our own
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[3.27961873e-03 1.75320666e-02 3.97840267e-02 1.41369795e-02
|
||||
1.12890007e-02 3.55765644e-02 1.34914442e-03 9.92463154e-03
|
||||
3.86416637e-02 8.40062254e-03 1.58912508e-02 7.41091395e-02
|
||||
2.28266792e-02 1.29662593e-03 1.47977394e-02 1.48877744e-02
|
||||
3.81874188e-02 7.92322465e-02 3.43012535e-02 1.47732343e-02
|
||||
2.59567145e-02 1.77816763e-02 1.07826171e-02 3.45241645e-02
|
||||
3.01752962e-02 4.99180856e-03 2.49838167e-04 4.38432672e-03
|
||||
5.93034176e-03 2.54644646e-02 1.76834212e-02 1.85785878e-02
|
||||
3.70827971e-02 9.47008799e-03 6.04008203e-02 2.45865699e-02
|
||||
2.43113284e-03 2.65643756e-02 1.40768645e-01 4.45840226e-03
|
||||
1.23365145e-02 3.49001938e-02 3.65693278e-02 2.08966600e-02
|
||||
1.45112450e-02 1.27524966e-02 7.26193455e-02 3.00129543e-02
|
||||
5.95594445e-03 1.06826023e-02 1.32609560e-02 1.42729886e-02
|
||||
2.87131961e-02 6.39892216e-02 4.19650622e-02 2.78272242e-02
|
||||
1.19023414e-02 2.32474290e-02 5.36904006e-02 7.48259891e-03
|
||||
2.74997869e-02 3.08549564e-02 4.12211324e-02 1.06229862e-02
|
||||
4.41816468e-02 5.87890188e-04 4.41486312e-02 1.41083418e-02
|
||||
5.35759569e-03 2.89869600e-02 2.20517254e-02 2.65534935e-02
|
||||
1.04481443e-02 2.47355725e-02 1.00634741e-02 2.73185363e-02
|
||||
1.13375152e-02 4.40236659e-02 1.20669787e-02 7.63258368e-02
|
||||
1.13208672e-04 1.15746031e-02 9.91094169e-03 4.09789363e-02
|
||||
5.98604688e-02 1.47082726e-02 1.54957609e-02 3.71921767e-02
|
||||
2.85688243e-03 7.15752533e-04 4.77553346e-02 2.33370315e-03
|
||||
3.30638512e-05 7.19284552e-02 5.16049171e-02 7.79000942e-03
|
||||
2.70465373e-02 5.84696315e-02 5.47099765e-02 6.36462687e-02]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[0.02126934 0.07602623 0.0014937 0.01294617 0.05421908 0.02237669
|
||||
0.0026081 0.00564956 0.00399247 0.05763394 0.00040953 0.06861085
|
||||
0.0098629 0.01199845 0.0097577 0.03260201 0.020964 0.01933058
|
||||
0.02255062 0.01801774 0.04039087 0.00472303 0.01003763 0.01487742
|
||||
0.05554042 0.00886682 0.05110883 0.02944194 0.00806407 0.01028231
|
||||
0.03613949 0.03352185 0.0512238 0.01525206 0.00660801 0.01073938
|
||||
0.06353697 0.00700232 0.0391902 0.08741274 0.01227458 0.01049472
|
||||
0.04691549 0.00963223 0.0143088 0.05177527 0.00850988 0.01121347
|
||||
0.02768957 0.02259051 0.02233576 0.01322543 0.02143332 0.01400329
|
||||
0.00102864 0.01322099 0.00611932 0.01011376 0.13281267 0.00684221
|
||||
0.05358851 0.02232779 0.00695738 0.03054765 0.00554475 0.05748797
|
||||
0.03507211 0.00563446 0.03123832 0.00033779 0.01122997 0.1098906
|
||||
0.07003926 0.03718926 0.0695405 0.00605451 0.0456042 0.00477722
|
||||
0.01224109 0.01072866 0.04273116 0.01873409 0.02903947 0.01927709
|
||||
0.00819724 0.00628788 0.00086553 0.02341603 0.0525063 0.03546779
|
||||
0.03012368 0.05069808 0.00327082 0.00517074 0.00071305 0.01194406
|
||||
0.05454172 0.02480935 0.00577016 0.02925853]
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1738,15 +1741,15 @@ but now splitting the data into a training set and a test set.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[ 1.96667129 1.00622755 0.25455904 7.36570403 -3.55965719]
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[ 2.04860436 -0.39444293 5.97533203 -0.78980112 0.09575221]
|
||||
Training R2
|
||||
0.9967912002709148
|
||||
0.9960913291755783
|
||||
Training MSE
|
||||
0.007098683295819775
|
||||
0.008823125370709272
|
||||
Test R2
|
||||
0.9957763445154583
|
||||
0.9906601091977738
|
||||
Test MSE
|
||||
0.008754614401844201
|
||||
0.01969004537138309
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -4221,6 +4224,13 @@ C(\boldsymbol{X},\boldsymbol{\beta})=\frac{1}{n}\left\{(\boldsymbol{y}-\boldsymb
|
||||
<p class="prev-next-title">Exercises week 35</p>
|
||||
</div>
|
||||
</a>
|
||||
<a class='right-next' id="next-link" href="exercisesweek36.html" title="next page">
|
||||
<div class="prev-next-info">
|
||||
<p class="prev-next-subtitle">next</p>
|
||||
<p class="prev-next-title">Exercises week 36</p>
|
||||
</div>
|
||||
<i class="fas fa-angle-right"></i>
|
||||
</a>
|
||||
</div>
|
||||
|
||||
</div>
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because one or more lines are too long
@@ -1077,7 +1077,7 @@
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
" return 1/(1 + np.exp(-x))\n"
|
||||
]
|
||||
},
|
||||
@@ -1655,7 +1655,7 @@
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
" return 1/(1 + np.exp(-x))\n"
|
||||
]
|
||||
},
|
||||
@@ -1673,7 +1673,7 @@
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
" return 1/(1 + np.exp(-x))\n"
|
||||
]
|
||||
},
|
||||
@@ -1691,7 +1691,7 @@
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
" return 1/(1 + np.exp(-x))\n"
|
||||
]
|
||||
},
|
||||
@@ -1709,7 +1709,7 @@
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
" return 1/(1 + np.exp(-x))\n"
|
||||
]
|
||||
},
|
||||
@@ -1727,7 +1727,7 @@
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
" return 1/(1 + np.exp(-x))\n"
|
||||
]
|
||||
},
|
||||
@@ -1745,7 +1745,7 @@
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
" return 1/(1 + np.exp(-x))\n"
|
||||
]
|
||||
},
|
||||
@@ -1763,7 +1763,7 @@
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
" return 1/(1 + np.exp(-x))\n"
|
||||
]
|
||||
},
|
||||
@@ -1781,11 +1781,11 @@
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
" return 1/(1 + np.exp(-x))\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n",
|
||||
" exp_term = np.exp(self.z_o)\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide\n",
|
||||
" self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)\n"
|
||||
]
|
||||
},
|
||||
@@ -1803,11 +1803,11 @@
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
" return 1/(1 + np.exp(-x))\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n",
|
||||
" exp_term = np.exp(self.z_o)\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide\n",
|
||||
" self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)\n"
|
||||
]
|
||||
},
|
||||
@@ -1825,11 +1825,11 @@
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
" return 1/(1 + np.exp(-x))\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n",
|
||||
" exp_term = np.exp(self.z_o)\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide\n",
|
||||
" self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)\n"
|
||||
]
|
||||
},
|
||||
@@ -1847,11 +1847,11 @@
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
" return 1/(1 + np.exp(-x))\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n",
|
||||
" exp_term = np.exp(self.z_o)\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide\n",
|
||||
" self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)\n"
|
||||
]
|
||||
},
|
||||
@@ -1869,11 +1869,11 @@
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
" return 1/(1 + np.exp(-x))\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n",
|
||||
" exp_term = np.exp(self.z_o)\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide\n",
|
||||
" self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)\n"
|
||||
]
|
||||
},
|
||||
@@ -1891,7 +1891,7 @@
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
" return 1/(1 + np.exp(-x))\n"
|
||||
]
|
||||
},
|
||||
@@ -1909,11 +1909,11 @@
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
" return 1/(1 + np.exp(-x))\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n",
|
||||
" exp_term = np.exp(self.z_o)\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide\n",
|
||||
" self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)\n"
|
||||
]
|
||||
},
|
||||
@@ -1931,11 +1931,11 @@
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
" return 1/(1 + np.exp(-x))\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n",
|
||||
" exp_term = np.exp(self.z_o)\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide\n",
|
||||
" self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)\n"
|
||||
]
|
||||
},
|
||||
@@ -1953,11 +1953,11 @@
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
" return 1/(1 + np.exp(-x))\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n",
|
||||
" exp_term = np.exp(self.z_o)\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide\n",
|
||||
" self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)\n"
|
||||
]
|
||||
},
|
||||
@@ -1975,11 +1975,11 @@
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
" return 1/(1 + np.exp(-x))\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n",
|
||||
" exp_term = np.exp(self.z_o)\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide\n",
|
||||
" self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)\n"
|
||||
]
|
||||
},
|
||||
@@ -1997,11 +1997,11 @@
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
" return 1/(1 + np.exp(-x))\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n",
|
||||
" exp_term = np.exp(self.z_o)\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide\n",
|
||||
" self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)\n"
|
||||
]
|
||||
},
|
||||
@@ -2019,11 +2019,11 @@
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
" return 1/(1 + np.exp(-x))\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n",
|
||||
" exp_term = np.exp(self.z_o)\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide\n",
|
||||
" self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)\n"
|
||||
]
|
||||
},
|
||||
@@ -2041,11 +2041,11 @@
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
" return 1/(1 + np.exp(-x))\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n",
|
||||
" exp_term = np.exp(self.z_o)\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide\n",
|
||||
" self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)\n"
|
||||
]
|
||||
},
|
||||
@@ -2063,11 +2063,11 @@
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
" return 1/(1 + np.exp(-x))\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:43: RuntimeWarning: overflow encountered in exp\n",
|
||||
" exp_term = np.exp(self.z_o)\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/1630775253.py:44: RuntimeWarning: invalid value encountered in true_divide\n",
|
||||
" self.probabilities = exp_term / np.sum(exp_term, axis=1, keepdims=True)\n"
|
||||
]
|
||||
},
|
||||
@@ -2128,15 +2128,15 @@
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
" return 1/(1 + np.exp(-x))\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
" return 1/(1 + np.exp(-x))\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
" return 1/(1 + np.exp(-x))\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
" return 1/(1 + np.exp(-x))\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20528/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16145/953065564.py:4: RuntimeWarning: overflow encountered in exp\n",
|
||||
" return 1/(1 + np.exp(-x))\n"
|
||||
]
|
||||
},
|
||||
@@ -2669,34 +2669,24 @@
|
||||
"Learning rate = 0.01\n",
|
||||
"Lambda = 0.1\n",
|
||||
"Accuracy score on test set: 0.9888888888888889\n",
|
||||
"\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"\n",
|
||||
"Learning rate = 0.01\n",
|
||||
"Lambda = 1.0\n",
|
||||
"Accuracy score on test set: 0.9722222222222222\n",
|
||||
"\n",
|
||||
"\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Learning rate = 0.01\n",
|
||||
"Lambda = 10.0\n",
|
||||
"Accuracy score on test set: 0.9527777777777777\n",
|
||||
"\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"\n",
|
||||
"Learning rate = 0.1\n",
|
||||
"Lambda = 1e-05\n",
|
||||
"Accuracy score on test set: 0.9027777777777778\n",
|
||||
"\n",
|
||||
"Learning rate = 0.1\n",
|
||||
"Lambda = 0.0001\n",
|
||||
"Accuracy score on test set: 0.8583333333333333\n",
|
||||
"\n"
|
||||
]
|
||||
},
|
||||
@@ -2704,6 +2694,10 @@
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Learning rate = 0.1\n",
|
||||
"Lambda = 0.0001\n",
|
||||
"Accuracy score on test set: 0.8583333333333333\n",
|
||||
"\n",
|
||||
"Learning rate = 0.1\n",
|
||||
"Lambda = 0.001\n",
|
||||
"Accuracy score on test set: 0.8722222222222222\n",
|
||||
@@ -2721,20 +2715,10 @@
|
||||
"Learning rate = 0.1\n",
|
||||
"Lambda = 0.1\n",
|
||||
"Accuracy score on test set: 0.8805555555555555\n",
|
||||
"\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"\n",
|
||||
"Learning rate = 0.1\n",
|
||||
"Lambda = 1.0\n",
|
||||
"Accuracy score on test set: 0.8722222222222222\n",
|
||||
"\n",
|
||||
"Learning rate = 0.1\n",
|
||||
"Lambda = 10.0\n",
|
||||
"Accuracy score on test set: 0.8666666666666667\n",
|
||||
"\n"
|
||||
]
|
||||
},
|
||||
@@ -2742,6 +2726,10 @@
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Learning rate = 0.1\n",
|
||||
"Lambda = 10.0\n",
|
||||
"Accuracy score on test set: 0.8666666666666667\n",
|
||||
"\n",
|
||||
"Learning rate = 1.0\n",
|
||||
"Lambda = 1e-05\n",
|
||||
"Accuracy score on test set: 0.08611111111111111\n",
|
||||
@@ -2749,10 +2737,6 @@
|
||||
"Learning rate = 1.0\n",
|
||||
"Lambda = 0.0001\n",
|
||||
"Accuracy score on test set: 0.10555555555555556\n",
|
||||
"\n",
|
||||
"Learning rate = 1.0\n",
|
||||
"Lambda = 0.001\n",
|
||||
"Accuracy score on test set: 0.10555555555555556\n",
|
||||
"\n"
|
||||
]
|
||||
},
|
||||
@@ -2760,6 +2744,10 @@
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Learning rate = 1.0\n",
|
||||
"Lambda = 0.001\n",
|
||||
"Accuracy score on test set: 0.10555555555555556\n",
|
||||
"\n",
|
||||
"Learning rate = 1.0\n",
|
||||
"Lambda = 0.01\n",
|
||||
"Accuracy score on test set: 0.17777777777777778\n",
|
||||
@@ -2785,6 +2773,10 @@
|
||||
"Learning rate = 10.0\n",
|
||||
"Lambda = 1e-05\n",
|
||||
"Accuracy score on test set: 0.17222222222222222\n",
|
||||
"\n",
|
||||
"Learning rate = 10.0\n",
|
||||
"Lambda = 0.0001\n",
|
||||
"Accuracy score on test set: 0.11666666666666667\n",
|
||||
"\n"
|
||||
]
|
||||
},
|
||||
@@ -2792,10 +2784,6 @@
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Learning rate = 10.0\n",
|
||||
"Lambda = 0.0001\n",
|
||||
"Accuracy score on test set: 0.11666666666666667\n",
|
||||
"\n",
|
||||
"Learning rate = 10.0\n",
|
||||
"Lambda = 0.001\n",
|
||||
"Accuracy score on test set: 0.10555555555555556\n",
|
||||
|
||||
@@ -3029,24 +3029,18 @@
|
||||
"File \u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/wrap_util.py:20\u001b[0m, in \u001b[0;36munary_to_nary.<locals>.nary_operator.<locals>.nary_f\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m 18\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m 19\u001b[0m x \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mtuple\u001b[39m(args[i] \u001b[38;5;28;01mfor\u001b[39;00m i \u001b[38;5;129;01min\u001b[39;00m argnum)\n\u001b[0;32m---> 20\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43munary_operator\u001b[49m\u001b[43m(\u001b[49m\u001b[43munary_f\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mx\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mnary_op_args\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mnary_op_kwargs\u001b[49m\u001b[43m)\u001b[49m\n",
|
||||
"File \u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/differential_operators.py:78\u001b[0m, in \u001b[0;36mhessian\u001b[0;34m(fun, x)\u001b[0m\n\u001b[1;32m 75\u001b[0m \u001b[38;5;129m@unary_to_nary\u001b[39m\n\u001b[1;32m 76\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mhessian\u001b[39m(fun, x):\n\u001b[1;32m 77\u001b[0m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mReturns a function that computes the exact Hessian.\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[0;32m---> 78\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mjacobian\u001b[49m\u001b[43m(\u001b[49m\u001b[43mjacobian\u001b[49m\u001b[43m(\u001b[49m\u001b[43mfun\u001b[49m\u001b[43m)\u001b[49m\u001b[43m)\u001b[49m\u001b[43m(\u001b[49m\u001b[43mx\u001b[49m\u001b[43m)\u001b[49m\n",
|
||||
"File \u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/wrap_util.py:20\u001b[0m, in \u001b[0;36munary_to_nary.<locals>.nary_operator.<locals>.nary_f\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m 18\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m 19\u001b[0m x \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mtuple\u001b[39m(args[i] \u001b[38;5;28;01mfor\u001b[39;00m i \u001b[38;5;129;01min\u001b[39;00m argnum)\n\u001b[0;32m---> 20\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43munary_operator\u001b[49m\u001b[43m(\u001b[49m\u001b[43munary_f\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mx\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mnary_op_args\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mnary_op_kwargs\u001b[49m\u001b[43m)\u001b[49m\n",
|
||||
"File \u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/differential_operators.py:57\u001b[0m, in \u001b[0;36mjacobian\u001b[0;34m(fun, x)\u001b[0m\n\u001b[1;32m 47\u001b[0m \u001b[38;5;129m@unary_to_nary\u001b[39m\n\u001b[1;32m 48\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mjacobian\u001b[39m(fun, x):\n\u001b[1;32m 49\u001b[0m \u001b[38;5;124;03m\"\"\"\u001b[39;00m\n\u001b[1;32m 50\u001b[0m \u001b[38;5;124;03m Returns a function which computes the Jacobian of `fun` with respect to\u001b[39;00m\n\u001b[1;32m 51\u001b[0m \u001b[38;5;124;03m positional argument number `argnum`, which must be a scalar or array. Unlike\u001b[39;00m\n\u001b[0;32m (...)\u001b[0m\n\u001b[1;32m 55\u001b[0m \u001b[38;5;124;03m (out1, out2, ...) then the Jacobian has shape (out1, out2, ..., in1, in2, ...).\u001b[39;00m\n\u001b[1;32m 56\u001b[0m \u001b[38;5;124;03m \"\"\"\u001b[39;00m\n\u001b[0;32m---> 57\u001b[0m vjp, ans \u001b[38;5;241m=\u001b[39m \u001b[43m_make_vjp\u001b[49m\u001b[43m(\u001b[49m\u001b[43mfun\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mx\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 58\u001b[0m ans_vspace \u001b[38;5;241m=\u001b[39m vspace(ans)\n\u001b[1;32m 59\u001b[0m jacobian_shape \u001b[38;5;241m=\u001b[39m ans_vspace\u001b[38;5;241m.\u001b[39mshape \u001b[38;5;241m+\u001b[39m vspace(x)\u001b[38;5;241m.\u001b[39mshape\n",
|
||||
"File \u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/core.py:10\u001b[0m, in \u001b[0;36mmake_vjp\u001b[0;34m(fun, x)\u001b[0m\n\u001b[1;32m 8\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mmake_vjp\u001b[39m(fun, x):\n\u001b[1;32m 9\u001b[0m start_node \u001b[38;5;241m=\u001b[39m VJPNode\u001b[38;5;241m.\u001b[39mnew_root()\n\u001b[0;32m---> 10\u001b[0m end_value, end_node \u001b[38;5;241m=\u001b[39m \u001b[43mtrace\u001b[49m\u001b[43m(\u001b[49m\u001b[43mstart_node\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mfun\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mx\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 11\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m end_node \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[1;32m 12\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mvjp\u001b[39m(g): \u001b[38;5;28;01mreturn\u001b[39;00m vspace(x)\u001b[38;5;241m.\u001b[39mzeros()\n",
|
||||
"File \u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/tracer.py:10\u001b[0m, in \u001b[0;36mtrace\u001b[0;34m(start_node, fun, x)\u001b[0m\n\u001b[1;32m 8\u001b[0m \u001b[38;5;28;01mwith\u001b[39;00m trace_stack\u001b[38;5;241m.\u001b[39mnew_trace() \u001b[38;5;28;01mas\u001b[39;00m t:\n\u001b[1;32m 9\u001b[0m start_box \u001b[38;5;241m=\u001b[39m new_box(x, t, start_node)\n\u001b[0;32m---> 10\u001b[0m end_box \u001b[38;5;241m=\u001b[39m \u001b[43mfun\u001b[49m\u001b[43m(\u001b[49m\u001b[43mstart_box\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 11\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m isbox(end_box) \u001b[38;5;129;01mand\u001b[39;00m end_box\u001b[38;5;241m.\u001b[39m_trace \u001b[38;5;241m==\u001b[39m start_box\u001b[38;5;241m.\u001b[39m_trace:\n\u001b[1;32m 12\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m end_box\u001b[38;5;241m.\u001b[39m_value, end_box\u001b[38;5;241m.\u001b[39m_node\n",
|
||||
"File \u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/wrap_util.py:15\u001b[0m, in \u001b[0;36munary_to_nary.<locals>.nary_operator.<locals>.nary_f.<locals>.unary_f\u001b[0;34m(x)\u001b[0m\n\u001b[1;32m 13\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m 14\u001b[0m subargs \u001b[38;5;241m=\u001b[39m subvals(args, \u001b[38;5;28mzip\u001b[39m(argnum, x))\n\u001b[0;32m---> 15\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mfun\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43msubargs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n",
|
||||
"File \u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/wrap_util.py:20\u001b[0m, in \u001b[0;36munary_to_nary.<locals>.nary_operator.<locals>.nary_f\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m 18\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m 19\u001b[0m x \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mtuple\u001b[39m(args[i] \u001b[38;5;28;01mfor\u001b[39;00m i \u001b[38;5;129;01min\u001b[39;00m argnum)\n\u001b[0;32m---> 20\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43munary_operator\u001b[49m\u001b[43m(\u001b[49m\u001b[43munary_f\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mx\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mnary_op_args\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mnary_op_kwargs\u001b[49m\u001b[43m)\u001b[49m\n",
|
||||
"File \u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/differential_operators.py:61\u001b[0m, in \u001b[0;36mjacobian\u001b[0;34m(fun, x)\u001b[0m\n\u001b[1;32m 59\u001b[0m jacobian_shape \u001b[38;5;241m=\u001b[39m ans_vspace\u001b[38;5;241m.\u001b[39mshape \u001b[38;5;241m+\u001b[39m vspace(x)\u001b[38;5;241m.\u001b[39mshape\n\u001b[1;32m 60\u001b[0m grads \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mmap\u001b[39m(vjp, ans_vspace\u001b[38;5;241m.\u001b[39mstandard_basis())\n\u001b[0;32m---> 61\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m np\u001b[38;5;241m.\u001b[39mreshape(\u001b[43mnp\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mstack\u001b[49m\u001b[43m(\u001b[49m\u001b[43mgrads\u001b[49m\u001b[43m)\u001b[49m, jacobian_shape)\n",
|
||||
"File \u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/numpy/numpy_wrapper.py:88\u001b[0m, in \u001b[0;36mstack\u001b[0;34m(arrays, axis)\u001b[0m\n\u001b[1;32m 83\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mstack\u001b[39m(arrays, axis\u001b[38;5;241m=\u001b[39m\u001b[38;5;241m0\u001b[39m):\n\u001b[1;32m 84\u001b[0m \u001b[38;5;66;03m# this code is basically copied from numpy/core/shape_base.py's stack\u001b[39;00m\n\u001b[1;32m 85\u001b[0m \u001b[38;5;66;03m# we need it here because we want to re-implement stack in terms of the\u001b[39;00m\n\u001b[1;32m 86\u001b[0m \u001b[38;5;66;03m# primitives defined in this file\u001b[39;00m\n\u001b[0;32m---> 88\u001b[0m arrays \u001b[38;5;241m=\u001b[39m [array(arr) \u001b[38;5;28;01mfor\u001b[39;00m arr \u001b[38;5;129;01min\u001b[39;00m arrays]\n\u001b[1;32m 89\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m arrays:\n\u001b[1;32m 90\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mValueError\u001b[39;00m(\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mneed at least one array to stack\u001b[39m\u001b[38;5;124m'\u001b[39m)\n",
|
||||
"File \u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/numpy/numpy_wrapper.py:88\u001b[0m, in \u001b[0;36m<listcomp>\u001b[0;34m(.0)\u001b[0m\n\u001b[1;32m 83\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mstack\u001b[39m(arrays, axis\u001b[38;5;241m=\u001b[39m\u001b[38;5;241m0\u001b[39m):\n\u001b[1;32m 84\u001b[0m \u001b[38;5;66;03m# this code is basically copied from numpy/core/shape_base.py's stack\u001b[39;00m\n\u001b[1;32m 85\u001b[0m \u001b[38;5;66;03m# we need it here because we want to re-implement stack in terms of the\u001b[39;00m\n\u001b[1;32m 86\u001b[0m \u001b[38;5;66;03m# primitives defined in this file\u001b[39;00m\n\u001b[0;32m---> 88\u001b[0m arrays \u001b[38;5;241m=\u001b[39m [array(arr) \u001b[38;5;28;01mfor\u001b[39;00m arr \u001b[38;5;129;01min\u001b[39;00m arrays]\n\u001b[1;32m 89\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m arrays:\n\u001b[1;32m 90\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mValueError\u001b[39;00m(\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mneed at least one array to stack\u001b[39m\u001b[38;5;124m'\u001b[39m)\n",
|
||||
"File \u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/core.py:14\u001b[0m, in \u001b[0;36mmake_vjp.<locals>.vjp\u001b[0;34m(g)\u001b[0m\n\u001b[0;32m---> 14\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mvjp\u001b[39m(g): \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mbackward_pass\u001b[49m\u001b[43m(\u001b[49m\u001b[43mg\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mend_node\u001b[49m\u001b[43m)\u001b[49m\n",
|
||||
"File \u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/core.py:21\u001b[0m, in \u001b[0;36mbackward_pass\u001b[0;34m(g, end_node)\u001b[0m\n\u001b[1;32m 19\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m node \u001b[38;5;129;01min\u001b[39;00m toposort(end_node):\n\u001b[1;32m 20\u001b[0m outgrad \u001b[38;5;241m=\u001b[39m outgrads\u001b[38;5;241m.\u001b[39mpop(node)\n\u001b[0;32m---> 21\u001b[0m ingrads \u001b[38;5;241m=\u001b[39m \u001b[43mnode\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mvjp\u001b[49m\u001b[43m(\u001b[49m\u001b[43moutgrad\u001b[49m\u001b[43m[\u001b[49m\u001b[38;5;241;43m0\u001b[39;49m\u001b[43m]\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 22\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m parent, ingrad \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mzip\u001b[39m(node\u001b[38;5;241m.\u001b[39mparents, ingrads):\n\u001b[1;32m 23\u001b[0m outgrads[parent] \u001b[38;5;241m=\u001b[39m add_outgrads(outgrads\u001b[38;5;241m.\u001b[39mget(parent), ingrad)\n",
|
||||
"File \u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/core.py:67\u001b[0m, in \u001b[0;36mdefvjp.<locals>.vjp_argnums.<locals>.<lambda>\u001b[0;34m(g)\u001b[0m\n\u001b[1;32m 64\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mNotImplementedError\u001b[39;00m(\n\u001b[1;32m 65\u001b[0m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mVJP of \u001b[39m\u001b[38;5;132;01m{}\u001b[39;00m\u001b[38;5;124m wrt argnum 0 not defined\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;241m.\u001b[39mformat(fun\u001b[38;5;241m.\u001b[39m\u001b[38;5;18m__name__\u001b[39m))\n\u001b[1;32m 66\u001b[0m vjp \u001b[38;5;241m=\u001b[39m vjpfun(ans, \u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs)\n\u001b[0;32m---> 67\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;01mlambda\u001b[39;00m g: (\u001b[43mvjp\u001b[49m\u001b[43m(\u001b[49m\u001b[43mg\u001b[49m\u001b[43m)\u001b[49m,)\n\u001b[1;32m 68\u001b[0m \u001b[38;5;28;01melif\u001b[39;00m L \u001b[38;5;241m==\u001b[39m \u001b[38;5;241m2\u001b[39m:\n\u001b[1;32m 69\u001b[0m argnum_0, argnum_1 \u001b[38;5;241m=\u001b[39m argnums\n",
|
||||
"File \u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/numpy/numpy_vjps.py:82\u001b[0m, in \u001b[0;36m<lambda>\u001b[0;34m(g)\u001b[0m\n\u001b[1;32m 80\u001b[0m defvjp(anp\u001b[38;5;241m.\u001b[39mlog10, \u001b[38;5;28;01mlambda\u001b[39;00m ans, x : \u001b[38;5;28;01mlambda\u001b[39;00m g: g \u001b[38;5;241m/\u001b[39m x \u001b[38;5;241m/\u001b[39m anp\u001b[38;5;241m.\u001b[39mlog(\u001b[38;5;241m10\u001b[39m))\n\u001b[1;32m 81\u001b[0m defvjp(anp\u001b[38;5;241m.\u001b[39mlog1p, \u001b[38;5;28;01mlambda\u001b[39;00m ans, x : \u001b[38;5;28;01mlambda\u001b[39;00m g: g \u001b[38;5;241m/\u001b[39m (x \u001b[38;5;241m+\u001b[39m \u001b[38;5;241m1\u001b[39m))\n\u001b[0;32m---> 82\u001b[0m defvjp(anp\u001b[38;5;241m.\u001b[39msin, \u001b[38;5;28;01mlambda\u001b[39;00m ans, x : \u001b[38;5;28;01mlambda\u001b[39;00m g: \u001b[43mg\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43m \u001b[49m\u001b[43manp\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mcos\u001b[49m\u001b[43m(\u001b[49m\u001b[43mx\u001b[49m\u001b[43m)\u001b[49m)\n\u001b[1;32m 83\u001b[0m defvjp(anp\u001b[38;5;241m.\u001b[39mcos, \u001b[38;5;28;01mlambda\u001b[39;00m ans, x : \u001b[38;5;28;01mlambda\u001b[39;00m g: \u001b[38;5;241m-\u001b[39m g \u001b[38;5;241m*\u001b[39m anp\u001b[38;5;241m.\u001b[39msin(x))\n\u001b[1;32m 84\u001b[0m defvjp(anp\u001b[38;5;241m.\u001b[39mtan, \u001b[38;5;28;01mlambda\u001b[39;00m ans, x : \u001b[38;5;28;01mlambda\u001b[39;00m g: g \u001b[38;5;241m/\u001b[39m anp\u001b[38;5;241m.\u001b[39mcos(x) \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39m\u001b[38;5;241m2\u001b[39m)\n",
|
||||
"File \u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/numpy/numpy_boxes.py:27\u001b[0m, in \u001b[0;36mArrayBox.__mul__\u001b[0;34m(self, other)\u001b[0m\n\u001b[0;32m---> 27\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21m__mul__\u001b[39m(\u001b[38;5;28mself\u001b[39m, other): \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43manp\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mmultiply\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mother\u001b[49m\u001b[43m)\u001b[49m\n",
|
||||
"File \u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/core.py:78\u001b[0m, in \u001b[0;36mdefvjp.<locals>.vjp_argnums.<locals>.<lambda>\u001b[0;34m(g)\u001b[0m\n\u001b[1;32m 76\u001b[0m vjp_0 \u001b[38;5;241m=\u001b[39m vjp_0_fun(ans, \u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs)\n\u001b[1;32m 77\u001b[0m vjp_1 \u001b[38;5;241m=\u001b[39m vjp_1_fun(ans, \u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs)\n\u001b[0;32m---> 78\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;01mlambda\u001b[39;00m g: (vjp_0(g), \u001b[43mvjp_1\u001b[49m\u001b[43m(\u001b[49m\u001b[43mg\u001b[49m\u001b[43m)\u001b[49m)\n\u001b[1;32m 79\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m 80\u001b[0m vjps \u001b[38;5;241m=\u001b[39m [vjps_dict[argnum](ans, \u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs) \u001b[38;5;28;01mfor\u001b[39;00m argnum \u001b[38;5;129;01min\u001b[39;00m argnums]\n",
|
||||
"File \u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/numpy/numpy_vjps.py:660\u001b[0m, in \u001b[0;36munbroadcast_f.<locals>.<lambda>\u001b[0;34m(g)\u001b[0m\n\u001b[1;32m 658\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21munbroadcast_f\u001b[39m(target, f):\n\u001b[1;32m 659\u001b[0m target_meta \u001b[38;5;241m=\u001b[39m anp\u001b[38;5;241m.\u001b[39mmetadata(target)\n\u001b[0;32m--> 660\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;01mlambda\u001b[39;00m g: unbroadcast(\u001b[43mf\u001b[49m\u001b[43m(\u001b[49m\u001b[43mg\u001b[49m\u001b[43m)\u001b[49m, target_meta)\n",
|
||||
"File \u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/numpy/numpy_vjps.py:35\u001b[0m, in \u001b[0;36m<lambda>\u001b[0;34m(g)\u001b[0m\n\u001b[1;32m 30\u001b[0m \u001b[38;5;66;03m# ----- Binary ufuncs -----\u001b[39;00m\n\u001b[1;32m 32\u001b[0m defvjp(anp\u001b[38;5;241m.\u001b[39madd, \u001b[38;5;28;01mlambda\u001b[39;00m ans, x, y : unbroadcast_f(x, \u001b[38;5;28;01mlambda\u001b[39;00m g: g),\n\u001b[1;32m 33\u001b[0m \u001b[38;5;28;01mlambda\u001b[39;00m ans, x, y : unbroadcast_f(y, \u001b[38;5;28;01mlambda\u001b[39;00m g: g))\n\u001b[1;32m 34\u001b[0m defvjp(anp\u001b[38;5;241m.\u001b[39mmultiply, \u001b[38;5;28;01mlambda\u001b[39;00m ans, x, y : unbroadcast_f(x, \u001b[38;5;28;01mlambda\u001b[39;00m g: y \u001b[38;5;241m*\u001b[39m g),\n\u001b[0;32m---> 35\u001b[0m \u001b[38;5;28;01mlambda\u001b[39;00m ans, x, y : unbroadcast_f(y, \u001b[38;5;28;01mlambda\u001b[39;00m g: \u001b[43mx\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43m \u001b[49m\u001b[43mg\u001b[49m))\n\u001b[1;32m 36\u001b[0m defvjp(anp\u001b[38;5;241m.\u001b[39msubtract, \u001b[38;5;28;01mlambda\u001b[39;00m ans, x, y : unbroadcast_f(x, \u001b[38;5;28;01mlambda\u001b[39;00m g: g),\n\u001b[1;32m 37\u001b[0m \u001b[38;5;28;01mlambda\u001b[39;00m ans, x, y : unbroadcast_f(y, \u001b[38;5;28;01mlambda\u001b[39;00m g: \u001b[38;5;241m-\u001b[39mg))\n\u001b[1;32m 38\u001b[0m defvjp(anp\u001b[38;5;241m.\u001b[39mdivide, \u001b[38;5;28;01mlambda\u001b[39;00m ans, x, y : unbroadcast_f(x, \u001b[38;5;28;01mlambda\u001b[39;00m g: g \u001b[38;5;241m/\u001b[39m y),\n\u001b[1;32m 39\u001b[0m \u001b[38;5;28;01mlambda\u001b[39;00m ans, x, y : unbroadcast_f(y, \u001b[38;5;28;01mlambda\u001b[39;00m g: \u001b[38;5;241m-\u001b[39m g \u001b[38;5;241m*\u001b[39m x \u001b[38;5;241m/\u001b[39m y\u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39m\u001b[38;5;241m2\u001b[39m))\n",
|
||||
"File \u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/numpy/numpy_boxes.py:35\u001b[0m, in \u001b[0;36mArrayBox.__rmul__\u001b[0;34m(self, other)\u001b[0m\n\u001b[0;32m---> 35\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21m__rmul__\u001b[39m(\u001b[38;5;28mself\u001b[39m, other): \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43manp\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mmultiply\u001b[49m\u001b[43m(\u001b[49m\u001b[43mother\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m)\u001b[49m\n",
|
||||
"File \u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/tracer.py:45\u001b[0m, in \u001b[0;36mprimitive.<locals>.f_wrapped\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m 43\u001b[0m argnums \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mtuple\u001b[39m(argnum \u001b[38;5;28;01mfor\u001b[39;00m argnum, _ \u001b[38;5;129;01min\u001b[39;00m boxed_args)\n\u001b[1;32m 44\u001b[0m ans \u001b[38;5;241m=\u001b[39m f_wrapped(\u001b[38;5;241m*\u001b[39margvals, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs)\n\u001b[0;32m---> 45\u001b[0m node \u001b[38;5;241m=\u001b[39m \u001b[43mnode_constructor\u001b[49m\u001b[43m(\u001b[49m\u001b[43mans\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mf_wrapped\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43margvals\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mkwargs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43margnums\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mparents\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 46\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m new_box(ans, trace, node)\n\u001b[1;32m 47\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n",
|
||||
"File \u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/core.py:36\u001b[0m, in \u001b[0;36mVJPNode.__init__\u001b[0;34m(self, value, fun, args, kwargs, parent_argnums, parents)\u001b[0m\n\u001b[1;32m 33\u001b[0m fun_name \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mgetattr\u001b[39m(fun, \u001b[38;5;124m'\u001b[39m\u001b[38;5;124m__name__\u001b[39m\u001b[38;5;124m'\u001b[39m, fun)\n\u001b[1;32m 34\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mNotImplementedError\u001b[39;00m(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mVJP of \u001b[39m\u001b[38;5;132;01m{}\u001b[39;00m\u001b[38;5;124m wrt argnums \u001b[39m\u001b[38;5;132;01m{}\u001b[39;00m\u001b[38;5;124m not defined\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 35\u001b[0m \u001b[38;5;241m.\u001b[39mformat(fun_name, parent_argnums))\n\u001b[0;32m---> 36\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mvjp \u001b[38;5;241m=\u001b[39m \u001b[43mvjpmaker\u001b[49m\u001b[43m(\u001b[49m\u001b[43mparent_argnums\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mvalue\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n",
|
||||
"File \u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/core.py:76\u001b[0m, in \u001b[0;36mdefvjp.<locals>.vjp_argnums\u001b[0;34m(argnums, ans, args, kwargs)\u001b[0m\n\u001b[1;32m 73\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mKeyError\u001b[39;00m:\n\u001b[1;32m 74\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mNotImplementedError\u001b[39;00m(\n\u001b[1;32m 75\u001b[0m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mVJP of \u001b[39m\u001b[38;5;132;01m{}\u001b[39;00m\u001b[38;5;124m wrt argnums 0, 1 not defined\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;241m.\u001b[39mformat(fun\u001b[38;5;241m.\u001b[39m\u001b[38;5;18m__name__\u001b[39m))\n\u001b[0;32m---> 76\u001b[0m vjp_0 \u001b[38;5;241m=\u001b[39m \u001b[43mvjp_0_fun\u001b[49m\u001b[43m(\u001b[49m\u001b[43mans\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 77\u001b[0m vjp_1 \u001b[38;5;241m=\u001b[39m vjp_1_fun(ans, \u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs)\n\u001b[1;32m 78\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;01mlambda\u001b[39;00m g: (vjp_0(g), vjp_1(g))\n",
|
||||
"File \u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/numpy/numpy_vjps.py:34\u001b[0m, in \u001b[0;36m<lambda>\u001b[0;34m(ans, x, y)\u001b[0m\n\u001b[1;32m 30\u001b[0m \u001b[38;5;66;03m# ----- Binary ufuncs -----\u001b[39;00m\n\u001b[1;32m 32\u001b[0m defvjp(anp\u001b[38;5;241m.\u001b[39madd, \u001b[38;5;28;01mlambda\u001b[39;00m ans, x, y : unbroadcast_f(x, \u001b[38;5;28;01mlambda\u001b[39;00m g: g),\n\u001b[1;32m 33\u001b[0m \u001b[38;5;28;01mlambda\u001b[39;00m ans, x, y : unbroadcast_f(y, \u001b[38;5;28;01mlambda\u001b[39;00m g: g))\n\u001b[0;32m---> 34\u001b[0m defvjp(anp\u001b[38;5;241m.\u001b[39mmultiply, \u001b[38;5;28;01mlambda\u001b[39;00m ans, x, y : \u001b[43munbroadcast_f\u001b[49m\u001b[43m(\u001b[49m\u001b[43mx\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43;01mlambda\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43mg\u001b[49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43my\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43m \u001b[49m\u001b[43mg\u001b[49m\u001b[43m)\u001b[49m,\n\u001b[1;32m 35\u001b[0m \u001b[38;5;28;01mlambda\u001b[39;00m ans, x, y : unbroadcast_f(y, \u001b[38;5;28;01mlambda\u001b[39;00m g: x \u001b[38;5;241m*\u001b[39m g))\n\u001b[1;32m 36\u001b[0m defvjp(anp\u001b[38;5;241m.\u001b[39msubtract, \u001b[38;5;28;01mlambda\u001b[39;00m ans, x, y : unbroadcast_f(x, \u001b[38;5;28;01mlambda\u001b[39;00m g: g),\n\u001b[1;32m 37\u001b[0m \u001b[38;5;28;01mlambda\u001b[39;00m ans, x, y : unbroadcast_f(y, \u001b[38;5;28;01mlambda\u001b[39;00m g: \u001b[38;5;241m-\u001b[39mg))\n\u001b[1;32m 38\u001b[0m defvjp(anp\u001b[38;5;241m.\u001b[39mdivide, \u001b[38;5;28;01mlambda\u001b[39;00m ans, x, y : unbroadcast_f(x, \u001b[38;5;28;01mlambda\u001b[39;00m g: g \u001b[38;5;241m/\u001b[39m y),\n\u001b[1;32m 39\u001b[0m \u001b[38;5;28;01mlambda\u001b[39;00m ans, x, y : unbroadcast_f(y, \u001b[38;5;28;01mlambda\u001b[39;00m g: \u001b[38;5;241m-\u001b[39m g \u001b[38;5;241m*\u001b[39m x \u001b[38;5;241m/\u001b[39m y\u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39m\u001b[38;5;241m2\u001b[39m))\n",
|
||||
"File \u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/numpy/numpy_vjps.py:659\u001b[0m, in \u001b[0;36munbroadcast_f\u001b[0;34m(target, f)\u001b[0m\n\u001b[1;32m 658\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21munbroadcast_f\u001b[39m(target, f):\n\u001b[0;32m--> 659\u001b[0m target_meta \u001b[38;5;241m=\u001b[39m \u001b[43manp\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mmetadata\u001b[49m\u001b[43m(\u001b[49m\u001b[43mtarget\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 660\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;01mlambda\u001b[39;00m g: unbroadcast(f(g), target_meta)\n",
|
||||
"File \u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/core.py:56\u001b[0m, in \u001b[0;36mdefvjp.<locals>.vjp_argnums\u001b[0;34m(argnums, ans, args, kwargs)\u001b[0m\n\u001b[1;32m 53\u001b[0m argnums \u001b[38;5;241m=\u001b[39m kwargs\u001b[38;5;241m.\u001b[39mget(\u001b[38;5;124m'\u001b[39m\u001b[38;5;124margnums\u001b[39m\u001b[38;5;124m'\u001b[39m, count())\n\u001b[1;32m 54\u001b[0m vjps_dict \u001b[38;5;241m=\u001b[39m {argnum : translate_vjp(vjpmaker, fun, argnum)\n\u001b[1;32m 55\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m argnum, vjpmaker \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mzip\u001b[39m(argnums, vjpmakers)}\n\u001b[0;32m---> 56\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mvjp_argnums\u001b[39m(argnums, ans, args, kwargs):\n\u001b[1;32m 57\u001b[0m L \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mlen\u001b[39m(argnums)\n\u001b[1;32m 58\u001b[0m \u001b[38;5;66;03m# These first two cases are just optimizations\u001b[39;00m\n",
|
||||
"\u001b[0;31mKeyboardInterrupt\u001b[0m: "
|
||||
]
|
||||
}
|
||||
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 14 KiB After Width: | Height: | Size: 13 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 18 KiB After Width: | Height: | Size: 18 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 26 KiB After Width: | Height: | Size: 25 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 18 KiB After Width: | Height: | Size: 17 KiB |
@@ -1798,10 +1798,10 @@
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"0.008947823579448178\n",
|
||||
"4.14822021080294\n",
|
||||
"[[0.73947737 2.16006961]\n",
|
||||
" [2.16006961 7.24782786]]\n"
|
||||
"0.01354598394614281\n",
|
||||
"4.037503978471253\n",
|
||||
"[[0.95927495 2.85834701]\n",
|
||||
" [2.85834701 9.70233292]]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
@@ -1845,10 +1845,10 @@
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"0.08657894597048958\n",
|
||||
"2.219222942590059\n",
|
||||
"[[1. 0.6343356]\n",
|
||||
" [0.6343356 1. ]]\n"
|
||||
"0.09309965420717024\n",
|
||||
"1.6546329613199204\n",
|
||||
"[[1. 0.57867898]\n",
|
||||
" [0.57867898 1. ]]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
@@ -1905,30 +1905,30 @@
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"[[ 0.29724806 0.26804287]\n",
|
||||
" [-0.15626984 -1.36853738]\n",
|
||||
" [-0.77070756 -2.13536532]\n",
|
||||
" [-0.45372697 -3.1582408 ]\n",
|
||||
" [ 0.52580392 2.72567956]\n",
|
||||
" [-0.86515815 -1.35704388]\n",
|
||||
" [-0.73738602 -2.12933164]\n",
|
||||
" [-0.10486183 1.06292011]\n",
|
||||
" [ 1.75670484 5.27381733]\n",
|
||||
" [ 0.50835355 0.81805914]]\n",
|
||||
"[[-0.49352496 -2.38394242]\n",
|
||||
" [ 0.18849928 0.73454039]\n",
|
||||
" [-1.58104393 -5.16350368]\n",
|
||||
" [ 0.34695383 0.23472346]\n",
|
||||
" [ 0.95953339 2.95819409]\n",
|
||||
" [ 1.31331481 3.59914165]\n",
|
||||
" [ 0.14846308 1.1180677 ]\n",
|
||||
" [ 0.26022531 0.23496851]\n",
|
||||
" [-0.12178678 -0.11868087]\n",
|
||||
" [-1.02063403 -1.21350883]]\n",
|
||||
" 0 1\n",
|
||||
"0 0.297248 0.268043\n",
|
||||
"1 -0.156270 -1.368537\n",
|
||||
"2 -0.770708 -2.135365\n",
|
||||
"3 -0.453727 -3.158241\n",
|
||||
"4 0.525804 2.725680\n",
|
||||
"5 -0.865158 -1.357044\n",
|
||||
"6 -0.737386 -2.129332\n",
|
||||
"7 -0.104862 1.062920\n",
|
||||
"8 1.756705 5.273817\n",
|
||||
"9 0.508354 0.818059\n",
|
||||
"0 -0.493525 -2.383942\n",
|
||||
"1 0.188499 0.734540\n",
|
||||
"2 -1.581044 -5.163504\n",
|
||||
"3 0.346954 0.234723\n",
|
||||
"4 0.959533 2.958194\n",
|
||||
"5 1.313315 3.599142\n",
|
||||
"6 0.148463 1.118068\n",
|
||||
"7 0.260225 0.234969\n",
|
||||
"8 -0.121787 -0.118681\n",
|
||||
"9 -1.020634 -1.213509\n",
|
||||
" 0 1\n",
|
||||
"0 1.000000 0.915549\n",
|
||||
"1 0.915549 1.000000\n"
|
||||
"0 1.000000 0.949087\n",
|
||||
"1 0.949087 1.000000\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
@@ -1974,37 +1974,37 @@
|
||||
"text": [
|
||||
" 0 1 2 3 4 5 6 7 \\\n",
|
||||
"0 0.0 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 \n",
|
||||
"1 0.0 0.086925 0.087377 0.087520 0.087878 0.088164 0.079680 0.079580 \n",
|
||||
"2 0.0 0.087377 0.089240 0.088074 0.089063 0.089924 0.079827 0.080060 \n",
|
||||
"3 0.0 0.087520 0.088074 0.094227 0.094252 0.094139 0.089441 0.088974 \n",
|
||||
"4 0.0 0.087878 0.089063 0.094252 0.094610 0.094812 0.089040 0.088778 \n",
|
||||
"5 0.0 0.088164 0.089924 0.094139 0.094812 0.095315 0.088488 0.088425 \n",
|
||||
"6 0.0 0.079680 0.079827 0.089441 0.089040 0.088488 0.087315 0.086524 \n",
|
||||
"7 0.0 0.079580 0.080060 0.088974 0.088778 0.088425 0.086524 0.085876 \n",
|
||||
"8 0.0 0.079499 0.080295 0.088502 0.088506 0.088348 0.085716 0.085210 \n",
|
||||
"9 0.0 0.079448 0.080548 0.088037 0.088238 0.088275 0.084906 0.084541 \n",
|
||||
"10 0.0 0.071751 0.071438 0.082839 0.082089 0.081194 0.082509 0.081481 \n",
|
||||
"11 0.0 0.071392 0.071284 0.082139 0.081533 0.080780 0.081559 0.080641 \n",
|
||||
"12 0.0 0.071071 0.071164 0.081471 0.081007 0.080395 0.080635 0.079827 \n",
|
||||
"13 0.0 0.070794 0.071084 0.080839 0.080518 0.080048 0.079741 0.079043 \n",
|
||||
"14 0.0 0.070562 0.071051 0.080249 0.080070 0.079745 0.078880 0.078293 \n",
|
||||
"1 0.0 0.072254 0.074732 0.075050 0.075400 0.075727 0.068297 0.068215 \n",
|
||||
"2 0.0 0.074732 0.078265 0.076259 0.077107 0.077954 0.068418 0.068622 \n",
|
||||
"3 0.0 0.075050 0.076259 0.082604 0.082322 0.081988 0.078133 0.077663 \n",
|
||||
"4 0.0 0.075400 0.077107 0.082322 0.082315 0.082269 0.077387 0.077091 \n",
|
||||
"5 0.0 0.075727 0.077954 0.081988 0.082269 0.082525 0.076570 0.076454 \n",
|
||||
"6 0.0 0.068297 0.068418 0.078133 0.077387 0.076570 0.075952 0.075224 \n",
|
||||
"7 0.0 0.068215 0.068622 0.077663 0.077091 0.076454 0.075224 0.074613 \n",
|
||||
"8 0.0 0.068168 0.068876 0.077210 0.076818 0.076371 0.074495 0.074004 \n",
|
||||
"9 0.0 0.068163 0.069190 0.076775 0.076573 0.076324 0.073764 0.073398 \n",
|
||||
"10 0.0 0.060924 0.060401 0.071600 0.070601 0.069522 0.071006 0.070141 \n",
|
||||
"11 0.0 0.060711 0.060369 0.071122 0.070241 0.069283 0.070364 0.069583 \n",
|
||||
"12 0.0 0.060534 0.060381 0.070671 0.069911 0.069080 0.069738 0.069042 \n",
|
||||
"13 0.0 0.060394 0.060441 0.070245 0.069612 0.068912 0.069125 0.068517 \n",
|
||||
"14 0.0 0.060291 0.060550 0.069845 0.069343 0.068782 0.068524 0.068007 \n",
|
||||
"\n",
|
||||
" 8 9 10 11 12 13 14 \n",
|
||||
"0 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 \n",
|
||||
"1 0.079499 0.079448 0.071751 0.071392 0.071071 0.070794 0.070562 \n",
|
||||
"2 0.080295 0.080548 0.071438 0.071284 0.071164 0.071084 0.071051 \n",
|
||||
"3 0.088502 0.088037 0.082839 0.082139 0.081471 0.080839 0.080249 \n",
|
||||
"4 0.088506 0.088238 0.082089 0.081533 0.081007 0.080518 0.080070 \n",
|
||||
"5 0.088348 0.088275 0.081194 0.080780 0.080395 0.080048 0.079745 \n",
|
||||
"6 0.085716 0.084906 0.082509 0.081559 0.080635 0.079741 0.078880 \n",
|
||||
"7 0.085210 0.084541 0.081481 0.080641 0.079827 0.079043 0.078293 \n",
|
||||
"8 0.084685 0.084157 0.080434 0.079704 0.078999 0.078326 0.077688 \n",
|
||||
"9 0.084157 0.083772 0.079379 0.078759 0.078165 0.077604 0.077079 \n",
|
||||
"10 0.080434 0.079379 0.079152 0.078033 0.076935 0.075863 0.074818 \n",
|
||||
"11 0.079704 0.078759 0.078033 0.077004 0.075996 0.075014 0.074061 \n",
|
||||
"12 0.078999 0.078165 0.076935 0.075996 0.075079 0.074187 0.073325 \n",
|
||||
"13 0.078326 0.077604 0.075863 0.075014 0.074187 0.073388 0.072618 \n",
|
||||
"14 0.077688 0.077079 0.074818 0.074061 0.073325 0.072618 0.071942 \n"
|
||||
"1 0.068168 0.068163 0.060924 0.060711 0.060534 0.060394 0.060291 \n",
|
||||
"2 0.068876 0.069190 0.060401 0.060369 0.060381 0.060441 0.060550 \n",
|
||||
"3 0.077210 0.076775 0.071600 0.071122 0.070671 0.070245 0.069845 \n",
|
||||
"4 0.076818 0.076573 0.070601 0.070241 0.069911 0.069612 0.069343 \n",
|
||||
"5 0.076371 0.076324 0.069522 0.069283 0.069080 0.068912 0.068782 \n",
|
||||
"6 0.074495 0.073764 0.071006 0.070364 0.069738 0.069125 0.068524 \n",
|
||||
"7 0.074004 0.073398 0.070141 0.069583 0.069042 0.068517 0.068007 \n",
|
||||
"8 0.073520 0.073044 0.069265 0.068792 0.068339 0.067905 0.067489 \n",
|
||||
"9 0.073044 0.072705 0.068375 0.067990 0.067628 0.067288 0.066969 \n",
|
||||
"10 0.069265 0.068375 0.067400 0.066672 0.065952 0.065237 0.064526 \n",
|
||||
"11 0.068792 0.067990 0.066672 0.066006 0.065350 0.064701 0.064057 \n",
|
||||
"12 0.068339 0.067628 0.065952 0.065350 0.064759 0.064176 0.063600 \n",
|
||||
"13 0.067905 0.067288 0.065237 0.064701 0.064176 0.063661 0.063155 \n",
|
||||
"14 0.067489 0.066969 0.064526 0.064057 0.063600 0.063155 0.062721 \n"
|
||||
]
|
||||
}
|
||||
],
|
||||
|
||||
File diff suppressed because one or more lines are too long
Binary file not shown.
|
Before Width: | Height: | Size: 22 KiB After Width: | Height: | Size: 22 KiB |
File diff suppressed because one or more lines are too long
Binary file not shown.
|
Before Width: | Height: | Size: 52 KiB After Width: | Height: | Size: 46 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 37 KiB After Width: | Height: | Size: 39 KiB |
File diff suppressed because one or more lines are too long
Binary file not shown.
|
Before Width: | Height: | Size: 36 KiB After Width: | Height: | Size: 40 KiB |
File diff suppressed because one or more lines are too long
Binary file not shown.
|
Before Width: | Height: | Size: 23 KiB After Width: | Height: | Size: 23 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 21 KiB After Width: | Height: | Size: 22 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 22 KiB After Width: | Height: | Size: 21 KiB |
@@ -0,0 +1,393 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "50b2fbbe",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)\n",
|
||||
"doconce format html exercisesweek36.do.txt -->\n",
|
||||
"<!-- dom:TITLE: Exercises week 36 -->"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "da488c0c",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"# Exercises week 36\n",
|
||||
"**September 4-8, 2023**\n",
|
||||
"\n",
|
||||
"Date: **Deadline is Sunday September 10 at midnight**"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "b84d6b13",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"## Overarching aims of the exercises this week\n",
|
||||
"\n",
|
||||
"This set of exercises form an important part of the first project. The\n",
|
||||
"analytical exercises deal with the material covered last week on the\n",
|
||||
"mathematical interpretations of ordinary least squares and of Ridge\n",
|
||||
"regression. The numerical exercises can be seen as a continuation of\n",
|
||||
"exercise 3 from week 35, with the inclusion of Ridge regression. This\n",
|
||||
"material enters also the discussions of the first project."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "96c9c28e",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"## Exercise 1: Analytical exercises\n",
|
||||
"\n",
|
||||
"The aim here is to derive the expression for the optimal parameters\n",
|
||||
"using Ridge regression. Furthermore, using the singular value\n",
|
||||
"decomposition, we will analyze the difference between the ordinary\n",
|
||||
"least squares approach and Ridge regression.\n",
|
||||
"\n",
|
||||
"The expression for the standard Mean Squared Error (MSE) which we used to define our cost function and the equations for the ordinary least squares (OLS) method, was given by the\n",
|
||||
"optimization problem"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "439f1456",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"{\\displaystyle \\min_{\\boldsymbol{\\beta}\\in {\\mathbb{R}}^{p}}}\\frac{1}{n}\\left\\{\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\right)^T\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\right)\\right\\}.\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "b51e09f7",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"which we can also write as"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "02c45981",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"{\\displaystyle \\min_{\\boldsymbol{\\beta}\\in\n",
|
||||
"{\\mathbb{R}}^{p}}}\\frac{1}{n}\\sum_{i=0}^{n-1}\\left(y_i-\\tilde{y}_i\\right)^2=\\frac{1}{n}\\vert\\vert \\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\vert\\vert_2^2,\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "cc0e91ea",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"where we have used the definition of a norm-2 vector, that is"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "b5805f35",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"\\vert\\vert \\boldsymbol{x}\\vert\\vert_2 = \\sqrt{\\sum_i x_i^2}.\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "6e3095bf",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"By minimizing the above equation with respect to the parameters\n",
|
||||
"$\\boldsymbol{\\beta}$ we could then obtain an analytical expression for the\n",
|
||||
"parameters $\\boldsymbol{\\beta}$.\n",
|
||||
"\n",
|
||||
"We can add a regularization parameter $\\lambda$ by\n",
|
||||
"defining a new cost function to be optimized, that is"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "da90fe04",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"{\\displaystyle \\min_{\\boldsymbol{\\beta}\\in\n",
|
||||
"{\\mathbb{R}}^{p}}}\\frac{1}{n}\\vert\\vert \\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\vert\\vert_2^2+\\lambda\\vert\\vert \\boldsymbol{\\beta}\\vert\\vert_2^2\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "1a106e07",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"which leads to the Ridge regression minimization problem. One can require as part of the optimization problem \n",
|
||||
"that $\\vert\\vert \\boldsymbol{\\beta}\\vert\\vert_2^2\\le t$, where $t$ is\n",
|
||||
"a finite number larger than zero. We will not implement that here."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "3917877b",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"### a) Expression for Ridge regression\n",
|
||||
"\n",
|
||||
"Show that the optimal parameters"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "78226f28",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"\\hat{\\boldsymbol{\\beta}}_{\\mathrm{Ridge}} = \\left(\\boldsymbol{X}^T\\boldsymbol{X}+\\lambda\\boldsymbol{I}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y},\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "951dfffa",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"with $\\boldsymbol{I}$ being a $p\\times p$ identity matrix with the constraint that"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "21d2770e",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"\\sum_{i=0}^{p-1} \\beta_i^2 \\leq t,\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "ec212498",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"with $t$ a finite positive number. \n",
|
||||
"\n",
|
||||
"The ordinary least squares result is"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "4ffabf6c",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"\\hat{\\boldsymbol{\\beta}}_{\\mathrm{OLS}} = \\left(\\boldsymbol{X}^T\\boldsymbol{X}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y},\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "f97a6f45",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"### b) The singular value decomposition\n",
|
||||
"\n",
|
||||
"Use the singular value decomposition of an n\\times p$ matrix $\\boldsymbol{X}$ (our design matrix)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "8761ed23",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"\\boldsymbol{X}=\\boldsymbol{U}\\boldsymbol{\\Sigma}\\boldsymbol{V}^T,\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "92f8479e",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"where $\\boldsymbol{U}$ and $\\boldsymbol{V}$ are orthogonal matrices of dimensions\n",
|
||||
"$n\\times n$ and $p\\times p$, respectively, and $\\boldsymbol{\\Sigma}$ is an\n",
|
||||
"$n\\times p$ matrix which contains the ingular values only. This material was discussed during the lectures of week 35.\n",
|
||||
"\n",
|
||||
"Show that you can write the \n",
|
||||
"OLS solutions in terms of the eigenvectors (the columns) of the orthogonal matrix $\\boldsymbol{U}$ as"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "9df91bda",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"\\tilde{\\boldsymbol{y}}_{\\mathrm{OLS}}=\\boldsymbol{X}\\boldsymbol{\\beta} = \\sum_{j=0}^{p-1}\\boldsymbol{u}_j\\boldsymbol{u}_j^T\\boldsymbol{y}.\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "e6de0312",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"For Ridge regression, show that the corresponding equation is"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "8e09d132",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"\\tilde{\\boldsymbol{y}}_{\\mathrm{Ridge}}=\\boldsymbol{X}\\boldsymbol{\\beta}_{\\mathrm{Ridge}} = \\boldsymbol{U\\Sigma V^T}\\left(\\boldsymbol{V}\\boldsymbol{\\Sigma}^2\\boldsymbol{V}^T+\\lambda\\boldsymbol{I} \\right)^{-1}(\\boldsymbol{U\\Sigma V^T})^T\\boldsymbol{y}=\\sum_{j=0}^{p-1}\\boldsymbol{u}_j\\boldsymbol{u}_j^T\\frac{\\sigma_j^2}{\\sigma_j^2+\\lambda}\\boldsymbol{y},\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "a9c924ab",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"with the vectors $\\boldsymbol{u}_j$ being the columns of $\\boldsymbol{U}$ from the SVD of the matrix $\\boldsymbol{X}$. \n",
|
||||
"\n",
|
||||
"Give an interpretation of the results. [Section 3.4 of Hastie et al's textbook gives a good discussion of the above results](https://link.springer.com/book/10.1007/978-0-387-84858-7)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "3b9328a1",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"## Exercise 2: Adding Ridge Regression\n",
|
||||
"\n",
|
||||
"This exercise is a continuation of exercise 3 from week 35, see <https://compphysics.github.io/MachineLearning/doc/LectureNotes/_build/html/exercisesweek35.html>. We will use the same function to\n",
|
||||
"generate our data set, still staying with a simple function $y(x)$\n",
|
||||
"which we want to fit using linear regression, but now extending the\n",
|
||||
"analysis to include the Ridge regression method.\n",
|
||||
"\n",
|
||||
"In this exercise you need to include the same elements from last week, that is\n",
|
||||
"1. scale your data by subtracting the mean value from each column in the design matrix.\n",
|
||||
"\n",
|
||||
"2. perform a split of the data in a training set and a test set.\n",
|
||||
"\n",
|
||||
"The addition to the analysis this time is the introduction of the hyperparameter $\\lambda$ when introducing Ridge regression.\n",
|
||||
"\n",
|
||||
"Extend the code from exercise 3 from [week 35](https://compphysics.github.io/MachineLearning/doc/LectureNotes/_build/html/exercisesweek35.html) to include Ridge regression with the hyperparameter $\\lambda$. The optimal parameters $\\hat{\\beta}$ for Ridge regression can be obtained by matrix inversion in a similar way as done for ordinary least squares. You need to add to your code the following equations"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "5b54b7b4",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"\\hat{\\boldsymbol{\\beta}}_{\\mathrm{Ridge}} = \\left(\\boldsymbol{X}^T\\boldsymbol{X}+\\lambda\\boldsymbol{I}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y}.\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "a7ebe31b",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"The ordinary least squares result you encoded last week is given by"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "874e0dd3",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"\\hat{\\boldsymbol{\\beta}}_{\\mathrm{OLS}} = \\left(\\boldsymbol{X}^T\\boldsymbol{X}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y},\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "9adcc39f",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"Use these results to compute the mean squared error for ordinary least\n",
|
||||
"squares and Ridge regression first for a polynomial of degree five\n",
|
||||
"with $n=100$ data points and five selected values of\n",
|
||||
"$\\lambda=[0.0001,0.001, 0.01,0.1,1.0]$. Compute thereafter the mean\n",
|
||||
"squared error for the same values of $\\lambda$ for polynomials of degree ten\n",
|
||||
"and $15$. Discuss your results for the training MSE and test MSE with\n",
|
||||
"Ridge regression and ordinary least squares."
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
@@ -225,8 +225,8 @@
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"[-0.38763091 -2.70501534 -0.3581571 -0.96251494 -1.26223899 0.35309734\n",
|
||||
" 2.28186376 -1.85104809 -0.37114298 -1.20893188]\n"
|
||||
"[-0.87136737 1.4300745 -0.3322326 -0.66758934 -1.00283636 -0.27625974\n",
|
||||
" 1.89249454 -0.25006757 0.73565195 0.33697405]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
@@ -662,26 +662,26 @@
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"[[0.78459267 0.75453081 0.05363779 0.57350724 0.69764852 0.65795279\n",
|
||||
" 0.51507839 0.20461136 0.38788697 0.96496641]\n",
|
||||
" [0.25028968 0.96081861 0.18931988 0.51108791 0.30337713 0.43036842\n",
|
||||
" 0.52839842 0.15321987 0.78561443 0.09030825]\n",
|
||||
" [0.10097956 0.50584526 0.34989509 0.55626454 0.69154964 0.2895238\n",
|
||||
" 0.13393141 0.15503141 0.26015755 0.42902155]\n",
|
||||
" [0.25789255 0.9492866 0.90252116 0.904221 0.51933924 0.14432948\n",
|
||||
" 0.54445121 0.02699523 0.18657863 0.971688 ]\n",
|
||||
" [0.13392097 0.27801122 0.50931378 0.04234339 0.22442417 0.44065609\n",
|
||||
" 0.74943449 0.42451192 0.33736485 0.97952271]\n",
|
||||
" [0.95824108 0.59950055 0.91346044 0.58042237 0.13228567 0.31519573\n",
|
||||
" 0.12427889 0.64736858 0.60236782 0.18036103]\n",
|
||||
" [0.95911004 0.82027884 0.27547877 0.84317815 0.89842298 0.68322599\n",
|
||||
" 0.02377668 0.39943328 0.00162091 0.0525221 ]\n",
|
||||
" [0.94076632 0.88335933 0.75492292 0.7860324 0.41923956 0.86181269\n",
|
||||
" 0.45979894 0.44190304 0.07829878 0.00458014]\n",
|
||||
" [0.42915006 0.68127341 0.23875722 0.31988705 0.54956992 0.24801014\n",
|
||||
" 0.65335632 0.97713364 0.05635863 0.12160173]\n",
|
||||
" [0.93386901 0.74935095 0.96534137 0.98400474 0.98581925 0.30313128\n",
|
||||
" 0.41386599 0.88450476 0.87099757 0.22566121]]\n"
|
||||
"[[0.52428467 0.89421873 0.57286194 0.35473061 0.34626037 0.77640601\n",
|
||||
" 0.60707801 0.60515083 0.41335011 0.16061448]\n",
|
||||
" [0.51508374 0.7651899 0.07657669 0.14045566 0.92863147 0.32541271\n",
|
||||
" 0.62761922 0.20965182 0.55051175 0.14072037]\n",
|
||||
" [0.70311568 0.03617972 0.22335823 0.45797073 0.2223808 0.86208133\n",
|
||||
" 0.64452882 0.0770789 0.7866024 0.45739087]\n",
|
||||
" [0.38469441 0.16105443 0.09873132 0.21913656 0.34561469 0.521986\n",
|
||||
" 0.89261458 0.75628425 0.25070907 0.96504592]\n",
|
||||
" [0.40506137 0.48241947 0.32158804 0.45112054 0.5783575 0.3267061\n",
|
||||
" 0.96261297 0.25216021 0.94271589 0.60651016]\n",
|
||||
" [0.91410101 0.48217977 0.39116042 0.99397049 0.80768431 0.68650491\n",
|
||||
" 0.04184636 0.5174581 0.86144422 0.46794929]\n",
|
||||
" [0.62405149 0.27498198 0.63550707 0.85902603 0.38644889 0.86858926\n",
|
||||
" 0.68282653 0.65554133 0.81051173 0.00281835]\n",
|
||||
" [0.66001778 0.64858537 0.90034533 0.62389761 0.5333734 0.75390337\n",
|
||||
" 0.97926642 0.9893405 0.61605739 0.51905011]\n",
|
||||
" [0.59277468 0.52678301 0.68347072 0.76707201 0.08821204 0.55220861\n",
|
||||
" 0.14648477 0.17606773 0.59609892 0.83723539]\n",
|
||||
" [0.40961677 0.07325301 0.34652523 0.72201591 0.66250644 0.6367311\n",
|
||||
" 0.79577619 0.85212606 0.86811137 0.43001767]]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
@@ -800,13 +800,13 @@
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"0.0626202708457115\n",
|
||||
"4.327883798133277\n",
|
||||
"0.3178411477108273\n",
|
||||
"[[ 1.01422853 3.21909039 2.82750687]\n",
|
||||
" [ 3.21909039 11.42694395 9.22887861]\n",
|
||||
" [ 2.82750687 9.22887861 17.2086376 ]]\n",
|
||||
"[24.72855434 0.09533574 4.82592 ]\n"
|
||||
"0.03499693245669077\n",
|
||||
"4.187268182169147\n",
|
||||
"-0.19627430151896047\n",
|
||||
"[[ 1.0622197 3.16232481 2.98018358]\n",
|
||||
" [ 3.16232481 10.48442771 8.58039072]\n",
|
||||
" [ 2.98018358 8.58039072 13.66351927]]\n",
|
||||
"[21.70244042 0.07978179 3.42794448]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
|
||||
File diff suppressed because one or more lines are too long
Binary file not shown.
|
Before Width: | Height: | Size: 10 KiB After Width: | Height: | Size: 9.9 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 18 KiB After Width: | Height: | Size: 19 KiB |
@@ -985,8 +985,8 @@
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"[-0.03483677 -1.06136773 0.56481272 -0.35947075 -1.64972151 -1.58590795\n",
|
||||
" -1.17428293 0.7225597 -1.14061742 -1.22893751]\n"
|
||||
"[-0.75408649 -0.05177092 -0.07339251 1.49372893 -1.21467644 0.24712854\n",
|
||||
" 0.67554302 -0.45143018 0.34212496 0.63262164]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
@@ -1424,26 +1424,26 @@
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"[[0.24279008 0.63112036 0.80947943 0.97509292 0.19425617 0.3957482\n",
|
||||
" 0.1655226 0.83760781 0.07995375 0.76400155]\n",
|
||||
" [0.75621895 0.20893514 0.93082503 0.79419162 0.02783644 0.21296315\n",
|
||||
" 0.64298419 0.34578026 0.60975366 0.46369869]\n",
|
||||
" [0.77859437 0.23477043 0.35438626 0.63115792 0.2460037 0.35568525\n",
|
||||
" 0.0825971 0.94117118 0.14900336 0.30035718]\n",
|
||||
" [0.36692356 0.78972773 0.67655635 0.67160204 0.80108096 0.31507591\n",
|
||||
" 0.21328866 0.41340248 0.3005849 0.40672425]\n",
|
||||
" [0.21922061 0.88274486 0.86572911 0.06486061 0.07565581 0.26678445\n",
|
||||
" 0.03265139 0.22090974 0.33135331 0.66973261]\n",
|
||||
" [0.7221662 0.96941962 0.39707147 0.24929083 0.31531613 0.33079801\n",
|
||||
" 0.06538944 0.42352791 0.94227931 0.27809912]\n",
|
||||
" [0.07195822 0.31719317 0.47248297 0.18264218 0.64033527 0.51146442\n",
|
||||
" 0.49545491 0.91936525 0.81656508 0.78329097]\n",
|
||||
" [0.58666142 0.01646892 0.11029323 0.67442363 0.6914791 0.87902877\n",
|
||||
" 0.98950411 0.27090196 0.08732305 0.89543736]\n",
|
||||
" [0.25253892 0.57505712 0.24848907 0.9631064 0.46312791 0.96431281\n",
|
||||
" 0.28744729 0.09772449 0.17676228 0.51656406]\n",
|
||||
" [0.68664664 0.63281351 0.62806444 0.36809474 0.98129668 0.62914124\n",
|
||||
" 0.37885034 0.6577093 0.57947706 0.24688732]]\n"
|
||||
"[[0.11167579 0.88778091 0.24908246 0.32566092 0.37585988 0.61893735\n",
|
||||
" 0.58574143 0.60081665 0.81065392 0.00509527]\n",
|
||||
" [0.02840884 0.39600607 0.96810393 0.28213741 0.19237495 0.54219245\n",
|
||||
" 0.85581869 0.17928538 0.43185254 0.37624303]\n",
|
||||
" [0.82045496 0.22828694 0.52773891 0.54526769 0.84708672 0.59118929\n",
|
||||
" 0.18221254 0.94640767 0.25328924 0.03192893]\n",
|
||||
" [0.98328759 0.68996179 0.81168086 0.08086066 0.10659682 0.66599246\n",
|
||||
" 0.78035464 0.57403561 0.96995249 0.78990036]\n",
|
||||
" [0.9909606 0.39920034 0.59813773 0.85982436 0.66665241 0.60522226\n",
|
||||
" 0.08889302 0.74433499 0.85630009 0.2745333 ]\n",
|
||||
" [0.60826013 0.94413158 0.32305813 0.27011455 0.38636467 0.02568505\n",
|
||||
" 0.0610602 0.92275042 0.09189927 0.67177934]\n",
|
||||
" [0.72097126 0.35819312 0.98060869 0.46573851 0.70482138 0.51749473\n",
|
||||
" 0.17012048 0.53752634 0.66726905 0.08874264]\n",
|
||||
" [0.79974359 0.05023564 0.84150451 0.05491301 0.90643349 0.75505597\n",
|
||||
" 0.7300372 0.57445493 0.39626502 0.22016672]\n",
|
||||
" [0.44463741 0.01448144 0.01167264 0.85595734 0.09989861 0.81007682\n",
|
||||
" 0.30844205 0.68075712 0.9067362 0.96317459]\n",
|
||||
" [0.87327422 0.12086283 0.50504743 0.60715594 0.9440887 0.54041662\n",
|
||||
" 0.7316348 0.42976761 0.63933182 0.22772039]]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
@@ -1557,13 +1557,13 @@
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"-0.19542686939553625\n",
|
||||
"3.3929303193461995\n",
|
||||
"-0.3886211842840971\n",
|
||||
"[[ 0.96552318 2.67354673 2.55640013]\n",
|
||||
" [ 2.67354673 8.3318361 7.41646135]\n",
|
||||
" [ 2.55640013 7.41646135 11.19890453]]\n",
|
||||
"[18.10686259 0.09289204 2.29650918]\n"
|
||||
"-0.05279147850537411\n",
|
||||
"3.9698377974429513\n",
|
||||
"0.22403397554517984\n",
|
||||
"[[ 1.03173163 3.03859372 3.15084741]\n",
|
||||
" [ 3.03859372 9.96514849 9.33589475]\n",
|
||||
" [ 3.15084741 9.33589475 15.3141149 ]]\n",
|
||||
"[23.20313883 0.08502083 3.02283536]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
@@ -1916,7 +1916,7 @@
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20702/1326197715.py:6: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.\n",
|
||||
"/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_16293/1326197715.py:6: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.\n",
|
||||
" data_pandas=data_pandas.append(pd.DataFrame(new_hobbit, index=['Pippin']))\n"
|
||||
]
|
||||
},
|
||||
|
||||
@@ -1519,7 +1519,7 @@
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"0.9949347794612255\n"
|
||||
"0.9959033816551833\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
@@ -1550,7 +1550,7 @@
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"0.01028995427580139\n"
|
||||
"0.009290411029763584\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
@@ -1585,31 +1585,23 @@
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"[3.27961873e-03 1.75320666e-02 3.97840267e-02 1.41369795e-02\n",
|
||||
" 1.12890007e-02 3.55765644e-02 1.34914442e-03 9.92463154e-03\n",
|
||||
" 3.86416637e-02 8.40062254e-03 1.58912508e-02 7.41091395e-02\n",
|
||||
" 2.28266792e-02 1.29662593e-03 1.47977394e-02 1.48877744e-02\n",
|
||||
" 3.81874188e-02 7.92322465e-02 3.43012535e-02 1.47732343e-02\n",
|
||||
" 2.59567145e-02 1.77816763e-02 1.07826171e-02 3.45241645e-02\n",
|
||||
" 3.01752962e-02 4.99180856e-03 2.49838167e-04 4.38432672e-03\n",
|
||||
" 5.93034176e-03 2.54644646e-02 1.76834212e-02 1.85785878e-02\n",
|
||||
" 3.70827971e-02 9.47008799e-03 6.04008203e-02 2.45865699e-02\n",
|
||||
" 2.43113284e-03 2.65643756e-02 1.40768645e-01 4.45840226e-03\n",
|
||||
" 1.23365145e-02 3.49001938e-02 3.65693278e-02 2.08966600e-02\n",
|
||||
" 1.45112450e-02 1.27524966e-02 7.26193455e-02 3.00129543e-02\n",
|
||||
" 5.95594445e-03 1.06826023e-02 1.32609560e-02 1.42729886e-02\n",
|
||||
" 2.87131961e-02 6.39892216e-02 4.19650622e-02 2.78272242e-02\n",
|
||||
" 1.19023414e-02 2.32474290e-02 5.36904006e-02 7.48259891e-03\n",
|
||||
" 2.74997869e-02 3.08549564e-02 4.12211324e-02 1.06229862e-02\n",
|
||||
" 4.41816468e-02 5.87890188e-04 4.41486312e-02 1.41083418e-02\n",
|
||||
" 5.35759569e-03 2.89869600e-02 2.20517254e-02 2.65534935e-02\n",
|
||||
" 1.04481443e-02 2.47355725e-02 1.00634741e-02 2.73185363e-02\n",
|
||||
" 1.13375152e-02 4.40236659e-02 1.20669787e-02 7.63258368e-02\n",
|
||||
" 1.13208672e-04 1.15746031e-02 9.91094169e-03 4.09789363e-02\n",
|
||||
" 5.98604688e-02 1.47082726e-02 1.54957609e-02 3.71921767e-02\n",
|
||||
" 2.85688243e-03 7.15752533e-04 4.77553346e-02 2.33370315e-03\n",
|
||||
" 3.30638512e-05 7.19284552e-02 5.16049171e-02 7.79000942e-03\n",
|
||||
" 2.70465373e-02 5.84696315e-02 5.47099765e-02 6.36462687e-02]\n"
|
||||
"[0.02126934 0.07602623 0.0014937 0.01294617 0.05421908 0.02237669\n",
|
||||
" 0.0026081 0.00564956 0.00399247 0.05763394 0.00040953 0.06861085\n",
|
||||
" 0.0098629 0.01199845 0.0097577 0.03260201 0.020964 0.01933058\n",
|
||||
" 0.02255062 0.01801774 0.04039087 0.00472303 0.01003763 0.01487742\n",
|
||||
" 0.05554042 0.00886682 0.05110883 0.02944194 0.00806407 0.01028231\n",
|
||||
" 0.03613949 0.03352185 0.0512238 0.01525206 0.00660801 0.01073938\n",
|
||||
" 0.06353697 0.00700232 0.0391902 0.08741274 0.01227458 0.01049472\n",
|
||||
" 0.04691549 0.00963223 0.0143088 0.05177527 0.00850988 0.01121347\n",
|
||||
" 0.02768957 0.02259051 0.02233576 0.01322543 0.02143332 0.01400329\n",
|
||||
" 0.00102864 0.01322099 0.00611932 0.01011376 0.13281267 0.00684221\n",
|
||||
" 0.05358851 0.02232779 0.00695738 0.03054765 0.00554475 0.05748797\n",
|
||||
" 0.03507211 0.00563446 0.03123832 0.00033779 0.01122997 0.1098906\n",
|
||||
" 0.07003926 0.03718926 0.0695405 0.00605451 0.0456042 0.00477722\n",
|
||||
" 0.01224109 0.01072866 0.04273116 0.01873409 0.02903947 0.01927709\n",
|
||||
" 0.00819724 0.00628788 0.00086553 0.02341603 0.0525063 0.03546779\n",
|
||||
" 0.03012368 0.05069808 0.00327082 0.00517074 0.00071305 0.01194406\n",
|
||||
" 0.05454172 0.02480935 0.00577016 0.02925853]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
@@ -1663,15 +1655,15 @@
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"[ 1.96667129 1.00622755 0.25455904 7.36570403 -3.55965719]\n",
|
||||
"[ 2.04860436 -0.39444293 5.97533203 -0.78980112 0.09575221]\n",
|
||||
"Training R2\n",
|
||||
"0.9967912002709148\n",
|
||||
"0.9960913291755783\n",
|
||||
"Training MSE\n",
|
||||
"0.007098683295819775\n",
|
||||
"0.008823125370709272\n",
|
||||
"Test R2\n",
|
||||
"0.9957763445154583\n",
|
||||
"0.9906601091977738\n",
|
||||
"Test MSE\n",
|
||||
"0.008754614401844201\n"
|
||||
"0.01969004537138309\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -45,3 +45,5 @@ parts:
|
||||
- file: week34.ipynb
|
||||
- file: exercisesweek35.ipynb
|
||||
- file: week35.ipynb
|
||||
- file: exercisesweek36.ipynb
|
||||
- file: week36.ipynb
|
||||
|
||||
Binary file not shown.
File diff suppressed because it is too large
Load Diff
@@ -263,7 +263,7 @@ MathJax.Hub.Config({
|
||||
<h2 id="code-for-svd-and-inversion-of-matrices" class="anchor">Code for SVD and Inversion of Matrices </h2>
|
||||
|
||||
<p>How do we use the SVD to invert a matrix \( \boldsymbol{X}^\boldsymbol{X} \) which is singular or near singular?
|
||||
The simple answer is to use the linear algebra function for pseudoinvers, that is
|
||||
The simple answer is to use the linear algebra function for the computation of the pseudoinverse of a given matrix \( \boldsymbol{X} \), that is
|
||||
</p>
|
||||
|
||||
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
|
||||
@@ -272,7 +272,9 @@ The simple answer is to use the linear algebra function for pseudoinvers, that i
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight" style="background: #f8f8f8">
|
||||
<pre style="line-height: 125%;">Ainv <span style="color: #666666">=</span> np<span style="color: #666666">.</span>linlag<span style="color: #666666">.</span>pinv(A)
|
||||
<pre style="line-height: 125%;"><span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">numpy</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">np</span>
|
||||
X <span style="color: #666666">=</span> np<span style="color: #666666">.</span>array( [ [<span style="color: #666666">1</span>,<span style="color: #666666">2</span>,<span style="color: #666666">3</span>],[<span style="color: #666666">2</span>,<span style="color: #666666">4</span>,<span style="color: #666666">5</span>],[<span style="color: #666666">3</span>,<span style="color: #666666">5</span>,<span style="color: #666666">6</span>]])
|
||||
Xinv <span style="color: #666666">=</span> np<span style="color: #666666">.</span>linlag<span style="color: #666666">.</span>pinv(X)
|
||||
</pre>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@@ -366,7 +366,7 @@ values and the column vectors of \( \boldsymbol{V} \).
|
||||
<h2 id="code-for-svd-and-inversion-of-matrices">Code for SVD and Inversion of Matrices </h2>
|
||||
|
||||
<p>How do we use the SVD to invert a matrix \( \boldsymbol{X}^\boldsymbol{X} \) which is singular or near singular?
|
||||
The simple answer is to use the linear algebra function for pseudoinvers, that is
|
||||
The simple answer is to use the linear algebra function for the computation of the pseudoinverse of a given matrix \( \boldsymbol{X} \), that is
|
||||
</p>
|
||||
|
||||
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
|
||||
@@ -375,7 +375,9 @@ The simple answer is to use the linear algebra function for pseudoinvers, that i
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight" style="background: #eeeedd">
|
||||
<pre style="font-size: 80%; line-height: 125%;">Ainv = np.linlag.pinv(A)
|
||||
<pre style="font-size: 80%; line-height: 125%;"><span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">numpy</span> <span style="color: #8B008B; font-weight: bold">as</span> <span style="color: #008b45; text-decoration: underline">np</span>
|
||||
X = np.array( [ [<span style="color: #B452CD">1</span>,<span style="color: #B452CD">2</span>,<span style="color: #B452CD">3</span>],[<span style="color: #B452CD">2</span>,<span style="color: #B452CD">4</span>,<span style="color: #B452CD">5</span>],[<span style="color: #B452CD">3</span>,<span style="color: #B452CD">5</span>,<span style="color: #B452CD">6</span>]])
|
||||
Xinv = np.linlag.pinv(X)
|
||||
</pre>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@@ -371,7 +371,7 @@ values and the column vectors of \( \boldsymbol{V} \).
|
||||
<h2 id="code-for-svd-and-inversion-of-matrices">Code for SVD and Inversion of Matrices </h2>
|
||||
|
||||
<p>How do we use the SVD to invert a matrix \( \boldsymbol{X}^\boldsymbol{X} \) which is singular or near singular?
|
||||
The simple answer is to use the linear algebra function for pseudoinvers, that is
|
||||
The simple answer is to use the linear algebra function for the computation of the pseudoinverse of a given matrix \( \boldsymbol{X} \), that is
|
||||
</p>
|
||||
|
||||
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
|
||||
@@ -380,7 +380,9 @@ The simple answer is to use the linear algebra function for pseudoinvers, that i
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight" style="background: #eeeedd">
|
||||
<pre style="line-height: 125%;">Ainv = np.linlag.pinv(A)
|
||||
<pre style="line-height: 125%;"><span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">numpy</span> <span style="color: #8B008B; font-weight: bold">as</span> <span style="color: #008b45; text-decoration: underline">np</span>
|
||||
X = np.array( [ [<span style="color: #B452CD">1</span>,<span style="color: #B452CD">2</span>,<span style="color: #B452CD">3</span>],[<span style="color: #B452CD">2</span>,<span style="color: #B452CD">4</span>,<span style="color: #B452CD">5</span>],[<span style="color: #B452CD">3</span>,<span style="color: #B452CD">5</span>,<span style="color: #B452CD">6</span>]])
|
||||
Xinv = np.linlag.pinv(X)
|
||||
</pre>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@@ -448,7 +448,7 @@ values and the column vectors of \( \boldsymbol{V} \).
|
||||
<h2 id="code-for-svd-and-inversion-of-matrices">Code for SVD and Inversion of Matrices </h2>
|
||||
|
||||
<p>How do we use the SVD to invert a matrix \( \boldsymbol{X}^\boldsymbol{X} \) which is singular or near singular?
|
||||
The simple answer is to use the linear algebra function for pseudoinvers, that is
|
||||
The simple answer is to use the linear algebra function for the computation of the pseudoinverse of a given matrix \( \boldsymbol{X} \), that is
|
||||
</p>
|
||||
|
||||
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
|
||||
@@ -457,7 +457,9 @@ The simple answer is to use the linear algebra function for pseudoinvers, that i
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class="highlight" style="background: #f8f8f8">
|
||||
<pre style="line-height: 125%;">Ainv <span style="color: #666666">=</span> np<span style="color: #666666">.</span>linlag<span style="color: #666666">.</span>pinv(A)
|
||||
<pre style="line-height: 125%;"><span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">numpy</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">np</span>
|
||||
X <span style="color: #666666">=</span> np<span style="color: #666666">.</span>array( [ [<span style="color: #666666">1</span>,<span style="color: #666666">2</span>,<span style="color: #666666">3</span>],[<span style="color: #666666">2</span>,<span style="color: #666666">4</span>,<span style="color: #666666">5</span>],[<span style="color: #666666">3</span>,<span style="color: #666666">5</span>,<span style="color: #666666">6</span>]])
|
||||
Xinv <span style="color: #666666">=</span> np<span style="color: #666666">.</span>linlag<span style="color: #666666">.</span>pinv(X)
|
||||
</pre>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user