From 7bd40a8c65e6492cdf18e5f6b3bbad330fb1cc49 Mon Sep 17 00:00:00 2001 From: Morten Hjorth-Jensen Date: Mon, 1 Sep 2025 08:51:55 +0200 Subject: [PATCH] update --- .../exercisesweek37-checkpoint.ipynb | 360 ++++++++++++++++++ .../_build/.doctrees/environment.pickle | Bin 264863 -> 264888 bytes .../_build/.doctrees/exercisesweek37.doctree | Bin 31903 -> 32658 bytes .../html/_sources/exercisesweek37.ipynb | 101 +++-- .../_build/html/exercisesweek37.html | 56 +-- doc/LectureNotes/_build/html/searchindex.js | 2 +- .../jupyter_execute/exercisesweek37.ipynb | 101 +++-- doc/LectureNotes/exercisesweek37.ipynb | 101 +++-- doc/src/week37/exercisesweek37.do.txt | 46 +-- 9 files changed, 589 insertions(+), 178 deletions(-) create mode 100644 doc/LectureNotes/.ipynb_checkpoints/exercisesweek37-checkpoint.ipynb diff --git a/doc/LectureNotes/.ipynb_checkpoints/exercisesweek37-checkpoint.ipynb b/doc/LectureNotes/.ipynb_checkpoints/exercisesweek37-checkpoint.ipynb new file mode 100644 index 000000000..68dd47235 --- /dev/null +++ b/doc/LectureNotes/.ipynb_checkpoints/exercisesweek37-checkpoint.ipynb @@ -0,0 +1,360 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "7d56b2d5", + "metadata": { + "editable": true + }, + "source": [ + "\n", + "" + ] + }, + { + "cell_type": "markdown", + "id": "c7a8e9c7", + "metadata": { + "editable": true + }, + "source": [ + "# Exercises week 36\n", + "**Implementing gradient descent for Ridge and ordinary Least Squares Regression**\n", + "\n", + "Date: **September 8-12, 2025**" + ] + }, + { + "cell_type": "markdown", + "id": "cf8f0ecb", + "metadata": { + "editable": true + }, + "source": [ + "## Learning goals\n", + "\n", + "After having completed these exercises you will have:\n", + "1. Your own code for the implementation of the simplest gradient descent approach applied to ordinary least squares (OLS) and Ridge regression\n", + "\n", + "2. Be able to compare the analytical expressions for OLS and Rudge regression with the gradient descent approach\n", + "\n", + "3. Explore the role of the learning rate in the gradient descent approach and the hyperparameter $\\lambda$ in Ridge regression\n", + "\n", + "4. Scale the data properly" + ] + }, + { + "cell_type": "markdown", + "id": "a67ae548", + "metadata": { + "editable": true + }, + "source": [ + "## Ridge regression and a new Synthetic Dataset\n", + "\n", + "We create a synthetic linear regression dataset with a sparse\n", + "underlying relationship. This means we have many features but only a\n", + "few of them actually contribute to the target. In our example, we’ll\n", + "use 10 features with only 3 non-zero weights in the true model. This\n", + "way, the target is generated as a linear combination of a few features\n", + "(with known coefficients) plus some random noise. The steps we include are:\n", + "\n", + "Decide on the number of samples and features (e.g. 100 samples, 10 features).\n", + "Define the **true** coefficient vector with mostly zeros (for sparsity). For example, we set $\\hat{\\boldsymbol{\\theta}} = [5.0, -3.0, 0.0, 0.0, 0.0, 0.0, 2.0, 0.0, 0.0, 0.0]$, meaning only features 0, 1, and 6 have a real effect on y.\n", + "\n", + "Then we sample feature values for $\\boldsymbol{X}$ randomly (e.g. from a normal distribution). We use a normal distribution so features are roughly centered around 0.\n", + "Then we compute the target values $y$ using the linear combination $\\boldsymbol{X}\\hat{\\boldsymbol{\\theta}}$ and add some noise (to simulate measurement error or unexplained variance).\n", + "\n", + "Below is the code to generate the dataset:" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "f2d4a55d", + "metadata": { + "collapsed": false, + "editable": true + }, + "outputs": [], + "source": [ + "import numpy as np\n", + "\n", + "# Set random seed for reproducibility\n", + "np.random.seed(0)\n", + "\n", + "# Define dataset size\n", + "n_samples = 100\n", + "n_features = 10\n", + "\n", + "# Define true coefficients (sparse linear relationship)\n", + "theta_true = np.array([5.0, -3.0, 0.0, 0.0, 0.0, 0.0, 2.0, 0.0, 0.0, 0.0])\n", + "\n", + "# Generate feature matrix X (n_samples x n_features) with random values\n", + "X = np.random.randn(n_samples, n_features) # standard normal distribution\n", + "\n", + "# Generate target values y with a linear combination of X and theta_true, plus noise\n", + "noise = 0.5 * np.random.randn(n_samples) # Gaussian noise\n", + "y = X.dot @ theta_true + noise" + ] + }, + { + "cell_type": "markdown", + "id": "a445583b", + "metadata": { + "editable": true + }, + "source": [ + "This code produces a dataset where only features 0, 1, and 6\n", + "significantly influence $\\boldsymbol{y}$. The rest of the features have zero true\n", + "coefficient, so they only contribute noise. For example, feature 0 has\n", + "a true weight of 5.0, feature 1 has -3.0, and feature 6 has 2.0, so\n", + "the expected relationship is:" + ] + }, + { + "cell_type": "markdown", + "id": "4a81ddf9", + "metadata": { + "editable": true + }, + "source": [ + "$$\n", + "y \\approx 5 \\times X_0 \\;-\\; 3 \\times X_1 \\;+\\; 2 \\times X_6 \\;+\\; \\text{noise}.\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "id": "ae590275", + "metadata": { + "editable": true + }, + "source": [ + "## Exercise 1, scale your data\n", + "\n", + "Before fitting a regression model, it is good practice to normalize or\n", + "standardize the features. This ensures all features are on a\n", + "comparable scale, which is especially important when using\n", + "regularization. Here we will perform standardization, scaling each\n", + "feature to have mean 0 and standard deviation 1:\n", + "\n", + "Compute the mean and standard deviation of each column (feature) in $bm{X}$.\n", + "Subtract the mean and divide by the standard deviation for each feature.\n", + "\n", + "We will also center the target $\\boldsymbol{y}$ to mean $0$. Centering $\\boldsymbol{y}$\n", + "(and each feature) means the model won’t require a separate intercept\n", + "term – the data is shifted such that the intercept is effectively 0\n", + ". (In practice, one could include an intercept in the model and not\n", + "penalize it, but here we simplify by centering.)" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "8b40c47a", + "metadata": { + "collapsed": false, + "editable": true + }, + "outputs": [], + "source": [ + "# Standardize features (zero mean, unit variance for each feature)\n", + "X_mean = X.mean(axis=0)\n", + "X_std = X.std(axis=0)\n", + "X_std[X_std == 0] = 1 # safeguard to avoid division by zero for constant features\n", + "X_norm = (X - X_mean) / X_std\n", + "\n", + "# Center the target to zero mean (optional, to simplify intercept handling)\n", + "y_mean = ?\n", + "y_centered = ?" + ] + }, + { + "cell_type": "markdown", + "id": "ff9c0c81", + "metadata": { + "editable": true + }, + "source": [ + "### 1a)\n", + "\n", + "Fill in the necessary details.\n", + "\n", + "After this preprocessing, each column of $\\boldsymbol{X}_norm$ has mean zero and standard deviation $1$\n", + "and $\\boldsymbol{y}_centered$ has mean 0. This makes the optimization landscape\n", + "nicer and ensures the regularization penalty $\\lambda \\sum_j\n", + "\\beta_j^2$ treats each coefficient fairly (since features are on the\n", + "same scale)." + ] + }, + { + "cell_type": "markdown", + "id": "d27c70e4", + "metadata": { + "editable": true + }, + "source": [ + "## Exercise 2, use the analytical formulae for OLS and Ridge regression to find the optimal paramters $\\boldsymbol{theta}$" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "9f1e5184", + "metadata": { + "collapsed": false, + "editable": true + }, + "outputs": [], + "source": [ + "# Set regularization parameter, either a single value or a vector of values\n", + "lambda = ?\n", + "\n", + "# Analytical form for OLS and Ridge solution: theta_Ridge = (X^T X + lambda * I)^{-1} X^T y and theta_OLS = (X^T X)^{-1} X^T y\n", + "I = np.eye(n_features)\n", + "theta_closed_formRidge = ?\n", + "theta_closed_formOLS = ?\n", + "\n", + "print(\"Closed-form Ridge coefficients:\", theta_closed_form)\n", + "print(\"Closed-form OLS coefficients:\", theta_closed_form)" + ] + }, + { + "cell_type": "markdown", + "id": "2ec556b9", + "metadata": { + "editable": true + }, + "source": [ + "This computes the ridge and OLS regression coefficients directly. The identity\n", + "matrix $I$ has the same size as $X^T X$ (which is n_features x\n", + "n_features), and lam * I adds $\\lambda$ to the diagonal of $X^T X. We\n", + "then invert this matrix and multiply by $X^T y. The result\n", + "for $\\boldsymbol{\\theta}$ is a NumPy array of shape (n_features,) containing the\n", + "fitted weights." + ] + }, + { + "cell_type": "markdown", + "id": "a821f0c5", + "metadata": { + "editable": true + }, + "source": [ + "### 2a)\n", + "\n", + "Finalize the OLS and Ridge regression determination of the optimal parameters $bm{\\theta}$." + ] + }, + { + "cell_type": "markdown", + "id": "d637130e", + "metadata": { + "editable": true + }, + "source": [ + "### 2b)\n", + "\n", + "Explore the results as function of different values of the hyperparameter $\\lambda$. See for example exercise 4 from week 36." + ] + }, + { + "cell_type": "markdown", + "id": "b455ce7e", + "metadata": { + "editable": true + }, + "source": [ + "## Implementing the simplest form for gradient descent\n", + "\n", + "Alternatively, we can fit the ridge regression model using gradient\n", + "descent. This is useful to visualize the iterative convergence and is\n", + "necessary if $n$ and $p$ are so large that the closed-form might be\n", + "too slow or memory-intensive. We derive the gradients from the cost\n", + "functions defined above. Use the gradients of the Ridge and OLS cost functions with respect to\n", + "the parameters $\\boldsymbol{\\theta}$ and set up (using the template below) your own gradient descent code for OLS and Ridge regression.\n", + "\n", + "Below is a template code for gradient descent implementation of ridge:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "cfa1eb29", + "metadata": { + "collapsed": false, + "editable": true + }, + "outputs": [], + "source": [ + "# Gradient descent parameters, learning rate eta first\n", + "eta = 0.1\n", + "# Then number of iterations\n", + "num_iters = 1000\n", + "\n", + "# Initialize weights for gradient descent\n", + "theta = np.zeros(n_features)\n", + "\n", + "# Arrays to store history for plotting\n", + "cost_history = np.zeros(num_iters)\n", + "\n", + "# Gradient descent loop\n", + "m = n_samples # number of examples\n", + "for t in range(num_iters):\n", + " # Compute prediction error\n", + " error = X_norm.dot(theta) - y_centered \n", + " # Compute cost for OLS and Ridge (MSE + regularization for Ridge) for monitoring\n", + " cost_OLS = ?\n", + " cost_Ridge = ?\n", + " cost_history[t] = ?\n", + " # Compute gradients for OSL and Ridge\n", + " grad_OLS = ?\n", + " grad_Ridge = ?\n", + " # Update parameters theta\n", + " theta_gdOLS = ?\n", + " theta_gdRidge = ? \n", + "\n", + "# After the loop, theta contains the fitted coefficients\n", + "theta_gdOLS = ?\n", + "theta_gdRidge = ?\n", + "print(\"Gradient Descent OLS coefficients:\", theta_gdOLS)\n", + "print(\"Gradient Descent Ridge coefficients:\", theta_gdRidge)" + ] + }, + { + "cell_type": "markdown", + "id": "dc78d58d", + "metadata": { + "editable": true + }, + "source": [ + "### 3a)\n", + "\n", + "Discuss the results as function of the learning rate paramaters and the number of iterations." + ] + }, + { + "cell_type": "markdown", + "id": "15060acb", + "metadata": { + "editable": true + }, + "source": [ + "### 3b)\n", + "\n", + "Add a stopping parameter as function of the number iterations. \n", + "\n", + "If everything worked correctly, the learned coefficients should be\n", + "close to the true values [5.0, -3.0, 0.0, …, 2.0, …] that we used to\n", + "generate the data. Keep in mind that due to regularization and noise,\n", + "the learned values will not exactly equal the true ones, but they\n", + "should be in the same ballpark." + ] + } + ], + "metadata": {}, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/doc/LectureNotes/_build/.doctrees/environment.pickle b/doc/LectureNotes/_build/.doctrees/environment.pickle index 0379211a45dcd93a92c446d742ad16126d470263..3aab3ce2401a11ec4d0b9ee42f9060a38f4575e9 100644 GIT binary patch delta 10142 zcmZXacR*Cf^T6F*;8d_sENCc-8e;<+iWOr;j4c{_K|$d_KtY5UgC!>CNo)^XHFk~W zYYTcY8~vI{jGDxfV2usCiAGIOV~O=QGjH#N- z4r*9-Y?4IE@G?-mB8V{}J_82UT!cPADmYcD+T@7yi+sGdJ2EXEFG;Loa!g?PYC`;Yc zWI}AN^-q+&%pP?%cN2A<&cKn`Bcpg|wb%z#fZc<=Lj}cD-xYh({X;q$U#1AYel3-NR^NlX3BzNzsKIAR!VuXlp)JmlgkiE<^1GiW2}5PKth-l45{Aoe znST5dNfIF@|5qoYPhOiy;pNRAoDuE1}w@i3+vyA2d%3>)(cyI?dJ+k{;; zj>Y>LNn^d(9ikkdKZv#fT_-vM^pNNd&_yDzSTA;os8Ot!ab~Qa(F?AH@qR`Ul-#9~ z9H1LSTY&Bn9Ra#bbO-1$kyo4-yXEkYi(>3=qc9=UI6v+?a2d%b{yTObjVc`B)-f=~ zqTME^S^qTd#RsvQhIc{`dt@|C2xJe)=9HIdQyFZENwZ$p5}^5qG;5e?!$EV6Gz;pO zno!W(b`&OrGggEJMvX*&h*FEhD-flH6Kfl-lRh-6y&qz1J`rj7Pv~hpn-F4rJYll& zbyA2Cm^{n)Atgj!J}e*%$>ShQP(>z3FfSR@lKB}~DLsvaQ2F@N;!WQF zy@EG0cBS_;7EBD0DfPLx(Q0yU<6){P$RkyGMdQcGk;bAd(gYb%8IeXmvpC2&48mCO zzR8qKBRnO<_<4$9%!d1CXAU<4(?X1wnYqRnX(4jKC%m@tTUMm;bZUsv=KVp2C)|3} zQr^c9QzMPx;M^ZRVt}z5gw9|Q@WDJYCqBk(KsnLJ1Bh~BFcRg&CPC(cBooVWmqa^gxP%86S%Tsd(!Dk&%aghV;POHOYgd6vFcx5*v@|K0+%-=XZr-~zE`a7n|adJkYroOWbZ5DQvS9f^L z_7zy)+Yx3<(I)wGe7LDQG1t!|`JsoX>Zmx+rcE}^@1RW{E!?BqJJL#YaIjpvHd&r0 zHFc2-w28GbujqKN#Gwn+vM+VPuXVVWW7hJyx*%w!t}ChF#HtxgZrH?9VCu9m9`taTy#XyWLmdfrvU^3(%z#Zw%48^S?vDsdU^;Z(VBp6CQ|k z{&zZQUE*KgXkCk9aDH=nrk(G0KTQVuF>j;q+Om%0_rn=VK8_mye55J=A$Ux>&W|A| zMEjl;Y1;lYikmvG7qd*AT=tbH@7Vkj1`8VR?8Xk7qyjn1)wo|U-K-kq&R|%ese0+b zAOM67%7KB=?nHo*Iew#%=PLGfSJXsH8^bK-X?et`!3?v`*$cueo(nl59e@yCF zne8&Eybtr^WEbYg2AjHNRoHry`c`A{CLOBI1edBWOL49nZ^+o1tcM&u#0-!+P?HT{ zDEi2iT|{d&Hh@jka#)lIQRRZzIFkKj+fPMRwKSNGG0Rnj+6-n1R#lrq*cy!^>au(j zW&TnTt_Hrz5_Qq>`fR4gaSfa(N1qaIG}2YMMhrqs4X$s@zSlS{CP??eI3QF5Xhol%u~JW$&Q*9 zAnr72eQ)N=W$_Z8BJ1^I2{Lje_g1U>F-Ty5J`^1Vc>tyd%4*mE22+~w!ax`fCT<+$ z!bw9gvPWw`PE1d2C6^6@-Y%%Gd;8fi)|#Q{qq>e@!!+&~$+9(e9?hz8(;#UKgG52L z6=ES6Ca#FXu}#`8jql~n2IW30*q}GebmTg zmag%73WK>#+Kh>;zs7Tua59m0LOQF=O`M;>Vzt&ilR=6h14kCCuMO@_Wsq7(J7t>7 zpl&uk2x$*aXPY(7n#o$}YDYd|Fw@B(=3};9V;vi-t_^zIU7qcj%Q%KRBhyP zR-iF#CC&uW-dn}`Y1;v7*^gS=cs={vL<#7po_x+=rJ@SC8yVz3!Uq2VM;6zICxBhx zT)DTQeu7<_*}Duy9~HP2JCX3vHtc)Cq4_Y22#vnyJXGO!Y?4aGIdJ?DHr>hEaTBu@ z+oHAYcVXbk;OTDcal)KEte?iezh!BJekx-x#*7U9DPWH@KHmouO|G8|Zo_^xP8Q72 zZjU>_+QQhI0d6}EvK$kQ)tlVZz@Hc_=u~p&XY`eD=3$I2Vb!B-yvF&z&}he2!U`EI zJY?|haU24UfGYhBE&wghgo2AWf!o(xpxm#=_k-U9CG*$(- zA#zmE^)BXpY;^Bmtb*CV#bVaQ)U~(|VWrX?57-lv_B>=$O^SN#q|;AWs;L|N6lOKm zc>W9$FVVplY=udaU$J-0dz@q(Mt0d)=o)xycwa`Q>aOYU#+yNgO67iPi93fJ^VO%Y z65-2)lotM;oW6v4tCtoY$q;>1-*UXS#{BX;#zdK3lLxELUi`S03!8CI)vO|iVM(?2 zRpRS4W_t6r8XNg?nDuBN3+wO->YN{+#AHyC(*byac~7pM%e*Bx;H$b-<1mUq_EAQ4 zKHNkJUes10HF!r|1g=zoEge|}@c(M<=pYVBgR~*F_#ut2YV*As!2#TxK{YvgCwEg> z^*C&#YJCC$gJ6u8qjMl4Bb^~w(VABWFGqrx^xU^1_$X6W zL)&67u}D2@$6;+E?B0Qw;}v~}k4Z_2P0N}9m#M?$h)5pnS6Y`IAD138O>TD=?fLO% z^4Cbd(QLrhiQWF z_2`us)A>P}wI7^(nb}j0$biDs!`jL3GB|8KK=e^%GdZm8gr8;M+$HQcmBXq@So8sh zb&D|TLwq2@>eKlb8jsJwd5&5M3urkte-`>qR(`W_ibh<5K@J}mta9}O=lBk^@n9Gy z55R-Ib2#L6!W(uzmC#Rq_Yj(zmP4ZntmMJZc}?~FQ|_xxYt833Of99~`t;h|s= zCk)>Rqu9i)U+{R1Z*Fq6yKOU1(b~|he6_|a+c2?V3lc_uC-oqoch)k@fjTN_2M#pz zW)4h7gpl}PGe8)q`1cy`eS?D(w5`G3kE*F2M|q+yIq(aPI5LVYg#3f( zC+804A$nymZhUqF_MfDza1*m9;f`B;qsGMBE^XzzxImD>lE3&A zjkSvTCXGe+FsqQk?g#v+#-$JW4fl1vR1tyCcw&XL@~O@!))&h`+n zE63Ylt!k>Ethlf5Rd^cY^W|MKEh&S~K;D1kRZ6K|Rut~98*x+q-U3#-n`u0p-plcu z<8rw>9meuTU-1S*^pk``Iv*B!MdP{8b>qISUoHn)U_Y6+Ojgk3`fqzJ-2b;M8;B_M2n>WMxj&TVoOUj4l@88()9NW06(MP6EEyAsWl`+RM0J!dunpEFeiy@ro|au*wND;WNeXZlZ%(BrnYp!Rl~# z@vb%vjS@3;(W8$=sJhovL}@wOCK{@YUIMZ>wkpfopDkhW1&8Dj*bPr|@hXTa3nIF|!IZ(~*E7ApiV zalGiKO^?RAx+udc=4ow}Bmrv#-KFS|sHb*M5WP6bM&DU(s%ENyv`EDxK7;95Jki;n ze6(CNSF_VZcimD^x+`oceMDthZ?$Nowq-c&Rpm_Q^C*>tUT6A<6!^B^UOt!#DWSmM zxkirqK-7da(jV5yoM||v2m`XkIur4eRg|hZLv+$b$ukAyRI<7}OTbErC}GRiLM@q% z@vJ;b_^E$C#=aw~={6jfr1iApB&G_feMDPzX|8}}jI3b)1btQ92I~Ge&)KYsUVuGD z72s2tpE|n`rwdhp!xmT)s0hA?HBgoE1Z+`B?l@G`R(XrXAZ_|m3RqaF7JM&T4%irO zGx{?DD-~%UFBh-~QQeE_qQ2U+62~XW;AbTT(cP&{@2(NBuu`ppb@*U}QJ)Lg91;Gq z!RePu_yV7ant@kPUsZM!rdyKnbElfVS-?JlY87vBc@X#(?0#o#b5(_RR$mpIkCTl| zXK%-$N9gImu}%mLY7ZoAJ*%nkuW_&>+Dehiy)#|PVW~vzv3LA@*9OXn+W5MV;UzcI)Mp}5d3z= z7Zxt(P{pTFux90Ht5;{m=Ni90FUD&OEy6W|n%R8O)p*Ed5u>$>uek1B^_t6e(RJ4V z6~BuQwXN}oYp6B2jG^f)uXbg|epdOx^fpotkUOnl^ z477M&23kBUi)M6E21-0810|l3K{I$j7Ttpm$D%t~i&W5EQPB(DrkeaKwq@hV+XCzM zurg)l!wH!wIZ8ZsCH*ws65STe#qXEnM)l7A|;H z3l|(&aKVuU7aUn|!I1?QJeq|Ip3A}oM;2UgWWj|-7F=-oiQ&vDhp%`ML3{{R^S2NeJS delta 10077 zcmZXZd0>sl7Qizzkt>@>Bw{HEQB_N9X%e+Gs4bzET4N0fa%Djhl3HS|n=95YjD4%3 zf~PK~`r1-e`ihFB>LE%krK;AT)_2bN=H8dQcmKHG@0>X^-^`ga=bP`FFP2nXyQJdI zsr@3nMx==Hu76O6=z{!m z$!VG6(@T|rxnELt()ba{8A%z`R^RwvY4DOvVVsuhsR!R~WW+r84 z4o^=XlktDeK!=>2#5%ffaJx!|e|)8$1^HeXnF*Q68JWo=pt7^P)X-WuhX!?=#n9;g z^dBmCq$H;$q!?%VH(-|*52(y9yBn8V26F2)BYZ&R+&|pi-17aLu4x{gJT@n7c!BkI zR9DdIW=`#2rL`{Go29i8PTP{w+IO7V2c@-<;mI}3+ubd#j&d5`GA<7Y3jVEhSqo>` zL&GONl-)BL#Rt1z1Pi06ZB>59-Pqr%8n-?+a^v6N*WHb#t*f=L-oOUR(_DwACKgz) zqU=Ln8Rcy53F^F^Ge>5(jeGIIiT9}>GbsTkb$kXDT%r1|sU_W?q@$2=dEnRo>RjV5 z{;v*3Zw~&EF=k-(+)GemeTILR@$WhQUBkZ@_=nSEeTjcKGuBu5hqGaQjej@;R=0BS zhhDcbB=oeEBcXGx0_hf#I}$q8S`O(Ul1I5*_`io}K_z(5-e$#}rdS*V-+an@#D4J4}n+vj3JTqL1yC z_gi^VMf9`Xa?aC>B=oi2@<|O-LVw#WP2cdLGW0o&cc3Yu-|d!}vrP$oZ@298^dn1j zzuj_sRsczl+}$i7_F5S<%m_<}Wp|9Ugpuqz{D&GP2_0E6P>3;b_)qMTQGY}@yKI~s zUCBrp;lr*GKYqGKa`jOfHb<4@4sB#lp^4=W*RnCN2^jSe)rg61x1Qh;s| zJX(J`QVM1KNZB=Sk}VRsx=le#eW*eFWQHZF|(2D*$CF!~#I1C4yY;nv|0 zV{z_T8P==D{m~)piQ%6d!fqQ)lWVg3Wb?C+X;Tqwu8?M_Mrr1%G9Ld-tEHJ8#sR~|-7<0wgk`ZP^r%W}Tr-m7Yr(=w&sj-Il#4zLI)GT8J z=xV0TGWXL-#(8x)THP)wv88_0$7jPsDR6s!m|pDDRr2<*OvJkJvNkg_~bSqDZtTX+DDpxo^CZs z?m5cqJ0{H>tqV$K)fZSbKo(!-5soC=yV`E%HD1|RvC7+VD({}QJu|lwCtJtg^Hyu! zmIbX%oxIpgG_CN?AnESI(}QyQ&;UE7wFP4jkvGl z;W7tSt>O4?#V1-{REK*zX0Dp6D}=6DTDsu$+UZQT_?4x@qKYt{tnVz>J!Jll9vj** zqyWdpPsV7f^(ownbO$#z<0g50X4ATLTk|xv+y)&cyI;2FXd3>HR83cQI_)O!D$u&x zyQdKaI`;0#*18zwq3_Xu-$Jc>_T?f?tG}A4sqQzEHSON7Nj7fJJ(e~EKlS|9+1!PL zTeX=y1fHSpbv^v4Ch@~!O`HFft10?unx-#~rD%%$X}+ejCw6ICadHEbgx z6&N;>TsT~~J33#jqD{tM@2^cR-Tb!pxoFeuVh`!h!CQvOm}_s%^Sz^FK$ zZghN*As2Ro6oxC{)PqQdQh=k{qgk4EmO!+SuKg33P@;oRi#2`ztP3}FJ}+mQI=P@w zcsUAR!JMPWc zYXND#h3xIkIvYEmQg_uJZ`PQB6rdjau*oLPs=z)o>Ai|tl#Hx$67stT)8K5Eh^s)PNfi$|93W@ z(_K%=L2`St83d}}9R_(2!~mnqv?}UWBpYIiYCtrDIK~p!a#K%Qu(xz^la}nY#!oq-{Tp1#=TqhBHW{r1eci z*AT8w!pJ9le-wjkO?YKA&LrV%D_g;2*UB(auvwWP?^f0Ku8|6nk79rtlEyMMUQ1_? zT}V4&9P6d={CHeyq)p9akRJ#aOkjyxTP~Ze(&)%x^|ZmANemJx*-o6|GI%o=2SVDz z(^#R#nKM|Fu6Aq|gFHY62_Li1HP*2)NL!@+!0zhVuDOhJ8Ek>!h0SOC z4MP!JyAZ>N@Z=&~H-y~{Snnnl7qEt$P_CWHJyjja2I->3%UE5rNd341XOij;S;Y=$ zY_bNI2WjuGWj(cR?+xsT);8M6AP=K~1PoM9KV|TsqY4F^8Kg+U`k!I;>AgN3caL+8 zotw<`w6BnT2#IA=fU3D2<`GE_86|?$uRB<>SrESy7MD?Rcc41fX}<7K15V+csp#fuHpnbeeb2IaT3hl9rc^T6`EM7Fxxm(FZS`VS zlgoGKzDk`aMS1|9QtU|BidMBKAjd^ktSKh!x zOWNi)aTOAtzr{A2DBTZ>M(V(AaHm<+;SM|{Ox*t$^VQ8(hwQ1w zuS(b?ja{C&@a$7IR%`n|!_`jJUOdMPO?dbvTdgtcHS4DDc82lEj9rvH1;1&$H`9!= zUhcdZ11Uf)EyrP@@16%KklsV2h4+=ymN9=Bx`J0y{^fZi@Z&OgLiF(D;1j}~UOd4> znOU8Os`ftol$MK{ac|YE0*7=(rU(7_MvdA2e1pb@l{ii~G%I1->8s8M^6^ZvOdd&3 zZPzPNUwJx{*HImVIYb$tzgk>{4>oacH4f87%8cr~0@vj=_*YsxG=%TcSi2TKs_|8A z4l9|eov6cMYeAS26p5&D&!Gi>Yp8XFI^te*=0Jl3%|vr z=W4Z>t`B$lL3pPJQ@8ja~aWn~~*D@i*0x{ybLK$~(;)s1M>HsMuBY?LY_= zc2(9H%x{BOaR}@qK!j_M96gkub#m`v{G4XhE`e9Vx_yUp2%H`Q5+A&BFOn_Xc~$jp z5^n%6L+&ufqDWR%Ef~qOOj!ku=F4=^v{2@weoW?_wVZp6l~)bM@V~WO`8*F)T~awD z3v5aqPs4~Hw2tMFHwdqe&OJ&*X`6Y;|FJS3b$)hh@$ocq@CA9V;hH z0N0N_s<)lLOyJQB#Q;?+8y7j@?i^f?gw{zMo|1%*Cv$j25iXv}A+-`VoyNClygeP) zE@|N*0gq9_MzgsuUs?~G1D_nMa(@HotVh{+C__0wCFgN?wiCXxbI5Cef%2OY=BbwE zV?1Gz{Pj~_T?Netr<bwXeolGJ0f>q&S-dmgc7r@|*R;O{(-wQr> z1j{N*d1qtTbsd4TmU4KW4Z9woUP$iCr0M}{4!(v+IcxwdPl(dy3^70@t>lnl34dRO z35jsb8k}9i2Wz=iW7>KSI|kBT+`z|a9J-OGX@n6C(b#=6hg~??9^Qg8OBl1&)$Wl( zp02f>xAS!xpY7oLHNq-@wSmpwCT-j7|=_d>{!YM_NZA^rf>~sYzV#v2>AS%!g_1)Jq&5i*!r)0CQ4n5B$bq%Sl@J4g(u; z#La;>V4F$Gx;HUH5+1$9;aN{O>$Xe#_8mS!Yd`-B9|2_W?mfO$?L4NL`A9RpvceHRaD8(S){`K#RzC`l1PW|jjLouyUtqbKqc`8 zLpebG6(~At>{wYm*Z5%-0WnPmx2p;WCBhli1vrJUW(@(uM>Jh`vnCFOiu#6%$r`WK z5)cnD^F*LZ2*bWmg>7#MFLk+&=w4cmtLu~%ltVC5TTyR|IU0-Vi@CZ2C7@XiMITc( zLf5#fyN!emlHwM))!fEnk*)-8U8NB|*sau<7a<^?=mz1D0^*qPMwEbMO1PkffW%B# zy`_NABK)M4OWU}OvtPJM45aa@fx|RTp}R_NJ_w^cIi{sa5#S@uoV_N5BU2@^Sd& zkz~L|dPqMJugl>(M@v<1fNKExOJM*X#fyHrJb0OiR@Dc&+Jo;Pk@EF&;jb$IfDrY`jiXR510hmm?YWK2?=< zo*1sH7SG23GEIBKRNEGc_nk$HT(!dci6&~XfwN9Gd|4pi-H&kYQVf2=%FA%=Q(HCP zf@y?l9;u5ati)AFR(`8p6;`Yf`0&IbKtI3NSZ;&BhZt(lec~LLO8C^-8+Bt7rW-Pt zzC}#Z=(|t1=Q<%I&D^4LoG?XEi-JH&KTD~sANclGv8%7kdO`g5lzRG34+ zTav4(uXkg%LQ%q2v5M-x7sHd9{$4rTQ~mb2hIsEw*DTHa3PYc4Uwkca3xdrwS;VU- zV?QncD!P9_SqFMOcYS?64%4?d4slv@4ig3A$|ip3bM{qB6yZu|)T2cMCHx;rTzrI!1?Qah2%gEUqRUo5lIlnOR%~Ixvg#q|>rE zKRPOlt4QZ$aUOI?7UxALWKbQC$Dj<)#=sH}#^8d7+;Ks4A{NK#I1C!YvoNTK2Vo%H zcn`1MbOr`mJOBeNo_@tKI{E@7o_m23551rnJn@R_Os8FOZLN<~^c_*b2i|@<{VjIn z;!)Z{>*r0}+!nx*mnk{QNZ*%{j+Bwkl#$MskgYYgR`IeFVToRl_F0}f)M3iclkwVKzwaZ9#%1Cv~NcGA{^-D`x`M#r4 z(uWU)gT_ob{<#Q&m(Rt|MN>|ZHMoEx3odvd3l}_%g$o|V!UfM^;ev;-aKRH;xZv?C zT=47_E*M#G!N`IOMiyK!vfzSet#HAER=8kf!384=E)-dC#UOwSMiyK!vfzS;s&K)` zf(u3#TrjfWg6F7kc_gR560tN08;!w6L$J{ZY%~BH_1{L_w^8qH)Oj2A-9}xvQO|AE zaU1p9M%}hiuk)zWHtMsDj)ZO0;XLZE&3d~IbS96wYop%gQD^h0uQuwc4SJgPS{$sE z?>#oBQEGZ(a#~_icERX&R!bi@x3{MkEE(km`)c^@>FNG9V|U~v=H8cGm#6SjEZ^qt oy}X-%skIUQHCG>V_b*t`DcFo#PvTz@{++?U)9OQa_au-10E60zzW@LL diff --git a/doc/LectureNotes/_build/.doctrees/exercisesweek37.doctree b/doc/LectureNotes/_build/.doctrees/exercisesweek37.doctree index 644fa8dbdc36b3ab5b8cc3be9d2275854b2200ef..45f954a046b8687de493ef2eceaf996f991eeeb2 100644 GIT binary patch delta 4121 zcmbVPX-r(#73Mx>Ach&Un!wnB2aMTfVTLupnAKpgSAAV4E=nL2f zV5cyBrs6rl&sr^R)?;@G3wGAW*;%*1d+jXmU;}t;k@s?}Ptb02sgv7W*5>D*@U`u) zY_D!taCy$}Gk{z==G?QPE|mM&cNH* zh3GnrY9}He!c?kqvce-g>n6-HWF8qL>=w46u{Q!p^m*4zIvtD$LL-QS9?Kjd2)C;xQaop zY7j3<)@CmZHI#}-70N{mj9KAeiDSG)DV;vM$3-aj5>0#nOpQakG`AZKL_>B*6v(~M z#It{6nDfb7sv`J^D|zbbYj}C{oEvbfB_FiBRz?-!#97D6K9mL(hGw})eUE{=1{Yp&KmBsV%2F0m*98D=**WC=8vb5|N-X7dV{Ik9^vMbRdNH7uydyj%0NC*a3PL_nD zLxu3)6NUSt9_g33mw!OLlqK|nw-T9m)5WI~PKR7kIz)Zbfqd~mob10SOv~e7@*XLr zct*t?tiaB&UM`y7AOXQ}7;;7wCsJTCy|3_uQBQ^%K6d9b|tmmhye@$Fcj&MA$7(S~QKV^s?Nz#RMEX)k!SK`6<%( zQJ_DaDwT=cH9*?<7FvKGjtj^HGGo4kv>wvJ z*>to4$IiTgQny@snJP|vBh+kpA(c6p5^dBqAJ9gfpAu`-R10Lf_c(Jdg{JGdEx{3aCl8XX7lq+nn&P>EM>S{+cz03f z|Aj8?u*KAe(QJiMDBjRIseo9;r)Btp*(Pt^3=e3R0*%eA`$Hf0)mLn`hHP z59aSO=t1a??@qEp-w@rHghNAyBWV#*-47cfhSc)!Mr_wVXGIXvDV-XB`ng#{X5$ zSBa>_$FoyMgo=m?BGz#OaDJ7D9wNGkSR-PEh@TNxKPBP}5tBr`LBv%o{G0pS2~vn4bswazgVb%%&lM0yBm(XTrO&+(=H5&~ yq1>;KP$+lu7ZK)8E?k(q8cTB8gvs}jH%fkkyixLoD0!z9YL@@WFsV?vvhY8C4^TV+ delta 3628 zcmb7HeN0=|73V(N7_c2MG#JwaKL`*51`H;A8Uh5IknmY>;syuThR2>A{DJ*k%7Bto z$tG8H?Q&eHRkt>*>$;{&sus1{A6=DJYL!Zzl1y#?ShSW!Qed`IS=!6CDZC9Lp@y z!NA-iJsk{iVXr63F?1m4jW8Rb%5tj4HXUSUXT8&2Ccyo#1?P<`Xs^R%rkx$N&JNl_ zJ2|hPiO>#L1BPla)R=h^cUf3%^m936#Qk}tbU{wWw1uJiY@f5!&pPX zhQ78Lp>`Opn}jQ-Jjk=;@!uPDN0MXeHM}&x04D2g>CngZ!|BldW-~VTd(*w}iM>pH zW~;k<(37yEu}@a+@xvPpFH)Nn^w{*J3Xgxuu@YyMC*rJjO&FbxpN_A@jhN^yLfr(X zy5*IRiuN|Cj10Dhl|?;x4XTTa`kR0(2DAyKmRaKAn%%6HtmDI0IF6H&?%!vC?<~&D zg<#<3si(o!v`0?gJOyyOr4ueUy+mcn2tOFPR|D6Y%TXSno(A<}B8M#a$G*~}R(-P$ zmEEmzrnx++k+jovw^mn={+=vZA|lo}%vKNcaUBF#%N~(liAEA8CiixJ39*f^bpE|=FWC0j?J7OuYfz|Vss8iTC445s}KL`C#lAJTm~zhGw3{-gV!y& zg^vq?;0?Y|zwYl?Q@hKdg4`&zi;?i3u|^aJ?&>?O`~#%Qw|n4(|PDsY|3v zl}g8oi`O2t?<^30OS1I_pYz{{EH1$l-Cc>bdicCshpybMfvbWpfupBP(CgDVJ;Aw| z$f6&AmKIlBNmj|;-F9B=&QI=kW9mIj9FU6P57}!k_qGWxZAH#Gx8F14rk#gmj5yDK6iGeLZL$e%Hs!$$0`( zxz%U^uJ=!C%BA^%%l)gU2D)v<9r;ompPigLwcY>Ilt|nnPwxGK*Dj_ys79XHDG(f| z^GJ6|=8PLT2Y)J~hU742?a6xQL%x9zL*0=g(!XjRNc8VP3=SX_JanYJMFJZI<6F8h zgpZ2})*Hdcy+4G{`UtrJ=rH((I-q>eXr_$|taqBFu|~i(%ec8{80$QgBMz!ev^y|E z<6_d)^Z<+wR>98(E%44@A9BH4_mwIa81Ii*E&_u?yAd&L!+Y@6Ap@4zhbpkdc?9jS zj(Rwi67?UyO{F9Nv%__Q)+@sX>WP$=RjMV%rd`u*Lt3(Hcm`Ae+sZGfDcLR<+C4K9 zp|>m}WN#eXLtRMG*h==oh$%rfv(Yg%K|63RYIh_%T7)1rs-oUZ*=XJ_6fH5bSVvEg zqA-!@9JhrHPNNWnRz4dHr&G04AE#XCw{y`7x5rBOQt8#(7~h>c>*Y9Rh89XGy_9ec z$wefwZiKO;d*QutJ#~k8H&@*(qmp;?*J7PgC*sz-WIBymkM9UBcPoW)1i)QF7}UC3UUsk)A0O>X<eAF zuEN&FTnEy;yzd%GLxvvkr6GTNV3)weYVSZFrP=(2dkN28wd0t;?s*;ECiB`+oCVc< zsstegf7hR#o={RKlbn^=`A^~5Lf`j_)Md{jiNo?+f*!(8pvx!QH4+@#A(y>ru9&LG zBy9|uNfmwvotYa!ZfN)Kfq%{Q^1o^+3sEpZzb6+fyx?zxKXW-UpI{*`P)rT-7Rnkw zOVJ$2Om&~Q5xRQd`#@(-vY8V@aQao$ao&TG#M<=VjW{gi0oXR*3)!JBpnm=s#b)xA zXW_;9Sz$mvr1HJtu{7jLcxWqv3~(c4kiO6GTC(GtB|P9(q^(pET`1Kx467rLUHysp zngxcqF3RGCXOCOp15Qg(^BbRXUs0+*;asfkV8Y>G7}ggGHojbVSEe^NCet8m!M_um z8CS6gt}QjALQpQ3ARSaLmmwoYiom^0WAu^bee#`3oPfOLooFvyU#?!gh^_m6O9c71 z<7*<~S^UQKJwrqVK9GE6MC>M_m6CtmB-^Epo#(38;-ffI`8ROl>cvmsrO^kOD1a) -
  • Exercise 2, use the analytical formulae for OLS and Ridge regression to find the optimal paramters \(\boldsymbol{theta}\)
      +
    • Exercise 2, use the analytical formulae for OLS and Ridge regression to find the optimal paramters \(\boldsymbol{\theta}\)
      • -
    • Implementing the simplest form for gradient descent
        +
      • Exercise 3, Implementing the simplest form for gradient descent @@ -459,13 +459,14 @@ y = X.dot @ theta_true + noise

        This code produces a dataset where only features 0, 1, and 6 significantly influence \(\boldsymbol{y}\). The rest of the features have zero true -coefficient, so they only contribute noise. For example, feature 0 has +coefficient. For example, feature 0 has a true weight of 5.0, feature 1 has -3.0, and feature 6 has 2.0, so the expected relationship is:

        \[ -y \approx 5 \times X_0 \;-\; 3 \times X_1 \;+\; 2 \times X_6 \;+\; \text{noise}. +y \approx 5 \times x_0 \;-\; 3 \times x_1 \;+\; 2 \times x_6 \;+\; \text{noise}. \]
        +

        You can remove the noise if you wish to.

        Exercise 1, scale your data#

        @@ -473,12 +474,14 @@ y \approx 5 \times X_0 \;-\; 3 \times X_1 \;+\; 2 \times X_6 \;+\; \text{noise}. standardize the features. This ensures all features are on a comparable scale, which is especially important when using regularization. Here we will perform standardization, scaling each -feature to have mean 0 and standard deviation 1:

        -

        Compute the mean and standard deviation of each column (feature) in \(bm{X}\). +feature to have mean 0 and standard deviation 1.

        +
        +

        1a)#

        +

        Compute the mean and standard deviation of each column (feature) in \(\boldsymbol{X}\). Subtract the mean and divide by the standard deviation for each feature.

        We will also center the target \(\boldsymbol{y}\) to mean \(0\). Centering \(\boldsymbol{y}\) -(and each feature) means the model won’t require a separate intercept -term – the data is shifted such that the intercept is effectively 0 +(and each feature) means the model does not require a separate intercept +term, the data is shifted such that the intercept is effectively 0 . (In practice, one could include an intercept in the model and not penalize it, but here we simplify by centering.)

        @@ -496,18 +499,16 @@ y_centered = ?
        -
        -

        1a)#

        Fill in the necessary details.

        -

        After this preprocessing, each column of \(\boldsymbol{X}_norm\) has mean zero and standard deviation \(1\) -and \(\boldsymbol{y}_centered\) has mean 0. This makes the optimization landscape +

        After this preprocessing, each column of \(\boldsymbol{X}_{\mathrm{norm}}\) has mean zero and standard deviation \(1\) +and \(\boldsymbol{y}_{\mathrm{centered}}\) has mean 0. This makes the optimization landscape nicer and ensures the regularization penalty \(\lambda \sum_j -\beta_j^2\) treats each coefficient fairly (since features are on the +\theta_j^2\) in Ridge regression treats each coefficient fairly (since features are on the same scale).

        -

        Exercise 2, use the analytical formulae for OLS and Ridge regression to find the optimal paramters \(\boldsymbol{theta}\)#

        +

        Exercise 2, use the analytical formulae for OLS and Ridge regression to find the optimal paramters \(\boldsymbol{\theta}\)#

        # Set regularization parameter, either a single value or a vector of values
@@ -524,23 +525,22 @@ print("Closed-form OLS coefficients:", theta_closed_form)
        -

        This computes the ridge and OLS regression coefficients directly. The identity -matrix \(I\) has the same size as \(X^T X\) (which is n_features x -n_features), and lam * I adds \(\lambda\) to the diagonal of \(X^T X. We -then invert this matrix and multiply by \)X^T y. The result -for \(\boldsymbol{\theta}\) is a NumPy array of shape (n_features,) containing the -fitted weights.

        +

        This computes the Ridge and OLS regression coefficients directly. The identity +matrix \(I\) has the same size as \(X^T X\). It adds \(\lambda\) to the diagonal of \(X^T X for Ridge regression. We +then invert this matrix and multiply by \)X^T y\(. The result +for \)\boldsymbol{\theta}\( is a NumPy array of shape (n\)_\(features,) containing the +fitted parameters \)\boldsymbol{\theta}$..

        2a)#

        -

        Finalize the OLS and Ridge regression determination of the optimal parameters \(bm{\theta}\).

        +

        Finalize, in the above code, the OLS and Ridge regression determination of the optimal parameters \(\boldsymbol{\theta}\).

        2b)#

        Explore the results as function of different values of the hyperparameter \(\lambda\). See for example exercise 4 from week 36.

        -
        -

        Implementing the simplest form for gradient descent#

        +
        +

        Exercise 3, Implementing the simplest form for gradient descent#

        Alternatively, we can fit the ridge regression model using gradient descent. This is useful to visualize the iterative convergence and is necessary if \(n\) and \(p\) are so large that the closed-form might be @@ -588,16 +588,16 @@ print("Gradient Descent Ridge coefficients:", theta_gdRidge)

        3a)#

        -

        Discuss the results as function of the learning rate paramaters and the number of iterations.

        +

        Discuss the results as function of the learning rate parameters and the number of iterations.

        3b)#

        -

        Add a stopping parameter as function of the number iterations.

        +

        Try to add a stopping parameter as function of the number iterations. How would you define a stopping criterion?

        If everything worked correctly, the learned coefficients should be close to the true values [5.0, -3.0, 0.0, …, 2.0, …] that we used to generate the data. Keep in mind that due to regularization and noise, the learned values will not exactly equal the true ones, but they -should be in the same ballpark.

        +should be in the same ballpark. Which method (OLS or Ridge) gives the best results?

        @@ -663,12 +663,12 @@ should be in the same ballpark.

      • 1a)
      • -
    • Exercise 2, use the analytical formulae for OLS and Ridge regression to find the optimal paramters \(\boldsymbol{theta}\)
        +
      • Exercise 2, use the analytical formulae for OLS and Ridge regression to find the optimal paramters \(\boldsymbol{\theta}\)
        • -
      • Implementing the simplest form for gradient descent
          +
        • Exercise 3, Implementing the simplest form for gradient descent diff --git a/doc/LectureNotes/_build/html/searchindex.js b/doc/LectureNotes/_build/html/searchindex.js index caec7a48e..c1a2d2214 100644 --- a/doc/LectureNotes/_build/html/searchindex.js +++ b/doc/LectureNotes/_build/html/searchindex.js @@ -1 +1 @@ -Search.setIndex({"alltitles": {"1a)": [[18, "a"]], "2a)": [[18, "id1"]], "2b)": [[18, "b"]], "3a)": [[18, "id2"]], "3b)": [[18, "id3"]], "A Classification Tree": [[9, "a-classification-tree"]], "A Frequentist approach to data analysis": [[0, "a-frequentist-approach-to-data-analysis"], [25, "a-frequentist-approach-to-data-analysis"]], "A better approach": [[8, "a-better-approach"]], "A first summary": [[25, "a-first-summary"]], "A quick Reminder on Lagrangian Multipliers": [[8, "a-quick-reminder-on-lagrangian-multipliers"]], "A simple example": [[4, "a-simple-example"]], "A soft classifier": [[8, "a-soft-classifier"]], "A top-down perspective on Neural networks": [[1, "a-top-down-perspective-on-neural-networks"]], "ADAM optimizer": [[13, "adam-optimizer"]], "Activation functions": [[12, "activation-functions"]], "Adaptive boosting: AdaBoost, Basic Algorithm": [[10, "adaptive-boosting-adaboost-basic-algorithm"]], "Adding error analysis and training set up": [[25, "adding-error-analysis-and-training-set-up"], [26, "adding-error-analysis-and-training-set-up"]], "Adjust hyperparameters": [[1, "adjust-hyperparameters"]], "Algorithms for Setting up Decision Trees": [[9, "algorithms-for-setting-up-decision-trees"]], "An Overview of Ensemble Methods": [[10, "an-overview-of-ensemble-methods"]], "An extrapolation example": [[4, "an-extrapolation-example"]], "An optimization/minimization problem": [[25, "an-optimization-minimization-problem"]], "And finally \\boldsymbol{X}\\boldsymbol{X}^T": [[26, "and-finally-boldsymbol-x-boldsymbol-x-t"]], "And what about using neural networks?": [[25, "and-what-about-using-neural-networks"]], "Another Example, now with a polynomial fit": [[27, "another-example-now-with-a-polynomial-fit"]], "Another example, the moons again": [[9, "another-example-the-moons-again"]], "Applied Data Analysis and Machine Learning": [[19, null]], "Autocorrelation function": [[22, "autocorrelation-function"]], "Automatic differentiation": [[13, "automatic-differentiation"]], "Back to Ridge and LASSO Regression": [[26, "back-to-ridge-and-lasso-regression"], [27, "back-to-ridge-and-lasso-regression"]], "Back to the Cancer Data": [[11, "back-to-the-cancer-data"]], "Bagging": [[10, "bagging"]], "Bagging Examples": [[10, "bagging-examples"]], "Basic Matrix Features": [[20, "basic-matrix-features"]], "Basic ideas of the Principal Component Analysis (PCA)": [[11, null]], "Basic math of the SVD": [[5, "basic-math-of-the-svd"], [26, "basic-math-of-the-svd"], [27, "basic-math-of-the-svd"]], "Basics": [[7, "basics"]], "Basics of a tree": [[9, "basics-of-a-tree"]], "Batch Normalization": [[1, "batch-normalization"]], "Bayes\u2019 Theorem and Ridge and Lasso Regression": [[5, "bayes-theorem-and-ridge-and-lasso-regression"]], "Boosting, a Bird\u2019s Eye View": [[10, "boosting-a-bird-s-eye-view"]], "Bootstrap": [[6, "bootstrap"]], "Bringing it together, first back propagation equation": [[12, "bringing-it-together-first-back-propagation-equation"]], "Building a Feed Forward Neural Network": [[1, null]], "Building a tree, regression": [[9, "building-a-tree-regression"]], "Building neural networks in Tensorflow and Keras": [[1, "building-neural-networks-in-tensorflow-and-keras"]], "CNNs in more detail, building convolutional neural networks in Tensorflow and Keras": [[3, "cnns-in-more-detail-building-convolutional-neural-networks-in-tensorflow-and-keras"]], "Cancer Data again now with Decision Trees and other Methods": [[9, "cancer-data-again-now-with-decision-trees-and-other-methods"]], "Choose cost function and optimizer": [[1, "choose-cost-function-and-optimizer"]], "Classical PCA Theorem": [[11, "classical-pca-theorem"]], "Clustering and Unsupervised Learning": [[14, null]], "Code for SVD and Inversion of Matrices": [[5, "code-for-svd-and-inversion-of-matrices"]], "Codes and Approaches": [[14, "codes-and-approaches"]], "Codes for the SVD": [[5, "codes-for-the-svd"], [26, "codes-for-the-svd"], [27, "codes-for-the-svd"]], "Coding Setup and Linear Regression": [[15, "coding-setup-and-linear-regression"]], "Collect and pre-process data": [[1, "collect-and-pre-process-data"]], "Communication channels": [[25, "communication-channels"]], "Compare Bagging on Trees with Random Forests": [[10, "compare-bagging-on-trees-with-random-forests"]], "Comparing with a numerical scheme": [[2, "comparing-with-a-numerical-scheme"]], "Comparison with OLS": [[27, "comparison-with-ols"]], "Computing the Gini index": [[9, "computing-the-gini-index"]], "Conditions on convex functions": [[27, "conditions-on-convex-functions"]], "Conjugate gradient method": [[13, "conjugate-gradient-method"]], "Convex function": [[27, "convex-function"]], "Convex functions": [[13, "convex-functions"], [27, "convex-functions"]], "Convolution Examples: Polynomial multiplication": [[3, "convolution-examples-polynomial-multiplication"]], "Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)": [[3, "convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms"]], "Convolutional Neural Network": [[12, "convolutional-neural-network"]], "Convolutional Neural Networks": [[3, null]], "Correlation Function and Design/Feature Matrix": [[26, "correlation-function-and-design-feature-matrix"]], "Correlation Matrix": [[11, "correlation-matrix"], [26, "correlation-matrix"]], "Correlation Matrix with Pandas": [[26, "correlation-matrix-with-pandas"]], "Course Format": [[25, "course-format"]], "Course setting": [[21, null]], "Covariance Matrix Examples": [[26, "covariance-matrix-examples"]], "Covariance and Correlation Matrix": [[26, "covariance-and-correlation-matrix"]], "Cross-validation": [[6, "cross-validation"]], "Deadlines for projects (tentative)": [[25, "deadlines-for-projects-tentative"]], "Decision trees, overarching aims": [[9, null]], "Deep learning methods": [[25, "deep-learning-methods"]], "Define model and architecture": [[1, "define-model-and-architecture"]], "Defining the cost function": [[1, "defining-the-cost-function"]], "Deliverables": [[15, "deliverables"], [16, "deliverables"]], "Derivatives and the chain rule": [[12, "derivatives-and-the-chain-rule"]], "Derivatives, example 1": [[26, "derivatives-example-1"]], "Deriving OLS from a probability distribution": [[5, "deriving-ols-from-a-probability-distribution"]], "Deriving and Implementing Ordinary Least Squares": [[16, "deriving-and-implementing-ordinary-least-squares"]], "Deriving and Implementing Ridge Regression": [[17, "deriving-and-implementing-ridge-regression"]], "Deriving the Lasso Regression Equations": [[26, "deriving-the-lasso-regression-equations"], [27, "deriving-the-lasso-regression-equations"], [27, "id6"]], "Deriving the Ridge Regression Equations": [[26, "deriving-the-ridge-regression-equations"], [27, "deriving-the-ridge-regression-equations"], [27, "id3"]], "Deriving the back propagation code for a multilayer perceptron model": [[12, "deriving-the-back-propagation-code-for-a-multilayer-perceptron-model"]], "Developing a code for doing neural networks with back propagation": [[1, "developing-a-code-for-doing-neural-networks-with-back-propagation"]], "Diagonalize the sample covariance matrix to obtain the principal components": [[11, "diagonalize-the-sample-covariance-matrix-to-obtain-the-principal-components"]], "Different kernels and Mercer\u2019s theorem": [[8, "different-kernels-and-mercer-s-theorem"]], "Disadvantages": [[9, "disadvantages"]], "Discriminative Modeling": [[25, "discriminative-modeling"]], "Domains and probabilities": [[22, "domains-and-probabilities"]], "Dropout": [[1, "dropout"]], "Economy-size SVD": [[26, "economy-size-svd"], [27, "economy-size-svd"]], "Elements of Probability Theory and Statistical Data Analysis": [[22, null]], "Ensemble Methods: From a Single Tree to Many Trees and Extreme Boosting, Meet the Jungle of Methods": [[10, null]], "Entropy and the ID3 algorithm": [[9, "entropy-and-the-id3-algorithm"]], "Essential elements of ML": [[25, "essential-elements-of-ml"]], "Evaluate model performance on test data": [[1, "evaluate-model-performance-on-test-data"]], "Example 2": [[26, "example-2"]], "Example 3": [[26, "example-3"]], "Example 4": [[26, "example-4"]], "Example Matrix": [[26, "example-matrix"], [27, "example-matrix"]], "Example of discriminative modeling, taken from Generative Deep Learning by David Foster": [[25, "example-of-discriminative-modeling-taken-from-generative-deep-learning-by-david-foster"]], "Example of generative modeling, taken from Generative Deep Learning by David Foster": [[25, "example-of-generative-modeling-taken-from-generative-deep-learning-by-david-foster"]], "Example of own Standard scaling": [[26, "example-of-own-standard-scaling"]], "Example relevant for the exercises": [[26, "example-relevant-for-the-exercises"]], "Example: Exponential decay": [[2, "example-exponential-decay"]], "Example: Population growth": [[2, "example-population-growth"]], "Example: The diffusion equation": [[2, "example-the-diffusion-equation"]], "Example: binary classification problem": [[1, "example-binary-classification-problem"]], "Examples": [[25, "examples"]], "Examples of likelihood functions used in logistic regression and neural networks": [[7, "examples-of-likelihood-functions-used-in-logistic-regression-and-neural-networks"]], "Exercise 1 - Choice of model and degrees of freedom": [[17, "exercise-1-choice-of-model-and-degrees-of-freedom"]], "Exercise 1 - Finding the derivative of Matrix-Vector expressions": [[16, "exercise-1-finding-the-derivative-of-matrix-vector-expressions"]], "Exercise 1 - Github Setup": [[15, "exercise-1-github-setup"]], "Exercise 1, scale your data": [[18, "exercise-1-scale-your-data"]], "Exercise 1: Setting up various Python environments": [[0, "exercise-1-setting-up-various-python-environments"]], "Exercise 2 - Deriving the expression for OLS": [[16, "exercise-2-deriving-the-expression-for-ols"]], "Exercise 2 - Deriving the expression for Ridge Regression": [[17, "exercise-2-deriving-the-expression-for-ridge-regression"]], "Exercise 2 - Setting up a Github repository": [[15, "exercise-2-setting-up-a-github-repository"]], "Exercise 2, use the analytical formulae for OLS and Ridge regression to find the optimal paramters \\boldsymbol{theta}": [[18, "exercise-2-use-the-analytical-formulae-for-ols-and-ridge-regression-to-find-the-optimal-paramters-boldsymbol-theta"]], "Exercise 2: making your own data and exploring scikit-learn": [[0, "exercise-2-making-your-own-data-and-exploring-scikit-learn"]], "Exercise 3 - Creating feature matrix and implementing OLS using the analytical expression": [[16, "exercise-3-creating-feature-matrix-and-implementing-ols-using-the-analytical-expression"]], "Exercise 3 - Fitting an OLS model to data": [[15, "exercise-3-fitting-an-ols-model-to-data"]], "Exercise 3 - Scaling data": [[17, "exercise-3-scaling-data"]], "Exercise 3 - Setting up a Python virtual environment": [[15, "exercise-3-setting-up-a-python-virtual-environment"]], "Exercise 3: Normalizing our data": [[0, "exercise-3-normalizing-our-data"]], "Exercise 4 - Fitting a polynomial": [[16, "exercise-4-fitting-a-polynomial"]], "Exercise 4 - Implementing Ridge Regression": [[17, "exercise-4-implementing-ridge-regression"]], "Exercise 4 - Testing multiple hyperparameters": [[17, "exercise-4-testing-multiple-hyperparameters"]], "Exercise 4 - The train-test split": [[15, "exercise-4-the-train-test-split"]], "Exercise 4: Adding Ridge Regression": [[0, "exercise-4-adding-ridge-regression"]], "Exercise 5 - Comparing your code with sklearn": [[16, "exercise-5-comparing-your-code-with-sklearn"]], "Exercise 5: Analytical exercises": [[0, "exercise-5-analytical-exercises"]], "Exercise: Cross-validation as resampling techniques, adding more complexity": [[6, "exercise-cross-validation-as-resampling-techniques-adding-more-complexity"]], "Exercise: Analysis of real data": [[6, "exercise-analysis-of-real-data"]], "Exercise: Bias-variance trade-off and resampling techniques": [[6, "exercise-bias-variance-trade-off-and-resampling-techniques"]], "Exercise: Lasso Regression on the Franke function with resampling": [[6, "exercise-lasso-regression-on-the-franke-function-with-resampling"]], "Exercise: Ordinary Least Square (OLS) on the Franke function": [[6, "exercise-ordinary-least-square-ols-on-the-franke-function"]], "Exercise: Ridge Regression on the Franke function with resampling": [[6, "exercise-ridge-regression-on-the-franke-function-with-resampling"]], "Exercises": [[0, "exercises"]], "Exercises and Projects": [[6, "exercises-and-projects"]], "Exercises week 34": [[15, null]], "Exercises week 35": [[16, null]], "Exercises week 36": [[17, null], [18, null]], "Expectation values": [[22, "expectation-values"]], "Extending to more than one variable": [[27, "extending-to-more-than-one-variable"]], "Extremely useful tools, strongly recommended": [[25, "extremely-useful-tools-strongly-recommended"]], "Feed-forward neural networks": [[12, "feed-forward-neural-networks"]], "Feed-forward pass": [[1, "feed-forward-pass"]], "Final back propagating equation": [[12, "final-back-propagating-equation"]], "Fine-tuning neural network hyperparameters": [[1, "fine-tuning-neural-network-hyperparameters"]], "Fitting an Equation of State for Dense Nuclear Matter": [[0, "fitting-an-equation-of-state-for-dense-nuclear-matter"]], "Fixing the singularity": [[26, "fixing-the-singularity"], [27, "fixing-the-singularity"]], "Frequently used scaling functions": [[26, "frequently-used-scaling-functions"]], "From OLS to Ridge and Lasso": [[27, "from-ols-to-ridge-and-lasso"]], "From one to many layers, the universal approximation theorem": [[12, "from-one-to-many-layers-the-universal-approximation-theorem"]], "Functionality in Scikit-Learn": [[26, "functionality-in-scikit-learn"]], "Further Dimensionality Remarks": [[3, "further-dimensionality-remarks"]], "Further properties (important for our analyses later)": [[5, "further-properties-important-for-our-analyses-later"], [26, "further-properties-important-for-our-analyses-later"], [27, "further-properties-important-for-our-analyses-later"]], "Gaussian Elimination": [[20, "gaussian-elimination"]], "General Features": [[9, "general-features"]], "General linear models and linear algebra": [[25, "general-linear-models-and-linear-algebra"]], "Generalizing the fitting procedure as a linear algebra problem": [[25, "generalizing-the-fitting-procedure-as-a-linear-algebra-problem"], [25, "id1"]], "Generative Adversarial Networks": [[4, "generative-adversarial-networks"]], "Generative Models": [[4, "generative-models"]], "Generative Versus Discriminative Modeling": [[25, "generative-versus-discriminative-modeling"]], "Geometric Interpretation and link with Singular Value Decomposition": [[11, "geometric-interpretation-and-link-with-singular-value-decomposition"]], "Gradient Boosting, Classification Example": [[10, "gradient-boosting-classification-example"]], "Gradient Boosting, Examples of Regression": [[10, "gradient-boosting-examples-of-regression"]], "Gradient Clipping": [[1, "gradient-clipping"]], "Gradient Descent Example": [[27, "id1"]], "Gradient boosting: Basics with Steepest Descent/Functional Gradient Descent": [[10, "gradient-boosting-basics-with-steepest-descent-functional-gradient-descent"]], "Gradient descent": [[2, "gradient-descent"]], "Gradient descent and Ridge": [[27, "gradient-descent-and-ridge"]], "Gradient descent example": [[27, "gradient-descent-example"]], "Grading": [[23, "grading"], [23, "id2"], [25, "grading"]], "How to take derivatives of Matrix-Vector expressions": [[16, "how-to-take-derivatives-of-matrix-vector-expressions"]], "Hyperplanes and all that": [[8, "hyperplanes-and-all-that"]], "Implementing the simplest form for gradient descent": [[18, "implementing-the-simplest-form-for-gradient-descent"]], "Important Matrix and vector handling packages": [[20, "important-matrix-and-vector-handling-packages"]], "Important technicalities: More on Rescaling data": [[26, "important-technicalities-more-on-rescaling-data"]], "Improving performance": [[1, "improving-performance"]], "In summary": [[23, "in-summary"]], "Including Stochastic Gradient Descent with Autograd": [[13, "including-stochastic-gradient-descent-with-autograd"]], "Incremental PCA": [[11, "incremental-pca"]], "Installing R, C++, cython or Julia": [[25, "installing-r-c-cython-or-julia"]], "Installing R, C++, cython, Numba etc": [[25, "installing-r-c-cython-numba-etc"]], "Instructor information": [[23, "instructor-information"]], "Interpretations and optimizing our parameters": [[25, "interpretations-and-optimizing-our-parameters"], [25, "id2"], [25, "id3"], [26, "interpretations-and-optimizing-our-parameters"], [26, "id1"], [26, "id2"]], "Interpreting the Ridge results": [[26, "interpreting-the-ridge-results"], [27, "interpreting-the-ridge-results"], [27, "id4"]], "Introducing JAX": [[13, "introducing-jax"]], "Introducing the Covariance and Correlation functions": [[11, "introducing-the-covariance-and-correlation-functions"], [26, "introducing-the-covariance-and-correlation-functions"]], "Introduction": [[0, "introduction"], [6, "introduction"], [19, "introduction"], [20, "introduction"]], "Iterative Fitting, Classification and AdaBoost": [[10, "iterative-fitting-classification-and-adaboost"]], "Iterative Fitting, Regression and Squared-error Cost Function": [[10, "iterative-fitting-regression-and-squared-error-cost-function"]], "Kernel PCA": [[11, "kernel-pca"]], "Kernels and non-linearity": [[8, "kernels-and-non-linearity"]], "LU Decomposition, the inverse of a matrix": [[20, "lu-decomposition-the-inverse-of-a-matrix"]], "Lasso Regression": [[27, "lasso-regression"]], "Lasso case": [[27, "lasso-case"]], "Layers": [[1, "layers"]], "Layers used to build CNNs": [[3, "layers-used-to-build-cnns"]], "Learning goals": [[15, "learning-goals"], [16, "learning-goals"], [17, "learning-goals"], [18, "learning-goals"]], "Learning outcomes": [[19, "learning-outcomes"], [25, "learning-outcomes"]], "Lectures and ComputerLab": [[25, "lectures-and-computerlab"]], "Limitations of supervised learning with deep networks": [[1, "limitations-of-supervised-learning-with-deep-networks"]], "Linear Algebra, Handling of Arrays and more Python Features": [[20, null]], "Linear Regression": [[0, null]], "Linear Regression Problems": [[26, "linear-regression-problems"], [27, "linear-regression-problems"]], "Linear Regression and the SVD": [[27, "linear-regression-and-the-svd"]], "Linear Regression, basic elements": [[0, "linear-regression-basic-elements"]], "Linking Bayes\u2019 Theorem with Ridge and Lasso Regression": [[5, "linking-bayes-theorem-with-ridge-and-lasso-regression"]], "Linking the regression analysis with a statistical interpretation": [[5, "linking-the-regression-analysis-with-a-statistical-interpretation"]], "Linking with the SVD": [[5, "linking-with-the-svd"], [26, "linking-with-the-svd"]], "Links to relevant courses at the University of Oslo": [[24, "links-to-relevant-courses-at-the-university-of-oslo"]], "Logistic Regression": [[7, null], [7, "id1"]], "MNIST and GANs": [[4, "mnist-and-gans"]], "Machine Learning": [[25, "machine-learning"]], "Machine learning": [[19, "machine-learning"]], "Main textbooks": [[25, "main-textbooks"]], "Making a tree": [[9, "making-a-tree"]], "Making your own Bootstrap: Changing the Level of the Decision Tree": [[10, "making-your-own-bootstrap-changing-the-level-of-the-decision-tree"]], "Making your own test-train splitting": [[26, "making-your-own-test-train-splitting"]], "Material for exercises week 35": [[26, "material-for-exercises-week-35"]], "Material for lab sessions sessions Tuesday and Wednesday": [[27, "material-for-lab-sessions-sessions-tuesday-and-wednesday"]], "Material for lecture Monday September 2": [[27, "material-for-lecture-monday-september-2"]], "Mathematical Interpretation of Ordinary Least Squares": [[5, "mathematical-interpretation-of-ordinary-least-squares"], [26, "mathematical-interpretation-of-ordinary-least-squares"], [27, "mathematical-interpretation-of-ordinary-least-squares"]], "Mathematical optimization of convex functions": [[8, "mathematical-optimization-of-convex-functions"]], "Mathematics of CNNs": [[3, "mathematics-of-cnns"]], "Mathematics of the SVD and implications": [[5, "mathematics-of-the-svd-and-implications"], [26, "mathematics-of-the-svd-and-implications"], [27, "mathematics-of-the-svd-and-implications"]], "Matrices in Python": [[25, "matrices-in-python"]], "Matrix multiplication": [[1, "matrix-multiplication"]], "Matrix-vector notation and activation": [[12, "matrix-vector-notation-and-activation"]], "Meet the covariance!": [[22, "meet-the-covariance"]], "Meet the Covariance Matrix": [[5, "meet-the-covariance-matrix"], [26, "meet-the-covariance-matrix"]], "Meet the Hessian Matrix": [[26, "meet-the-hessian-matrix"]], "Meet the Pandas": [[25, "meet-the-pandas"]], "Min-Max Scaling": [[26, "min-max-scaling"]], "Momentum based GD": [[13, "momentum-based-gd"]], "More complicated Example: The Ising model": [[6, "more-complicated-example-the-ising-model"]], "More interpretations": [[26, "more-interpretations"], [27, "more-interpretations"], [27, "id5"]], "More on Dimensionalities": [[3, "more-on-dimensionalities"]], "More on Rescaling data": [[6, "more-on-rescaling-data"]], "More on Steepest descent": [[27, "more-on-steepest-descent"]], "More on convex functions": [[27, "more-on-convex-functions"]], "More preprocessing": [[26, "more-preprocessing"]], "Multilayer perceptrons": [[12, "multilayer-perceptrons"]], "Network requirements": [[2, "network-requirements"]], "Neural Networks vs CNNs": [[3, "neural-networks-vs-cnns"]], "Neural networks": [[12, null]], "Note about SVD Calculations": [[26, "note-about-svd-calculations"], [27, "note-about-svd-calculations"]], "Note on Scikit-Learn": [[27, "note-on-scikit-learn"]], "Numerical experiments and the covariance, central limit theorem": [[22, "numerical-experiments-and-the-covariance-central-limit-theorem"]], "Numpy and arrays": [[20, "numpy-and-arrays"], [25, "numpy-and-arrays"]], "Numpy examples and Important Matrix and vector handling packages": [[25, "numpy-examples-and-important-matrix-and-vector-handling-packages"]], "Optimization and gradient descent, the central part of any Machine Learning algortithm": [[27, "optimization-and-gradient-descent-the-central-part-of-any-machine-learning-algortithm"]], "Optimization, the central part of any Machine Learning algortithm": [[13, null]], "Optimizing our parameters": [[25, "optimizing-our-parameters"]], "Optimizing our parameters, more details": [[25, "optimizing-our-parameters-more-details"]], "Optimizing the cost function": [[1, "optimizing-the-cost-function"]], "Organizing our data": [[0, "organizing-our-data"], [25, "organizing-our-data"]], "Other Matrix and Vector Operations": [[20, "other-matrix-and-vector-operations"]], "Other Types of Recurrent Neural Networks": [[4, "other-types-of-recurrent-neural-networks"]], "Other courses on Data science and Machine Learning at UiO": [[25, "other-courses-on-data-science-and-machine-learning-at-uio"]], "Other courses on Data science and Machine Learning at UiO, contn": [[25, "other-courses-on-data-science-and-machine-learning-at-uio-contn"]], "Other popular texts": [[25, "other-popular-texts"]], "Other techniques": [[11, "other-techniques"]], "Other types of networks": [[12, "other-types-of-networks"]], "Other ways of visualizing the trees": [[9, "other-ways-of-visualizing-the-trees"]], "Our model for the nuclear binding energies": [[25, "our-model-for-the-nuclear-binding-energies"]], "Overview of first week": [[25, "overview-of-first-week"]], "Own code for Ordinary Least Squares": [[25, "own-code-for-ordinary-least-squares"], [26, "own-code-for-ordinary-least-squares"]], "PCA and scikit-learn": [[11, "pca-and-scikit-learn"]], "Pandas AI": [[25, "pandas-ai"]], "Partial Differential Equations": [[2, "partial-differential-equations"]], "Plans for week 35": [[26, "plans-for-week-35"]], "Plans for week 36": [[27, "plans-for-week-36"]], "Practical tips": [[13, "practical-tips"]], "Practicalities": [[23, "practicalities"], [23, "id1"]], "Predicting New Points With A Trained Recurrent Neural Network": [[4, "predicting-new-points-with-a-trained-recurrent-neural-network"]], "Preprocessing our data": [[26, "preprocessing-our-data"]], "Prerequisites": [[25, "prerequisites"]], "Prerequisites and background": [[19, "prerequisites-and-background"]], "Prerequisites: Collect and pre-process data": [[3, "prerequisites-collect-and-pre-process-data"]], "Probability Distribution Functions": [[22, "probability-distribution-functions"]], "Program example for gradient descent with Ridge Regression": [[27, "program-example-for-gradient-descent-with-ridge-regression"]], "Program for stochastic gradient": [[13, "program-for-stochastic-gradient"]], "Properties of PDFs": [[22, "properties-of-pdfs"]], "Pros and cons of trees, pros": [[9, "pros-and-cons-of-trees-pros"]], "Python installers": [[19, "python-installers"], [25, "python-installers"]], "RMS prop": [[13, "rms-prop"]], "Random Numbers": [[22, "random-numbers"]], "Random forests": [[10, "random-forests"]], "Randomized PCA": [[11, "randomized-pca"]], "Reading material": [[25, "reading-material"]], "Reading recommendations:": [[26, "reading-recommendations"]], "Reading suggestions week 34": [[25, "reading-suggestions-week-34"]], "Recurrent neural networks": [[12, "recurrent-neural-networks"]], "Recurrent neural networks: Overarching view": [[4, null]], "Reducing the number of degrees of freedom, overarching view": [[0, "reducing-the-number-of-degrees-of-freedom-overarching-view"], [26, "reducing-the-number-of-degrees-of-freedom-overarching-view"]], "Reformulating the problem": [[2, "reformulating-the-problem"]], "Regression Case": [[10, "regression-case"]], "Regression analysis, overarching aims": [[25, "regression-analysis-overarching-aims"]], "Regression analysis, overarching aims II": [[25, "regression-analysis-overarching-aims-ii"]], "Regularization": [[1, "regularization"]], "Reminder from last week": [[26, "reminder-from-last-week"]], "Reminder on Newton-Raphson\u2019s method": [[27, "reminder-on-newton-raphson-s-method"]], "Reminder on Statistics": [[6, "reminder-on-statistics"]], "Replace or not": [[13, "replace-or-not"]], "Required Technologies": [[19, "required-technologies"]], "Resampling Methods": [[6, null]], "Resampling methods": [[6, "id1"]], "Residual Error": [[26, "residual-error"], [27, "residual-error"]], "Resources on differential equations and deep learning": [[2, "resources-on-differential-equations-and-deep-learning"]], "Revisiting Ordinary Least Squares": [[27, "revisiting-ordinary-least-squares"]], "Revisiting our Linear Regression Solvers": [[13, "revisiting-our-linear-regression-solvers"]], "Rewriting the Covariance and/or Correlation Matrix": [[26, "rewriting-the-covariance-and-or-correlation-matrix"]], "Rewriting the fitting procedure as a linear algebra problem": [[25, "rewriting-the-fitting-procedure-as-a-linear-algebra-problem"]], "Rewriting the fitting procedure as a linear algebra problem, more details": [[25, "rewriting-the-fitting-procedure-as-a-linear-algebra-problem-more-details"]], "Ridge Regression": [[27, "ridge-regression"]], "Ridge and LASSO Regression": [[26, "ridge-and-lasso-regression"], [27, "ridge-and-lasso-regression"], [27, "id2"]], "Ridge and Lasso Regression": [[5, null], [5, "id1"]], "Ridge regression and a new Synthetic Dataset": [[18, "ridge-regression-and-a-new-synthetic-dataset"]], "SVD analysis": [[27, "svd-analysis"]], "Same code but now with momentum gradient descent": [[13, "same-code-but-now-with-momentum-gradient-descent"]], "Schedule first week": [[25, "schedule-first-week"]], "Schematic Regression Procedure": [[9, "schematic-regression-procedure"]], "Setting up the Back propagation algorithm": [[12, "setting-up-the-back-propagation-algorithm"]], "Setting up the Matrix to be inverted": [[26, "setting-up-the-matrix-to-be-inverted"], [27, "setting-up-the-matrix-to-be-inverted"]], "Setting up the network using Autograd; The full program": [[2, "setting-up-the-network-using-autograd-the-full-program"]], "Similar (second order function now) problem but now with AdaGrad": [[13, "similar-second-order-function-now-problem-but-now-with-adagrad"]], "Simple Python Code to read in Data and perform Classification": [[9, "simple-python-code-to-read-in-data-and-perform-classification"]], "Simple case": [[26, "simple-case"], [27, "simple-case"]], "Simple code for solving the above problem": [[27, "simple-code-for-solving-the-above-problem"]], "Simple example to illustrate Ordinary Least Squares, Ridge and Lasso Regression": [[27, "simple-example-to-illustrate-ordinary-least-squares-ridge-and-lasso-regression"]], "Simple geometric interpretation": [[27, "simple-geometric-interpretation"]], "Simple linear regression model using scikit-learn": [[0, "simple-linear-regression-model-using-scikit-learn"], [25, "simple-linear-regression-model-using-scikit-learn"]], "Simple program": [[27, "simple-program"]], "Software and needed installations": [[25, "software-and-needed-installations"]], "Solving Differential Equations with Deep Learning": [[2, null]], "Solving the one dimensional Poisson equation": [[2, "solving-the-one-dimensional-poisson-equation"]], "Solving the wave equation with Neural Networks": [[2, "solving-the-wave-equation-with-neural-networks"]], "Some famous Matrices": [[20, "some-famous-matrices"]], "Some simple problems": [[13, "some-simple-problems"], [27, "some-simple-problems"]], "Some useful matrix and vector expressions": [[26, "some-useful-matrix-and-vector-expressions"]], "Splitting our Data in Training and Test data": [[0, "splitting-our-data-in-training-and-test-data"], [26, "splitting-our-data-in-training-and-test-data"]], "Standard steepest descent": [[13, "standard-steepest-descent"]], "Statistical analysis and optimization of data": [[19, "statistical-analysis-and-optimization-of-data"], [25, "statistical-analysis-and-optimization-of-data"]], "Steepest descent": [[13, "steepest-descent"], [27, "steepest-descent"]], "Stochastic Gradient Descent (SGD)": [[13, "stochastic-gradient-descent-sgd"]], "Stochastic variables and the main concepts, the discrete case": [[22, "stochastic-variables-and-the-main-concepts-the-discrete-case"]], "Support Vector Machines, overarching aims": [[8, null]], "Systematic reduction": [[3, "systematic-reduction"]], "Teachers": [[25, "teachers"]], "Teachers and Grading": [[23, null]], "Teaching Assistants Fall semester 2023": [[23, "teaching-assistants-fall-semester-2023"]], "Tentative deadllines for projects": [[23, "tentative-deadllines-for-projects"]], "Testing the Means Squared Error as function of Complexity": [[0, "testing-the-means-squared-error-as-function-of-complexity"], [26, "testing-the-means-squared-error-as-function-of-complexity"]], "Textbooks": [[24, null]], "The Algorithm before theorem": [[11, "the-algorithm-before-theorem"]], "The Breast Cancer Data, now with Keras": [[1, "the-breast-cancer-data-now-with-keras"]], "The CART algorithm for Classification": [[9, "the-cart-algorithm-for-classification"]], "The CART algorithm for Regression": [[9, "the-cart-algorithm-for-regression"]], "The CIFAR01 data set": [[3, "the-cifar01-data-set"]], "The Hessian matrix": [[27, "the-hessian-matrix"]], "The Hessian matrix for Ridge Regression": [[27, "the-hessian-matrix-for-ridge-regression"]], "The Jacobian": [[26, "the-jacobian"]], "The MNIST dataset again": [[3, "the-mnist-dataset-again"]], "The OLS case": [[27, "the-ols-case"]], "The RELU function family": [[1, "the-relu-function-family"]], "The Ridge case": [[27, "the-ridge-case"]], "The SVD, a Fantastic Algorithm": [[26, "the-svd-a-fantastic-algorithm"], [27, "the-svd-a-fantastic-algorithm"]], "The Softmax function": [[1, "the-softmax-function"]], "The \\chi^2 function": [[0, "the-chi-2-function"], [25, "the-chi-2-function"], [25, "id4"], [25, "id5"], [25, "id6"], [25, "id7"], [25, "id8"]], "The bias-variance tradeoff": [[6, "the-bias-variance-tradeoff"]], "The code for solving the ODE": [[2, "the-code-for-solving-the-ode"]], "The complete code with a simple data set": [[26, "the-complete-code-with-a-simple-data-set"]], "The cost/loss function": [[26, "the-cost-loss-function"]], "The course has two central parts": [[19, "the-course-has-two-central-parts"]], "The derivative of the cost/loss function": [[27, "the-derivative-of-the-cost-loss-function"]], "The equations": [[27, "the-equations"]], "The equations for ordinary least squares": [[26, "the-equations-for-ordinary-least-squares"]], "The first Case": [[27, "the-first-case"]], "The ideal": [[27, "the-ideal"]], "The logistic function": [[7, "the-logistic-function"]], "The mean squared error and its derivative": [[26, "the-mean-squared-error-and-its-derivative"]], "The moons example": [[8, "the-moons-example"]], "The multilayer perceptron (MLP)": [[12, "the-multilayer-perceptron-mlp"]], "The network with one input layer, specified number of hidden layers, and one output layer": [[2, "the-network-with-one-input-layer-specified-number-of-hidden-layers-and-one-output-layer"]], "The plethora of machine learning algorithms/methods": [[25, "the-plethora-of-machine-learning-algorithms-methods"]], "The sensitiveness of the gradient descent": [[27, "the-sensitiveness-of-the-gradient-descent"]], "The singular value decomposition": [[5, "the-singular-value-decomposition"], [26, "the-singular-value-decomposition"], [27, "the-singular-value-decomposition"]], "The two-dimensional case": [[8, "the-two-dimensional-case"]], "To our real data: nuclear binding energies. Brief reminder on masses and binding energies": [[25, "to-our-real-data-nuclear-binding-energies-brief-reminder-on-masses-and-binding-energies"]], "Topics covered in this course: Statistical analysis and optimization of data": [[25, "topics-covered-in-this-course-statistical-analysis-and-optimization-of-data"]], "Towards the PCA theorem": [[11, "towards-the-pca-theorem"]], "Train and test datasets": [[1, "train-and-test-datasets"]], "Two-dimensional Objects": [[3, "two-dimensional-objects"]], "Type of problem": [[2, "type-of-problem"]], "Types of Machine Learning": [[25, "types-of-machine-learning"]], "Useful Python libraries": [[19, "useful-python-libraries"], [25, "useful-python-libraries"]], "Using Autograd": [[13, "using-autograd"]], "Using forward Euler to solve the ODE": [[2, "using-forward-euler-to-solve-the-ode"]], "Using gradient descent methods, limitations": [[13, "using-gradient-descent-methods-limitations"], [27, "using-gradient-descent-methods-limitations"]], "Visualization": [[1, "visualization"], [1, "id1"]], "Visualizing the Tree, Classification": [[9, "visualizing-the-tree-classification"]], "Week 34: Introduction to the course, Logistics and Practicalities": [[25, null]], "Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression": [[26, null]], "Week 36: Linear Regression and Gradient descent": [[27, null]], "What Is Generative Modeling?": [[25, "what-is-generative-modeling"]], "What does it mean?": [[26, "what-does-it-mean"], [27, "what-does-it-mean"]], "What is Machine Learning?": [[0, "what-is-machine-learning"]], "What is a good model?": [[0, "what-is-a-good-model"], [25, "what-is-a-good-model"]], "What is a good model? Can we define it?": [[25, "what-is-a-good-model-can-we-define-it"]], "Which activation function should I use?": [[1, "which-activation-function-should-i-use"]], "Why Linear Regression (aka Ordinary Least Squares and family)": [[25, "why-linear-regression-aka-ordinary-least-squares-and-family"]], "Wisconsin Cancer Data": [[7, "wisconsin-cancer-data"]], "With Lasso Regression": [[27, "with-lasso-regression"]], "Writing Our First Generative Adversarial Network": [[4, "writing-our-first-generative-adversarial-network"]], "Writing our own PCA code": [[11, "writing-our-own-pca-code"]], "Writing the Cost Function": [[27, "writing-the-cost-function"]], "XGBoost: Extreme Gradient Boosting": [[10, "xgboost-extreme-gradient-boosting"]], "Yet another Example": [[27, "yet-another-example"]], "a) Expression for Ridge regression": [[17, "a-expression-for-ridge-regression"]], "scikit-learn implementation": [[1, "scikit-learn-implementation"]]}, "docnames": ["chapter1", "chapter10", "chapter11", "chapter12", "chapter13", "chapter2", "chapter3", "chapter4", "chapter5", "chapter6", "chapter7", "chapter8", "chapter9", "chapteroptimization", "clustering", "exercisesweek34", "exercisesweek35", "exercisesweek36", "exercisesweek37", "intro", "linalg", "schedule", "statistics", "teachers", "textbooks", "week34", "week35", "week36"], "envversion": {"sphinx": 62, "sphinx.domains.c": 3, "sphinx.domains.changeset": 1, "sphinx.domains.citation": 1, "sphinx.domains.cpp": 9, "sphinx.domains.index": 1, "sphinx.domains.javascript": 3, "sphinx.domains.math": 2, "sphinx.domains.python": 4, "sphinx.domains.rst": 2, "sphinx.domains.std": 2, "sphinx.ext.intersphinx": 1}, "filenames": ["chapter1.ipynb", "chapter10.ipynb", "chapter11.ipynb", "chapter12.ipynb", "chapter13.ipynb", "chapter2.ipynb", "chapter3.ipynb", "chapter4.ipynb", "chapter5.ipynb", "chapter6.ipynb", "chapter7.ipynb", "chapter8.ipynb", "chapter9.ipynb", "chapteroptimization.ipynb", "clustering.ipynb", "exercisesweek34.ipynb", "exercisesweek35.ipynb", "exercisesweek36.ipynb", "exercisesweek37.ipynb", "intro.md", "linalg.ipynb", "schedule.md", "statistics.ipynb", "teachers.md", "textbooks.md", "week34.ipynb", "week35.ipynb", "week36.ipynb"], "indexentries": {}, "objects": {}, "objnames": {}, "objtypes": {}, "terms": {"": [0, 1, 2, 3, 4, 5, 6, 7, 9, 11, 12, 13, 15, 16, 17, 19, 20, 22, 23, 25, 26], "0": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 22, 23, 25, 26, 27], "00": [0, 1, 5, 11, 25, 26], "000": [1, 3], "00000000e": [], "001": [2, 8, 13, 27], "004": 5, "004113634617443131": 26, "004113634617443139": 26, "00411363461744314": 26, "004113634617443147": 26, "00727646693": [0, 25], "0086649156": [0, 25], "01": [0, 1, 2, 5, 9, 11, 13, 17, 24, 25, 26], "0110": 22, "01719003e": [], "02": [0, 4, 7, 12, 25], "02334824": [], "02857": 4, "02f": 6, "03077640549": 4, "03097597e": [], "031": 5, "04": 11, "0458": 9, "05": [4, 6], "062292565": 4, "062435": [], "06730814": [], "07": [], "0713": [0, 25], "07285": 3, "08": 22, "08078025e": [], "08336233266": 4, "08376632": 26, "083766322923899": 26, "0837663229239043": 26, "0917": 9, "0n": [0, 25], "0x113e21950": 17, "1": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 20, 21, 22, 23, 24, 25, 27], "10": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 16, 17, 18, 20, 21, 22, 23, 25, 26, 27], "100": [0, 1, 2, 3, 4, 5, 6, 8, 9, 10, 11, 13, 14, 15, 16, 17, 18, 20, 22, 23, 25, 26, 27], "1000": [0, 1, 2, 4, 5, 8, 11, 13, 14, 18, 19, 22, 25, 27], "10000": [2, 5, 6, 10, 11, 13, 22], "100000": 8, "10001": 10, "1001": 22, "1002": 22, "1003": 22, "1005": 22, "1009": 22, "101": 16, "1011": 22, "1013": 22, "1013904243": 22, "1015": 22, "102": 16, "1023": 22, "1024": 3, "1026": 22, "1027": 22, "103": 1, "1030": 22, "1037": 22, "1038": 22, "1040": 22, "1047": 22, "107": 16, "108": [], "10th": 9, "10x": [0, 25], "11": [0, 2, 5, 6, 7, 8, 9, 10, 11, 12, 13, 16, 20, 22, 24, 25, 26, 27], "110": [], "1100": 22, "1101": 22, "111": [1, 7, 12], "112": 16, "11340253": [], "11590451": [], "116": 16, "117": 16, "118": 16, "12": [0, 1, 2, 3, 4, 5, 6, 8, 9, 11, 12, 18, 20, 22, 24, 25, 26, 27], "120": 3, "121": [8, 9, 10, 16], "1215pm": [23, 25], "122": [8, 9, 10], "124": [0, 25], "125": 16, "127": [4, 16], "128": [3, 4, 13], "129": 16, "1298": 9, "12pm": [23, 25], "13": [0, 2, 9, 12, 20, 22, 25], "131": 16, "133": 7, "135": 16, "136": 16, "14": [0, 2, 4, 6, 8, 9, 10, 12, 20, 22, 24, 26], "141": 16, "143": 16, "1446729567": 4, "149": 16, "14g": 6, "15": [0, 2, 4, 6, 7, 8, 9, 12, 13, 22, 25, 27], "150": [4, 8], "152": 16, "153760": [], "156": 16, "157": [], "158": [], "159": 16, "15g": 6, "15pm": 25, "16": [1, 2, 3, 4, 5, 8, 9, 10, 22, 25, 27], "160": 16, "1603": 3, "161": 16, "162": 16, "16231451": 4, "163": 16, "16384": 3, "164": 16, "167": 16, "17": [1, 2, 8, 22], "172": 16, "173": 16, "176": 16, "178": 16, "179": 16, "1797": 1, "18": [2, 6, 7, 8, 9, 10, 22, 25], "1807": 4, "18392847": [], "19": [2, 22, 25], "1940": [], "1943": 12, "19569961": 26, "1970": [20, 25], "1973": 9, "1979": 6, "1_1": 12, "1_2": 12, "1_3": 12, "1cm": [0, 8, 10, 22, 25], "1d": [1, 2, 3], "1e": [2, 4, 13, 14], "1e10": 14, "1e4": 6, "1f": 1, "1k": 20, "1n": [0, 25], "1x": [0, 25], "2": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 19, 20, 22, 24], "20": [0, 1, 2, 6, 7, 8, 16, 17, 22, 23, 25, 26, 27], "200": [0, 2, 3, 4, 8, 9, 10], "2000": [0, 26], "2004": [13, 27], "2006": 24, "20072279": [], "2008": 25, "2010": 1, "2011": 1, "2014": 4, "2015": 1, "2016": [0, 25], "2018": [0, 6, 26], "2021": [6, 14, 26], "2022": 25, "2025": [18, 25, 26, 27], "21": [0, 1, 5, 7, 9, 12, 20, 25, 26, 27], "2116753732": 4, "215pm": [23, 25], "2167072": [], "22": [0, 1, 5, 12, 13, 20, 25, 26, 27], "221": 8, "225": 4, "22948497": [], "23": [1, 12, 20], "24": [0, 1, 20, 25], "25": [2, 3, 4, 5, 6, 8, 9, 11, 26], "250": [2, 4, 7, 9], "25000": [], "250154": [], "253775": [], "255": 3, "256": 4, "26": [], "26303845": [], "264": [], "265": [], "265109911": 4, "266": [], "269": [], "27": 1, "270": [], "278": 27, "27n_": 22, "28": [1, 3, 4], "283": 27, "2830637392": 4, "2861": 22, "2873": 9, "2882": 22, "2886": 22, "2890": [0, 25], "2892": 22, "29": 26, "2915": 22, "2931": 25, "29364655": [], "294399745619595": [], "296247": [], "2968": 25, "2980": 25, "298273": [], "298375": [], "2990": 25, "2_": 12, "2_1": 12, "2_2": 12, "2_3": 12, "2_i": 12, "2_m": [6, 22], "2_t": 13, "2_x": 22, "2a": 17, "2b": 22, "2cm": 8, "2d": [1, 3, 11, 12, 19, 25], "2e": 6, "2f": [0, 7, 9, 10, 11, 12, 25], "2g": 2, "2g_i": 2, "2k": 3, "2m": 6, "2mvizaqfst8": 26, "2n": [0, 2, 3, 25, 26], "2nd": 9, "2p": 22, "2pt": 4, "2x": [0, 3, 8, 13, 25], "2x_ix_jy_iy_j": 8, "2x_j": 8, "2y_i": 10, "2y_j": 8, "3": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 18, 19, 20, 21, 22, 23, 25, 27], "30": [0, 1, 4, 6, 7, 10, 13, 23], "30000": [0, 25], "3072": 3, "31": [12, 20, 22], "315": [6, 26], "3155": [0, 5, 6, 26, 27], "32": [3, 4, 6, 12, 13, 20, 22], "3200": 1, "3250": 1, "3297": [], "33": [12, 20, 23], "3303": [], "3310": [], "332331": [], "333": 7, "3331": [], "3337": [], "34": 20, "3436": [0, 25], "3437": [0, 25], "35": [0, 6, 25, 27], "3581341341": 4, "359": [5, 27], "36": [0, 5, 6, 22], "37": 27, "370782966": 4, "38": 22, "39": [0, 23, 25], "3d": [2, 3, 4, 6, 13, 16], "3f": [1, 3, 9], "3n": 20, "3x": [2, 8], "3x_i": 2, "3y": 8, "4": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 18, 20, 22, 25, 27], "40": [1, 6, 23, 25], "400": 4, "4000": 25, "4050": [24, 25], "41": 20, "4155": [2, 15], "41589548": [], "42": [1, 4, 8, 9, 10, 20], "43": [0, 7, 20], "4310": 25, "436462435": 4, "44": [0, 20, 27], "45": [23, 25], "46": [23, 25], "462": 7, "47": [23, 25], "479465113": 4, "47958494": [], "48": [], "48257387": [23, 25], "49": [5, 6, 11], "49152": 3, "4940954": [0, 25], "4990": 22, "4992": 22, "4997": 22, "4c4c7f": [9, 10], "4d": 3, "4f": 6, "4pm": [23, 25], "4y": 8, "4y_i": 10, "5": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 18, 20, 22, 25, 26, 27], "50": [1, 2, 3, 4, 6, 7, 8, 10, 13, 25, 26], "500": [1, 3, 4, 6, 9, 10, 13], "5018": 22, "506": [], "507d50": [9, 10], "50j": 13, "50x10": 1, "51": 10, "510": 1, "512132": [], "5177783846": 4, "53": 9, "54": [6, 22], "5411205": [], "54894451": [], "55": 1, "56": 1, "56536": [0, 25], "569": 1, "57": [0, 8, 23, 25], "571": [5, 27], "58": [10, 23, 25], "591317992": 4, "5cm": 22, "5f": 8, "5x": 8, "5y": 8, "6": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 18, 20, 22, 23, 25, 26, 27], "60": [1, 3], "60000": 4, "6019067271": 4, "606439": [], "625": 7, "63": 1, "64": [1, 3, 4, 13, 20, 25], "64x50": 1, "65": [1, 8, 9], "6887363571": 4, "69": [16, 22], "69069n_": 22, "691": [], "6n_": 22, "7": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 20, 22, 24, 25, 26], "70": [1, 7], "70653767": 4, "71": 1, "724": 3, "73": [], "7304881": [], "75": [5, 6, 8, 11], "76": [23, 25], "765": 7, "77": [23, 25], "7718": 9, "7782028952": 4, "77893972": [], "78": [], "7d7d58": [9, 10], "8": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 14, 18, 20, 22, 23, 25, 27], "80": [0, 1, 5, 8, 17, 26], "800": [4, 7], "81": 1, "815am": [23, 25], "85": 1, "8702784034": 4, "88": 25, "8f": 6, "8g": 6, "8n": 20, "8x8": 1, "9": [0, 1, 2, 4, 5, 6, 7, 8, 9, 11, 12, 13, 20, 22, 25], "90": 1, "9040": 9, "91": [23, 25], "92": [23, 25], "93": 16, "931": [0, 25], "933": [5, 27], "937": 22, "938": 22, "939": [0, 22, 25], "94": 22, "95": [1, 11], "954": 22, "955820c21e8b": 4, "96": 6, "960": 22, "961": 22, "962": 22, "9649652536": 4, "96611194e": [], "9780387310732": 24, "9780387848570": 24, "9781098134174": 25, "9781492032632": 24, "9781801819312": 25, "97898392": 26, "98": [0, 1, 16], "985": 22, "986": 22, "989": 22, "9898ff": [9, 10], "99": [13, 16], "991": 22, "992": 22, "993": 22, "996": 5, "999": [9, 22], "9x": 6, "9y": 6, "A": [2, 3, 5, 6, 7, 10, 11, 12, 13, 15, 16, 19, 20, 21, 22, 23, 24, 26, 27], "AND": 2, "And": [0, 3, 4, 5, 6, 9, 13, 19, 22, 27], "As": [0, 1, 2, 3, 4, 5, 6, 8, 10, 12, 13, 15, 16, 20, 22, 25, 26, 27], "At": [0, 4, 6, 13, 25], "BE": [0, 25], "Be": [2, 18, 19, 25], "Being": 13, "But": [0, 1, 2, 3, 5, 6, 9, 10, 16, 22, 26], "By": [0, 3, 5, 6, 12, 13, 17, 20, 25, 26, 27], "For": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 18, 19, 20, 22, 24, 25, 26, 27], "IF": [6, 26], "IN": 24, "If": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16, 18, 19, 20, 22, 25, 26, 27], "In": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 19, 20, 22, 24, 25, 26, 27], "Ising": [5, 12, 26, 27], "It": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 19, 20, 22, 25, 26, 27], "Its": [1, 2, 4, 11], "No": [6, 9, 25, 26], "Not": [0, 1, 5, 6, 26, 27], "OR": 22, "Of": 22, "On": [0, 3, 22, 23, 24, 25], "One": [0, 1, 3, 4, 5, 6, 7, 8, 11, 12, 13, 17, 22, 26, 27], "Or": [0, 1, 6, 25], "Such": [0, 6, 12, 16, 22], "That": [0, 5, 7, 10, 11, 12, 14, 22, 25], "The": [4, 10, 13, 14, 16, 17, 18, 20, 21, 22, 23, 24], "Then": [0, 1, 6, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 25, 27], "There": [0, 3, 4, 5, 6, 8, 9, 11, 12, 14, 15, 20, 22, 23, 25, 26, 27], "These": [0, 3, 4, 5, 8, 9, 10, 11, 12, 13, 14, 17, 20, 22, 23, 25, 26, 27], "To": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16, 20, 22, 26, 27], "With": [0, 5, 6, 8, 9, 10, 11, 12, 14, 16, 20, 22, 25, 26], "_": [0, 1, 2, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 16, 17, 20, 25, 26, 27], "_0": [5, 8, 10, 11, 13, 26, 27], "_1": [2, 5, 6, 8, 10, 11, 12, 13, 14, 20, 26, 27], "_2": [2, 5, 8, 11, 12, 13, 20, 26], "_3": 20, "_4": 20, "_9": 13, "__class__": 10, "__doc__": 6, "__future__": [8, 9], "__init__": 1, "__main__": 2, "__name__": [2, 10], "_auto1": [2, 3, 4, 5, 6, 7, 12, 13, 20, 22, 26, 27], "_auto10": [6, 12], "_auto11": 6, "_auto12": 6, "_auto2": [2, 3, 4, 5, 6, 12, 13, 20, 22], "_auto3": [3, 4, 5, 6, 12, 13, 20], "_auto4": [4, 6, 12, 13, 20], "_auto5": [4, 6, 12, 13, 20], "_auto6": [4, 6, 12, 20], "_auto7": [4, 6, 12, 20], "_auto8": [6, 12], "_auto9": [6, 12], "_build": [0, 19, 24, 25], "_c": 1, "_center": 18, "_compon": 11, "_depth": 9, "_export": [15, 16], "_fraction": 9, "_i": [0, 1, 2, 5, 6, 7, 8, 11, 12, 13, 25, 26, 27], "_j": [0, 1, 2, 3, 5, 6, 8, 13, 26, 27], "_k": [13, 27], "_l": 12, "_lambda": 6, "_leaf": 9, "_m": 10, "_multilayer_perceptron": [], "_n": [2, 5, 8, 11, 13, 26, 27], "_node": 9, "_norm": 18, "_p": [5, 8, 26, 27], "_ratio": 11, "_sampl": 9, "_split": [6, 9], "_t": 13, "_test": 6, "_varianc": 11, "_weight": 9, "a0": 3, "a0faa0": [9, 10], "a1": [0, 25], "a2": [0, 25], "a3": [0, 25], "a4": [0, 25], "a_": [0, 1, 16, 20, 25, 26], "a_0": [0, 25], "a_1a": [0, 25], "a_2a": [0, 25], "a_3": [0, 25], "a_3a": [0, 25], "a_4": [0, 25], "a_4a": [0, 25], "a_h": 1, "a_i": [0, 1, 2, 12, 25], "a_j": [1, 12], "a_k": [0, 1, 12], "aaron": 24, "ab": [0, 2, 5, 13, 14, 25, 26], "ab_channel": 19, "abandon": 1, "abid": 22, "abil": [0, 10], "abl": [0, 1, 4, 5, 6, 7, 10, 12, 13, 16, 18, 26, 27], "about": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16, 19, 20, 23], "abov": [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 22, 24, 25, 26], "abovement": [6, 25], "abscissa": [13, 27], "absolut": [0, 2, 5, 6, 13, 25, 26, 27], "absorb": [26, 27], "abstract": 1, "acceler": 13, "accept": [0, 3, 6, 9, 26], "access": [3, 11, 22, 25], "accid": [4, 6], "accompani": [0, 25, 26], "accomplish": [8, 9, 13], "accord": [0, 1, 2, 5, 6, 9, 12, 13, 14, 22, 25, 27], "accordingli": 11, "account": [0, 3, 5, 13, 15, 16, 22, 25], "accumul": [12, 13, 22], "accur": [0, 3, 4, 6, 10, 13], "accuraci": [0, 1, 3, 4, 5, 6, 7, 9, 10, 11, 12, 25, 26, 27], "accuracy_scor": [0, 1, 10, 25], "accuracy_score_numpi": 1, "achiev": [0, 1, 5, 6, 8, 12, 20, 25], "aco": 22, "acquaint": 19, "acquir": [1, 19, 25], "acr": [], "across": [1, 3, 6, 9, 17, 19, 25], "act": [1, 3, 20], "action": 22, "activ": [0, 2, 3, 4, 9, 15, 21, 23, 25], "actual": [0, 1, 4, 5, 6, 8, 11, 15, 16, 18, 20, 22, 25, 26, 27], "ad": [1, 3, 4, 5, 8, 13, 15, 16, 20, 27], "ada_clf": 10, "adaboostclassifi": 10, "adadelta": 13, "adam": [1, 3, 4, 25], "adapt": [4, 6, 13, 17, 24, 27], "add": [0, 1, 2, 3, 4, 5, 6, 8, 10, 11, 12, 15, 16, 17, 18, 22, 23, 25, 26, 27], "add_subplot": [1, 7, 12, 14], "addendum": 5, "addit": [0, 2, 3, 5, 6, 7, 8, 9, 10, 12, 13, 15, 19, 20, 22, 23, 24, 25, 26], "addition": [12, 13, 27], "address": [1, 9, 11, 13, 25], "adjac": [3, 12], "adjoint": [5, 26], "adjust": [0, 5, 12, 13, 27], "admir": [0, 25], "advanc": [4, 6, 12, 24, 25], "advantag": [1, 3, 5, 6, 10, 13, 20, 27], "adversari": 25, "afecionado": 25, "affect": [3, 15], "affin": [0, 3, 8, 11, 26], "afford": 3, "aficionado": 25, "aforement": 14, "african": [], "after": [0, 1, 2, 4, 5, 6, 9, 11, 12, 13, 15, 16, 17, 18, 19, 20, 22, 25, 26, 27], "afterward": [0, 25], "ag": [0, 7, 25, 26], "ag_0": 2, "again": [0, 1, 4, 5, 6, 7, 8, 10, 11, 12, 13, 22, 25, 26, 27], "against": [1, 4, 7, 10], "agegroup": 7, "agegroupmean": 7, "aggreg": [9, 10], "agorithm": 10, "agre": [5, 6, 22, 26, 27], "agreement": 13, "ahead": 9, "ai": [0, 24], "aid": 11, "aim": [0, 1, 4, 6, 7, 11, 14, 16, 17, 19, 20, 26], "ainv": 5, "airplan": 3, "aka": 5, "al": [0, 2, 4, 16, 17, 24, 25, 26, 27], "alarm": [5, 7], "aldo": 26, "algebra": [0, 3, 5, 13, 19, 26, 27], "algorithm": [0, 1, 2, 4, 5, 6, 7, 8, 13, 14, 16, 19, 20, 22, 24], "align": [0, 2, 5, 6, 7, 8, 13, 22, 25, 26, 27], "all": [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, 14, 15, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27], "allevi": [1, 13, 27], "alloc": [3, 20], "allow": [0, 1, 2, 3, 5, 6, 8, 10, 13, 15, 19, 20, 25, 26, 27], "almost": [0, 1, 6, 8, 11, 13, 22, 27], "alon": [2, 9], "along": [2, 3, 4, 5, 6, 9, 10, 11, 15, 19, 20, 25, 26, 27], "alpha": [0, 1, 2, 3, 4, 6, 7, 8, 9, 10, 13, 14, 22, 25, 26, 27], "alpha_": 10, "alpha_0": 3, "alpha_1": 3, "alpha_2": 3, "alpha_i": [3, 13], "alpha_k": 13, "alpha_m": 10, "alpha_n": 3, "alpha_opt": 13, "alreadi": [2, 3, 4, 5, 6, 10, 12, 15, 19, 20, 22, 25, 26, 27], "also": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 19, 20, 22, 25, 26, 27], "alter": 1, "altern": [0, 1, 4, 5, 6, 8, 9, 11, 13, 15, 18, 20, 25, 26], "although": [0, 1, 5, 6, 8, 10, 13, 16, 25], "alwai": [0, 3, 5, 6, 12, 13, 16, 22, 25, 26, 27], "am": 4, "ame2016": [0, 25], "american": [], "among": [0, 3, 5, 9, 10, 12, 20, 25, 26], "amongst": 5, "amount": [0, 1, 3, 4, 6, 8, 10, 14, 19], "an": [1, 2, 3, 5, 6, 7, 8, 9, 11, 12, 13, 14, 16, 17, 18, 19, 20, 22, 23, 24, 26, 27], "an_": 22, "anaconda": [0, 1, 19, 25], "analogi": 13, "analys": 6, "analysi": [1, 3, 4, 7, 14, 20, 24], "analyt": [2, 3, 5, 6, 7, 12, 13, 17, 19, 25, 26, 27], "analyz": [0, 1, 3, 4, 5, 6, 16, 22, 26, 27], "andrew": 1, "angl": [0, 3, 9, 26], "anharmon": 3, "ani": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 14, 15, 16, 22, 25, 26], "anim": [4, 12], "ann": 12, "annot": [0, 1, 3, 7, 8, 25], "announc": 25, "anoth": [0, 1, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 15, 20, 22, 25, 26], "ansatz": [0, 25], "answer": [0, 1, 3, 5, 6, 20, 23, 25], "antialias": [2, 6], "anticip": 4, "anymor": [1, 8], "anyon": [4, 8, 15], "anyth": [1, 15, 16, 22], "anytim": [23, 25], "apach": 1, "apart": [11, 13, 27], "api": [1, 19, 25], "appar": 2, "appear": [0, 1, 3, 13, 20, 22], "append": [1, 3, 4, 8, 9, 13, 25], "appli": [0, 1, 3, 4, 6, 7, 8, 9, 10, 11, 12, 13, 18, 22, 24, 25, 26], "applic": [0, 1, 3, 4, 5, 6, 7, 9, 12, 13, 16, 20, 22, 24, 25, 26, 27], "apply_gradi": 4, "approach": [1, 2, 4, 5, 6, 9, 10, 11, 12, 13, 15, 16, 18, 19, 22, 24, 26, 27], "appropri": [2, 6, 9, 12, 13, 17, 19, 22], "approv": 25, "approx": [0, 2, 3, 6, 10, 11, 13, 18, 22, 25, 27], "approxim": [0, 1, 2, 3, 4, 5, 6, 7, 10, 11, 13, 22, 25, 26, 27], "apt": [0, 19, 25], "aq": 22, "ar": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27], "aragorn": 25, "arang": [1, 3, 4, 6, 7, 9, 10, 12, 13, 25], "arbitrari": [1, 4, 6, 8, 12, 13, 22, 27], "arbitrarili": [0, 1, 11, 25], "arc": 6, "architectur": [3, 4, 12], "area": [0, 3, 6, 24, 25], "argmax": [1, 11], "argmin": [4, 10, 14], "argsort": 11, "argu": [1, 13], "argument": [0, 2, 3, 5, 11, 12, 13, 17, 25, 26], "aris": [0, 6, 12, 13, 22, 25, 27], "arithmet": [0, 13, 20, 25], "arm": [6, 26], "armadillo": 20, "around": [0, 1, 4, 5, 6, 11, 18, 22, 25], "arrai": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 12, 13, 14, 16, 18, 19, 22, 26, 27], "arrang": [3, 25], "arraybox": 13, "arriv": [0, 6, 9, 11, 20, 22, 25], "arrow": 12, "arrowprop": 8, "art": [0, 1, 19], "articl": [0, 3, 4, 6, 10, 25, 26, 27], "artifici": [0, 2, 7, 12, 24, 25], "artificialneuron": 12, "arug": 13, "arxiv": [3, 4], "asarrai": [0, 6, 9, 26], "asid": 26, "ask": [5, 6, 11, 12, 15], "aspect": [0, 6, 19, 25, 26], "assembl": 3, "assembli": [0, 25], "assert": 4, "assess": [0, 6, 25, 26], "assici": 4, "assign": [0, 7, 8, 9, 12, 13, 14, 15, 21, 23, 24, 25], "associ": [0, 6, 9, 12, 14, 22, 25], "assum": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 17, 20, 22, 25, 26, 27], "assumpt": [0, 3, 5, 6, 9, 11, 22, 25, 26], "ast": [0, 5, 6, 25], "astyp": [4, 9, 10], "asymmetri": [0, 25], "asymptot": [4, 6], "atom": [0, 25], "attempt": [0, 4, 6, 7, 8, 10, 25, 26], "attend": 25, "attent": [0, 20, 25], "attract": [0, 10, 25], "attribut": [0, 9, 25], "audi": [0, 25], "audio": [3, 4], "august": [25, 26], "aurelien": [0, 24, 25], "austfjel": 6, "auth": 15, "authent": 15, "author": [0, 1, 10, 22], "authour": 25, "auto": [9, 10, 22], "auto_exampl": 26, "autocor": 22, "autocorrelation_tim": 22, "autocorrelform": 22, "autocovari": 22, "autoencod": [4, 19, 25], "autoencond": 19, "autograd": [19, 25], "autom": [0, 19, 24, 25], "automac": 20, "automag": 25, "automat": [0, 1, 2, 3, 4, 11, 16, 19, 20, 25], "automobil": 3, "autonom": 4, "avail": [0, 1, 4, 6, 10, 11, 19, 20, 21, 23, 24, 25], "averag": [0, 1, 3, 6, 9, 10, 13, 14, 22, 23, 25, 26], "avoid": [0, 4, 5, 6, 9, 11, 13, 18, 20, 26], "awai": [2, 3, 6, 26], "awar": [2, 10], "award": [23, 25], "ax": [0, 1, 2, 3, 4, 6, 7, 8, 9, 10, 11, 12, 13, 14, 20, 25], "axes3d": [2, 6, 13, 27], "axes_grid1": 6, "axhlin": 8, "axi": [0, 1, 2, 3, 4, 6, 7, 8, 9, 10, 11, 12, 13, 14, 18, 20, 22, 25, 26, 27], "axiom": 5, "axvlin": [4, 8], "axvspan": 4, "b": [0, 1, 3, 4, 5, 6, 8, 9, 10, 12, 13, 14, 15, 16, 17, 22, 23, 25, 26, 27], "b1": 8, "b2": 8, "b3": 8, "b_": [0, 1, 20], "b_0": 0, "b_1": [0, 2, 12, 13], "b_2": [0, 13], "b_5": 13, "b_group": 9, "b_i": [0, 1, 2, 12, 25], "b_ia_": [0, 25], "b_ia_i": 0, "b_index": 9, "b_j": [1, 12], "b_k": [0, 1, 12, 13], "b_m": 12, "b_score": 9, "b_valu": 9, "babcock": 25, "bachelor": [21, 23], "back": [0, 3, 4, 5, 6, 8, 9, 10, 15, 16, 20, 22, 25], "backbon": 20, "backend": [1, 4], "background": [24, 25], "backpropag": 1, "backtrack": 9, "backup": 20, "backward": [1, 2, 4, 12, 20], "bad": [6, 17, 26], "badli": 22, "bag": [9, 19, 25], "bag_clf": 10, "baggin": 25, "baggingboot": 10, "baggingclassifi": 10, "baggingtre": 10, "balanc": 6, "ballpark": 18, "band": 20, "bandwidth": 20, "bar": [0, 6, 11, 25], "barber": 24, "bare": [4, 10], "base": [0, 1, 3, 4, 5, 7, 8, 9, 10, 14, 15, 16, 17, 19, 22, 23, 24, 25, 26, 27], "basi": [5, 7, 8, 10, 11, 12, 13, 20, 26, 27], "basic": [6, 8, 12, 13, 14, 15, 19, 22, 25], "batch": [3, 4, 11, 12, 13, 27], "batch_shap": 4, "batch_siz": [1, 3, 4], "batchnorm": 4, "bay": 7, "bayesian": [5, 19, 24, 25], "becaus": [0, 1, 2, 3, 4, 5, 6, 8, 9, 12, 13, 14, 25, 26, 27], "becom": [0, 1, 2, 5, 6, 7, 9, 12, 13, 22, 25, 26, 27], "been": [0, 1, 2, 3, 4, 5, 6, 11, 12, 13, 19, 20, 25, 26], "befor": [0, 1, 2, 3, 4, 5, 6, 7, 8, 12, 13, 14, 16, 17, 18, 20, 22, 25, 26], "beforehand": [0, 22, 25], "begin": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 14, 15, 20, 22, 23, 25, 26, 27], "behav": [1, 6, 13, 27], "behavior": [0, 1, 13, 25, 27], "behaviour": 12, "behind": [0, 1, 6, 8, 13, 25, 27], "being": [0, 1, 2, 3, 4, 5, 7, 8, 10, 11, 12, 13, 17, 22, 25, 26, 27], "believ": [9, 20], "belong": [7, 8, 9, 13, 14, 27], "below": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 18, 20, 22, 25, 26, 27], "benchmark": 10, "benefici": [1, 13], "benefit": [0, 1, 4, 11, 13, 19, 25, 27], "bengio": [1, 24, 25, 26], "benign": [1, 7], "besid": [4, 5, 27], "bessel": [5, 26], "best": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 15, 16, 23, 25, 26, 27], "beta": [1, 3, 10, 11, 13, 16, 17, 25, 26, 27], "beta_": [3, 13, 17, 26], "beta_0": [1, 3, 13, 26], "beta_1": [1, 3, 10, 13, 26], "beta_1x_i": 13, "beta_2": [3, 13], "beta_3": 3, "beta_i": 3, "beta_j": [13, 18, 26], "beta_k": 13, "beta_linreg": 13, "beta_m": 10, "beta_mg_m": 10, "beta_n": 3, "better": [0, 1, 2, 3, 4, 6, 9, 10, 11, 12, 13, 25, 26], "between": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 14, 15, 16, 17, 22, 25, 26, 27], "beyond": [0, 1, 5, 6, 8, 13, 25, 26, 27], "bf": [13, 14, 20, 22, 27], "bg": 25, "bgd": 13, "bia": [0, 1, 2, 3, 5, 8, 9, 10, 12, 13, 25, 26, 27], "bias": [1, 2, 3, 5, 6, 9, 12], "big": [0, 1, 2, 5, 6, 14], "bigger": [1, 6, 26], "bigr": 12, "bike": 9, "bilbo": 25, "billion": [3, 12, 19], "bin": [7, 22], "binari": [0, 3, 5, 7, 9, 10, 12, 25], "binarycrossentropi": 4, "bind": 0, "binomi": [19, 22, 25], "binsboot": 6, "bioinformat": 0, "biolog": [1, 12], "bios1100": [19, 25], "bird": [0, 3], "birth": 25, "bishop": [24, 25], "bit": [1, 4, 20, 22, 25], "bitwis": 22, "bivari": 2, "bk": 13, "bla": [20, 25], "black": [8, 9, 14], "block": [6, 10, 19, 20, 22, 25], "blog": 25, "blogpost": 4, "blue": [0, 3], "bm": 18, "bmatrix": [0, 1, 3, 5, 7, 8, 11, 13, 20, 25, 26, 27], "bmi": 1, "bodi": [0, 1, 4, 12], "bold": 1, "boldfac": [0, 5, 16, 26, 27], "boldsymbol": [0, 1, 2, 3, 5, 6, 7, 8, 10, 11, 13, 14, 16, 17, 25, 27], "boltzmann": [12, 19, 25], "book": [17, 24, 25, 26], "book1": 24, "boolean": [4, 17], "boost": [1, 9, 19, 25], "boostrap": 10, "bootstrap": [1, 13, 19, 25], "borrow": 25, "boston_dataset": [], "bot": 8, "both": [0, 1, 4, 5, 6, 8, 9, 10, 13, 14, 15, 16, 17, 19, 20, 22, 23, 25, 26, 27], "bottl": 7, "bound": [8, 12], "boundari": [2, 4, 8, 11, 12], "box": [4, 9], "boyd": [8, 13, 27], "bracket": [4, 22], "brain": [1, 7, 12], "branch": [9, 25], "break": [0, 4, 6, 11, 14, 25], "breast": [5, 7, 11], "breviti": 13, "brew": [0, 19, 25], "brg": 8, "brief": 26, "briefli": [0, 16, 25], "bring": [0, 5, 6, 10, 26], "britt": [23, 25], "broad": 0, "broadli": 25, "brought": [13, 19, 25], "brownle": 4, "browser": [15, 25], "brute": [3, 5, 11, 26], "buffer_s": 4, "bui": 4, "build": [0, 4, 5, 6, 10, 16, 20, 22, 25], "built": [1, 3, 4, 6], "bunch": 11, "busi": [], "byte": [20, 25], "c": [0, 1, 2, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 19, 20, 21, 22, 23, 24, 26, 27], "c1": [8, 11], "c2": [8, 11], "c_": [8, 9, 10, 13, 22, 27], "c_0": 22, "c_1": 12, "c_2": 12, "c_3": 12, "c_4": 12, "c_i": [12, 13], "c_k": 22, "ca": [1, 25], "cach": 10, "cal": [0, 8, 10, 12, 13, 27], "calcul": [0, 1, 2, 4, 5, 6, 8, 9, 10, 11, 12, 13, 14, 16, 20, 22, 25], "call": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 19, 20, 22, 23, 25, 26, 27], "calor": [0, 26], "cambridg": [13, 24, 27], "can": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 23, 24, 26, 27], "cancel": [0, 13, 25, 26], "cancer": [5, 10], "cancerpd": 7, "candid": [8, 9, 10], "cannot": [0, 1, 4, 5, 6, 7, 8, 9, 22, 26, 27], "canopi": [0, 19, 25], "canva": [15, 16, 25], "cap": 5, "capabl": [0, 1, 8, 13, 19, 25], "capac": [2, 23], "capita": [], "captur": [4, 11, 12, 25], "car": [3, 4], "card": [0, 7, 25], "cardin": 1, "care": [11, 15], "carefulli": 13, "carlo": [0, 6, 19, 22, 24, 25], "carri": [2, 6, 7], "cart": 10, "case": [0, 1, 2, 3, 4, 5, 6, 7, 11, 12, 13, 14, 15, 16, 19, 20, 25], "casella": 24, "cast": 1, "cat": [3, 4], "catch": 0, "categor": [0, 1, 3, 9, 11, 25], "categori": [0, 1, 3, 7, 10, 12, 14, 25], "categorical_crossentropi": [1, 3], "caus": [0, 5, 6, 22, 25, 26, 27], "causal": 0, "causat": [0, 25], "cax": 1, "cb": [6, 25], "cbar": 1, "cc": [0, 1, 5, 13, 25, 26, 27], "ccc": [5, 12, 27], "cdf": 22, "cdot": [0, 2, 6, 12, 13, 14, 20, 22, 25, 27], "celebr": [13, 27], "cell": 4, "center": [0, 1, 6, 7, 8, 9, 11, 14, 18, 22, 25, 26], "central": [0, 3, 5, 6, 8, 16, 20, 25, 26], "centroid": [14, 22], "centroid_differ": 14, "centuri": 3, "certain": [0, 3, 6, 7, 9, 22, 25, 26], "cg": 13, "cha": [], "chain": [0, 1, 13, 19, 22, 25], "challeng": 15, "chanc": [1, 5, 13, 22], "chang": [0, 1, 2, 3, 4, 5, 6, 8, 9, 11, 12, 13, 14, 15, 16, 20, 22, 25, 26, 27], "channel": 3, "chapter": [0, 6, 10, 11, 16, 17, 20, 24, 25, 26, 27], "chapter3": 0, "charact": [0, 3, 5, 25, 26, 27], "character": [8, 9, 10, 12, 22], "characterist": [0, 1, 3, 10, 13, 25], "charg": [0, 25], "charl": [], "chase": 4, "chatgpt": 15, "chd": 7, "chddata": 7, "cheap": [5, 26, 27], "cheaper": [1, 13], "check": [1, 3, 4, 5, 11, 13, 15, 16, 20, 25], "checkmark": 3, "checkpoint": 4, "checkpoint_dir": 4, "checkpoint_prefix": 4, "chen": 10, "cheng": 26, "chiaramont": 2, "childcar": 16, "children": 16, "choic": [0, 1, 2, 3, 4, 6, 9, 12, 13, 14, 20, 25, 26, 27], "choleski": [5, 20, 26, 27], "choos": [2, 3, 6, 9, 10, 11, 13, 14, 15, 27], "chosen": [0, 1, 2, 6, 8, 9, 10, 13, 16, 22, 25, 27], "chosen_datapoint": 1, "christian": 24, "christoph": [24, 25], "cifar": 3, "cifar10": 3, "circ": [1, 12], "circl": [0, 8, 12, 26], "circuit": 3, "circumfer": 9, "circumv": [1, 5, 13, 26, 27], "ckpt": 4, "clariti": 22, "class": [0, 1, 3, 4, 6, 7, 8, 9, 11, 12, 13, 22, 25], "class_nam": [3, 9], "class_val": 9, "class_valu": 9, "classic": [7, 9, 13], "classif": [0, 3, 5, 6, 7, 8, 11, 12, 19, 24, 25, 26], "classifi": [0, 1, 4, 7, 9, 10, 11, 25], "classificaton": 1, "classifii": 10, "clean": 1, "clear": [1, 5, 10, 12, 13], "clearli": [0, 3, 5, 6, 7, 8, 22, 26, 27], "clever": [1, 10], "clf": [0, 6, 8, 9, 10, 25, 26], "clf3": 0, "clf_lasso": 6, "clf_ridg": 6, "cli": 15, "clip": [3, 22], "clone": [15, 23], "close": [0, 1, 2, 4, 6, 8, 9, 11, 12, 13, 14, 18, 22, 24, 25, 27], "closer": [3, 5, 13, 26, 27], "closest": [8, 11, 13, 14], "closur": [19, 25], "cloud": [19, 25], "cluster": [0, 1, 4, 6, 11, 19, 25], "cluster_label": 14, "cm": [1, 2, 3, 6, 8, 13, 27], "cmap": [0, 1, 2, 3, 4, 6, 8, 9, 10, 25], "cmap_arg": 6, "cmd": [9, 15], "cn_": 22, "cnn": 12, "cnn_kera": 3, "cntk": [19, 25], "co": [0, 2, 3, 6, 9, 13, 25], "code": [0, 3, 4, 6, 7, 8, 18, 19, 20, 22, 24], "coef": [0, 25], "coef0": 8, "coef_": [0, 5, 6, 8, 9, 13, 16, 25, 26, 27], "coeff": 5, "coeffici": [0, 3, 5, 6, 7, 8, 9, 13, 18, 20, 25, 26], "coerc": [0, 6, 25], "coin": [10, 22], "coin_toss": 10, "col": [0, 11, 25, 26], "colab": [19, 25], "cold": 9, "colinear": [], "collaps": 8, "collect": [2, 6, 10, 11, 17, 19, 22, 24, 25], "collinear": [5, 26, 27], "color": [0, 3, 4, 6, 8, 9, 10, 22], "color_channel": 3, "color_cod": 6, "colorbar": [1, 6], "colsample_bytre": 10, "colsaobject": 10, "column": [0, 1, 2, 5, 6, 7, 8, 9, 11, 12, 16, 17, 18, 20, 25, 26, 27], "columntransform": 9, "com": [4, 6, 15, 16, 19, 24, 25, 27], "combin": [1, 2, 5, 6, 7, 10, 15, 18, 22], "come": [0, 1, 3, 4, 5, 12, 13, 14, 15, 25, 26, 27], "command": [0, 1, 15], "comment": [0, 4, 5, 6], "commerci": [0, 19, 25], "commit": 15, "commod": [0, 25], "common": [0, 1, 3, 5, 6, 7, 9, 11, 13, 14, 16, 22, 25, 26, 27], "commonli": [0, 1, 4, 6, 7, 9, 13, 14, 26], "commun": [0, 12, 15], "commut": 3, "commutatitav": 3, "compact": [0, 1, 3, 5, 6, 7, 9, 11, 12, 13, 14, 25, 26], "compair": 0, "compar": [0, 3, 4, 5, 6, 11, 13, 18, 20, 25, 26, 27], "comparison": [2, 4, 13], "compat": 7, "compet": 0, "competit": 10, "compil": [0, 1, 3, 4, 13, 19, 20, 25], "complet": [0, 2, 3, 4, 9, 12, 15, 16, 17, 18, 25], "completenn": 12, "complex": [1, 5, 8, 9, 11, 12, 13, 16, 25, 27], "complic": [0, 1, 9, 13, 25, 27], "compoment": 26, "compon": [0, 1, 3, 4, 5, 6, 7, 9, 14, 16, 19, 25, 26, 27], "components_": 11, "compos": [9, 12, 13, 14, 19, 25], "compphys": [0, 6, 16, 19, 21, 23, 24, 25, 26], "compress": [0, 25, 26], "compris": 6, "compromis": [5, 26, 27], "compulsori": [19, 25], "comput": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 15, 16, 17, 18, 19, 20, 21, 22, 24, 25, 26, 27], "computation": [0, 3, 6, 9, 13, 22, 25, 27], "computationalscienceuio": 25, "concaten": [2, 4, 6, 14], "concav": [1, 13, 26, 27], "concentr": 10, "concept": [0, 2, 19, 25, 26], "conceptu": [12, 13, 27], "concern": [0, 1, 4, 7, 25, 27], "concic": 25, "conclud": [0, 5, 13], "conclus": 1, "cond": 2, "conda": [0, 1, 19, 25], "condis": 26, "condit": [0, 2, 4, 5, 6, 8, 9, 11, 13, 22, 25, 26], "conduct": 19, "condwav": 2, "confid": [0, 5, 6, 7, 8, 25, 26], "configur": 3, "confirm": [5, 12], "confus": [5, 6, 7, 10, 20, 26], "confusion_matrix": 9, "congruenti": 22, "conjug": [4, 8], "conjugaci": 13, "conjunct": 3, "connect": [0, 1, 3, 4, 9, 11, 12, 13, 20, 25, 26, 27], "consequ": [5, 6, 8, 10, 12, 13, 26, 27], "conserv": [5, 14, 26, 27], "consid": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 12, 13, 16, 20, 22, 25, 26, 27], "consider": [0, 1, 5, 13, 25, 26, 27], "consist": [1, 2, 3, 4, 6, 12, 13, 22, 26, 27], "constant": [0, 2, 4, 5, 6, 8, 12, 13, 16, 18, 22, 25, 26, 27], "constitu": [0, 25], "constitut": [2, 6], "constrain": [1, 3, 5, 7, 11, 27], "constraint": [5, 6, 8, 13, 26, 27], "construct": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 20, 22, 25, 26], "contact": [0, 25], "contain": [0, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 15, 18, 20, 22, 24, 25, 26, 27], "contemporari": 25, "content": [1, 15, 19, 20, 25, 27], "context": [6, 10, 13, 27], "contigu": 20, "continu": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 20, 22, 25, 26, 27], "contour": [9, 10, 13], "contourf": [8, 9, 10], "contrast": [1, 4, 9, 10, 12, 25], "contribut": [0, 3, 5, 13, 18, 22, 25, 26, 27], "contributor": 0, "control": [0, 1, 3, 9, 13, 15, 19, 25], "conv": [3, 4], "conv2d": [3, 4], "conv2dtranspos": 4, "convei": 25, "conveni": [5, 6, 12, 13, 20, 25, 27], "convent": [12, 26], "converg": [1, 2, 4, 5, 8, 13, 14, 18, 26, 27], "convergencewarn": [], "convert": [0, 1, 4, 5, 9, 11, 13, 20, 25, 26, 27], "converttomatrix": 4, "convex": [4, 5, 7, 26], "convinc": [13, 27], "convolut": [1, 4, 19, 25], "cool": [4, 9], "coolwarm": 6, "coordin": [5, 12, 14, 26, 27], "coorel": [], "copi": [0, 1, 14, 15, 26], "core": 10, "corel": 25, "coronari": 7, "corr": [5, 7, 11, 26], "correalt": [11, 19], "correct": [0, 1, 2, 3, 4, 5, 7, 13, 15, 20, 22, 25, 26, 27], "correctli": [1, 2, 6, 7, 10, 18], "correl": [0, 1, 3, 5, 6, 7, 10, 12, 13, 19, 22, 25, 27], "correlation_matrix": [5, 7, 11, 26], "correspond": [0, 3, 5, 6, 8, 9, 11, 12, 19, 20, 22, 25, 26, 27], "cortex": 12, "cosin": [3, 6], "cost": [0, 2, 3, 5, 6, 7, 8, 9, 12, 13, 16, 17, 18, 25], "cost_deep_grad": 2, "cost_funct": 2, "cost_function_deep": 2, "cost_function_deep_grad": 2, "cost_function_grad": 2, "cost_grad": 2, "cost_histori": 18, "cost_ol": 18, "cost_ridg": 18, "cost_sum": 2, "costol": 13, "could": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16, 17, 18, 20, 22, 25, 26, 27], "coulomb": [0, 25], "count": [0, 9, 15, 21, 22, 23, 25], "counterpart": 25, "countor": 13, "coupl": [4, 5, 6], "cours": [0, 1, 3, 5, 11, 15, 16, 17, 23, 26], "coursework": 15, "courvil": [24, 25, 26], "cov": [5, 6, 11, 20, 22, 25, 26], "cov_xi": [5, 11, 26], "cov_xx": [5, 11, 26], "cov_yi": [5, 11, 26], "covari": [0, 7, 19, 20, 25, 27], "covariance_matrix": [5, 11, 14], "cover": [0, 5, 19, 23, 24, 26, 27], "covert": [0, 25], "covxi": 22, "covxx": 22, "covxz": 22, "covyi": 22, "covyz": 22, "covzz": 22, "cpu": 1, "craft": 3, "creat": [1, 3, 4, 5, 9, 10, 11, 12, 15, 18, 19, 25], "create_biases_and_weight": 1, "create_convolutional_neural_network_kera": 3, "create_neural_network_kera": 1, "create_x": [5, 11], "credit": [0, 7, 23, 25], "crim": [], "crime": [], "criteria": [0, 4, 9, 10, 14, 22, 25], "criterion": [9, 10, 13, 27], "critic": [6, 26], "cross": [0, 1, 3, 7, 9, 10, 13, 15, 19, 22, 25, 26, 27], "cross_entropi": 4, "cross_val_scor": 6, "cross_valid": [7, 10], "crossvalid": 6, "crucial": [1, 22], "cs231": 3, "csr_matrix": [20, 25], "csv": [0, 4, 6, 7, 9], "ctnk": 1, "cubic": 0, "cumbersom": 5, "cumsum": [10, 11, 25], "cumul": [7, 10, 22], "cumulative_heads_ratio": 10, "cup": 5, "current": [1, 2, 3, 4, 13, 14, 15, 16, 24, 27], "curs": [0, 26], "curv": [6, 7, 10, 12], "curvatur": [13, 27], "custom": [6, 14], "custom_cmap": [9, 10], "custom_cmap2": [9, 10], "cutpoint": 9, "cv": [6, 7, 10], "cvxbook": [13, 27], "cvxopt": [5, 8, 26], "cycl": [1, 12], "d": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 16, 17, 20, 22, 23, 25, 26, 27], "d2_g_t": 2, "d_f": [13, 27], "d_g_t": 2, "d_net_out": 2, "da": 3, "dagger": [5, 20, 26, 27], "dai": [1, 9, 19], "damp": 3, "darget": 9, "darkr": 22, "dat": [0, 25], "dat_id": [0, 6, 7, 9, 25], "data": [2, 4, 5, 8, 10, 12, 13, 14, 16, 20, 24, 27], "data1": 14, "data2": 14, "data3": 14, "data4": 14, "data_id": [0, 6, 7, 9, 25], "data_indic": 1, "data_panda": 25, "data_path": [0, 6, 7, 9, 25], "databas": 1, "datafil": [0, 6, 7, 9, 25], "datafram": [0, 4, 5, 7, 9, 11, 25, 26], "datapoint": [1, 5, 6, 7, 11, 13, 16, 27], "datasci": [15, 16], "dataset": [0, 4, 6, 7, 8, 9, 10, 11, 13, 14, 16, 25, 27], "datatyp": 4, "date": [15, 18, 25, 26, 27], "daughter": 10, "david": 24, "dbh": 1, "dbo": 1, "dcomposit": 20, "ddot": 2, "dead": 1, "deadlin": 15, "deal": [0, 1, 3, 5, 6, 8, 11, 13, 14, 20, 22, 25, 26, 27], "dealt": 0, "debt": 7, "debug": [0, 5, 6, 26, 27], "decad": [0, 3], "decai": [0, 13, 22, 25], "decemb": [23, 25], "decent": 10, "decid": [0, 2, 3, 5, 6, 9, 18, 26, 27], "decim": [0, 25], "decis": [0, 1, 8, 11, 19, 24, 25], "decision_funct": 8, "decision_tre": 9, "decisiontreeclassifi": [9, 10], "decisiontreeregressor": [0, 9, 10], "declar": [0, 4, 20, 25], "decompos": [5, 6, 20, 26, 27], "decomposit": [0, 6, 12, 25], "decompost": [5, 26, 27], "deconvolut": 3, "decorrel": [10, 13], "decreas": [1, 2, 4, 5, 6, 10, 11, 13, 27], "deduc": [0, 25], "deep": [3, 7, 12, 13, 19, 24, 26, 27], "deep_neural_network": 2, "deep_param": 2, "deep_tree_clf": [9, 10], "deep_tree_clf1": 9, "deep_tree_clf2": 9, "deepen": [5, 19, 25], "deeper": [0, 3, 4, 25], "deeplearningbook": [24, 25, 27], "deer": 3, "def": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 16, 17, 22, 25, 26, 27], "def_covari": 22, "default": [0, 1, 2, 4, 6, 7, 20, 25, 26], "default_tim": 4, "defect": [5, 26, 27], "defici": [5, 26, 27], "defin": [0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 18, 20, 22, 26, 27], "definit": [1, 2, 5, 6, 7, 8, 10, 11, 12, 13, 20, 22, 26, 27], "defint": 22, "degre": [3, 5, 6, 8, 9, 10, 11, 15, 16, 22, 25, 27], "deisenroth": 26, "del": 1, "delet": [6, 15], "delimit": 4, "deliv": [15, 21, 25], "delta": [0, 2, 3, 6, 8, 12, 13, 14, 25], "delta_": [1, 20], "delta_0": 3, "delta_1": 3, "delta_2": 3, "delta_3": 3, "delta_4": 3, "delta_5": 3, "delta_h": [0, 1, 25], "delta_j": [3, 12], "delta_k": 12, "delta_l": [1, 3], "delta_momentum": 13, "delta_n": [0, 3, 25], "delug": 19, "delv": 0, "demand": [13, 27], "demonstr": [0, 3, 5, 6, 7, 11, 12, 19, 25, 26, 27], "den": 4, "denomin": [1, 5], "denot": [1, 2, 6, 7, 13, 22, 27], "dens": [1, 3, 4], "densiti": [0, 2, 6, 22], "depart": [23, 25, 26, 27], "depend": [0, 1, 2, 4, 5, 6, 7, 8, 11, 12, 13, 15, 16, 19, 20, 22, 25, 26, 27], "depict": 22, "deploy": [0, 19, 25], "depth": [0, 3, 9, 10, 20], "deriv": [0, 1, 2, 6, 7, 8, 10, 11, 13, 18, 19, 25], "derivati": 13, "derivative_fn": 13, "descend": [5, 9, 11, 26, 27], "descent": [0, 1, 3, 7, 8, 12, 25, 26], "describ": [0, 2, 4, 5, 6, 8, 10, 11, 12, 13, 20, 25], "descript": [0, 8, 9, 25], "design": [0, 1, 3, 4, 5, 6, 7, 10, 11, 12, 13, 17, 25, 27], "designmatrix": [0, 25], "desir": [0, 2, 4, 5, 13, 14, 25, 26, 27], "desktop": 15, "despit": [1, 12], "destroi": 20, "det": [5, 20, 26, 27], "detail": [0, 6, 11, 13, 14, 18, 20, 26, 27], "detect": [3, 8, 12], "determin": [0, 2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 13, 18, 20, 22, 25, 26, 27], "determinist": [7, 13, 22, 27], "dev": 1, "develop": [0, 3, 5, 8, 10, 11, 12, 19, 20, 25, 26], "deviat": [0, 1, 2, 4, 5, 6, 17, 18, 22, 25, 26], "devis": 12, "df": [4, 8, 11, 13, 25], "df1": 25, "di": [], "diag": [5, 8, 26, 27], "diagnost": [1, 10], "diagon": [0, 5, 7, 13, 18, 20, 22, 25, 26, 27], "diagonaliz": [5, 26, 27], "diagram": 10, "diagsvd": 6, "dice": [6, 22], "dict": [6, 8], "dictionari": [], "did": [0, 1, 5, 6, 7, 10, 11, 14, 16, 25], "die": 1, "diff": 2, "diff1": 2, "diff2": 2, "diff_ag": 2, "diffeent": 8, "differ": [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 24, 25, 26, 27], "differenti": [0, 3, 16, 19, 20, 25, 26, 27], "difficult": [0, 1, 6, 10, 13, 22, 25], "difficulti": [0, 1, 13, 25, 27], "diffonedim": 2, "digit": [0, 1, 3, 4, 6, 23, 25], "dilemma": 13, "dilut": 1, "dim": [4, 11, 14, 20], "dimens": [0, 1, 2, 3, 4, 5, 8, 11, 14, 16, 20, 25, 26, 27], "dimension": [0, 4, 5, 6, 9, 11, 13, 14, 19, 20, 25, 26, 27], "dimensionless": [0, 3, 25], "diment": 20, "dimnsion": 4, "diod": 3, "direct": [0, 1, 2, 4, 11, 12, 13, 14, 25, 26, 27], "directli": [1, 4, 5, 6, 18, 22, 26, 27], "disadvantag": [0, 25], "disappear": [3, 6], "disc_loss": 4, "disc_tap": 4, "discard": [6, 11], "disciplin": [0, 3, 12], "disclaim": 22, "discord": 25, "discourag": [13, 15, 27], "discov": [0, 25], "discover": 5, "discret": [1, 3, 5, 7, 13], "discrimin": [4, 7, 10, 11], "discriminator_loss": 4, "discriminator_loss_list": 4, "discriminator_model": 4, "discriminator_optim": 4, "discuss": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 18, 19, 20, 22, 24, 25, 26, 27], "diseas": 7, "disguis": [6, 26], "disord": [1, 7], "displai": [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 22, 25, 26], "displaystyl": [0, 5, 17, 25, 26, 27], "disregard": [0, 25], "dissimilar": [11, 14], "dist": 14, "distanc": [8, 9, 11, 14, 22], "distance_list": 9, "distinct": [3, 7, 8, 9, 10, 14], "distinctli": 8, "distinguish": [0, 4, 7, 8, 22, 25], "distplot": [], "distribut": [0, 1, 4, 6, 7, 10, 11, 13, 14, 18, 19, 20, 25, 26, 27], "distrubut": [0, 19, 25], "dive": [0, 8, 20, 25], "diverg": [1, 13, 27], "divid": [0, 1, 3, 5, 6, 7, 8, 9, 11, 12, 18, 22, 25, 26], "divis": [6, 8, 9, 13, 18, 20, 22], "dna": 7, "dnn": [0, 1, 2, 4, 12, 25], "dnn1": 4, "dnn2_gru2": 4, "dnn_kera": 1, "dnn_model": 1, "dnn_numpi": 1, "dnn_scikit": [0, 1, 25], "do": [0, 2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 13, 14, 15, 16, 20, 25, 26, 27], "doc": [0, 15, 16, 19, 21, 23, 24, 25], "document": [4, 13, 15], "doe": [0, 1, 2, 3, 4, 5, 6, 8, 10, 11, 12, 13, 15, 16, 17, 20, 22, 25], "doesn": [3, 9, 12, 25], "dog": [1, 3, 4], "domain": [5, 8, 13, 27], "domin": [0, 25], "don": [0, 1, 3, 5, 6, 8, 11, 13, 15, 16, 19, 25, 26], "done": [0, 2, 3, 4, 5, 6, 9, 10, 11, 13, 16, 20, 25, 26, 27], "dot": [0, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 18, 20, 22, 25, 26, 27], "doubl": [3, 4, 16, 20, 25], "doubli": 1, "down": [0, 3, 6, 9, 11, 12, 13, 27], "download": [0, 1, 3, 5, 6, 15, 20, 24, 25], "downsampl": 3, "dozen": 1, "dq": 6, "drag": 13, "dramat": 11, "drastic": 4, "draw": [4, 6, 10, 13, 27], "drawback": [0, 1, 3, 13, 26, 27], "drawn": [1, 4, 6, 7, 11, 22, 25], "drive": [3, 4], "driven": 3, "drop": [0, 1, 5, 6, 11, 13, 22, 25, 26, 27], "dropna": [0, 6, 25], "dropout": 4, "dt": [2, 3, 13, 22], "dtype": [0, 1, 3, 4, 14, 20, 25], "dub": [0, 25], "due": [1, 2, 5, 6, 8, 10, 12, 13, 18, 23, 25, 26, 27], "dummi": [], "dure": [0, 1, 3, 4, 8, 9, 11, 19, 25], "dwell": [], "dwh": 1, "dwo": 1, "dx": [2, 3, 8, 22], "dx_1": 22, "dx_1p": 6, "dx_2p": 6, "dx_mp": 6, "dx_n": 22, "dxp": 6, "dy": [1, 8, 22], "dynam": 4, "dz": 8, "e": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 22, 23, 25, 26, 27], "e_": [0, 2, 25], "each": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 19, 20, 21, 22, 23, 25, 26, 27], "eapprox": [0, 25], "earli": [1, 13], "earlier": [0, 5, 7, 8, 9, 11, 12, 13, 25, 26], "earthexplor": 6, "eas": [6, 9, 14], "easi": [0, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 19, 20, 25, 26, 27], "easier": [5, 6, 8, 9, 13, 15, 22, 25, 26, 27], "easiest": 13, "easili": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 20, 25, 26, 27], "eastern": [23, 25], "ebind": [0, 25], "eblock": 9, "econometr": 25, "economi": 5, "ecosystem": [19, 25], "ect": 21, "edg": 3, "edgecolor": 6, "editor": 15, "edu": [13, 27], "educ": [0, 25], "eff": 22, "effect": [1, 4, 10, 13, 16, 17, 18, 22], "effic": 1, "effici": [0, 3, 10, 13, 19, 20, 22, 25], "efron": 6, "egrad": 13, "eig": [5, 11, 13, 20, 22, 25, 26, 27], "eigen": 22, "eigenpair": [5, 11, 26, 27], "eigenvalu": [0, 5, 8, 11, 13, 20, 25, 26, 27], "eigenvector": [5, 11, 13, 26, 27], "eight": [20, 25], "eigval": [20, 22, 25], "eigvalu": [11, 13, 27], "eigvec": [20, 22, 25], "eigvector": [11, 13, 27], "eir": [23, 25], "eispack": [20, 25], "either": [1, 5, 6, 7, 8, 9, 10, 11, 13, 18, 22, 25, 26, 27], "eivind": 23, "eivinsto": 23, "ekstr\u00f8m": 4, "elabor": 22, "elarn": 3, "electr": [0, 3, 12, 25], "electron": 25, "eleg": 11, "element": [1, 2, 3, 4, 5, 6, 7, 8, 11, 12, 13, 19, 20, 24, 26], "elementari": [10, 13, 20], "elementwis": [3, 13], "elementwise_grad": [2, 13], "elessar": 25, "elif": 14, "elim": 20, "elimin": [3, 8], "elin": [23, 25], "ellipsi": 16, "els": [1, 3, 4, 7, 9, 12, 13, 16, 20], "elu": 1, "elus": [0, 25], "email": [21, 23, 25], "embed": [0, 11, 26], "embodi": 6, "emit": 22, "emner": 24, "emphas": [0, 10, 19, 25], "emphasi": [0, 19, 24, 25], "empir": [1, 11, 22], "emploi": [0, 1, 5, 6, 11, 13, 22, 25, 26, 27], "employ": 0, "empti": [6, 10, 15], "emul": 12, "en": [19, 24], "enabl": 11, "enbodi": 6, "encod": [0, 3, 5, 9, 11, 14, 25, 26, 27], "encompass": [0, 22], "encount": [0, 1, 5, 7, 13, 15, 22, 25, 26, 27], "encourag": 15, "end": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 20, 22, 23, 25, 26, 27], "endpoint": [3, 6], "energi": [0, 4, 6], "enforc": 12, "eng": 24, "engin": [0, 1, 3, 4, 19, 25], "enorm": 3, "enough": [0, 6, 13, 25, 27], "ensembl": [1, 9, 25], "ensur": [0, 1, 2, 3, 5, 6, 11, 13, 18, 22, 26, 27], "entail": 25, "enter": [5, 6, 26, 27], "enthought": [0, 19, 25], "entir": [1, 3, 7, 9, 19, 22, 25], "entiti": [9, 12, 20, 25], "entri": [0, 5, 8, 11, 12, 20, 25, 26], "entropi": [1, 3, 7, 10, 13, 25, 27], "enumer": [0, 1, 2, 3, 4, 6, 8, 25, 26], "env": 22, "environ": [2, 19, 25], "environemnt": 15, "eo": [0, 6], "eol": 0, "eosfit": 0, "epoch": [0, 1, 3, 4, 12, 13, 25], "epsilon": [0, 5, 6, 7, 13, 25, 26, 27], "epsilon_": [0, 25], "epsilon_0": [0, 25], "epsilon_1": [0, 25], "epsilon_2": [0, 25], "epsilon_i": [0, 25, 26], "eq": [3, 13, 14, 20, 22, 27], "eqnarrai": [3, 5, 6], "equal": [0, 1, 2, 3, 4, 5, 6, 8, 9, 11, 12, 13, 14, 16, 18, 20, 22, 25, 26, 27], "equat": [1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 17, 20, 22, 25], "equilibrium": [2, 12], "equiv": [3, 13, 20, 22, 27], "equival": [0, 1, 5, 7, 8, 11, 13, 19, 20, 25, 26, 27], "erf": 22, "eriador": 25, "err": [0, 10], "err_": 6, "err_sqr": 2, "errat": [13, 27], "erron": 2, "error": [1, 2, 4, 5, 6, 7, 9, 11, 12, 13, 15, 16, 17, 18, 19, 20, 22], "error_estimate_corr_tim": 22, "error_hidden": 1, "error_output": 1, "escap": [13, 27], "especi": [1, 3, 9, 12, 13, 15, 18], "essenti": [0, 5, 6, 9, 10, 12, 14, 15, 22, 26, 27], "establish": [0, 6, 10, 11, 16], "estim": [0, 1, 5, 6, 7, 10, 11, 13, 19, 22, 25, 26, 27], "estimated_mse_fold": 6, "estimated_mse_kfold": 6, "estimated_mse_sklearn": 6, "et": [0, 2, 4, 16, 17, 24, 25, 26, 27], "eta": [0, 1, 3, 8, 12, 13, 18, 25, 27], "eta0": [8, 13], "eta_": 13, "eta_t": 13, "eta_v": [0, 1, 3, 25], "etc": [0, 1, 3, 5, 7, 8, 9, 11, 12, 13, 14, 19, 20, 22, 26, 27], "ethic": 19, "euclidean": [0, 14, 26], "evalu": [0, 2, 3, 4, 5, 6, 9, 13, 15, 16, 17, 22, 25, 26, 27], "evalut": 13, "even": [0, 1, 3, 4, 5, 6, 8, 9, 10, 11, 12, 13, 14, 19, 20, 22, 25, 26, 27], "evenli": 4, "event": [5, 7, 10, 22], "eventu": [0, 5, 6, 11, 12, 13, 23, 26, 27], "everi": [0, 1, 2, 3, 4, 5, 6, 9, 10, 11, 12, 13, 14, 15, 19, 22, 23, 25, 26, 27], "everyth": [4, 12, 16, 18], "everywher": [4, 13, 27], "evolv": 0, "exact": [0, 5, 11, 12, 13, 20, 22, 25, 26], "exactli": [0, 3, 4, 6, 12, 18, 19, 26], "exam": 25, "examin": 6, "exampl": [0, 5, 11, 12, 13, 15, 16, 18, 19, 20, 22, 24], "exce": [1, 12, 13], "excel": [0, 1, 4, 5, 10, 25, 26], "except": [3, 4, 6, 8, 9, 20], "excess": [0, 25], "excit": 0, "exclud": [1, 6, 12, 26], "exclus": [0, 1, 3, 6, 22, 25], "execut": [2, 5, 13, 15, 26, 27], "exemplifi": 13, "exercic": [23, 25], "exercis": [5, 19, 21, 23, 25, 27], "exhaust": 6, "exhibit": [0, 5, 6, 8, 25, 26], "exist": [0, 1, 2, 3, 5, 6, 7, 8, 9, 13, 20, 25, 27], "exit": [5, 20, 26, 27], "exp": [0, 1, 2, 5, 6, 7, 8, 10, 11, 12, 13, 16, 17, 22, 26, 27], "exp_term": 1, "expand": [5, 7, 11, 13, 27], "expans": [0, 3, 5, 8, 10, 12, 13, 25, 26, 27], "expect": [0, 1, 5, 6, 7, 11, 12, 13, 15, 18, 19, 25, 26], "expectation_value_of_h_wrt_p": 22, "expens": [6, 10, 13, 16, 27], "experi": [0, 1, 6, 8, 13, 15, 19, 25, 26, 27], "experiment": [0, 4, 6, 9, 22, 25], "expert": [1, 9], "explain": [0, 6, 9, 10, 11, 13, 16, 25, 27], "explained_variance_ratio_": 11, "explanatori": [0, 25], "explicit": [0, 3, 6, 13, 20, 25, 26, 27], "explicitli": [0, 4], "explod": 1, "exploit": [0, 3, 12, 13, 25], "explor": [1, 4, 6, 8, 13, 18, 19, 25, 27], "expon": 1, "exponenti": [0, 1, 5, 6, 10, 13, 22, 25, 27], "export": [9, 15, 16], "export_graphviz": 9, "export_text": 9, "exporttext": 9, "expos": 19, "express": [0, 2, 3, 5, 6, 7, 10, 12, 13, 18, 20, 22, 25, 27], "exptmean": 22, "exptvari": 22, "extend": [0, 2, 7, 11, 13, 19, 25], "extens": [0, 12, 15, 19, 25], "extent": [0, 1, 6, 24], "extern": [3, 6, 9], "extra": [1, 3, 5, 15, 23, 25, 26, 27], "extract": [0, 3, 5, 6, 7, 8, 11, 13, 16, 17, 20, 25, 26], "extrapol": [0, 25], "extrem": [0, 1, 4, 5, 6, 7, 8, 9, 13, 15, 16, 20, 26, 27], "extremum": [13, 27], "extrins": 11, "ey": [0, 5, 6, 13, 14, 18, 20, 25, 26, 27], "f": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 12, 13, 14, 15, 16, 17, 20, 22, 23, 25, 26, 27], "f1": 13, "f11": [0, 25], "f12": [0, 25], "f13": [0, 25], "f1_grad": 13, "f1d": 13, "f2": 13, "f2_grad_x1": 13, "f2_grad_x1_analyt": 13, "f2_grad_x2": 13, "f2_grad_x2_analyt": 13, "f3": 13, "f3_grad": 13, "f3_grad_analyt": 13, "f4": 13, "f4_grad": 13, "f4_grad_analyt": 13, "f5": 13, "f5_grad": 13, "f6": 13, "f6_for": 13, "f6_for_grad": 13, "f6_grad_analyt": 13, "f6_while": 13, "f6_while_grad": 13, "f7": 13, "f7_grad": 13, "f7_grad_analyt": 13, "f8": 13, "f8_grad": 13, "f9": [0, 13, 25], "f9_altern": 13, "f9_alternative_grad": 13, "f9_grad": 13, "f_": 10, "f_0": [3, 10], "f_1": [10, 13, 27], "f_2": [12, 13, 27], "f_3": 12, "f_d": 22, "f_grad": 13, "f_grad_analyt": 13, "f_i": [0, 6, 12, 16], "f_m": [3, 10], "f_n": 3, "f_vec": 2, "face": [13, 25, 27], "facecolor": [6, 8, 22], "facil": [0, 19], "facilit": 12, "fact": [0, 1, 3, 5, 9, 11, 12, 13, 25, 26, 27], "factor": [0, 1, 3, 5, 6, 9, 10, 11, 13, 20, 22, 25, 26, 27], "factori": 13, "fade": 6, "fafab0": [9, 10], "fail": [0, 6, 13, 23, 25, 27], "failur": 7, "fairli": [1, 2, 18, 22], "faisal": [16, 26], "fake": 4, "fake_loss": 4, "fake_output": 4, "fall": [8, 9, 21], "fals": [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 14, 16, 17, 20, 25, 26, 27], "famili": [0, 7, 8, 22, 26], "familiar": [0, 3, 5, 6, 8, 15, 19, 20, 22, 25], "famou": [6, 12], "far": [0, 3, 4, 5, 6, 8, 11, 12, 13, 14, 16, 25, 26, 27], "fashion": [0, 9, 10, 25], "fast": [1, 3, 6, 10, 12, 13, 19, 22, 25, 27], "faster": [1, 11, 13], "fastest": [13, 20, 27], "favor": 7, "favorit": 22, "fc": 3, "featur": [0, 1, 3, 5, 6, 7, 8, 10, 11, 12, 13, 15, 17, 18, 19, 22, 25, 27], "feature_nam": [1, 7, 9], "feautur": 9, "fed": 1, "feed": [0, 2, 3, 11, 19, 25], "feed_forward": 1, "feed_forward_out": 1, "feed_forward_train": 1, "feedback": [4, 25], "feeddorward": 4, "feedforward": [1, 4, 12], "feel": [0, 5, 6, 11, 13, 15, 16, 19, 23, 25], "feet": [], "fetch": [6, 15], "few": [1, 3, 4, 5, 9, 17, 18, 22, 25], "fewer": [0, 9, 11, 25], "ffnn": [1, 12], "field": [0, 3, 6, 12, 19], "fifth": [0, 6, 25], "fig": [0, 1, 2, 3, 4, 6, 7, 12, 13, 14, 25], "fig_id": [0, 6, 7, 9, 25], "figaxi": 22, "figsiz": [0, 1, 2, 3, 4, 6, 7, 8, 9, 10, 25], "figur": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 14, 16, 19, 25, 26, 27], "figure_id": [0, 6, 7, 9, 25], "figurefil": [0, 6, 7, 9, 25], "file": [0, 4, 5, 6, 7, 9, 15, 25], "file_prefix": 4, "filenam": 25, "fill": [5, 9, 18, 26, 27], "filter": [3, 4], "final": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 18, 21, 22, 23, 25, 27], "financ": 0, "find": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 19, 22, 25, 26, 27], "fine": [0, 14], "finish": 2, "finit": [3, 5, 6, 12, 13, 17, 22, 26, 27], "finnicki": 15, "first": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 16, 18, 20, 22, 23, 24, 26], "firsteigvector": 11, "fit": [1, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 17, 18, 22, 26], "fit_beta": 26, "fit_intercept": [0, 5, 6, 16, 26, 27], "fit_mod": 9, "fit_theta": 6, "fit_transform": [0, 6, 8, 9, 11, 15], "fiti": [0, 25], "five": [0, 9, 25, 26], "fix": [0, 3, 4, 6, 10, 11, 12, 13, 25], "flag": 4, "flat": [12, 13, 27], "flatten": [1, 3, 4, 5, 20], "flexibl": [1, 6, 8, 10, 12, 25], "flip": [23, 25], "float": [0, 3, 4, 5, 9, 11, 13, 14, 20, 25, 26, 27], "float32": [4, 9], "float64": [4, 20, 25], "flop": [5, 20, 26, 27], "flow": [1, 4, 12], "fluctuat": 5, "fly": 11, "fm": 0, "fmax": 3, "fmesh": 13, "fn": 7, "focu": [0, 3, 4, 5, 6, 15, 19, 24, 25, 26, 27], "focus": [1, 6, 7, 20, 26], "fold": [6, 9], "folder": [0, 4, 6, 15, 25], "follow": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 19, 20, 22, 23, 24, 25, 26, 27], "font": [7, 22, 25], "fontdict": 22, "fontsiz": [1, 6, 8, 9, 10, 22], "fontweight": 1, "footprint": 3, "foral": [8, 26], "forc": [0, 5, 6, 10, 11, 26, 27], "forcast": 4, "forecast": [4, 12], "forest": [0, 1, 9, 19, 25], "forget": 11, "form": [0, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 15, 16, 19, 20, 22, 25, 26, 27], "formal": [3, 4, 14, 22], "format": [0, 1, 3, 4, 6, 7, 8, 9, 10, 11, 19, 22, 24], "format_data": 4, "formatstrformatt": [6, 13, 27], "formul": [4, 6, 11, 14], "formula": [3, 13, 22, 27], "forth": [4, 12], "fortran": [0, 19, 20, 25], "fortran2003": [19, 25], "fortran90": 22, "fortun": [0, 11, 26], "forward": [0, 3, 6, 19, 20, 25], "found": [1, 2, 4, 5, 6, 12, 13, 25, 26], "foundat": [19, 25], "four": [4, 5, 6, 8, 12, 20, 21, 23, 25, 27], "fourier": [0, 25], "fourierdef1": 3, "fourierdef2": 3, "fourierseriessign": 3, "fourth": [12, 25, 26], "fp": 7, "frac": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 20, 22, 25, 26, 27], "fraction": 9, "frame": 7, "framework": [1, 8, 10, 22], "frank": [5, 11], "frankefunct": [5, 6, 11], "fredli": [23, 25], "free": [0, 6, 11, 13, 15, 16, 19, 20, 22, 23, 24, 25], "freecodecamp": 19, "freedom": [5, 27], "freeli": 0, "freez": 15, "frequenc": [3, 6, 7, 22], "frequent": [0, 8, 9, 13, 27], "frequentist": 19, "fresh": 10, "fridai": [15, 23, 25], "friedman": [6, 24, 25], "friendli": 4, "frodo": 25, "frog": 3, "from": [0, 1, 2, 3, 4, 6, 7, 8, 9, 11, 13, 14, 15, 16, 17, 18, 19, 20, 22, 23, 24], "from_cod": 9, "from_logit": [3, 4], "from_tensor_slic": 4, "front": [0, 4, 5, 25, 26, 27], "frustrat": 15, "fulfil": [2, 5, 12, 26, 27], "full": [1, 3, 5, 7, 9, 10, 13, 22, 25, 26, 27], "full_matric": [5, 26, 27], "fulli": [3, 6, 12, 22], "fun": [19, 25], "func": 2, "function": [2, 3, 4, 5, 9, 14, 15, 16, 17, 18, 19, 20], "functionali": 11, "fundament": [0, 6, 19, 25], "funtion": 2, "further": [2, 7, 9, 25], "furthermor": [0, 3, 5, 6, 7, 11, 12, 13, 19, 25, 26, 27], "futur": [0, 4, 8, 9, 25], "fy": [15, 21, 23, 24, 25], "fys5419": [24, 25], "fys5429": [24, 25], "f\u00f8470": [23, 25], "g": [0, 1, 2, 3, 4, 6, 8, 9, 10, 11, 13, 15, 18, 22, 25, 26, 27], "g0": 2, "g_": [2, 9, 10], "g_0": 2, "g_1": [2, 10], "g_2": [2, 10], "g_analyt": 2, "g_dnn_ag": 2, "g_euler": 2, "g_i": 2, "g_m": [3, 10], "g_n": 3, "g_re": 2, "g_t": 2, "g_t_d2t": 2, "g_t_d2x": 2, "g_t_dt": 2, "g_t_hessian": 2, "g_t_hessian_func": 2, "g_t_jacobian": 2, "g_t_jacobian_func": 2, "g_trial": 2, "g_trial_deep": 2, "g_vec": 2, "gain": [1, 5, 7, 9, 10, 13, 26, 27], "galleri": [0, 25], "game": 4, "gamge": 25, "gamma": [0, 2, 8, 9, 10, 11, 13, 25, 27], "gamma1": 8, "gamma2": 8, "gamma_": [0, 25], "gamma_0": 10, "gamma_1": 10, "gamma_1x": 10, "gamma_i": [0, 8, 22, 25], "gamma_j": 13, "gamma_k": [13, 27], "gamma_m": 10, "gamma_x": [0, 25], "gap": 8, "gate": [4, 12], "gather": [0, 1, 12, 26], "gaug": 12, "gaussbacksub": 20, "gaussian": [4, 5, 6, 8, 14, 18, 22, 25], "gaussian_point": 14, "gaussian_rbf": 8, "gave": 13, "gavra": 25, "gbc": 25, "gca": [2, 6, 8, 13], "gd": [1, 27], "gd_clf": 10, "gdclassiffiercgain": 10, "gdclassiffierconfus": 10, "gdclassiffierroc": 10, "gdm": 13, "gdregress": 10, "ge": [1, 5, 7, 22, 26, 27], "gen_loss": 4, "gen_tap": 4, "gender": [0, 25], "genener": 4, "gener": [0, 1, 2, 3, 5, 6, 8, 10, 11, 12, 13, 14, 15, 16, 18, 20, 22, 24, 26, 27], "generaliz": 16, "generallay": 12, "generate_and_save_imag": 4, "generate_imag": 4, "generate_latent_point": 4, "generate_simple_clustering_dataset": 14, "generated_imag": 4, "generator_loss": 4, "generator_loss_list": 4, "generator_model": 4, "generator_optim": 4, "genom": 19, "geodes": 11, "geometr": [0, 13, 25], "geometri": 5, "georg": 24, "geotif": 6, "geq": [2, 5, 8, 9, 13, 26, 27], "geron": [0, 24, 25], "get": [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 13, 15, 19, 20, 22, 23, 25, 26, 27], "get_dummi": 9, "get_paramet": 2, "get_split": 9, "get_yaxi": 8, "get_yticklabel": 6, "gh": 15, "gibb": [19, 25], "gif": 4, "gini": 10, "gini_index": 9, "ginvers": 13, "git": [0, 15, 19, 25], "giter": 13, "github": [0, 19, 21, 23, 24, 25, 26], "gitignor": 15, "gitlab": [0, 15, 19, 25], "give": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 12, 13, 14, 19, 22, 25, 26, 27], "given": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 17, 20, 22, 25, 26, 27], "global": [6, 7, 13, 27], "glorot": 1, "gnew": 13, "go": [0, 1, 3, 5, 6, 8, 9, 11, 12, 13, 15, 16, 25, 26, 27], "goal": [0, 7, 9, 25], "goe": [0, 1, 2, 5, 6, 13, 14, 15, 20, 25, 26, 27], "golden": 13, "gone": [5, 26, 27], "gong": 1, "good": [1, 3, 4, 5, 6, 9, 10, 11, 13, 15, 18, 19, 22, 24, 26, 27], "goodfellow": [4, 24, 25, 26, 27], "googl": [1, 4, 19, 25], "got": [1, 6], "gotten": 25, "gov": 6, "govern": 25, "gp": 24, "gpu": [1, 13, 19, 25], "grad": [2, 13], "grad_analyt": 13, "grad_ol": 18, "grad_ridg": 18, "grade": 21, "gradient": [0, 3, 4, 7, 8, 9, 12, 19, 25, 26], "gradientboostingclassifi": 10, "gradientboostingregressor": 10, "gradients_of_discrimin": 4, "gradients_of_gener": 4, "gradienttap": 4, "gradual": [1, 14], "grai": [4, 6], "graph": [1, 9, 11, 12, 13, 16, 27], "graph_from_dot_data": 9, "graphic": [0, 1, 9, 15, 25], "grasp": 0, "gray_r": [1, 3], "grayscal": 3, "great": [5, 13, 15, 27], "greater": [1, 7, 22, 26], "greatli": 13, "greedi": 9, "green": [0, 3, 9, 22], "grei": 4, "grid": [1, 3, 6, 7, 8, 12, 22, 26], "grossli": [13, 27], "ground": [0, 25], "group": [0, 6, 7, 9, 14, 15, 19, 21, 23, 25], "groupbi": [0, 25], "grow": [1, 3, 9, 10], "growth": [0, 25], "gru": 4, "guarante": [0, 4, 13, 22, 25, 26, 27], "guess": [1, 4, 10, 13, 14, 27], "guestrin": 10, "gui": 15, "guid": 1, "h": [0, 1, 5, 6, 8, 13, 15, 22, 23, 24, 25, 26, 27], "h1": 2, "h_": [0, 13, 25, 27], "h_1": [2, 13, 27], "h_2": [2, 13, 27], "h_m": 10, "ha": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 18, 20, 22, 25, 26, 27], "haanen": [23, 25], "habit": [0, 26], "had": [0, 1, 6, 7, 13, 25, 27], "hadamard": [1, 12, 13], "half": [1, 8, 9], "halv": 10, "hand": [0, 1, 2, 3, 5, 11, 12, 13, 19, 20, 22, 23, 24, 25, 26, 27], "handi": 3, "handl": [0, 1, 2, 5, 9, 11, 15, 18, 19, 26, 27], "handle_unknown": 9, "handsid": 12, "handwrit": 12, "handwritten": [1, 5], "happen": [1, 2, 3, 4, 5, 6, 10, 13, 22, 26, 27], "hard": [1, 7, 8, 10, 13, 27], "hardcopi": [19, 25], "harder": [0, 1, 26], "harmon": 3, "hasn": [], "hassl": [0, 19, 25], "hast": [19, 25], "hasti": [0, 6, 16, 17, 24, 25, 26], "hat": [0, 1, 5, 6, 7, 9, 10, 11, 12, 13, 16, 17, 18, 20, 26, 27], "have": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 23, 25, 26, 27], "haven": 1, "he": 7, "head": [4, 10, 22], "header": [0, 25], "heads_proba": 10, "health": [0, 26], "hear": [0, 13, 25], "heart": [0, 7, 25], "heatmap": [0, 1, 3, 7, 17, 25], "heavili": 0, "heavisid": 1, "height": [1, 3, 6, 26], "held": 13, "help": [0, 1, 4, 12, 13, 15, 16, 25], "helper": [4, 14], "henc": [0, 5, 6, 8, 9, 10, 12, 13, 25, 26, 27], "henrik": [23, 25], "her": 7, "here": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 17, 18, 19, 20, 22, 25, 26, 27], "hereaft": [0, 8, 12, 25], "hermitian": 20, "hessenberg": 20, "hessian": [0, 2, 5, 13], "heterogen": [9, 10], "hi": 7, "hidden": [1, 3, 4, 12], "hidden_bia": 1, "hidden_bias_gradi": 1, "hidden_layer_s": [0, 1, 25], "hidden_neuron": 4, "hidden_weight": 1, "hidden_weights_gradi": 1, "hierarch": [5, 26, 27], "high": [0, 1, 2, 3, 4, 5, 6, 9, 10, 11, 13, 14, 19, 20, 25, 26, 27], "higher": [0, 1, 3, 5, 6, 8, 13, 25, 26, 27], "highest": [1, 2], "highli": [0, 3, 4, 10, 19, 20, 24, 25, 26, 27], "highwai": [], "hing": 8, "hint": [13, 15, 16, 26, 27], "hip": 19, "hire": 0, "hist": [4, 6, 7, 22], "histogram": [6, 7, 22], "histor": [7, 11], "histori": [3, 4, 12, 15, 18], "hitherto": 5, "hjorth": [23, 25, 26, 27], "hobbi": 22, "hoc": [5, 26, 27], "hoff": 24, "hold": [1, 3, 6, 13, 14, 27], "holder": [0, 25], "home": [], "homepag": 25, "homework": [6, 13, 27], "homogen": [1, 3, 9, 10, 13], "honchar": 2, "hopefulli": [0, 11, 15, 22, 25], "horizont": 11, "horlyk": [23, 25], "hors": [3, 7, 25], "hot": [1, 9], "hour": [1, 19, 21, 22, 23, 25], "how": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 19, 20, 22, 25, 26, 27], "howev": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 19, 20, 22, 25, 26, 27], "hspace": [0, 4, 8, 10, 22, 25], "hstack": 1, "htf": 25, "html": [0, 16, 19, 21, 23, 24, 25, 26, 27], "http": [0, 3, 4, 6, 13, 15, 16, 19, 20, 21, 23, 24, 25, 26, 27], "huang": [0, 25], "huber": [0, 25], "huge": [1, 3, 4, 19], "human": [0, 1, 3, 6, 9, 12, 26], "humid": 9, "hundr": 1, "hungri": 1, "hybrid": 21, "hydrogen": [0, 25], "hyperbol": [1, 4, 12], "hyperparam": 8, "hyperparamet": [3, 4, 5, 6, 9, 13, 18, 26, 27], "hyperplan": 11, "h\u00f8rlyk": [23, 25], "i": [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 26, 27], "i0": [0, 25], "i1": [0, 6, 8, 12, 25, 26], "i2": [0, 8, 12, 25], "i3": [0, 12, 25], "i5": [0, 25], "i_": [13, 27], "i_1": [5, 6], "i_2": [5, 6], "ian": 24, "ic": 1, "id": [7, 13, 27], "ida": [23, 25], "idea": [0, 1, 2, 3, 4, 6, 9, 10, 12, 13, 20, 26, 27], "ideal": [0, 2, 6, 8, 13, 22, 25], "idem": 6, "ident": [5, 6, 12, 13, 17, 18, 20, 26, 27], "identifi": [0, 1, 7, 9, 11, 12, 13, 14, 25, 26], "ieor": 22, "ifi": 24, "ifs": [19, 25], "ignor": [0, 1, 3, 9, 15, 26], "ii": [20, 22], "iii": [20, 25], "ij": [0, 1, 3, 6, 8, 12, 14, 16, 20, 22, 25, 26], "ik": [0, 20, 25, 26], "illustr": [5, 7, 10, 12, 13, 14, 19, 25], "im": 6, "imag": [1, 3, 4, 6, 9, 11, 12, 14, 24, 25], "image_at_epoch_": 4, "image_batch": 4, "image_height": 3, "image_path": [0, 6, 7, 9, 25], "image_width": 3, "imageio": 6, "images_from_seed_imag": 4, "imagin": 1, "immedi": [0, 3, 4, 6, 19, 25], "implement": [0, 2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 13, 14, 22, 25, 26, 27], "impli": [3, 5, 6, 7, 13, 20, 26, 27], "implicit": 3, "implicitli": [11, 22], "import": [0, 1, 2, 3, 4, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 22], "importantli": 3, "impos": [0, 6, 11, 12, 25], "imposs": [0, 5, 25, 26, 27], "impress": [0, 12, 25], "improv": [0, 4, 5, 9, 10, 11, 13, 15, 26, 27], "impur": 9, "imread": 6, "imshow": [1, 3, 4, 6], "in3050": [24, 25], "in3310": 25, "in4080": [24, 25], "in4300": [24, 25], "in4310": 24, "in5400": 3, "in5550": 24, "in_out_neuron": 4, "inaccur": [13, 27], "inact": 12, "inadequ": [0, 25], "inch": [6, 26], "includ": [0, 1, 2, 3, 4, 5, 6, 7, 11, 12, 15, 16, 17, 18, 19, 22, 23, 24, 25, 26, 27], "include_bia": [6, 9], "incom": [12, 16], "incorrect": 1, "incoveni": 8, "increas": [0, 1, 3, 4, 5, 6, 9, 12, 13, 22, 25, 26], "increasingli": 22, "ind": 6, "inde": [0, 2, 4, 5, 6, 13, 25, 26, 27], "indefinit": 4, "independ": [0, 5, 6, 7, 8, 12, 13, 22, 25, 26, 27], "index": [0, 1, 3, 4, 10, 14, 19, 20, 22, 24, 25], "index_col": [0, 25], "indic": [0, 1, 3, 4, 5, 6, 9, 10, 11, 13, 16, 25, 26], "indispens": 6, "individu": [1, 6, 7, 10, 12, 22, 25, 26], "indu": [], "indx": 20, "indx1": 2, "indx2": 2, "indx3": 2, "ineffici": [3, 13], "inequ": [8, 13], "inequaltii": 27, "inertia": 13, "inf1000": [19, 25], "inf1100": [19, 25], "inf1100l": [19, 25], "inf1110": [19, 25], "inf3000": 25, "infeas": 9, "infer": [0, 1, 4, 6, 24, 25], "inferenc": 1, "infil": [0, 6, 7, 9, 25], "infin": [5, 6, 7, 11, 26, 27], "infinit": 3, "infinitesim": 22, "influenc": [6, 10, 18], "influenti": 1, "info": 25, "inform": [0, 1, 3, 4, 6, 9, 11, 12, 13, 14, 20, 24, 25, 27], "inforom": 15, "infti": [3, 6, 13, 22, 27], "ingeni": [13, 27], "ingredi": [0, 9, 25], "inher": 6, "inherit": [20, 25], "initi": [0, 1, 2, 6, 10, 13, 14, 18, 20, 22, 25, 27], "inject": 14, "inlin": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 20, 22, 25, 26, 27], "inner": [0, 13, 26], "inp": 4, "inplac": 13, "input": [0, 1, 3, 4, 5, 6, 7, 8, 12, 13, 14, 16, 22, 25, 26, 27], "input_dim": 1, "input_shap": [3, 4], "inputs": 1, "inputs_shuffl": [0, 1, 26], "insert": [3, 5, 6, 8, 10, 22, 26, 27], "insid": [4, 7], "insight": [0, 1, 5, 19, 25, 26, 27], "insist": [6, 13, 26], "inspir": [0, 1, 12, 25], "instabl": 2, "instal": [0, 1, 5, 6, 9, 15], "instanc": [0, 1, 2, 4, 6, 9, 11, 13, 16, 25, 26, 27], "instanti": 10, "instead": [0, 1, 2, 3, 4, 5, 6, 8, 9, 11, 13, 14, 17, 20, 22, 25, 26], "institut": 1, "instruct": [0, 1, 15], "int": [0, 1, 2, 3, 4, 5, 6, 11, 13, 14, 20, 22, 26], "int32": 10, "int_": [3, 6, 22], "int_0": 22, "int_a": 22, "intak": [0, 26], "integ": [1, 2, 13, 14, 20, 22, 25], "integer_vector": 1, "integr": [3, 6, 22, 25], "intellig": [0, 14, 24, 25], "intend": 10, "intens": [1, 18], "intention": 14, "interact": [0, 6, 9, 12, 19, 25], "intercept": [0, 6, 8, 11, 13, 16, 17, 18, 25, 26, 27], "intercept_": [0, 6, 8, 9, 13, 25, 26], "interchang": [5, 12, 20], "interconnect": 1, "interest": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 12, 19, 22, 25, 26, 27], "interfac": [0, 1, 15, 20, 26], "interior": [0, 9, 25], "intermedi": [20, 26], "intern": [1, 10, 12], "interpol": [1, 3, 4, 6, 12], "interpr": [5, 26, 27], "interpret": [0, 1, 6, 9, 10, 12, 13, 15, 16, 20, 22], "interv": [0, 3, 5, 6, 7, 13, 22, 25, 26, 27], "intial": [13, 27], "intract": [0, 4, 26], "intrins": [3, 11, 20, 22, 25], "intro": [19, 24, 25], "introduc": [0, 1, 5, 6, 8, 10, 12, 20, 22, 25, 27], "introduct": [1, 2, 4, 13, 24, 26, 27], "introductori": [0, 4, 20, 24, 25, 26], "intuit": [0, 5, 6, 8, 12, 13, 25], "inv": [0, 5, 13, 17, 25, 26, 27], "invalu": [0, 13, 19, 25, 27], "invari": 1, "invd": 5, "inver": 8, "invers": [0, 3, 6, 13, 25, 26, 27], "inverse_transform": 8, "invert": [0, 5, 7, 10, 13, 16, 18, 25], "invh": 13, "invok": 8, "involv": [0, 2, 6, 7, 11, 12, 25, 26], "io": [0, 19, 21, 23, 24, 25, 26], "ip": [0, 8, 22, 25], "ipca": 11, "ipynb": [19, 25], "ipython": [0, 5, 7, 9, 11, 14, 19, 25, 26], "iq": 6, "iri": [8, 9], "irreduc": 6, "irrelev": [5, 26, 27], "irrespect": [0, 25], "isn": 5, "isnul": [], "isomap": 11, "issu": [1, 9, 15, 20], "it_arrai": 13, "item": [0, 13, 25], "items": [20, 25], "iter": [1, 2, 4, 6, 8, 13, 14, 18, 22, 27], "its": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 19, 20, 22, 25, 27], "itself": [5, 6, 12, 22, 25, 26], "j": [0, 1, 2, 3, 4, 5, 6, 8, 9, 11, 12, 13, 14, 15, 16, 20, 22, 24, 25, 26, 27], "j1": 20, "j_": 6, "j_lasso_sk": 6, "j_ridge_sk": 6, "j_sk": 6, "jackknif": [6, 19, 25], "jacobian": [2, 13, 27], "jason": 4, "jax": [19, 25], "jensen": [23, 25, 26, 27], "jerom": 24, "ji": [12, 20], "jit": 13, "jj": [0, 5, 6, 25], "jk": [0, 1, 6, 12, 20, 25], "jl": [0, 25], "jm": 20, "jnp": 13, "job": [2, 8, 10, 15], "join": [0, 4, 6, 7, 9, 25], "joint": [4, 5], "judg": [13, 27], "judgement": 6, "julia": [19, 20], "jump": 22, "junk": 4, "jupit": 25, "jupyt": [0, 15, 16, 19, 24, 25], "just": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 19, 22, 25, 26, 27], "justif": 0, "justifi": [3, 10], "k": [0, 1, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 19, 20, 22, 23, 25, 26, 27], "k0": 7, "k1": 7, "kaggl": 6, "kappa_d": 22, "karl": [23, 25], "karush": 8, "katrin": [23, 25], "keep": [0, 1, 4, 5, 6, 11, 13, 14, 15, 18, 20, 25, 26, 27], "keepdim": [1, 6, 10, 20], "kei": [1, 3, 6, 12], "kept": [4, 6, 14], "kera": [0, 4, 19, 25], "kernel": [0, 1, 3, 19, 25, 26], "kernel_regular": [1, 3], "kernel_s": 4, "kernelpca": 11, "kev": [0, 25], "kevin": [24, 25], "keyword": [20, 25], "kfold": 6, "kg": 1, "ki": 20, "kick": [1, 13], "kiener": 2, "kilomet": [6, 26], "kind": [0, 2, 3, 4, 8, 12, 13, 14, 25, 26], "kj": [6, 12, 20, 26], "kjm": [19, 25], "kkt": 8, "kl": 22, "km": [12, 25], "kmean": 14, "kmeanspoint": 14, "kn_k": 14, "know": [0, 1, 2, 5, 6, 8, 13, 15, 16, 17, 19, 25, 26, 27], "knowledg": [0, 19, 25], "known": [1, 3, 4, 5, 6, 7, 8, 9, 12, 18, 20, 22, 24, 26], "kondev": [0, 25], "kp": 22, "kpca": 11, "kroneck": 14, "kuhn": 8, "kvalsund": [23, 25], "kwown": [0, 25], "l": [0, 1, 2, 3, 5, 6, 7, 8, 10, 11, 12, 13, 20, 22, 25, 27], "l0": 7, "l1": [0, 1, 3, 7, 25], "l1_l2": [1, 3], "l1regl": 5, "l2": [1, 3], "l_": 20, "l_1": 7, "l_2": [7, 13, 27], "l_j": 12, "la": 13, "la_i": 12, "la_k": 12, "lab": [19, 25], "label": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 14, 15, 19, 20, 22, 25, 26, 27], "labelencod": [7, 10], "labels": [6, 8, 9], "labels_shuffl": [0, 1, 26], "laboratori": 21, "lack": [0, 25], "lagari": 2, "lagrang": [8, 11], "lam": 18, "lambda": [0, 1, 2, 3, 5, 6, 7, 8, 10, 12, 13, 17, 18, 22, 25, 26, 27], "lambda_": 11, "lambda_0": 11, "lambda_1": [5, 8, 11, 26, 27], "lambda_2": [8, 11], "lambda_i": [8, 11], "lambda_iy_i": 8, "lambda_jy_iy_j": 8, "lambda_k": 8, "lambda_n": [5, 8, 26, 27], "lamda": 1, "land": 8, "landmark": 8, "landscap": [13, 18, 27], "langl": [0, 6, 11, 22, 25, 26], "languag": [0, 1, 4, 8, 19, 20, 24, 25], "lapack": [20, 25], "laplac": 5, "laptop": [15, 19], "larg": [0, 1, 2, 4, 5, 6, 8, 9, 10, 11, 13, 18, 19, 20, 22, 24, 25, 26, 27], "larger": [0, 3, 5, 6, 8, 10, 11, 13, 17, 22, 25, 26, 27], "largest": [4, 8, 11], "lasso": [0, 7, 19, 25], "lasso_sk": 6, "last": [0, 1, 3, 4, 5, 6, 7, 8, 12, 16, 17, 20, 22, 23, 25, 27], "latent": 4, "latent_dim": 4, "latent_point": 4, "latent_space_value_rang": 4, "later": [0, 1, 4, 7, 8, 12, 13, 14, 15, 19, 25], "latest": [4, 15, 19], "latest_checkpoint": 4, "latex": 25, "latter": [0, 3, 6, 7, 8, 11, 13, 20, 22, 25, 26, 27], "lattic": 12, "law": 0, "layer": [0, 4, 13, 25], "lbfg": [7, 9, 10], "lcc": [5, 6], "lda": 11, "ldot": [0, 6, 11, 25], "le": [5, 7, 10, 13, 17, 22, 26, 27], "lead": [0, 1, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 16, 17, 20, 22, 25, 26, 27], "leaf": 9, "leaki": 1, "leakyrelu": 4, "lear": [13, 27], "learn": [3, 4, 5, 6, 7, 8, 9, 10, 12, 20, 23, 24], "learnabl": 3, "learner": 10, "learnig": 25, "learning_r": [8, 10], "learning_rate_init": [0, 1, 25], "learning_schedul": 13, "least": [0, 7, 8, 10, 11, 17, 18, 19, 20, 22], "leat": 13, "leav": [0, 1, 3, 5, 6, 9, 11, 25, 27], "lectur": [0, 1, 5, 10, 11, 12, 13, 19, 20, 21, 23, 24, 26], "lecturenot": [0, 19, 24, 25], "left": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 20, 22, 25, 26, 27], "leftarrow": [8, 12], "legend": [0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 13, 15, 25, 26, 27], "leinonen": 25, "len": [0, 1, 2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 16, 17, 20, 25, 26, 27], "length": [0, 1, 3, 4, 8, 9, 13, 16, 19, 25, 26, 27], "length_of_sequ": 4, "leq": [0, 5, 7, 8, 13, 14, 22, 25, 26, 27], "less": [0, 1, 3, 4, 5, 6, 8, 9, 13, 19, 22, 25, 26, 27], "lessen": 1, "let": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 20, 22, 25, 26, 27], "letter": [0, 16, 20, 22, 25, 26], "level": [0, 1, 5, 6, 9, 19, 20, 21, 23, 25], "li": [8, 11], "lib": [], "liblinear": 10, "librari": [0, 1, 2, 3, 4, 5, 6, 9, 10, 11, 20, 22, 24, 26, 27], "licens": [0, 1, 19, 25], "lie": [0, 6, 11, 22, 25, 26], "life": [0, 1, 8, 12, 25], "lifetim": 13, "like": [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, 15, 16, 19, 20, 22, 25, 26, 27], "likelihood": [0, 1, 5, 9, 25, 26], "lim_": 22, "limit": [0, 5, 6, 8, 12, 20, 25, 26], "lin_clf": 8, "lin_model": [], "lin_reg": 9, "linalg": [0, 2, 5, 6, 8, 11, 13, 17, 20, 22, 25, 26, 27], "line": [0, 3, 6, 8, 11, 13, 15, 16, 25, 27], "line1": 8, "line2": 8, "line3": 8, "line_model": 15, "line_ms": 15, "line_predict": 15, "linear": [1, 3, 5, 6, 7, 9, 10, 11, 12, 16, 17, 18, 19, 22], "linear_model": [0, 5, 6, 7, 8, 9, 10, 11, 13, 15, 16, 25, 26, 27], "linear_regress": 6, "linearli": [5, 26, 27], "linearloc": [6, 13, 27], "linearregress": [0, 6, 7, 9, 15, 16, 25, 26], "linearsvc": 8, "lineat": 27, "liner": [1, 3], "linerar": 10, "linewidth": [0, 2, 4, 6, 8, 9, 10], "link": [0, 4, 9, 12, 15, 19, 21, 23, 25], "linlag": 5, "linpack": [20, 25], "linreg": [0, 25], "linspac": [0, 2, 3, 4, 6, 8, 9, 10, 13, 16, 17, 20, 22, 25, 26], "linu": 4, "linux": [0, 1, 19, 25], "liquid": [0, 25], "list": [1, 2, 3, 4, 9, 15, 19, 25], "listedcolormap": [9, 10], "literatur": [1, 7, 14, 24], "littl": [1, 3, 9, 12], "live": [8, 16], "ll": [0, 18, 22, 25, 26], "lle": [0, 26], "lloyd": [4, 14], "lmb": [0, 2, 5, 6, 26, 27], "lmbd": [0, 1, 3, 25], "lmbd_val": [0, 1, 3, 25], "lmbda": [13, 27], "ln": [1, 13, 27], "load": [1, 4, 6, 7, 9, 10], "load_boston": [], "load_breast_canc": [1, 7, 9, 10, 11], "load_data": [3, 4], "load_digit": [1, 3], "load_iri": [8, 9], "loc": [3, 6, 7, 8, 9, 10, 25], "local": [0, 1, 3, 7, 12, 13, 15, 26, 27], "locat": [2, 3, 8, 15], "log": [0, 1, 2, 4, 5, 6, 7, 9, 10, 11, 13, 15, 20, 25], "log10": [0, 5, 6, 26, 27], "log_": [0, 25], "log_clf": 10, "logarithm": [0, 5, 7, 17, 20, 25], "logic": [0, 1, 9, 25], "login": 15, "logist": [0, 1, 2, 8, 9, 10, 11, 12, 13, 19, 26, 27], "logisticregress": [7, 9, 10, 11], "logit": 7, "logreg": [7, 9, 10, 11], "logspac": [0, 1, 3, 5, 6, 25, 26, 27], "long": [0, 1, 3, 4, 12, 13, 25, 27], "longer": [2, 3, 8, 10, 14, 20, 22, 25], "loocv": 6, "look": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 15, 16, 20, 22, 25, 26, 27], "loop": [1, 4, 6, 10, 12, 14, 16, 17, 18, 19, 20, 25], "lose": 1, "loss": [0, 1, 3, 4, 5, 6, 7, 8, 10, 11, 13, 20, 25], "loss_fil": 4, "lossfil": 4, "lost": 4, "lot": [1, 4, 6, 16], "low": [0, 6, 9, 10, 11, 25, 26], "lower": [0, 1, 3, 6, 9, 10, 16, 20, 26], "lowercas": [20, 25], "lowest": [9, 13, 22], "lr": [1, 3, 4, 10], "lstat": [], "lstm": 4, "lstm_2layer": 4, "lstsq": [0, 25, 26], "lt": 6, "lu": [0, 5, 25, 26, 27], "lubksb": 20, "luckili": 2, "ludcmp": 20, "lux": 20, "lvert": 1, "lw": [0, 25], "m": [0, 1, 2, 3, 5, 6, 8, 9, 10, 11, 12, 13, 15, 18, 20, 22, 23, 24, 25, 26, 27], "m_": [9, 12], "m_1": 14, "m_h": [0, 25], "m_k": 14, "m_l": 12, "m_n": [0, 25], "m_p": [0, 25], "m_t": 13, "ma": 11, "machin": [1, 3, 4, 5, 6, 7, 9, 10, 11, 12, 15, 16, 20, 24, 26], "machinelearn": [0, 6, 16, 19, 21, 23, 24, 25, 26], "mackai": 24, "made": [0, 1, 3, 4, 5, 6, 7, 9, 11, 12, 25, 26], "mae": [0, 25], "magic": 4, "magnitud": [1, 6, 7, 13, 26], "mai": [0, 1, 2, 3, 5, 6, 7, 8, 9, 11, 12, 13, 19, 20, 22, 25, 26, 27], "mail": [21, 23], "main": [0, 1, 3, 4, 5, 6, 7, 9, 20, 24, 26, 27], "mainli": [0, 5, 6, 7, 9, 25, 26], "maintain": 6, "major": [1, 6, 9, 10, 13, 20, 25, 27], "make": [1, 2, 3, 4, 5, 6, 7, 8, 11, 12, 13, 15, 16, 18, 19, 20, 22, 24, 25, 27], "make_axes_locat": 6, "make_moon": [8, 9, 10], "make_pipelin": [0, 6, 10, 26], "makedir": [0, 6, 7, 9, 25], "malcondit": 20, "malign": [1, 7, 9], "mammographi": 5, "manag": [0, 2, 3, 15, 19, 25], "mandatori": [23, 25], "mani": [0, 1, 3, 4, 5, 6, 7, 8, 9, 11, 13, 14, 15, 16, 17, 18, 19, 20, 22, 24, 25, 26, 27], "manifold": 11, "manner": 3, "manual": [6, 26], "map": [0, 1, 2, 6, 7, 8, 11, 12, 14, 22, 25], "marc": 26, "margin": [0, 5, 8], "marit": [0, 25], "mark": 25, "marker": [7, 20, 25], "markov": [19, 25], "marsaglia": 22, "mass": [0, 1, 5, 13, 26, 27], "massag": [0, 25], "masses2016": [0, 25], "masses2016ol": [0, 25], "masses2016tre": 0, "masseval2016": [0, 25], "master": [21, 23], "mat": [19, 25], "mat1100": [19, 25], "mat1110": [19, 25], "mat1120": [19, 25], "match": [1, 4, 5, 13, 14, 15, 26, 27], "materi": [4, 5, 7, 13, 15, 20, 21, 23], "math": [3, 7, 12, 13, 20, 22, 24, 25], "mathbb": [0, 4, 5, 6, 7, 8, 11, 12, 13, 14, 17, 20, 22, 25, 26, 27], "mathbf": [0, 5, 6, 7, 8, 13, 20, 25, 26, 27], "mathcal": [1, 5, 6, 7, 13], "matheemat": 3, "mathemat": [0, 6, 11, 12, 13, 19, 20, 22, 24, 25], "mathemati": 25, "mathrm": [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 17, 22, 25, 26, 27], "matmul": [1, 2, 5], "matnat": 24, "matplotlib": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 19, 20, 22, 25, 26, 27], "matric": [0, 1, 3, 4, 6, 7, 8, 11, 13, 16, 17, 19, 26, 27], "matrix": [0, 2, 3, 4, 6, 7, 8, 10, 13, 17, 18, 22], "matshow": 1, "matter": [2, 3, 13, 26, 27], "max": [0, 1, 2, 3, 4, 9, 10, 12, 13, 23, 25, 27], "max_depth": [0, 9, 10], "max_diff": 2, "max_diff1": 2, "max_diff2": 2, "max_it": [0, 1, 8, 13, 25], "max_iter": 14, "max_leaf_nod": 10, "max_sampl": 10, "maxdegre": [0, 6, 10, 26], "maxdepth": 10, "maxim": [1, 4, 5, 7, 8, 11], "maximum": [0, 2, 3, 5, 7, 8, 9, 10, 13, 14, 25, 26, 27], "maxpolydegre": [5, 6, 26, 27], "maxpooling2d": 3, "mbox": [5, 6, 26, 27], "mcculloch": 12, "md": 11, "mdoel": 4, "mean": [1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, 14, 15, 17, 18, 19, 20, 22, 25], "mean_absolute_error": [0, 25], "mean_divisor": 14, "mean_i": 22, "mean_matrix": 14, "mean_squared_error": [0, 4, 6, 7, 10, 15, 25, 26], "mean_squared_log_error": [0, 25], "mean_vector": 14, "mean_x": 22, "meaning": [0, 4, 7, 25], "meansquarederror": [0, 25], "meant": [3, 7, 10, 13], "measur": [0, 1, 2, 5, 6, 9, 11, 12, 14, 16, 18, 22, 25, 26], "mechan": [0, 4, 22, 25], "median": [0, 25, 26], "medicin": 12, "medium": [4, 8, 13], "medv": [], "meet": [0, 23], "mehta": [0, 25, 26, 27], "memori": [3, 4, 11, 12, 13, 18, 20], "mention": [0, 12, 13, 22, 25, 27], "mere": 0, "meshgrid": [2, 5, 6, 8, 9, 10, 11], "mess": 15, "messag": [5, 13], "messi": 2, "met": [0, 3, 8, 26], "meteorolog": 9, "meter": [6, 26], "method": [0, 1, 2, 3, 4, 5, 7, 8, 11, 12, 14, 15, 16, 17, 19, 20, 22, 24, 26], "metion": 6, "metric": [0, 1, 3, 6, 7, 9, 10, 14, 15, 25, 26], "metropoli": [19, 25], "mev": [0, 22, 25], "mgd": 13, "mglearn": [19, 25], "mgrid": 13, "mhjensen": [], "mi": 10, "mia": [23, 25], "microsoft": 24, "mid": 1, "midel": 4, "midnight": 15, "midpoint": 9, "might": [0, 1, 2, 4, 6, 9, 13, 15, 17, 18, 26, 27], "migth": 17, "mild": 9, "millimet": [6, 26], "million": [0, 25, 26], "mimic": 12, "min": [0, 2, 5, 8, 9, 27], "min_": [0, 2, 5, 14, 17, 25, 26, 27], "min_samples_leaf": 9, "mind": [0, 6, 13, 15, 18, 25, 26, 27], "mindboard": 4, "mine": [19, 25], "mini": [1, 11, 12, 13, 27], "minibatch": [1, 11, 13], "minibathc": 13, "miniforge3": [], "minim": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 26, 27], "minima": [0, 1, 7, 13, 25, 27], "minimum": [0, 1, 2, 6, 8, 9, 11, 13, 26, 27], "minmaxscal": [0, 26], "minor": 22, "minst": 1, "minu": 7, "mirjalili": 25, "mirror": 9, "misc": 6, "misclassif": [8, 9, 10], "misclassifi": [8, 10], "miser": 0, "mismatch": 1, "miss": [7, 10], "mistak": 4, "mit": 24, "mix": [1, 2, 25], "mixtur": 13, "mk": [9, 20], "mkdir": [0, 6, 7, 9, 25], "ml": [0, 1, 10, 13, 20, 26, 27], "mlab": 22, "mle": [5, 7], "mlp": 1, "mlpclassifi": 1, "mlpregressor": [0, 25], "mm": 20, "mml": 26, "mn": [12, 22], "mnist": [1, 11], "mod": 22, "mode": [21, 23, 25], "model": [2, 3, 5, 7, 8, 9, 10, 11, 13, 14, 16, 18, 19, 22, 24, 26, 27], "model_select": [0, 1, 3, 5, 6, 7, 9, 10, 11, 15, 16, 17, 25, 26, 27], "moder": 10, "modern": [0, 6, 7, 19, 25], "modif": [2, 12, 13], "modifi": [0, 1, 3, 5, 7, 8, 10, 12, 13, 25, 26, 27], "modul": [0, 16, 20, 25], "modular": 22, "modulo": 22, "moe": [11, 26], "moment": [5, 6, 13, 22], "mondai": [23, 25], "monitor": [13, 18], "monoton": [5, 12, 22], "mont": [0, 6, 19, 22, 24, 25], "montli": 16, "moor": [5, 6], "more": [0, 1, 2, 4, 5, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 19, 22], "moreov": [0, 3], "morten": [23, 25, 26, 27], "mortenhj": 25, "most": [0, 1, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 19, 22, 25, 26, 27], "mostli": [1, 11, 18], "motion": [0, 13], "motiv": [1, 4], "move": [0, 4, 5, 6, 7, 9, 12, 13, 14, 15, 16, 22, 26, 27], "mpl": [7, 25], "mpl_toolkit": [2, 6, 13, 27], "mplot3d": [2, 6, 13, 27], "mplregressor": 1, "mse": [0, 4, 5, 6, 9, 10, 15, 16, 17, 18, 25, 26, 27], "mse_simpletre": 10, "mselassopredict": [5, 27], "mselassotrain": [5, 27], "mseownridgepredict": [6, 26, 27], "msepredict": [5, 27], "mseridgepredict": [0, 5, 6, 26, 27], "msetrain": [5, 27], "msle": [0, 25], "mt": [7, 12], "mu": [0, 6, 11, 13, 22, 25], "mu0": 22, "mu1": 22, "mu2": 22, "mu_": [6, 22, 26], "mu_i": [6, 26], "mu_n": 11, "mu_x": 22, "much": [0, 1, 2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 13, 15, 20, 22, 25, 26, 27], "multi": [0, 1, 3, 7, 19, 25], "multiclass": [1, 7], "multidimension": [11, 12, 25], "multilay": 1, "multinomi": 7, "multipl": [2, 4, 5, 6, 7, 12, 13, 15, 22, 26, 27], "multipli": [3, 5, 6, 11, 13, 18, 20, 22, 26, 27], "multiplum": 8, "multivari": [0, 2, 10, 11, 19, 22, 25], "multivariate_norm": [11, 14], "multpli": 16, "murphi": [11, 24, 25], "must": [1, 2, 5, 6, 8, 10, 12, 13, 14, 15, 22, 26, 27], "mutat": 7, "mutual": [1, 3, 6, 13], "mx_": 22, "my": 25, "myenv": [], "myriad": [0, 19, 25], "mz1": 22, "mz2": 22, "m\u00f8svatn": 6, "n": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 22, 25, 26, 27], "n1": 20, "n2": 20, "n_": [1, 2, 3, 8, 12, 22], "n_0": [12, 22], "n_boostrap": [6, 10], "n_bootstrap": 6, "n_categori": [1, 3], "n_cluster": 14, "n_compon": 11, "n_epoch": 13, "n_estim": 10, "n_examples_to_gener": 4, "n_featur": [1, 18], "n_filter": 3, "n_hidden": 2, "n_hidden_neuron": [0, 1, 25], "n_i": 22, "n_input": [0, 1, 3, 26], "n_instanc": 9, "n_job": 10, "n_k": 14, "n_l": [12, 22], "n_layer": 1, "n_m": 9, "n_neuron": 1, "n_neurons_connect": 3, "n_neurons_layer1": 1, "n_neurons_layer2": 1, "n_point": 14, "n_sampl": [6, 8, 9, 10, 14, 18], "n_split": 6, "n_step": 4, "n_t": 2, "n_x": 2, "nabla": [1, 13, 27], "nabla_": [2, 13, 27], "nabla_w": 13, "nag": 13, "naimi": [0, 25], "naiv": 7, "naive_kmean": 14, "name": [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 14, 15, 19, 20, 22, 23, 25, 26, 27], "narrow": 13, "nation": [1, 5], "nativ": [19, 25], "natur": [0, 1, 4, 8, 9, 12, 13, 22, 24, 25, 27], "navier": 12, "navig": 15, "nb": 22, "nb_": 20, "nbconvert": 25, "nd": 14, "ndarrai": 6, "ne": [9, 10, 20, 22, 26, 27], "nearest": [1, 3, 6, 11], "nearli": [13, 27], "neat": 25, "neccesari": 6, "necess": 2, "necessari": [0, 1, 3, 4, 8, 14, 18, 25], "necessarili": [0, 4, 11, 22, 25], "necesserali": 5, "neck": 7, "need": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 20, 22, 26, 27], "neg": [0, 1, 3, 5, 6, 7, 10, 13, 20, 22, 25, 27], "neg_mean_squared_error": 6, "neglect": 22, "neglig": 22, "neighbor": [3, 6, 11], "neither": [4, 13], "neq": [13, 14, 22, 27], "nervou": 12, "nest": [9, 12], "nesterov": 13, "net": [2, 4, 12], "netlib": [20, 25], "network": [0, 9, 13, 19, 24, 26], "neural": [0, 13, 19, 24, 26], "neural_network": [0, 1, 2, 25], "neuralnetwork": 1, "neuron": [1, 2, 3, 4, 12], "neutral": [0, 25], "neutron": [0, 25], "never": [1, 4, 6, 9, 22], "new": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 17, 20, 25, 26, 27], "new_chang": 13, "new_hobbit": 25, "newaxi": [0, 3, 6, 9], "newli": [0, 25], "newton": [1, 7, 8, 13, 22], "next": [0, 1, 2, 3, 4, 5, 6, 8, 9, 13, 14, 15, 16, 25, 26, 27], "next_guess": 13, "next_input": 4, "ng": 1, "ni": 14, "nice": [0, 1, 5, 11, 25, 26, 27], "nicer": 18, "niter": [13, 27], "nitric": [], "nlambda": [0, 5, 6, 26, 27], "nlp": 24, "nm": 22, "nm_n": [0, 25], "nmse": 6, "nn": [2, 5, 6, 12, 20, 25], "nn_model": 1, "nnmin": 2, "node": [1, 3, 9, 10, 12], "nois": [0, 4, 5, 6, 8, 9, 10, 13, 18, 25, 26, 27], "noise_dimens": 4, "noisi": [1, 6], "non": [0, 1, 3, 5, 6, 7, 9, 10, 11, 12, 13, 14, 18, 20, 22, 25, 26, 27], "none": [0, 1, 2, 4, 5, 9, 10, 13, 22, 25, 26], "nonlinear": [3, 6, 8, 9, 11, 12], "nonneg": [6, 9, 13, 27], "nonparametr": 6, "nonsens": 22, "nonsingular": 20, "nonumb": [3, 7, 8, 13, 20], "nor": [1, 4, 13], "norm": [0, 1, 5, 6, 8, 11, 13, 25, 26, 27], "normal": [3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 16, 17, 18, 19, 20, 22, 25, 26, 27], "normali": [20, 25], "norwai": [6, 25, 27], "notat": [0, 2, 5, 6, 13, 14, 22, 25, 26, 27], "note": [0, 1, 2, 3, 4, 5, 6, 7, 8, 11, 12, 13, 14, 15, 16, 19, 20, 22, 24, 25], "notebook": [0, 1, 3, 9, 15, 16, 19, 25], "noth": [1, 2, 5, 8, 12, 14, 22, 26, 27], "notic": [4, 5, 12, 13, 20, 22, 25], "notion": 3, "novel": [3, 6, 10, 25], "novemb": [1, 23, 25], "now": [0, 2, 4, 5, 6, 7, 8, 10, 11, 12, 14, 15, 16, 19, 20, 22, 25, 26], "nowadai": [0, 1, 3, 9, 19, 25], "nox": [], "np": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 16, 17, 18, 20, 22, 25, 26, 27], "npr": 2, "nsampl": 6, "nt": 2, "nu": 22, "nuclear": [5, 26, 27], "nuclei": [0, 22, 25], "nucleon": [0, 25], "nucleu": [0, 25], "num": 4, "num_coordin": 2, "num_hidden_neuron": 2, "num_it": [2, 18], "num_neuron": 2, "num_neurons_hidden": 2, "num_point": 2, "num_tre": 10, "num_valu": 2, "number": [1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 18, 20, 21, 23, 25, 27], "numberid": 7, "numberparamet": 3, "numer": [0, 5, 6, 9, 10, 11, 12, 13, 19, 20, 24, 25, 26, 27], "numpi": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 22, 26, 27], "nunmpi": [5, 26], "nx": 2, "ny": 22, "o": [0, 1, 4, 5, 6, 7, 8, 9, 11, 20, 23, 24, 25, 26, 27], "obei": [6, 11, 13, 26], "object": [0, 1, 4, 8, 10, 15, 20, 25], "obliqu": [5, 26, 27], "observ": [0, 1, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 22, 25, 27], "obtain": [0, 1, 5, 6, 7, 8, 9, 10, 12, 13, 14, 17, 20, 22, 25, 26, 27], "obviou": [5, 6, 11, 22, 26, 27], "obviouli": 25, "obvious": [0, 4, 5, 6, 20, 25], "oc": [26, 27], "occupi": [], "occur": [0, 6, 8, 9, 20, 22, 25], "octob": [23, 25], "od": 0, "odd": [0, 3, 7, 25, 26], "odenum": 2, "odesi": 2, "oen": 0, "off": [1, 3, 4, 5, 9, 13, 22], "offer": [6, 11, 19, 20, 21, 23, 25], "offic": [23, 25], "offici": [21, 25], "often": [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 19, 20, 22, 25, 26, 27], "ofter": [20, 25], "ol": [0, 13, 17, 26], "old": [1, 5, 10, 13, 15], "ols_paramet": 16, "ols_sk": 6, "ols_svd": 6, "olsbeta": 27, "olstheta": [0, 5], "omega": [2, 3, 6], "omega_0": 3, "omit": [0, 5, 25, 26, 27], "onc": [1, 6, 9, 11, 13], "one": [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 18, 19, 20, 22, 23, 25, 26], "onehot": 1, "onehot_vector": 1, "onehotencod": 9, "ones": [0, 2, 5, 6, 8, 9, 10, 11, 13, 16, 18, 20, 25, 26, 27], "ones_lik": 4, "ong": 26, "onl": 3, "onli": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 18, 20, 22, 25, 26, 27], "onlin": [11, 15, 21], "onto": [5, 11, 26, 27], "open": [0, 1, 4, 6, 7, 9, 15, 19, 21, 23, 25], "oper": [0, 1, 3, 5, 6, 10, 11, 12, 13, 15, 16, 19, 22, 25, 26, 27], "operation": 22, "oplu": 22, "opmiz": 13, "opportun": 0, "oppos": [6, 13], "opposit": [1, 5, 8, 26, 27], "opt": [1, 5, 25, 27], "optim": [0, 2, 3, 4, 5, 6, 7, 9, 10, 11, 14, 16, 17], "optimis": [1, 3], "option": [0, 1, 3, 5, 6, 8, 11, 15, 18, 20, 26], "optmiz": [1, 8, 13, 26], "oral": 25, "orang": 0, "order": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 15, 20, 22, 25, 26, 27], "ordinari": [0, 2, 3, 7, 11, 13, 17, 18, 19], "oreilli": [24, 25], "org": [0, 3, 4, 16, 19, 20, 24, 25, 26, 27], "organ": [6, 7, 10, 20], "orient": [1, 5, 22, 26, 27], "origin": [0, 3, 5, 6, 8, 11, 12, 13, 15, 20, 25, 26, 27], "orthogn": [5, 26, 27], "orthogon": [0, 5, 6, 8, 11, 13, 20, 25, 26, 27], "orthonorm": [5, 26, 27], "os": [23, 25], "oscar": 1, "oscil": [3, 13], "oskar": 25, "oskarlei": 25, "osl": 18, "oslo": [0, 19, 21, 23, 25, 26, 27], "osx": [0, 19, 25], "other": [0, 1, 2, 3, 5, 6, 7, 8, 10, 13, 14, 16, 19, 21, 22, 23, 24, 26, 27], "otherwis": [0, 1, 4, 7, 13, 20, 25], "ouput": [5, 7, 12], "our": [1, 2, 3, 6, 7, 8, 9, 10, 12, 14, 15, 16, 17, 18, 19, 20, 22], "ourmodel": 0, "ourselv": [0, 5, 6, 8, 11, 13, 25, 26, 27], "out": [0, 1, 2, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16, 19, 20, 22, 25, 26], "out_fil": 9, "outcom": [0, 7, 9, 10, 12, 22, 26], "outdoor": 9, "outer": [6, 12, 13], "outfil": 4, "outlier": [0, 8, 25, 26], "outlin": [6, 10, 11], "outlook": 9, "outperform": 10, "output": [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 20, 22, 25, 26, 27], "output_bia": 1, "output_bias_gradi": 1, "output_shap": 4, "output_weight": 1, "output_weights_gradi": 1, "outputlayer1": 12, "outputlayer2": 12, "outsid": 4, "over": [0, 1, 3, 4, 5, 6, 9, 10, 12, 13, 15, 16, 20, 25, 26, 27], "over1": 13, "overal": [1, 10], "overcast": 9, "overcom": [12, 13], "overdetermin": [0, 25], "overfit": [0, 1, 3, 6, 9, 10, 13], "overflow": 5, "overhead": 12, "overlap": [3, 7, 8, 9], "overlin": [0, 5, 6, 9, 10, 11, 14, 20, 25, 26], "overst": 0, "overtrain": 4, "overview": 3, "own": [4, 5, 6, 8, 12, 13, 16, 18, 19, 20, 27], "owner": [], "ownmsepredict": 0, "ownmsetrain": 0, "ownridgebeta": 26, "ownridgetheta": [0, 6, 26, 27], "ownypredictridg": 0, "ownytilderidg": 0, "oxid": [], "p": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 16, 17, 18, 20, 22, 25, 26, 27], "p0": 2, "p1": 2, "p_": [2, 4, 8, 9], "p_hidden": 2, "p_i": [5, 22], "p_j": 22, "p_n": 22, "p_output": 2, "p_x": 22, "pack": [0, 25], "packag": [0, 1, 3, 4, 5, 8, 11, 13, 15, 19, 22, 26, 27], "packtpub": 25, "packtpublish": 25, "pad": [3, 4], "page": [0, 19, 25, 27], "pai": [0, 1, 9, 13, 15], "pair": [0, 2, 3, 9, 19, 22, 25], "paltform": 15, "panda": [0, 4, 5, 6, 7, 9, 11, 19, 27], "panel": 25, "paper": 1, "paradigm": [0, 25], "parallel": [10, 13, 19, 20, 25], "param": 2, "paramat": [2, 18], "paramet": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 16, 17, 18, 22, 27], "parameter": [0, 6, 10, 25, 26], "parametr": [0, 6, 25, 26], "paramt": [3, 5], "part": [0, 1, 3, 5, 6, 10, 17, 20, 21, 22, 23, 25, 26], "partial": [0, 1, 5, 6, 7, 8, 10, 11, 12, 13, 16, 22, 25, 26, 27], "particip": [15, 19, 21, 23, 25], "particl": [0, 4, 13, 22, 25], "particular": [0, 1, 2, 3, 5, 6, 9, 10, 11, 12, 13, 16, 22, 24, 25, 26, 27], "particularli": [5, 6, 8, 11, 13, 22, 26, 27], "partit": [1, 4, 9], "partli": [6, 25], "partner": 15, "pass": [2, 3, 12, 14], "past": [10, 22], "patch": [6, 22], "path": [0, 4, 6, 7, 9, 19, 25], "pathcollect": 17, "patient": 7, "patter": 4, "pattern": [0, 3, 4, 12, 24, 25], "pauli": [0, 25], "pc": [11, 15, 19], "pca": [0, 7, 19, 25, 26], "pd": [0, 4, 5, 6, 7, 9, 11, 25, 26, 27], "pde": 2, "pdf": [0, 3, 4, 5, 6, 9, 15, 16, 24, 25], "pedagog": [0, 25, 26], "penal": [6, 18, 26], "penalti": [6, 13, 18, 26], "penros": [5, 6], "pentagon": [13, 27], "peopl": [1, 9, 13, 19], "per": [0, 1, 6, 21, 23, 25], "percentag": [10, 11, 23], "perceptron": [0, 1, 7, 25], "peregrin": 25, "perfect": [0, 1, 13, 25], "perfectli": [4, 6], "perform": [0, 2, 3, 4, 5, 6, 8, 10, 11, 12, 13, 14, 16, 18, 19, 20, 22, 25, 26, 27], "performac": 4, "perhap": [0, 5, 13, 25, 26, 27], "perimet": 1, "period": [1, 4, 22], "permiss": 15, "permut": 11, "persist": 13, "person": [5, 6, 7, 16, 21, 23, 25, 26], "perspect": 24, "pertin": [12, 25], "petal": [8, 9], "peter": [24, 26], "phantom": 22, "phase": [6, 12], "phenomena": 22, "phi": 8, "phi_k": 8, "philosophi": 13, "phone": [23, 25], "photo": [4, 25], "phrase": [0, 25], "physic": [0, 1, 4, 7, 12, 13, 22, 23, 24, 25, 26, 27], "pi": [2, 3, 5, 6, 7, 9, 12, 13, 22], "pick": [1, 9, 10, 11, 13, 14], "pickl": 1, "pictur": [0, 25], "pie": [19, 25], "piec": [11, 14], "pillow": [0, 19, 25], "pinv": [5, 6, 13, 26, 27], "pip": [0, 1, 15, 19, 25], "pip3": [0, 1, 25], "pipelin": [0, 6, 8, 10, 26], "pippin": 25, "pit": 4, "pitfal": [6, 26], "pitt": 12, "pixel": [1, 3, 4, 25], "pixel_height": [1, 3], "pixel_width": [1, 3], "place": [0, 4, 6, 8, 13, 15, 20, 25, 27], "plai": [0, 3, 4, 5, 6, 8, 11, 19, 25, 26, 27], "plain": [8, 10, 12, 13, 14, 27], "plan": [6, 9, 23, 24, 25], "plane": [8, 9], "plateau": [5, 27], "platform": [19, 25], "plausibl": 12, "pleas": [13, 23, 25], "plenti": 1, "plethora": [3, 12], "plot": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 18, 19, 20, 22, 25, 26, 27], "plot_all_sc": 26, "plot_confusion_matrix": [7, 10], "plot_count": 6, "plot_cumulative_gain": [7, 10], "plot_data": 1, "plot_dataset": 8, "plot_decision_boundari": [9, 10], "plot_import": 10, "plot_max": 4, "plot_min": 4, "plot_model": 4, "plot_numb": 4, "plot_predict": 8, "plot_regression_predict": 9, "plot_result": 4, "plot_roc": [7, 10], "plot_surfac": [2, 6, 13], "plot_train": 9, "plot_tre": [9, 10], "plt": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 20, 22, 25, 26, 27], "plu": [0, 3, 5, 7, 18, 25, 26], "pm": 8, "pmatrix": 2, "pml": 24, "pn": 3, "png": [0, 4, 6, 7, 9, 25], "point": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 13, 14, 20, 22, 23, 25, 26, 27], "point_1": 4, "point_2": 4, "poisson": [19, 22, 25], "poli": [6, 8], "poly100_kernel_svm_clf": 8, "poly3": 0, "poly3_plot": 0, "poly_featur": [8, 9, 15], "poly_features10": 9, "poly_fit": 9, "poly_fit10": 9, "poly_kernel_svm_clf": 8, "poly_model": 15, "poly_ms": 15, "poly_predict": 15, "polydegre": [0, 5, 6, 10, 26], "polygon": [13, 27], "polym": 12, "polynomi": [0, 5, 6, 7, 8, 9, 10, 11, 15, 17, 25, 26], "polynomial_featur": [6, 15, 16, 17], "polynomial_svm_clf": 8, "polynomialfeatur": [0, 6, 8, 9, 15, 16, 26], "polytrop": [0, 6], "pool": 3, "pool_siz": 3, "poor": [1, 13, 27], "poorli": [0, 26], "popul": [0, 5, 25, 26], "popular": [0, 1, 3, 6, 7, 8, 9, 11, 12, 15, 19, 20, 22, 26], "popularli": [0, 25], "portabl": 10, "portion": [11, 13], "pose": [0, 4, 5, 6, 11, 22, 25], "posit": [0, 1, 2, 3, 5, 7, 8, 10, 11, 13, 14, 20, 22, 25, 26, 27], "possibl": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 19, 20, 22, 23, 25, 26, 27], "possibli": [6, 8, 13], "posterior": 5, "postpon": [0, 26], "postul": 5, "potenti": [0, 3, 5, 6, 12, 13, 26], "pott": 12, "power": [0, 1, 5, 6, 8, 9, 12, 13, 25, 26, 27], "pp": [5, 6], "practic": [0, 5, 6, 7, 8, 16, 18, 22, 26], "practition": [0, 1, 3, 25], "pre": 25, "preced": [1, 11, 12, 22], "preceed": 4, "preceq": 8, "precis": [0, 2, 5, 11, 13, 20, 22, 25, 26], "pred": 6, "predicit": 0, "predict": [0, 1, 5, 6, 7, 8, 9, 10, 15, 16, 17, 18, 19, 24, 25, 26, 27], "predict_prob": 1, "predict_proba": [7, 10], "predictor": [0, 5, 6, 7, 9, 10, 11, 25, 26], "prefer": [0, 1, 6, 8, 9, 11, 13, 15, 19, 25], "prepar": [0, 6, 20, 25, 26], "preprocess": [0, 4, 6, 7, 8, 9, 10, 11, 15, 16, 17, 18], "prerequisit": 0, "presenc": 13, "present": [0, 5, 6, 7, 9, 12, 13, 20, 22, 25, 26, 27], "preserv": [3, 11, 20], "press": [13, 15, 24, 27], "pretrain": [1, 4], "pretti": [0, 4, 8, 9, 19, 25], "prev_centroid": 14, "prevent": [13, 22], "previou": [0, 1, 2, 3, 4, 5, 6, 8, 10, 11, 12, 13, 15, 16, 20, 22, 26, 27], "previous": [2, 3, 9, 10, 22], "price": [0, 4, 9, 13], "primal": 8, "primari": [0, 7, 25], "prime": 22, "princip": [0, 5, 7, 19, 25, 26, 27], "principl": [0, 6, 7, 8, 14, 25], "print": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 18, 20, 22, 25, 26, 27], "print_funct": [8, 9], "printout": [0, 25], "prior": [0, 5, 6, 25], "privat": 0, "prob": [1, 22], "probabilist": [0, 24, 25, 26], "probabl": [0, 1, 3, 4, 6, 7, 10, 13, 19, 25, 26], "problem": [0, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 17, 19, 20, 22], "probml": 24, "proce": [0, 5, 6, 7, 8, 9, 10, 11, 13, 20, 25, 26], "procedur": [2, 4, 5, 6, 8, 10, 11, 13, 26, 27], "proceed": 20, "process": [0, 2, 4, 6, 9, 10, 12, 13, 19, 20, 22, 24, 25, 27], "prod": 24, "prod_": [1, 5, 7], "produc": [0, 3, 4, 5, 6, 9, 10, 11, 12, 13, 18, 19, 20, 22, 25, 26], "product": [0, 1, 3, 5, 6, 7, 8, 12, 13, 16, 17, 19, 20, 25, 26], "profess": [0, 25], "program": [0, 1, 4, 5, 6, 8, 12, 14, 15, 19, 20, 21, 22, 23, 25, 26], "programm": 20, "progress": [1, 4, 14], "prohibit": 6, "project": [0, 1, 2, 3, 5, 11, 13, 15, 19, 21, 26, 27], "project_root_dir": [0, 6, 7, 9, 25], "promin": 12, "promis": 8, "promot": [23, 25], "prone": [9, 15], "pronounc": [13, 19, 25], "proof": [0, 11, 12, 13, 25, 27], "propag": [2, 3, 13], "proper": [0, 2, 6, 7], "properli": [1, 6, 8, 10, 13, 18], "properti": [0, 1, 3, 12, 13, 16, 20, 25], "proport": [0, 1, 5, 9, 11, 13, 22, 25, 26], "propos": [1, 4, 6, 10, 25], "propto": [5, 13, 27], "proton": [0, 25], "prove": [3, 13, 27], "provid": [0, 1, 3, 4, 5, 6, 8, 9, 10, 12, 13, 19, 20, 22, 25, 26, 27], "proxi": [1, 13], "prune": 9, "pseudo": [20, 22], "pseudoinv": 5, "pseudoinvers": [5, 6], "pseudorandom": [6, 22], "psychologi": [0, 25], "pt": 13, "public": [0, 15, 19, 25], "pull": 15, "punish": [0, 1, 25], "pure": [3, 9, 22], "purest": 9, "puriti": 9, "purpos": [0, 3, 10, 12, 14, 25], "push": 15, "put": 1, "py": 5, "pycod": 25, "pydata": 19, "pydot": 9, "pyhton2": 25, "pylab": [7, 25], "pypi": 19, "pyplot": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 20, 22, 25, 26, 27], "pythagora": 5, "python": [1, 2, 3, 5, 6, 8, 11, 12, 13, 14, 22, 26], "python2": 0, "python3": [0, 19, 25], "pytorch": [0, 19, 25], "q": [5, 6, 8, 11, 22], "qp": 8, "qquad": [2, 11, 13, 20], "qr": [5, 6, 20, 26, 27], "quad": [1, 13, 20], "quadrat": [0, 8, 9, 13, 25], "qualit": [4, 9, 22], "qualiti": [0, 9, 19, 25, 26], "quantifi": 1, "quantil": 10, "quantit": [0, 6, 9, 25], "quantiti": [0, 2, 5, 6, 7, 9, 10, 11, 12, 14, 16, 20, 22, 25, 26, 27], "quantum": [4, 12, 24, 25], "quartil": [0, 26], "quench": 5, "queri": 9, "question": [0, 5, 6, 9, 11, 12, 13, 23, 25, 26], "qugan": 4, "quick": [4, 22], "quickli": [1, 3, 9, 11, 13, 27], "quit": [1, 5, 6, 9, 10, 12, 15, 26, 27], "quot": 4, "r": [0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 19, 20, 22, 26, 27], "r2": [0, 5, 6, 25, 26, 27], "r2_score": [0, 25], "r2score": [0, 25], "r_1": 9, "r_2": 9, "r_j": 9, "r_m": 9, "rad": [], "radial": [8, 12], "radioact": 22, "radiu": [0, 1, 26], "rain": 9, "ramp": 1, "ran0": 22, "ran1": 22, "ran2": 22, "ran3": 22, "rand": [0, 4, 5, 6, 9, 10, 13, 15, 20, 25, 26, 27], "randint": [6, 9, 13], "randn": [0, 1, 2, 5, 6, 9, 11, 13, 15, 18, 25, 26, 27], "random": [0, 1, 2, 3, 4, 5, 6, 8, 9, 13, 14, 15, 16, 17, 18, 19, 20, 25, 26, 27], "random_forest_model": 10, "random_index": 13, "random_indic": [1, 3], "random_st": [7, 8, 9, 10, 11], "randomforestclassifi": 10, "randomli": [1, 6, 9, 13, 14, 18, 27], "rang": [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, 14, 18, 20, 22, 25, 26, 27], "rangl": [0, 6, 11, 22, 25, 26], "rangle_x": 22, "rank": [5, 26, 27], "rankdir": 4, "raphson": [1, 8, 13], "rapidli": 0, "rare": [1, 13], "raschka": [25, 26], "rasckha": 25, "rashcka": 27, "rate": [1, 2, 3, 4, 8, 9, 10, 12, 13, 18, 27], "rather": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 20, 22, 25, 26, 27], "ratio": [4, 7, 9, 10, 11], "rational": [0, 25], "ravel": [5, 6, 7, 8, 9, 10, 11, 13, 20], "raw": 3, "rbf": [8, 11, 12], "rbf_kernel_svm_clf": 8, "rbf_pca": 11, "rc": 22, "rcond": [0, 25, 26], "rcparam": [1, 3, 7, 8, 9, 10, 22, 25], "re": [2, 4, 13, 15, 27], "reach": [1, 4, 5, 6, 9, 10, 12, 13, 14, 27], "read": [0, 2, 3, 4, 5, 6, 7, 8, 11, 12, 16, 17, 20, 22, 24, 27], "read_csv": [0, 6, 7, 9], "read_fwf": [0, 25], "reader": [0, 6, 20, 22, 25, 26], "readi": [0, 1, 5, 6, 8, 10, 11, 12, 20, 25], "readili": 1, "readm": 15, "readthedoc": 19, "real": [0, 1, 4, 7, 10, 11, 12, 16, 18, 20, 26], "real_loss": 4, "real_output": 4, "realist": [8, 25], "realiti": 22, "realiz": [1, 12], "realli": [0, 1, 25], "rearrang": 13, "reason": [0, 1, 3, 4, 10, 13, 24, 25, 27], "reassign": 1, "recal": [5, 6, 9, 10, 11, 12, 20, 22, 25, 26, 27], "recast": 3, "receiv": [1, 3, 10, 12, 22], "recent": [0, 6, 13, 24], "recept": [3, 12], "receptive_field": 3, "recip": [0, 6, 7, 20, 25, 26], "reciproc": 5, "recogn": [0, 4, 5, 10, 25], "recognit": [0, 1, 3, 12, 24, 25], "recommen": 25, "recommend": [0, 2, 3, 4, 5, 6, 8, 13, 15, 19, 20, 24, 27], "reconsid": 9, "reconstruct": 11, "record": [10, 21, 23, 25], "recreat": 15, "rectangl": [9, 13, 27], "rectangular": [5, 26, 27], "rectifi": [1, 3, 12], "recur": [0, 19, 25], "recurr": [0, 1, 19, 25], "recurs": [9, 19, 20, 25], "red": [0, 3, 4, 6, 8, 9], "redefin": [0, 10, 25, 26, 27], "redefinit": 27, "reduc": [1, 3, 5, 6, 9, 10, 11, 13, 25, 27], "reduct": [0, 10, 11, 19, 22, 25, 26], "refer": [0, 1, 2, 3, 5, 6, 11, 12, 13, 14, 20, 24, 25, 26, 27], "referenc": 2, "refin": 12, "refit": 6, "reflect": [0, 1, 4, 5, 22, 25], "refresh": [19, 25], "refreshprogrammingskil": 25, "reg": [10, 11], "regard": [1, 9, 13], "regardless": [12, 16], "region": [3, 4, 6, 9, 12], "regist": [6, 22], "reglasso": [5, 27], "regr_1": [0, 9], "regr_2": [0, 9], "regr_3": [0, 9], "regress": [1, 8, 11, 12, 16, 19, 20], "regressor": [0, 7, 10], "regridg": [0, 5, 6, 26, 27], "regular": [0, 3, 4, 5, 6, 7, 9, 13, 17, 18, 23, 25, 26, 27], "regularli": 15, "reilli": [0, 24, 25], "reinforc": [0, 8, 19, 25], "reiter": 1, "reject": 7, "rel": [0, 4, 6, 7, 9, 12, 13, 22, 25, 26], "relat": [0, 1, 3, 4, 5, 11, 13, 14, 20, 22, 25, 26, 27], "relationship": [0, 4, 9, 18, 25], "relativeerror": [0, 25, 26], "releas": [1, 19, 25], "relev": [0, 1, 5, 7, 11, 19, 22, 25, 27], "reli": [0, 6, 8], "reliabl": [7, 22], "relu": [3, 4, 25], "remain": [1, 2, 4, 6, 12, 20, 22, 26], "remaind": 22, "reman": 2, "remark": 1, "rememb": [0, 8, 13, 20, 25], "remind": [0, 5, 11, 13, 20, 22], "remot": 15, "remov": [4, 5, 6, 26, 27], "renam": 15, "render": [0, 25, 26], "reorder": [5, 7, 26, 27], "reorgan": [0, 25], "repeat": [0, 1, 3, 4, 5, 6, 9, 10, 11, 13, 14, 20, 22, 25, 26, 27], "repeated": 25, "repeatedli": [0, 6, 10, 13], "repet": 3, "repetit": [6, 25, 26], "rephras": [13, 27], "replac": [0, 1, 3, 4, 5, 6, 10, 12, 14, 19, 25, 26, 27], "replica": 6, "repo": 15, "report": 25, "repositori": [4, 25], "repres": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 22, 25, 26, 27], "represent": [0, 1, 3, 6, 22, 25], "representd": 3, "reproduc": [0, 5, 6, 9, 12, 15, 16, 18, 19, 22, 25, 26], "repuls": [0, 25], "request": [0, 13], "requir": [0, 1, 3, 4, 5, 6, 8, 9, 11, 12, 13, 15, 17, 18, 20, 25, 26, 27], "res1": 2, "res2": 2, "res3": 2, "res_analyt": 2, "res_analytical1": 2, "res_analytical2": 2, "res_analytical3": 2, "resaml": 6, "resampl": [0, 7, 10, 19, 25, 26], "rescal": [0, 11, 12], "rescu": 5, "reseach": 6, "research": [0, 4, 13, 19, 24, 25], "resembl": [6, 22], "reserv": [1, 5, 6, 22], "reshap": [0, 1, 2, 3, 4, 6, 8, 9, 10, 20, 25, 26], "residenti": [], "residu": [0, 5, 13, 25], "resiz": [5, 26, 27], "resourc": 25, "respect": [0, 1, 2, 3, 5, 6, 7, 8, 10, 11, 12, 13, 14, 16, 17, 18, 22, 25, 26, 27], "respond": 12, "respons": [0, 7, 9, 12, 25, 26], "rest": [0, 5, 18, 26, 27], "restat": [0, 12, 25], "restor": 4, "restored_discrimin": 4, "restored_gener": 4, "restrict": [0, 3, 9, 12, 25], "result": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 25], "retail": [], "retain": [5, 6, 26, 27], "return": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 13, 14, 16, 17, 20, 22, 25, 26, 27], "return_data": 14, "return_sequ": 4, "return_x_i": 9, "reus": [1, 3, 6], "reveal": [0, 12, 25], "revers": [1, 20], "review": [19, 20], "revisit": 14, "revolut": 25, "reward": [0, 4, 25], "rewrit": [0, 3, 5, 6, 7, 8, 10, 11, 12, 13, 16, 20, 22, 27], "rewritten": [2, 6, 8, 10, 22], "rewrot": 13, "rf": 10, "rgb": 3, "rgoj5yh7evk": 19, "rh": 6, "rho": [0, 10, 13], "rho_1": 10, "rho_2": 10, "rho_m": 10, "rich": [0, 25], "ride": 9, "rideclass": 9, "ridedata": 9, "ridg": [7, 11, 13, 19, 25], "ridge_paramet": 17, "ridge_sk": 6, "ridgebeta": 27, "ridgetheta": 5, "right": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 12, 13, 14, 16, 17, 20, 22, 25, 26, 27], "right_sid": 2, "rightarrow": [0, 1, 5, 6, 8, 11, 12, 13, 22, 25, 26, 27], "rigor": [0, 25, 26, 27], "ring": 6, "rise": [0, 25], "risk": [0, 13, 25, 27], "rival": 4, "river": [], "rlm": 25, "rm": 22, "rmse": [], "rmsporp": 13, "rmsprop": [1, 3, 4, 13], "rnd_clf": 10, "rng": 22, "rnn": [4, 12], "rnn1": 4, "rnn2": 4, "rnn_2layer": 4, "rnn_input": 4, "rnn_output": 4, "rnn_train": 4, "rntrick1": 22, "rntrick2": 22, "rntrick3": 22, "rntrick4": 22, "ro": [0, 13, 25, 27], "robert": 24, "robust": [0, 25], "robustscal": [0, 26], "roc": [7, 10], "role": [0, 2, 5, 6, 8, 18, 19, 25, 26, 27], "roll": 6, "room": [0, 23, 25], "root": [0, 5, 9, 13, 15, 22, 26, 27], "rot": 25, "rotat": [1, 8, 9, 10], "rotation_matrix": 9, "roughli": [1, 3, 18], "round": [7, 9, 13], "routin": [13, 20, 25, 27], "row": [0, 1, 2, 5, 6, 9, 11, 16, 20, 25, 26, 27], "rr": [5, 26, 27], "rrr": [5, 26, 27], "rudg": 18, "rug": [13, 27], "rule": [0, 1, 5, 6, 13, 25, 26, 27], "run": [0, 1, 2, 4, 5, 6, 8, 9, 11, 13, 15, 19, 25, 26, 27], "runtim": [1, 6, 14, 15], "rust": [0, 19, 20, 25], "rvert": 1, "rvert_2": 1, "s_": [3, 6], "s_1": 6, "s_i": [6, 7], "s_j": 6, "s_k": 6, "saddl": [13, 27], "safeguard": 18, "sai": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 20, 22, 25, 26, 27], "said": [6, 9, 13, 27], "sake": [0, 5, 7, 11, 25, 26, 27], "sale": [0, 25], "sam": 25, "same": [0, 1, 2, 3, 4, 5, 6, 8, 9, 11, 12, 14, 15, 16, 18, 20, 22, 25, 26, 27], "samm": 10, "sampl": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 13, 14, 18, 19, 20, 22, 25, 26], "sample_vari": 14, "sampleexptvari": 22, "samwis": 25, "sastri": 11, "satisfactori": [0, 25], "satisfi": [1, 2, 3, 6, 8, 13, 20, 22, 27], "satur": [1, 6], "save": [0, 4, 6, 7, 9, 13, 25], "save_fig": [0, 6, 7, 9, 10, 25], "savefig": [0, 4, 6, 7, 9, 22, 25], "savetxt": 4, "saw": [5, 26], "scalabl": 10, "scalar": [2, 5, 6, 10, 26], "scale": [0, 1, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 19, 20, 23, 25, 27], "scale_mean": 4, "scale_std": 4, "scaler": [0, 7, 8, 9, 10, 11, 17, 26], "scan": [5, 7], "scari": 5, "scatter": [0, 1, 6, 7, 8, 9, 14, 15, 17, 25, 26], "scenario": [6, 13, 27], "schedul": 13, "scheme": [1, 13, 27], "schrage": 22, "sch\u00f8yen": [6, 26], "scienc": [0, 1, 10, 12, 13, 19, 21, 22, 23, 24, 27], "scientif": [0, 19, 25], "scientist": [0, 25], "scikit": [3, 5, 6, 8, 9, 10, 13, 15, 16, 19, 20, 24], "scikit_learn": 0, "scikitlearn": 25, "scikitplot": [7, 10], "scipi": [0, 3, 5, 6, 13, 19, 20, 25, 26, 27], "scl": 6, "scm": 15, "score": [0, 1, 3, 6, 7, 9, 10, 11, 15, 16, 23, 25, 26], "scores_kfold": 6, "scratch": [1, 13, 16], "sdg": 13, "sdv4f4s2sb8": 27, "seaborn": [0, 1, 3, 6, 7, 25], "seamless": [0, 19, 25], "search": [0, 1, 3, 5, 9, 13, 15, 25, 27], "sebastian": 25, "sebastianraschka": 25, "sec": 6, "second": [0, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 14, 15, 16, 19, 20, 22, 23, 25, 26, 27], "secondeigvector": 11, "secondli": 12, "section": [4, 11, 16, 20, 22, 26], "sector": 0, "see": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 15, 16, 18, 19, 20, 22, 25, 26, 27], "seed": [0, 1, 2, 3, 4, 5, 6, 8, 9, 11, 13, 14, 18, 22, 25, 26, 27], "seed_imag": 4, "seek": [1, 2, 8], "seem": [1, 3, 4], "seemingli": [0, 25], "seen": [0, 1, 3, 5, 10, 12, 22], "segment": [13, 27], "seismic": 6, "seldomli": [0, 25], "select": [1, 5, 6, 8, 9, 10, 11, 15, 21, 22, 23, 24, 25, 26, 27], "selevet": 15, "self": [1, 5, 26], "sell": 4, "semest": [7, 21], "semi": [8, 13, 27], "semilogx": 6, "send": [5, 12, 13, 23, 25], "senior": [21, 23], "sens": [0, 4, 6, 8, 25], "sensibl": 3, "sensit": [0, 5, 6, 9, 13, 25, 26], "sent": 2, "sentenc": [4, 12], "separ": [0, 1, 2, 4, 6, 8, 9, 12, 14, 18, 19, 22, 25], "septemb": [18, 25], "sequenc": [3, 4, 7, 9, 10, 12, 13, 19, 20, 22, 25, 27], "sequenti": [1, 3, 4, 10, 12, 22], "seri": [0, 1, 2, 3, 4, 5, 6, 10, 11, 12, 13, 20, 25, 26, 27], "serif": [7, 22, 25], "serv": [0, 1, 2, 3, 5, 7, 13, 24, 25, 26, 27], "session": [1, 15, 21, 23, 25], "set": [1, 4, 5, 6, 7, 8, 10, 11, 13, 14, 16, 17, 18, 19, 20, 22, 23], "set_major_formatt": 6, "set_major_loc": 6, "set_tick": [1, 8], "set_ticklabel": 1, "set_titl": [0, 1, 2, 3, 7, 12, 14, 25], "set_xlabel": [0, 1, 2, 3, 7, 12, 25], "set_xlim": [7, 12], "set_xticklabel": 1, "set_ylabel": [0, 1, 2, 3, 7, 25], "set_ylim": [7, 12], "set_ytick": 7, "set_yticklabel": [1, 6], "set_zlim": 6, "seth": 4, "setminu": 6, "setosa": [8, 9], "setosa_or_versicolor": 8, "setp": 6, "setup": [1, 4, 6, 8, 19, 25, 26, 27], "sever": [0, 3, 5, 6, 7, 8, 9, 11, 12, 13, 16, 19, 20, 22, 25, 26, 27], "sgd": [1, 3, 27], "sgd_clf": 8, "sgdclassifi": 8, "sgdreg": 13, "sgdregressor": 13, "sgn": [5, 26, 27], "shallow": 13, "shape": [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 16, 18, 20, 25, 26, 27], "share": [1, 3, 15, 25], "shareabl": 15, "she": 7, "shift": [1, 6, 12, 15, 18, 22, 26], "ship": 3, "shire": 25, "short": [4, 5], "shortcom": [13, 27], "shorten": 4, "shorter": 22, "shorthand": 25, "shortli": [20, 25], "should": [0, 2, 3, 5, 6, 8, 9, 11, 12, 15, 18, 20, 22, 25, 26], "show": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 20, 22, 25, 26, 27], "show_shap": 4, "shown": [0, 4, 5, 8, 12, 13, 20, 26, 27], "shrink": [3, 5, 6, 8, 11, 26, 27], "shrinkag": [5, 6, 26, 27], "shrunk": 11, "shuffl": [0, 1, 4, 6, 13, 26], "side": [0, 2, 5, 8, 12, 13, 20, 25, 27], "sigh": [19, 25], "sigma": [0, 1, 5, 6, 7, 10, 11, 12, 13, 20, 22, 25, 26, 27], "sigma0": 22, "sigma1": 22, "sigma2": 22, "sigma_": [5, 20, 25, 26, 27], "sigma_0": [5, 26, 27], "sigma_1": [5, 26, 27], "sigma_2": [5, 26, 27], "sigma_fn": [7, 12], "sigma_i": [0, 5, 25, 26, 27], "sigma_j": [5, 26, 27], "sigma_m": [6, 22], "sigma_n": [11, 22], "sigma_t": 13, "sigma_x": 22, "sigmoid": [1, 2, 4, 7, 8, 10, 12], "sigmundson": [6, 26], "sign": [1, 2, 7, 8, 10, 22, 23], "signal": [1, 3, 10, 12], "signifi": 4, "signific": 1, "significantli": [1, 13, 18, 22, 27], "sim": [4, 5, 6, 13, 22], "similar": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 14, 19, 20, 25, 27], "similarli": [0, 1, 3, 5, 8, 10, 13, 22, 25, 26, 27], "simpl": [1, 2, 3, 5, 6, 7, 8, 10, 11, 12, 14, 16, 17, 19, 20, 22], "simplepredict": 10, "simpler": [0, 1, 5, 6, 7, 13, 16, 19, 25, 27], "simplernn": 4, "simplest": [0, 1, 3, 4, 9, 10, 12, 14, 25], "simpletre": 10, "simpli": [0, 1, 2, 4, 5, 6, 8, 9, 10, 11, 12, 19, 20, 22, 25, 26, 27], "simplic": [2, 5, 6, 7, 8, 9, 10, 11, 12, 14, 26, 27], "simplicti": [5, 26, 27], "simplifi": [0, 6, 9, 18, 19, 25, 26], "simplist": [3, 6, 22], "simul": [6, 18], "simultan": 6, "sin": [0, 1, 2, 3, 4, 9, 12, 13, 20, 25], "sinc": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 13, 16, 18, 20, 22, 24, 25, 26, 27], "sine": [3, 12], "singl": [0, 1, 2, 3, 5, 6, 7, 8, 9, 12, 13, 18, 20, 22, 25, 26, 27], "singular": [0, 6, 13, 20, 25], "sinusoid": 3, "site": [0, 21, 26], "situat": [0, 4, 5, 7, 13, 22, 25, 26, 27], "six": [3, 22], "size": [0, 1, 2, 3, 4, 5, 6, 8, 9, 10, 11, 13, 18, 20, 22, 25], "sketch": 10, "ski": 9, "skill": 0, "skip": 11, "skl": [0, 6, 25, 26], "sklearn": [0, 1, 3, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 17, 25, 26, 27], "skplt": [7, 10], "sl": [6, 26], "slack": 8, "slice": [2, 20, 25], "slide": [0, 3, 16, 22, 25, 26, 27], "slight": [6, 13], "slightli": [1, 2, 3, 5, 6, 7, 10, 22, 26, 27], "slope": [8, 11, 12], "slow": [0, 2, 8, 13, 18, 26, 27], "slower": [5, 20, 25, 26, 27], "slowest": 20, "slowli": 12, "slp": 1, "small": [0, 1, 2, 3, 5, 6, 8, 9, 10, 11, 12, 13, 19, 20, 22, 25, 26, 27], "smaller": [0, 1, 2, 5, 6, 8, 9, 11, 13, 22, 25, 26, 27], "smallest": [0, 4, 14, 25], "smallest_row_index": 14, "smooth": [0, 3, 6, 13, 25, 27], "sn": [0, 1, 3, 6, 7, 25], "sne": 11, "so": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 19, 20, 22, 23, 25, 26, 27], "soar": 6, "social": 0, "soft": [1, 7, 10, 12], "soften": 8, "softmax": [3, 7], "softwar": [0, 8, 19, 20], "sol": 8, "sole": [0, 6, 25], "solid": [0, 7], "solut": [0, 1, 2, 3, 5, 6, 8, 10, 11, 13, 18, 20, 22, 25, 26, 27], "soluton": 2, "solv": [0, 1, 3, 5, 6, 8, 10, 11, 12, 13, 16, 20, 25, 26], "solve_expdec": 2, "solve_ode_deep_neural_network": 2, "solve_ode_neural_network": 2, "solve_pde_deep_neural_network": 2, "solveod": 2, "solveode_popul": 2, "solver": [2, 7, 8, 9, 10, 20, 25], "some": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 15, 16, 18, 22, 25], "some_model": [6, 26], "somehow": 4, "someon": 16, "someth": [0, 1, 3, 4, 7, 9, 11, 15, 22, 25, 26], "sometim": [0, 1, 11, 12, 13, 14, 26], "soon": [20, 23, 26], "sophist": [0, 25], "sopt": 13, "sort": [5, 6, 9, 11, 22], "sound": [3, 5], "sourc": [0, 1, 3, 6, 19, 20, 22, 25], "space": [0, 1, 4, 5, 8, 9, 11, 12, 13, 14, 22, 26, 27], "span": [0, 3, 5, 9, 11, 20, 25, 26, 27], "spare": 1, "spars": [3, 6, 18, 20, 25], "sparse_mtx": [20, 25], "sparsecategoricalcrossentropi": 3, "sparsiti": [10, 18], "spatial": [1, 2, 3, 12], "speak": 22, "special": [6, 7, 10, 12, 13, 20, 22, 25, 26, 27], "specif": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 15, 16, 19, 20, 22, 24, 25, 26, 27], "specifi": [0, 3, 5, 6, 7, 9, 11, 13, 14, 22, 25, 27], "specifici": [0, 10, 25], "spectacular": 3, "spectral": 1, "speech": [0, 1, 3, 4, 12], "speed": [1, 2, 4, 13], "spend": [16, 22], "sphere": [0, 26], "spin": 6, "spite": 0, "spline": 8, "split": [1, 3, 4, 5, 6, 8, 9, 10, 11, 14, 16, 17, 22, 25, 27], "splite": 0, "splitter": [1, 10], "spontan": 22, "spot": 3, "spread": [0, 11, 22, 25, 26], "springer": [24, 25], "spuriou": 13, "sqquar": 27, "sqrsignal": 3, "sqrt": [3, 4, 5, 6, 8, 10, 11, 13, 22, 26, 27], "squar": [1, 2, 3, 4, 7, 8, 9, 11, 13, 14, 15, 17, 18, 19, 20, 22], "squarederror": 10, "squaredeuclidean": 14, "squash": 12, "srtm": 6, "srtm_data_norway_1": 6, "stabil": 5, "stabl": [0, 4, 5, 6, 9, 16, 19, 25, 26, 27], "stack": [3, 4], "stage": [5, 13, 15], "stai": [0, 2, 4, 5, 11, 25, 26], "stand": [0, 5, 9, 12, 25, 26, 27], "standard": [0, 1, 4, 5, 6, 7, 8, 10, 12, 17, 18, 20, 22, 25, 27], "standardscal": [0, 6, 7, 8, 9, 10, 11, 17, 26], "stanford": [13, 27], "start": [0, 1, 2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 13, 14, 15, 16, 20, 22, 23, 25, 26, 27], "start_tim": 14, "stat": 6, "state": [1, 2, 4, 5, 6, 7, 8, 10, 11, 12, 13, 19, 22, 25, 26, 27], "statement": [0, 7, 20, 25], "stationari": 27, "statist": [0, 1, 3, 4, 7, 9, 10, 11, 12, 13, 14, 20, 24, 26, 27], "statu": [0, 7, 15, 25], "stavang": 6, "std": [0, 4, 6, 18, 25, 26], "steep": [13, 27], "step": [0, 1, 2, 4, 6, 7, 9, 10, 11, 12, 13, 14, 15, 18, 20, 25, 27], "step_fn": [7, 12], "step_length": 13, "steps_list": 9, "stereo": 3, "still": [0, 2, 3, 5, 6, 11, 13, 22, 26, 27], "stimuli": 12, "stk": [24, 25], "stk2100": [24, 25], "stk3155": [15, 21, 23], "stk4021": [24, 25], "stk4051": [24, 25], "stk4155": [21, 23], "stk5000": 24, "stochast": [0, 1, 5, 6, 8, 11, 12, 27], "stock": 4, "stoke": 12, "stone": [0, 7], "stop": [1, 4, 9, 13, 14, 18, 27], "storag": [5, 26, 27], "store": [0, 1, 2, 3, 6, 11, 13, 18, 22, 25], "storehaug": [23, 25], "str": [1, 3, 4], "straight": [0, 6, 8, 13, 25, 27], "straightforward": [0, 2, 3, 5, 6, 8, 9, 10, 13, 20, 25, 26, 27], "strategi": [0, 1, 9, 25], "stratifi": 6, "strength": [0, 5, 14, 26, 27], "stretch": 11, "strict": [8, 13, 27], "strictli": [8, 13, 27], "stride": [4, 20], "strike": 6, "string": 1, "stroke": 7, "strong": [3, 6, 9, 10, 12, 20, 22], "strongli": [0, 8, 15, 19, 20], "stronli": [], "structur": [0, 1, 2, 3, 6, 9, 10, 12, 19, 25], "stuck": [1, 13, 27], "student": [0, 15, 21, 23, 24, 25], "studi": [0, 3, 4, 5, 6, 7, 8, 11, 12, 13, 19, 24, 25, 26, 27], "studier": 24, "style": [7, 9, 20, 25], "st\u00f8land": 23, "sub": [9, 12], "subdivid": [0, 20, 25], "subfield": 0, "subject": [6, 8, 22], "submit": 25, "subplot": [0, 1, 3, 4, 6, 7, 8, 9, 10, 14, 25], "subplots_adjust": [8, 22], "subprogram": [20, 25], "subract": [0, 26], "subroutin": [0, 25], "subscript": 1, "subsequ": [1, 4, 5, 6, 12, 20, 22, 26, 27], "subset": [1, 6, 9, 12, 13, 19, 25, 27], "subspac": [0, 8, 11, 26], "substanti": [9, 10], "substep": 11, "substitut": [3, 6, 12, 16, 20], "subsubset": 9, "subtask": 6, "subtl": 1, "subtract": [0, 4, 5, 6, 11, 13, 18, 20, 22, 26], "subtre": 9, "succeed": [0, 4, 25], "success": [3, 7, 9, 13, 22], "successfulli": [4, 9], "sudo": [0, 19, 25], "suffer": [0, 1, 2, 5, 10, 25, 26, 27], "suffici": [1, 6, 8, 11, 13, 27], "suggest": [1, 13, 24, 27], "suit": [8, 12], "suitabl": [0, 15, 22, 26], "sum": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 20, 22, 25, 26, 27], "sum_": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 20, 22, 25, 26, 27], "sum_i": [0, 2, 5, 6, 8, 13, 26, 27], "sum_j": [6, 18], "sum_ja_": 0, "sum_k": [6, 8, 12, 20], "sum_logist": 13, "sum_m": 3, "sum_n": 3, "sum_nx_": 3, "summar": [5, 6, 9], "summari": [1, 3, 4, 10, 21, 27], "summat": [0, 3, 16, 26, 27], "sunni": 9, "super": [5, 26, 27], "superfici": 3, "superscript": [1, 12], "supervis": [0, 5, 6, 7, 9, 12, 19, 25, 26, 27], "supplement": 7, "support": [0, 1, 9, 10, 11, 13, 19, 25, 26], "suppos": [0, 5, 6, 7, 8, 10, 11, 12, 13, 20, 25, 26, 27], "suppress": [5, 13, 27], "sure": [0, 1, 4, 6, 16], "surf": 6, "surfac": [0, 6, 25], "surpass": 6, "surpris": [0, 25], "surround": [3, 19], "survei": [0, 5, 6, 25, 26], "svc": [8, 9, 10], "svd": [0, 6, 11, 25], "svdinv": 5, "svm": [8, 9, 10, 11], "svm_clf": [8, 10], "swath": [5, 26, 27], "switch": 0, "sy": [13, 27], "symbol": [1, 5, 11, 13, 19, 22, 25, 26, 27], "symmeteri": 1, "symmetr": [0, 5, 8, 11, 12, 13, 20, 25, 26], "symmetri": 6, "sympi": [0, 19, 25], "synonim": 22, "syntax": 13, "system": [0, 1, 3, 4, 6, 7, 9, 10, 12, 13, 15, 19, 20, 25, 27], "systemat": [4, 6], "t": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 23, 25, 27], "t0": [3, 6, 13], "t1": [2, 13], "t2": 2, "t3": 2, "t_": 2, "t_0": [2, 9, 13], "t_1": 13, "t_b": 10, "t_i": [1, 2, 5, 12, 26, 27], "t_j": 12, "t_k": 9, "tabl": [9, 22, 23, 25], "tabul": [0, 25], "tabular": 25, "tackl": 4, "tag": [2, 3, 4, 5, 6, 7, 12, 13, 14, 20, 22, 26, 27], "taht": [0, 25], "tail": 22, "tailor": [2, 8, 11, 25], "taiwan": [0, 25], "take": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 17, 19, 20, 22, 25, 26, 27], "taken": [0, 1, 3, 6, 10, 13, 20], "tan": 3, "tangent": [1, 4, 12, 13, 27], "tanh": [1, 4, 7, 8, 12], "target": [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 15, 16, 18, 25, 26, 27], "target_nam": 9, "task": [0, 1, 3, 6, 9, 11, 12, 14, 25], "tau": [3, 5, 22], "taught": 25, "tax": [], "taylor": [2, 13, 27], "taylornr": [13, 27], "tc": 8, "teach": [15, 21, 25], "team": 1, "teaser": 0, "technic": [0, 5, 6, 13, 27], "techniqu": [0, 1, 8, 10, 13, 19, 22, 24, 25, 26], "technologi": [0, 1], "tell": [0, 4, 6, 10, 11, 13, 16, 22], "temp": 1, "temp1": 1, "temp2": 1, "temperatur": [0, 9, 25], "templat": 18, "temporarili": 1, "ten": [3, 25], "tend": [3, 5, 6, 8, 9, 10, 12, 13, 14, 26], "tendenc": [0, 25], "tension": 6, "tensor": 3, "tensorflow": [0, 2, 4, 8, 14, 19, 20, 24, 25, 26], "term": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 18, 22, 25, 26, 27], "term1": [5, 6, 11], "term2": [5, 6, 11], "term3": [5, 6, 11], "term4": [5, 6, 11], "termin": [0, 4, 5, 9, 10, 13, 15, 26, 27], "terminarl": 15, "terrain": 6, "terrain1": 6, "test": [3, 4, 5, 6, 7, 8, 9, 10, 13, 16, 20, 22, 25, 27], "test_acc": 3, "test_accuraci": [1, 3], "test_error": 6, "test_imag": [3, 4], "test_ind": 6, "test_input": 4, "test_label": [3, 4], "test_loss": 3, "test_pr": 1, "test_predict": 1, "test_rnn": 4, "test_scor": [7, 10], "test_siz": [0, 1, 3, 5, 6, 10, 15, 17, 26, 27], "test_split": 9, "testerror": [0, 6, 26], "testi": 4, "testpredict": 4, "testx": 4, "text": [0, 1, 2, 4, 5, 8, 9, 11, 13, 15, 18, 20, 22, 24, 26, 27], "textbook": [16, 26, 27], "textual": 9, "textur": 1, "tf": [1, 3, 4, 13, 14, 27], "th": [0, 1, 2, 5, 6, 7, 9, 12, 13, 14, 20, 22, 25, 26], "than": [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, 17, 19, 22, 25, 26], "thank": [4, 6, 26], "theano": [1, 19, 25], "thei": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 15, 16, 18, 20, 22, 25, 26, 27], "them": [0, 1, 3, 4, 6, 8, 9, 10, 11, 12, 13, 18, 20, 25, 26], "theme": [0, 15, 25], "themselv": [0, 22, 25], "thenc": 6, "theorem": [2, 6, 7, 26, 27], "theoret": [0, 4, 10], "theori": [0, 1, 3, 8, 9, 12, 13, 19, 24, 25], "thereaft": [0, 5, 6, 11, 12, 20, 25], "therebi": [0, 5, 7, 11, 25, 26, 27], "therefor": [0, 1, 2, 3, 4, 6, 7, 8, 11, 13, 22, 25, 26, 27], "therein": 11, "thereof": [0, 6, 13, 25], "theta": [0, 1, 4, 5, 6, 7, 13, 16, 22, 25, 26, 27], "theta_": [0, 1, 6, 7, 13, 25, 26, 27], "theta_0": [0, 5, 6, 7, 16, 25, 26, 27], "theta_0x_": [0, 25, 26], "theta_1": [0, 5, 6, 7, 25, 26, 27], "theta_1x_": [0, 25, 26], "theta_1x_0": [0, 25], "theta_1x_1": [0, 7, 25], "theta_1x_2": [0, 25], "theta_1x_i": [7, 26, 27], "theta_2": [0, 25, 26], "theta_2x_": [0, 25, 26], "theta_2x_0": [0, 25], "theta_2x_1": [0, 25], "theta_2x_2": [0, 7, 25], "theta_2x_i": 26, "theta_3x_i": 26, "theta_4x_i": 26, "theta_closed_form": 18, "theta_closed_formol": 18, "theta_closed_formridg": 18, "theta_gdol": 18, "theta_gdridg": 18, "theta_i": [0, 1, 5, 25, 26, 27], "theta_j": [0, 5, 6, 25, 26], "theta_k": 27, "theta_linreg": [13, 27], "theta_ol": 18, "theta_p": 7, "theta_px_p": 7, "theta_ridg": 18, "theta_t": 13, "theta_tru": 18, "thetavalu": 5, "thi": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 24, 26, 27], "thing": [0, 1, 2, 4, 5, 7, 9, 15, 16, 22, 25], "think": [0, 1, 3, 4, 6, 9, 12, 13, 14, 22, 25, 26, 27], "third": [0, 3, 6, 13, 23, 25, 27], "thirti": 7, "thorughout": 25, "those": [0, 3, 5, 6, 8, 9, 10, 11, 20, 25, 26, 27], "though": [1, 2, 3, 4, 13, 16, 17, 20, 22], "thought": [6, 14, 22], "thousand": [0, 1, 26], "three": [0, 1, 3, 5, 6, 8, 9, 12, 20, 21, 22, 23, 25, 26, 27], "threshold": [1, 3, 9, 10, 11, 12, 13], "through": [0, 1, 2, 3, 4, 5, 6, 8, 11, 12, 13, 14, 15, 19, 20, 22, 25, 26, 27], "throughout": [0, 4, 5, 14, 15, 19, 20, 22, 25], "throw": [3, 6, 22], "thu": [0, 1, 2, 5, 6, 7, 8, 10, 11, 12, 13, 23, 25, 26, 27], "thumb": [0, 6, 26], "thursdai": [], "tibshirani": [6, 24, 25], "tick_param": 6, "ticker": [6, 13, 22, 27], "tif": 6, "tight_layout": [1, 7], "tightli": 11, "tild": [0, 5, 6, 7, 11, 22, 25, 26, 27], "till": [0, 4, 7, 8, 9, 10, 12, 20, 25, 26], "time": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 25, 26, 27], "timeit": 4, "timer": 4, "tini": 1, "tip": 3, "titl": [0, 1, 2, 3, 4, 6, 7, 8, 9, 10, 13, 15, 22, 25, 27], "tmp": 13, "tn": [2, 3, 7], "to_categor": [1, 3, 4], "to_categorical_numpi": 1, "to_numer": [0, 6, 25], "todai": 3, "togeth": [0, 3, 6, 8, 11, 13, 19, 25], "toi": 14, "told": 13, "toler": [2, 14], "tolist": 4, "tomographi": 12, "too": [0, 2, 4, 5, 6, 9, 11, 13, 17, 18, 22, 24, 26, 27], "took": [8, 25], "tool": [0, 1, 3, 6, 13, 15, 19, 26], "toolbox": 8, "top": [0, 3, 5, 6, 9, 10, 19, 25], "topic": [0, 5, 6, 7, 8, 19, 26, 27], "topolog": [3, 12], "topologi": [1, 12], "torkjellsdatt": [23, 25], "toss": [10, 22], "total": [0, 1, 2, 3, 4, 6, 7, 8, 10, 11, 12, 13, 14, 20, 22, 23, 25, 26, 27], "total_loss": 4, "totalclustervari": 14, "totalscatt": 14, "toward": [1, 2, 7, 12, 13, 15, 27], "town": [], "tp": [4, 7], "tpng": 9, "tpu": [13, 19, 25], "tqdm": 6, "tr": [], "track": [3, 13, 14, 15, 20, 26, 27], "tract": [], "tractabl": [0, 25, 26], "trade": [5, 9], "tradeoff": [0, 5, 25, 26, 27], "tradit": [0, 1, 4, 6, 25], "train": [2, 3, 5, 6, 8, 9, 10, 11, 12, 13, 16, 17, 27], "train_accuraci": [0, 1, 3, 25], "train_dataset": 4, "train_end": [0, 1, 26], "train_error": 6, "train_imag": [3, 4], "train_ind": 6, "train_label": [3, 4], "train_pr": 1, "train_siz": [0, 1, 3, 26], "train_step": 4, "train_test_split": [0, 1, 3, 5, 6, 7, 9, 10, 11, 15, 16, 17, 25, 26, 27], "train_test_split_numpi": [0, 1, 26], "trainable_vari": 4, "trained_model": [6, 26], "trainerror": [0, 26], "traini": 4, "training_checkpoint": 4, "training_dataset": 4, "training_gradi": 13, "trainingerror": 6, "trainpredict": 4, "trainscor": 4, "trainx": 4, "trait": [0, 25], "trajectori": 4, "transfer": [9, 25], "transform": [0, 5, 6, 7, 8, 9, 10, 11, 12, 13, 17, 19, 20, 25, 26, 27], "transit": [6, 12], "translat": [1, 4, 6, 10, 25, 26], "transpos": [1, 5, 11, 20, 26, 27], "travers": [0, 5], "treat": [0, 1, 3, 6, 12, 13, 18, 22, 25, 26, 27], "tree": [0, 1, 19, 25], "tree_clf": [9, 10], "tree_clf_": 9, "tree_clf_sr": 9, "tree_reg": 9, "tree_reg1": 9, "tree_reg2": 9, "trend": 22, "treue": 7, "trevor": 24, "tri": [2, 3, 4, 9, 13, 16], "triain": 0, "trial": [0, 2, 4, 6, 13, 22, 25, 27], "triangl": [13, 27], "triangular": 20, "trick": [3, 4, 8, 11, 13, 22], "trickier": 22, "tridiagon": 20, "trillion": 19, "trivial": [0, 1, 5, 11, 22, 25, 27], "troubl": [0, 8, 12, 15, 26], "truck": 3, "true": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 14, 16, 17, 18, 20, 22, 25, 26, 27], "true_beta": 26, "true_fun": 6, "true_theta": 6, "truli": 25, "try": [0, 1, 2, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 19, 20, 22, 25, 26, 27], "tucker": 8, "tuesdai": [23, 25], "tumor": [7, 9], "tumour": 7, "tunabl": 1, "tune": [4, 9, 13, 20, 25], "turn": [0, 1, 5, 6, 7, 8, 9, 10, 11, 12, 13, 20, 22, 25, 26, 27], "tutori": [1, 4], "tv": 2, "tveito": 2, "tweak": [1, 4, 10, 22], "twice": [13, 27], "twist": 11, "two": [0, 1, 2, 4, 5, 6, 7, 9, 10, 11, 12, 13, 15, 17, 20, 21, 22, 24, 25, 26, 27], "tx": [13, 27], "tx_1": [13, 27], "txt": [4, 15], "ty": [13, 27], "type": [0, 1, 3, 6, 8, 10, 13, 20, 22, 26, 27], "typic": [0, 1, 2, 3, 4, 5, 7, 9, 10, 12, 13, 15, 16, 22, 25, 26, 27], "u": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 20, 22, 24, 25, 26, 27], "u_": 20, "u_i": 12, "u_m": 10, "ua": [0, 25], "ubuntu": [0, 19, 25], "uci": [], "uio": [15, 23, 24], "un": 14, "unabl": 15, "unari": [20, 25], "unbalanc": [6, 9], "unbias": [0, 5, 6, 25], "uncent": [6, 26], "uncertainti": [0, 5, 25], "uncertitud": 22, "unchang": [1, 3], "uncorrel": [10, 22], "undefin": [5, 26, 27], "under": [0, 1, 5, 6, 10, 13, 19, 25, 26, 27], "underdetermin": [0, 25], "underfit": [1, 6], "underflowproblem": 5, "undergo": 5, "undergradu": [21, 23], "underli": [0, 1, 9, 13, 18, 22, 25], "underset": [4, 14], "understand": [0, 1, 3, 5, 6, 10, 13, 14, 15, 19, 25, 26, 27], "understood": [8, 13], "undesir": 8, "undetermin": [5, 8], "undo": 4, "unexpect": 6, "unexpected": 22, "unexplain": 18, "unfair": [6, 26], "unfortun": [1, 8, 9, 10], "unicode_liter": [8, 9], "uniform": [0, 1, 5, 6, 11, 13, 22, 25, 27], "uniformli": [13, 22, 27], "unifrompdf": 22, "unimport": [13, 27], "union": [5, 6], "uniqu": [0, 2, 6, 13, 14, 20, 25], "unique_cluster_label": 14, "unit": [0, 1, 3, 4, 5, 10, 12, 18, 22, 25, 26, 27], "unitari": [5, 6, 20, 26, 27], "unitarili": [20, 25], "uniti": 22, "univari": 22, "univers": [0, 1, 2, 13, 19, 21, 23, 25, 26, 27], "unix": 1, "unknow": [0, 20, 25], "unknown": [0, 1, 3, 4, 5, 6, 8, 10, 13, 20, 25, 26, 27], "unknowwn": 12, "unlabel": 1, "unless": [0, 3, 6, 11, 13, 25, 27], "unlik": [1, 3, 8, 13, 27], "unnecessarili": 9, "unord": 3, "unravel": 1, "unrol": [3, 11], "unseen": [0, 7, 9, 15], "unstabl": 1, "unsupervis": [0, 1, 4, 12, 19, 25], "unsymmetr": [20, 25], "until": [1, 2, 4, 9, 12, 13, 14, 27], "untouch": 0, "unusu": 12, "up": [1, 3, 4, 5, 6, 8, 10, 11, 13, 14, 16, 18, 19, 20, 22, 23], "updat": [1, 2, 10, 12, 13, 14, 15, 18], "uploa": 25, "upload": [15, 19, 24], "upon": [0, 1, 6, 7, 11, 20], "upper": [0, 8, 9, 16, 20, 26], "uppercas": [20, 25], "upsampl": 4, "upscal": 4, "url": [25, 26], "us": [4, 5, 6, 8, 9, 10, 11, 12, 14, 15, 17, 20, 22, 24], "usag": [0, 8, 19, 25, 26], "usd": [], "usd10000": [], "use_bia": 4, "usecol": [0, 25], "useless": 1, "user": [0, 1, 2, 4, 6, 7, 15, 19, 20, 25, 26], "usernam": 15, "usetex": 22, "usg": 6, "usr": 22, "usual": [0, 3, 4, 7, 12, 13, 14, 25], "ut": 5, "util": [1, 3, 4, 6, 7, 10, 14, 25], "ux": 20, "v": [2, 4, 5, 6, 11, 13, 15, 19, 26, 27], "v0": 22, "v1": 22, "v2": 22, "v_0": 11, "va": 1, "vahid": 25, "val": 13, "val_accuraci": 3, "val_loss": 4, "vale": 2, "valid": [0, 1, 4, 7, 9, 10, 13, 19, 22, 25, 26], "validation_data": 3, "validation_split": 4, "valu": [0, 1, 2, 3, 4, 6, 7, 8, 9, 10, 12, 13, 14, 16, 17, 18, 19, 20, 25], "valuat": 9, "valy": 4, "van": [0, 25, 26, 27], "vandenbergh": [8, 13, 27], "vandermond": [0, 25], "vanilla": [0, 6, 11, 14, 26], "vanish": [1, 4, 13, 22, 27], "var": [5, 6, 10, 11, 22, 26], "var_x": 22, "varabl": 8, "varepsilon": [5, 6], "varepsilon_": [5, 6], "varepsilon_i": [5, 6], "vari": [0, 1, 3, 5, 6, 10, 25], "variabl": [0, 1, 2, 5, 6, 7, 8, 10, 11, 12, 13, 14, 20, 25, 26], "varianc": [0, 1, 5, 7, 9, 10, 11, 13, 14, 18, 19, 20, 22, 25, 26, 27], "variance_i": [5, 11, 26], "variance_x": [5, 11, 26], "variant": [0, 1, 6, 8, 12, 13, 25, 26, 27], "variat": [3, 4, 11, 25], "varieti": [0, 3, 12, 19, 25], "variou": [1, 3, 5, 6, 7, 8, 9, 11, 12, 13, 16, 19, 20, 22, 25, 26, 27], "varydimens": 4, "vastli": 3, "vaue": 1, "vault": 0, "vdot": [2, 13, 27], "vec": 6, "vector": [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 13, 14, 17, 18, 19, 27], "vector_mean": 14, "ventur": [0, 8, 19, 25], "venv": 15, "verbos": [1, 3, 4], "veri": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 22, 24, 25, 26, 27], "verifi": [3, 11, 20, 25], "versatil": [8, 25], "versicolor": [8, 9], "version": [0, 3, 10, 13, 14, 15, 19, 20, 22, 25], "versu": 1, "vert": [0, 1, 5, 6, 7, 8, 9, 11, 13, 16, 17, 25, 26, 27], "vert_1": [5, 6, 26, 27], "vert_2": [5, 6, 11, 17, 26, 27], "via": [0, 5, 6, 7, 8, 9, 10, 11, 12, 19, 20, 21, 22, 23, 25, 26, 27], "vidal": 11, "video": [0, 1, 12, 19, 21, 23, 25, 26, 27], "view": [1, 3, 5, 6, 12, 13, 22, 24, 25, 27], "violat": 8, "virginica": 9, "viridi": [0, 1, 2, 3, 25], "virtual": 1, "viscos": 13, "viscou": 13, "visibl": 15, "vision": [0, 3], "visual": [0, 3, 11, 12, 18, 19, 25, 26], "visualis": 1, "visualstudio": [15, 16], "viz": [6, 8, 22], "vmap": 13, "vmax": [1, 6], "vmin": [1, 6], "voic": 3, "volum": [0, 3, 25], "vote": [10, 25], "voting_clf": 10, "votingclassifi": 10, "votingsimpl": 10, "vstack": [5, 11, 20, 22, 25, 26], "vt": [5, 26, 27], "w": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 20, 22, 25, 26, 27], "w1": 8, "w2": [8, 11], "w3": 8, "w_": [1, 12], "w_1": [8, 20], "w_1x_": 8, "w_1x_1": 8, "w_2": [8, 20], "w_2x_": 8, "w_2x_2": 8, "w_3": 20, "w_4": 20, "w_hidden": 2, "w_i": [1, 2, 10], "w_ix_i": 12, "w_j": 20, "w_m": 20, "w_output": 2, "w_px_": 8, "w_px_p": 8, "wa": [1, 3, 4, 5, 6, 7, 10, 11, 12, 14, 17, 20, 25, 26], "wai": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 15, 18, 20, 22, 25, 26, 27], "walk": 9, "walker": 22, "wang": [0, 25], "want": [0, 1, 2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 19, 22, 25, 26, 27], "warn": 4, "warrant": 6, "wast": 3, "watch": [19, 27], "wave": 3, "wavelet": 8, "we": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 23, 24, 26, 27], "weak": [9, 10, 14], "weather": [1, 12], "web": [19, 21, 23, 25], "webpag": 25, "websit": [6, 20, 21, 25], "wedg": [8, 22], "wednesdai": [23, 25], "wee": 11, "week": [0, 5, 6, 7, 21, 23], "weekli": [15, 16, 19, 21, 23, 24, 25], "weight": [1, 2, 3, 6, 7, 9, 10, 12, 13, 18, 22], "weigth": 2, "welcom": [8, 15, 19], "well": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 15, 16, 19, 20, 22, 24, 25, 26, 27], "went": 8, "were": [0, 1, 3, 4, 5, 6, 7, 8, 10, 11, 12, 14, 22, 25], "wessel": [0, 25, 26, 27], "what": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 19, 20, 22], "whatev": 3, "when": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 22, 25, 26, 27], "whenev": [13, 15, 22], "where": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 23, 25, 26, 27], "wherea": [6, 22], "wherein": [1, 12], "whether": [0, 3, 5, 7, 9, 22, 25], "which": [0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 25, 26, 27], "whichev": [1, 3], "while": [0, 1, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 15, 16, 22, 25, 26, 27], "white": 9, "whiteboard": 26, "who": [0, 15], "whole": [1, 3, 4, 5, 9, 11, 13], "whose": [0, 6, 10, 22, 26], "whow": [11, 26], "why": [0, 1, 3, 6, 13, 15, 16, 17, 26, 27], "wide": [0, 1, 3, 6, 7, 12, 19, 20, 25], "widehat": 6, "width": [0, 3, 8, 9, 25], "wieringen": [0, 25, 26, 27], "win": 10, "wind": 9, "wing": [23, 25], "winther": 2, "wiothout": 6, "wiscons": 7, "wisconsin": 10, "wisdom": [6, 26], "wise": [1, 5, 12, 13, 26, 27], "wish": [0, 2, 5, 7, 8, 11, 13, 14, 20, 25, 26, 27], "with_std": [0, 26], "wither": 6, "within": [0, 2, 3, 4, 7, 9, 12, 13, 14, 22, 24, 25, 27], "withinclust": 14, "without": [0, 1, 5, 6, 8, 9, 11, 12, 13, 15, 25, 26, 27], "won": [0, 15, 18, 25], "wonder": 8, "word": [0, 1, 3, 4, 5, 6, 7, 14, 22, 25, 26, 27], "work": [0, 1, 4, 6, 7, 8, 9, 13, 15, 16, 18, 19, 21, 22, 23, 25, 26], "workshop": 25, "world": [0, 8, 16, 26], "worldwid": [0, 25], "worri": 15, "wors": [0, 1, 3, 4, 6, 25], "worth": 9, "would": [0, 1, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 16, 20, 22, 25, 26, 27], "wrap": [6, 20, 25], "write": [0, 1, 2, 3, 5, 6, 7, 8, 12, 13, 15, 16, 20, 25, 26], "written": [0, 2, 3, 5, 11, 12, 13, 16, 19, 20, 22, 25, 26, 27], "wrong": [1, 8, 15], "wrongli": 10, "wrote": [5, 11, 26], "wrt": [10, 13], "wth": [10, 13], "www": [19, 20, 24, 25, 27], "wx_1": 8, "x": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 22, 25, 27], "x0": 8, "x1": [4, 8, 9, 10, 13], "x1_exampl": 8, "x1d": 8, "x2": [8, 9, 10, 13], "x2d": [8, 11], "x2d_train": 11, "x2dsl": 11, "x3": 8, "x_": [0, 2, 3, 5, 6, 8, 10, 11, 13, 14, 20, 22, 25, 26, 27], "x_0": [0, 5, 11, 18, 20, 25, 26], "x_1": [0, 2, 5, 6, 7, 8, 9, 10, 11, 13, 18, 20, 22, 25, 26, 27], "x_2": [0, 2, 5, 6, 7, 8, 9, 10, 11, 13, 20, 22, 25, 26, 27], "x_3": [8, 20, 22], "x_4": 20, "x_6": 18, "x_center": 11, "x_data": 1, "x_data_ful": 1, "x_hidden": 2, "x_i": [0, 1, 2, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 20, 22, 25, 26, 27], "x_input": 2, "x_ix_": [0, 25], "x_iy_i": 8, "x_j": [0, 2, 8, 9, 12, 16, 22, 26], "x_jy_j": 8, "x_k": [12, 14, 20, 22, 26], "x_l": 22, "x_m": [6, 12, 20, 22], "x_mean": 18, "x_n": [0, 2, 3, 6, 8, 11, 12, 13, 20, 22, 25, 27], "x_new": [9, 10], "x_norm": 18, "x_offset": [6, 26], "x_output": 2, "x_p": [3, 7, 9], "x_poli": 9, "x_poly10": 9, "x_pred": 4, "x_prev": 2, "x_reduc": 11, "x_scale": 8, "x_small": 13, "x_std": 18, "x_test": [0, 1, 3, 5, 6, 7, 9, 10, 11, 15, 16, 17, 26, 27], "x_test_": 17, "x_test_own": 6, "x_test_scal": [0, 6, 7, 9, 10, 11, 26], "x_tot": 4, "x_train": [0, 1, 3, 4, 5, 6, 7, 9, 10, 11, 15, 16, 17, 25, 26, 27], "x_train_": 17, "x_train_mean": [6, 26], "x_train_own": 6, "x_train_scal": [0, 6, 7, 9, 10, 11, 26], "x_val": 1, "xarrai": [19, 25], "xavier": 1, "xbnew": [13, 27], "xcode": [0, 19, 25], "xdclassiffierconfus": 10, "xdclassiffierroc": 10, "xg_clf": 10, "xgb": 10, "xgbclassifi": 10, "xgboost": 9, "xgboot": 10, "xgbregressor": 10, "xgparam": 10, "xgtree": 10, "xi": [8, 13], "xi_": 8, "xi_1": 8, "xi_i": 8, "xk": 8, "xla": [13, 19, 25], "xlabel": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 13, 22, 25, 26, 27], "xlim": [6, 10], "xm": 9, "xmesh": 13, "xnew": [0, 13, 25, 27], "xp": 22, "xpanda": [0, 26], "xpd": [5, 11, 26], "xplot": 0, "xscale": [0, 26], "xsr": 9, "xt_x": [13, 27], "xtest": 6, "xtick": [3, 6, 8, 9], "xtrain": 6, "xu": [0, 25], "xx": [0, 20, 25], "xy": [0, 6, 8, 20, 25], "xytext": 8, "xz": [20, 25], "y": [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 22, 25, 26, 27], "y1": 4, "y2": 4, "y3": 4, "y_": [0, 1, 5, 6, 10, 11, 20, 25, 26], "y_0": [0, 5, 11, 20, 25, 26], "y_1": [0, 5, 8, 9, 11, 13, 20, 25, 26, 27], "y_1y_1": 8, "y_1y_1k": 8, "y_1y_2": 8, "y_1y_2k": 8, "y_1y_n": 8, "y_1y_nk": 8, "y_2": [0, 5, 8, 9, 11, 20, 25, 26], "y_2y_1": 8, "y_2y_1k": 8, "y_2y_2": 8, "y_2y_2k": 8, "y_3": [0, 9, 20], "y_4": 20, "y_center": 18, "y_data": [0, 1, 5, 6, 25, 26, 27], "y_data_ful": 1, "y_decis": 8, "y_fit": [0, 26], "y_i": [0, 1, 5, 6, 7, 8, 9, 10, 11, 12, 13, 20, 25, 26, 27], "y_if_": 10, "y_ix_": [0, 25], "y_ix_i": [7, 8, 13, 26, 27], "y_iy_jk": 8, "y_j": [6, 8, 12], "y_k": 12, "y_m": 20, "y_mean": 18, "y_model": [0, 4, 5, 6, 25, 26, 27], "y_n": [8, 13, 27], "y_ny_1": 8, "y_ny_1k": 8, "y_ny_2": 8, "y_ny_2k": 8, "y_ny_n": 8, "y_ny_nk": 8, "y_offset": [6, 17, 26], "y_plot": 9, "y_pred": [0, 1, 4, 6, 7, 8, 9, 10, 26], "y_pred1": 9, "y_pred2": 9, "y_pred_rf": 10, "y_pred_tre": 10, "y_proba": [7, 10], "y_scaler": [6, 26], "y_test": [0, 1, 3, 4, 5, 6, 7, 9, 10, 11, 15, 16, 17, 26, 27], "y_test_onehot": 1, "y_test_predict": [], "y_tot": 4, "y_train": [0, 1, 3, 4, 5, 6, 7, 9, 10, 11, 15, 16, 17, 25, 26, 27], "y_train_mean": [6, 26], "y_train_onehot": 1, "y_train_predict": [], "y_train_scal": [6, 26], "y_val": 1, "ye": [3, 6, 7], "year": [0, 19, 25], "yet": [0, 1, 6, 8, 11, 13, 25], "yi": 13, "yield": [0, 2, 5, 6, 8, 10, 12, 13, 14, 20, 22, 25, 27], "yk": 8, "ylabel": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 13, 22, 25, 26, 27], "ylim": [3, 6], "ym": 9, "ymesh": 13, "yn": 0, "yo": [8, 9, 10], "yoshua": [1, 24], "you": [0, 1, 3, 4, 5, 6, 8, 9, 10, 11, 13, 15, 16, 17, 18, 19, 20, 22, 23, 24, 25, 26, 27], "young": 0, "your": [1, 2, 4, 5, 6, 8, 11, 13, 15, 17, 19, 20, 25, 27], "your_model_object": 16, "yourself": [11, 13, 25, 27], "youtu": 26, "youtub": [19, 27], "ypred": 6, "ypredict": [0, 13, 25, 26, 27], "ypredict2": [13, 27], "ypredictlasso": [5, 27], "ypredictol": [0, 5, 27], "ypredictown": [6, 26], "ypredictownridg": [6, 26, 27], "ypredictridg": [0, 5, 6, 26, 27], "ypredictskl": [6, 26], "ytest": 6, "ytick": [3, 6, 8, 9], "ytild": [0, 6, 25, 26], "ytildelasso": [5, 27], "ytildenp": [0, 25, 26], "ytildeol": [0, 5, 27], "ytildeownridg": [6, 26, 27], "ytilderidg": [5, 6, 26, 27], "ytrain": 6, "yuxi": 25, "yx": [20, 25], "yy": [20, 25], "yz": [20, 25], "z": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 20, 22, 25, 26], "z_": [1, 2, 12, 20, 25], "z_0": [20, 25], "z_1": [20, 25], "z_2": [20, 25], "z_c": 1, "z_h": 1, "z_hidden": 2, "z_i": [1, 12], "z_j": [1, 12], "z_k": [12, 26], "z_m": 1, "z_mod": 9, "z_o": 1, "z_output": 2, "zaman": 22, "zaxi": 6, "zero": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 18, 20, 22, 25, 26, 27], "zeros_lik": 4, "zeroth": 26, "zfill": 4, "zip": [4, 6], "zm_h": [0, 25], "zn": [], "zone": [], "zoom": 25, "zx": [20, 25], "zy": [20, 25], "zz": [20, 25], "\u00f8yvind": [6, 26]}, "titles": ["3. Linear Regression", "14. Building a Feed Forward Neural Network", "15. Solving Differential Equations with Deep Learning", "16. Convolutional Neural Networks", "17. Recurrent neural networks: Overarching view", "4. Ridge and Lasso Regression", "5. Resampling Methods", "6. Logistic Regression", "8. Support Vector Machines, overarching aims", "9. Decision trees, overarching aims", "10. Ensemble Methods: From a Single Tree to Many Trees and Extreme Boosting, Meet the Jungle of Methods", "11. Basic ideas of the Principal Component Analysis (PCA)", "13. Neural networks", "7. Optimization, the central part of any Machine Learning algortithm", "12. Clustering and Unsupervised Learning", "Exercises week 34", "Exercises week 35", "Exercises week 36", "Exercises week 36", "Applied Data Analysis and Machine Learning", "2. Linear Algebra, Handling of Arrays and more Python Features", "Course setting", "1. Elements of Probability Theory and Statistical Data Analysis", "Teachers and Grading", "Textbooks", "Week 34: Introduction to the course, Logistics and Practicalities", "Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression", "Week 36: Linear Regression and Gradient descent"], "titleterms": {"": [8, 10, 27], "1": [0, 15, 16, 17, 18, 26], "1a": 18, "2": [0, 15, 16, 17, 18, 25, 26, 27], "2023": 23, "2a": 18, "2b": 18, "3": [0, 15, 16, 17, 26], "34": [15, 25], "35": [16, 26], "36": [17, 18, 27], "3a": 18, "3b": 18, "4": [0, 15, 16, 17, 26], "5": [0, 16], "A": [0, 1, 4, 8, 9, 25], "And": [25, 26], "In": 23, "Ising": 6, "The": [0, 1, 2, 3, 5, 6, 7, 8, 9, 11, 12, 15, 19, 25, 26, 27], "To": 25, "With": [4, 27], "about": [25, 26, 27], "abov": 27, "activ": [1, 12], "ad": [0, 6, 25, 26], "adaboost": 10, "adagrad": 13, "adam": 13, "adapt": 10, "adjust": 1, "adversari": 4, "again": [3, 9], "ai": 25, "aim": [8, 9, 25], "aka": 25, "algebra": [20, 25], "algorithm": [9, 10, 11, 12, 25, 26, 27], "algortithm": [13, 27], "all": 8, "an": [0, 4, 10, 15, 25], "analys": [5, 26, 27], "analysi": [0, 5, 6, 11, 19, 22, 25, 26, 27], "analyt": [0, 16, 18], "ani": [13, 27], "anoth": [9, 27], "appli": 19, "approach": [0, 8, 14, 25], "approxim": 12, "architectur": 1, "arrai": [20, 25], "assist": 23, "autocorrel": 22, "autograd": [2, 13], "automat": 13, "back": [1, 11, 12, 26, 27], "background": 19, "bag": 10, "base": 13, "basic": [0, 5, 7, 9, 10, 11, 20, 26, 27], "batch": 1, "bay": 5, "befor": 11, "better": 8, "bia": 6, "binari": 1, "bind": 25, "bird": 10, "boldsymbol": [18, 26], "boost": 10, "bootstrap": [6, 10], "boston": [], "breast": 1, "brief": 25, "bring": 12, "build": [1, 3, 9], "c": 25, "calcul": [26, 27], "can": 25, "cancer": [1, 7, 9, 11], "cart": 9, "case": [8, 10, 22, 26, 27], "central": [13, 19, 22, 27], "chain": 12, "chang": 10, "channel": 25, "chi": [0, 25], "choic": 17, "choos": 1, "cifar01": 3, "classic": 11, "classif": [1, 9, 10], "classifi": 8, "clip": 1, "cluster": 14, "cnn": 3, "code": [1, 2, 5, 9, 11, 12, 13, 14, 15, 16, 25, 26, 27], "collect": [1, 3], "commun": 25, "compar": [2, 10, 16], "comparison": 27, "complet": 26, "complex": [0, 6, 26], "complic": 6, "compon": 11, "comput": 9, "computerlab": 25, "con": 9, "concept": 22, "condit": 27, "conjug": 13, "contn": 25, "convex": [8, 13, 27], "convolut": [3, 12], "correl": [11, 26], "correspond": [], "cost": [1, 10, 26, 27], "cours": [19, 21, 24, 25], "covari": [5, 11, 22, 26], "cover": 25, "creat": 16, "cross": 6, "cython": 25, "data": [0, 1, 3, 6, 7, 9, 11, 15, 17, 18, 19, 22, 25, 26], "dataset": [1, 3, 18], "david": 25, "deadlin": 25, "deadllin": 23, "decai": 2, "decis": [9, 10], "decomposit": [5, 11, 20, 26, 27], "deeep": [], "deep": [1, 2, 25], "defin": [1, 25], "degre": [0, 17, 26], "deliver": [15, 16], "dens": 0, "deriv": [5, 12, 16, 17, 26, 27], "descent": [2, 10, 13, 18, 27], "design": 26, "detail": [3, 25], "develop": 1, "diagon": 11, "differ": 8, "differenti": [2, 13], "diffus": 2, "dimension": [2, 3, 8], "disadvantag": 9, "discret": 22, "discrimin": 25, "distribut": [5, 22], "do": 1, "doe": [26, 27], "domain": 22, "down": 1, "dropout": 1, "economi": [26, 27], "element": [0, 22, 25], "elimin": 20, "energi": 25, "ensembl": 10, "entropi": 9, "environ": [0, 15], "equat": [0, 2, 12, 26, 27], "error": [0, 10, 25, 26, 27], "essenti": 25, "etc": 25, "euler": 2, "evalu": 1, "exampl": [1, 2, 3, 4, 6, 7, 8, 9, 10, 25, 26, 27], "exercis": [0, 6, 15, 16, 17, 18, 26], "expect": 22, "experi": 22, "explor": 0, "exponenti": 2, "express": [16, 17, 26], "extend": 27, "extrapol": 4, "extrem": [10, 25], "ey": 10, "fall": 23, "famili": [1, 25], "famou": 20, "fantast": [26, 27], "featur": [9, 16, 20, 26], "feed": [1, 12], "final": [12, 26], "find": [16, 18], "fine": 1, "first": [4, 12, 25, 27], "fit": [0, 10, 15, 16, 25, 27], "fix": [26, 27], "forc": 3, "forest": 10, "form": 18, "format": 25, "formula": 18, "forward": [1, 2, 12], "foster": 25, "fourier": 3, "frank": 6, "freedom": [0, 17, 26], "frequent": 26, "frequentist": [0, 25], "from": [5, 10, 12, 25, 26, 27], "full": 2, "function": [0, 1, 6, 7, 8, 10, 11, 12, 13, 22, 25, 26, 27], "further": [3, 5, 26, 27], "gan": 4, "gaussian": 20, "gd": 13, "gener": [4, 9, 25], "geometr": [11, 27], "gini": 9, "github": 15, "goal": [15, 16, 17, 18], "good": [0, 25], "grade": [23, 25], "gradient": [1, 2, 10, 13, 18, 27], "growth": 2, "ha": 19, "handl": [20, 25], "hessian": [26, 27], "hidden": 2, "hous": [], "how": 16, "hyperparamet": [1, 17], "hyperplan": 8, "i": [0, 1, 25], "id3": 9, "idea": 11, "ideal": 27, "ii": 25, "illustr": 27, "implement": [1, 16, 17, 18], "implic": [5, 26, 27], "import": [5, 20, 25, 26, 27], "improv": 1, "includ": 13, "increment": 11, "index": 9, "inform": 23, "input": 2, "instal": [19, 25], "instructor": 23, "interpret": [5, 11, 25, 26, 27], "introduc": [11, 13, 26], "introduct": [0, 6, 19, 20, 25], "invers": [5, 20], "invert": [26, 27], "iter": 10, "its": 26, "jacobian": 26, "jax": 13, "julia": 25, "jungl": 10, "kera": [1, 3], "kernel": [8, 11], "lab": 27, "lagrangian": 8, "lasso": [5, 6, 26, 27], "last": 26, "later": [5, 26, 27], "layer": [1, 2, 3, 12], "learn": [0, 1, 2, 11, 13, 14, 15, 16, 17, 18, 19, 25, 26, 27], "least": [5, 6, 16, 25, 26, 27], "lectur": [25, 27], "level": 10, "librari": [19, 25], "likelihood": 7, "limit": [1, 13, 22, 27], "linear": [0, 8, 13, 15, 20, 25, 26, 27], "link": [5, 11, 24, 26], "logist": [7, 25], "loss": [26, 27], "lu": 20, "machin": [0, 8, 13, 19, 25, 27], "main": [22, 25], "make": [0, 9, 10, 26], "mani": [10, 12], "mass": 25, "materi": [25, 26, 27], "math": [5, 26, 27], "mathemat": [3, 5, 8, 26, 27], "matric": [5, 20, 25], "matrix": [1, 5, 11, 12, 16, 20, 25, 26, 27], "matter": 0, "max": 26, "mean": [0, 26, 27], "meet": [5, 10, 22, 25, 26], "mercer": 8, "method": [6, 9, 10, 13, 25, 27], "min": 26, "minim": 25, "ml": 25, "mlp": 12, "mnist": [3, 4], "model": [0, 1, 4, 6, 12, 15, 17, 25], "momentum": 13, "mondai": 27, "moon": [8, 9], "more": [3, 6, 20, 25, 26, 27], "multilay": 12, "multipl": [1, 3, 17], "multipli": 8, "need": 25, "network": [1, 2, 3, 4, 7, 12, 25], "neural": [1, 2, 3, 4, 7, 12, 25], "new": [4, 18], "newton": 27, "non": 8, "normal": [0, 1], "notat": 12, "note": [26, 27], "now": [1, 9, 13, 27], "nuclear": [0, 25], "numba": 25, "number": [0, 2, 22, 26], "numer": [2, 22], "numpi": [20, 25], "object": 3, "obtain": 11, "od": 2, "off": 6, "ol": [5, 6, 15, 16, 18, 27], "one": [2, 12, 27], "oper": 20, "optim": [1, 8, 13, 18, 19, 25, 26, 27], "order": 13, "ordinari": [5, 6, 16, 25, 26, 27], "organ": [0, 25], "oslo": 24, "other": [4, 9, 11, 12, 20, 25], "our": [0, 4, 5, 11, 13, 25, 26, 27], "outcom": [19, 25], "output": 2, "overarch": [0, 4, 8, 9, 25, 26], "overview": [10, 25], "own": [0, 10, 11, 25, 26], "packag": [20, 25], "panda": [25, 26], "paramet": [25, 26], "paramt": 18, "part": [13, 19, 27], "partial": 2, "pass": 1, "pca": 11, "pdf": 22, "perceptron": 12, "perform": [1, 9], "period": 3, "perspect": 1, "plan": [26, 27], "plethora": 25, "point": 4, "poisson": 2, "polynomi": [3, 16, 27], "popul": 2, "popular": 25, "practic": [13, 23, 25], "pre": [1, 3], "predict": 4, "preprocess": 26, "prerequisit": [3, 19, 25], "princip": 11, "principl": 3, "pro": 9, "probabl": [5, 22], "problem": [1, 2, 13, 25, 26, 27], "procedur": [9, 25], "process": [1, 3], "program": [2, 13, 27], "project": [6, 23, 25], "prop": 13, "propag": [1, 12], "properti": [5, 22, 26, 27], "python": [0, 9, 15, 19, 20, 25], "quick": 8, "r": 25, "random": [10, 11, 22], "raphson": 27, "read": [9, 25, 26], "real": [6, 25], "recommend": [25, 26], "recurr": [4, 12], "reduc": [0, 26], "reduct": 3, "reformul": 2, "regress": [0, 5, 6, 7, 9, 10, 13, 15, 17, 18, 25, 26, 27], "regular": 1, "relat": [], "relev": [24, 26], "relu": 1, "remark": 3, "remind": [6, 8, 25, 26, 27], "replac": 13, "repositori": 15, "requir": [2, 19], "resampl": 6, "rescal": [6, 26], "residu": [26, 27], "resourc": 2, "result": [26, 27], "revisit": [13, 27], "rewrit": [25, 26], "ridg": [0, 5, 6, 17, 18, 26, 27], "rm": 13, "rule": 12, "same": 13, "sampl": 11, "scale": [17, 18, 26], "schedul": 25, "schemat": 9, "scheme": 2, "scienc": 25, "scikit": [0, 1, 11, 25, 26, 27], "second": 13, "semest": 23, "sensit": 27, "septemb": 27, "session": 27, "set": [0, 2, 3, 9, 12, 15, 21, 25, 26, 27], "setup": 15, "sgd": 13, "should": 1, "similar": 13, "simpl": [0, 4, 9, 13, 25, 26, 27], "simplest": 18, "singl": 10, "singular": [5, 11, 26, 27], "size": [26, 27], "sklearn": 16, "soft": 8, "softmax": 1, "softwar": 25, "solv": [2, 27], "solver": 13, "some": [13, 20, 26, 27], "specifi": 2, "split": [0, 15, 26], "squar": [0, 5, 6, 10, 16, 25, 26, 27], "standard": [13, 26], "state": 0, "statist": [5, 6, 19, 22, 25], "steepest": [10, 13, 27], "stochast": [13, 22], "strongli": 25, "suggest": 25, "summari": [23, 25], "superposit": 3, "supervis": 1, "support": 8, "svd": [5, 26, 27], "synthet": 18, "systemat": 3, "t": 26, "take": 16, "taken": 25, "teach": 23, "teacher": [23, 25], "technic": 26, "techniqu": [6, 11], "technologi": 19, "tensorflow": [1, 3], "tent": [23, 25], "test": [0, 1, 15, 17, 26], "text": 25, "textbook": [24, 25], "than": 27, "theorem": [5, 8, 11, 12, 22], "theori": 22, "theta": 18, "thi": 25, "tip": 13, "togeth": 12, "tool": 25, "top": 1, "topic": 25, "toward": 11, "trade": 6, "tradeoff": 6, "train": [0, 1, 4, 15, 25, 26], "transform": 3, "tree": [9, 10], "tuesdai": 27, "tune": 1, "two": [3, 8, 19], "type": [2, 4, 12, 25], "uio": 25, "univers": [12, 24], "unsupervis": 14, "up": [0, 2, 9, 12, 15, 25, 26, 27], "us": [0, 1, 2, 3, 7, 13, 16, 18, 19, 25, 26, 27], "v": 3, "valid": 6, "valu": [5, 11, 22, 26, 27], "variabl": [22, 27], "varianc": 6, "variou": 0, "vector": [8, 12, 16, 20, 25, 26], "versu": 25, "view": [0, 4, 10, 26], "virtual": 15, "visual": [1, 9], "wai": 9, "wave": 2, "we": 25, "wednesdai": 27, "week": [15, 16, 17, 18, 25, 26, 27], "weekli": [], "what": [0, 25, 26, 27], "which": 1, "why": 25, "wisconsin": 7, "write": [4, 11, 27], "x": 26, "xgboost": 10, "yet": 27, "your": [0, 10, 16, 18, 26]}}) \ No newline at end of file +Search.setIndex({"alltitles": {"1a)": [[18, "a"]], "2a)": [[18, "id1"]], "2b)": [[18, "b"]], "3a)": [[18, "id2"]], "3b)": [[18, "id3"]], "A Classification Tree": [[9, "a-classification-tree"]], "A Frequentist approach to data analysis": [[0, "a-frequentist-approach-to-data-analysis"], [25, "a-frequentist-approach-to-data-analysis"]], "A better approach": [[8, "a-better-approach"]], "A first summary": [[25, "a-first-summary"]], "A quick Reminder on Lagrangian Multipliers": [[8, "a-quick-reminder-on-lagrangian-multipliers"]], "A simple example": [[4, "a-simple-example"]], "A soft classifier": [[8, "a-soft-classifier"]], "A top-down perspective on Neural networks": [[1, "a-top-down-perspective-on-neural-networks"]], "ADAM optimizer": [[13, "adam-optimizer"]], "Activation functions": [[12, "activation-functions"]], "Adaptive boosting: AdaBoost, Basic Algorithm": [[10, "adaptive-boosting-adaboost-basic-algorithm"]], "Adding error analysis and training set up": [[25, "adding-error-analysis-and-training-set-up"], [26, "adding-error-analysis-and-training-set-up"]], "Adjust hyperparameters": [[1, "adjust-hyperparameters"]], "Algorithms for Setting up Decision Trees": [[9, "algorithms-for-setting-up-decision-trees"]], "An Overview of Ensemble Methods": [[10, "an-overview-of-ensemble-methods"]], "An extrapolation example": [[4, "an-extrapolation-example"]], "An optimization/minimization problem": [[25, "an-optimization-minimization-problem"]], "And finally \\boldsymbol{X}\\boldsymbol{X}^T": [[26, "and-finally-boldsymbol-x-boldsymbol-x-t"]], "And what about using neural networks?": [[25, "and-what-about-using-neural-networks"]], "Another Example, now with a polynomial fit": [[27, "another-example-now-with-a-polynomial-fit"]], "Another example, the moons again": [[9, "another-example-the-moons-again"]], "Applied Data Analysis and Machine Learning": [[19, null]], "Autocorrelation function": [[22, "autocorrelation-function"]], "Automatic differentiation": [[13, "automatic-differentiation"]], "Back to Ridge and LASSO Regression": [[26, "back-to-ridge-and-lasso-regression"], [27, "back-to-ridge-and-lasso-regression"]], "Back to the Cancer Data": [[11, "back-to-the-cancer-data"]], "Bagging": [[10, "bagging"]], "Bagging Examples": [[10, "bagging-examples"]], "Basic Matrix Features": [[20, "basic-matrix-features"]], "Basic ideas of the Principal Component Analysis (PCA)": [[11, null]], "Basic math of the SVD": [[5, "basic-math-of-the-svd"], [26, "basic-math-of-the-svd"], [27, "basic-math-of-the-svd"]], "Basics": [[7, "basics"]], "Basics of a tree": [[9, "basics-of-a-tree"]], "Batch Normalization": [[1, "batch-normalization"]], "Bayes\u2019 Theorem and Ridge and Lasso Regression": [[5, "bayes-theorem-and-ridge-and-lasso-regression"]], "Boosting, a Bird\u2019s Eye View": [[10, "boosting-a-bird-s-eye-view"]], "Bootstrap": [[6, "bootstrap"]], "Bringing it together, first back propagation equation": [[12, "bringing-it-together-first-back-propagation-equation"]], "Building a Feed Forward Neural Network": [[1, null]], "Building a tree, regression": [[9, "building-a-tree-regression"]], "Building neural networks in Tensorflow and Keras": [[1, "building-neural-networks-in-tensorflow-and-keras"]], "CNNs in more detail, building convolutional neural networks in Tensorflow and Keras": [[3, "cnns-in-more-detail-building-convolutional-neural-networks-in-tensorflow-and-keras"]], "Cancer Data again now with Decision Trees and other Methods": [[9, "cancer-data-again-now-with-decision-trees-and-other-methods"]], "Choose cost function and optimizer": [[1, "choose-cost-function-and-optimizer"]], "Classical PCA Theorem": [[11, "classical-pca-theorem"]], "Clustering and Unsupervised Learning": [[14, null]], "Code for SVD and Inversion of Matrices": [[5, "code-for-svd-and-inversion-of-matrices"]], "Codes and Approaches": [[14, "codes-and-approaches"]], "Codes for the SVD": [[5, "codes-for-the-svd"], [26, "codes-for-the-svd"], [27, "codes-for-the-svd"]], "Coding Setup and Linear Regression": [[15, "coding-setup-and-linear-regression"]], "Collect and pre-process data": [[1, "collect-and-pre-process-data"]], "Communication channels": [[25, "communication-channels"]], "Compare Bagging on Trees with Random Forests": [[10, "compare-bagging-on-trees-with-random-forests"]], "Comparing with a numerical scheme": [[2, "comparing-with-a-numerical-scheme"]], "Comparison with OLS": [[27, "comparison-with-ols"]], "Computing the Gini index": [[9, "computing-the-gini-index"]], "Conditions on convex functions": [[27, "conditions-on-convex-functions"]], "Conjugate gradient method": [[13, "conjugate-gradient-method"]], "Convex function": [[27, "convex-function"]], "Convex functions": [[13, "convex-functions"], [27, "convex-functions"]], "Convolution Examples: Polynomial multiplication": [[3, "convolution-examples-polynomial-multiplication"]], "Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)": [[3, "convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms"]], "Convolutional Neural Network": [[12, "convolutional-neural-network"]], "Convolutional Neural Networks": [[3, null]], "Correlation Function and Design/Feature Matrix": [[26, "correlation-function-and-design-feature-matrix"]], "Correlation Matrix": [[11, "correlation-matrix"], [26, "correlation-matrix"]], "Correlation Matrix with Pandas": [[26, "correlation-matrix-with-pandas"]], "Course Format": [[25, "course-format"]], "Course setting": [[21, null]], "Covariance Matrix Examples": [[26, "covariance-matrix-examples"]], "Covariance and Correlation Matrix": [[26, "covariance-and-correlation-matrix"]], "Cross-validation": [[6, "cross-validation"]], "Deadlines for projects (tentative)": [[25, "deadlines-for-projects-tentative"]], "Decision trees, overarching aims": [[9, null]], "Deep learning methods": [[25, "deep-learning-methods"]], "Define model and architecture": [[1, "define-model-and-architecture"]], "Defining the cost function": [[1, "defining-the-cost-function"]], "Deliverables": [[15, "deliverables"], [16, "deliverables"]], "Derivatives and the chain rule": [[12, "derivatives-and-the-chain-rule"]], "Derivatives, example 1": [[26, "derivatives-example-1"]], "Deriving OLS from a probability distribution": [[5, "deriving-ols-from-a-probability-distribution"]], "Deriving and Implementing Ordinary Least Squares": [[16, "deriving-and-implementing-ordinary-least-squares"]], "Deriving and Implementing Ridge Regression": [[17, "deriving-and-implementing-ridge-regression"]], "Deriving the Lasso Regression Equations": [[26, "deriving-the-lasso-regression-equations"], [27, "deriving-the-lasso-regression-equations"], [27, "id6"]], "Deriving the Ridge Regression Equations": [[26, "deriving-the-ridge-regression-equations"], [27, "deriving-the-ridge-regression-equations"], [27, "id3"]], "Deriving the back propagation code for a multilayer perceptron model": [[12, "deriving-the-back-propagation-code-for-a-multilayer-perceptron-model"]], "Developing a code for doing neural networks with back propagation": [[1, "developing-a-code-for-doing-neural-networks-with-back-propagation"]], "Diagonalize the sample covariance matrix to obtain the principal components": [[11, "diagonalize-the-sample-covariance-matrix-to-obtain-the-principal-components"]], "Different kernels and Mercer\u2019s theorem": [[8, "different-kernels-and-mercer-s-theorem"]], "Disadvantages": [[9, "disadvantages"]], "Discriminative Modeling": [[25, "discriminative-modeling"]], "Domains and probabilities": [[22, "domains-and-probabilities"]], "Dropout": [[1, "dropout"]], "Economy-size SVD": [[26, "economy-size-svd"], [27, "economy-size-svd"]], "Elements of Probability Theory and Statistical Data Analysis": [[22, null]], "Ensemble Methods: From a Single Tree to Many Trees and Extreme Boosting, Meet the Jungle of Methods": [[10, null]], "Entropy and the ID3 algorithm": [[9, "entropy-and-the-id3-algorithm"]], "Essential elements of ML": [[25, "essential-elements-of-ml"]], "Evaluate model performance on test data": [[1, "evaluate-model-performance-on-test-data"]], "Example 2": [[26, "example-2"]], "Example 3": [[26, "example-3"]], "Example 4": [[26, "example-4"]], "Example Matrix": [[26, "example-matrix"], [27, "example-matrix"]], "Example of discriminative modeling, taken from Generative Deep Learning by David Foster": [[25, "example-of-discriminative-modeling-taken-from-generative-deep-learning-by-david-foster"]], "Example of generative modeling, taken from Generative Deep Learning by David Foster": [[25, "example-of-generative-modeling-taken-from-generative-deep-learning-by-david-foster"]], "Example of own Standard scaling": [[26, "example-of-own-standard-scaling"]], "Example relevant for the exercises": [[26, "example-relevant-for-the-exercises"]], "Example: Exponential decay": [[2, "example-exponential-decay"]], "Example: Population growth": [[2, "example-population-growth"]], "Example: The diffusion equation": [[2, "example-the-diffusion-equation"]], "Example: binary classification problem": [[1, "example-binary-classification-problem"]], "Examples": [[25, "examples"]], "Examples of likelihood functions used in logistic regression and neural networks": [[7, "examples-of-likelihood-functions-used-in-logistic-regression-and-neural-networks"]], "Exercise 1 - Choice of model and degrees of freedom": [[17, "exercise-1-choice-of-model-and-degrees-of-freedom"]], "Exercise 1 - Finding the derivative of Matrix-Vector expressions": [[16, "exercise-1-finding-the-derivative-of-matrix-vector-expressions"]], "Exercise 1 - Github Setup": [[15, "exercise-1-github-setup"]], "Exercise 1, scale your data": [[18, "exercise-1-scale-your-data"]], "Exercise 1: Setting up various Python environments": [[0, "exercise-1-setting-up-various-python-environments"]], "Exercise 2 - Deriving the expression for OLS": [[16, "exercise-2-deriving-the-expression-for-ols"]], "Exercise 2 - Deriving the expression for Ridge Regression": [[17, "exercise-2-deriving-the-expression-for-ridge-regression"]], "Exercise 2 - Setting up a Github repository": [[15, "exercise-2-setting-up-a-github-repository"]], "Exercise 2, use the analytical formulae for OLS and Ridge regression to find the optimal paramters \\boldsymbol{\\theta}": [[18, "exercise-2-use-the-analytical-formulae-for-ols-and-ridge-regression-to-find-the-optimal-paramters-boldsymbol-theta"]], "Exercise 2: making your own data and exploring scikit-learn": [[0, "exercise-2-making-your-own-data-and-exploring-scikit-learn"]], "Exercise 3 - Creating feature matrix and implementing OLS using the analytical expression": [[16, "exercise-3-creating-feature-matrix-and-implementing-ols-using-the-analytical-expression"]], "Exercise 3 - Fitting an OLS model to data": [[15, "exercise-3-fitting-an-ols-model-to-data"]], "Exercise 3 - Scaling data": [[17, "exercise-3-scaling-data"]], "Exercise 3 - Setting up a Python virtual environment": [[15, "exercise-3-setting-up-a-python-virtual-environment"]], "Exercise 3, Implementing the simplest form for gradient descent": [[18, "exercise-3-implementing-the-simplest-form-for-gradient-descent"]], "Exercise 3: Normalizing our data": [[0, "exercise-3-normalizing-our-data"]], "Exercise 4 - Fitting a polynomial": [[16, "exercise-4-fitting-a-polynomial"]], "Exercise 4 - Implementing Ridge Regression": [[17, "exercise-4-implementing-ridge-regression"]], "Exercise 4 - Testing multiple hyperparameters": [[17, "exercise-4-testing-multiple-hyperparameters"]], "Exercise 4 - The train-test split": [[15, "exercise-4-the-train-test-split"]], "Exercise 4: Adding Ridge Regression": [[0, "exercise-4-adding-ridge-regression"]], "Exercise 5 - Comparing your code with sklearn": [[16, "exercise-5-comparing-your-code-with-sklearn"]], "Exercise 5: Analytical exercises": [[0, "exercise-5-analytical-exercises"]], "Exercise: Cross-validation as resampling techniques, adding more complexity": [[6, "exercise-cross-validation-as-resampling-techniques-adding-more-complexity"]], "Exercise: Analysis of real data": [[6, "exercise-analysis-of-real-data"]], "Exercise: Bias-variance trade-off and resampling techniques": [[6, "exercise-bias-variance-trade-off-and-resampling-techniques"]], "Exercise: Lasso Regression on the Franke function with resampling": [[6, "exercise-lasso-regression-on-the-franke-function-with-resampling"]], "Exercise: Ordinary Least Square (OLS) on the Franke function": [[6, "exercise-ordinary-least-square-ols-on-the-franke-function"]], "Exercise: Ridge Regression on the Franke function with resampling": [[6, "exercise-ridge-regression-on-the-franke-function-with-resampling"]], "Exercises": [[0, "exercises"]], "Exercises and Projects": [[6, "exercises-and-projects"]], "Exercises week 34": [[15, null]], "Exercises week 35": [[16, null]], "Exercises week 36": [[17, null], [18, null]], "Expectation values": [[22, "expectation-values"]], "Extending to more than one variable": [[27, "extending-to-more-than-one-variable"]], "Extremely useful tools, strongly recommended": [[25, "extremely-useful-tools-strongly-recommended"]], "Feed-forward neural networks": [[12, "feed-forward-neural-networks"]], "Feed-forward pass": [[1, "feed-forward-pass"]], "Final back propagating equation": [[12, "final-back-propagating-equation"]], "Fine-tuning neural network hyperparameters": [[1, "fine-tuning-neural-network-hyperparameters"]], "Fitting an Equation of State for Dense Nuclear Matter": [[0, "fitting-an-equation-of-state-for-dense-nuclear-matter"]], "Fixing the singularity": [[26, "fixing-the-singularity"], [27, "fixing-the-singularity"]], "Frequently used scaling functions": [[26, "frequently-used-scaling-functions"]], "From OLS to Ridge and Lasso": [[27, "from-ols-to-ridge-and-lasso"]], "From one to many layers, the universal approximation theorem": [[12, "from-one-to-many-layers-the-universal-approximation-theorem"]], "Functionality in Scikit-Learn": [[26, "functionality-in-scikit-learn"]], "Further Dimensionality Remarks": [[3, "further-dimensionality-remarks"]], "Further properties (important for our analyses later)": [[5, "further-properties-important-for-our-analyses-later"], [26, "further-properties-important-for-our-analyses-later"], [27, "further-properties-important-for-our-analyses-later"]], "Gaussian Elimination": [[20, "gaussian-elimination"]], "General Features": [[9, "general-features"]], "General linear models and linear algebra": [[25, "general-linear-models-and-linear-algebra"]], "Generalizing the fitting procedure as a linear algebra problem": [[25, "generalizing-the-fitting-procedure-as-a-linear-algebra-problem"], [25, "id1"]], "Generative Adversarial Networks": [[4, "generative-adversarial-networks"]], "Generative Models": [[4, "generative-models"]], "Generative Versus Discriminative Modeling": [[25, "generative-versus-discriminative-modeling"]], "Geometric Interpretation and link with Singular Value Decomposition": [[11, "geometric-interpretation-and-link-with-singular-value-decomposition"]], "Gradient Boosting, Classification Example": [[10, "gradient-boosting-classification-example"]], "Gradient Boosting, Examples of Regression": [[10, "gradient-boosting-examples-of-regression"]], "Gradient Clipping": [[1, "gradient-clipping"]], "Gradient Descent Example": [[27, "id1"]], "Gradient boosting: Basics with Steepest Descent/Functional Gradient Descent": [[10, "gradient-boosting-basics-with-steepest-descent-functional-gradient-descent"]], "Gradient descent": [[2, "gradient-descent"]], "Gradient descent and Ridge": [[27, "gradient-descent-and-ridge"]], "Gradient descent example": [[27, "gradient-descent-example"]], "Grading": [[23, "grading"], [23, "id2"], [25, "grading"]], "How to take derivatives of Matrix-Vector expressions": [[16, "how-to-take-derivatives-of-matrix-vector-expressions"]], "Hyperplanes and all that": [[8, "hyperplanes-and-all-that"]], "Important Matrix and vector handling packages": [[20, "important-matrix-and-vector-handling-packages"]], "Important technicalities: More on Rescaling data": [[26, "important-technicalities-more-on-rescaling-data"]], "Improving performance": [[1, "improving-performance"]], "In summary": [[23, "in-summary"]], "Including Stochastic Gradient Descent with Autograd": [[13, "including-stochastic-gradient-descent-with-autograd"]], "Incremental PCA": [[11, "incremental-pca"]], "Installing R, C++, cython or Julia": [[25, "installing-r-c-cython-or-julia"]], "Installing R, C++, cython, Numba etc": [[25, "installing-r-c-cython-numba-etc"]], "Instructor information": [[23, "instructor-information"]], "Interpretations and optimizing our parameters": [[25, "interpretations-and-optimizing-our-parameters"], [25, "id2"], [25, "id3"], [26, "interpretations-and-optimizing-our-parameters"], [26, "id1"], [26, "id2"]], "Interpreting the Ridge results": [[26, "interpreting-the-ridge-results"], [27, "interpreting-the-ridge-results"], [27, "id4"]], "Introducing JAX": [[13, "introducing-jax"]], "Introducing the Covariance and Correlation functions": [[11, "introducing-the-covariance-and-correlation-functions"], [26, "introducing-the-covariance-and-correlation-functions"]], "Introduction": [[0, "introduction"], [6, "introduction"], [19, "introduction"], [20, "introduction"]], "Iterative Fitting, Classification and AdaBoost": [[10, "iterative-fitting-classification-and-adaboost"]], "Iterative Fitting, Regression and Squared-error Cost Function": [[10, "iterative-fitting-regression-and-squared-error-cost-function"]], "Kernel PCA": [[11, "kernel-pca"]], "Kernels and non-linearity": [[8, "kernels-and-non-linearity"]], "LU Decomposition, the inverse of a matrix": [[20, "lu-decomposition-the-inverse-of-a-matrix"]], "Lasso Regression": [[27, "lasso-regression"]], "Lasso case": [[27, "lasso-case"]], "Layers": [[1, "layers"]], "Layers used to build CNNs": [[3, "layers-used-to-build-cnns"]], "Learning goals": [[15, "learning-goals"], [16, "learning-goals"], [17, "learning-goals"], [18, "learning-goals"]], "Learning outcomes": [[19, "learning-outcomes"], [25, "learning-outcomes"]], "Lectures and ComputerLab": [[25, "lectures-and-computerlab"]], "Limitations of supervised learning with deep networks": [[1, "limitations-of-supervised-learning-with-deep-networks"]], "Linear Algebra, Handling of Arrays and more Python Features": [[20, null]], "Linear Regression": [[0, null]], "Linear Regression Problems": [[26, "linear-regression-problems"], [27, "linear-regression-problems"]], "Linear Regression and the SVD": [[27, "linear-regression-and-the-svd"]], "Linear Regression, basic elements": [[0, "linear-regression-basic-elements"]], "Linking Bayes\u2019 Theorem with Ridge and Lasso Regression": [[5, "linking-bayes-theorem-with-ridge-and-lasso-regression"]], "Linking the regression analysis with a statistical interpretation": [[5, "linking-the-regression-analysis-with-a-statistical-interpretation"]], "Linking with the SVD": [[5, "linking-with-the-svd"], [26, "linking-with-the-svd"]], "Links to relevant courses at the University of Oslo": [[24, "links-to-relevant-courses-at-the-university-of-oslo"]], "Logistic Regression": [[7, null], [7, "id1"]], "MNIST and GANs": [[4, "mnist-and-gans"]], "Machine Learning": [[25, "machine-learning"]], "Machine learning": [[19, "machine-learning"]], "Main textbooks": [[25, "main-textbooks"]], "Making a tree": [[9, "making-a-tree"]], "Making your own Bootstrap: Changing the Level of the Decision Tree": [[10, "making-your-own-bootstrap-changing-the-level-of-the-decision-tree"]], "Making your own test-train splitting": [[26, "making-your-own-test-train-splitting"]], "Material for exercises week 35": [[26, "material-for-exercises-week-35"]], "Material for lab sessions sessions Tuesday and Wednesday": [[27, "material-for-lab-sessions-sessions-tuesday-and-wednesday"]], "Material for lecture Monday September 2": [[27, "material-for-lecture-monday-september-2"]], "Mathematical Interpretation of Ordinary Least Squares": [[5, "mathematical-interpretation-of-ordinary-least-squares"], [26, "mathematical-interpretation-of-ordinary-least-squares"], [27, "mathematical-interpretation-of-ordinary-least-squares"]], "Mathematical optimization of convex functions": [[8, "mathematical-optimization-of-convex-functions"]], "Mathematics of CNNs": [[3, "mathematics-of-cnns"]], "Mathematics of the SVD and implications": [[5, "mathematics-of-the-svd-and-implications"], [26, "mathematics-of-the-svd-and-implications"], [27, "mathematics-of-the-svd-and-implications"]], "Matrices in Python": [[25, "matrices-in-python"]], "Matrix multiplication": [[1, "matrix-multiplication"]], "Matrix-vector notation and activation": [[12, "matrix-vector-notation-and-activation"]], "Meet the covariance!": [[22, "meet-the-covariance"]], "Meet the Covariance Matrix": [[5, "meet-the-covariance-matrix"], [26, "meet-the-covariance-matrix"]], "Meet the Hessian Matrix": [[26, "meet-the-hessian-matrix"]], "Meet the Pandas": [[25, "meet-the-pandas"]], "Min-Max Scaling": [[26, "min-max-scaling"]], "Momentum based GD": [[13, "momentum-based-gd"]], "More complicated Example: The Ising model": [[6, "more-complicated-example-the-ising-model"]], "More interpretations": [[26, "more-interpretations"], [27, "more-interpretations"], [27, "id5"]], "More on Dimensionalities": [[3, "more-on-dimensionalities"]], "More on Rescaling data": [[6, "more-on-rescaling-data"]], "More on Steepest descent": [[27, "more-on-steepest-descent"]], "More on convex functions": [[27, "more-on-convex-functions"]], "More preprocessing": [[26, "more-preprocessing"]], "Multilayer perceptrons": [[12, "multilayer-perceptrons"]], "Network requirements": [[2, "network-requirements"]], "Neural Networks vs CNNs": [[3, "neural-networks-vs-cnns"]], "Neural networks": [[12, null]], "Note about SVD Calculations": [[26, "note-about-svd-calculations"], [27, "note-about-svd-calculations"]], "Note on Scikit-Learn": [[27, "note-on-scikit-learn"]], "Numerical experiments and the covariance, central limit theorem": [[22, "numerical-experiments-and-the-covariance-central-limit-theorem"]], "Numpy and arrays": [[20, "numpy-and-arrays"], [25, "numpy-and-arrays"]], "Numpy examples and Important Matrix and vector handling packages": [[25, "numpy-examples-and-important-matrix-and-vector-handling-packages"]], "Optimization and gradient descent, the central part of any Machine Learning algortithm": [[27, "optimization-and-gradient-descent-the-central-part-of-any-machine-learning-algortithm"]], "Optimization, the central part of any Machine Learning algortithm": [[13, null]], "Optimizing our parameters": [[25, "optimizing-our-parameters"]], "Optimizing our parameters, more details": [[25, "optimizing-our-parameters-more-details"]], "Optimizing the cost function": [[1, "optimizing-the-cost-function"]], "Organizing our data": [[0, "organizing-our-data"], [25, "organizing-our-data"]], "Other Matrix and Vector Operations": [[20, "other-matrix-and-vector-operations"]], "Other Types of Recurrent Neural Networks": [[4, "other-types-of-recurrent-neural-networks"]], "Other courses on Data science and Machine Learning at UiO": [[25, "other-courses-on-data-science-and-machine-learning-at-uio"]], "Other courses on Data science and Machine Learning at UiO, contn": [[25, "other-courses-on-data-science-and-machine-learning-at-uio-contn"]], "Other popular texts": [[25, "other-popular-texts"]], "Other techniques": [[11, "other-techniques"]], "Other types of networks": [[12, "other-types-of-networks"]], "Other ways of visualizing the trees": [[9, "other-ways-of-visualizing-the-trees"]], "Our model for the nuclear binding energies": [[25, "our-model-for-the-nuclear-binding-energies"]], "Overview of first week": [[25, "overview-of-first-week"]], "Own code for Ordinary Least Squares": [[25, "own-code-for-ordinary-least-squares"], [26, "own-code-for-ordinary-least-squares"]], "PCA and scikit-learn": [[11, "pca-and-scikit-learn"]], "Pandas AI": [[25, "pandas-ai"]], "Partial Differential Equations": [[2, "partial-differential-equations"]], "Plans for week 35": [[26, "plans-for-week-35"]], "Plans for week 36": [[27, "plans-for-week-36"]], "Practical tips": [[13, "practical-tips"]], "Practicalities": [[23, "practicalities"], [23, "id1"]], "Predicting New Points With A Trained Recurrent Neural Network": [[4, "predicting-new-points-with-a-trained-recurrent-neural-network"]], "Preprocessing our data": [[26, "preprocessing-our-data"]], "Prerequisites": [[25, "prerequisites"]], "Prerequisites and background": [[19, "prerequisites-and-background"]], "Prerequisites: Collect and pre-process data": [[3, "prerequisites-collect-and-pre-process-data"]], "Probability Distribution Functions": [[22, "probability-distribution-functions"]], "Program example for gradient descent with Ridge Regression": [[27, "program-example-for-gradient-descent-with-ridge-regression"]], "Program for stochastic gradient": [[13, "program-for-stochastic-gradient"]], "Properties of PDFs": [[22, "properties-of-pdfs"]], "Pros and cons of trees, pros": [[9, "pros-and-cons-of-trees-pros"]], "Python installers": [[19, "python-installers"], [25, "python-installers"]], "RMS prop": [[13, "rms-prop"]], "Random Numbers": [[22, "random-numbers"]], "Random forests": [[10, "random-forests"]], "Randomized PCA": [[11, "randomized-pca"]], "Reading material": [[25, "reading-material"]], "Reading recommendations:": [[26, "reading-recommendations"]], "Reading suggestions week 34": [[25, "reading-suggestions-week-34"]], "Recurrent neural networks": [[12, "recurrent-neural-networks"]], "Recurrent neural networks: Overarching view": [[4, null]], "Reducing the number of degrees of freedom, overarching view": [[0, "reducing-the-number-of-degrees-of-freedom-overarching-view"], [26, "reducing-the-number-of-degrees-of-freedom-overarching-view"]], "Reformulating the problem": [[2, "reformulating-the-problem"]], "Regression Case": [[10, "regression-case"]], "Regression analysis, overarching aims": [[25, "regression-analysis-overarching-aims"]], "Regression analysis, overarching aims II": [[25, "regression-analysis-overarching-aims-ii"]], "Regularization": [[1, "regularization"]], "Reminder from last week": [[26, "reminder-from-last-week"]], "Reminder on Newton-Raphson\u2019s method": [[27, "reminder-on-newton-raphson-s-method"]], "Reminder on Statistics": [[6, "reminder-on-statistics"]], "Replace or not": [[13, "replace-or-not"]], "Required Technologies": [[19, "required-technologies"]], "Resampling Methods": [[6, null]], "Resampling methods": [[6, "id1"]], "Residual Error": [[26, "residual-error"], [27, "residual-error"]], "Resources on differential equations and deep learning": [[2, "resources-on-differential-equations-and-deep-learning"]], "Revisiting Ordinary Least Squares": [[27, "revisiting-ordinary-least-squares"]], "Revisiting our Linear Regression Solvers": [[13, "revisiting-our-linear-regression-solvers"]], "Rewriting the Covariance and/or Correlation Matrix": [[26, "rewriting-the-covariance-and-or-correlation-matrix"]], "Rewriting the fitting procedure as a linear algebra problem": [[25, "rewriting-the-fitting-procedure-as-a-linear-algebra-problem"]], "Rewriting the fitting procedure as a linear algebra problem, more details": [[25, "rewriting-the-fitting-procedure-as-a-linear-algebra-problem-more-details"]], "Ridge Regression": [[27, "ridge-regression"]], "Ridge and LASSO Regression": [[26, "ridge-and-lasso-regression"], [27, "ridge-and-lasso-regression"], [27, "id2"]], "Ridge and Lasso Regression": [[5, null], [5, "id1"]], "Ridge regression and a new Synthetic Dataset": [[18, "ridge-regression-and-a-new-synthetic-dataset"]], "SVD analysis": [[27, "svd-analysis"]], "Same code but now with momentum gradient descent": [[13, "same-code-but-now-with-momentum-gradient-descent"]], "Schedule first week": [[25, "schedule-first-week"]], "Schematic Regression Procedure": [[9, "schematic-regression-procedure"]], "Setting up the Back propagation algorithm": [[12, "setting-up-the-back-propagation-algorithm"]], "Setting up the Matrix to be inverted": [[26, "setting-up-the-matrix-to-be-inverted"], [27, "setting-up-the-matrix-to-be-inverted"]], "Setting up the network using Autograd; The full program": [[2, "setting-up-the-network-using-autograd-the-full-program"]], "Similar (second order function now) problem but now with AdaGrad": [[13, "similar-second-order-function-now-problem-but-now-with-adagrad"]], "Simple Python Code to read in Data and perform Classification": [[9, "simple-python-code-to-read-in-data-and-perform-classification"]], "Simple case": [[26, "simple-case"], [27, "simple-case"]], "Simple code for solving the above problem": [[27, "simple-code-for-solving-the-above-problem"]], "Simple example to illustrate Ordinary Least Squares, Ridge and Lasso Regression": [[27, "simple-example-to-illustrate-ordinary-least-squares-ridge-and-lasso-regression"]], "Simple geometric interpretation": [[27, "simple-geometric-interpretation"]], "Simple linear regression model using scikit-learn": [[0, "simple-linear-regression-model-using-scikit-learn"], [25, "simple-linear-regression-model-using-scikit-learn"]], "Simple program": [[27, "simple-program"]], "Software and needed installations": [[25, "software-and-needed-installations"]], "Solving Differential Equations with Deep Learning": [[2, null]], "Solving the one dimensional Poisson equation": [[2, "solving-the-one-dimensional-poisson-equation"]], "Solving the wave equation with Neural Networks": [[2, "solving-the-wave-equation-with-neural-networks"]], "Some famous Matrices": [[20, "some-famous-matrices"]], "Some simple problems": [[13, "some-simple-problems"], [27, "some-simple-problems"]], "Some useful matrix and vector expressions": [[26, "some-useful-matrix-and-vector-expressions"]], "Splitting our Data in Training and Test data": [[0, "splitting-our-data-in-training-and-test-data"], [26, "splitting-our-data-in-training-and-test-data"]], "Standard steepest descent": [[13, "standard-steepest-descent"]], "Statistical analysis and optimization of data": [[19, "statistical-analysis-and-optimization-of-data"], [25, "statistical-analysis-and-optimization-of-data"]], "Steepest descent": [[13, "steepest-descent"], [27, "steepest-descent"]], "Stochastic Gradient Descent (SGD)": [[13, "stochastic-gradient-descent-sgd"]], "Stochastic variables and the main concepts, the discrete case": [[22, "stochastic-variables-and-the-main-concepts-the-discrete-case"]], "Support Vector Machines, overarching aims": [[8, null]], "Systematic reduction": [[3, "systematic-reduction"]], "Teachers": [[25, "teachers"]], "Teachers and Grading": [[23, null]], "Teaching Assistants Fall semester 2023": [[23, "teaching-assistants-fall-semester-2023"]], "Tentative deadllines for projects": [[23, "tentative-deadllines-for-projects"]], "Testing the Means Squared Error as function of Complexity": [[0, "testing-the-means-squared-error-as-function-of-complexity"], [26, "testing-the-means-squared-error-as-function-of-complexity"]], "Textbooks": [[24, null]], "The Algorithm before theorem": [[11, "the-algorithm-before-theorem"]], "The Breast Cancer Data, now with Keras": [[1, "the-breast-cancer-data-now-with-keras"]], "The CART algorithm for Classification": [[9, "the-cart-algorithm-for-classification"]], "The CART algorithm for Regression": [[9, "the-cart-algorithm-for-regression"]], "The CIFAR01 data set": [[3, "the-cifar01-data-set"]], "The Hessian matrix": [[27, "the-hessian-matrix"]], "The Hessian matrix for Ridge Regression": [[27, "the-hessian-matrix-for-ridge-regression"]], "The Jacobian": [[26, "the-jacobian"]], "The MNIST dataset again": [[3, "the-mnist-dataset-again"]], "The OLS case": [[27, "the-ols-case"]], "The RELU function family": [[1, "the-relu-function-family"]], "The Ridge case": [[27, "the-ridge-case"]], "The SVD, a Fantastic Algorithm": [[26, "the-svd-a-fantastic-algorithm"], [27, "the-svd-a-fantastic-algorithm"]], "The Softmax function": [[1, "the-softmax-function"]], "The \\chi^2 function": [[0, "the-chi-2-function"], [25, "the-chi-2-function"], [25, "id4"], [25, "id5"], [25, "id6"], [25, "id7"], [25, "id8"]], "The bias-variance tradeoff": [[6, "the-bias-variance-tradeoff"]], "The code for solving the ODE": [[2, "the-code-for-solving-the-ode"]], "The complete code with a simple data set": [[26, "the-complete-code-with-a-simple-data-set"]], "The cost/loss function": [[26, "the-cost-loss-function"]], "The course has two central parts": [[19, "the-course-has-two-central-parts"]], "The derivative of the cost/loss function": [[27, "the-derivative-of-the-cost-loss-function"]], "The equations": [[27, "the-equations"]], "The equations for ordinary least squares": [[26, "the-equations-for-ordinary-least-squares"]], "The first Case": [[27, "the-first-case"]], "The ideal": [[27, "the-ideal"]], "The logistic function": [[7, "the-logistic-function"]], "The mean squared error and its derivative": [[26, "the-mean-squared-error-and-its-derivative"]], "The moons example": [[8, "the-moons-example"]], "The multilayer perceptron (MLP)": [[12, "the-multilayer-perceptron-mlp"]], "The network with one input layer, specified number of hidden layers, and one output layer": [[2, "the-network-with-one-input-layer-specified-number-of-hidden-layers-and-one-output-layer"]], "The plethora of machine learning algorithms/methods": [[25, "the-plethora-of-machine-learning-algorithms-methods"]], "The sensitiveness of the gradient descent": [[27, "the-sensitiveness-of-the-gradient-descent"]], "The singular value decomposition": [[5, "the-singular-value-decomposition"], [26, "the-singular-value-decomposition"], [27, "the-singular-value-decomposition"]], "The two-dimensional case": [[8, "the-two-dimensional-case"]], "To our real data: nuclear binding energies. Brief reminder on masses and binding energies": [[25, "to-our-real-data-nuclear-binding-energies-brief-reminder-on-masses-and-binding-energies"]], "Topics covered in this course: Statistical analysis and optimization of data": [[25, "topics-covered-in-this-course-statistical-analysis-and-optimization-of-data"]], "Towards the PCA theorem": [[11, "towards-the-pca-theorem"]], "Train and test datasets": [[1, "train-and-test-datasets"]], "Two-dimensional Objects": [[3, "two-dimensional-objects"]], "Type of problem": [[2, "type-of-problem"]], "Types of Machine Learning": [[25, "types-of-machine-learning"]], "Useful Python libraries": [[19, "useful-python-libraries"], [25, "useful-python-libraries"]], "Using Autograd": [[13, "using-autograd"]], "Using forward Euler to solve the ODE": [[2, "using-forward-euler-to-solve-the-ode"]], "Using gradient descent methods, limitations": [[13, "using-gradient-descent-methods-limitations"], [27, "using-gradient-descent-methods-limitations"]], "Visualization": [[1, "visualization"], [1, "id1"]], "Visualizing the Tree, Classification": [[9, "visualizing-the-tree-classification"]], "Week 34: Introduction to the course, Logistics and Practicalities": [[25, null]], "Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression": [[26, null]], "Week 36: Linear Regression and Gradient descent": [[27, null]], "What Is Generative Modeling?": [[25, "what-is-generative-modeling"]], "What does it mean?": [[26, "what-does-it-mean"], [27, "what-does-it-mean"]], "What is Machine Learning?": [[0, "what-is-machine-learning"]], "What is a good model?": [[0, "what-is-a-good-model"], [25, "what-is-a-good-model"]], "What is a good model? Can we define it?": [[25, "what-is-a-good-model-can-we-define-it"]], "Which activation function should I use?": [[1, "which-activation-function-should-i-use"]], "Why Linear Regression (aka Ordinary Least Squares and family)": [[25, "why-linear-regression-aka-ordinary-least-squares-and-family"]], "Wisconsin Cancer Data": [[7, "wisconsin-cancer-data"]], "With Lasso Regression": [[27, "with-lasso-regression"]], "Writing Our First Generative Adversarial Network": [[4, "writing-our-first-generative-adversarial-network"]], "Writing our own PCA code": [[11, "writing-our-own-pca-code"]], "Writing the Cost Function": [[27, "writing-the-cost-function"]], "XGBoost: Extreme Gradient Boosting": [[10, "xgboost-extreme-gradient-boosting"]], "Yet another Example": [[27, "yet-another-example"]], "a) Expression for Ridge regression": [[17, "a-expression-for-ridge-regression"]], "scikit-learn implementation": [[1, "scikit-learn-implementation"]]}, "docnames": ["chapter1", "chapter10", "chapter11", "chapter12", "chapter13", "chapter2", "chapter3", "chapter4", "chapter5", "chapter6", "chapter7", "chapter8", "chapter9", "chapteroptimization", "clustering", "exercisesweek34", "exercisesweek35", "exercisesweek36", "exercisesweek37", "intro", "linalg", "schedule", "statistics", "teachers", "textbooks", "week34", "week35", "week36"], "envversion": {"sphinx": 62, "sphinx.domains.c": 3, "sphinx.domains.changeset": 1, "sphinx.domains.citation": 1, "sphinx.domains.cpp": 9, "sphinx.domains.index": 1, "sphinx.domains.javascript": 3, "sphinx.domains.math": 2, "sphinx.domains.python": 4, "sphinx.domains.rst": 2, "sphinx.domains.std": 2, "sphinx.ext.intersphinx": 1}, "filenames": ["chapter1.ipynb", "chapter10.ipynb", "chapter11.ipynb", "chapter12.ipynb", "chapter13.ipynb", "chapter2.ipynb", "chapter3.ipynb", "chapter4.ipynb", "chapter5.ipynb", "chapter6.ipynb", "chapter7.ipynb", "chapter8.ipynb", "chapter9.ipynb", "chapteroptimization.ipynb", "clustering.ipynb", "exercisesweek34.ipynb", "exercisesweek35.ipynb", "exercisesweek36.ipynb", "exercisesweek37.ipynb", "intro.md", "linalg.ipynb", "schedule.md", "statistics.ipynb", "teachers.md", "textbooks.md", "week34.ipynb", "week35.ipynb", "week36.ipynb"], "indexentries": {}, "objects": {}, "objnames": {}, "objtypes": {}, "terms": {"": [0, 1, 2, 3, 4, 5, 6, 7, 9, 11, 12, 13, 15, 16, 17, 19, 20, 22, 23, 25, 26], "0": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 22, 23, 25, 26, 27], "00": [0, 1, 5, 11, 25, 26], "000": [1, 3], "00000000e": [], "001": [2, 8, 13, 27], "004": 5, "004113634617443131": 26, "004113634617443139": 26, "00411363461744314": 26, "004113634617443147": 26, "00727646693": [0, 25], "0086649156": [0, 25], "01": [0, 1, 2, 5, 9, 11, 13, 17, 24, 25, 26], "0110": 22, "01719003e": [], "02": [0, 4, 7, 12, 25], "02334824": [], "02857": 4, "02f": 6, "03077640549": 4, "03097597e": [], "031": 5, "04": 11, "0458": 9, "05": [4, 6], "062292565": 4, "062435": [], "06730814": [], "07": [], "0713": [0, 25], "07285": 3, "08": 22, "08078025e": [], "08336233266": 4, "08376632": 26, "083766322923899": 26, "0837663229239043": 26, "0917": 9, "0n": [0, 25], "0x113e21950": 17, "1": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 20, 21, 22, 23, 24, 25, 27], "10": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 16, 17, 18, 20, 21, 22, 23, 25, 26, 27], "100": [0, 1, 2, 3, 4, 5, 6, 8, 9, 10, 11, 13, 14, 15, 16, 17, 18, 20, 22, 23, 25, 26, 27], "1000": [0, 1, 2, 4, 5, 8, 11, 13, 14, 18, 19, 22, 25, 27], "10000": [2, 5, 6, 10, 11, 13, 22], "100000": 8, "10001": 10, "1001": 22, "1002": 22, "1003": 22, "1005": 22, "1009": 22, "101": 16, "1011": 22, "1013": 22, "1013904243": 22, "1015": 22, "102": 16, "1023": 22, "1024": 3, "1026": 22, "1027": 22, "103": 1, "1030": 22, "1037": 22, "1038": 22, "1040": 22, "1047": 22, "107": 16, "108": [], "10th": 9, "10x": [0, 25], "11": [0, 2, 5, 6, 7, 8, 9, 10, 11, 12, 13, 16, 20, 22, 24, 25, 26, 27], "110": [], "1100": 22, "1101": 22, "111": [1, 7, 12], "112": 16, "11340253": [], "11590451": [], "116": 16, "117": 16, "118": 16, "12": [0, 1, 2, 3, 4, 5, 6, 8, 9, 11, 12, 18, 20, 22, 24, 25, 26, 27], "120": 3, "121": [8, 9, 10, 16], "1215pm": [23, 25], "122": [8, 9, 10], "124": [0, 25], "125": 16, "127": [4, 16], "128": [3, 4, 13], "129": 16, "1298": 9, "12pm": [23, 25], "13": [0, 2, 9, 12, 20, 22, 25], "131": 16, "133": 7, "135": 16, "136": 16, "14": [0, 2, 4, 6, 8, 9, 10, 12, 20, 22, 24, 26], "141": 16, "143": 16, "1446729567": 4, "149": 16, "14g": 6, "15": [0, 2, 4, 6, 7, 8, 9, 12, 13, 22, 25, 27], "150": [4, 8], "152": 16, "153760": [], "156": 16, "157": [], "158": [], "159": 16, "15g": 6, "15pm": 25, "16": [1, 2, 3, 4, 5, 8, 9, 10, 22, 25, 27], "160": 16, "1603": 3, "161": 16, "162": 16, "16231451": 4, "163": 16, "16384": 3, "164": 16, "167": 16, "17": [1, 2, 8, 22], "172": 16, "173": 16, "176": 16, "178": 16, "179": 16, "1797": 1, "18": [2, 6, 7, 8, 9, 10, 22, 25], "1807": 4, "18392847": [], "19": [2, 22, 25], "1940": [], "1943": 12, "19569961": 26, "1970": [20, 25], "1973": 9, "1979": 6, "1_1": 12, "1_2": 12, "1_3": 12, "1cm": [0, 8, 10, 22, 25], "1d": [1, 2, 3], "1e": [2, 4, 13, 14], "1e10": 14, "1e4": 6, "1f": 1, "1k": 20, "1n": [0, 25], "1x": [0, 25], "2": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 19, 20, 22, 24], "20": [0, 1, 2, 6, 7, 8, 16, 17, 22, 23, 25, 26, 27], "200": [0, 2, 3, 4, 8, 9, 10], "2000": [0, 26], "2004": [13, 27], "2006": 24, "20072279": [], "2008": 25, "2010": 1, "2011": 1, "2014": 4, "2015": 1, "2016": [0, 25], "2018": [0, 6, 26], "2021": [6, 14, 26], "2022": 25, "2025": [18, 25, 26, 27], "21": [0, 1, 5, 7, 9, 12, 20, 25, 26, 27], "2116753732": 4, "215pm": [23, 25], "2167072": [], "22": [0, 1, 5, 12, 13, 20, 25, 26, 27], "221": 8, "225": 4, "22948497": [], "23": [1, 12, 20], "24": [0, 1, 20, 25], "25": [2, 3, 4, 5, 6, 8, 9, 11, 26], "250": [2, 4, 7, 9], "25000": [], "250154": [], "253775": [], "255": 3, "256": 4, "26": [], "26303845": [], "264": [], "265": [], "265109911": 4, "266": [], "269": [], "27": 1, "270": [], "278": 27, "27n_": 22, "28": [1, 3, 4], "283": 27, "2830637392": 4, "2861": 22, "2873": 9, "2882": 22, "2886": 22, "2890": [0, 25], "2892": 22, "29": 26, "2915": 22, "2931": 25, "29364655": [], "294399745619595": [], "296247": [], "2968": 25, "2980": 25, "298273": [], "298375": [], "2990": 25, "2_": 12, "2_1": 12, "2_2": 12, "2_3": 12, "2_i": 12, "2_m": [6, 22], "2_t": 13, "2_x": 22, "2a": 17, "2b": 22, "2cm": 8, "2d": [1, 3, 11, 12, 19, 25], "2e": 6, "2f": [0, 7, 9, 10, 11, 12, 25], "2g": 2, "2g_i": 2, "2k": 3, "2m": 6, "2mvizaqfst8": 26, "2n": [0, 2, 3, 25, 26], "2nd": 9, "2p": 22, "2pt": 4, "2x": [0, 3, 8, 13, 25], "2x_ix_jy_iy_j": 8, "2x_j": 8, "2y_i": 10, "2y_j": 8, "3": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 19, 20, 21, 22, 23, 25, 27], "30": [0, 1, 4, 6, 7, 10, 13, 23], "30000": [0, 25], "3072": 3, "31": [12, 20, 22], "315": [6, 26], "3155": [0, 5, 6, 26, 27], "32": [3, 4, 6, 12, 13, 20, 22], "3200": 1, "3250": 1, "3297": [], "33": [12, 20, 23], "3303": [], "3310": [], "332331": [], "333": 7, "3331": [], "3337": [], "34": 20, "3436": [0, 25], "3437": [0, 25], "35": [0, 6, 25, 27], "3581341341": 4, "359": [5, 27], "36": [0, 5, 6, 22], "37": 27, "370782966": 4, "38": 22, "39": [0, 23, 25], "3d": [2, 3, 4, 6, 13, 16], "3f": [1, 3, 9], "3n": 20, "3x": [2, 8], "3x_i": 2, "3y": 8, "4": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 18, 20, 22, 25, 27], "40": [1, 6, 23, 25], "400": 4, "4000": 25, "4050": [24, 25], "41": 20, "4155": [2, 15], "41589548": [], "42": [1, 4, 8, 9, 10, 20], "43": [0, 7, 20], "4310": 25, "436462435": 4, "44": [0, 20, 27], "45": [23, 25], "46": [23, 25], "462": 7, "47": [23, 25], "479465113": 4, "47958494": [], "48": [], "48257387": [23, 25], "49": [5, 6, 11], "49152": 3, "4940954": [0, 25], "4990": 22, "4992": 22, "4997": 22, "4c4c7f": [9, 10], "4d": 3, "4f": 6, "4pm": [23, 25], "4y": 8, "4y_i": 10, "5": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 18, 20, 22, 25, 26, 27], "50": [1, 2, 3, 4, 6, 7, 8, 10, 13, 25, 26], "500": [1, 3, 4, 6, 9, 10, 13], "5018": 22, "506": [], "507d50": [9, 10], "50j": 13, "50x10": 1, "51": 10, "510": 1, "512132": [], "5177783846": 4, "53": 9, "54": [6, 22], "5411205": [], "54894451": [], "55": 1, "56": 1, "56536": [0, 25], "569": 1, "57": [0, 8, 23, 25], "571": [5, 27], "58": [10, 23, 25], "591317992": 4, "5cm": 22, "5f": 8, "5x": 8, "5y": 8, "6": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 18, 20, 22, 23, 25, 26, 27], "60": [1, 3], "60000": 4, "6019067271": 4, "606439": [], "625": 7, "63": 1, "64": [1, 3, 4, 13, 20, 25], "64x50": 1, "65": [1, 8, 9], "6887363571": 4, "69": [16, 22], "69069n_": 22, "691": [], "6n_": 22, "7": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 20, 22, 24, 25, 26], "70": [1, 7], "70653767": 4, "71": 1, "724": 3, "73": [], "7304881": [], "75": [5, 6, 8, 11], "76": [23, 25], "765": 7, "77": [23, 25], "7718": 9, "7782028952": 4, "77893972": [], "78": [], "7d7d58": [9, 10], "8": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 14, 18, 20, 22, 23, 25, 27], "80": [0, 1, 5, 8, 17, 26], "800": [4, 7], "81": 1, "815am": [23, 25], "85": 1, "8702784034": 4, "88": 25, "8f": 6, "8g": 6, "8n": 20, "8x8": 1, "9": [0, 1, 2, 4, 5, 6, 7, 8, 9, 11, 12, 13, 20, 22, 25], "90": 1, "9040": 9, "91": [23, 25], "92": [23, 25], "93": 16, "931": [0, 25], "933": [5, 27], "937": 22, "938": 22, "939": [0, 22, 25], "94": 22, "95": [1, 11], "954": 22, "955820c21e8b": 4, "96": 6, "960": 22, "961": 22, "962": 22, "9649652536": 4, "96611194e": [], "9780387310732": 24, "9780387848570": 24, "9781098134174": 25, "9781492032632": 24, "9781801819312": 25, "97898392": 26, "98": [0, 1, 16], "985": 22, "986": 22, "989": 22, "9898ff": [9, 10], "99": [13, 16], "991": 22, "992": 22, "993": 22, "996": 5, "999": [9, 22], "9x": 6, "9y": 6, "A": [2, 3, 5, 6, 7, 10, 11, 12, 13, 15, 16, 19, 20, 21, 22, 23, 24, 26, 27], "AND": 2, "And": [0, 3, 4, 5, 6, 9, 13, 19, 22, 27], "As": [0, 1, 2, 3, 4, 5, 6, 8, 10, 12, 13, 15, 16, 20, 22, 25, 26, 27], "At": [0, 4, 6, 13, 25], "BE": [0, 25], "Be": [2, 18, 19, 25], "Being": 13, "But": [0, 1, 2, 3, 5, 6, 9, 10, 16, 22, 26], "By": [0, 3, 5, 6, 12, 13, 17, 20, 25, 26, 27], "For": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 18, 19, 20, 22, 24, 25, 26, 27], "IF": [6, 26], "IN": 24, "If": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16, 18, 19, 20, 22, 25, 26, 27], "In": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 19, 20, 22, 24, 25, 26, 27], "Ising": [5, 12, 26, 27], "It": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 18, 19, 20, 22, 25, 26, 27], "Its": [1, 2, 4, 11], "No": [6, 9, 25, 26], "Not": [0, 1, 5, 6, 26, 27], "OR": 22, "Of": 22, "On": [0, 3, 22, 23, 24, 25], "One": [0, 1, 3, 4, 5, 6, 7, 8, 11, 12, 13, 17, 22, 26, 27], "Or": [0, 1, 6, 25], "Such": [0, 6, 12, 16, 22], "That": [0, 5, 7, 10, 11, 12, 14, 22, 25], "The": [4, 10, 13, 14, 16, 17, 18, 20, 21, 22, 23, 24], "Then": [0, 1, 6, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 25, 27], "There": [0, 3, 4, 5, 6, 8, 9, 11, 12, 14, 15, 20, 22, 23, 25, 26, 27], "These": [0, 3, 4, 5, 8, 9, 10, 11, 12, 13, 14, 17, 20, 22, 23, 25, 26, 27], "To": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16, 20, 22, 26, 27], "With": [0, 5, 6, 8, 9, 10, 11, 12, 14, 16, 20, 22, 25, 26], "_": [0, 1, 2, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 16, 17, 18, 20, 25, 26, 27], "_0": [5, 8, 10, 11, 13, 26, 27], "_1": [2, 5, 6, 8, 10, 11, 12, 13, 14, 20, 26, 27], "_2": [2, 5, 8, 11, 12, 13, 20, 26], "_3": 20, "_4": 20, "_9": 13, "__class__": 10, "__doc__": 6, "__future__": [8, 9], "__init__": 1, "__main__": 2, "__name__": [2, 10], "_auto1": [2, 3, 4, 5, 6, 7, 12, 13, 20, 22, 26, 27], "_auto10": [6, 12], "_auto11": 6, "_auto12": 6, "_auto2": [2, 3, 4, 5, 6, 12, 13, 20, 22], "_auto3": [3, 4, 5, 6, 12, 13, 20], "_auto4": [4, 6, 12, 13, 20], "_auto5": [4, 6, 12, 13, 20], "_auto6": [4, 6, 12, 20], "_auto7": [4, 6, 12, 20], "_auto8": [6, 12], "_auto9": [6, 12], "_build": [0, 19, 24, 25], "_c": 1, "_center": [], "_compon": 11, "_depth": 9, "_export": [15, 16], "_fraction": 9, "_i": [0, 1, 2, 5, 6, 7, 8, 11, 12, 13, 25, 26, 27], "_j": [0, 1, 2, 3, 5, 6, 8, 13, 26, 27], "_k": [13, 27], "_l": 12, "_lambda": 6, "_leaf": 9, "_m": 10, "_multilayer_perceptron": [], "_n": [2, 5, 8, 11, 13, 26, 27], "_node": 9, "_norm": [], "_p": [5, 8, 26, 27], "_ratio": 11, "_sampl": 9, "_split": [6, 9], "_t": 13, "_test": 6, "_varianc": 11, "_weight": 9, "a0": 3, "a0faa0": [9, 10], "a1": [0, 25], "a2": [0, 25], "a3": [0, 25], "a4": [0, 25], "a_": [0, 1, 16, 20, 25, 26], "a_0": [0, 25], "a_1a": [0, 25], "a_2a": [0, 25], "a_3": [0, 25], "a_3a": [0, 25], "a_4": [0, 25], "a_4a": [0, 25], "a_h": 1, "a_i": [0, 1, 2, 12, 25], "a_j": [1, 12], "a_k": [0, 1, 12], "aaron": 24, "ab": [0, 2, 5, 13, 14, 25, 26], "ab_channel": 19, "abandon": 1, "abid": 22, "abil": [0, 10], "abl": [0, 1, 4, 5, 6, 7, 10, 12, 13, 16, 18, 26, 27], "about": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16, 19, 20, 23], "abov": [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 22, 24, 25, 26], "abovement": [6, 25], "abscissa": [13, 27], "absolut": [0, 2, 5, 6, 13, 25, 26, 27], "absorb": [26, 27], "abstract": 1, "acceler": 13, "accept": [0, 3, 6, 9, 26], "access": [3, 11, 22, 25], "accid": [4, 6], "accompani": [0, 25, 26], "accomplish": [8, 9, 13], "accord": [0, 1, 2, 5, 6, 9, 12, 13, 14, 22, 25, 27], "accordingli": 11, "account": [0, 3, 5, 13, 15, 16, 22, 25], "accumul": [12, 13, 22], "accur": [0, 3, 4, 6, 10, 13], "accuraci": [0, 1, 3, 4, 5, 6, 7, 9, 10, 11, 12, 25, 26, 27], "accuracy_scor": [0, 1, 10, 25], "accuracy_score_numpi": 1, "achiev": [0, 1, 5, 6, 8, 12, 20, 25], "aco": 22, "acquaint": 19, "acquir": [1, 19, 25], "acr": [], "across": [1, 3, 6, 9, 17, 19, 25], "act": [1, 3, 20], "action": 22, "activ": [0, 2, 3, 4, 9, 15, 21, 23, 25], "actual": [0, 1, 4, 5, 6, 8, 11, 15, 16, 18, 20, 22, 25, 26, 27], "ad": [1, 3, 4, 5, 8, 13, 15, 16, 20, 27], "ada_clf": 10, "adaboostclassifi": 10, "adadelta": 13, "adam": [1, 3, 4, 25], "adapt": [4, 6, 13, 17, 24, 27], "add": [0, 1, 2, 3, 4, 5, 6, 8, 10, 11, 12, 15, 16, 17, 18, 22, 23, 25, 26, 27], "add_subplot": [1, 7, 12, 14], "addendum": 5, "addit": [0, 2, 3, 5, 6, 7, 8, 9, 10, 12, 13, 15, 19, 20, 22, 23, 24, 25, 26], "addition": [12, 13, 27], "address": [1, 9, 11, 13, 25], "adjac": [3, 12], "adjoint": [5, 26], "adjust": [0, 5, 12, 13, 27], "admir": [0, 25], "advanc": [4, 6, 12, 24, 25], "advantag": [1, 3, 5, 6, 10, 13, 20, 27], "adversari": 25, "afecionado": 25, "affect": [3, 15], "affin": [0, 3, 8, 11, 26], "afford": 3, "aficionado": 25, "aforement": 14, "african": [], "after": [0, 1, 2, 4, 5, 6, 9, 11, 12, 13, 15, 16, 17, 18, 19, 20, 22, 25, 26, 27], "afterward": [0, 25], "ag": [0, 7, 25, 26], "ag_0": 2, "again": [0, 1, 4, 5, 6, 7, 8, 10, 11, 12, 13, 22, 25, 26, 27], "against": [1, 4, 7, 10], "agegroup": 7, "agegroupmean": 7, "aggreg": [9, 10], "agorithm": 10, "agre": [5, 6, 22, 26, 27], "agreement": 13, "ahead": 9, "ai": [0, 24], "aid": 11, "aim": [0, 1, 4, 6, 7, 11, 14, 16, 17, 19, 20, 26], "ainv": 5, "airplan": 3, "aka": 5, "al": [0, 2, 4, 16, 17, 24, 25, 26, 27], "alarm": [5, 7], "aldo": 26, "algebra": [0, 3, 5, 13, 19, 26, 27], "algorithm": [0, 1, 2, 4, 5, 6, 7, 8, 13, 14, 16, 19, 20, 22, 24], "align": [0, 2, 5, 6, 7, 8, 13, 22, 25, 26, 27], "all": [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, 14, 15, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27], "allevi": [1, 13, 27], "alloc": [3, 20], "allow": [0, 1, 2, 3, 5, 6, 8, 10, 13, 15, 19, 20, 25, 26, 27], "almost": [0, 1, 6, 8, 11, 13, 22, 27], "alon": [2, 9], "along": [2, 3, 4, 5, 6, 9, 10, 11, 15, 19, 20, 25, 26, 27], "alpha": [0, 1, 2, 3, 4, 6, 7, 8, 9, 10, 13, 14, 22, 25, 26, 27], "alpha_": 10, "alpha_0": 3, "alpha_1": 3, "alpha_2": 3, "alpha_i": [3, 13], "alpha_k": 13, "alpha_m": 10, "alpha_n": 3, "alpha_opt": 13, "alreadi": [2, 3, 4, 5, 6, 10, 12, 15, 19, 20, 22, 25, 26, 27], "also": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 19, 20, 22, 25, 26, 27], "alter": 1, "altern": [0, 1, 4, 5, 6, 8, 9, 11, 13, 15, 18, 20, 25, 26], "although": [0, 1, 5, 6, 8, 10, 13, 16, 25], "alwai": [0, 3, 5, 6, 12, 13, 16, 22, 25, 26, 27], "am": 4, "ame2016": [0, 25], "american": [], "among": [0, 3, 5, 9, 10, 12, 20, 25, 26], "amongst": 5, "amount": [0, 1, 3, 4, 6, 8, 10, 14, 19], "an": [1, 2, 3, 5, 6, 7, 8, 9, 11, 12, 13, 14, 16, 17, 18, 19, 20, 22, 23, 24, 26, 27], "an_": 22, "anaconda": [0, 1, 19, 25], "analogi": 13, "analys": 6, "analysi": [1, 3, 4, 7, 14, 20, 24], "analyt": [2, 3, 5, 6, 7, 12, 13, 17, 19, 25, 26, 27], "analyz": [0, 1, 3, 4, 5, 6, 16, 22, 26, 27], "andrew": 1, "angl": [0, 3, 9, 26], "anharmon": 3, "ani": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 14, 15, 16, 22, 25, 26], "anim": [4, 12], "ann": 12, "annot": [0, 1, 3, 7, 8, 25], "announc": 25, "anoth": [0, 1, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 15, 20, 22, 25, 26], "ansatz": [0, 25], "answer": [0, 1, 3, 5, 6, 20, 23, 25], "antialias": [2, 6], "anticip": 4, "anymor": [1, 8], "anyon": [4, 8, 15], "anyth": [1, 15, 16, 22], "anytim": [23, 25], "apach": 1, "apart": [11, 13, 27], "api": [1, 19, 25], "appar": 2, "appear": [0, 1, 3, 13, 20, 22], "append": [1, 3, 4, 8, 9, 13, 25], "appli": [0, 1, 3, 4, 6, 7, 8, 9, 10, 11, 12, 13, 18, 22, 24, 25, 26], "applic": [0, 1, 3, 4, 5, 6, 7, 9, 12, 13, 16, 20, 22, 24, 25, 26, 27], "apply_gradi": 4, "approach": [1, 2, 4, 5, 6, 9, 10, 11, 12, 13, 15, 16, 18, 19, 22, 24, 26, 27], "appropri": [2, 6, 9, 12, 13, 17, 19, 22], "approv": 25, "approx": [0, 2, 3, 6, 10, 11, 13, 18, 22, 25, 27], "approxim": [0, 1, 2, 3, 4, 5, 6, 7, 10, 11, 13, 22, 25, 26, 27], "apt": [0, 19, 25], "aq": 22, "ar": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27], "aragorn": 25, "arang": [1, 3, 4, 6, 7, 9, 10, 12, 13, 25], "arbitrari": [1, 4, 6, 8, 12, 13, 22, 27], "arbitrarili": [0, 1, 11, 25], "arc": 6, "architectur": [3, 4, 12], "area": [0, 3, 6, 24, 25], "argmax": [1, 11], "argmin": [4, 10, 14], "argsort": 11, "argu": [1, 13], "argument": [0, 2, 3, 5, 11, 12, 13, 17, 25, 26], "aris": [0, 6, 12, 13, 22, 25, 27], "arithmet": [0, 13, 20, 25], "arm": [6, 26], "armadillo": 20, "around": [0, 1, 4, 5, 6, 11, 18, 22, 25], "arrai": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 12, 13, 14, 16, 18, 19, 22, 26, 27], "arrang": [3, 25], "arraybox": 13, "arriv": [0, 6, 9, 11, 20, 22, 25], "arrow": 12, "arrowprop": 8, "art": [0, 1, 19], "articl": [0, 3, 4, 6, 10, 25, 26, 27], "artifici": [0, 2, 7, 12, 24, 25], "artificialneuron": 12, "arug": 13, "arxiv": [3, 4], "asarrai": [0, 6, 9, 26], "asid": 26, "ask": [5, 6, 11, 12, 15], "aspect": [0, 6, 19, 25, 26], "assembl": 3, "assembli": [0, 25], "assert": 4, "assess": [0, 6, 25, 26], "assici": 4, "assign": [0, 7, 8, 9, 12, 13, 14, 15, 21, 23, 24, 25], "associ": [0, 6, 9, 12, 14, 22, 25], "assum": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 17, 20, 22, 25, 26, 27], "assumpt": [0, 3, 5, 6, 9, 11, 22, 25, 26], "ast": [0, 5, 6, 25], "astyp": [4, 9, 10], "asymmetri": [0, 25], "asymptot": [4, 6], "atom": [0, 25], "attempt": [0, 4, 6, 7, 8, 10, 25, 26], "attend": 25, "attent": [0, 20, 25], "attract": [0, 10, 25], "attribut": [0, 9, 25], "audi": [0, 25], "audio": [3, 4], "august": [25, 26], "aurelien": [0, 24, 25], "austfjel": 6, "auth": 15, "authent": 15, "author": [0, 1, 10, 22], "authour": 25, "auto": [9, 10, 22], "auto_exampl": 26, "autocor": 22, "autocorrelation_tim": 22, "autocorrelform": 22, "autocovari": 22, "autoencod": [4, 19, 25], "autoencond": 19, "autograd": [19, 25], "autom": [0, 19, 24, 25], "automac": 20, "automag": 25, "automat": [0, 1, 2, 3, 4, 11, 16, 19, 20, 25], "automobil": 3, "autonom": 4, "avail": [0, 1, 4, 6, 10, 11, 19, 20, 21, 23, 24, 25], "averag": [0, 1, 3, 6, 9, 10, 13, 14, 22, 23, 25, 26], "avoid": [0, 4, 5, 6, 9, 11, 13, 18, 20, 26], "awai": [2, 3, 6, 26], "awar": [2, 10], "award": [23, 25], "ax": [0, 1, 2, 3, 4, 6, 7, 8, 9, 10, 11, 12, 13, 14, 20, 25], "axes3d": [2, 6, 13, 27], "axes_grid1": 6, "axhlin": 8, "axi": [0, 1, 2, 3, 4, 6, 7, 8, 9, 10, 11, 12, 13, 14, 18, 20, 22, 25, 26, 27], "axiom": 5, "axvlin": [4, 8], "axvspan": 4, "b": [0, 1, 3, 4, 5, 6, 8, 9, 10, 12, 13, 14, 15, 16, 17, 22, 23, 25, 26, 27], "b1": 8, "b2": 8, "b3": 8, "b_": [0, 1, 20], "b_0": 0, "b_1": [0, 2, 12, 13], "b_2": [0, 13], "b_5": 13, "b_group": 9, "b_i": [0, 1, 2, 12, 25], "b_ia_": [0, 25], "b_ia_i": 0, "b_index": 9, "b_j": [1, 12], "b_k": [0, 1, 12, 13], "b_m": 12, "b_score": 9, "b_valu": 9, "babcock": 25, "bachelor": [21, 23], "back": [0, 3, 4, 5, 6, 8, 9, 10, 15, 16, 20, 22, 25], "backbon": 20, "backend": [1, 4], "background": [24, 25], "backpropag": 1, "backtrack": 9, "backup": 20, "backward": [1, 2, 4, 12, 20], "bad": [6, 17, 26], "badli": 22, "bag": [9, 19, 25], "bag_clf": 10, "baggin": 25, "baggingboot": 10, "baggingclassifi": 10, "baggingtre": 10, "balanc": 6, "ballpark": 18, "band": 20, "bandwidth": 20, "bar": [0, 6, 11, 25], "barber": 24, "bare": [4, 10], "base": [0, 1, 3, 4, 5, 7, 8, 9, 10, 14, 15, 16, 17, 19, 22, 23, 24, 25, 26, 27], "basi": [5, 7, 8, 10, 11, 12, 13, 20, 26, 27], "basic": [6, 8, 12, 13, 14, 15, 19, 22, 25], "batch": [3, 4, 11, 12, 13, 27], "batch_shap": 4, "batch_siz": [1, 3, 4], "batchnorm": 4, "bay": 7, "bayesian": [5, 19, 24, 25], "becaus": [0, 1, 2, 3, 4, 5, 6, 8, 9, 12, 13, 14, 25, 26, 27], "becom": [0, 1, 2, 5, 6, 7, 9, 12, 13, 22, 25, 26, 27], "been": [0, 1, 2, 3, 4, 5, 6, 11, 12, 13, 19, 20, 25, 26], "befor": [0, 1, 2, 3, 4, 5, 6, 7, 8, 12, 13, 14, 16, 17, 18, 20, 22, 25, 26], "beforehand": [0, 22, 25], "begin": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 14, 15, 20, 22, 23, 25, 26, 27], "behav": [1, 6, 13, 27], "behavior": [0, 1, 13, 25, 27], "behaviour": 12, "behind": [0, 1, 6, 8, 13, 25, 27], "being": [0, 1, 2, 3, 4, 5, 7, 8, 10, 11, 12, 13, 17, 22, 25, 26, 27], "believ": [9, 20], "belong": [7, 8, 9, 13, 14, 27], "below": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 18, 20, 22, 25, 26, 27], "benchmark": 10, "benefici": [1, 13], "benefit": [0, 1, 4, 11, 13, 19, 25, 27], "bengio": [1, 24, 25, 26], "benign": [1, 7], "besid": [4, 5, 27], "bessel": [5, 26], "best": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 15, 16, 18, 23, 25, 26, 27], "beta": [1, 3, 10, 11, 13, 16, 17, 25, 26, 27], "beta_": [3, 13, 17, 26], "beta_0": [1, 3, 13, 26], "beta_1": [1, 3, 10, 13, 26], "beta_1x_i": 13, "beta_2": [3, 13], "beta_3": 3, "beta_i": 3, "beta_j": [13, 26], "beta_k": 13, "beta_linreg": 13, "beta_m": 10, "beta_mg_m": 10, "beta_n": 3, "better": [0, 1, 2, 3, 4, 6, 9, 10, 11, 12, 13, 25, 26], "between": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 14, 15, 16, 17, 22, 25, 26, 27], "beyond": [0, 1, 5, 6, 8, 13, 25, 26, 27], "bf": [13, 14, 20, 22, 27], "bg": 25, "bgd": 13, "bia": [0, 1, 2, 3, 5, 8, 9, 10, 12, 13, 25, 26, 27], "bias": [1, 2, 3, 5, 6, 9, 12], "big": [0, 1, 2, 5, 6, 14], "bigger": [1, 6, 26], "bigr": 12, "bike": 9, "bilbo": 25, "billion": [3, 12, 19], "bin": [7, 22], "binari": [0, 3, 5, 7, 9, 10, 12, 25], "binarycrossentropi": 4, "bind": 0, "binomi": [19, 22, 25], "binsboot": 6, "bioinformat": 0, "biolog": [1, 12], "bios1100": [19, 25], "bird": [0, 3], "birth": 25, "bishop": [24, 25], "bit": [1, 4, 20, 22, 25], "bitwis": 22, "bivari": 2, "bk": 13, "bla": [20, 25], "black": [8, 9, 14], "block": [6, 10, 19, 20, 22, 25], "blog": 25, "blogpost": 4, "blue": [0, 3], "bm": [], "bmatrix": [0, 1, 3, 5, 7, 8, 11, 13, 20, 25, 26, 27], "bmi": 1, "bodi": [0, 1, 4, 12], "bold": 1, "boldfac": [0, 5, 16, 26, 27], "boldsymbol": [0, 1, 2, 3, 5, 6, 7, 8, 10, 11, 13, 14, 16, 17, 25, 27], "boltzmann": [12, 19, 25], "book": [17, 24, 25, 26], "book1": 24, "boolean": [4, 17], "boost": [1, 9, 19, 25], "boostrap": 10, "bootstrap": [1, 13, 19, 25], "borrow": 25, "boston_dataset": [], "bot": 8, "both": [0, 1, 4, 5, 6, 8, 9, 10, 13, 14, 15, 16, 17, 19, 20, 22, 23, 25, 26, 27], "bottl": 7, "bound": [8, 12], "boundari": [2, 4, 8, 11, 12], "box": [4, 9], "boyd": [8, 13, 27], "bracket": [4, 22], "brain": [1, 7, 12], "branch": [9, 25], "break": [0, 4, 6, 11, 14, 25], "breast": [5, 7, 11], "breviti": 13, "brew": [0, 19, 25], "brg": 8, "brief": 26, "briefli": [0, 16, 25], "bring": [0, 5, 6, 10, 26], "britt": [23, 25], "broad": 0, "broadli": 25, "brought": [13, 19, 25], "brownle": 4, "browser": [15, 25], "brute": [3, 5, 11, 26], "buffer_s": 4, "bui": 4, "build": [0, 4, 5, 6, 10, 16, 20, 22, 25], "built": [1, 3, 4, 6], "bunch": 11, "busi": [], "byte": [20, 25], "c": [0, 1, 2, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 19, 20, 21, 22, 23, 24, 26, 27], "c1": [8, 11], "c2": [8, 11], "c_": [8, 9, 10, 13, 22, 27], "c_0": 22, "c_1": 12, "c_2": 12, "c_3": 12, "c_4": 12, "c_i": [12, 13], "c_k": 22, "ca": [1, 25], "cach": 10, "cal": [0, 8, 10, 12, 13, 27], "calcul": [0, 1, 2, 4, 5, 6, 8, 9, 10, 11, 12, 13, 14, 16, 20, 22, 25], "call": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 19, 20, 22, 23, 25, 26, 27], "calor": [0, 26], "cambridg": [13, 24, 27], "can": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 23, 24, 26, 27], "cancel": [0, 13, 25, 26], "cancer": [5, 10], "cancerpd": 7, "candid": [8, 9, 10], "cannot": [0, 1, 4, 5, 6, 7, 8, 9, 22, 26, 27], "canopi": [0, 19, 25], "canva": [15, 16, 25], "cap": 5, "capabl": [0, 1, 8, 13, 19, 25], "capac": [2, 23], "capita": [], "captur": [4, 11, 12, 25], "car": [3, 4], "card": [0, 7, 25], "cardin": 1, "care": [11, 15], "carefulli": 13, "carlo": [0, 6, 19, 22, 24, 25], "carri": [2, 6, 7], "cart": 10, "case": [0, 1, 2, 3, 4, 5, 6, 7, 11, 12, 13, 14, 15, 16, 19, 20, 25], "casella": 24, "cast": 1, "cat": [3, 4], "catch": 0, "categor": [0, 1, 3, 9, 11, 25], "categori": [0, 1, 3, 7, 10, 12, 14, 25], "categorical_crossentropi": [1, 3], "caus": [0, 5, 6, 22, 25, 26, 27], "causal": 0, "causat": [0, 25], "cax": 1, "cb": [6, 25], "cbar": 1, "cc": [0, 1, 5, 13, 25, 26, 27], "ccc": [5, 12, 27], "cdf": 22, "cdot": [0, 2, 6, 12, 13, 14, 20, 22, 25, 27], "celebr": [13, 27], "cell": 4, "center": [0, 1, 6, 7, 8, 9, 11, 14, 18, 22, 25, 26], "central": [0, 3, 5, 6, 8, 16, 20, 25, 26], "centroid": [14, 22], "centroid_differ": 14, "centuri": 3, "certain": [0, 3, 6, 7, 9, 22, 25, 26], "cg": 13, "cha": [], "chain": [0, 1, 13, 19, 22, 25], "challeng": 15, "chanc": [1, 5, 13, 22], "chang": [0, 1, 2, 3, 4, 5, 6, 8, 9, 11, 12, 13, 14, 15, 16, 20, 22, 25, 26, 27], "channel": 3, "chapter": [0, 6, 10, 11, 16, 17, 20, 24, 25, 26, 27], "chapter3": 0, "charact": [0, 3, 5, 25, 26, 27], "character": [8, 9, 10, 12, 22], "characterist": [0, 1, 3, 10, 13, 25], "charg": [0, 25], "charl": [], "chase": 4, "chatgpt": 15, "chd": 7, "chddata": 7, "cheap": [5, 26, 27], "cheaper": [1, 13], "check": [1, 3, 4, 5, 11, 13, 15, 16, 20, 25], "checkmark": 3, "checkpoint": 4, "checkpoint_dir": 4, "checkpoint_prefix": 4, "chen": 10, "cheng": 26, "chiaramont": 2, "childcar": 16, "children": 16, "choic": [0, 1, 2, 3, 4, 6, 9, 12, 13, 14, 20, 25, 26, 27], "choleski": [5, 20, 26, 27], "choos": [2, 3, 6, 9, 10, 11, 13, 14, 15, 27], "chosen": [0, 1, 2, 6, 8, 9, 10, 13, 16, 22, 25, 27], "chosen_datapoint": 1, "christian": 24, "christoph": [24, 25], "cifar": 3, "cifar10": 3, "circ": [1, 12], "circl": [0, 8, 12, 26], "circuit": 3, "circumfer": 9, "circumv": [1, 5, 13, 26, 27], "ckpt": 4, "clariti": 22, "class": [0, 1, 3, 4, 6, 7, 8, 9, 11, 12, 13, 22, 25], "class_nam": [3, 9], "class_val": 9, "class_valu": 9, "classic": [7, 9, 13], "classif": [0, 3, 5, 6, 7, 8, 11, 12, 19, 24, 25, 26], "classifi": [0, 1, 4, 7, 9, 10, 11, 25], "classificaton": 1, "classifii": 10, "clean": 1, "clear": [1, 5, 10, 12, 13], "clearli": [0, 3, 5, 6, 7, 8, 22, 26, 27], "clever": [1, 10], "clf": [0, 6, 8, 9, 10, 25, 26], "clf3": 0, "clf_lasso": 6, "clf_ridg": 6, "cli": 15, "clip": [3, 22], "clone": [15, 23], "close": [0, 1, 2, 4, 6, 8, 9, 11, 12, 13, 14, 18, 22, 24, 25, 27], "closer": [3, 5, 13, 26, 27], "closest": [8, 11, 13, 14], "closur": [19, 25], "cloud": [19, 25], "cluster": [0, 1, 4, 6, 11, 19, 25], "cluster_label": 14, "cm": [1, 2, 3, 6, 8, 13, 27], "cmap": [0, 1, 2, 3, 4, 6, 8, 9, 10, 25], "cmap_arg": 6, "cmd": [9, 15], "cn_": 22, "cnn": 12, "cnn_kera": 3, "cntk": [19, 25], "co": [0, 2, 3, 6, 9, 13, 25], "code": [0, 3, 4, 6, 7, 8, 18, 19, 20, 22, 24], "coef": [0, 25], "coef0": 8, "coef_": [0, 5, 6, 8, 9, 13, 16, 25, 26, 27], "coeff": 5, "coeffici": [0, 3, 5, 6, 7, 8, 9, 13, 18, 20, 25, 26], "coerc": [0, 6, 25], "coin": [10, 22], "coin_toss": 10, "col": [0, 11, 25, 26], "colab": [19, 25], "cold": 9, "colinear": [], "collaps": 8, "collect": [2, 6, 10, 11, 17, 19, 22, 24, 25], "collinear": [5, 26, 27], "color": [0, 3, 4, 6, 8, 9, 10, 22], "color_channel": 3, "color_cod": 6, "colorbar": [1, 6], "colsample_bytre": 10, "colsaobject": 10, "column": [0, 1, 2, 5, 6, 7, 8, 9, 11, 12, 16, 17, 18, 20, 25, 26, 27], "columntransform": 9, "com": [4, 6, 15, 16, 19, 24, 25, 27], "combin": [1, 2, 5, 6, 7, 10, 15, 18, 22], "come": [0, 1, 3, 4, 5, 12, 13, 14, 15, 25, 26, 27], "command": [0, 1, 15], "comment": [0, 4, 5, 6], "commerci": [0, 19, 25], "commit": 15, "commod": [0, 25], "common": [0, 1, 3, 5, 6, 7, 9, 11, 13, 14, 16, 22, 25, 26, 27], "commonli": [0, 1, 4, 6, 7, 9, 13, 14, 26], "commun": [0, 12, 15], "commut": 3, "commutatitav": 3, "compact": [0, 1, 3, 5, 6, 7, 9, 11, 12, 13, 14, 25, 26], "compair": 0, "compar": [0, 3, 4, 5, 6, 11, 13, 18, 20, 25, 26, 27], "comparison": [2, 4, 13], "compat": 7, "compet": 0, "competit": 10, "compil": [0, 1, 3, 4, 13, 19, 20, 25], "complet": [0, 2, 3, 4, 9, 12, 15, 16, 17, 18, 25], "completenn": 12, "complex": [1, 5, 8, 9, 11, 12, 13, 16, 25, 27], "complic": [0, 1, 9, 13, 25, 27], "compoment": 26, "compon": [0, 1, 3, 4, 5, 6, 7, 9, 14, 16, 19, 25, 26, 27], "components_": 11, "compos": [9, 12, 13, 14, 19, 25], "compphys": [0, 6, 16, 19, 21, 23, 24, 25, 26], "compress": [0, 25, 26], "compris": 6, "compromis": [5, 26, 27], "compulsori": [19, 25], "comput": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 15, 16, 17, 18, 19, 20, 21, 22, 24, 25, 26, 27], "computation": [0, 3, 6, 9, 13, 22, 25, 27], "computationalscienceuio": 25, "concaten": [2, 4, 6, 14], "concav": [1, 13, 26, 27], "concentr": 10, "concept": [0, 2, 19, 25, 26], "conceptu": [12, 13, 27], "concern": [0, 1, 4, 7, 25, 27], "concic": 25, "conclud": [0, 5, 13], "conclus": 1, "cond": 2, "conda": [0, 1, 19, 25], "condis": 26, "condit": [0, 2, 4, 5, 6, 8, 9, 11, 13, 22, 25, 26], "conduct": 19, "condwav": 2, "confid": [0, 5, 6, 7, 8, 25, 26], "configur": 3, "confirm": [5, 12], "confus": [5, 6, 7, 10, 20, 26], "confusion_matrix": 9, "congruenti": 22, "conjug": [4, 8], "conjugaci": 13, "conjunct": 3, "connect": [0, 1, 3, 4, 9, 11, 12, 13, 20, 25, 26, 27], "consequ": [5, 6, 8, 10, 12, 13, 26, 27], "conserv": [5, 14, 26, 27], "consid": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 12, 13, 16, 20, 22, 25, 26, 27], "consider": [0, 1, 5, 13, 25, 26, 27], "consist": [1, 2, 3, 4, 6, 12, 13, 22, 26, 27], "constant": [0, 2, 4, 5, 6, 8, 12, 13, 16, 18, 22, 25, 26, 27], "constitu": [0, 25], "constitut": [2, 6], "constrain": [1, 3, 5, 7, 11, 27], "constraint": [5, 6, 8, 13, 26, 27], "construct": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 20, 22, 25, 26], "contact": [0, 25], "contain": [0, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 15, 18, 20, 22, 24, 25, 26, 27], "contemporari": 25, "content": [1, 15, 19, 20, 25, 27], "context": [6, 10, 13, 27], "contigu": 20, "continu": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 20, 22, 25, 26, 27], "contour": [9, 10, 13], "contourf": [8, 9, 10], "contrast": [1, 4, 9, 10, 12, 25], "contribut": [0, 3, 5, 13, 18, 22, 25, 26, 27], "contributor": 0, "control": [0, 1, 3, 9, 13, 15, 19, 25], "conv": [3, 4], "conv2d": [3, 4], "conv2dtranspos": 4, "convei": 25, "conveni": [5, 6, 12, 13, 20, 25, 27], "convent": [12, 26], "converg": [1, 2, 4, 5, 8, 13, 14, 18, 26, 27], "convergencewarn": [], "convert": [0, 1, 4, 5, 9, 11, 13, 20, 25, 26, 27], "converttomatrix": 4, "convex": [4, 5, 7, 26], "convinc": [13, 27], "convolut": [1, 4, 19, 25], "cool": [4, 9], "coolwarm": 6, "coordin": [5, 12, 14, 26, 27], "coorel": [], "copi": [0, 1, 14, 15, 26], "core": 10, "corel": 25, "coronari": 7, "corr": [5, 7, 11, 26], "correalt": [11, 19], "correct": [0, 1, 2, 3, 4, 5, 7, 13, 15, 20, 22, 25, 26, 27], "correctli": [1, 2, 6, 7, 10, 18], "correl": [0, 1, 3, 5, 6, 7, 10, 12, 13, 19, 22, 25, 27], "correlation_matrix": [5, 7, 11, 26], "correspond": [0, 3, 5, 6, 8, 9, 11, 12, 19, 20, 22, 25, 26, 27], "cortex": 12, "cosin": [3, 6], "cost": [0, 2, 3, 5, 6, 7, 8, 9, 12, 13, 16, 17, 18, 25], "cost_deep_grad": 2, "cost_funct": 2, "cost_function_deep": 2, "cost_function_deep_grad": 2, "cost_function_grad": 2, "cost_grad": 2, "cost_histori": 18, "cost_ol": 18, "cost_ridg": 18, "cost_sum": 2, "costol": 13, "could": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16, 17, 18, 20, 22, 25, 26, 27], "coulomb": [0, 25], "count": [0, 9, 15, 21, 22, 23, 25], "counterpart": 25, "countor": 13, "coupl": [4, 5, 6], "cours": [0, 1, 3, 5, 11, 15, 16, 17, 23, 26], "coursework": 15, "courvil": [24, 25, 26], "cov": [5, 6, 11, 20, 22, 25, 26], "cov_xi": [5, 11, 26], "cov_xx": [5, 11, 26], "cov_yi": [5, 11, 26], "covari": [0, 7, 19, 20, 25, 27], "covariance_matrix": [5, 11, 14], "cover": [0, 5, 19, 23, 24, 26, 27], "covert": [0, 25], "covxi": 22, "covxx": 22, "covxz": 22, "covyi": 22, "covyz": 22, "covzz": 22, "cpu": 1, "craft": 3, "creat": [1, 3, 4, 5, 9, 10, 11, 12, 15, 18, 19, 25], "create_biases_and_weight": 1, "create_convolutional_neural_network_kera": 3, "create_neural_network_kera": 1, "create_x": [5, 11], "credit": [0, 7, 23, 25], "crim": [], "crime": [], "criteria": [0, 4, 9, 10, 14, 22, 25], "criterion": [9, 10, 13, 18, 27], "critic": [6, 26], "cross": [0, 1, 3, 7, 9, 10, 13, 15, 19, 22, 25, 26, 27], "cross_entropi": 4, "cross_val_scor": 6, "cross_valid": [7, 10], "crossvalid": 6, "crucial": [1, 22], "cs231": 3, "csr_matrix": [20, 25], "csv": [0, 4, 6, 7, 9], "ctnk": 1, "cubic": 0, "cumbersom": 5, "cumsum": [10, 11, 25], "cumul": [7, 10, 22], "cumulative_heads_ratio": 10, "cup": 5, "current": [1, 2, 3, 4, 13, 14, 15, 16, 24, 27], "curs": [0, 26], "curv": [6, 7, 10, 12], "curvatur": [13, 27], "custom": [6, 14], "custom_cmap": [9, 10], "custom_cmap2": [9, 10], "cutpoint": 9, "cv": [6, 7, 10], "cvxbook": [13, 27], "cvxopt": [5, 8, 26], "cycl": [1, 12], "d": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 16, 17, 20, 22, 23, 25, 26, 27], "d2_g_t": 2, "d_f": [13, 27], "d_g_t": 2, "d_net_out": 2, "da": 3, "dagger": [5, 20, 26, 27], "dai": [1, 9, 19], "damp": 3, "darget": 9, "darkr": 22, "dat": [0, 25], "dat_id": [0, 6, 7, 9, 25], "data": [2, 4, 5, 8, 10, 12, 13, 14, 16, 20, 24, 27], "data1": 14, "data2": 14, "data3": 14, "data4": 14, "data_id": [0, 6, 7, 9, 25], "data_indic": 1, "data_panda": 25, "data_path": [0, 6, 7, 9, 25], "databas": 1, "datafil": [0, 6, 7, 9, 25], "datafram": [0, 4, 5, 7, 9, 11, 25, 26], "datapoint": [1, 5, 6, 7, 11, 13, 16, 27], "datasci": [15, 16], "dataset": [0, 4, 6, 7, 8, 9, 10, 11, 13, 14, 16, 25, 27], "datatyp": 4, "date": [15, 18, 25, 26, 27], "daughter": 10, "david": 24, "dbh": 1, "dbo": 1, "dcomposit": 20, "ddot": 2, "dead": 1, "deadlin": 15, "deal": [0, 1, 3, 5, 6, 8, 11, 13, 14, 20, 22, 25, 26, 27], "dealt": 0, "debt": 7, "debug": [0, 5, 6, 26, 27], "decad": [0, 3], "decai": [0, 13, 22, 25], "decemb": [23, 25], "decent": 10, "decid": [0, 2, 3, 5, 6, 9, 18, 26, 27], "decim": [0, 25], "decis": [0, 1, 8, 11, 19, 24, 25], "decision_funct": 8, "decision_tre": 9, "decisiontreeclassifi": [9, 10], "decisiontreeregressor": [0, 9, 10], "declar": [0, 4, 20, 25], "decompos": [5, 6, 20, 26, 27], "decomposit": [0, 6, 12, 25], "decompost": [5, 26, 27], "deconvolut": 3, "decorrel": [10, 13], "decreas": [1, 2, 4, 5, 6, 10, 11, 13, 27], "deduc": [0, 25], "deep": [3, 7, 12, 13, 19, 24, 26, 27], "deep_neural_network": 2, "deep_param": 2, "deep_tree_clf": [9, 10], "deep_tree_clf1": 9, "deep_tree_clf2": 9, "deepen": [5, 19, 25], "deeper": [0, 3, 4, 25], "deeplearningbook": [24, 25, 27], "deer": 3, "def": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 16, 17, 22, 25, 26, 27], "def_covari": 22, "default": [0, 1, 2, 4, 6, 7, 20, 25, 26], "default_tim": 4, "defect": [5, 26, 27], "defici": [5, 26, 27], "defin": [0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 18, 20, 22, 26, 27], "definit": [1, 2, 5, 6, 7, 8, 10, 11, 12, 13, 20, 22, 26, 27], "defint": 22, "degre": [3, 5, 6, 8, 9, 10, 11, 15, 16, 22, 25, 27], "deisenroth": 26, "del": 1, "delet": [6, 15], "delimit": 4, "deliv": [15, 21, 25], "delta": [0, 2, 3, 6, 8, 12, 13, 14, 25], "delta_": [1, 20], "delta_0": 3, "delta_1": 3, "delta_2": 3, "delta_3": 3, "delta_4": 3, "delta_5": 3, "delta_h": [0, 1, 25], "delta_j": [3, 12], "delta_k": 12, "delta_l": [1, 3], "delta_momentum": 13, "delta_n": [0, 3, 25], "delug": 19, "delv": 0, "demand": [13, 27], "demonstr": [0, 3, 5, 6, 7, 11, 12, 19, 25, 26, 27], "den": 4, "denomin": [1, 5], "denot": [1, 2, 6, 7, 13, 22, 27], "dens": [1, 3, 4], "densiti": [0, 2, 6, 22], "depart": [23, 25, 26, 27], "depend": [0, 1, 2, 4, 5, 6, 7, 8, 11, 12, 13, 15, 16, 19, 20, 22, 25, 26, 27], "depict": 22, "deploy": [0, 19, 25], "depth": [0, 3, 9, 10, 20], "deriv": [0, 1, 2, 6, 7, 8, 10, 11, 13, 18, 19, 25], "derivati": 13, "derivative_fn": 13, "descend": [5, 9, 11, 26, 27], "descent": [0, 1, 3, 7, 8, 12, 25, 26], "describ": [0, 2, 4, 5, 6, 8, 10, 11, 12, 13, 20, 25], "descript": [0, 8, 9, 25], "design": [0, 1, 3, 4, 5, 6, 7, 10, 11, 12, 13, 17, 25, 27], "designmatrix": [0, 25], "desir": [0, 2, 4, 5, 13, 14, 25, 26, 27], "desktop": 15, "despit": [1, 12], "destroi": 20, "det": [5, 20, 26, 27], "detail": [0, 6, 11, 13, 14, 18, 20, 26, 27], "detect": [3, 8, 12], "determin": [0, 2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 13, 18, 20, 22, 25, 26, 27], "determinist": [7, 13, 22, 27], "dev": 1, "develop": [0, 3, 5, 8, 10, 11, 12, 19, 20, 25, 26], "deviat": [0, 1, 2, 4, 5, 6, 17, 18, 22, 25, 26], "devis": 12, "df": [4, 8, 11, 13, 25], "df1": 25, "di": [], "diag": [5, 8, 26, 27], "diagnost": [1, 10], "diagon": [0, 5, 7, 13, 18, 20, 22, 25, 26, 27], "diagonaliz": [5, 26, 27], "diagram": 10, "diagsvd": 6, "dice": [6, 22], "dict": [6, 8], "dictionari": [], "did": [0, 1, 5, 6, 7, 10, 11, 14, 16, 25], "die": 1, "diff": 2, "diff1": 2, "diff2": 2, "diff_ag": 2, "diffeent": 8, "differ": [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 24, 25, 26, 27], "differenti": [0, 3, 16, 19, 20, 25, 26, 27], "difficult": [0, 1, 6, 10, 13, 22, 25], "difficulti": [0, 1, 13, 25, 27], "diffonedim": 2, "digit": [0, 1, 3, 4, 6, 23, 25], "dilemma": 13, "dilut": 1, "dim": [4, 11, 14, 20], "dimens": [0, 1, 2, 3, 4, 5, 8, 11, 14, 16, 20, 25, 26, 27], "dimension": [0, 4, 5, 6, 9, 11, 13, 14, 19, 20, 25, 26, 27], "dimensionless": [0, 3, 25], "diment": 20, "dimnsion": 4, "diod": 3, "direct": [0, 1, 2, 4, 11, 12, 13, 14, 25, 26, 27], "directli": [1, 4, 5, 6, 18, 22, 26, 27], "disadvantag": [0, 25], "disappear": [3, 6], "disc_loss": 4, "disc_tap": 4, "discard": [6, 11], "disciplin": [0, 3, 12], "disclaim": 22, "discord": 25, "discourag": [13, 15, 27], "discov": [0, 25], "discover": 5, "discret": [1, 3, 5, 7, 13], "discrimin": [4, 7, 10, 11], "discriminator_loss": 4, "discriminator_loss_list": 4, "discriminator_model": 4, "discriminator_optim": 4, "discuss": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 18, 19, 20, 22, 24, 25, 26, 27], "diseas": 7, "disguis": [6, 26], "disord": [1, 7], "displai": [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 22, 25, 26], "displaystyl": [0, 5, 17, 25, 26, 27], "disregard": [0, 25], "dissimilar": [11, 14], "dist": 14, "distanc": [8, 9, 11, 14, 22], "distance_list": 9, "distinct": [3, 7, 8, 9, 10, 14], "distinctli": 8, "distinguish": [0, 4, 7, 8, 22, 25], "distplot": [], "distribut": [0, 1, 4, 6, 7, 10, 11, 13, 14, 18, 19, 20, 25, 26, 27], "distrubut": [0, 19, 25], "dive": [0, 8, 20, 25], "diverg": [1, 13, 27], "divid": [0, 1, 3, 5, 6, 7, 8, 9, 11, 12, 18, 22, 25, 26], "divis": [6, 8, 9, 13, 18, 20, 22], "dna": 7, "dnn": [0, 1, 2, 4, 12, 25], "dnn1": 4, "dnn2_gru2": 4, "dnn_kera": 1, "dnn_model": 1, "dnn_numpi": 1, "dnn_scikit": [0, 1, 25], "do": [0, 2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 13, 14, 15, 16, 20, 25, 26, 27], "doc": [0, 15, 16, 19, 21, 23, 24, 25], "document": [4, 13, 15], "doe": [0, 1, 2, 3, 4, 5, 6, 8, 10, 11, 12, 13, 15, 16, 17, 18, 20, 22, 25], "doesn": [3, 9, 12, 25], "dog": [1, 3, 4], "domain": [5, 8, 13, 27], "domin": [0, 25], "don": [0, 1, 3, 5, 6, 8, 11, 13, 15, 16, 19, 25, 26], "done": [0, 2, 3, 4, 5, 6, 9, 10, 11, 13, 16, 20, 25, 26, 27], "dot": [0, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 18, 20, 22, 25, 26, 27], "doubl": [3, 4, 16, 20, 25], "doubli": 1, "down": [0, 3, 6, 9, 11, 12, 13, 27], "download": [0, 1, 3, 5, 6, 15, 20, 24, 25], "downsampl": 3, "dozen": 1, "dq": 6, "drag": 13, "dramat": 11, "drastic": 4, "draw": [4, 6, 10, 13, 27], "drawback": [0, 1, 3, 13, 26, 27], "drawn": [1, 4, 6, 7, 11, 22, 25], "drive": [3, 4], "driven": 3, "drop": [0, 1, 5, 6, 11, 13, 22, 25, 26, 27], "dropna": [0, 6, 25], "dropout": 4, "dt": [2, 3, 13, 22], "dtype": [0, 1, 3, 4, 14, 20, 25], "dub": [0, 25], "due": [1, 2, 5, 6, 8, 10, 12, 13, 18, 23, 25, 26, 27], "dummi": [], "dure": [0, 1, 3, 4, 8, 9, 11, 19, 25], "dwell": [], "dwh": 1, "dwo": 1, "dx": [2, 3, 8, 22], "dx_1": 22, "dx_1p": 6, "dx_2p": 6, "dx_mp": 6, "dx_n": 22, "dxp": 6, "dy": [1, 8, 22], "dynam": 4, "dz": 8, "e": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 22, 23, 25, 26, 27], "e_": [0, 2, 25], "each": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 19, 20, 21, 22, 23, 25, 26, 27], "eapprox": [0, 25], "earli": [1, 13], "earlier": [0, 5, 7, 8, 9, 11, 12, 13, 25, 26], "earthexplor": 6, "eas": [6, 9, 14], "easi": [0, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 19, 20, 25, 26, 27], "easier": [5, 6, 8, 9, 13, 15, 22, 25, 26, 27], "easiest": 13, "easili": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 20, 25, 26, 27], "eastern": [23, 25], "ebind": [0, 25], "eblock": 9, "econometr": 25, "economi": 5, "ecosystem": [19, 25], "ect": 21, "edg": 3, "edgecolor": 6, "editor": 15, "edu": [13, 27], "educ": [0, 25], "eff": 22, "effect": [1, 4, 10, 13, 16, 17, 18, 22], "effic": 1, "effici": [0, 3, 10, 13, 19, 20, 22, 25], "efron": 6, "egrad": 13, "eig": [5, 11, 13, 20, 22, 25, 26, 27], "eigen": 22, "eigenpair": [5, 11, 26, 27], "eigenvalu": [0, 5, 8, 11, 13, 20, 25, 26, 27], "eigenvector": [5, 11, 13, 26, 27], "eight": [20, 25], "eigval": [20, 22, 25], "eigvalu": [11, 13, 27], "eigvec": [20, 22, 25], "eigvector": [11, 13, 27], "eir": [23, 25], "eispack": [20, 25], "either": [1, 5, 6, 7, 8, 9, 10, 11, 13, 18, 22, 25, 26, 27], "eivind": 23, "eivinsto": 23, "ekstr\u00f8m": 4, "elabor": 22, "elarn": 3, "electr": [0, 3, 12, 25], "electron": 25, "eleg": 11, "element": [1, 2, 3, 4, 5, 6, 7, 8, 11, 12, 13, 19, 20, 24, 26], "elementari": [10, 13, 20], "elementwis": [3, 13], "elementwise_grad": [2, 13], "elessar": 25, "elif": 14, "elim": 20, "elimin": [3, 8], "elin": [23, 25], "ellipsi": 16, "els": [1, 3, 4, 7, 9, 12, 13, 16, 20], "elu": 1, "elus": [0, 25], "email": [21, 23, 25], "embed": [0, 11, 26], "embodi": 6, "emit": 22, "emner": 24, "emphas": [0, 10, 19, 25], "emphasi": [0, 19, 24, 25], "empir": [1, 11, 22], "emploi": [0, 1, 5, 6, 11, 13, 22, 25, 26, 27], "employ": 0, "empti": [6, 10, 15], "emul": 12, "en": [19, 24], "enabl": 11, "enbodi": 6, "encod": [0, 3, 5, 9, 11, 14, 25, 26, 27], "encompass": [0, 22], "encount": [0, 1, 5, 7, 13, 15, 22, 25, 26, 27], "encourag": 15, "end": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 20, 22, 23, 25, 26, 27], "endpoint": [3, 6], "energi": [0, 4, 6], "enforc": 12, "eng": 24, "engin": [0, 1, 3, 4, 19, 25], "enorm": 3, "enough": [0, 6, 13, 25, 27], "ensembl": [1, 9, 25], "ensur": [0, 1, 2, 3, 5, 6, 11, 13, 18, 22, 26, 27], "entail": 25, "enter": [5, 6, 26, 27], "enthought": [0, 19, 25], "entir": [1, 3, 7, 9, 19, 22, 25], "entiti": [9, 12, 20, 25], "entri": [0, 5, 8, 11, 12, 20, 25, 26], "entropi": [1, 3, 7, 10, 13, 25, 27], "enumer": [0, 1, 2, 3, 4, 6, 8, 25, 26], "env": 22, "environ": [2, 19, 25], "environemnt": 15, "eo": [0, 6], "eol": 0, "eosfit": 0, "epoch": [0, 1, 3, 4, 12, 13, 25], "epsilon": [0, 5, 6, 7, 13, 25, 26, 27], "epsilon_": [0, 25], "epsilon_0": [0, 25], "epsilon_1": [0, 25], "epsilon_2": [0, 25], "epsilon_i": [0, 25, 26], "eq": [3, 13, 14, 20, 22, 27], "eqnarrai": [3, 5, 6], "equal": [0, 1, 2, 3, 4, 5, 6, 8, 9, 11, 12, 13, 14, 16, 18, 20, 22, 25, 26, 27], "equat": [1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 17, 20, 22, 25], "equilibrium": [2, 12], "equiv": [3, 13, 20, 22, 27], "equival": [0, 1, 5, 7, 8, 11, 13, 19, 20, 25, 26, 27], "erf": 22, "eriador": 25, "err": [0, 10], "err_": 6, "err_sqr": 2, "errat": [13, 27], "erron": 2, "error": [1, 2, 4, 5, 6, 7, 9, 11, 12, 13, 15, 16, 17, 18, 19, 20, 22], "error_estimate_corr_tim": 22, "error_hidden": 1, "error_output": 1, "escap": [13, 27], "especi": [1, 3, 9, 12, 13, 15, 18], "essenti": [0, 5, 6, 9, 10, 12, 14, 15, 22, 26, 27], "establish": [0, 6, 10, 11, 16], "estim": [0, 1, 5, 6, 7, 10, 11, 13, 19, 22, 25, 26, 27], "estimated_mse_fold": 6, "estimated_mse_kfold": 6, "estimated_mse_sklearn": 6, "et": [0, 2, 4, 16, 17, 24, 25, 26, 27], "eta": [0, 1, 3, 8, 12, 13, 18, 25, 27], "eta0": [8, 13], "eta_": 13, "eta_t": 13, "eta_v": [0, 1, 3, 25], "etc": [0, 1, 3, 5, 7, 8, 9, 11, 12, 13, 14, 19, 20, 22, 26, 27], "ethic": 19, "euclidean": [0, 14, 26], "evalu": [0, 2, 3, 4, 5, 6, 9, 13, 15, 16, 17, 22, 25, 26, 27], "evalut": 13, "even": [0, 1, 3, 4, 5, 6, 8, 9, 10, 11, 12, 13, 14, 19, 20, 22, 25, 26, 27], "evenli": 4, "event": [5, 7, 10, 22], "eventu": [0, 5, 6, 11, 12, 13, 23, 26, 27], "everi": [0, 1, 2, 3, 4, 5, 6, 9, 10, 11, 12, 13, 14, 15, 19, 22, 23, 25, 26, 27], "everyth": [4, 12, 16, 18], "everywher": [4, 13, 27], "evolv": 0, "exact": [0, 5, 11, 12, 13, 20, 22, 25, 26], "exactli": [0, 3, 4, 6, 12, 18, 19, 26], "exam": 25, "examin": 6, "exampl": [0, 5, 11, 12, 13, 15, 16, 18, 19, 20, 22, 24], "exce": [1, 12, 13], "excel": [0, 1, 4, 5, 10, 25, 26], "except": [3, 4, 6, 8, 9, 20], "excess": [0, 25], "excit": 0, "exclud": [1, 6, 12, 26], "exclus": [0, 1, 3, 6, 22, 25], "execut": [2, 5, 13, 15, 26, 27], "exemplifi": 13, "exercic": [23, 25], "exercis": [5, 19, 21, 23, 25, 27], "exhaust": 6, "exhibit": [0, 5, 6, 8, 25, 26], "exist": [0, 1, 2, 3, 5, 6, 7, 8, 9, 13, 20, 25, 27], "exit": [5, 20, 26, 27], "exp": [0, 1, 2, 5, 6, 7, 8, 10, 11, 12, 13, 16, 17, 22, 26, 27], "exp_term": 1, "expand": [5, 7, 11, 13, 27], "expans": [0, 3, 5, 8, 10, 12, 13, 25, 26, 27], "expect": [0, 1, 5, 6, 7, 11, 12, 13, 15, 18, 19, 25, 26], "expectation_value_of_h_wrt_p": 22, "expens": [6, 10, 13, 16, 27], "experi": [0, 1, 6, 8, 13, 15, 19, 25, 26, 27], "experiment": [0, 4, 6, 9, 22, 25], "expert": [1, 9], "explain": [0, 6, 9, 10, 11, 13, 16, 25, 27], "explained_variance_ratio_": 11, "explanatori": [0, 25], "explicit": [0, 3, 6, 13, 20, 25, 26, 27], "explicitli": [0, 4], "explod": 1, "exploit": [0, 3, 12, 13, 25], "explor": [1, 4, 6, 8, 13, 18, 19, 25, 27], "expon": 1, "exponenti": [0, 1, 5, 6, 10, 13, 22, 25, 27], "export": [9, 15, 16], "export_graphviz": 9, "export_text": 9, "exporttext": 9, "expos": 19, "express": [0, 2, 3, 5, 6, 7, 10, 12, 13, 18, 20, 22, 25, 27], "exptmean": 22, "exptvari": 22, "extend": [0, 2, 7, 11, 13, 19, 25], "extens": [0, 12, 15, 19, 25], "extent": [0, 1, 6, 24], "extern": [3, 6, 9], "extra": [1, 3, 5, 15, 23, 25, 26, 27], "extract": [0, 3, 5, 6, 7, 8, 11, 13, 16, 17, 20, 25, 26], "extrapol": [0, 25], "extrem": [0, 1, 4, 5, 6, 7, 8, 9, 13, 15, 16, 20, 26, 27], "extremum": [13, 27], "extrins": 11, "ey": [0, 5, 6, 13, 14, 18, 20, 25, 26, 27], "f": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 12, 13, 14, 15, 16, 17, 20, 22, 23, 25, 26, 27], "f1": 13, "f11": [0, 25], "f12": [0, 25], "f13": [0, 25], "f1_grad": 13, "f1d": 13, "f2": 13, "f2_grad_x1": 13, "f2_grad_x1_analyt": 13, "f2_grad_x2": 13, "f2_grad_x2_analyt": 13, "f3": 13, "f3_grad": 13, "f3_grad_analyt": 13, "f4": 13, "f4_grad": 13, "f4_grad_analyt": 13, "f5": 13, "f5_grad": 13, "f6": 13, "f6_for": 13, "f6_for_grad": 13, "f6_grad_analyt": 13, "f6_while": 13, "f6_while_grad": 13, "f7": 13, "f7_grad": 13, "f7_grad_analyt": 13, "f8": 13, "f8_grad": 13, "f9": [0, 13, 25], "f9_altern": 13, "f9_alternative_grad": 13, "f9_grad": 13, "f_": 10, "f_0": [3, 10], "f_1": [10, 13, 27], "f_2": [12, 13, 27], "f_3": 12, "f_d": 22, "f_grad": 13, "f_grad_analyt": 13, "f_i": [0, 6, 12, 16], "f_m": [3, 10], "f_n": 3, "f_vec": 2, "face": [13, 25, 27], "facecolor": [6, 8, 22], "facil": [0, 19], "facilit": 12, "fact": [0, 1, 3, 5, 9, 11, 12, 13, 25, 26, 27], "factor": [0, 1, 3, 5, 6, 9, 10, 11, 13, 20, 22, 25, 26, 27], "factori": 13, "fade": 6, "fafab0": [9, 10], "fail": [0, 6, 13, 23, 25, 27], "failur": 7, "fairli": [1, 2, 18, 22], "faisal": [16, 26], "fake": 4, "fake_loss": 4, "fake_output": 4, "fall": [8, 9, 21], "fals": [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 14, 16, 17, 20, 25, 26, 27], "famili": [0, 7, 8, 22, 26], "familiar": [0, 3, 5, 6, 8, 15, 19, 20, 22, 25], "famou": [6, 12], "far": [0, 3, 4, 5, 6, 8, 11, 12, 13, 14, 16, 25, 26, 27], "fashion": [0, 9, 10, 25], "fast": [1, 3, 6, 10, 12, 13, 19, 22, 25, 27], "faster": [1, 11, 13], "fastest": [13, 20, 27], "favor": 7, "favorit": 22, "fc": 3, "featur": [0, 1, 3, 5, 6, 7, 8, 10, 11, 12, 13, 15, 17, 18, 19, 22, 25, 27], "feature_nam": [1, 7, 9], "feautur": 9, "fed": 1, "feed": [0, 2, 3, 11, 19, 25], "feed_forward": 1, "feed_forward_out": 1, "feed_forward_train": 1, "feedback": [4, 25], "feeddorward": 4, "feedforward": [1, 4, 12], "feel": [0, 5, 6, 11, 13, 15, 16, 19, 23, 25], "feet": [], "fetch": [6, 15], "few": [1, 3, 4, 5, 9, 17, 18, 22, 25], "fewer": [0, 9, 11, 25], "ffnn": [1, 12], "field": [0, 3, 6, 12, 19], "fifth": [0, 6, 25], "fig": [0, 1, 2, 3, 4, 6, 7, 12, 13, 14, 25], "fig_id": [0, 6, 7, 9, 25], "figaxi": 22, "figsiz": [0, 1, 2, 3, 4, 6, 7, 8, 9, 10, 25], "figur": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 14, 16, 19, 25, 26, 27], "figure_id": [0, 6, 7, 9, 25], "figurefil": [0, 6, 7, 9, 25], "file": [0, 4, 5, 6, 7, 9, 15, 25], "file_prefix": 4, "filenam": 25, "fill": [5, 9, 18, 26, 27], "filter": [3, 4], "final": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 18, 21, 22, 23, 25, 27], "financ": 0, "find": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 19, 22, 25, 26, 27], "fine": [0, 14], "finish": 2, "finit": [3, 5, 6, 12, 13, 17, 22, 26, 27], "finnicki": 15, "first": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 16, 18, 20, 22, 23, 24, 26], "firsteigvector": 11, "fit": [1, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 17, 18, 22, 26], "fit_beta": 26, "fit_intercept": [0, 5, 6, 16, 26, 27], "fit_mod": 9, "fit_theta": 6, "fit_transform": [0, 6, 8, 9, 11, 15], "fiti": [0, 25], "five": [0, 9, 25, 26], "fix": [0, 3, 4, 6, 10, 11, 12, 13, 25], "flag": 4, "flat": [12, 13, 27], "flatten": [1, 3, 4, 5, 20], "flexibl": [1, 6, 8, 10, 12, 25], "flip": [23, 25], "float": [0, 3, 4, 5, 9, 11, 13, 14, 20, 25, 26, 27], "float32": [4, 9], "float64": [4, 20, 25], "flop": [5, 20, 26, 27], "flow": [1, 4, 12], "fluctuat": 5, "fly": 11, "fm": 0, "fmax": 3, "fmesh": 13, "fn": 7, "focu": [0, 3, 4, 5, 6, 15, 19, 24, 25, 26, 27], "focus": [1, 6, 7, 20, 26], "fold": [6, 9], "folder": [0, 4, 6, 15, 25], "follow": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 19, 20, 22, 23, 24, 25, 26, 27], "font": [7, 22, 25], "fontdict": 22, "fontsiz": [1, 6, 8, 9, 10, 22], "fontweight": 1, "footprint": 3, "foral": [8, 26], "forc": [0, 5, 6, 10, 11, 26, 27], "forcast": 4, "forecast": [4, 12], "forest": [0, 1, 9, 19, 25], "forget": 11, "form": [0, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 15, 16, 19, 20, 22, 25, 26, 27], "formal": [3, 4, 14, 22], "format": [0, 1, 3, 4, 6, 7, 8, 9, 10, 11, 19, 22, 24], "format_data": 4, "formatstrformatt": [6, 13, 27], "formul": [4, 6, 11, 14], "formula": [3, 13, 22, 27], "forth": [4, 12], "fortran": [0, 19, 20, 25], "fortran2003": [19, 25], "fortran90": 22, "fortun": [0, 11, 26], "forward": [0, 3, 6, 19, 20, 25], "found": [1, 2, 4, 5, 6, 12, 13, 25, 26], "foundat": [19, 25], "four": [4, 5, 6, 8, 12, 20, 21, 23, 25, 27], "fourier": [0, 25], "fourierdef1": 3, "fourierdef2": 3, "fourierseriessign": 3, "fourth": [12, 25, 26], "fp": 7, "frac": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 20, 22, 25, 26, 27], "fraction": 9, "frame": 7, "framework": [1, 8, 10, 22], "frank": [5, 11], "frankefunct": [5, 6, 11], "fredli": [23, 25], "free": [0, 6, 11, 13, 15, 16, 19, 20, 22, 23, 24, 25], "freecodecamp": 19, "freedom": [5, 27], "freeli": 0, "freez": 15, "frequenc": [3, 6, 7, 22], "frequent": [0, 8, 9, 13, 27], "frequentist": 19, "fresh": 10, "fridai": [15, 23, 25], "friedman": [6, 24, 25], "friendli": 4, "frodo": 25, "frog": 3, "from": [0, 1, 2, 3, 4, 6, 7, 8, 9, 11, 13, 14, 15, 16, 17, 18, 19, 20, 22, 23, 24], "from_cod": 9, "from_logit": [3, 4], "from_tensor_slic": 4, "front": [0, 4, 5, 25, 26, 27], "frustrat": 15, "fulfil": [2, 5, 12, 26, 27], "full": [1, 3, 5, 7, 9, 10, 13, 22, 25, 26, 27], "full_matric": [5, 26, 27], "fulli": [3, 6, 12, 22], "fun": [19, 25], "func": 2, "function": [2, 3, 4, 5, 9, 14, 15, 16, 17, 18, 19, 20], "functionali": 11, "fundament": [0, 6, 19, 25], "funtion": 2, "further": [2, 7, 9, 25], "furthermor": [0, 3, 5, 6, 7, 11, 12, 13, 19, 25, 26, 27], "futur": [0, 4, 8, 9, 25], "fy": [15, 21, 23, 24, 25], "fys5419": [24, 25], "fys5429": [24, 25], "f\u00f8470": [23, 25], "g": [0, 1, 2, 3, 4, 6, 8, 9, 10, 11, 13, 15, 18, 22, 25, 26, 27], "g0": 2, "g_": [2, 9, 10], "g_0": 2, "g_1": [2, 10], "g_2": [2, 10], "g_analyt": 2, "g_dnn_ag": 2, "g_euler": 2, "g_i": 2, "g_m": [3, 10], "g_n": 3, "g_re": 2, "g_t": 2, "g_t_d2t": 2, "g_t_d2x": 2, "g_t_dt": 2, "g_t_hessian": 2, "g_t_hessian_func": 2, "g_t_jacobian": 2, "g_t_jacobian_func": 2, "g_trial": 2, "g_trial_deep": 2, "g_vec": 2, "gain": [1, 5, 7, 9, 10, 13, 26, 27], "galleri": [0, 25], "game": 4, "gamge": 25, "gamma": [0, 2, 8, 9, 10, 11, 13, 25, 27], "gamma1": 8, "gamma2": 8, "gamma_": [0, 25], "gamma_0": 10, "gamma_1": 10, "gamma_1x": 10, "gamma_i": [0, 8, 22, 25], "gamma_j": 13, "gamma_k": [13, 27], "gamma_m": 10, "gamma_x": [0, 25], "gap": 8, "gate": [4, 12], "gather": [0, 1, 12, 26], "gaug": 12, "gaussbacksub": 20, "gaussian": [4, 5, 6, 8, 14, 18, 22, 25], "gaussian_point": 14, "gaussian_rbf": 8, "gave": 13, "gavra": 25, "gbc": 25, "gca": [2, 6, 8, 13], "gd": [1, 27], "gd_clf": 10, "gdclassiffiercgain": 10, "gdclassiffierconfus": 10, "gdclassiffierroc": 10, "gdm": 13, "gdregress": 10, "ge": [1, 5, 7, 22, 26, 27], "gen_loss": 4, "gen_tap": 4, "gender": [0, 25], "genener": 4, "gener": [0, 1, 2, 3, 5, 6, 8, 10, 11, 12, 13, 14, 15, 16, 18, 20, 22, 24, 26, 27], "generaliz": 16, "generallay": 12, "generate_and_save_imag": 4, "generate_imag": 4, "generate_latent_point": 4, "generate_simple_clustering_dataset": 14, "generated_imag": 4, "generator_loss": 4, "generator_loss_list": 4, "generator_model": 4, "generator_optim": 4, "genom": 19, "geodes": 11, "geometr": [0, 13, 25], "geometri": 5, "georg": 24, "geotif": 6, "geq": [2, 5, 8, 9, 13, 26, 27], "geron": [0, 24, 25], "get": [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 13, 15, 19, 20, 22, 23, 25, 26, 27], "get_dummi": 9, "get_paramet": 2, "get_split": 9, "get_yaxi": 8, "get_yticklabel": 6, "gh": 15, "gibb": [19, 25], "gif": 4, "gini": 10, "gini_index": 9, "ginvers": 13, "git": [0, 15, 19, 25], "giter": 13, "github": [0, 19, 21, 23, 24, 25, 26], "gitignor": 15, "gitlab": [0, 15, 19, 25], "give": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 12, 13, 14, 18, 19, 22, 25, 26, 27], "given": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 17, 20, 22, 25, 26, 27], "global": [6, 7, 13, 27], "glorot": 1, "gnew": 13, "go": [0, 1, 3, 5, 6, 8, 9, 11, 12, 13, 15, 16, 25, 26, 27], "goal": [0, 7, 9, 25], "goe": [0, 1, 2, 5, 6, 13, 14, 15, 20, 25, 26, 27], "golden": 13, "gone": [5, 26, 27], "gong": 1, "good": [1, 3, 4, 5, 6, 9, 10, 11, 13, 15, 18, 19, 22, 24, 26, 27], "goodfellow": [4, 24, 25, 26, 27], "googl": [1, 4, 19, 25], "got": [1, 6], "gotten": 25, "gov": 6, "govern": 25, "gp": 24, "gpu": [1, 13, 19, 25], "grad": [2, 13], "grad_analyt": 13, "grad_ol": 18, "grad_ridg": 18, "grade": 21, "gradient": [0, 3, 4, 7, 8, 9, 12, 19, 25, 26], "gradientboostingclassifi": 10, "gradientboostingregressor": 10, "gradients_of_discrimin": 4, "gradients_of_gener": 4, "gradienttap": 4, "gradual": [1, 14], "grai": [4, 6], "graph": [1, 9, 11, 12, 13, 16, 27], "graph_from_dot_data": 9, "graphic": [0, 1, 9, 15, 25], "grasp": 0, "gray_r": [1, 3], "grayscal": 3, "great": [5, 13, 15, 27], "greater": [1, 7, 22, 26], "greatli": 13, "greedi": 9, "green": [0, 3, 9, 22], "grei": 4, "grid": [1, 3, 6, 7, 8, 12, 22, 26], "grossli": [13, 27], "ground": [0, 25], "group": [0, 6, 7, 9, 14, 15, 19, 21, 23, 25], "groupbi": [0, 25], "grow": [1, 3, 9, 10], "growth": [0, 25], "gru": 4, "guarante": [0, 4, 13, 22, 25, 26, 27], "guess": [1, 4, 10, 13, 14, 27], "guestrin": 10, "gui": 15, "guid": 1, "h": [0, 1, 5, 6, 8, 13, 15, 22, 23, 24, 25, 26, 27], "h1": 2, "h_": [0, 13, 25, 27], "h_1": [2, 13, 27], "h_2": [2, 13, 27], "h_m": 10, "ha": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 18, 20, 22, 25, 26, 27], "haanen": [23, 25], "habit": [0, 26], "had": [0, 1, 6, 7, 13, 25, 27], "hadamard": [1, 12, 13], "half": [1, 8, 9], "halv": 10, "hand": [0, 1, 2, 3, 5, 11, 12, 13, 19, 20, 22, 23, 24, 25, 26, 27], "handi": 3, "handl": [0, 1, 2, 5, 9, 11, 15, 18, 19, 26, 27], "handle_unknown": 9, "handsid": 12, "handwrit": 12, "handwritten": [1, 5], "happen": [1, 2, 3, 4, 5, 6, 10, 13, 22, 26, 27], "hard": [1, 7, 8, 10, 13, 27], "hardcopi": [19, 25], "harder": [0, 1, 26], "harmon": 3, "hasn": [], "hassl": [0, 19, 25], "hast": [19, 25], "hasti": [0, 6, 16, 17, 24, 25, 26], "hat": [0, 1, 5, 6, 7, 9, 10, 11, 12, 13, 16, 17, 18, 20, 26, 27], "have": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 23, 25, 26, 27], "haven": 1, "he": 7, "head": [4, 10, 22], "header": [0, 25], "heads_proba": 10, "health": [0, 26], "hear": [0, 13, 25], "heart": [0, 7, 25], "heatmap": [0, 1, 3, 7, 17, 25], "heavili": 0, "heavisid": 1, "height": [1, 3, 6, 26], "held": 13, "help": [0, 1, 4, 12, 13, 15, 16, 25], "helper": [4, 14], "henc": [0, 5, 6, 8, 9, 10, 12, 13, 25, 26, 27], "henrik": [23, 25], "her": 7, "here": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 17, 18, 19, 20, 22, 25, 26, 27], "hereaft": [0, 8, 12, 25], "hermitian": 20, "hessenberg": 20, "hessian": [0, 2, 5, 13], "heterogen": [9, 10], "hi": 7, "hidden": [1, 3, 4, 12], "hidden_bia": 1, "hidden_bias_gradi": 1, "hidden_layer_s": [0, 1, 25], "hidden_neuron": 4, "hidden_weight": 1, "hidden_weights_gradi": 1, "hierarch": [5, 26, 27], "high": [0, 1, 2, 3, 4, 5, 6, 9, 10, 11, 13, 14, 19, 20, 25, 26, 27], "higher": [0, 1, 3, 5, 6, 8, 13, 25, 26, 27], "highest": [1, 2], "highli": [0, 3, 4, 10, 19, 20, 24, 25, 26, 27], "highwai": [], "hing": 8, "hint": [13, 15, 16, 26, 27], "hip": 19, "hire": 0, "hist": [4, 6, 7, 22], "histogram": [6, 7, 22], "histor": [7, 11], "histori": [3, 4, 12, 15, 18], "hitherto": 5, "hjorth": [23, 25, 26, 27], "hobbi": 22, "hoc": [5, 26, 27], "hoff": 24, "hold": [1, 3, 6, 13, 14, 27], "holder": [0, 25], "home": [], "homepag": 25, "homework": [6, 13, 27], "homogen": [1, 3, 9, 10, 13], "honchar": 2, "hopefulli": [0, 11, 15, 22, 25], "horizont": 11, "horlyk": [23, 25], "hors": [3, 7, 25], "hot": [1, 9], "hour": [1, 19, 21, 22, 23, 25], "how": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 18, 19, 20, 22, 25, 26, 27], "howev": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 19, 20, 22, 25, 26, 27], "hspace": [0, 4, 8, 10, 22, 25], "hstack": 1, "htf": 25, "html": [0, 16, 19, 21, 23, 24, 25, 26, 27], "http": [0, 3, 4, 6, 13, 15, 16, 19, 20, 21, 23, 24, 25, 26, 27], "huang": [0, 25], "huber": [0, 25], "huge": [1, 3, 4, 19], "human": [0, 1, 3, 6, 9, 12, 26], "humid": 9, "hundr": 1, "hungri": 1, "hybrid": 21, "hydrogen": [0, 25], "hyperbol": [1, 4, 12], "hyperparam": 8, "hyperparamet": [3, 4, 5, 6, 9, 13, 18, 26, 27], "hyperplan": 11, "h\u00f8rlyk": [23, 25], "i": [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 26, 27], "i0": [0, 25], "i1": [0, 6, 8, 12, 25, 26], "i2": [0, 8, 12, 25], "i3": [0, 12, 25], "i5": [0, 25], "i_": [13, 27], "i_1": [5, 6], "i_2": [5, 6], "ian": 24, "ic": 1, "id": [7, 13, 27], "ida": [23, 25], "idea": [0, 1, 2, 3, 4, 6, 9, 10, 12, 13, 20, 26, 27], "ideal": [0, 2, 6, 8, 13, 22, 25], "idem": 6, "ident": [5, 6, 12, 13, 17, 18, 20, 26, 27], "identifi": [0, 1, 7, 9, 11, 12, 13, 14, 25, 26], "ieor": 22, "ifi": 24, "ifs": [19, 25], "ignor": [0, 1, 3, 9, 15, 26], "ii": [20, 22], "iii": [20, 25], "ij": [0, 1, 3, 6, 8, 12, 14, 16, 20, 22, 25, 26], "ik": [0, 20, 25, 26], "illustr": [5, 7, 10, 12, 13, 14, 19, 25], "im": 6, "imag": [1, 3, 4, 6, 9, 11, 12, 14, 24, 25], "image_at_epoch_": 4, "image_batch": 4, "image_height": 3, "image_path": [0, 6, 7, 9, 25], "image_width": 3, "imageio": 6, "images_from_seed_imag": 4, "imagin": 1, "immedi": [0, 3, 4, 6, 19, 25], "implement": [0, 2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 13, 14, 22, 25, 26, 27], "impli": [3, 5, 6, 7, 13, 20, 26, 27], "implicit": 3, "implicitli": [11, 22], "import": [0, 1, 2, 3, 4, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 22], "importantli": 3, "impos": [0, 6, 11, 12, 25], "imposs": [0, 5, 25, 26, 27], "impress": [0, 12, 25], "improv": [0, 4, 5, 9, 10, 11, 13, 15, 26, 27], "impur": 9, "imread": 6, "imshow": [1, 3, 4, 6], "in3050": [24, 25], "in3310": 25, "in4080": [24, 25], "in4300": [24, 25], "in4310": 24, "in5400": 3, "in5550": 24, "in_out_neuron": 4, "inaccur": [13, 27], "inact": 12, "inadequ": [0, 25], "inch": [6, 26], "includ": [0, 1, 2, 3, 4, 5, 6, 7, 11, 12, 15, 16, 17, 18, 19, 22, 23, 24, 25, 26, 27], "include_bia": [6, 9], "incom": [12, 16], "incorrect": 1, "incoveni": 8, "increas": [0, 1, 3, 4, 5, 6, 9, 12, 13, 22, 25, 26], "increasingli": 22, "ind": 6, "inde": [0, 2, 4, 5, 6, 13, 25, 26, 27], "indefinit": 4, "independ": [0, 5, 6, 7, 8, 12, 13, 22, 25, 26, 27], "index": [0, 1, 3, 4, 10, 14, 19, 20, 22, 24, 25], "index_col": [0, 25], "indic": [0, 1, 3, 4, 5, 6, 9, 10, 11, 13, 16, 25, 26], "indispens": 6, "individu": [1, 6, 7, 10, 12, 22, 25, 26], "indu": [], "indx": 20, "indx1": 2, "indx2": 2, "indx3": 2, "ineffici": [3, 13], "inequ": [8, 13], "inequaltii": 27, "inertia": 13, "inf1000": [19, 25], "inf1100": [19, 25], "inf1100l": [19, 25], "inf1110": [19, 25], "inf3000": 25, "infeas": 9, "infer": [0, 1, 4, 6, 24, 25], "inferenc": 1, "infil": [0, 6, 7, 9, 25], "infin": [5, 6, 7, 11, 26, 27], "infinit": 3, "infinitesim": 22, "influenc": [6, 10, 18], "influenti": 1, "info": 25, "inform": [0, 1, 3, 4, 6, 9, 11, 12, 13, 14, 20, 24, 25, 27], "inforom": 15, "infti": [3, 6, 13, 22, 27], "ingeni": [13, 27], "ingredi": [0, 9, 25], "inher": 6, "inherit": [20, 25], "initi": [0, 1, 2, 6, 10, 13, 14, 18, 20, 22, 25, 27], "inject": 14, "inlin": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 20, 22, 25, 26, 27], "inner": [0, 13, 26], "inp": 4, "inplac": 13, "input": [0, 1, 3, 4, 5, 6, 7, 8, 12, 13, 14, 16, 22, 25, 26, 27], "input_dim": 1, "input_shap": [3, 4], "inputs": 1, "inputs_shuffl": [0, 1, 26], "insert": [3, 5, 6, 8, 10, 22, 26, 27], "insid": [4, 7], "insight": [0, 1, 5, 19, 25, 26, 27], "insist": [6, 13, 26], "inspir": [0, 1, 12, 25], "instabl": 2, "instal": [0, 1, 5, 6, 9, 15], "instanc": [0, 1, 2, 4, 6, 9, 11, 13, 16, 25, 26, 27], "instanti": 10, "instead": [0, 1, 2, 3, 4, 5, 6, 8, 9, 11, 13, 14, 17, 20, 22, 25, 26], "institut": 1, "instruct": [0, 1, 15], "int": [0, 1, 2, 3, 4, 5, 6, 11, 13, 14, 20, 22, 26], "int32": 10, "int_": [3, 6, 22], "int_0": 22, "int_a": 22, "intak": [0, 26], "integ": [1, 2, 13, 14, 20, 22, 25], "integer_vector": 1, "integr": [3, 6, 22, 25], "intellig": [0, 14, 24, 25], "intend": 10, "intens": [1, 18], "intention": 14, "interact": [0, 6, 9, 12, 19, 25], "intercept": [0, 6, 8, 11, 13, 16, 17, 18, 25, 26, 27], "intercept_": [0, 6, 8, 9, 13, 25, 26], "interchang": [5, 12, 20], "interconnect": 1, "interest": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 12, 19, 22, 25, 26, 27], "interfac": [0, 1, 15, 20, 26], "interior": [0, 9, 25], "intermedi": [20, 26], "intern": [1, 10, 12], "interpol": [1, 3, 4, 6, 12], "interpr": [5, 26, 27], "interpret": [0, 1, 6, 9, 10, 12, 13, 15, 16, 20, 22], "interv": [0, 3, 5, 6, 7, 13, 22, 25, 26, 27], "intial": [13, 27], "intract": [0, 4, 26], "intrins": [3, 11, 20, 22, 25], "intro": [19, 24, 25], "introduc": [0, 1, 5, 6, 8, 10, 12, 20, 22, 25, 27], "introduct": [1, 2, 4, 13, 24, 26, 27], "introductori": [0, 4, 20, 24, 25, 26], "intuit": [0, 5, 6, 8, 12, 13, 25], "inv": [0, 5, 13, 17, 25, 26, 27], "invalu": [0, 13, 19, 25, 27], "invari": 1, "invd": 5, "inver": 8, "invers": [0, 3, 6, 13, 25, 26, 27], "inverse_transform": 8, "invert": [0, 5, 7, 10, 13, 16, 18, 25], "invh": 13, "invok": 8, "involv": [0, 2, 6, 7, 11, 12, 25, 26], "io": [0, 19, 21, 23, 24, 25, 26], "ip": [0, 8, 22, 25], "ipca": 11, "ipynb": [19, 25], "ipython": [0, 5, 7, 9, 11, 14, 19, 25, 26], "iq": 6, "iri": [8, 9], "irreduc": 6, "irrelev": [5, 26, 27], "irrespect": [0, 25], "isn": 5, "isnul": [], "isomap": 11, "issu": [1, 9, 15, 20], "it_arrai": 13, "item": [0, 13, 25], "items": [20, 25], "iter": [1, 2, 4, 6, 8, 13, 14, 18, 22, 27], "its": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 19, 20, 22, 25, 27], "itself": [5, 6, 12, 22, 25, 26], "j": [0, 1, 2, 3, 4, 5, 6, 8, 9, 11, 12, 13, 14, 15, 16, 20, 22, 24, 25, 26, 27], "j1": 20, "j_": 6, "j_lasso_sk": 6, "j_ridge_sk": 6, "j_sk": 6, "jackknif": [6, 19, 25], "jacobian": [2, 13, 27], "jason": 4, "jax": [19, 25], "jensen": [23, 25, 26, 27], "jerom": 24, "ji": [12, 20], "jit": 13, "jj": [0, 5, 6, 25], "jk": [0, 1, 6, 12, 20, 25], "jl": [0, 25], "jm": 20, "jnp": 13, "job": [2, 8, 10, 15], "join": [0, 4, 6, 7, 9, 25], "joint": [4, 5], "judg": [13, 27], "judgement": 6, "julia": [19, 20], "jump": 22, "junk": 4, "jupit": 25, "jupyt": [0, 15, 16, 19, 24, 25], "just": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 19, 22, 25, 26, 27], "justif": 0, "justifi": [3, 10], "k": [0, 1, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 19, 20, 22, 23, 25, 26, 27], "k0": 7, "k1": 7, "kaggl": 6, "kappa_d": 22, "karl": [23, 25], "karush": 8, "katrin": [23, 25], "keep": [0, 1, 4, 5, 6, 11, 13, 14, 15, 18, 20, 25, 26, 27], "keepdim": [1, 6, 10, 20], "kei": [1, 3, 6, 12], "kept": [4, 6, 14], "kera": [0, 4, 19, 25], "kernel": [0, 1, 3, 19, 25, 26], "kernel_regular": [1, 3], "kernel_s": 4, "kernelpca": 11, "kev": [0, 25], "kevin": [24, 25], "keyword": [20, 25], "kfold": 6, "kg": 1, "ki": 20, "kick": [1, 13], "kiener": 2, "kilomet": [6, 26], "kind": [0, 2, 3, 4, 8, 12, 13, 14, 25, 26], "kj": [6, 12, 20, 26], "kjm": [19, 25], "kkt": 8, "kl": 22, "km": [12, 25], "kmean": 14, "kmeanspoint": 14, "kn_k": 14, "know": [0, 1, 2, 5, 6, 8, 13, 15, 16, 17, 19, 25, 26, 27], "knowledg": [0, 19, 25], "known": [1, 3, 4, 5, 6, 7, 8, 9, 12, 18, 20, 22, 24, 26], "kondev": [0, 25], "kp": 22, "kpca": 11, "kroneck": 14, "kuhn": 8, "kvalsund": [23, 25], "kwown": [0, 25], "l": [0, 1, 2, 3, 5, 6, 7, 8, 10, 11, 12, 13, 20, 22, 25, 27], "l0": 7, "l1": [0, 1, 3, 7, 25], "l1_l2": [1, 3], "l1regl": 5, "l2": [1, 3], "l_": 20, "l_1": 7, "l_2": [7, 13, 27], "l_j": 12, "la": 13, "la_i": 12, "la_k": 12, "lab": [19, 25], "label": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 14, 15, 19, 20, 22, 25, 26, 27], "labelencod": [7, 10], "labels": [6, 8, 9], "labels_shuffl": [0, 1, 26], "laboratori": 21, "lack": [0, 25], "lagari": 2, "lagrang": [8, 11], "lam": [], "lambda": [0, 1, 2, 3, 5, 6, 7, 8, 10, 12, 13, 17, 18, 22, 25, 26, 27], "lambda_": 11, "lambda_0": 11, "lambda_1": [5, 8, 11, 26, 27], "lambda_2": [8, 11], "lambda_i": [8, 11], "lambda_iy_i": 8, "lambda_jy_iy_j": 8, "lambda_k": 8, "lambda_n": [5, 8, 26, 27], "lamda": 1, "land": 8, "landmark": 8, "landscap": [13, 18, 27], "langl": [0, 6, 11, 22, 25, 26], "languag": [0, 1, 4, 8, 19, 20, 24, 25], "lapack": [20, 25], "laplac": 5, "laptop": [15, 19], "larg": [0, 1, 2, 4, 5, 6, 8, 9, 10, 11, 13, 18, 19, 20, 22, 24, 25, 26, 27], "larger": [0, 3, 5, 6, 8, 10, 11, 13, 17, 22, 25, 26, 27], "largest": [4, 8, 11], "lasso": [0, 7, 19, 25], "lasso_sk": 6, "last": [0, 1, 3, 4, 5, 6, 7, 8, 12, 16, 17, 20, 22, 23, 25, 27], "latent": 4, "latent_dim": 4, "latent_point": 4, "latent_space_value_rang": 4, "later": [0, 1, 4, 7, 8, 12, 13, 14, 15, 19, 25], "latest": [4, 15, 19], "latest_checkpoint": 4, "latex": 25, "latter": [0, 3, 6, 7, 8, 11, 13, 20, 22, 25, 26, 27], "lattic": 12, "law": 0, "layer": [0, 4, 13, 25], "lbfg": [7, 9, 10], "lcc": [5, 6], "lda": 11, "ldot": [0, 6, 11, 25], "le": [5, 7, 10, 13, 17, 22, 26, 27], "lead": [0, 1, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 16, 17, 20, 22, 25, 26, 27], "leaf": 9, "leaki": 1, "leakyrelu": 4, "lear": [13, 27], "learn": [3, 4, 5, 6, 7, 8, 9, 10, 12, 20, 23, 24], "learnabl": 3, "learner": 10, "learnig": 25, "learning_r": [8, 10], "learning_rate_init": [0, 1, 25], "learning_schedul": 13, "least": [0, 7, 8, 10, 11, 17, 18, 19, 20, 22], "leat": 13, "leav": [0, 1, 3, 5, 6, 9, 11, 25, 27], "lectur": [0, 1, 5, 10, 11, 12, 13, 19, 20, 21, 23, 24, 26], "lecturenot": [0, 19, 24, 25], "left": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 20, 22, 25, 26, 27], "leftarrow": [8, 12], "legend": [0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 13, 15, 25, 26, 27], "leinonen": 25, "len": [0, 1, 2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 16, 17, 20, 25, 26, 27], "length": [0, 1, 3, 4, 8, 9, 13, 16, 19, 25, 26, 27], "length_of_sequ": 4, "leq": [0, 5, 7, 8, 13, 14, 22, 25, 26, 27], "less": [0, 1, 3, 4, 5, 6, 8, 9, 13, 19, 22, 25, 26, 27], "lessen": 1, "let": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 20, 22, 25, 26, 27], "letter": [0, 16, 20, 22, 25, 26], "level": [0, 1, 5, 6, 9, 19, 20, 21, 23, 25], "li": [8, 11], "lib": [], "liblinear": 10, "librari": [0, 1, 2, 3, 4, 5, 6, 9, 10, 11, 20, 22, 24, 26, 27], "licens": [0, 1, 19, 25], "lie": [0, 6, 11, 22, 25, 26], "life": [0, 1, 8, 12, 25], "lifetim": 13, "like": [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, 15, 16, 19, 20, 22, 25, 26, 27], "likelihood": [0, 1, 5, 9, 25, 26], "lim_": 22, "limit": [0, 5, 6, 8, 12, 20, 25, 26], "lin_clf": 8, "lin_model": [], "lin_reg": 9, "linalg": [0, 2, 5, 6, 8, 11, 13, 17, 20, 22, 25, 26, 27], "line": [0, 3, 6, 8, 11, 13, 15, 16, 25, 27], "line1": 8, "line2": 8, "line3": 8, "line_model": 15, "line_ms": 15, "line_predict": 15, "linear": [1, 3, 5, 6, 7, 9, 10, 11, 12, 16, 17, 18, 19, 22], "linear_model": [0, 5, 6, 7, 8, 9, 10, 11, 13, 15, 16, 25, 26, 27], "linear_regress": 6, "linearli": [5, 26, 27], "linearloc": [6, 13, 27], "linearregress": [0, 6, 7, 9, 15, 16, 25, 26], "linearsvc": 8, "lineat": 27, "liner": [1, 3], "linerar": 10, "linewidth": [0, 2, 4, 6, 8, 9, 10], "link": [0, 4, 9, 12, 15, 19, 21, 23, 25], "linlag": 5, "linpack": [20, 25], "linreg": [0, 25], "linspac": [0, 2, 3, 4, 6, 8, 9, 10, 13, 16, 17, 20, 22, 25, 26], "linu": 4, "linux": [0, 1, 19, 25], "liquid": [0, 25], "list": [1, 2, 3, 4, 9, 15, 19, 25], "listedcolormap": [9, 10], "literatur": [1, 7, 14, 24], "littl": [1, 3, 9, 12], "live": [8, 16], "ll": [0, 18, 22, 25, 26], "lle": [0, 26], "lloyd": [4, 14], "lmb": [0, 2, 5, 6, 26, 27], "lmbd": [0, 1, 3, 25], "lmbd_val": [0, 1, 3, 25], "lmbda": [13, 27], "ln": [1, 13, 27], "load": [1, 4, 6, 7, 9, 10], "load_boston": [], "load_breast_canc": [1, 7, 9, 10, 11], "load_data": [3, 4], "load_digit": [1, 3], "load_iri": [8, 9], "loc": [3, 6, 7, 8, 9, 10, 25], "local": [0, 1, 3, 7, 12, 13, 15, 26, 27], "locat": [2, 3, 8, 15], "log": [0, 1, 2, 4, 5, 6, 7, 9, 10, 11, 13, 15, 20, 25], "log10": [0, 5, 6, 26, 27], "log_": [0, 25], "log_clf": 10, "logarithm": [0, 5, 7, 17, 20, 25], "logic": [0, 1, 9, 25], "login": 15, "logist": [0, 1, 2, 8, 9, 10, 11, 12, 13, 19, 26, 27], "logisticregress": [7, 9, 10, 11], "logit": 7, "logreg": [7, 9, 10, 11], "logspac": [0, 1, 3, 5, 6, 25, 26, 27], "long": [0, 1, 3, 4, 12, 13, 25, 27], "longer": [2, 3, 8, 10, 14, 20, 22, 25], "loocv": 6, "look": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 15, 16, 20, 22, 25, 26, 27], "loop": [1, 4, 6, 10, 12, 14, 16, 17, 18, 19, 20, 25], "lose": 1, "loss": [0, 1, 3, 4, 5, 6, 7, 8, 10, 11, 13, 20, 25], "loss_fil": 4, "lossfil": 4, "lost": 4, "lot": [1, 4, 6, 16], "low": [0, 6, 9, 10, 11, 25, 26], "lower": [0, 1, 3, 6, 9, 10, 16, 20, 26], "lowercas": [20, 25], "lowest": [9, 13, 22], "lr": [1, 3, 4, 10], "lstat": [], "lstm": 4, "lstm_2layer": 4, "lstsq": [0, 25, 26], "lt": 6, "lu": [0, 5, 25, 26, 27], "lubksb": 20, "luckili": 2, "ludcmp": 20, "lux": 20, "lvert": 1, "lw": [0, 25], "m": [0, 1, 2, 3, 5, 6, 8, 9, 10, 11, 12, 13, 15, 18, 20, 22, 23, 24, 25, 26, 27], "m_": [9, 12], "m_1": 14, "m_h": [0, 25], "m_k": 14, "m_l": 12, "m_n": [0, 25], "m_p": [0, 25], "m_t": 13, "ma": 11, "machin": [1, 3, 4, 5, 6, 7, 9, 10, 11, 12, 15, 16, 20, 24, 26], "machinelearn": [0, 6, 16, 19, 21, 23, 24, 25, 26], "mackai": 24, "made": [0, 1, 3, 4, 5, 6, 7, 9, 11, 12, 25, 26], "mae": [0, 25], "magic": 4, "magnitud": [1, 6, 7, 13, 26], "mai": [0, 1, 2, 3, 5, 6, 7, 8, 9, 11, 12, 13, 19, 20, 22, 25, 26, 27], "mail": [21, 23], "main": [0, 1, 3, 4, 5, 6, 7, 9, 20, 24, 26, 27], "mainli": [0, 5, 6, 7, 9, 25, 26], "maintain": 6, "major": [1, 6, 9, 10, 13, 20, 25, 27], "make": [1, 2, 3, 4, 5, 6, 7, 8, 11, 12, 13, 15, 16, 18, 19, 20, 22, 24, 25, 27], "make_axes_locat": 6, "make_moon": [8, 9, 10], "make_pipelin": [0, 6, 10, 26], "makedir": [0, 6, 7, 9, 25], "malcondit": 20, "malign": [1, 7, 9], "mammographi": 5, "manag": [0, 2, 3, 15, 19, 25], "mandatori": [23, 25], "mani": [0, 1, 3, 4, 5, 6, 7, 8, 9, 11, 13, 14, 15, 16, 17, 18, 19, 20, 22, 24, 25, 26, 27], "manifold": 11, "manner": 3, "manual": [6, 26], "map": [0, 1, 2, 6, 7, 8, 11, 12, 14, 22, 25], "marc": 26, "margin": [0, 5, 8], "marit": [0, 25], "mark": 25, "marker": [7, 20, 25], "markov": [19, 25], "marsaglia": 22, "mass": [0, 1, 5, 13, 26, 27], "massag": [0, 25], "masses2016": [0, 25], "masses2016ol": [0, 25], "masses2016tre": 0, "masseval2016": [0, 25], "master": [21, 23], "mat": [19, 25], "mat1100": [19, 25], "mat1110": [19, 25], "mat1120": [19, 25], "match": [1, 4, 5, 13, 14, 15, 26, 27], "materi": [4, 5, 7, 13, 15, 20, 21, 23], "math": [3, 7, 12, 13, 20, 22, 24, 25], "mathbb": [0, 4, 5, 6, 7, 8, 11, 12, 13, 14, 17, 20, 22, 25, 26, 27], "mathbf": [0, 5, 6, 7, 8, 13, 20, 25, 26, 27], "mathcal": [1, 5, 6, 7, 13], "matheemat": 3, "mathemat": [0, 6, 11, 12, 13, 19, 20, 22, 24, 25], "mathemati": 25, "mathrm": [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 17, 18, 22, 25, 26, 27], "matmul": [1, 2, 5], "matnat": 24, "matplotlib": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 19, 20, 22, 25, 26, 27], "matric": [0, 1, 3, 4, 6, 7, 8, 11, 13, 16, 17, 19, 26, 27], "matrix": [0, 2, 3, 4, 6, 7, 8, 10, 13, 17, 18, 22], "matshow": 1, "matter": [2, 3, 13, 26, 27], "max": [0, 1, 2, 3, 4, 9, 10, 12, 13, 23, 25, 27], "max_depth": [0, 9, 10], "max_diff": 2, "max_diff1": 2, "max_diff2": 2, "max_it": [0, 1, 8, 13, 25], "max_iter": 14, "max_leaf_nod": 10, "max_sampl": 10, "maxdegre": [0, 6, 10, 26], "maxdepth": 10, "maxim": [1, 4, 5, 7, 8, 11], "maximum": [0, 2, 3, 5, 7, 8, 9, 10, 13, 14, 25, 26, 27], "maxpolydegre": [5, 6, 26, 27], "maxpooling2d": 3, "mbox": [5, 6, 26, 27], "mcculloch": 12, "md": 11, "mdoel": 4, "mean": [1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, 14, 15, 17, 18, 19, 20, 22, 25], "mean_absolute_error": [0, 25], "mean_divisor": 14, "mean_i": 22, "mean_matrix": 14, "mean_squared_error": [0, 4, 6, 7, 10, 15, 25, 26], "mean_squared_log_error": [0, 25], "mean_vector": 14, "mean_x": 22, "meaning": [0, 4, 7, 25], "meansquarederror": [0, 25], "meant": [3, 7, 10, 13], "measur": [0, 1, 2, 5, 6, 9, 11, 12, 14, 16, 18, 22, 25, 26], "mechan": [0, 4, 22, 25], "median": [0, 25, 26], "medicin": 12, "medium": [4, 8, 13], "medv": [], "meet": [0, 23], "mehta": [0, 25, 26, 27], "memori": [3, 4, 11, 12, 13, 18, 20], "mention": [0, 12, 13, 22, 25, 27], "mere": 0, "meshgrid": [2, 5, 6, 8, 9, 10, 11], "mess": 15, "messag": [5, 13], "messi": 2, "met": [0, 3, 8, 26], "meteorolog": 9, "meter": [6, 26], "method": [0, 1, 2, 3, 4, 5, 7, 8, 11, 12, 14, 15, 16, 17, 18, 19, 20, 22, 24, 26], "metion": 6, "metric": [0, 1, 3, 6, 7, 9, 10, 14, 15, 25, 26], "metropoli": [19, 25], "mev": [0, 22, 25], "mgd": 13, "mglearn": [19, 25], "mgrid": 13, "mhjensen": [], "mi": 10, "mia": [23, 25], "microsoft": 24, "mid": 1, "midel": 4, "midnight": 15, "midpoint": 9, "might": [0, 1, 2, 4, 6, 9, 13, 15, 17, 18, 26, 27], "migth": 17, "mild": 9, "millimet": [6, 26], "million": [0, 25, 26], "mimic": 12, "min": [0, 2, 5, 8, 9, 27], "min_": [0, 2, 5, 14, 17, 25, 26, 27], "min_samples_leaf": 9, "mind": [0, 6, 13, 15, 18, 25, 26, 27], "mindboard": 4, "mine": [19, 25], "mini": [1, 11, 12, 13, 27], "minibatch": [1, 11, 13], "minibathc": 13, "miniforge3": [], "minim": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 26, 27], "minima": [0, 1, 7, 13, 25, 27], "minimum": [0, 1, 2, 6, 8, 9, 11, 13, 26, 27], "minmaxscal": [0, 26], "minor": 22, "minst": 1, "minu": 7, "mirjalili": 25, "mirror": 9, "misc": 6, "misclassif": [8, 9, 10], "misclassifi": [8, 10], "miser": 0, "mismatch": 1, "miss": [7, 10], "mistak": 4, "mit": 24, "mix": [1, 2, 25], "mixtur": 13, "mk": [9, 20], "mkdir": [0, 6, 7, 9, 25], "ml": [0, 1, 10, 13, 20, 26, 27], "mlab": 22, "mle": [5, 7], "mlp": 1, "mlpclassifi": 1, "mlpregressor": [0, 25], "mm": 20, "mml": 26, "mn": [12, 22], "mnist": [1, 11], "mod": 22, "mode": [21, 23, 25], "model": [2, 3, 5, 7, 8, 9, 10, 11, 13, 14, 16, 18, 19, 22, 24, 26, 27], "model_select": [0, 1, 3, 5, 6, 7, 9, 10, 11, 15, 16, 17, 25, 26, 27], "moder": 10, "modern": [0, 6, 7, 19, 25], "modif": [2, 12, 13], "modifi": [0, 1, 3, 5, 7, 8, 10, 12, 13, 25, 26, 27], "modul": [0, 16, 20, 25], "modular": 22, "modulo": 22, "moe": [11, 26], "moment": [5, 6, 13, 22], "mondai": [23, 25], "monitor": [13, 18], "monoton": [5, 12, 22], "mont": [0, 6, 19, 22, 24, 25], "montli": 16, "moor": [5, 6], "more": [0, 1, 2, 4, 5, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 19, 22], "moreov": [0, 3], "morten": [23, 25, 26, 27], "mortenhj": 25, "most": [0, 1, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 19, 22, 25, 26, 27], "mostli": [1, 11, 18], "motion": [0, 13], "motiv": [1, 4], "move": [0, 4, 5, 6, 7, 9, 12, 13, 14, 15, 16, 22, 26, 27], "mpl": [7, 25], "mpl_toolkit": [2, 6, 13, 27], "mplot3d": [2, 6, 13, 27], "mplregressor": 1, "mse": [0, 4, 5, 6, 9, 10, 15, 16, 17, 18, 25, 26, 27], "mse_simpletre": 10, "mselassopredict": [5, 27], "mselassotrain": [5, 27], "mseownridgepredict": [6, 26, 27], "msepredict": [5, 27], "mseridgepredict": [0, 5, 6, 26, 27], "msetrain": [5, 27], "msle": [0, 25], "mt": [7, 12], "mu": [0, 6, 11, 13, 22, 25], "mu0": 22, "mu1": 22, "mu2": 22, "mu_": [6, 22, 26], "mu_i": [6, 26], "mu_n": 11, "mu_x": 22, "much": [0, 1, 2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 13, 15, 20, 22, 25, 26, 27], "multi": [0, 1, 3, 7, 19, 25], "multiclass": [1, 7], "multidimension": [11, 12, 25], "multilay": 1, "multinomi": 7, "multipl": [2, 4, 5, 6, 7, 12, 13, 15, 22, 26, 27], "multipli": [3, 5, 6, 11, 13, 18, 20, 22, 26, 27], "multiplum": 8, "multivari": [0, 2, 10, 11, 19, 22, 25], "multivariate_norm": [11, 14], "multpli": 16, "murphi": [11, 24, 25], "must": [1, 2, 5, 6, 8, 10, 12, 13, 14, 15, 22, 26, 27], "mutat": 7, "mutual": [1, 3, 6, 13], "mx_": 22, "my": 25, "myenv": [], "myriad": [0, 19, 25], "mz1": 22, "mz2": 22, "m\u00f8svatn": 6, "n": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 22, 25, 26, 27], "n1": 20, "n2": 20, "n_": [1, 2, 3, 8, 12, 22], "n_0": [12, 22], "n_boostrap": [6, 10], "n_bootstrap": 6, "n_categori": [1, 3], "n_cluster": 14, "n_compon": 11, "n_epoch": 13, "n_estim": 10, "n_examples_to_gener": 4, "n_featur": [1, 18], "n_filter": 3, "n_hidden": 2, "n_hidden_neuron": [0, 1, 25], "n_i": 22, "n_input": [0, 1, 3, 26], "n_instanc": 9, "n_job": 10, "n_k": 14, "n_l": [12, 22], "n_layer": 1, "n_m": 9, "n_neuron": 1, "n_neurons_connect": 3, "n_neurons_layer1": 1, "n_neurons_layer2": 1, "n_point": 14, "n_sampl": [6, 8, 9, 10, 14, 18], "n_split": 6, "n_step": 4, "n_t": 2, "n_x": 2, "nabla": [1, 13, 27], "nabla_": [2, 13, 27], "nabla_w": 13, "nag": 13, "naimi": [0, 25], "naiv": 7, "naive_kmean": 14, "name": [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 14, 15, 19, 20, 22, 23, 25, 26, 27], "narrow": 13, "nation": [1, 5], "nativ": [19, 25], "natur": [0, 1, 4, 8, 9, 12, 13, 22, 24, 25, 27], "navier": 12, "navig": 15, "nb": 22, "nb_": 20, "nbconvert": 25, "nd": 14, "ndarrai": 6, "ne": [9, 10, 20, 22, 26, 27], "nearest": [1, 3, 6, 11], "nearli": [13, 27], "neat": 25, "neccesari": 6, "necess": 2, "necessari": [0, 1, 3, 4, 8, 14, 18, 25], "necessarili": [0, 4, 11, 22, 25], "necesserali": 5, "neck": 7, "need": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 20, 22, 26, 27], "neg": [0, 1, 3, 5, 6, 7, 10, 13, 20, 22, 25, 27], "neg_mean_squared_error": 6, "neglect": 22, "neglig": 22, "neighbor": [3, 6, 11], "neither": [4, 13], "neq": [13, 14, 22, 27], "nervou": 12, "nest": [9, 12], "nesterov": 13, "net": [2, 4, 12], "netlib": [20, 25], "network": [0, 9, 13, 19, 24, 26], "neural": [0, 13, 19, 24, 26], "neural_network": [0, 1, 2, 25], "neuralnetwork": 1, "neuron": [1, 2, 3, 4, 12], "neutral": [0, 25], "neutron": [0, 25], "never": [1, 4, 6, 9, 22], "new": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 17, 20, 25, 26, 27], "new_chang": 13, "new_hobbit": 25, "newaxi": [0, 3, 6, 9], "newli": [0, 25], "newton": [1, 7, 8, 13, 22], "next": [0, 1, 2, 3, 4, 5, 6, 8, 9, 13, 14, 15, 16, 25, 26, 27], "next_guess": 13, "next_input": 4, "ng": 1, "ni": 14, "nice": [0, 1, 5, 11, 25, 26, 27], "nicer": 18, "niter": [13, 27], "nitric": [], "nlambda": [0, 5, 6, 26, 27], "nlp": 24, "nm": 22, "nm_n": [0, 25], "nmse": 6, "nn": [2, 5, 6, 12, 20, 25], "nn_model": 1, "nnmin": 2, "node": [1, 3, 9, 10, 12], "nois": [0, 4, 5, 6, 8, 9, 10, 13, 18, 25, 26, 27], "noise_dimens": 4, "noisi": [1, 6], "non": [0, 1, 3, 5, 6, 7, 9, 10, 11, 12, 13, 14, 18, 20, 22, 25, 26, 27], "none": [0, 1, 2, 4, 5, 9, 10, 13, 22, 25, 26], "nonlinear": [3, 6, 8, 9, 11, 12], "nonneg": [6, 9, 13, 27], "nonparametr": 6, "nonsens": 22, "nonsingular": 20, "nonumb": [3, 7, 8, 13, 20], "nor": [1, 4, 13], "norm": [0, 1, 5, 6, 8, 11, 13, 18, 25, 26, 27], "normal": [3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 16, 17, 18, 19, 20, 22, 25, 26, 27], "normali": [20, 25], "norwai": [6, 25, 27], "notat": [0, 2, 5, 6, 13, 14, 22, 25, 26, 27], "note": [0, 1, 2, 3, 4, 5, 6, 7, 8, 11, 12, 13, 14, 15, 16, 19, 20, 22, 24, 25], "notebook": [0, 1, 3, 9, 15, 16, 19, 25], "noth": [1, 2, 5, 8, 12, 14, 22, 26, 27], "notic": [4, 5, 12, 13, 20, 22, 25], "notion": 3, "novel": [3, 6, 10, 25], "novemb": [1, 23, 25], "now": [0, 2, 4, 5, 6, 7, 8, 10, 11, 12, 14, 15, 16, 19, 20, 22, 25, 26], "nowadai": [0, 1, 3, 9, 19, 25], "nox": [], "np": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 16, 17, 18, 20, 22, 25, 26, 27], "npr": 2, "nsampl": 6, "nt": 2, "nu": 22, "nuclear": [5, 26, 27], "nuclei": [0, 22, 25], "nucleon": [0, 25], "nucleu": [0, 25], "num": 4, "num_coordin": 2, "num_hidden_neuron": 2, "num_it": [2, 18], "num_neuron": 2, "num_neurons_hidden": 2, "num_point": 2, "num_tre": 10, "num_valu": 2, "number": [1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 18, 20, 21, 23, 25, 27], "numberid": 7, "numberparamet": 3, "numer": [0, 5, 6, 9, 10, 11, 12, 13, 19, 20, 24, 25, 26, 27], "numpi": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 22, 26, 27], "nunmpi": [5, 26], "nx": 2, "ny": 22, "o": [0, 1, 4, 5, 6, 7, 8, 9, 11, 20, 23, 24, 25, 26, 27], "obei": [6, 11, 13, 26], "object": [0, 1, 4, 8, 10, 15, 20, 25], "obliqu": [5, 26, 27], "observ": [0, 1, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 22, 25, 27], "obtain": [0, 1, 5, 6, 7, 8, 9, 10, 12, 13, 14, 17, 20, 22, 25, 26, 27], "obviou": [5, 6, 11, 22, 26, 27], "obviouli": 25, "obvious": [0, 4, 5, 6, 20, 25], "oc": [26, 27], "occupi": [], "occur": [0, 6, 8, 9, 20, 22, 25], "octob": [23, 25], "od": 0, "odd": [0, 3, 7, 25, 26], "odenum": 2, "odesi": 2, "oen": 0, "off": [1, 3, 4, 5, 9, 13, 22], "offer": [6, 11, 19, 20, 21, 23, 25], "offic": [23, 25], "offici": [21, 25], "often": [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 19, 20, 22, 25, 26, 27], "ofter": [20, 25], "ol": [0, 13, 17, 26], "old": [1, 5, 10, 13, 15], "ols_paramet": 16, "ols_sk": 6, "ols_svd": 6, "olsbeta": 27, "olstheta": [0, 5], "omega": [2, 3, 6], "omega_0": 3, "omit": [0, 5, 25, 26, 27], "onc": [1, 6, 9, 11, 13], "one": [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 18, 19, 20, 22, 23, 25, 26], "onehot": 1, "onehot_vector": 1, "onehotencod": 9, "ones": [0, 2, 5, 6, 8, 9, 10, 11, 13, 16, 18, 20, 25, 26, 27], "ones_lik": 4, "ong": 26, "onl": 3, "onli": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 18, 20, 22, 25, 26, 27], "onlin": [11, 15, 21], "onto": [5, 11, 26, 27], "open": [0, 1, 4, 6, 7, 9, 15, 19, 21, 23, 25], "oper": [0, 1, 3, 5, 6, 10, 11, 12, 13, 15, 16, 19, 22, 25, 26, 27], "operation": 22, "oplu": 22, "opmiz": 13, "opportun": 0, "oppos": [6, 13], "opposit": [1, 5, 8, 26, 27], "opt": [1, 5, 25, 27], "optim": [0, 2, 3, 4, 5, 6, 7, 9, 10, 11, 14, 16, 17], "optimis": [1, 3], "option": [0, 1, 3, 5, 6, 8, 11, 15, 18, 20, 26], "optmiz": [1, 8, 13, 26], "oral": 25, "orang": 0, "order": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 15, 20, 22, 25, 26, 27], "ordinari": [0, 2, 3, 7, 11, 13, 17, 18, 19], "oreilli": [24, 25], "org": [0, 3, 4, 16, 19, 20, 24, 25, 26, 27], "organ": [6, 7, 10, 20], "orient": [1, 5, 22, 26, 27], "origin": [0, 3, 5, 6, 8, 11, 12, 13, 15, 20, 25, 26, 27], "orthogn": [5, 26, 27], "orthogon": [0, 5, 6, 8, 11, 13, 20, 25, 26, 27], "orthonorm": [5, 26, 27], "os": [23, 25], "oscar": 1, "oscil": [3, 13], "oskar": 25, "oskarlei": 25, "osl": 18, "oslo": [0, 19, 21, 23, 25, 26, 27], "osx": [0, 19, 25], "other": [0, 1, 2, 3, 5, 6, 7, 8, 10, 13, 14, 16, 19, 21, 22, 23, 24, 26, 27], "otherwis": [0, 1, 4, 7, 13, 20, 25], "ouput": [5, 7, 12], "our": [1, 2, 3, 6, 7, 8, 9, 10, 12, 14, 15, 16, 17, 18, 19, 20, 22], "ourmodel": 0, "ourselv": [0, 5, 6, 8, 11, 13, 25, 26, 27], "out": [0, 1, 2, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16, 19, 20, 22, 25, 26], "out_fil": 9, "outcom": [0, 7, 9, 10, 12, 22, 26], "outdoor": 9, "outer": [6, 12, 13], "outfil": 4, "outlier": [0, 8, 25, 26], "outlin": [6, 10, 11], "outlook": 9, "outperform": 10, "output": [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 20, 22, 25, 26, 27], "output_bia": 1, "output_bias_gradi": 1, "output_shap": 4, "output_weight": 1, "output_weights_gradi": 1, "outputlayer1": 12, "outputlayer2": 12, "outsid": 4, "over": [0, 1, 3, 4, 5, 6, 9, 10, 12, 13, 15, 16, 20, 25, 26, 27], "over1": 13, "overal": [1, 10], "overcast": 9, "overcom": [12, 13], "overdetermin": [0, 25], "overfit": [0, 1, 3, 6, 9, 10, 13], "overflow": 5, "overhead": 12, "overlap": [3, 7, 8, 9], "overlin": [0, 5, 6, 9, 10, 11, 14, 20, 25, 26], "overst": 0, "overtrain": 4, "overview": 3, "own": [4, 5, 6, 8, 12, 13, 16, 18, 19, 20, 27], "owner": [], "ownmsepredict": 0, "ownmsetrain": 0, "ownridgebeta": 26, "ownridgetheta": [0, 6, 26, 27], "ownypredictridg": 0, "ownytilderidg": 0, "oxid": [], "p": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 16, 17, 18, 20, 22, 25, 26, 27], "p0": 2, "p1": 2, "p_": [2, 4, 8, 9], "p_hidden": 2, "p_i": [5, 22], "p_j": 22, "p_n": 22, "p_output": 2, "p_x": 22, "pack": [0, 25], "packag": [0, 1, 3, 4, 5, 8, 11, 13, 15, 19, 22, 26, 27], "packtpub": 25, "packtpublish": 25, "pad": [3, 4], "page": [0, 19, 25, 27], "pai": [0, 1, 9, 13, 15], "pair": [0, 2, 3, 9, 19, 22, 25], "paltform": 15, "panda": [0, 4, 5, 6, 7, 9, 11, 19, 27], "panel": 25, "paper": 1, "paradigm": [0, 25], "parallel": [10, 13, 19, 20, 25], "param": 2, "paramat": 2, "paramet": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 16, 17, 18, 22, 27], "parameter": [0, 6, 10, 25, 26], "parametr": [0, 6, 25, 26], "paramt": [3, 5], "part": [0, 1, 3, 5, 6, 10, 17, 20, 21, 22, 23, 25, 26], "partial": [0, 1, 5, 6, 7, 8, 10, 11, 12, 13, 16, 22, 25, 26, 27], "particip": [15, 19, 21, 23, 25], "particl": [0, 4, 13, 22, 25], "particular": [0, 1, 2, 3, 5, 6, 9, 10, 11, 12, 13, 16, 22, 24, 25, 26, 27], "particularli": [5, 6, 8, 11, 13, 22, 26, 27], "partit": [1, 4, 9], "partli": [6, 25], "partner": 15, "pass": [2, 3, 12, 14], "past": [10, 22], "patch": [6, 22], "path": [0, 4, 6, 7, 9, 19, 25], "pathcollect": 17, "patient": 7, "patter": 4, "pattern": [0, 3, 4, 12, 24, 25], "pauli": [0, 25], "pc": [11, 15, 19], "pca": [0, 7, 19, 25, 26], "pd": [0, 4, 5, 6, 7, 9, 11, 25, 26, 27], "pde": 2, "pdf": [0, 3, 4, 5, 6, 9, 15, 16, 24, 25], "pedagog": [0, 25, 26], "penal": [6, 18, 26], "penalti": [6, 13, 18, 26], "penros": [5, 6], "pentagon": [13, 27], "peopl": [1, 9, 13, 19], "per": [0, 1, 6, 21, 23, 25], "percentag": [10, 11, 23], "perceptron": [0, 1, 7, 25], "peregrin": 25, "perfect": [0, 1, 13, 25], "perfectli": [4, 6], "perform": [0, 2, 3, 4, 5, 6, 8, 10, 11, 12, 13, 14, 16, 18, 19, 20, 22, 25, 26, 27], "performac": 4, "perhap": [0, 5, 13, 25, 26, 27], "perimet": 1, "period": [1, 4, 22], "permiss": 15, "permut": 11, "persist": 13, "person": [5, 6, 7, 16, 21, 23, 25, 26], "perspect": 24, "pertin": [12, 25], "petal": [8, 9], "peter": [24, 26], "phantom": 22, "phase": [6, 12], "phenomena": 22, "phi": 8, "phi_k": 8, "philosophi": 13, "phone": [23, 25], "photo": [4, 25], "phrase": [0, 25], "physic": [0, 1, 4, 7, 12, 13, 22, 23, 24, 25, 26, 27], "pi": [2, 3, 5, 6, 7, 9, 12, 13, 22], "pick": [1, 9, 10, 11, 13, 14], "pickl": 1, "pictur": [0, 25], "pie": [19, 25], "piec": [11, 14], "pillow": [0, 19, 25], "pinv": [5, 6, 13, 26, 27], "pip": [0, 1, 15, 19, 25], "pip3": [0, 1, 25], "pipelin": [0, 6, 8, 10, 26], "pippin": 25, "pit": 4, "pitfal": [6, 26], "pitt": 12, "pixel": [1, 3, 4, 25], "pixel_height": [1, 3], "pixel_width": [1, 3], "place": [0, 4, 6, 8, 13, 15, 20, 25, 27], "plai": [0, 3, 4, 5, 6, 8, 11, 19, 25, 26, 27], "plain": [8, 10, 12, 13, 14, 27], "plan": [6, 9, 23, 24, 25], "plane": [8, 9], "plateau": [5, 27], "platform": [19, 25], "plausibl": 12, "pleas": [13, 23, 25], "plenti": 1, "plethora": [3, 12], "plot": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 18, 19, 20, 22, 25, 26, 27], "plot_all_sc": 26, "plot_confusion_matrix": [7, 10], "plot_count": 6, "plot_cumulative_gain": [7, 10], "plot_data": 1, "plot_dataset": 8, "plot_decision_boundari": [9, 10], "plot_import": 10, "plot_max": 4, "plot_min": 4, "plot_model": 4, "plot_numb": 4, "plot_predict": 8, "plot_regression_predict": 9, "plot_result": 4, "plot_roc": [7, 10], "plot_surfac": [2, 6, 13], "plot_train": 9, "plot_tre": [9, 10], "plt": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 20, 22, 25, 26, 27], "plu": [0, 3, 5, 7, 18, 25, 26], "pm": 8, "pmatrix": 2, "pml": 24, "pn": 3, "png": [0, 4, 6, 7, 9, 25], "point": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 13, 14, 20, 22, 23, 25, 26, 27], "point_1": 4, "point_2": 4, "poisson": [19, 22, 25], "poli": [6, 8], "poly100_kernel_svm_clf": 8, "poly3": 0, "poly3_plot": 0, "poly_featur": [8, 9, 15], "poly_features10": 9, "poly_fit": 9, "poly_fit10": 9, "poly_kernel_svm_clf": 8, "poly_model": 15, "poly_ms": 15, "poly_predict": 15, "polydegre": [0, 5, 6, 10, 26], "polygon": [13, 27], "polym": 12, "polynomi": [0, 5, 6, 7, 8, 9, 10, 11, 15, 17, 25, 26], "polynomial_featur": [6, 15, 16, 17], "polynomial_svm_clf": 8, "polynomialfeatur": [0, 6, 8, 9, 15, 16, 26], "polytrop": [0, 6], "pool": 3, "pool_siz": 3, "poor": [1, 13, 27], "poorli": [0, 26], "popul": [0, 5, 25, 26], "popular": [0, 1, 3, 6, 7, 8, 9, 11, 12, 15, 19, 20, 22, 26], "popularli": [0, 25], "portabl": 10, "portion": [11, 13], "pose": [0, 4, 5, 6, 11, 22, 25], "posit": [0, 1, 2, 3, 5, 7, 8, 10, 11, 13, 14, 20, 22, 25, 26, 27], "possibl": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 19, 20, 22, 23, 25, 26, 27], "possibli": [6, 8, 13], "posterior": 5, "postpon": [0, 26], "postul": 5, "potenti": [0, 3, 5, 6, 12, 13, 26], "pott": 12, "power": [0, 1, 5, 6, 8, 9, 12, 13, 25, 26, 27], "pp": [5, 6], "practic": [0, 5, 6, 7, 8, 16, 18, 22, 26], "practition": [0, 1, 3, 25], "pre": 25, "preced": [1, 11, 12, 22], "preceed": 4, "preceq": 8, "precis": [0, 2, 5, 11, 13, 20, 22, 25, 26], "pred": 6, "predicit": 0, "predict": [0, 1, 5, 6, 7, 8, 9, 10, 15, 16, 17, 18, 19, 24, 25, 26, 27], "predict_prob": 1, "predict_proba": [7, 10], "predictor": [0, 5, 6, 7, 9, 10, 11, 25, 26], "prefer": [0, 1, 6, 8, 9, 11, 13, 15, 19, 25], "prepar": [0, 6, 20, 25, 26], "preprocess": [0, 4, 6, 7, 8, 9, 10, 11, 15, 16, 17, 18], "prerequisit": 0, "presenc": 13, "present": [0, 5, 6, 7, 9, 12, 13, 20, 22, 25, 26, 27], "preserv": [3, 11, 20], "press": [13, 15, 24, 27], "pretrain": [1, 4], "pretti": [0, 4, 8, 9, 19, 25], "prev_centroid": 14, "prevent": [13, 22], "previou": [0, 1, 2, 3, 4, 5, 6, 8, 10, 11, 12, 13, 15, 16, 20, 22, 26, 27], "previous": [2, 3, 9, 10, 22], "price": [0, 4, 9, 13], "primal": 8, "primari": [0, 7, 25], "prime": 22, "princip": [0, 5, 7, 19, 25, 26, 27], "principl": [0, 6, 7, 8, 14, 25], "print": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 18, 20, 22, 25, 26, 27], "print_funct": [8, 9], "printout": [0, 25], "prior": [0, 5, 6, 25], "privat": 0, "prob": [1, 22], "probabilist": [0, 24, 25, 26], "probabl": [0, 1, 3, 4, 6, 7, 10, 13, 19, 25, 26], "problem": [0, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 17, 19, 20, 22], "probml": 24, "proce": [0, 5, 6, 7, 8, 9, 10, 11, 13, 20, 25, 26], "procedur": [2, 4, 5, 6, 8, 10, 11, 13, 26, 27], "proceed": 20, "process": [0, 2, 4, 6, 9, 10, 12, 13, 19, 20, 22, 24, 25, 27], "prod": 24, "prod_": [1, 5, 7], "produc": [0, 3, 4, 5, 6, 9, 10, 11, 12, 13, 18, 19, 20, 22, 25, 26], "product": [0, 1, 3, 5, 6, 7, 8, 12, 13, 16, 17, 19, 20, 25, 26], "profess": [0, 25], "program": [0, 1, 4, 5, 6, 8, 12, 14, 15, 19, 20, 21, 22, 23, 25, 26], "programm": 20, "progress": [1, 4, 14], "prohibit": 6, "project": [0, 1, 2, 3, 5, 11, 13, 15, 19, 21, 26, 27], "project_root_dir": [0, 6, 7, 9, 25], "promin": 12, "promis": 8, "promot": [23, 25], "prone": [9, 15], "pronounc": [13, 19, 25], "proof": [0, 11, 12, 13, 25, 27], "propag": [2, 3, 13], "proper": [0, 2, 6, 7], "properli": [1, 6, 8, 10, 13, 18], "properti": [0, 1, 3, 12, 13, 16, 20, 25], "proport": [0, 1, 5, 9, 11, 13, 22, 25, 26], "propos": [1, 4, 6, 10, 25], "propto": [5, 13, 27], "proton": [0, 25], "prove": [3, 13, 27], "provid": [0, 1, 3, 4, 5, 6, 8, 9, 10, 12, 13, 19, 20, 22, 25, 26, 27], "proxi": [1, 13], "prune": 9, "pseudo": [20, 22], "pseudoinv": 5, "pseudoinvers": [5, 6], "pseudorandom": [6, 22], "psychologi": [0, 25], "pt": 13, "public": [0, 15, 19, 25], "pull": 15, "punish": [0, 1, 25], "pure": [3, 9, 22], "purest": 9, "puriti": 9, "purpos": [0, 3, 10, 12, 14, 25], "push": 15, "put": 1, "py": 5, "pycod": 25, "pydata": 19, "pydot": 9, "pyhton2": 25, "pylab": [7, 25], "pypi": 19, "pyplot": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 20, 22, 25, 26, 27], "pythagora": 5, "python": [1, 2, 3, 5, 6, 8, 11, 12, 13, 14, 22, 26], "python2": 0, "python3": [0, 19, 25], "pytorch": [0, 19, 25], "q": [5, 6, 8, 11, 22], "qp": 8, "qquad": [2, 11, 13, 20], "qr": [5, 6, 20, 26, 27], "quad": [1, 13, 20], "quadrat": [0, 8, 9, 13, 25], "qualit": [4, 9, 22], "qualiti": [0, 9, 19, 25, 26], "quantifi": 1, "quantil": 10, "quantit": [0, 6, 9, 25], "quantiti": [0, 2, 5, 6, 7, 9, 10, 11, 12, 14, 16, 20, 22, 25, 26, 27], "quantum": [4, 12, 24, 25], "quartil": [0, 26], "quench": 5, "queri": 9, "question": [0, 5, 6, 9, 11, 12, 13, 23, 25, 26], "qugan": 4, "quick": [4, 22], "quickli": [1, 3, 9, 11, 13, 27], "quit": [1, 5, 6, 9, 10, 12, 15, 26, 27], "quot": 4, "r": [0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 19, 20, 22, 26, 27], "r2": [0, 5, 6, 25, 26, 27], "r2_score": [0, 25], "r2score": [0, 25], "r_1": 9, "r_2": 9, "r_j": 9, "r_m": 9, "rad": [], "radial": [8, 12], "radioact": 22, "radiu": [0, 1, 26], "rain": 9, "ramp": 1, "ran0": 22, "ran1": 22, "ran2": 22, "ran3": 22, "rand": [0, 4, 5, 6, 9, 10, 13, 15, 20, 25, 26, 27], "randint": [6, 9, 13], "randn": [0, 1, 2, 5, 6, 9, 11, 13, 15, 18, 25, 26, 27], "random": [0, 1, 2, 3, 4, 5, 6, 8, 9, 13, 14, 15, 16, 17, 18, 19, 20, 25, 26, 27], "random_forest_model": 10, "random_index": 13, "random_indic": [1, 3], "random_st": [7, 8, 9, 10, 11], "randomforestclassifi": 10, "randomli": [1, 6, 9, 13, 14, 18, 27], "rang": [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, 14, 18, 20, 22, 25, 26, 27], "rangl": [0, 6, 11, 22, 25, 26], "rangle_x": 22, "rank": [5, 26, 27], "rankdir": 4, "raphson": [1, 8, 13], "rapidli": 0, "rare": [1, 13], "raschka": [25, 26], "rasckha": 25, "rashcka": 27, "rate": [1, 2, 3, 4, 8, 9, 10, 12, 13, 18, 27], "rather": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 20, 22, 25, 26, 27], "ratio": [4, 7, 9, 10, 11], "rational": [0, 25], "ravel": [5, 6, 7, 8, 9, 10, 11, 13, 20], "raw": 3, "rbf": [8, 11, 12], "rbf_kernel_svm_clf": 8, "rbf_pca": 11, "rc": 22, "rcond": [0, 25, 26], "rcparam": [1, 3, 7, 8, 9, 10, 22, 25], "re": [2, 4, 13, 15, 27], "reach": [1, 4, 5, 6, 9, 10, 12, 13, 14, 27], "read": [0, 2, 3, 4, 5, 6, 7, 8, 11, 12, 16, 17, 20, 22, 24, 27], "read_csv": [0, 6, 7, 9], "read_fwf": [0, 25], "reader": [0, 6, 20, 22, 25, 26], "readi": [0, 1, 5, 6, 8, 10, 11, 12, 20, 25], "readili": 1, "readm": 15, "readthedoc": 19, "real": [0, 1, 4, 7, 10, 11, 12, 16, 18, 20, 26], "real_loss": 4, "real_output": 4, "realist": [8, 25], "realiti": 22, "realiz": [1, 12], "realli": [0, 1, 25], "rearrang": 13, "reason": [0, 1, 3, 4, 10, 13, 24, 25, 27], "reassign": 1, "recal": [5, 6, 9, 10, 11, 12, 20, 22, 25, 26, 27], "recast": 3, "receiv": [1, 3, 10, 12, 22], "recent": [0, 6, 13, 24], "recept": [3, 12], "receptive_field": 3, "recip": [0, 6, 7, 20, 25, 26], "reciproc": 5, "recogn": [0, 4, 5, 10, 25], "recognit": [0, 1, 3, 12, 24, 25], "recommen": 25, "recommend": [0, 2, 3, 4, 5, 6, 8, 13, 15, 19, 20, 24, 27], "reconsid": 9, "reconstruct": 11, "record": [10, 21, 23, 25], "recreat": 15, "rectangl": [9, 13, 27], "rectangular": [5, 26, 27], "rectifi": [1, 3, 12], "recur": [0, 19, 25], "recurr": [0, 1, 19, 25], "recurs": [9, 19, 20, 25], "red": [0, 3, 4, 6, 8, 9], "redefin": [0, 10, 25, 26, 27], "redefinit": 27, "reduc": [1, 3, 5, 6, 9, 10, 11, 13, 25, 27], "reduct": [0, 10, 11, 19, 22, 25, 26], "refer": [0, 1, 2, 3, 5, 6, 11, 12, 13, 14, 20, 24, 25, 26, 27], "referenc": 2, "refin": 12, "refit": 6, "reflect": [0, 1, 4, 5, 22, 25], "refresh": [19, 25], "refreshprogrammingskil": 25, "reg": [10, 11], "regard": [1, 9, 13], "regardless": [12, 16], "region": [3, 4, 6, 9, 12], "regist": [6, 22], "reglasso": [5, 27], "regr_1": [0, 9], "regr_2": [0, 9], "regr_3": [0, 9], "regress": [1, 8, 11, 12, 16, 19, 20], "regressor": [0, 7, 10], "regridg": [0, 5, 6, 26, 27], "regular": [0, 3, 4, 5, 6, 7, 9, 13, 17, 18, 23, 25, 26, 27], "regularli": 15, "reilli": [0, 24, 25], "reinforc": [0, 8, 19, 25], "reiter": 1, "reject": 7, "rel": [0, 4, 6, 7, 9, 12, 13, 22, 25, 26], "relat": [0, 1, 3, 4, 5, 11, 13, 14, 20, 22, 25, 26, 27], "relationship": [0, 4, 9, 18, 25], "relativeerror": [0, 25, 26], "releas": [1, 19, 25], "relev": [0, 1, 5, 7, 11, 19, 22, 25, 27], "reli": [0, 6, 8], "reliabl": [7, 22], "relu": [3, 4, 25], "remain": [1, 2, 4, 6, 12, 20, 22, 26], "remaind": 22, "reman": 2, "remark": 1, "rememb": [0, 8, 13, 20, 25], "remind": [0, 5, 11, 13, 20, 22], "remot": 15, "remov": [4, 5, 6, 18, 26, 27], "renam": 15, "render": [0, 25, 26], "reorder": [5, 7, 26, 27], "reorgan": [0, 25], "repeat": [0, 1, 3, 4, 5, 6, 9, 10, 11, 13, 14, 20, 22, 25, 26, 27], "repeated": 25, "repeatedli": [0, 6, 10, 13], "repet": 3, "repetit": [6, 25, 26], "rephras": [13, 27], "replac": [0, 1, 3, 4, 5, 6, 10, 12, 14, 19, 25, 26, 27], "replica": 6, "repo": 15, "report": 25, "repositori": [4, 25], "repres": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 22, 25, 26, 27], "represent": [0, 1, 3, 6, 22, 25], "representd": 3, "reproduc": [0, 5, 6, 9, 12, 15, 16, 18, 19, 22, 25, 26], "repuls": [0, 25], "request": [0, 13], "requir": [0, 1, 3, 4, 5, 6, 8, 9, 11, 12, 13, 15, 17, 18, 20, 25, 26, 27], "res1": 2, "res2": 2, "res3": 2, "res_analyt": 2, "res_analytical1": 2, "res_analytical2": 2, "res_analytical3": 2, "resaml": 6, "resampl": [0, 7, 10, 19, 25, 26], "rescal": [0, 11, 12], "rescu": 5, "reseach": 6, "research": [0, 4, 13, 19, 24, 25], "resembl": [6, 22], "reserv": [1, 5, 6, 22], "reshap": [0, 1, 2, 3, 4, 6, 8, 9, 10, 20, 25, 26], "residenti": [], "residu": [0, 5, 13, 25], "resiz": [5, 26, 27], "resourc": 25, "respect": [0, 1, 2, 3, 5, 6, 7, 8, 10, 11, 12, 13, 14, 16, 17, 18, 22, 25, 26, 27], "respond": 12, "respons": [0, 7, 9, 12, 25, 26], "rest": [0, 5, 18, 26, 27], "restat": [0, 12, 25], "restor": 4, "restored_discrimin": 4, "restored_gener": 4, "restrict": [0, 3, 9, 12, 25], "result": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 25], "retail": [], "retain": [5, 6, 26, 27], "return": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 13, 14, 16, 17, 20, 22, 25, 26, 27], "return_data": 14, "return_sequ": 4, "return_x_i": 9, "reus": [1, 3, 6], "reveal": [0, 12, 25], "revers": [1, 20], "review": [19, 20], "revisit": 14, "revolut": 25, "reward": [0, 4, 25], "rewrit": [0, 3, 5, 6, 7, 8, 10, 11, 12, 13, 16, 20, 22, 27], "rewritten": [2, 6, 8, 10, 22], "rewrot": 13, "rf": 10, "rgb": 3, "rgoj5yh7evk": 19, "rh": 6, "rho": [0, 10, 13], "rho_1": 10, "rho_2": 10, "rho_m": 10, "rich": [0, 25], "ride": 9, "rideclass": 9, "ridedata": 9, "ridg": [7, 11, 13, 19, 25], "ridge_paramet": 17, "ridge_sk": 6, "ridgebeta": 27, "ridgetheta": 5, "right": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 12, 13, 14, 16, 17, 20, 22, 25, 26, 27], "right_sid": 2, "rightarrow": [0, 1, 5, 6, 8, 11, 12, 13, 22, 25, 26, 27], "rigor": [0, 25, 26, 27], "ring": 6, "rise": [0, 25], "risk": [0, 13, 25, 27], "rival": 4, "river": [], "rlm": 25, "rm": 22, "rmse": [], "rmsporp": 13, "rmsprop": [1, 3, 4, 13], "rnd_clf": 10, "rng": 22, "rnn": [4, 12], "rnn1": 4, "rnn2": 4, "rnn_2layer": 4, "rnn_input": 4, "rnn_output": 4, "rnn_train": 4, "rntrick1": 22, "rntrick2": 22, "rntrick3": 22, "rntrick4": 22, "ro": [0, 13, 25, 27], "robert": 24, "robust": [0, 25], "robustscal": [0, 26], "roc": [7, 10], "role": [0, 2, 5, 6, 8, 18, 19, 25, 26, 27], "roll": 6, "room": [0, 23, 25], "root": [0, 5, 9, 13, 15, 22, 26, 27], "rot": 25, "rotat": [1, 8, 9, 10], "rotation_matrix": 9, "roughli": [1, 3, 18], "round": [7, 9, 13], "routin": [13, 20, 25, 27], "row": [0, 1, 2, 5, 6, 9, 11, 16, 20, 25, 26, 27], "rr": [5, 26, 27], "rrr": [5, 26, 27], "rudg": 18, "rug": [13, 27], "rule": [0, 1, 5, 6, 13, 25, 26, 27], "run": [0, 1, 2, 4, 5, 6, 8, 9, 11, 13, 15, 19, 25, 26, 27], "runtim": [1, 6, 14, 15], "rust": [0, 19, 20, 25], "rvert": 1, "rvert_2": 1, "s_": [3, 6], "s_1": 6, "s_i": [6, 7], "s_j": 6, "s_k": 6, "saddl": [13, 27], "safeguard": 18, "sai": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 20, 22, 25, 26, 27], "said": [6, 9, 13, 27], "sake": [0, 5, 7, 11, 25, 26, 27], "sale": [0, 25], "sam": 25, "same": [0, 1, 2, 3, 4, 5, 6, 8, 9, 11, 12, 14, 15, 16, 18, 20, 22, 25, 26, 27], "samm": 10, "sampl": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 13, 14, 18, 19, 20, 22, 25, 26], "sample_vari": 14, "sampleexptvari": 22, "samwis": 25, "sastri": 11, "satisfactori": [0, 25], "satisfi": [1, 2, 3, 6, 8, 13, 20, 22, 27], "satur": [1, 6], "save": [0, 4, 6, 7, 9, 13, 25], "save_fig": [0, 6, 7, 9, 10, 25], "savefig": [0, 4, 6, 7, 9, 22, 25], "savetxt": 4, "saw": [5, 26], "scalabl": 10, "scalar": [2, 5, 6, 10, 26], "scale": [0, 1, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 19, 20, 23, 25, 27], "scale_mean": 4, "scale_std": 4, "scaler": [0, 7, 8, 9, 10, 11, 17, 26], "scan": [5, 7], "scari": 5, "scatter": [0, 1, 6, 7, 8, 9, 14, 15, 17, 25, 26], "scenario": [6, 13, 27], "schedul": 13, "scheme": [1, 13, 27], "schrage": 22, "sch\u00f8yen": [6, 26], "scienc": [0, 1, 10, 12, 13, 19, 21, 22, 23, 24, 27], "scientif": [0, 19, 25], "scientist": [0, 25], "scikit": [3, 5, 6, 8, 9, 10, 13, 15, 16, 19, 20, 24], "scikit_learn": 0, "scikitlearn": 25, "scikitplot": [7, 10], "scipi": [0, 3, 5, 6, 13, 19, 20, 25, 26, 27], "scl": 6, "scm": 15, "score": [0, 1, 3, 6, 7, 9, 10, 11, 15, 16, 23, 25, 26], "scores_kfold": 6, "scratch": [1, 13, 16], "sdg": 13, "sdv4f4s2sb8": 27, "seaborn": [0, 1, 3, 6, 7, 25], "seamless": [0, 19, 25], "search": [0, 1, 3, 5, 9, 13, 15, 25, 27], "sebastian": 25, "sebastianraschka": 25, "sec": 6, "second": [0, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 14, 15, 16, 19, 20, 22, 23, 25, 26, 27], "secondeigvector": 11, "secondli": 12, "section": [4, 11, 16, 20, 22, 26], "sector": 0, "see": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 15, 16, 18, 19, 20, 22, 25, 26, 27], "seed": [0, 1, 2, 3, 4, 5, 6, 8, 9, 11, 13, 14, 18, 22, 25, 26, 27], "seed_imag": 4, "seek": [1, 2, 8], "seem": [1, 3, 4], "seemingli": [0, 25], "seen": [0, 1, 3, 5, 10, 12, 22], "segment": [13, 27], "seismic": 6, "seldomli": [0, 25], "select": [1, 5, 6, 8, 9, 10, 11, 15, 21, 22, 23, 24, 25, 26, 27], "selevet": 15, "self": [1, 5, 26], "sell": 4, "semest": [7, 21], "semi": [8, 13, 27], "semilogx": 6, "send": [5, 12, 13, 23, 25], "senior": [21, 23], "sens": [0, 4, 6, 8, 25], "sensibl": 3, "sensit": [0, 5, 6, 9, 13, 25, 26], "sent": 2, "sentenc": [4, 12], "separ": [0, 1, 2, 4, 6, 8, 9, 12, 14, 18, 19, 22, 25], "septemb": [18, 25], "sequenc": [3, 4, 7, 9, 10, 12, 13, 19, 20, 22, 25, 27], "sequenti": [1, 3, 4, 10, 12, 22], "seri": [0, 1, 2, 3, 4, 5, 6, 10, 11, 12, 13, 20, 25, 26, 27], "serif": [7, 22, 25], "serv": [0, 1, 2, 3, 5, 7, 13, 24, 25, 26, 27], "session": [1, 15, 21, 23, 25], "set": [1, 4, 5, 6, 7, 8, 10, 11, 13, 14, 16, 17, 18, 19, 20, 22, 23], "set_major_formatt": 6, "set_major_loc": 6, "set_tick": [1, 8], "set_ticklabel": 1, "set_titl": [0, 1, 2, 3, 7, 12, 14, 25], "set_xlabel": [0, 1, 2, 3, 7, 12, 25], "set_xlim": [7, 12], "set_xticklabel": 1, "set_ylabel": [0, 1, 2, 3, 7, 25], "set_ylim": [7, 12], "set_ytick": 7, "set_yticklabel": [1, 6], "set_zlim": 6, "seth": 4, "setminu": 6, "setosa": [8, 9], "setosa_or_versicolor": 8, "setp": 6, "setup": [1, 4, 6, 8, 19, 25, 26, 27], "sever": [0, 3, 5, 6, 7, 8, 9, 11, 12, 13, 16, 19, 20, 22, 25, 26, 27], "sgd": [1, 3, 27], "sgd_clf": 8, "sgdclassifi": 8, "sgdreg": 13, "sgdregressor": 13, "sgn": [5, 26, 27], "shallow": 13, "shape": [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 16, 18, 20, 25, 26, 27], "share": [1, 3, 15, 25], "shareabl": 15, "she": 7, "shift": [1, 6, 12, 15, 18, 22, 26], "ship": 3, "shire": 25, "short": [4, 5], "shortcom": [13, 27], "shorten": 4, "shorter": 22, "shorthand": 25, "shortli": [20, 25], "should": [0, 2, 3, 5, 6, 8, 9, 11, 12, 15, 18, 20, 22, 25, 26], "show": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 20, 22, 25, 26, 27], "show_shap": 4, "shown": [0, 4, 5, 8, 12, 13, 20, 26, 27], "shrink": [3, 5, 6, 8, 11, 26, 27], "shrinkag": [5, 6, 26, 27], "shrunk": 11, "shuffl": [0, 1, 4, 6, 13, 26], "side": [0, 2, 5, 8, 12, 13, 20, 25, 27], "sigh": [19, 25], "sigma": [0, 1, 5, 6, 7, 10, 11, 12, 13, 20, 22, 25, 26, 27], "sigma0": 22, "sigma1": 22, "sigma2": 22, "sigma_": [5, 20, 25, 26, 27], "sigma_0": [5, 26, 27], "sigma_1": [5, 26, 27], "sigma_2": [5, 26, 27], "sigma_fn": [7, 12], "sigma_i": [0, 5, 25, 26, 27], "sigma_j": [5, 26, 27], "sigma_m": [6, 22], "sigma_n": [11, 22], "sigma_t": 13, "sigma_x": 22, "sigmoid": [1, 2, 4, 7, 8, 10, 12], "sigmundson": [6, 26], "sign": [1, 2, 7, 8, 10, 22, 23], "signal": [1, 3, 10, 12], "signifi": 4, "signific": 1, "significantli": [1, 13, 18, 22, 27], "sim": [4, 5, 6, 13, 22], "similar": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 14, 19, 20, 25, 27], "similarli": [0, 1, 3, 5, 8, 10, 13, 22, 25, 26, 27], "simpl": [1, 2, 3, 5, 6, 7, 8, 10, 11, 12, 14, 16, 17, 19, 20, 22], "simplepredict": 10, "simpler": [0, 1, 5, 6, 7, 13, 16, 19, 25, 27], "simplernn": 4, "simplest": [0, 1, 3, 4, 9, 10, 12, 14, 25], "simpletre": 10, "simpli": [0, 1, 2, 4, 5, 6, 8, 9, 10, 11, 12, 19, 20, 22, 25, 26, 27], "simplic": [2, 5, 6, 7, 8, 9, 10, 11, 12, 14, 26, 27], "simplicti": [5, 26, 27], "simplifi": [0, 6, 9, 18, 19, 25, 26], "simplist": [3, 6, 22], "simul": [6, 18], "simultan": 6, "sin": [0, 1, 2, 3, 4, 9, 12, 13, 20, 25], "sinc": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 13, 16, 18, 20, 22, 24, 25, 26, 27], "sine": [3, 12], "singl": [0, 1, 2, 3, 5, 6, 7, 8, 9, 12, 13, 18, 20, 22, 25, 26, 27], "singular": [0, 6, 13, 20, 25], "sinusoid": 3, "site": [0, 21, 26], "situat": [0, 4, 5, 7, 13, 22, 25, 26, 27], "six": [3, 22], "size": [0, 1, 2, 3, 4, 5, 6, 8, 9, 10, 11, 13, 18, 20, 22, 25], "sketch": 10, "ski": 9, "skill": 0, "skip": 11, "skl": [0, 6, 25, 26], "sklearn": [0, 1, 3, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 17, 25, 26, 27], "skplt": [7, 10], "sl": [6, 26], "slack": 8, "slice": [2, 20, 25], "slide": [0, 3, 16, 22, 25, 26, 27], "slight": [6, 13], "slightli": [1, 2, 3, 5, 6, 7, 10, 22, 26, 27], "slope": [8, 11, 12], "slow": [0, 2, 8, 13, 18, 26, 27], "slower": [5, 20, 25, 26, 27], "slowest": 20, "slowli": 12, "slp": 1, "small": [0, 1, 2, 3, 5, 6, 8, 9, 10, 11, 12, 13, 19, 20, 22, 25, 26, 27], "smaller": [0, 1, 2, 5, 6, 8, 9, 11, 13, 22, 25, 26, 27], "smallest": [0, 4, 14, 25], "smallest_row_index": 14, "smooth": [0, 3, 6, 13, 25, 27], "sn": [0, 1, 3, 6, 7, 25], "sne": 11, "so": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 19, 20, 22, 23, 25, 26, 27], "soar": 6, "social": 0, "soft": [1, 7, 10, 12], "soften": 8, "softmax": [3, 7], "softwar": [0, 8, 19, 20], "sol": 8, "sole": [0, 6, 25], "solid": [0, 7], "solut": [0, 1, 2, 3, 5, 6, 8, 10, 11, 13, 18, 20, 22, 25, 26, 27], "soluton": 2, "solv": [0, 1, 3, 5, 6, 8, 10, 11, 12, 13, 16, 20, 25, 26], "solve_expdec": 2, "solve_ode_deep_neural_network": 2, "solve_ode_neural_network": 2, "solve_pde_deep_neural_network": 2, "solveod": 2, "solveode_popul": 2, "solver": [2, 7, 8, 9, 10, 20, 25], "some": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 15, 16, 18, 22, 25], "some_model": [6, 26], "somehow": 4, "someon": 16, "someth": [0, 1, 3, 4, 7, 9, 11, 15, 22, 25, 26], "sometim": [0, 1, 11, 12, 13, 14, 26], "soon": [20, 23, 26], "sophist": [0, 25], "sopt": 13, "sort": [5, 6, 9, 11, 22], "sound": [3, 5], "sourc": [0, 1, 3, 6, 19, 20, 22, 25], "space": [0, 1, 4, 5, 8, 9, 11, 12, 13, 14, 22, 26, 27], "span": [0, 3, 5, 9, 11, 20, 25, 26, 27], "spare": 1, "spars": [3, 6, 18, 20, 25], "sparse_mtx": [20, 25], "sparsecategoricalcrossentropi": 3, "sparsiti": [10, 18], "spatial": [1, 2, 3, 12], "speak": 22, "special": [6, 7, 10, 12, 13, 20, 22, 25, 26, 27], "specif": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 15, 16, 19, 20, 22, 24, 25, 26, 27], "specifi": [0, 3, 5, 6, 7, 9, 11, 13, 14, 22, 25, 27], "specifici": [0, 10, 25], "spectacular": 3, "spectral": 1, "speech": [0, 1, 3, 4, 12], "speed": [1, 2, 4, 13], "spend": [16, 22], "sphere": [0, 26], "spin": 6, "spite": 0, "spline": 8, "split": [1, 3, 4, 5, 6, 8, 9, 10, 11, 14, 16, 17, 22, 25, 27], "splite": 0, "splitter": [1, 10], "spontan": 22, "spot": 3, "spread": [0, 11, 22, 25, 26], "springer": [24, 25], "spuriou": 13, "sqquar": 27, "sqrsignal": 3, "sqrt": [3, 4, 5, 6, 8, 10, 11, 13, 22, 26, 27], "squar": [1, 2, 3, 4, 7, 8, 9, 11, 13, 14, 15, 17, 18, 19, 20, 22], "squarederror": 10, "squaredeuclidean": 14, "squash": 12, "srtm": 6, "srtm_data_norway_1": 6, "stabil": 5, "stabl": [0, 4, 5, 6, 9, 16, 19, 25, 26, 27], "stack": [3, 4], "stage": [5, 13, 15], "stai": [0, 2, 4, 5, 11, 25, 26], "stand": [0, 5, 9, 12, 25, 26, 27], "standard": [0, 1, 4, 5, 6, 7, 8, 10, 12, 17, 18, 20, 22, 25, 27], "standardscal": [0, 6, 7, 8, 9, 10, 11, 17, 26], "stanford": [13, 27], "start": [0, 1, 2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 13, 14, 15, 16, 20, 22, 23, 25, 26, 27], "start_tim": 14, "stat": 6, "state": [1, 2, 4, 5, 6, 7, 8, 10, 11, 12, 13, 19, 22, 25, 26, 27], "statement": [0, 7, 20, 25], "stationari": 27, "statist": [0, 1, 3, 4, 7, 9, 10, 11, 12, 13, 14, 20, 24, 26, 27], "statu": [0, 7, 15, 25], "stavang": 6, "std": [0, 4, 6, 18, 25, 26], "steep": [13, 27], "step": [0, 1, 2, 4, 6, 7, 9, 10, 11, 12, 13, 14, 15, 18, 20, 25, 27], "step_fn": [7, 12], "step_length": 13, "steps_list": 9, "stereo": 3, "still": [0, 2, 3, 5, 6, 11, 13, 22, 26, 27], "stimuli": 12, "stk": [24, 25], "stk2100": [24, 25], "stk3155": [15, 21, 23], "stk4021": [24, 25], "stk4051": [24, 25], "stk4155": [21, 23], "stk5000": 24, "stochast": [0, 1, 5, 6, 8, 11, 12, 27], "stock": 4, "stoke": 12, "stone": [0, 7], "stop": [1, 4, 9, 13, 14, 18, 27], "storag": [5, 26, 27], "store": [0, 1, 2, 3, 6, 11, 13, 18, 22, 25], "storehaug": [23, 25], "str": [1, 3, 4], "straight": [0, 6, 8, 13, 25, 27], "straightforward": [0, 2, 3, 5, 6, 8, 9, 10, 13, 20, 25, 26, 27], "strategi": [0, 1, 9, 25], "stratifi": 6, "strength": [0, 5, 14, 26, 27], "stretch": 11, "strict": [8, 13, 27], "strictli": [8, 13, 27], "stride": [4, 20], "strike": 6, "string": 1, "stroke": 7, "strong": [3, 6, 9, 10, 12, 20, 22], "strongli": [0, 8, 15, 19, 20], "stronli": [], "structur": [0, 1, 2, 3, 6, 9, 10, 12, 19, 25], "stuck": [1, 13, 27], "student": [0, 15, 21, 23, 24, 25], "studi": [0, 3, 4, 5, 6, 7, 8, 11, 12, 13, 19, 24, 25, 26, 27], "studier": 24, "style": [7, 9, 20, 25], "st\u00f8land": 23, "sub": [9, 12], "subdivid": [0, 20, 25], "subfield": 0, "subject": [6, 8, 22], "submit": 25, "subplot": [0, 1, 3, 4, 6, 7, 8, 9, 10, 14, 25], "subplots_adjust": [8, 22], "subprogram": [20, 25], "subract": [0, 26], "subroutin": [0, 25], "subscript": 1, "subsequ": [1, 4, 5, 6, 12, 20, 22, 26, 27], "subset": [1, 6, 9, 12, 13, 19, 25, 27], "subspac": [0, 8, 11, 26], "substanti": [9, 10], "substep": 11, "substitut": [3, 6, 12, 16, 20], "subsubset": 9, "subtask": 6, "subtl": 1, "subtract": [0, 4, 5, 6, 11, 13, 18, 20, 22, 26], "subtre": 9, "succeed": [0, 4, 25], "success": [3, 7, 9, 13, 22], "successfulli": [4, 9], "sudo": [0, 19, 25], "suffer": [0, 1, 2, 5, 10, 25, 26, 27], "suffici": [1, 6, 8, 11, 13, 27], "suggest": [1, 13, 24, 27], "suit": [8, 12], "suitabl": [0, 15, 22, 26], "sum": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 20, 22, 25, 26, 27], "sum_": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 20, 22, 25, 26, 27], "sum_i": [0, 2, 5, 6, 8, 13, 26, 27], "sum_j": [6, 18], "sum_ja_": 0, "sum_k": [6, 8, 12, 20], "sum_logist": 13, "sum_m": 3, "sum_n": 3, "sum_nx_": 3, "summar": [5, 6, 9], "summari": [1, 3, 4, 10, 21, 27], "summat": [0, 3, 16, 26, 27], "sunni": 9, "super": [5, 26, 27], "superfici": 3, "superscript": [1, 12], "supervis": [0, 5, 6, 7, 9, 12, 19, 25, 26, 27], "supplement": 7, "support": [0, 1, 9, 10, 11, 13, 19, 25, 26], "suppos": [0, 5, 6, 7, 8, 10, 11, 12, 13, 20, 25, 26, 27], "suppress": [5, 13, 27], "sure": [0, 1, 4, 6, 16], "surf": 6, "surfac": [0, 6, 25], "surpass": 6, "surpris": [0, 25], "surround": [3, 19], "survei": [0, 5, 6, 25, 26], "svc": [8, 9, 10], "svd": [0, 6, 11, 25], "svdinv": 5, "svm": [8, 9, 10, 11], "svm_clf": [8, 10], "swath": [5, 26, 27], "switch": 0, "sy": [13, 27], "symbol": [1, 5, 11, 13, 19, 22, 25, 26, 27], "symmeteri": 1, "symmetr": [0, 5, 8, 11, 12, 13, 20, 25, 26], "symmetri": 6, "sympi": [0, 19, 25], "synonim": 22, "syntax": 13, "system": [0, 1, 3, 4, 6, 7, 9, 10, 12, 13, 15, 19, 20, 25, 27], "systemat": [4, 6], "t": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 23, 25, 27], "t0": [3, 6, 13], "t1": [2, 13], "t2": 2, "t3": 2, "t_": 2, "t_0": [2, 9, 13], "t_1": 13, "t_b": 10, "t_i": [1, 2, 5, 12, 26, 27], "t_j": 12, "t_k": 9, "tabl": [9, 22, 23, 25], "tabul": [0, 25], "tabular": 25, "tackl": 4, "tag": [2, 3, 4, 5, 6, 7, 12, 13, 14, 20, 22, 26, 27], "taht": [0, 25], "tail": 22, "tailor": [2, 8, 11, 25], "taiwan": [0, 25], "take": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 17, 19, 20, 22, 25, 26, 27], "taken": [0, 1, 3, 6, 10, 13, 20], "tan": 3, "tangent": [1, 4, 12, 13, 27], "tanh": [1, 4, 7, 8, 12], "target": [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 15, 16, 18, 25, 26, 27], "target_nam": 9, "task": [0, 1, 3, 6, 9, 11, 12, 14, 25], "tau": [3, 5, 22], "taught": 25, "tax": [], "taylor": [2, 13, 27], "taylornr": [13, 27], "tc": 8, "teach": [15, 21, 25], "team": 1, "teaser": 0, "technic": [0, 5, 6, 13, 27], "techniqu": [0, 1, 8, 10, 13, 19, 22, 24, 25, 26], "technologi": [0, 1], "tell": [0, 4, 6, 10, 11, 13, 16, 22], "temp": 1, "temp1": 1, "temp2": 1, "temperatur": [0, 9, 25], "templat": 18, "temporarili": 1, "ten": [3, 25], "tend": [3, 5, 6, 8, 9, 10, 12, 13, 14, 26], "tendenc": [0, 25], "tension": 6, "tensor": 3, "tensorflow": [0, 2, 4, 8, 14, 19, 20, 24, 25, 26], "term": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 18, 22, 25, 26, 27], "term1": [5, 6, 11], "term2": [5, 6, 11], "term3": [5, 6, 11], "term4": [5, 6, 11], "termin": [0, 4, 5, 9, 10, 13, 15, 26, 27], "terminarl": 15, "terrain": 6, "terrain1": 6, "test": [3, 4, 5, 6, 7, 8, 9, 10, 13, 16, 20, 22, 25, 27], "test_acc": 3, "test_accuraci": [1, 3], "test_error": 6, "test_imag": [3, 4], "test_ind": 6, "test_input": 4, "test_label": [3, 4], "test_loss": 3, "test_pr": 1, "test_predict": 1, "test_rnn": 4, "test_scor": [7, 10], "test_siz": [0, 1, 3, 5, 6, 10, 15, 17, 26, 27], "test_split": 9, "testerror": [0, 6, 26], "testi": 4, "testpredict": 4, "testx": 4, "text": [0, 1, 2, 4, 5, 8, 9, 11, 13, 15, 18, 20, 22, 24, 26, 27], "textbook": [16, 26, 27], "textual": 9, "textur": 1, "tf": [1, 3, 4, 13, 14, 27], "th": [0, 1, 2, 5, 6, 7, 9, 12, 13, 14, 20, 22, 25, 26], "than": [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, 17, 19, 22, 25, 26], "thank": [4, 6, 26], "theano": [1, 19, 25], "thei": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 15, 16, 18, 20, 22, 25, 26, 27], "them": [0, 1, 3, 4, 6, 8, 9, 10, 11, 12, 13, 18, 20, 25, 26], "theme": [0, 15, 25], "themselv": [0, 22, 25], "thenc": 6, "theorem": [2, 6, 7, 26, 27], "theoret": [0, 4, 10], "theori": [0, 1, 3, 8, 9, 12, 13, 19, 24, 25], "thereaft": [0, 5, 6, 11, 12, 20, 25], "therebi": [0, 5, 7, 11, 25, 26, 27], "therefor": [0, 1, 2, 3, 4, 6, 7, 8, 11, 13, 22, 25, 26, 27], "therein": 11, "thereof": [0, 6, 13, 25], "theta": [0, 1, 4, 5, 6, 7, 13, 16, 22, 25, 26, 27], "theta_": [0, 1, 6, 7, 13, 25, 26, 27], "theta_0": [0, 5, 6, 7, 16, 25, 26, 27], "theta_0x_": [0, 25, 26], "theta_1": [0, 5, 6, 7, 25, 26, 27], "theta_1x_": [0, 25, 26], "theta_1x_0": [0, 25], "theta_1x_1": [0, 7, 25], "theta_1x_2": [0, 25], "theta_1x_i": [7, 26, 27], "theta_2": [0, 25, 26], "theta_2x_": [0, 25, 26], "theta_2x_0": [0, 25], "theta_2x_1": [0, 25], "theta_2x_2": [0, 7, 25], "theta_2x_i": 26, "theta_3x_i": 26, "theta_4x_i": 26, "theta_closed_form": 18, "theta_closed_formol": 18, "theta_closed_formridg": 18, "theta_gdol": 18, "theta_gdridg": 18, "theta_i": [0, 1, 5, 25, 26, 27], "theta_j": [0, 5, 6, 18, 25, 26], "theta_k": 27, "theta_linreg": [13, 27], "theta_ol": 18, "theta_p": 7, "theta_px_p": 7, "theta_ridg": 18, "theta_t": 13, "theta_tru": 18, "thetavalu": 5, "thi": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 24, 26, 27], "thing": [0, 1, 2, 4, 5, 7, 9, 15, 16, 22, 25], "think": [0, 1, 3, 4, 6, 9, 12, 13, 14, 22, 25, 26, 27], "third": [0, 3, 6, 13, 23, 25, 27], "thirti": 7, "thorughout": 25, "those": [0, 3, 5, 6, 8, 9, 10, 11, 20, 25, 26, 27], "though": [1, 2, 3, 4, 13, 16, 17, 20, 22], "thought": [6, 14, 22], "thousand": [0, 1, 26], "three": [0, 1, 3, 5, 6, 8, 9, 12, 20, 21, 22, 23, 25, 26, 27], "threshold": [1, 3, 9, 10, 11, 12, 13], "through": [0, 1, 2, 3, 4, 5, 6, 8, 11, 12, 13, 14, 15, 19, 20, 22, 25, 26, 27], "throughout": [0, 4, 5, 14, 15, 19, 20, 22, 25], "throw": [3, 6, 22], "thu": [0, 1, 2, 5, 6, 7, 8, 10, 11, 12, 13, 23, 25, 26, 27], "thumb": [0, 6, 26], "thursdai": [], "tibshirani": [6, 24, 25], "tick_param": 6, "ticker": [6, 13, 22, 27], "tif": 6, "tight_layout": [1, 7], "tightli": 11, "tild": [0, 5, 6, 7, 11, 22, 25, 26, 27], "till": [0, 4, 7, 8, 9, 10, 12, 20, 25, 26], "time": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 25, 26, 27], "timeit": 4, "timer": 4, "tini": 1, "tip": 3, "titl": [0, 1, 2, 3, 4, 6, 7, 8, 9, 10, 13, 15, 22, 25, 27], "tmp": 13, "tn": [2, 3, 7], "to_categor": [1, 3, 4], "to_categorical_numpi": 1, "to_numer": [0, 6, 25], "todai": 3, "togeth": [0, 3, 6, 8, 11, 13, 19, 25], "toi": 14, "told": 13, "toler": [2, 14], "tolist": 4, "tomographi": 12, "too": [0, 2, 4, 5, 6, 9, 11, 13, 17, 18, 22, 24, 26, 27], "took": [8, 25], "tool": [0, 1, 3, 6, 13, 15, 19, 26], "toolbox": 8, "top": [0, 3, 5, 6, 9, 10, 19, 25], "topic": [0, 5, 6, 7, 8, 19, 26, 27], "topolog": [3, 12], "topologi": [1, 12], "torkjellsdatt": [23, 25], "toss": [10, 22], "total": [0, 1, 2, 3, 4, 6, 7, 8, 10, 11, 12, 13, 14, 20, 22, 23, 25, 26, 27], "total_loss": 4, "totalclustervari": 14, "totalscatt": 14, "toward": [1, 2, 7, 12, 13, 15, 27], "town": [], "tp": [4, 7], "tpng": 9, "tpu": [13, 19, 25], "tqdm": 6, "tr": [], "track": [3, 13, 14, 15, 20, 26, 27], "tract": [], "tractabl": [0, 25, 26], "trade": [5, 9], "tradeoff": [0, 5, 25, 26, 27], "tradit": [0, 1, 4, 6, 25], "train": [2, 3, 5, 6, 8, 9, 10, 11, 12, 13, 16, 17, 27], "train_accuraci": [0, 1, 3, 25], "train_dataset": 4, "train_end": [0, 1, 26], "train_error": 6, "train_imag": [3, 4], "train_ind": 6, "train_label": [3, 4], "train_pr": 1, "train_siz": [0, 1, 3, 26], "train_step": 4, "train_test_split": [0, 1, 3, 5, 6, 7, 9, 10, 11, 15, 16, 17, 25, 26, 27], "train_test_split_numpi": [0, 1, 26], "trainable_vari": 4, "trained_model": [6, 26], "trainerror": [0, 26], "traini": 4, "training_checkpoint": 4, "training_dataset": 4, "training_gradi": 13, "trainingerror": 6, "trainpredict": 4, "trainscor": 4, "trainx": 4, "trait": [0, 25], "trajectori": 4, "transfer": [9, 25], "transform": [0, 5, 6, 7, 8, 9, 10, 11, 12, 13, 17, 19, 20, 25, 26, 27], "transit": [6, 12], "translat": [1, 4, 6, 10, 25, 26], "transpos": [1, 5, 11, 20, 26, 27], "travers": [0, 5], "treat": [0, 1, 3, 6, 12, 13, 18, 22, 25, 26, 27], "tree": [0, 1, 19, 25], "tree_clf": [9, 10], "tree_clf_": 9, "tree_clf_sr": 9, "tree_reg": 9, "tree_reg1": 9, "tree_reg2": 9, "trend": 22, "treue": 7, "trevor": 24, "tri": [2, 3, 4, 9, 13, 16], "triain": 0, "trial": [0, 2, 4, 6, 13, 22, 25, 27], "triangl": [13, 27], "triangular": 20, "trick": [3, 4, 8, 11, 13, 22], "trickier": 22, "tridiagon": 20, "trillion": 19, "trivial": [0, 1, 5, 11, 22, 25, 27], "troubl": [0, 8, 12, 15, 26], "truck": 3, "true": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 14, 16, 17, 18, 20, 22, 25, 26, 27], "true_beta": 26, "true_fun": 6, "true_theta": 6, "truli": 25, "try": [0, 1, 2, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 18, 19, 20, 22, 25, 26, 27], "tucker": 8, "tuesdai": [23, 25], "tumor": [7, 9], "tumour": 7, "tunabl": 1, "tune": [4, 9, 13, 20, 25], "turn": [0, 1, 5, 6, 7, 8, 9, 10, 11, 12, 13, 20, 22, 25, 26, 27], "tutori": [1, 4], "tv": 2, "tveito": 2, "tweak": [1, 4, 10, 22], "twice": [13, 27], "twist": 11, "two": [0, 1, 2, 4, 5, 6, 7, 9, 10, 11, 12, 13, 15, 17, 20, 21, 22, 24, 25, 26, 27], "tx": [13, 27], "tx_1": [13, 27], "txt": [4, 15], "ty": [13, 27], "type": [0, 1, 3, 6, 8, 10, 13, 20, 22, 26, 27], "typic": [0, 1, 2, 3, 4, 5, 7, 9, 10, 12, 13, 15, 16, 22, 25, 26, 27], "u": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 20, 22, 24, 25, 26, 27], "u_": 20, "u_i": 12, "u_m": 10, "ua": [0, 25], "ubuntu": [0, 19, 25], "uci": [], "uio": [15, 23, 24], "un": 14, "unabl": 15, "unari": [20, 25], "unbalanc": [6, 9], "unbias": [0, 5, 6, 25], "uncent": [6, 26], "uncertainti": [0, 5, 25], "uncertitud": 22, "unchang": [1, 3], "uncorrel": [10, 22], "undefin": [5, 26, 27], "under": [0, 1, 5, 6, 10, 13, 19, 25, 26, 27], "underdetermin": [0, 25], "underfit": [1, 6], "underflowproblem": 5, "undergo": 5, "undergradu": [21, 23], "underli": [0, 1, 9, 13, 18, 22, 25], "underset": [4, 14], "understand": [0, 1, 3, 5, 6, 10, 13, 14, 15, 19, 25, 26, 27], "understood": [8, 13], "undesir": 8, "undetermin": [5, 8], "undo": 4, "unexpect": 6, "unexpected": 22, "unexplain": 18, "unfair": [6, 26], "unfortun": [1, 8, 9, 10], "unicode_liter": [8, 9], "uniform": [0, 1, 5, 6, 11, 13, 22, 25, 27], "uniformli": [13, 22, 27], "unifrompdf": 22, "unimport": [13, 27], "union": [5, 6], "uniqu": [0, 2, 6, 13, 14, 20, 25], "unique_cluster_label": 14, "unit": [0, 1, 3, 4, 5, 10, 12, 18, 22, 25, 26, 27], "unitari": [5, 6, 20, 26, 27], "unitarili": [20, 25], "uniti": 22, "univari": 22, "univers": [0, 1, 2, 13, 19, 21, 23, 25, 26, 27], "unix": 1, "unknow": [0, 20, 25], "unknown": [0, 1, 3, 4, 5, 6, 8, 10, 13, 20, 25, 26, 27], "unknowwn": 12, "unlabel": 1, "unless": [0, 3, 6, 11, 13, 25, 27], "unlik": [1, 3, 8, 13, 27], "unnecessarili": 9, "unord": 3, "unravel": 1, "unrol": [3, 11], "unseen": [0, 7, 9, 15], "unstabl": 1, "unsupervis": [0, 1, 4, 12, 19, 25], "unsymmetr": [20, 25], "until": [1, 2, 4, 9, 12, 13, 14, 27], "untouch": 0, "unusu": 12, "up": [1, 3, 4, 5, 6, 8, 10, 11, 13, 14, 16, 18, 19, 20, 22, 23], "updat": [1, 2, 10, 12, 13, 14, 15, 18], "uploa": 25, "upload": [15, 19, 24], "upon": [0, 1, 6, 7, 11, 20], "upper": [0, 8, 9, 16, 20, 26], "uppercas": [20, 25], "upsampl": 4, "upscal": 4, "url": [25, 26], "us": [4, 5, 6, 8, 9, 10, 11, 12, 14, 15, 17, 20, 22, 24], "usag": [0, 8, 19, 25, 26], "usd": [], "usd10000": [], "use_bia": 4, "usecol": [0, 25], "useless": 1, "user": [0, 1, 2, 4, 6, 7, 15, 19, 20, 25, 26], "usernam": 15, "usetex": 22, "usg": 6, "usr": 22, "usual": [0, 3, 4, 7, 12, 13, 14, 25], "ut": 5, "util": [1, 3, 4, 6, 7, 10, 14, 25], "ux": 20, "v": [2, 4, 5, 6, 11, 13, 15, 19, 26, 27], "v0": 22, "v1": 22, "v2": 22, "v_0": 11, "va": 1, "vahid": 25, "val": 13, "val_accuraci": 3, "val_loss": 4, "vale": 2, "valid": [0, 1, 4, 7, 9, 10, 13, 19, 22, 25, 26], "validation_data": 3, "validation_split": 4, "valu": [0, 1, 2, 3, 4, 6, 7, 8, 9, 10, 12, 13, 14, 16, 17, 18, 19, 20, 25], "valuat": 9, "valy": 4, "van": [0, 25, 26, 27], "vandenbergh": [8, 13, 27], "vandermond": [0, 25], "vanilla": [0, 6, 11, 14, 26], "vanish": [1, 4, 13, 22, 27], "var": [5, 6, 10, 11, 22, 26], "var_x": 22, "varabl": 8, "varepsilon": [5, 6], "varepsilon_": [5, 6], "varepsilon_i": [5, 6], "vari": [0, 1, 3, 5, 6, 10, 25], "variabl": [0, 1, 2, 5, 6, 7, 8, 10, 11, 12, 13, 14, 20, 25, 26], "varianc": [0, 1, 5, 7, 9, 10, 11, 13, 14, 18, 19, 20, 22, 25, 26, 27], "variance_i": [5, 11, 26], "variance_x": [5, 11, 26], "variant": [0, 1, 6, 8, 12, 13, 25, 26, 27], "variat": [3, 4, 11, 25], "varieti": [0, 3, 12, 19, 25], "variou": [1, 3, 5, 6, 7, 8, 9, 11, 12, 13, 16, 19, 20, 22, 25, 26, 27], "varydimens": 4, "vastli": 3, "vaue": 1, "vault": 0, "vdot": [2, 13, 27], "vec": 6, "vector": [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 13, 14, 17, 18, 19, 27], "vector_mean": 14, "ventur": [0, 8, 19, 25], "venv": 15, "verbos": [1, 3, 4], "veri": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 22, 24, 25, 26, 27], "verifi": [3, 11, 20, 25], "versatil": [8, 25], "versicolor": [8, 9], "version": [0, 3, 10, 13, 14, 15, 19, 20, 22, 25], "versu": 1, "vert": [0, 1, 5, 6, 7, 8, 9, 11, 13, 16, 17, 25, 26, 27], "vert_1": [5, 6, 26, 27], "vert_2": [5, 6, 11, 17, 26, 27], "via": [0, 5, 6, 7, 8, 9, 10, 11, 12, 19, 20, 21, 22, 23, 25, 26, 27], "vidal": 11, "video": [0, 1, 12, 19, 21, 23, 25, 26, 27], "view": [1, 3, 5, 6, 12, 13, 22, 24, 25, 27], "violat": 8, "virginica": 9, "viridi": [0, 1, 2, 3, 25], "virtual": 1, "viscos": 13, "viscou": 13, "visibl": 15, "vision": [0, 3], "visual": [0, 3, 11, 12, 18, 19, 25, 26], "visualis": 1, "visualstudio": [15, 16], "viz": [6, 8, 22], "vmap": 13, "vmax": [1, 6], "vmin": [1, 6], "voic": 3, "volum": [0, 3, 25], "vote": [10, 25], "voting_clf": 10, "votingclassifi": 10, "votingsimpl": 10, "vstack": [5, 11, 20, 22, 25, 26], "vt": [5, 26, 27], "w": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 20, 22, 25, 26, 27], "w1": 8, "w2": [8, 11], "w3": 8, "w_": [1, 12], "w_1": [8, 20], "w_1x_": 8, "w_1x_1": 8, "w_2": [8, 20], "w_2x_": 8, "w_2x_2": 8, "w_3": 20, "w_4": 20, "w_hidden": 2, "w_i": [1, 2, 10], "w_ix_i": 12, "w_j": 20, "w_m": 20, "w_output": 2, "w_px_": 8, "w_px_p": 8, "wa": [1, 3, 4, 5, 6, 7, 10, 11, 12, 14, 17, 20, 25, 26], "wai": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 15, 18, 20, 22, 25, 26, 27], "walk": 9, "walker": 22, "wang": [0, 25], "want": [0, 1, 2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 19, 22, 25, 26, 27], "warn": 4, "warrant": 6, "wast": 3, "watch": [19, 27], "wave": 3, "wavelet": 8, "we": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 23, 24, 26, 27], "weak": [9, 10, 14], "weather": [1, 12], "web": [19, 21, 23, 25], "webpag": 25, "websit": [6, 20, 21, 25], "wedg": [8, 22], "wednesdai": [23, 25], "wee": 11, "week": [0, 5, 6, 7, 21, 23], "weekli": [15, 16, 19, 21, 23, 24, 25], "weight": [1, 2, 3, 6, 7, 9, 10, 12, 13, 18, 22], "weigth": 2, "welcom": [8, 15, 19], "well": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 15, 16, 19, 20, 22, 24, 25, 26, 27], "went": 8, "were": [0, 1, 3, 4, 5, 6, 7, 8, 10, 11, 12, 14, 22, 25], "wessel": [0, 25, 26, 27], "what": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 19, 20, 22], "whatev": 3, "when": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 22, 25, 26, 27], "whenev": [13, 15, 22], "where": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 23, 25, 26, 27], "wherea": [6, 22], "wherein": [1, 12], "whether": [0, 3, 5, 7, 9, 22, 25], "which": [0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 25, 26, 27], "whichev": [1, 3], "while": [0, 1, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 15, 16, 22, 25, 26, 27], "white": 9, "whiteboard": 26, "who": [0, 15], "whole": [1, 3, 4, 5, 9, 11, 13], "whose": [0, 6, 10, 22, 26], "whow": [11, 26], "why": [0, 1, 3, 6, 13, 15, 16, 17, 26, 27], "wide": [0, 1, 3, 6, 7, 12, 19, 20, 25], "widehat": 6, "width": [0, 3, 8, 9, 25], "wieringen": [0, 25, 26, 27], "win": 10, "wind": 9, "wing": [23, 25], "winther": 2, "wiothout": 6, "wiscons": 7, "wisconsin": 10, "wisdom": [6, 26], "wise": [1, 5, 12, 13, 26, 27], "wish": [0, 2, 5, 7, 8, 11, 13, 14, 18, 20, 25, 26, 27], "with_std": [0, 26], "wither": 6, "within": [0, 2, 3, 4, 7, 9, 12, 13, 14, 22, 24, 25, 27], "withinclust": 14, "without": [0, 1, 5, 6, 8, 9, 11, 12, 13, 15, 25, 26, 27], "won": [0, 15, 25], "wonder": 8, "word": [0, 1, 3, 4, 5, 6, 7, 14, 22, 25, 26, 27], "work": [0, 1, 4, 6, 7, 8, 9, 13, 15, 16, 18, 19, 21, 22, 23, 25, 26], "workshop": 25, "world": [0, 8, 16, 26], "worldwid": [0, 25], "worri": 15, "wors": [0, 1, 3, 4, 6, 25], "worth": 9, "would": [0, 1, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 16, 18, 20, 22, 25, 26, 27], "wrap": [6, 20, 25], "write": [0, 1, 2, 3, 5, 6, 7, 8, 12, 13, 15, 16, 20, 25, 26], "written": [0, 2, 3, 5, 11, 12, 13, 16, 19, 20, 22, 25, 26, 27], "wrong": [1, 8, 15], "wrongli": 10, "wrote": [5, 11, 26], "wrt": [10, 13], "wth": [10, 13], "www": [19, 20, 24, 25, 27], "wx_1": 8, "x": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 22, 25, 27], "x0": 8, "x1": [4, 8, 9, 10, 13], "x1_exampl": 8, "x1d": 8, "x2": [8, 9, 10, 13], "x2d": [8, 11], "x2d_train": 11, "x2dsl": 11, "x3": 8, "x_": [0, 2, 3, 5, 6, 8, 10, 11, 13, 14, 20, 22, 25, 26, 27], "x_0": [0, 5, 11, 18, 20, 25, 26], "x_1": [0, 2, 5, 6, 7, 8, 9, 10, 11, 13, 18, 20, 22, 25, 26, 27], "x_2": [0, 2, 5, 6, 7, 8, 9, 10, 11, 13, 20, 22, 25, 26, 27], "x_3": [8, 20, 22], "x_4": 20, "x_6": 18, "x_center": 11, "x_data": 1, "x_data_ful": 1, "x_hidden": 2, "x_i": [0, 1, 2, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 20, 22, 25, 26, 27], "x_input": 2, "x_ix_": [0, 25], "x_iy_i": 8, "x_j": [0, 2, 8, 9, 12, 16, 22, 26], "x_jy_j": 8, "x_k": [12, 14, 20, 22, 26], "x_l": 22, "x_m": [6, 12, 20, 22], "x_mean": 18, "x_n": [0, 2, 3, 6, 8, 11, 12, 13, 20, 22, 25, 27], "x_new": [9, 10], "x_norm": 18, "x_offset": [6, 26], "x_output": 2, "x_p": [3, 7, 9], "x_poli": 9, "x_poly10": 9, "x_pred": 4, "x_prev": 2, "x_reduc": 11, "x_scale": 8, "x_small": 13, "x_std": 18, "x_test": [0, 1, 3, 5, 6, 7, 9, 10, 11, 15, 16, 17, 26, 27], "x_test_": 17, "x_test_own": 6, "x_test_scal": [0, 6, 7, 9, 10, 11, 26], "x_tot": 4, "x_train": [0, 1, 3, 4, 5, 6, 7, 9, 10, 11, 15, 16, 17, 25, 26, 27], "x_train_": 17, "x_train_mean": [6, 26], "x_train_own": 6, "x_train_scal": [0, 6, 7, 9, 10, 11, 26], "x_val": 1, "xarrai": [19, 25], "xavier": 1, "xbnew": [13, 27], "xcode": [0, 19, 25], "xdclassiffierconfus": 10, "xdclassiffierroc": 10, "xg_clf": 10, "xgb": 10, "xgbclassifi": 10, "xgboost": 9, "xgboot": 10, "xgbregressor": 10, "xgparam": 10, "xgtree": 10, "xi": [8, 13], "xi_": 8, "xi_1": 8, "xi_i": 8, "xk": 8, "xla": [13, 19, 25], "xlabel": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 13, 22, 25, 26, 27], "xlim": [6, 10], "xm": 9, "xmesh": 13, "xnew": [0, 13, 25, 27], "xp": 22, "xpanda": [0, 26], "xpd": [5, 11, 26], "xplot": 0, "xscale": [0, 26], "xsr": 9, "xt_x": [13, 27], "xtest": 6, "xtick": [3, 6, 8, 9], "xtrain": 6, "xu": [0, 25], "xx": [0, 20, 25], "xy": [0, 6, 8, 20, 25], "xytext": 8, "xz": [20, 25], "y": [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 22, 25, 26, 27], "y1": 4, "y2": 4, "y3": 4, "y_": [0, 1, 5, 6, 10, 11, 20, 25, 26], "y_0": [0, 5, 11, 20, 25, 26], "y_1": [0, 5, 8, 9, 11, 13, 20, 25, 26, 27], "y_1y_1": 8, "y_1y_1k": 8, "y_1y_2": 8, "y_1y_2k": 8, "y_1y_n": 8, "y_1y_nk": 8, "y_2": [0, 5, 8, 9, 11, 20, 25, 26], "y_2y_1": 8, "y_2y_1k": 8, "y_2y_2": 8, "y_2y_2k": 8, "y_3": [0, 9, 20], "y_4": 20, "y_center": 18, "y_data": [0, 1, 5, 6, 25, 26, 27], "y_data_ful": 1, "y_decis": 8, "y_fit": [0, 26], "y_i": [0, 1, 5, 6, 7, 8, 9, 10, 11, 12, 13, 20, 25, 26, 27], "y_if_": 10, "y_ix_": [0, 25], "y_ix_i": [7, 8, 13, 26, 27], "y_iy_jk": 8, "y_j": [6, 8, 12], "y_k": 12, "y_m": 20, "y_mean": 18, "y_model": [0, 4, 5, 6, 25, 26, 27], "y_n": [8, 13, 27], "y_ny_1": 8, "y_ny_1k": 8, "y_ny_2": 8, "y_ny_2k": 8, "y_ny_n": 8, "y_ny_nk": 8, "y_offset": [6, 17, 26], "y_plot": 9, "y_pred": [0, 1, 4, 6, 7, 8, 9, 10, 26], "y_pred1": 9, "y_pred2": 9, "y_pred_rf": 10, "y_pred_tre": 10, "y_proba": [7, 10], "y_scaler": [6, 26], "y_test": [0, 1, 3, 4, 5, 6, 7, 9, 10, 11, 15, 16, 17, 26, 27], "y_test_onehot": 1, "y_test_predict": [], "y_tot": 4, "y_train": [0, 1, 3, 4, 5, 6, 7, 9, 10, 11, 15, 16, 17, 25, 26, 27], "y_train_mean": [6, 26], "y_train_onehot": 1, "y_train_predict": [], "y_train_scal": [6, 26], "y_val": 1, "ye": [3, 6, 7], "year": [0, 19, 25], "yet": [0, 1, 6, 8, 11, 13, 25], "yi": 13, "yield": [0, 2, 5, 6, 8, 10, 12, 13, 14, 20, 22, 25, 27], "yk": 8, "ylabel": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 13, 22, 25, 26, 27], "ylim": [3, 6], "ym": 9, "ymesh": 13, "yn": 0, "yo": [8, 9, 10], "yoshua": [1, 24], "you": [0, 1, 3, 4, 5, 6, 8, 9, 10, 11, 13, 15, 16, 17, 18, 19, 20, 22, 23, 24, 25, 26, 27], "young": 0, "your": [1, 2, 4, 5, 6, 8, 11, 13, 15, 17, 19, 20, 25, 27], "your_model_object": 16, "yourself": [11, 13, 25, 27], "youtu": 26, "youtub": [19, 27], "ypred": 6, "ypredict": [0, 13, 25, 26, 27], "ypredict2": [13, 27], "ypredictlasso": [5, 27], "ypredictol": [0, 5, 27], "ypredictown": [6, 26], "ypredictownridg": [6, 26, 27], "ypredictridg": [0, 5, 6, 26, 27], "ypredictskl": [6, 26], "ytest": 6, "ytick": [3, 6, 8, 9], "ytild": [0, 6, 25, 26], "ytildelasso": [5, 27], "ytildenp": [0, 25, 26], "ytildeol": [0, 5, 27], "ytildeownridg": [6, 26, 27], "ytilderidg": [5, 6, 26, 27], "ytrain": 6, "yuxi": 25, "yx": [20, 25], "yy": [20, 25], "yz": [20, 25], "z": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 20, 22, 25, 26], "z_": [1, 2, 12, 20, 25], "z_0": [20, 25], "z_1": [20, 25], "z_2": [20, 25], "z_c": 1, "z_h": 1, "z_hidden": 2, "z_i": [1, 12], "z_j": [1, 12], "z_k": [12, 26], "z_m": 1, "z_mod": 9, "z_o": 1, "z_output": 2, "zaman": 22, "zaxi": 6, "zero": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 18, 20, 22, 25, 26, 27], "zeros_lik": 4, "zeroth": 26, "zfill": 4, "zip": [4, 6], "zm_h": [0, 25], "zn": [], "zone": [], "zoom": 25, "zx": [20, 25], "zy": [20, 25], "zz": [20, 25], "\u00f8yvind": [6, 26]}, "titles": ["3. Linear Regression", "14. Building a Feed Forward Neural Network", "15. Solving Differential Equations with Deep Learning", "16. Convolutional Neural Networks", "17. Recurrent neural networks: Overarching view", "4. Ridge and Lasso Regression", "5. Resampling Methods", "6. Logistic Regression", "8. Support Vector Machines, overarching aims", "9. Decision trees, overarching aims", "10. Ensemble Methods: From a Single Tree to Many Trees and Extreme Boosting, Meet the Jungle of Methods", "11. Basic ideas of the Principal Component Analysis (PCA)", "13. Neural networks", "7. Optimization, the central part of any Machine Learning algortithm", "12. Clustering and Unsupervised Learning", "Exercises week 34", "Exercises week 35", "Exercises week 36", "Exercises week 36", "Applied Data Analysis and Machine Learning", "2. Linear Algebra, Handling of Arrays and more Python Features", "Course setting", "1. Elements of Probability Theory and Statistical Data Analysis", "Teachers and Grading", "Textbooks", "Week 34: Introduction to the course, Logistics and Practicalities", "Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression", "Week 36: Linear Regression and Gradient descent"], "titleterms": {"": [8, 10, 27], "1": [0, 15, 16, 17, 18, 26], "1a": 18, "2": [0, 15, 16, 17, 18, 25, 26, 27], "2023": 23, "2a": 18, "2b": 18, "3": [0, 15, 16, 17, 18, 26], "34": [15, 25], "35": [16, 26], "36": [17, 18, 27], "3a": 18, "3b": 18, "4": [0, 15, 16, 17, 26], "5": [0, 16], "A": [0, 1, 4, 8, 9, 25], "And": [25, 26], "In": 23, "Ising": 6, "The": [0, 1, 2, 3, 5, 6, 7, 8, 9, 11, 12, 15, 19, 25, 26, 27], "To": 25, "With": [4, 27], "about": [25, 26, 27], "abov": 27, "activ": [1, 12], "ad": [0, 6, 25, 26], "adaboost": 10, "adagrad": 13, "adam": 13, "adapt": 10, "adjust": 1, "adversari": 4, "again": [3, 9], "ai": 25, "aim": [8, 9, 25], "aka": 25, "algebra": [20, 25], "algorithm": [9, 10, 11, 12, 25, 26, 27], "algortithm": [13, 27], "all": 8, "an": [0, 4, 10, 15, 25], "analys": [5, 26, 27], "analysi": [0, 5, 6, 11, 19, 22, 25, 26, 27], "analyt": [0, 16, 18], "ani": [13, 27], "anoth": [9, 27], "appli": 19, "approach": [0, 8, 14, 25], "approxim": 12, "architectur": 1, "arrai": [20, 25], "assist": 23, "autocorrel": 22, "autograd": [2, 13], "automat": 13, "back": [1, 11, 12, 26, 27], "background": 19, "bag": 10, "base": 13, "basic": [0, 5, 7, 9, 10, 11, 20, 26, 27], "batch": 1, "bay": 5, "befor": 11, "better": 8, "bia": 6, "binari": 1, "bind": 25, "bird": 10, "boldsymbol": [18, 26], "boost": 10, "bootstrap": [6, 10], "boston": [], "breast": 1, "brief": 25, "bring": 12, "build": [1, 3, 9], "c": 25, "calcul": [26, 27], "can": 25, "cancer": [1, 7, 9, 11], "cart": 9, "case": [8, 10, 22, 26, 27], "central": [13, 19, 22, 27], "chain": 12, "chang": 10, "channel": 25, "chi": [0, 25], "choic": 17, "choos": 1, "cifar01": 3, "classic": 11, "classif": [1, 9, 10], "classifi": 8, "clip": 1, "cluster": 14, "cnn": 3, "code": [1, 2, 5, 9, 11, 12, 13, 14, 15, 16, 25, 26, 27], "collect": [1, 3], "commun": 25, "compar": [2, 10, 16], "comparison": 27, "complet": 26, "complex": [0, 6, 26], "complic": 6, "compon": 11, "comput": 9, "computerlab": 25, "con": 9, "concept": 22, "condit": 27, "conjug": 13, "contn": 25, "convex": [8, 13, 27], "convolut": [3, 12], "correl": [11, 26], "correspond": [], "cost": [1, 10, 26, 27], "cours": [19, 21, 24, 25], "covari": [5, 11, 22, 26], "cover": 25, "creat": 16, "cross": 6, "cython": 25, "data": [0, 1, 3, 6, 7, 9, 11, 15, 17, 18, 19, 22, 25, 26], "dataset": [1, 3, 18], "david": 25, "deadlin": 25, "deadllin": 23, "decai": 2, "decis": [9, 10], "decomposit": [5, 11, 20, 26, 27], "deeep": [], "deep": [1, 2, 25], "defin": [1, 25], "degre": [0, 17, 26], "deliver": [15, 16], "dens": 0, "deriv": [5, 12, 16, 17, 26, 27], "descent": [2, 10, 13, 18, 27], "design": 26, "detail": [3, 25], "develop": 1, "diagon": 11, "differ": 8, "differenti": [2, 13], "diffus": 2, "dimension": [2, 3, 8], "disadvantag": 9, "discret": 22, "discrimin": 25, "distribut": [5, 22], "do": 1, "doe": [26, 27], "domain": 22, "down": 1, "dropout": 1, "economi": [26, 27], "element": [0, 22, 25], "elimin": 20, "energi": 25, "ensembl": 10, "entropi": 9, "environ": [0, 15], "equat": [0, 2, 12, 26, 27], "error": [0, 10, 25, 26, 27], "essenti": 25, "etc": 25, "euler": 2, "evalu": 1, "exampl": [1, 2, 3, 4, 6, 7, 8, 9, 10, 25, 26, 27], "exercis": [0, 6, 15, 16, 17, 18, 26], "expect": 22, "experi": 22, "explor": 0, "exponenti": 2, "express": [16, 17, 26], "extend": 27, "extrapol": 4, "extrem": [10, 25], "ey": 10, "fall": 23, "famili": [1, 25], "famou": 20, "fantast": [26, 27], "featur": [9, 16, 20, 26], "feed": [1, 12], "final": [12, 26], "find": [16, 18], "fine": 1, "first": [4, 12, 25, 27], "fit": [0, 10, 15, 16, 25, 27], "fix": [26, 27], "forc": 3, "forest": 10, "form": 18, "format": 25, "formula": 18, "forward": [1, 2, 12], "foster": 25, "fourier": 3, "frank": 6, "freedom": [0, 17, 26], "frequent": 26, "frequentist": [0, 25], "from": [5, 10, 12, 25, 26, 27], "full": 2, "function": [0, 1, 6, 7, 8, 10, 11, 12, 13, 22, 25, 26, 27], "further": [3, 5, 26, 27], "gan": 4, "gaussian": 20, "gd": 13, "gener": [4, 9, 25], "geometr": [11, 27], "gini": 9, "github": 15, "goal": [15, 16, 17, 18], "good": [0, 25], "grade": [23, 25], "gradient": [1, 2, 10, 13, 18, 27], "growth": 2, "ha": 19, "handl": [20, 25], "hessian": [26, 27], "hidden": 2, "hous": [], "how": 16, "hyperparamet": [1, 17], "hyperplan": 8, "i": [0, 1, 25], "id3": 9, "idea": 11, "ideal": 27, "ii": 25, "illustr": 27, "implement": [1, 16, 17, 18], "implic": [5, 26, 27], "import": [5, 20, 25, 26, 27], "improv": 1, "includ": 13, "increment": 11, "index": 9, "inform": 23, "input": 2, "instal": [19, 25], "instructor": 23, "interpret": [5, 11, 25, 26, 27], "introduc": [11, 13, 26], "introduct": [0, 6, 19, 20, 25], "invers": [5, 20], "invert": [26, 27], "iter": 10, "its": 26, "jacobian": 26, "jax": 13, "julia": 25, "jungl": 10, "kera": [1, 3], "kernel": [8, 11], "lab": 27, "lagrangian": 8, "lasso": [5, 6, 26, 27], "last": 26, "later": [5, 26, 27], "layer": [1, 2, 3, 12], "learn": [0, 1, 2, 11, 13, 14, 15, 16, 17, 18, 19, 25, 26, 27], "least": [5, 6, 16, 25, 26, 27], "lectur": [25, 27], "level": 10, "librari": [19, 25], "likelihood": 7, "limit": [1, 13, 22, 27], "linear": [0, 8, 13, 15, 20, 25, 26, 27], "link": [5, 11, 24, 26], "logist": [7, 25], "loss": [26, 27], "lu": 20, "machin": [0, 8, 13, 19, 25, 27], "main": [22, 25], "make": [0, 9, 10, 26], "mani": [10, 12], "mass": 25, "materi": [25, 26, 27], "math": [5, 26, 27], "mathemat": [3, 5, 8, 26, 27], "matric": [5, 20, 25], "matrix": [1, 5, 11, 12, 16, 20, 25, 26, 27], "matter": 0, "max": 26, "mean": [0, 26, 27], "meet": [5, 10, 22, 25, 26], "mercer": 8, "method": [6, 9, 10, 13, 25, 27], "min": 26, "minim": 25, "ml": 25, "mlp": 12, "mnist": [3, 4], "model": [0, 1, 4, 6, 12, 15, 17, 25], "momentum": 13, "mondai": 27, "moon": [8, 9], "more": [3, 6, 20, 25, 26, 27], "multilay": 12, "multipl": [1, 3, 17], "multipli": 8, "need": 25, "network": [1, 2, 3, 4, 7, 12, 25], "neural": [1, 2, 3, 4, 7, 12, 25], "new": [4, 18], "newton": 27, "non": 8, "normal": [0, 1], "notat": 12, "note": [26, 27], "now": [1, 9, 13, 27], "nuclear": [0, 25], "numba": 25, "number": [0, 2, 22, 26], "numer": [2, 22], "numpi": [20, 25], "object": 3, "obtain": 11, "od": 2, "off": 6, "ol": [5, 6, 15, 16, 18, 27], "one": [2, 12, 27], "oper": 20, "optim": [1, 8, 13, 18, 19, 25, 26, 27], "order": 13, "ordinari": [5, 6, 16, 25, 26, 27], "organ": [0, 25], "oslo": 24, "other": [4, 9, 11, 12, 20, 25], "our": [0, 4, 5, 11, 13, 25, 26, 27], "outcom": [19, 25], "output": 2, "overarch": [0, 4, 8, 9, 25, 26], "overview": [10, 25], "own": [0, 10, 11, 25, 26], "packag": [20, 25], "panda": [25, 26], "paramet": [25, 26], "paramt": 18, "part": [13, 19, 27], "partial": 2, "pass": 1, "pca": 11, "pdf": 22, "perceptron": 12, "perform": [1, 9], "period": 3, "perspect": 1, "plan": [26, 27], "plethora": 25, "point": 4, "poisson": 2, "polynomi": [3, 16, 27], "popul": 2, "popular": 25, "practic": [13, 23, 25], "pre": [1, 3], "predict": 4, "preprocess": 26, "prerequisit": [3, 19, 25], "princip": 11, "principl": 3, "pro": 9, "probabl": [5, 22], "problem": [1, 2, 13, 25, 26, 27], "procedur": [9, 25], "process": [1, 3], "program": [2, 13, 27], "project": [6, 23, 25], "prop": 13, "propag": [1, 12], "properti": [5, 22, 26, 27], "python": [0, 9, 15, 19, 20, 25], "quick": 8, "r": 25, "random": [10, 11, 22], "raphson": 27, "read": [9, 25, 26], "real": [6, 25], "recommend": [25, 26], "recurr": [4, 12], "reduc": [0, 26], "reduct": 3, "reformul": 2, "regress": [0, 5, 6, 7, 9, 10, 13, 15, 17, 18, 25, 26, 27], "regular": 1, "relat": [], "relev": [24, 26], "relu": 1, "remark": 3, "remind": [6, 8, 25, 26, 27], "replac": 13, "repositori": 15, "requir": [2, 19], "resampl": 6, "rescal": [6, 26], "residu": [26, 27], "resourc": 2, "result": [26, 27], "revisit": [13, 27], "rewrit": [25, 26], "ridg": [0, 5, 6, 17, 18, 26, 27], "rm": 13, "rule": 12, "same": 13, "sampl": 11, "scale": [17, 18, 26], "schedul": 25, "schemat": 9, "scheme": 2, "scienc": 25, "scikit": [0, 1, 11, 25, 26, 27], "second": 13, "semest": 23, "sensit": 27, "septemb": 27, "session": 27, "set": [0, 2, 3, 9, 12, 15, 21, 25, 26, 27], "setup": 15, "sgd": 13, "should": 1, "similar": 13, "simpl": [0, 4, 9, 13, 25, 26, 27], "simplest": 18, "singl": 10, "singular": [5, 11, 26, 27], "size": [26, 27], "sklearn": 16, "soft": 8, "softmax": 1, "softwar": 25, "solv": [2, 27], "solver": 13, "some": [13, 20, 26, 27], "specifi": 2, "split": [0, 15, 26], "squar": [0, 5, 6, 10, 16, 25, 26, 27], "standard": [13, 26], "state": 0, "statist": [5, 6, 19, 22, 25], "steepest": [10, 13, 27], "stochast": [13, 22], "strongli": 25, "suggest": 25, "summari": [23, 25], "superposit": 3, "supervis": 1, "support": 8, "svd": [5, 26, 27], "synthet": 18, "systemat": 3, "t": 26, "take": 16, "taken": 25, "teach": 23, "teacher": [23, 25], "technic": 26, "techniqu": [6, 11], "technologi": 19, "tensorflow": [1, 3], "tent": [23, 25], "test": [0, 1, 15, 17, 26], "text": 25, "textbook": [24, 25], "than": 27, "theorem": [5, 8, 11, 12, 22], "theori": 22, "theta": 18, "thi": 25, "tip": 13, "togeth": 12, "tool": 25, "top": 1, "topic": 25, "toward": 11, "trade": 6, "tradeoff": 6, "train": [0, 1, 4, 15, 25, 26], "transform": 3, "tree": [9, 10], "tuesdai": 27, "tune": 1, "two": [3, 8, 19], "type": [2, 4, 12, 25], "uio": 25, "univers": [12, 24], "unsupervis": 14, "up": [0, 2, 9, 12, 15, 25, 26, 27], "us": [0, 1, 2, 3, 7, 13, 16, 18, 19, 25, 26, 27], "v": 3, "valid": 6, "valu": [5, 11, 22, 26, 27], "variabl": [22, 27], "varianc": 6, "variou": 0, "vector": [8, 12, 16, 20, 25, 26], "versu": 25, "view": [0, 4, 10, 26], "virtual": 15, "visual": [1, 9], "wai": 9, "wave": 2, "we": 25, "wednesdai": 27, "week": [15, 16, 17, 18, 25, 26, 27], "weekli": [], "what": [0, 25, 26, 27], "which": 1, "why": 25, "wisconsin": 7, "write": [4, 11, 27], "x": 26, "xgboost": 10, "yet": 27, "your": [0, 10, 16, 18, 26]}}) \ No newline at end of file diff --git a/doc/LectureNotes/_build/jupyter_execute/exercisesweek37.ipynb b/doc/LectureNotes/_build/jupyter_execute/exercisesweek37.ipynb index b63ddcc09..373a3f02a 100644 --- a/doc/LectureNotes/_build/jupyter_execute/exercisesweek37.ipynb +++ b/doc/LectureNotes/_build/jupyter_execute/exercisesweek37.ipynb @@ -2,7 +2,7 @@ "cells": [ { "cell_type": "markdown", - "id": "7d56b2d5", + "id": "d3aa801d", "metadata": { "editable": true }, @@ -14,7 +14,7 @@ }, { "cell_type": "markdown", - "id": "c7a8e9c7", + "id": "7c64e6da", "metadata": { "editable": true }, @@ -27,7 +27,7 @@ }, { "cell_type": "markdown", - "id": "cf8f0ecb", + "id": "51e35698", "metadata": { "editable": true }, @@ -46,7 +46,7 @@ }, { "cell_type": "markdown", - "id": "a67ae548", + "id": "74fb184e", "metadata": { "editable": true }, @@ -72,7 +72,7 @@ { "cell_type": "code", "execution_count": 1, - "id": "f2d4a55d", + "id": "9e6acfef", "metadata": { "collapsed": false, "editable": true @@ -101,33 +101,43 @@ }, { "cell_type": "markdown", - "id": "a445583b", + "id": "f2d03ca8", "metadata": { "editable": true }, "source": [ "This code produces a dataset where only features 0, 1, and 6\n", "significantly influence $\\boldsymbol{y}$. The rest of the features have zero true\n", - "coefficient, so they only contribute noise. For example, feature 0 has\n", + "coefficient. For example, feature 0 has\n", "a true weight of 5.0, feature 1 has -3.0, and feature 6 has 2.0, so\n", "the expected relationship is:" ] }, { "cell_type": "markdown", - "id": "4a81ddf9", + "id": "d2d64f9b", "metadata": { "editable": true }, "source": [ "$$\n", - "y \\approx 5 \\times X_0 \\;-\\; 3 \\times X_1 \\;+\\; 2 \\times X_6 \\;+\\; \\text{noise}.\n", + "y \\approx 5 \\times x_0 \\;-\\; 3 \\times x_1 \\;+\\; 2 \\times x_6 \\;+\\; \\text{noise}.\n", "$$" ] }, { "cell_type": "markdown", - "id": "ae590275", + "id": "b4248e9d", + "metadata": { + "editable": true + }, + "source": [ + "You can remove the noise if you wish to." + ] + }, + { + "cell_type": "markdown", + "id": "5fed181f", "metadata": { "editable": true }, @@ -138,14 +148,24 @@ "standardize the features. This ensures all features are on a\n", "comparable scale, which is especially important when using\n", "regularization. Here we will perform standardization, scaling each\n", - "feature to have mean 0 and standard deviation 1:\n", + "feature to have mean 0 and standard deviation 1." + ] + }, + { + "cell_type": "markdown", + "id": "6ec0227c", + "metadata": { + "editable": true + }, + "source": [ + "### 1a)\n", "\n", - "Compute the mean and standard deviation of each column (feature) in $bm{X}$.\n", + "Compute the mean and standard deviation of each column (feature) in $\\boldsymbol{X}$.\n", "Subtract the mean and divide by the standard deviation for each feature.\n", "\n", "We will also center the target $\\boldsymbol{y}$ to mean $0$. Centering $\\boldsymbol{y}$\n", - "(and each feature) means the model won’t require a separate intercept\n", - "term – the data is shifted such that the intercept is effectively 0\n", + "(and each feature) means the model does not require a separate intercept\n", + "term, the data is shifted such that the intercept is effectively 0\n", ". (In practice, one could include an intercept in the model and not\n", "penalize it, but here we simplify by centering.)" ] @@ -153,7 +173,7 @@ { "cell_type": "code", "execution_count": 2, - "id": "8b40c47a", + "id": "a140aac7", "metadata": { "collapsed": false, "editable": true @@ -173,36 +193,34 @@ }, { "cell_type": "markdown", - "id": "ff9c0c81", + "id": "57ad18f5", "metadata": { "editable": true }, "source": [ - "### 1a)\n", - "\n", "Fill in the necessary details.\n", "\n", - "After this preprocessing, each column of $\\boldsymbol{X}_norm$ has mean zero and standard deviation $1$\n", - "and $\\boldsymbol{y}_centered$ has mean 0. This makes the optimization landscape\n", + "After this preprocessing, each column of $\\boldsymbol{X}_{\\mathrm{norm}}$ has mean zero and standard deviation $1$\n", + "and $\\boldsymbol{y}_{\\mathrm{centered}}$ has mean 0. This makes the optimization landscape\n", "nicer and ensures the regularization penalty $\\lambda \\sum_j\n", - "\\beta_j^2$ treats each coefficient fairly (since features are on the\n", + "\\theta_j^2$ in Ridge regression treats each coefficient fairly (since features are on the\n", "same scale)." ] }, { "cell_type": "markdown", - "id": "d27c70e4", + "id": "2886697d", "metadata": { "editable": true }, "source": [ - "## Exercise 2, use the analytical formulae for OLS and Ridge regression to find the optimal paramters $\\boldsymbol{theta}$" + "## Exercise 2, use the analytical formulae for OLS and Ridge regression to find the optimal paramters $\\boldsymbol{\\theta}$" ] }, { "cell_type": "code", "execution_count": 3, - "id": "9f1e5184", + "id": "97ac6cb6", "metadata": { "collapsed": false, "editable": true @@ -223,34 +241,33 @@ }, { "cell_type": "markdown", - "id": "2ec556b9", + "id": "3efb067b", "metadata": { "editable": true }, "source": [ - "This computes the ridge and OLS regression coefficients directly. The identity\n", - "matrix $I$ has the same size as $X^T X$ (which is n_features x\n", - "n_features), and lam * I adds $\\lambda$ to the diagonal of $X^T X. We\n", - "then invert this matrix and multiply by $X^T y. The result\n", - "for $\\boldsymbol{\\theta}$ is a NumPy array of shape (n_features,) containing the\n", - "fitted weights." + "This computes the Ridge and OLS regression coefficients directly. The identity\n", + "matrix $I$ has the same size as $X^T X$. It adds $\\lambda$ to the diagonal of $X^T X for Ridge regression. We\n", + "then invert this matrix and multiply by $X^T y$. The result\n", + "for $\\boldsymbol{\\theta}$ is a NumPy array of shape (n$\\_$features,) containing the\n", + "fitted parameters $\\boldsymbol{\\theta}$.." ] }, { "cell_type": "markdown", - "id": "a821f0c5", + "id": "53be2bf8", "metadata": { "editable": true }, "source": [ "### 2a)\n", "\n", - "Finalize the OLS and Ridge regression determination of the optimal parameters $bm{\\theta}$." + "Finalize, in the above code, the OLS and Ridge regression determination of the optimal parameters $\\boldsymbol{\\theta}$." ] }, { "cell_type": "markdown", - "id": "d637130e", + "id": "e4126591", "metadata": { "editable": true }, @@ -262,12 +279,12 @@ }, { "cell_type": "markdown", - "id": "b455ce7e", + "id": "642d0850", "metadata": { "editable": true }, "source": [ - "## Implementing the simplest form for gradient descent\n", + "## Exercise 3, Implementing the simplest form for gradient descent\n", "\n", "Alternatively, we can fit the ridge regression model using gradient\n", "descent. This is useful to visualize the iterative convergence and is\n", @@ -282,7 +299,7 @@ { "cell_type": "code", "execution_count": 4, - "id": "cfa1eb29", + "id": "a67af634", "metadata": { "collapsed": false, "editable": true @@ -325,32 +342,32 @@ }, { "cell_type": "markdown", - "id": "dc78d58d", + "id": "1c8c35dc", "metadata": { "editable": true }, "source": [ "### 3a)\n", "\n", - "Discuss the results as function of the learning rate paramaters and the number of iterations." + "Discuss the results as function of the learning rate parameters and the number of iterations." ] }, { "cell_type": "markdown", - "id": "15060acb", + "id": "899fec5c", "metadata": { "editable": true }, "source": [ "### 3b)\n", "\n", - "Add a stopping parameter as function of the number iterations. \n", + "Try to add a stopping parameter as function of the number iterations. How would you define a stopping criterion? \n", "\n", "If everything worked correctly, the learned coefficients should be\n", "close to the true values [5.0, -3.0, 0.0, …, 2.0, …] that we used to\n", "generate the data. Keep in mind that due to regularization and noise,\n", "the learned values will not exactly equal the true ones, but they\n", - "should be in the same ballpark." + "should be in the same ballpark. Which method (OLS or Ridge) gives the best results?" ] } ], diff --git a/doc/LectureNotes/exercisesweek37.ipynb b/doc/LectureNotes/exercisesweek37.ipynb index 68dd47235..56fe53e19 100644 --- a/doc/LectureNotes/exercisesweek37.ipynb +++ b/doc/LectureNotes/exercisesweek37.ipynb @@ -2,7 +2,7 @@ "cells": [ { "cell_type": "markdown", - "id": "7d56b2d5", + "id": "d3aa801d", "metadata": { "editable": true }, @@ -14,7 +14,7 @@ }, { "cell_type": "markdown", - "id": "c7a8e9c7", + "id": "7c64e6da", "metadata": { "editable": true }, @@ -27,7 +27,7 @@ }, { "cell_type": "markdown", - "id": "cf8f0ecb", + "id": "51e35698", "metadata": { "editable": true }, @@ -46,7 +46,7 @@ }, { "cell_type": "markdown", - "id": "a67ae548", + "id": "74fb184e", "metadata": { "editable": true }, @@ -72,7 +72,7 @@ { "cell_type": "code", "execution_count": 1, - "id": "f2d4a55d", + "id": "9e6acfef", "metadata": { "collapsed": false, "editable": true @@ -101,33 +101,43 @@ }, { "cell_type": "markdown", - "id": "a445583b", + "id": "f2d03ca8", "metadata": { "editable": true }, "source": [ "This code produces a dataset where only features 0, 1, and 6\n", "significantly influence $\\boldsymbol{y}$. The rest of the features have zero true\n", - "coefficient, so they only contribute noise. For example, feature 0 has\n", + "coefficient. For example, feature 0 has\n", "a true weight of 5.0, feature 1 has -3.0, and feature 6 has 2.0, so\n", "the expected relationship is:" ] }, { "cell_type": "markdown", - "id": "4a81ddf9", + "id": "d2d64f9b", "metadata": { "editable": true }, "source": [ "$$\n", - "y \\approx 5 \\times X_0 \\;-\\; 3 \\times X_1 \\;+\\; 2 \\times X_6 \\;+\\; \\text{noise}.\n", + "y \\approx 5 \\times x_0 \\;-\\; 3 \\times x_1 \\;+\\; 2 \\times x_6 \\;+\\; \\text{noise}.\n", "$$" ] }, { "cell_type": "markdown", - "id": "ae590275", + "id": "b4248e9d", + "metadata": { + "editable": true + }, + "source": [ + "You can remove the noise if you wish to." + ] + }, + { + "cell_type": "markdown", + "id": "5fed181f", "metadata": { "editable": true }, @@ -138,14 +148,24 @@ "standardize the features. This ensures all features are on a\n", "comparable scale, which is especially important when using\n", "regularization. Here we will perform standardization, scaling each\n", - "feature to have mean 0 and standard deviation 1:\n", + "feature to have mean 0 and standard deviation 1." + ] + }, + { + "cell_type": "markdown", + "id": "6ec0227c", + "metadata": { + "editable": true + }, + "source": [ + "### 1a)\n", "\n", - "Compute the mean and standard deviation of each column (feature) in $bm{X}$.\n", + "Compute the mean and standard deviation of each column (feature) in $\\boldsymbol{X}$.\n", "Subtract the mean and divide by the standard deviation for each feature.\n", "\n", "We will also center the target $\\boldsymbol{y}$ to mean $0$. Centering $\\boldsymbol{y}$\n", - "(and each feature) means the model won’t require a separate intercept\n", - "term – the data is shifted such that the intercept is effectively 0\n", + "(and each feature) means the model does not require a separate intercept\n", + "term, the data is shifted such that the intercept is effectively 0\n", ". (In practice, one could include an intercept in the model and not\n", "penalize it, but here we simplify by centering.)" ] @@ -153,7 +173,7 @@ { "cell_type": "code", "execution_count": 2, - "id": "8b40c47a", + "id": "a140aac7", "metadata": { "collapsed": false, "editable": true @@ -173,36 +193,34 @@ }, { "cell_type": "markdown", - "id": "ff9c0c81", + "id": "57ad18f5", "metadata": { "editable": true }, "source": [ - "### 1a)\n", - "\n", "Fill in the necessary details.\n", "\n", - "After this preprocessing, each column of $\\boldsymbol{X}_norm$ has mean zero and standard deviation $1$\n", - "and $\\boldsymbol{y}_centered$ has mean 0. This makes the optimization landscape\n", + "After this preprocessing, each column of $\\boldsymbol{X}_{\\mathrm{norm}}$ has mean zero and standard deviation $1$\n", + "and $\\boldsymbol{y}_{\\mathrm{centered}}$ has mean 0. This makes the optimization landscape\n", "nicer and ensures the regularization penalty $\\lambda \\sum_j\n", - "\\beta_j^2$ treats each coefficient fairly (since features are on the\n", + "\\theta_j^2$ in Ridge regression treats each coefficient fairly (since features are on the\n", "same scale)." ] }, { "cell_type": "markdown", - "id": "d27c70e4", + "id": "2886697d", "metadata": { "editable": true }, "source": [ - "## Exercise 2, use the analytical formulae for OLS and Ridge regression to find the optimal paramters $\\boldsymbol{theta}$" + "## Exercise 2, use the analytical formulae for OLS and Ridge regression to find the optimal paramters $\\boldsymbol{\\theta}$" ] }, { "cell_type": "code", "execution_count": 3, - "id": "9f1e5184", + "id": "97ac6cb6", "metadata": { "collapsed": false, "editable": true @@ -223,34 +241,33 @@ }, { "cell_type": "markdown", - "id": "2ec556b9", + "id": "3efb067b", "metadata": { "editable": true }, "source": [ - "This computes the ridge and OLS regression coefficients directly. The identity\n", - "matrix $I$ has the same size as $X^T X$ (which is n_features x\n", - "n_features), and lam * I adds $\\lambda$ to the diagonal of $X^T X. We\n", - "then invert this matrix and multiply by $X^T y. The result\n", - "for $\\boldsymbol{\\theta}$ is a NumPy array of shape (n_features,) containing the\n", - "fitted weights." + "This computes the Ridge and OLS regression coefficients directly. The identity\n", + "matrix $I$ has the same size as $X^T X$. It adds $\\lambda$ to the diagonal of $X^T X for Ridge regression. We\n", + "then invert this matrix and multiply by $X^T y$. The result\n", + "for $\\boldsymbol{\\theta}$ is a NumPy array of shape (n$\\_$features,) containing the\n", + "fitted parameters $\\boldsymbol{\\theta}$.." ] }, { "cell_type": "markdown", - "id": "a821f0c5", + "id": "53be2bf8", "metadata": { "editable": true }, "source": [ "### 2a)\n", "\n", - "Finalize the OLS and Ridge regression determination of the optimal parameters $bm{\\theta}$." + "Finalize, in the above code, the OLS and Ridge regression determination of the optimal parameters $\\boldsymbol{\\theta}$." ] }, { "cell_type": "markdown", - "id": "d637130e", + "id": "e4126591", "metadata": { "editable": true }, @@ -262,12 +279,12 @@ }, { "cell_type": "markdown", - "id": "b455ce7e", + "id": "642d0850", "metadata": { "editable": true }, "source": [ - "## Implementing the simplest form for gradient descent\n", + "## Exercise 3, Implementing the simplest form for gradient descent\n", "\n", "Alternatively, we can fit the ridge regression model using gradient\n", "descent. This is useful to visualize the iterative convergence and is\n", @@ -282,7 +299,7 @@ { "cell_type": "code", "execution_count": 4, - "id": "cfa1eb29", + "id": "a67af634", "metadata": { "collapsed": false, "editable": true @@ -325,32 +342,32 @@ }, { "cell_type": "markdown", - "id": "dc78d58d", + "id": "1c8c35dc", "metadata": { "editable": true }, "source": [ "### 3a)\n", "\n", - "Discuss the results as function of the learning rate paramaters and the number of iterations." + "Discuss the results as function of the learning rate parameters and the number of iterations." ] }, { "cell_type": "markdown", - "id": "15060acb", + "id": "899fec5c", "metadata": { "editable": true }, "source": [ "### 3b)\n", "\n", - "Add a stopping parameter as function of the number iterations. \n", + "Try to add a stopping parameter as function of the number iterations. How would you define a stopping criterion? \n", "\n", "If everything worked correctly, the learned coefficients should be\n", "close to the true values [5.0, -3.0, 0.0, …, 2.0, …] that we used to\n", "generate the data. Keep in mind that due to regularization and noise,\n", "the learned values will not exactly equal the true ones, but they\n", - "should be in the same ballpark." + "should be in the same ballpark. Which method (OLS or Ridge) gives the best results?" ] } ], diff --git a/doc/src/week37/exercisesweek37.do.txt b/doc/src/week37/exercisesweek37.do.txt index 0a19d1538..d03f4b781 100644 --- a/doc/src/week37/exercisesweek37.do.txt +++ b/doc/src/week37/exercisesweek37.do.txt @@ -52,31 +52,33 @@ y = X.dot @ theta_true + noise This code produces a dataset where only features 0, 1, and 6 significantly influence $\bm{y}$. The rest of the features have zero true -coefficient, so they only contribute noise. For example, feature 0 has +coefficient. For example, feature 0 has a true weight of 5.0, feature 1 has -3.0, and feature 6 has 2.0, so the expected relationship is: !bt \[ -y \approx 5 \times X_0 \;-\; 3 \times X_1 \;+\; 2 \times X_6 \;+\; \text{noise}. +y \approx 5 \times x_0 \;-\; 3 \times x_1 \;+\; 2 \times x_6 \;+\; \text{noise}. \] !et - +You can remove the noise if you wish to. ===== Exercise 1, scale your data ===== Before fitting a regression model, it is good practice to normalize or standardize the features. This ensures all features are on a comparable scale, which is especially important when using regularization. Here we will perform standardization, scaling each -feature to have mean 0 and standard deviation 1: +feature to have mean 0 and standard deviation 1. -Compute the mean and standard deviation of each column (feature) in $bm{X}$. +=== 1a) === + +Compute the mean and standard deviation of each column (feature) in $\bm{X}$. Subtract the mean and divide by the standard deviation for each feature. We will also center the target $\bm{y}$ to mean $0$. Centering $\bm{y}$ -(and each feature) means the model won’t require a separate intercept -term – the data is shifted such that the intercept is effectively 0 +(and each feature) means the model does not require a separate intercept +term, the data is shifted such that the intercept is effectively 0 . (In practice, one could include an intercept in the model and not penalize it, but here we simplify by centering.) @@ -92,17 +94,16 @@ y_mean = ? y_centered = ? !ec -=== 1a) === Fill in the necessary details. -After this preprocessing, each column of $\bm{X}_norm$ has mean zero and standard deviation $1$ -and $\bm{y}_centered$ has mean 0. This makes the optimization landscape +After this preprocessing, each column of $\bm{X}_{\mathrm{norm}}$ has mean zero and standard deviation $1$ +and $\bm{y}_{\mathrm{centered}}$ has mean 0. This makes the optimization landscape nicer and ensures the regularization penalty $\lambda \sum_j -\beta_j^2$ treats each coefficient fairly (since features are on the +\theta_j^2$ in Ridge regression treats each coefficient fairly (since features are on the same scale). -===== Exercise 2, use the analytical formulae for OLS and Ridge regression to find the optimal paramters $\bm{theta}$ ===== +===== Exercise 2, use the analytical formulae for OLS and Ridge regression to find the optimal paramters $\bm{\theta}$ ===== !bc pycod # Set regularization parameter, either a single value or a vector of values @@ -117,20 +118,19 @@ print("Closed-form Ridge coefficients:", theta_closed_form) print("Closed-form OLS coefficients:", theta_closed_form) !ec -This computes the ridge and OLS regression coefficients directly. The identity -matrix $I$ has the same size as $X^T X$ (which is n_features x -n_features), and lam * I adds $\lambda$ to the diagonal of $X^T X. We -then invert this matrix and multiply by $X^T y. The result -for $\bm{\theta}$ is a NumPy array of shape (n_features,) containing the -fitted weights. +This computes the Ridge and OLS regression coefficients directly. The identity +matrix $I$ has the same size as $X^T X$. It adds $\lambda$ to the diagonal of $X^T X for Ridge regression. We +then invert this matrix and multiply by $X^T y$. The result +for $\bm{\theta}$ is a NumPy array of shape (n$\_$features,) containing the +fitted parameters $\bm{\theta}$.. === 2a) === -Finalize the OLS and Ridge regression determination of the optimal parameters $bm{\theta}$. +Finalize, in the above code, the OLS and Ridge regression determination of the optimal parameters $\bm{\theta}$. === 2b) === Explore the results as function of different values of the hyperparameter $\lambda$. See for example exercise 4 from week 36. -===== Implementing the simplest form for gradient descent ===== +===== Exercise 3, Implementing the simplest form for gradient descent ===== Alternatively, we can fit the ridge regression model using gradient descent. This is useful to visualize the iterative convergence and is @@ -178,10 +178,10 @@ print("Gradient Descent Ridge coefficients:", theta_gdRidge) !ec === 3a) === -Discuss the results as function of the learning rate paramaters and the number of iterations. +Discuss the results as function of the learning rate parameters and the number of iterations. === 3b) === -Add a stopping parameter as function of the number iterations. +Try to add a stopping parameter as function of the number iterations. How would you define a stopping criterion? @@ -189,5 +189,5 @@ If everything worked correctly, the learned coefficients should be close to the true values [5.0, -3.0, 0.0, …, 2.0, …] that we used to generate the data. Keep in mind that due to regularization and noise, the learned values will not exactly equal the true ones, but they -should be in the same ballpark. +should be in the same ballpark. Which method (OLS or Ridge) gives the best results?