added execises week 42
This commit is contained in:
Morten Hjorth-Jensen
@@ -1,4 +1,4 @@
|
||||
# Sphinx build info version 1
|
||||
# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
|
||||
config: 0bfc9c8c9117066c9421f9443a3502e5
|
||||
config: 73ce6691cda151d4aabc9466268ebbb7
|
||||
tags: 645f666f9bcd5a90fca523b33c5a78b7
|
||||
|
||||
@@ -0,0 +1,719 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Exercises week 42\n",
|
||||
"\n",
|
||||
"**October 13-17, 2025**\n",
|
||||
"\n",
|
||||
"Date: **Deadline is Friday October 17 at midnight**\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Overarching aims of the exercises this week\n",
|
||||
"\n",
|
||||
"The aim of the exercises this week is to train the neural network you implemented last week.\n",
|
||||
"\n",
|
||||
"To train neural networks, we use gradient descent, since there is no analytical expression for the optimal parameters. This means you will need to compute the gradient of the cost function wrt. the network parameters. And then you will need to implement some gradient method.\n",
|
||||
"\n",
|
||||
"You will begin by computing gradients for a network with one layer, then two layers, then any number of layers. Keeping track of the shapes and doing things step by step will be very important this week.\n",
|
||||
"\n",
|
||||
"We recommend that you do the exercises this week by editing and running this notebook file, as it includes some checks along the way that you have implemented the neural network correctly, and running small parts of the code at a time will be important for understanding the methods. If you have trouble running a notebook, you can run this notebook in google colab instead(https://colab.research.google.com/drive/1FfvbN0XlhV-lATRPyGRTtTBnJr3zNuHL#offline=true&sandboxMode=true), though we recommend that you set up VSCode and your python environment to run code like this locally.\n",
|
||||
"\n",
|
||||
"First, some setup code that you will need.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import autograd.numpy as np # We need to use this numpy wrapper to make automatic differentiation work later\n",
|
||||
"from autograd import grad, elementwise_grad\n",
|
||||
"from sklearn import datasets\n",
|
||||
"import matplotlib.pyplot as plt\n",
|
||||
"from sklearn.metrics import accuracy_score\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"# Defining some activation functions\n",
|
||||
"def ReLU(z):\n",
|
||||
" return np.where(z > 0, z, 0)\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"# Derivative of the ReLU function\n",
|
||||
"def ReLU_der(z):\n",
|
||||
" return np.where(z > 0, 1, 0)\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"def sigmoid(z):\n",
|
||||
" return 1 / (1 + np.exp(-z))\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"def mse(predict, target):\n",
|
||||
" return np.mean((predict - target) ** 2)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Exercise 1 - Understand the feed forward pass\n",
|
||||
"\n",
|
||||
"**a)** Complete last weeks' exercises if you haven't already (recommended).\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Exercise 2 - Gradient with one layer using autograd\n",
|
||||
"\n",
|
||||
"For the first few exercises, we will not use batched inputs. Only a single input vector is passed through the layer at a time.\n",
|
||||
"\n",
|
||||
"In this exercise you will compute the gradient of a single layer. You only need to change the code in the cells right below an exercise, the rest works out of the box. Feel free to make changes and see how stuff works though!\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**a)** If the weights and bias of a layer has shapes (10, 4) and (10), what will the shapes of the gradients of the cost function wrt. these weights and this bias be?\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**b)** Complete the feed_forward_one_layer function. It should use the sigmoid activation function. Also define the weigth and bias with the correct shapes.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 41,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def feed_forward_one_layer(W, b, x):\n",
|
||||
" z = ...\n",
|
||||
" a = ...\n",
|
||||
" return a\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"def cost_one_layer(W, b, x, target):\n",
|
||||
" predict = feed_forward_one_layer(W, b, x)\n",
|
||||
" return mse(predict, target)\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"x = np.random.rand(2)\n",
|
||||
"target = np.random.rand(3)\n",
|
||||
"\n",
|
||||
"W = ...\n",
|
||||
"b = ..."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**c)** Compute the gradient of the cost function wrt. the weigth and bias by running the cell below. You will not need to change anything, just make sure it runs by defining things correctly in the cell above. This code uses the autograd package which uses backprogagation to compute the gradient!\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"autograd_one_layer = grad(cost_one_layer, [0, 1])\n",
|
||||
"W_g, b_g = autograd_one_layer(W, b, x, target)\n",
|
||||
"print(W_g, b_g)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Exercise 3 - Gradient with one layer writing backpropagation by hand\n",
|
||||
"\n",
|
||||
"Before you use the gradient you found using autograd, you will have to find the gradient \"manually\", to better understand how the backpropagation computation works. To do backpropagation \"manually\", you will need to write out expressions for many derivatives along the computation.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"We want to find the gradient of the cost function wrt. the weight and bias. This is quite hard to do directly, so we instead use the chain rule to combine multiple derivatives which are easier to compute.\n",
|
||||
"\n",
|
||||
"$$\n",
|
||||
"\\frac{dC}{dW} = \\frac{dC}{da}\\frac{da}{dz}\\frac{dz}{dW}\n",
|
||||
"$$\n",
|
||||
"\n",
|
||||
"$$\n",
|
||||
"\\frac{dC}{db} = \\frac{dC}{da}\\frac{da}{dz}\\frac{dz}{db}\n",
|
||||
"$$\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**a)** Which intermediary results can be reused between the two expressions?\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**b)** What is the derivative of the cost wrt. the final activation? You can use the autograd calculation to make sure you get the correct result. Remember that we compute the mean in mse.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"z = W @ x + b\n",
|
||||
"a = sigmoid(z)\n",
|
||||
"\n",
|
||||
"predict = a\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"def mse_der(predict, target):\n",
|
||||
" return ...\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"print(mse_der(predict, target))\n",
|
||||
"\n",
|
||||
"cost_autograd = grad(mse, 0)\n",
|
||||
"print(cost_autograd(predict, target))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**c)** What is the expression for the derivative of the sigmoid activation function? You can use the autograd calculation to make sure you get the correct result.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def sigmoid_der(z):\n",
|
||||
" return ...\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"print(sigmoid_der(z))\n",
|
||||
"\n",
|
||||
"sigmoid_autograd = elementwise_grad(sigmoid, 0)\n",
|
||||
"print(sigmoid_autograd(z))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**d)** Using the two derivatives you just computed, compute this intermetidary gradient you will use later:\n",
|
||||
"\n",
|
||||
"$$\n",
|
||||
"\\frac{dC}{dz} = \\frac{dC}{da}\\frac{da}{dz}\n",
|
||||
"$$\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 54,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"dC_da = ...\n",
|
||||
"dC_dz = ..."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**e)** What is the derivative of the intermediary z wrt. the weight and bias? What should the shapes be? The one for the weights is a little tricky, it can be easier to play around in the next exercise first. You can also try computing it with autograd to get a hint.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**f)** Now combine the expressions you have worked with so far to compute the gradients! Note that you always need to do a feed forward pass while saving the zs and as before you do backpropagation, as they are used in the derivative expressions\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"dC_da = ...\n",
|
||||
"dC_dz = ...\n",
|
||||
"dC_dW = ...\n",
|
||||
"dC_db = ...\n",
|
||||
"\n",
|
||||
"print(dC_dW, dC_db)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"You should get the same results as with autograd.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"W_g, b_g = autograd_one_layer(W, b, x, target)\n",
|
||||
"print(W_g, b_g)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Exercise 4 - Gradient with two layers writing backpropagation by hand\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Now that you have implemented backpropagation for one layer, you have found most of the expressions you will need for more layers. Let's move up to two layers.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 59,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"x = np.random.rand(2)\n",
|
||||
"target = np.random.rand(4)\n",
|
||||
"\n",
|
||||
"W1 = np.random.rand(3, 2)\n",
|
||||
"b1 = np.random.rand(3)\n",
|
||||
"\n",
|
||||
"W2 = np.random.rand(4, 3)\n",
|
||||
"b2 = np.random.rand(4)\n",
|
||||
"\n",
|
||||
"layers = [(W1, b1), (W2, b2)]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 60,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"z1 = W1 @ x + b1\n",
|
||||
"a1 = sigmoid(z1)\n",
|
||||
"z2 = W2 @ a1 + b2\n",
|
||||
"a2 = sigmoid(z2)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"We begin by computing the gradients of the last layer, as the gradients must be propagated backwards from the end.\n",
|
||||
"\n",
|
||||
"**a)** Compute the gradients of the last layer, just like you did the single layer in the previous exercise.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 61,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"dC_da2 = ...\n",
|
||||
"dC_dz2 = ...\n",
|
||||
"dC_dW2 = ...\n",
|
||||
"dC_db2 = ..."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"To find the derivative of the cost wrt. the activation of the first layer, we need a new expression, the one furthest to the right in the following.\n",
|
||||
"\n",
|
||||
"$$\n",
|
||||
"\\frac{dC}{da_1} = \\frac{dC}{dz_2}\\frac{dz_2}{da_1}\n",
|
||||
"$$\n",
|
||||
"\n",
|
||||
"**b)** What is the derivative of the second layer intermetiate wrt. the first layer activation? (First recall how you compute $z_2$)\n",
|
||||
"\n",
|
||||
"$$\n",
|
||||
"\\frac{dz_2}{da_1}\n",
|
||||
"$$\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**c)** Use this expression, together with expressions which are equivelent to ones for the last layer to compute all the derivatives of the first layer.\n",
|
||||
"\n",
|
||||
"$$\n",
|
||||
"\\frac{dC}{dW_1} = \\frac{dC}{da_1}\\frac{da_1}{dz_1}\\frac{dz_1}{dW_1}\n",
|
||||
"$$\n",
|
||||
"\n",
|
||||
"$$\n",
|
||||
"\\frac{dC}{db_1} = \\frac{dC}{da_1}\\frac{da_1}{dz_1}\\frac{dz_1}{db_1}\n",
|
||||
"$$\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 63,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"dC_da1 = ...\n",
|
||||
"dC_dz1 = ...\n",
|
||||
"dC_dW1 = ...\n",
|
||||
"dC_db1 = ..."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(dC_dW1, dC_db1)\n",
|
||||
"print(dC_dW2, dC_db2)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**d)** Make sure you got the same gradient as the following code which uses autograd to do backpropagation.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 67,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def feed_forward_two_layers(layers, x):\n",
|
||||
" W1, b1 = layers[0]\n",
|
||||
" z1 = W1 @ x + b1\n",
|
||||
" a1 = sigmoid(z1)\n",
|
||||
"\n",
|
||||
" W2, b2 = layers[1]\n",
|
||||
" z2 = W2 @ a1 + b2\n",
|
||||
" a2 = sigmoid(z2)\n",
|
||||
"\n",
|
||||
" return a2"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def cost_two_layers(layers, x, target):\n",
|
||||
" predict = feed_forward_two_layers(layers, x)\n",
|
||||
" return mse(predict, target)\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"grad_two_layers = grad(cost_two_layers, 0)\n",
|
||||
"grad_two_layers(layers, x, target)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**e)** How would you use the gradient from this layer to compute the gradient of an even earlier layer? Would the expressions be any different?\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Exercise 5 - Gradient with any number of layers writing backpropagation by hand\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Well done on getting this far! Now it's time to compute the gradient with any number of layers.\n",
|
||||
"\n",
|
||||
"First, some code from the general neural network code from last week. Note that we are still sending in one input vector at a time. We will change it to use batched inputs later.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def create_layers(network_input_size, layer_output_sizes):\n",
|
||||
" layers = []\n",
|
||||
"\n",
|
||||
" i_size = network_input_size\n",
|
||||
" for layer_output_size in layer_output_sizes:\n",
|
||||
" W = np.random.randn(layer_output_size, i_size)\n",
|
||||
" b = np.random.randn(layer_output_size)\n",
|
||||
" layers.append((W, b))\n",
|
||||
"\n",
|
||||
" i_size = layer_output_size\n",
|
||||
" return layers\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"def feed_forward(input, layers, activation_funcs):\n",
|
||||
" a = input\n",
|
||||
" for (W, b), activation_func in zip(layers, activation_funcs):\n",
|
||||
" z = W @ a + b\n",
|
||||
" a = activation_func(z)\n",
|
||||
" return a\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"def cost(layers, input, activation_funcs, target):\n",
|
||||
" predict = feed_forward(input, layers, activation_funcs)\n",
|
||||
" return mse(predict, target)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"You might have already have noticed a very important detail in backpropagation: You need the values from the forward pass to compute all the gradients! The feed forward method above is great for efficiency and for using autograd, as it only cares about computing the final output, but now we need to also save the results along the way.\n",
|
||||
"\n",
|
||||
"Here is a function which does that for you.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def feed_forward_saver(input, layers, activation_funcs):\n",
|
||||
" layer_inputs = []\n",
|
||||
" zs = []\n",
|
||||
" a = input\n",
|
||||
" for (W, b), activation_func in zip(layers, activation_funcs):\n",
|
||||
" layer_inputs.append(a)\n",
|
||||
" z = W @ a + b\n",
|
||||
" a = activation_func(z)\n",
|
||||
"\n",
|
||||
" zs.append(z)\n",
|
||||
"\n",
|
||||
" return layer_inputs, zs, a"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**a)** Now, complete the backpropagation function so that it returns the gradient of the cost function wrt. all the weigths and biases. Use the autograd calculation below to make sure you get the correct answer.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def backpropagation(\n",
|
||||
" input, layers, activation_funcs, target, activation_ders, cost_der=mse_der\n",
|
||||
"):\n",
|
||||
" layer_inputs, zs, predict = feed_forward_saver(input, layers, activation_funcs)\n",
|
||||
"\n",
|
||||
" layer_grads = [() for layer in layers]\n",
|
||||
"\n",
|
||||
" # We loop over the layers, from the last to the first\n",
|
||||
" for i in reversed(range(len(layers))):\n",
|
||||
" layer_input, z, activation_der = layer_inputs[i], zs[i], activation_ders[i]\n",
|
||||
"\n",
|
||||
" if i == len(layers) - 1:\n",
|
||||
" # For last layer we use cost derivative as dC_da(L) can be computed directly\n",
|
||||
" dC_da = ...\n",
|
||||
" else:\n",
|
||||
" # For other layers we build on previous z derivative, as dC_da(i) = dC_dz(i+1) * dz(i+1)_da(i)\n",
|
||||
" (W, b) = layers[i + 1]\n",
|
||||
" dC_da = ...\n",
|
||||
"\n",
|
||||
" dC_dz = ...\n",
|
||||
" dC_dW = ...\n",
|
||||
" dC_db = ...\n",
|
||||
"\n",
|
||||
" layer_grads[i] = (dC_dW, dC_db)\n",
|
||||
"\n",
|
||||
" return layer_grads"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"network_input_size = 2\n",
|
||||
"layer_output_sizes = [3, 4]\n",
|
||||
"activation_funcs = [sigmoid, ReLU]\n",
|
||||
"activation_ders = [sigmoid_der, ReLU_der]\n",
|
||||
"\n",
|
||||
"layers = create_layers(network_input_size, layer_output_sizes)\n",
|
||||
"\n",
|
||||
"x = np.random.rand(network_input_size)\n",
|
||||
"target = np.random.rand(4)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"layer_grads = backpropagation(x, layers, activation_funcs, target, activation_ders)\n",
|
||||
"print(layer_grads)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"cost_grad = grad(cost, 0)\n",
|
||||
"cost_grad(layers, x, [sigmoid, ReLU], target)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Exercise 6 - Batched inputs\n",
|
||||
"\n",
|
||||
"Make new versions of all the functions in exercise 5 which now take batched inputs instead. See last weeks exercise 5 for details on how to batch inputs to neural networks. You will also need to update the backpropogation function.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Exercise 7 - Training\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**a)** Complete exercise 6 and 7 from last week, but use your own backpropogation implementation to compute the gradient.\n",
|
||||
"- IMPORTANT: Do not implement the derivative terms for softmax and cross-entropy separately, it will be very hard!\n",
|
||||
"- Instead, use the fact that the derivatives multiplied together simplify to **prediction - target** (see [source1](https://medium.com/data-science/derivative-of-the-softmax-function-and-the-categorical-cross-entropy-loss-ffceefc081d1), [source2](https://shivammehta25.github.io/posts/deriving-categorical-cross-entropy-and-softmax/))\n",
|
||||
"\n",
|
||||
"**b)** Use stochastic gradient descent with momentum when you train your network.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Exercise 8 (Optional) - Object orientation\n",
|
||||
"\n",
|
||||
"Passing in the layers, activations functions, activation derivatives and cost derivatives into the functions each time leads to code which is easy to understand in isoloation, but messier when used in a larger context with data splitting, data scaling, gradient methods and so forth. Creating an object which stores these values can lead to code which is much easier to use.\n",
|
||||
"\n",
|
||||
"**a)** Write a neural network class. You are free to implement it how you see fit, though we strongly recommend to not save any input or output values as class attributes, nor let the neural network class handle gradient methods internally. Gradient methods should be handled outside, by performing general operations on the layer_grads list using functions or classes separate to the neural network.\n",
|
||||
"\n",
|
||||
"We provide here a skeleton structure which should get you started.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"class NeuralNetwork:\n",
|
||||
" def __init__(\n",
|
||||
" self,\n",
|
||||
" network_input_size,\n",
|
||||
" layer_output_sizes,\n",
|
||||
" activation_funcs,\n",
|
||||
" activation_ders,\n",
|
||||
" cost_fun,\n",
|
||||
" cost_der,\n",
|
||||
" ):\n",
|
||||
" pass\n",
|
||||
"\n",
|
||||
" def predict(self, inputs):\n",
|
||||
" # Simple feed forward pass\n",
|
||||
" pass\n",
|
||||
"\n",
|
||||
" def cost(self, inputs, targets):\n",
|
||||
" pass\n",
|
||||
"\n",
|
||||
" def _feed_forward_saver(self, inputs):\n",
|
||||
" pass\n",
|
||||
"\n",
|
||||
" def compute_gradient(self, inputs, targets):\n",
|
||||
" pass\n",
|
||||
"\n",
|
||||
" def update_weights(self, layer_grads):\n",
|
||||
" pass\n",
|
||||
"\n",
|
||||
" # These last two methods are not needed in the project, but they can be nice to have! The first one has a layers parameter so that you can use autograd on it\n",
|
||||
" def autograd_compliant_predict(self, layers, inputs):\n",
|
||||
" pass\n",
|
||||
"\n",
|
||||
" def autograd_gradient(self, inputs, targets):\n",
|
||||
" pass"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.9.15"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
@@ -6,11 +6,11 @@ html[data-theme="light"] .highlight span.linenos.special { color: #000000; backg
|
||||
html[data-theme="light"] .highlight .hll { background-color: #fae4c2 }
|
||||
html[data-theme="light"] .highlight { background: #fefefe; color: #080808 }
|
||||
html[data-theme="light"] .highlight .c { color: #515151 } /* Comment */
|
||||
html[data-theme="light"] .highlight .err { color: #A12236 } /* Error */
|
||||
html[data-theme="light"] .highlight .k { color: #6730C5 } /* Keyword */
|
||||
html[data-theme="light"] .highlight .l { color: #7F4707 } /* Literal */
|
||||
html[data-theme="light"] .highlight .err { color: #a12236 } /* Error */
|
||||
html[data-theme="light"] .highlight .k { color: #6730c5 } /* Keyword */
|
||||
html[data-theme="light"] .highlight .l { color: #7f4707 } /* Literal */
|
||||
html[data-theme="light"] .highlight .n { color: #080808 } /* Name */
|
||||
html[data-theme="light"] .highlight .o { color: #00622F } /* Operator */
|
||||
html[data-theme="light"] .highlight .o { color: #00622f } /* Operator */
|
||||
html[data-theme="light"] .highlight .p { color: #080808 } /* Punctuation */
|
||||
html[data-theme="light"] .highlight .ch { color: #515151 } /* Comment.Hashbang */
|
||||
html[data-theme="light"] .highlight .cm { color: #515151 } /* Comment.Multiline */
|
||||
@@ -18,135 +18,135 @@ html[data-theme="light"] .highlight .cp { color: #515151 } /* Comment.Preproc */
|
||||
html[data-theme="light"] .highlight .cpf { color: #515151 } /* Comment.PreprocFile */
|
||||
html[data-theme="light"] .highlight .c1 { color: #515151 } /* Comment.Single */
|
||||
html[data-theme="light"] .highlight .cs { color: #515151 } /* Comment.Special */
|
||||
html[data-theme="light"] .highlight .gd { color: #005B82 } /* Generic.Deleted */
|
||||
html[data-theme="light"] .highlight .gd { color: #005b82 } /* Generic.Deleted */
|
||||
html[data-theme="light"] .highlight .ge { font-style: italic } /* Generic.Emph */
|
||||
html[data-theme="light"] .highlight .gh { color: #005B82 } /* Generic.Heading */
|
||||
html[data-theme="light"] .highlight .gh { color: #005b82 } /* Generic.Heading */
|
||||
html[data-theme="light"] .highlight .gs { font-weight: bold } /* Generic.Strong */
|
||||
html[data-theme="light"] .highlight .gu { color: #005B82 } /* Generic.Subheading */
|
||||
html[data-theme="light"] .highlight .kc { color: #6730C5 } /* Keyword.Constant */
|
||||
html[data-theme="light"] .highlight .kd { color: #6730C5 } /* Keyword.Declaration */
|
||||
html[data-theme="light"] .highlight .kn { color: #6730C5 } /* Keyword.Namespace */
|
||||
html[data-theme="light"] .highlight .kp { color: #6730C5 } /* Keyword.Pseudo */
|
||||
html[data-theme="light"] .highlight .kr { color: #6730C5 } /* Keyword.Reserved */
|
||||
html[data-theme="light"] .highlight .kt { color: #7F4707 } /* Keyword.Type */
|
||||
html[data-theme="light"] .highlight .ld { color: #7F4707 } /* Literal.Date */
|
||||
html[data-theme="light"] .highlight .m { color: #7F4707 } /* Literal.Number */
|
||||
html[data-theme="light"] .highlight .s { color: #00622F } /* Literal.String */
|
||||
html[data-theme="light"] .highlight .gu { color: #005b82 } /* Generic.Subheading */
|
||||
html[data-theme="light"] .highlight .kc { color: #6730c5 } /* Keyword.Constant */
|
||||
html[data-theme="light"] .highlight .kd { color: #6730c5 } /* Keyword.Declaration */
|
||||
html[data-theme="light"] .highlight .kn { color: #6730c5 } /* Keyword.Namespace */
|
||||
html[data-theme="light"] .highlight .kp { color: #6730c5 } /* Keyword.Pseudo */
|
||||
html[data-theme="light"] .highlight .kr { color: #6730c5 } /* Keyword.Reserved */
|
||||
html[data-theme="light"] .highlight .kt { color: #7f4707 } /* Keyword.Type */
|
||||
html[data-theme="light"] .highlight .ld { color: #7f4707 } /* Literal.Date */
|
||||
html[data-theme="light"] .highlight .m { color: #7f4707 } /* Literal.Number */
|
||||
html[data-theme="light"] .highlight .s { color: #00622f } /* Literal.String */
|
||||
html[data-theme="light"] .highlight .na { color: #912583 } /* Name.Attribute */
|
||||
html[data-theme="light"] .highlight .nb { color: #7F4707 } /* Name.Builtin */
|
||||
html[data-theme="light"] .highlight .nc { color: #005B82 } /* Name.Class */
|
||||
html[data-theme="light"] .highlight .no { color: #005B82 } /* Name.Constant */
|
||||
html[data-theme="light"] .highlight .nd { color: #7F4707 } /* Name.Decorator */
|
||||
html[data-theme="light"] .highlight .ni { color: #00622F } /* Name.Entity */
|
||||
html[data-theme="light"] .highlight .ne { color: #6730C5 } /* Name.Exception */
|
||||
html[data-theme="light"] .highlight .nf { color: #005B82 } /* Name.Function */
|
||||
html[data-theme="light"] .highlight .nl { color: #7F4707 } /* Name.Label */
|
||||
html[data-theme="light"] .highlight .nb { color: #7f4707 } /* Name.Builtin */
|
||||
html[data-theme="light"] .highlight .nc { color: #005b82 } /* Name.Class */
|
||||
html[data-theme="light"] .highlight .no { color: #005b82 } /* Name.Constant */
|
||||
html[data-theme="light"] .highlight .nd { color: #7f4707 } /* Name.Decorator */
|
||||
html[data-theme="light"] .highlight .ni { color: #00622f } /* Name.Entity */
|
||||
html[data-theme="light"] .highlight .ne { color: #6730c5 } /* Name.Exception */
|
||||
html[data-theme="light"] .highlight .nf { color: #005b82 } /* Name.Function */
|
||||
html[data-theme="light"] .highlight .nl { color: #7f4707 } /* Name.Label */
|
||||
html[data-theme="light"] .highlight .nn { color: #080808 } /* Name.Namespace */
|
||||
html[data-theme="light"] .highlight .nx { color: #080808 } /* Name.Other */
|
||||
html[data-theme="light"] .highlight .py { color: #005B82 } /* Name.Property */
|
||||
html[data-theme="light"] .highlight .nt { color: #005B82 } /* Name.Tag */
|
||||
html[data-theme="light"] .highlight .nv { color: #A12236 } /* Name.Variable */
|
||||
html[data-theme="light"] .highlight .ow { color: #6730C5 } /* Operator.Word */
|
||||
html[data-theme="light"] .highlight .py { color: #005b82 } /* Name.Property */
|
||||
html[data-theme="light"] .highlight .nt { color: #005b82 } /* Name.Tag */
|
||||
html[data-theme="light"] .highlight .nv { color: #a12236 } /* Name.Variable */
|
||||
html[data-theme="light"] .highlight .ow { color: #6730c5 } /* Operator.Word */
|
||||
html[data-theme="light"] .highlight .pm { color: #080808 } /* Punctuation.Marker */
|
||||
html[data-theme="light"] .highlight .w { color: #080808 } /* Text.Whitespace */
|
||||
html[data-theme="light"] .highlight .mb { color: #7F4707 } /* Literal.Number.Bin */
|
||||
html[data-theme="light"] .highlight .mf { color: #7F4707 } /* Literal.Number.Float */
|
||||
html[data-theme="light"] .highlight .mh { color: #7F4707 } /* Literal.Number.Hex */
|
||||
html[data-theme="light"] .highlight .mi { color: #7F4707 } /* Literal.Number.Integer */
|
||||
html[data-theme="light"] .highlight .mo { color: #7F4707 } /* Literal.Number.Oct */
|
||||
html[data-theme="light"] .highlight .sa { color: #00622F } /* Literal.String.Affix */
|
||||
html[data-theme="light"] .highlight .sb { color: #00622F } /* Literal.String.Backtick */
|
||||
html[data-theme="light"] .highlight .sc { color: #00622F } /* Literal.String.Char */
|
||||
html[data-theme="light"] .highlight .dl { color: #00622F } /* Literal.String.Delimiter */
|
||||
html[data-theme="light"] .highlight .sd { color: #00622F } /* Literal.String.Doc */
|
||||
html[data-theme="light"] .highlight .s2 { color: #00622F } /* Literal.String.Double */
|
||||
html[data-theme="light"] .highlight .se { color: #00622F } /* Literal.String.Escape */
|
||||
html[data-theme="light"] .highlight .sh { color: #00622F } /* Literal.String.Heredoc */
|
||||
html[data-theme="light"] .highlight .si { color: #00622F } /* Literal.String.Interpol */
|
||||
html[data-theme="light"] .highlight .sx { color: #00622F } /* Literal.String.Other */
|
||||
html[data-theme="light"] .highlight .sr { color: #A12236 } /* Literal.String.Regex */
|
||||
html[data-theme="light"] .highlight .s1 { color: #00622F } /* Literal.String.Single */
|
||||
html[data-theme="light"] .highlight .ss { color: #005B82 } /* Literal.String.Symbol */
|
||||
html[data-theme="light"] .highlight .bp { color: #7F4707 } /* Name.Builtin.Pseudo */
|
||||
html[data-theme="light"] .highlight .fm { color: #005B82 } /* Name.Function.Magic */
|
||||
html[data-theme="light"] .highlight .vc { color: #A12236 } /* Name.Variable.Class */
|
||||
html[data-theme="light"] .highlight .vg { color: #A12236 } /* Name.Variable.Global */
|
||||
html[data-theme="light"] .highlight .vi { color: #A12236 } /* Name.Variable.Instance */
|
||||
html[data-theme="light"] .highlight .vm { color: #7F4707 } /* Name.Variable.Magic */
|
||||
html[data-theme="light"] .highlight .il { color: #7F4707 } /* Literal.Number.Integer.Long */
|
||||
html[data-theme="light"] .highlight .mb { color: #7f4707 } /* Literal.Number.Bin */
|
||||
html[data-theme="light"] .highlight .mf { color: #7f4707 } /* Literal.Number.Float */
|
||||
html[data-theme="light"] .highlight .mh { color: #7f4707 } /* Literal.Number.Hex */
|
||||
html[data-theme="light"] .highlight .mi { color: #7f4707 } /* Literal.Number.Integer */
|
||||
html[data-theme="light"] .highlight .mo { color: #7f4707 } /* Literal.Number.Oct */
|
||||
html[data-theme="light"] .highlight .sa { color: #00622f } /* Literal.String.Affix */
|
||||
html[data-theme="light"] .highlight .sb { color: #00622f } /* Literal.String.Backtick */
|
||||
html[data-theme="light"] .highlight .sc { color: #00622f } /* Literal.String.Char */
|
||||
html[data-theme="light"] .highlight .dl { color: #00622f } /* Literal.String.Delimiter */
|
||||
html[data-theme="light"] .highlight .sd { color: #00622f } /* Literal.String.Doc */
|
||||
html[data-theme="light"] .highlight .s2 { color: #00622f } /* Literal.String.Double */
|
||||
html[data-theme="light"] .highlight .se { color: #00622f } /* Literal.String.Escape */
|
||||
html[data-theme="light"] .highlight .sh { color: #00622f } /* Literal.String.Heredoc */
|
||||
html[data-theme="light"] .highlight .si { color: #00622f } /* Literal.String.Interpol */
|
||||
html[data-theme="light"] .highlight .sx { color: #00622f } /* Literal.String.Other */
|
||||
html[data-theme="light"] .highlight .sr { color: #a12236 } /* Literal.String.Regex */
|
||||
html[data-theme="light"] .highlight .s1 { color: #00622f } /* Literal.String.Single */
|
||||
html[data-theme="light"] .highlight .ss { color: #005b82 } /* Literal.String.Symbol */
|
||||
html[data-theme="light"] .highlight .bp { color: #7f4707 } /* Name.Builtin.Pseudo */
|
||||
html[data-theme="light"] .highlight .fm { color: #005b82 } /* Name.Function.Magic */
|
||||
html[data-theme="light"] .highlight .vc { color: #a12236 } /* Name.Variable.Class */
|
||||
html[data-theme="light"] .highlight .vg { color: #a12236 } /* Name.Variable.Global */
|
||||
html[data-theme="light"] .highlight .vi { color: #a12236 } /* Name.Variable.Instance */
|
||||
html[data-theme="light"] .highlight .vm { color: #7f4707 } /* Name.Variable.Magic */
|
||||
html[data-theme="light"] .highlight .il { color: #7f4707 } /* Literal.Number.Integer.Long */
|
||||
html[data-theme="dark"] .highlight pre { line-height: 125%; }
|
||||
html[data-theme="dark"] .highlight td.linenos .normal { color: inherit; background-color: transparent; padding-left: 5px; padding-right: 5px; }
|
||||
html[data-theme="dark"] .highlight span.linenos { color: inherit; background-color: transparent; padding-left: 5px; padding-right: 5px; }
|
||||
html[data-theme="dark"] .highlight td.linenos .special { color: #000000; background-color: #ffffc0; padding-left: 5px; padding-right: 5px; }
|
||||
html[data-theme="dark"] .highlight span.linenos.special { color: #000000; background-color: #ffffc0; padding-left: 5px; padding-right: 5px; }
|
||||
html[data-theme="dark"] .highlight .hll { background-color: #ffd9002e }
|
||||
html[data-theme="dark"] .highlight { background: #2b2b2b; color: #F8F8F2 }
|
||||
html[data-theme="dark"] .highlight .c { color: #FFD900 } /* Comment */
|
||||
html[data-theme="dark"] .highlight .err { color: #FFA07A } /* Error */
|
||||
html[data-theme="dark"] .highlight .k { color: #DCC6E0 } /* Keyword */
|
||||
html[data-theme="dark"] .highlight .l { color: #FFD900 } /* Literal */
|
||||
html[data-theme="dark"] .highlight .n { color: #F8F8F2 } /* Name */
|
||||
html[data-theme="dark"] .highlight .o { color: #ABE338 } /* Operator */
|
||||
html[data-theme="dark"] .highlight .p { color: #F8F8F2 } /* Punctuation */
|
||||
html[data-theme="dark"] .highlight .ch { color: #FFD900 } /* Comment.Hashbang */
|
||||
html[data-theme="dark"] .highlight .cm { color: #FFD900 } /* Comment.Multiline */
|
||||
html[data-theme="dark"] .highlight .cp { color: #FFD900 } /* Comment.Preproc */
|
||||
html[data-theme="dark"] .highlight .cpf { color: #FFD900 } /* Comment.PreprocFile */
|
||||
html[data-theme="dark"] .highlight .c1 { color: #FFD900 } /* Comment.Single */
|
||||
html[data-theme="dark"] .highlight .cs { color: #FFD900 } /* Comment.Special */
|
||||
html[data-theme="dark"] .highlight .gd { color: #00E0E0 } /* Generic.Deleted */
|
||||
html[data-theme="dark"] .highlight { background: #2b2b2b; color: #f8f8f2 }
|
||||
html[data-theme="dark"] .highlight .c { color: #ffd900 } /* Comment */
|
||||
html[data-theme="dark"] .highlight .err { color: #ffa07a } /* Error */
|
||||
html[data-theme="dark"] .highlight .k { color: #dcc6e0 } /* Keyword */
|
||||
html[data-theme="dark"] .highlight .l { color: #ffd900 } /* Literal */
|
||||
html[data-theme="dark"] .highlight .n { color: #f8f8f2 } /* Name */
|
||||
html[data-theme="dark"] .highlight .o { color: #abe338 } /* Operator */
|
||||
html[data-theme="dark"] .highlight .p { color: #f8f8f2 } /* Punctuation */
|
||||
html[data-theme="dark"] .highlight .ch { color: #ffd900 } /* Comment.Hashbang */
|
||||
html[data-theme="dark"] .highlight .cm { color: #ffd900 } /* Comment.Multiline */
|
||||
html[data-theme="dark"] .highlight .cp { color: #ffd900 } /* Comment.Preproc */
|
||||
html[data-theme="dark"] .highlight .cpf { color: #ffd900 } /* Comment.PreprocFile */
|
||||
html[data-theme="dark"] .highlight .c1 { color: #ffd900 } /* Comment.Single */
|
||||
html[data-theme="dark"] .highlight .cs { color: #ffd900 } /* Comment.Special */
|
||||
html[data-theme="dark"] .highlight .gd { color: #00e0e0 } /* Generic.Deleted */
|
||||
html[data-theme="dark"] .highlight .ge { font-style: italic } /* Generic.Emph */
|
||||
html[data-theme="dark"] .highlight .gh { color: #00E0E0 } /* Generic.Heading */
|
||||
html[data-theme="dark"] .highlight .gh { color: #00e0e0 } /* Generic.Heading */
|
||||
html[data-theme="dark"] .highlight .gs { font-weight: bold } /* Generic.Strong */
|
||||
html[data-theme="dark"] .highlight .gu { color: #00E0E0 } /* Generic.Subheading */
|
||||
html[data-theme="dark"] .highlight .kc { color: #DCC6E0 } /* Keyword.Constant */
|
||||
html[data-theme="dark"] .highlight .kd { color: #DCC6E0 } /* Keyword.Declaration */
|
||||
html[data-theme="dark"] .highlight .kn { color: #DCC6E0 } /* Keyword.Namespace */
|
||||
html[data-theme="dark"] .highlight .kp { color: #DCC6E0 } /* Keyword.Pseudo */
|
||||
html[data-theme="dark"] .highlight .kr { color: #DCC6E0 } /* Keyword.Reserved */
|
||||
html[data-theme="dark"] .highlight .kt { color: #FFD900 } /* Keyword.Type */
|
||||
html[data-theme="dark"] .highlight .ld { color: #FFD900 } /* Literal.Date */
|
||||
html[data-theme="dark"] .highlight .m { color: #FFD900 } /* Literal.Number */
|
||||
html[data-theme="dark"] .highlight .s { color: #ABE338 } /* Literal.String */
|
||||
html[data-theme="dark"] .highlight .na { color: #FFD900 } /* Name.Attribute */
|
||||
html[data-theme="dark"] .highlight .nb { color: #FFD900 } /* Name.Builtin */
|
||||
html[data-theme="dark"] .highlight .nc { color: #00E0E0 } /* Name.Class */
|
||||
html[data-theme="dark"] .highlight .no { color: #00E0E0 } /* Name.Constant */
|
||||
html[data-theme="dark"] .highlight .nd { color: #FFD900 } /* Name.Decorator */
|
||||
html[data-theme="dark"] .highlight .ni { color: #ABE338 } /* Name.Entity */
|
||||
html[data-theme="dark"] .highlight .ne { color: #DCC6E0 } /* Name.Exception */
|
||||
html[data-theme="dark"] .highlight .nf { color: #00E0E0 } /* Name.Function */
|
||||
html[data-theme="dark"] .highlight .nl { color: #FFD900 } /* Name.Label */
|
||||
html[data-theme="dark"] .highlight .nn { color: #F8F8F2 } /* Name.Namespace */
|
||||
html[data-theme="dark"] .highlight .nx { color: #F8F8F2 } /* Name.Other */
|
||||
html[data-theme="dark"] .highlight .py { color: #00E0E0 } /* Name.Property */
|
||||
html[data-theme="dark"] .highlight .nt { color: #00E0E0 } /* Name.Tag */
|
||||
html[data-theme="dark"] .highlight .nv { color: #FFA07A } /* Name.Variable */
|
||||
html[data-theme="dark"] .highlight .ow { color: #DCC6E0 } /* Operator.Word */
|
||||
html[data-theme="dark"] .highlight .pm { color: #F8F8F2 } /* Punctuation.Marker */
|
||||
html[data-theme="dark"] .highlight .w { color: #F8F8F2 } /* Text.Whitespace */
|
||||
html[data-theme="dark"] .highlight .mb { color: #FFD900 } /* Literal.Number.Bin */
|
||||
html[data-theme="dark"] .highlight .mf { color: #FFD900 } /* Literal.Number.Float */
|
||||
html[data-theme="dark"] .highlight .mh { color: #FFD900 } /* Literal.Number.Hex */
|
||||
html[data-theme="dark"] .highlight .mi { color: #FFD900 } /* Literal.Number.Integer */
|
||||
html[data-theme="dark"] .highlight .mo { color: #FFD900 } /* Literal.Number.Oct */
|
||||
html[data-theme="dark"] .highlight .sa { color: #ABE338 } /* Literal.String.Affix */
|
||||
html[data-theme="dark"] .highlight .sb { color: #ABE338 } /* Literal.String.Backtick */
|
||||
html[data-theme="dark"] .highlight .sc { color: #ABE338 } /* Literal.String.Char */
|
||||
html[data-theme="dark"] .highlight .dl { color: #ABE338 } /* Literal.String.Delimiter */
|
||||
html[data-theme="dark"] .highlight .sd { color: #ABE338 } /* Literal.String.Doc */
|
||||
html[data-theme="dark"] .highlight .s2 { color: #ABE338 } /* Literal.String.Double */
|
||||
html[data-theme="dark"] .highlight .se { color: #ABE338 } /* Literal.String.Escape */
|
||||
html[data-theme="dark"] .highlight .sh { color: #ABE338 } /* Literal.String.Heredoc */
|
||||
html[data-theme="dark"] .highlight .si { color: #ABE338 } /* Literal.String.Interpol */
|
||||
html[data-theme="dark"] .highlight .sx { color: #ABE338 } /* Literal.String.Other */
|
||||
html[data-theme="dark"] .highlight .sr { color: #FFA07A } /* Literal.String.Regex */
|
||||
html[data-theme="dark"] .highlight .s1 { color: #ABE338 } /* Literal.String.Single */
|
||||
html[data-theme="dark"] .highlight .ss { color: #00E0E0 } /* Literal.String.Symbol */
|
||||
html[data-theme="dark"] .highlight .bp { color: #FFD900 } /* Name.Builtin.Pseudo */
|
||||
html[data-theme="dark"] .highlight .fm { color: #00E0E0 } /* Name.Function.Magic */
|
||||
html[data-theme="dark"] .highlight .vc { color: #FFA07A } /* Name.Variable.Class */
|
||||
html[data-theme="dark"] .highlight .vg { color: #FFA07A } /* Name.Variable.Global */
|
||||
html[data-theme="dark"] .highlight .vi { color: #FFA07A } /* Name.Variable.Instance */
|
||||
html[data-theme="dark"] .highlight .vm { color: #FFD900 } /* Name.Variable.Magic */
|
||||
html[data-theme="dark"] .highlight .il { color: #FFD900 } /* Literal.Number.Integer.Long */
|
||||
html[data-theme="dark"] .highlight .gu { color: #00e0e0 } /* Generic.Subheading */
|
||||
html[data-theme="dark"] .highlight .kc { color: #dcc6e0 } /* Keyword.Constant */
|
||||
html[data-theme="dark"] .highlight .kd { color: #dcc6e0 } /* Keyword.Declaration */
|
||||
html[data-theme="dark"] .highlight .kn { color: #dcc6e0 } /* Keyword.Namespace */
|
||||
html[data-theme="dark"] .highlight .kp { color: #dcc6e0 } /* Keyword.Pseudo */
|
||||
html[data-theme="dark"] .highlight .kr { color: #dcc6e0 } /* Keyword.Reserved */
|
||||
html[data-theme="dark"] .highlight .kt { color: #ffd900 } /* Keyword.Type */
|
||||
html[data-theme="dark"] .highlight .ld { color: #ffd900 } /* Literal.Date */
|
||||
html[data-theme="dark"] .highlight .m { color: #ffd900 } /* Literal.Number */
|
||||
html[data-theme="dark"] .highlight .s { color: #abe338 } /* Literal.String */
|
||||
html[data-theme="dark"] .highlight .na { color: #ffd900 } /* Name.Attribute */
|
||||
html[data-theme="dark"] .highlight .nb { color: #ffd900 } /* Name.Builtin */
|
||||
html[data-theme="dark"] .highlight .nc { color: #00e0e0 } /* Name.Class */
|
||||
html[data-theme="dark"] .highlight .no { color: #00e0e0 } /* Name.Constant */
|
||||
html[data-theme="dark"] .highlight .nd { color: #ffd900 } /* Name.Decorator */
|
||||
html[data-theme="dark"] .highlight .ni { color: #abe338 } /* Name.Entity */
|
||||
html[data-theme="dark"] .highlight .ne { color: #dcc6e0 } /* Name.Exception */
|
||||
html[data-theme="dark"] .highlight .nf { color: #00e0e0 } /* Name.Function */
|
||||
html[data-theme="dark"] .highlight .nl { color: #ffd900 } /* Name.Label */
|
||||
html[data-theme="dark"] .highlight .nn { color: #f8f8f2 } /* Name.Namespace */
|
||||
html[data-theme="dark"] .highlight .nx { color: #f8f8f2 } /* Name.Other */
|
||||
html[data-theme="dark"] .highlight .py { color: #00e0e0 } /* Name.Property */
|
||||
html[data-theme="dark"] .highlight .nt { color: #00e0e0 } /* Name.Tag */
|
||||
html[data-theme="dark"] .highlight .nv { color: #ffa07a } /* Name.Variable */
|
||||
html[data-theme="dark"] .highlight .ow { color: #dcc6e0 } /* Operator.Word */
|
||||
html[data-theme="dark"] .highlight .pm { color: #f8f8f2 } /* Punctuation.Marker */
|
||||
html[data-theme="dark"] .highlight .w { color: #f8f8f2 } /* Text.Whitespace */
|
||||
html[data-theme="dark"] .highlight .mb { color: #ffd900 } /* Literal.Number.Bin */
|
||||
html[data-theme="dark"] .highlight .mf { color: #ffd900 } /* Literal.Number.Float */
|
||||
html[data-theme="dark"] .highlight .mh { color: #ffd900 } /* Literal.Number.Hex */
|
||||
html[data-theme="dark"] .highlight .mi { color: #ffd900 } /* Literal.Number.Integer */
|
||||
html[data-theme="dark"] .highlight .mo { color: #ffd900 } /* Literal.Number.Oct */
|
||||
html[data-theme="dark"] .highlight .sa { color: #abe338 } /* Literal.String.Affix */
|
||||
html[data-theme="dark"] .highlight .sb { color: #abe338 } /* Literal.String.Backtick */
|
||||
html[data-theme="dark"] .highlight .sc { color: #abe338 } /* Literal.String.Char */
|
||||
html[data-theme="dark"] .highlight .dl { color: #abe338 } /* Literal.String.Delimiter */
|
||||
html[data-theme="dark"] .highlight .sd { color: #abe338 } /* Literal.String.Doc */
|
||||
html[data-theme="dark"] .highlight .s2 { color: #abe338 } /* Literal.String.Double */
|
||||
html[data-theme="dark"] .highlight .se { color: #abe338 } /* Literal.String.Escape */
|
||||
html[data-theme="dark"] .highlight .sh { color: #abe338 } /* Literal.String.Heredoc */
|
||||
html[data-theme="dark"] .highlight .si { color: #abe338 } /* Literal.String.Interpol */
|
||||
html[data-theme="dark"] .highlight .sx { color: #abe338 } /* Literal.String.Other */
|
||||
html[data-theme="dark"] .highlight .sr { color: #ffa07a } /* Literal.String.Regex */
|
||||
html[data-theme="dark"] .highlight .s1 { color: #abe338 } /* Literal.String.Single */
|
||||
html[data-theme="dark"] .highlight .ss { color: #00e0e0 } /* Literal.String.Symbol */
|
||||
html[data-theme="dark"] .highlight .bp { color: #ffd900 } /* Name.Builtin.Pseudo */
|
||||
html[data-theme="dark"] .highlight .fm { color: #00e0e0 } /* Name.Function.Magic */
|
||||
html[data-theme="dark"] .highlight .vc { color: #ffa07a } /* Name.Variable.Class */
|
||||
html[data-theme="dark"] .highlight .vg { color: #ffa07a } /* Name.Variable.Global */
|
||||
html[data-theme="dark"] .highlight .vi { color: #ffa07a } /* Name.Variable.Instance */
|
||||
html[data-theme="dark"] .highlight .vm { color: #ffd900 } /* Name.Variable.Magic */
|
||||
html[data-theme="dark"] .highlight .il { color: #ffd900 } /* Literal.Number.Integer.Long */
|
||||
@@ -28,7 +28,7 @@
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-brands-400.woff2" />
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-regular-400.woff2" />
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/sphinx-book-theme.css?v=eba8b062" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css?v=13237357" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
|
||||
@@ -246,6 +246,16 @@
|
||||
|
||||
|
||||
|
||||
<li class="toctree-l1"><a class="reference internal" href="exercisesweek42.html">Exercises week 42</a></li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
|
||||
<ul class="nav bd-sidenav">
|
||||
|
||||
@@ -28,7 +28,7 @@
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-brands-400.woff2" />
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-regular-400.woff2" />
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/sphinx-book-theme.css?v=eba8b062" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css?v=13237357" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
|
||||
@@ -246,6 +246,16 @@
|
||||
|
||||
|
||||
|
||||
<li class="toctree-l1"><a class="reference internal" href="exercisesweek42.html">Exercises week 42</a></li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
|
||||
<ul class="nav bd-sidenav">
|
||||
|
||||
@@ -28,7 +28,7 @@
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-brands-400.woff2" />
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-regular-400.woff2" />
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/sphinx-book-theme.css?v=eba8b062" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css?v=13237357" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
|
||||
@@ -246,6 +246,16 @@
|
||||
|
||||
|
||||
|
||||
<li class="toctree-l1"><a class="reference internal" href="exercisesweek42.html">Exercises week 42</a></li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
|
||||
<ul class="nav bd-sidenav">
|
||||
|
||||
@@ -28,7 +28,7 @@
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-brands-400.woff2" />
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-regular-400.woff2" />
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/sphinx-book-theme.css?v=eba8b062" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css?v=13237357" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
|
||||
@@ -246,6 +246,16 @@
|
||||
|
||||
|
||||
|
||||
<li class="toctree-l1"><a class="reference internal" href="exercisesweek42.html">Exercises week 42</a></li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
|
||||
<ul class="nav bd-sidenav">
|
||||
|
||||
@@ -28,7 +28,7 @@
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-brands-400.woff2" />
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-regular-400.woff2" />
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/sphinx-book-theme.css?v=eba8b062" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css?v=13237357" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
|
||||
@@ -246,6 +246,16 @@
|
||||
|
||||
|
||||
|
||||
<li class="toctree-l1"><a class="reference internal" href="exercisesweek42.html">Exercises week 42</a></li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
|
||||
<ul class="nav bd-sidenav">
|
||||
|
||||
@@ -28,7 +28,7 @@
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-brands-400.woff2" />
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-regular-400.woff2" />
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/sphinx-book-theme.css?v=eba8b062" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css?v=13237357" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
|
||||
@@ -246,6 +246,16 @@
|
||||
|
||||
|
||||
|
||||
<li class="toctree-l1"><a class="reference internal" href="exercisesweek42.html">Exercises week 42</a></li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
|
||||
<ul class="nav bd-sidenav">
|
||||
|
||||
@@ -28,7 +28,7 @@
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-brands-400.woff2" />
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-regular-400.woff2" />
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/sphinx-book-theme.css?v=eba8b062" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css?v=13237357" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
|
||||
@@ -246,6 +246,16 @@
|
||||
|
||||
|
||||
|
||||
<li class="toctree-l1"><a class="reference internal" href="exercisesweek42.html">Exercises week 42</a></li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
|
||||
<ul class="nav bd-sidenav">
|
||||
|
||||
@@ -28,7 +28,7 @@
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-brands-400.woff2" />
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-regular-400.woff2" />
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/sphinx-book-theme.css?v=eba8b062" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css?v=13237357" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
|
||||
@@ -246,6 +246,16 @@
|
||||
|
||||
|
||||
|
||||
<li class="toctree-l1"><a class="reference internal" href="exercisesweek42.html">Exercises week 42</a></li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
|
||||
<ul class="nav bd-sidenav">
|
||||
|
||||
@@ -28,7 +28,7 @@
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-brands-400.woff2" />
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-regular-400.woff2" />
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/sphinx-book-theme.css?v=eba8b062" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css?v=13237357" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
|
||||
@@ -246,6 +246,16 @@
|
||||
|
||||
|
||||
|
||||
<li class="toctree-l1"><a class="reference internal" href="exercisesweek42.html">Exercises week 42</a></li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
|
||||
<ul class="nav bd-sidenav">
|
||||
|
||||
@@ -28,7 +28,7 @@
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-brands-400.woff2" />
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-regular-400.woff2" />
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/sphinx-book-theme.css?v=eba8b062" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css?v=13237357" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
|
||||
@@ -246,6 +246,16 @@
|
||||
|
||||
|
||||
|
||||
<li class="toctree-l1"><a class="reference internal" href="exercisesweek42.html">Exercises week 42</a></li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
|
||||
<ul class="nav bd-sidenav">
|
||||
|
||||
@@ -28,7 +28,7 @@
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-brands-400.woff2" />
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-regular-400.woff2" />
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/sphinx-book-theme.css?v=eba8b062" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css?v=13237357" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
|
||||
@@ -246,6 +246,16 @@
|
||||
|
||||
|
||||
|
||||
<li class="toctree-l1"><a class="reference internal" href="exercisesweek42.html">Exercises week 42</a></li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
|
||||
<ul class="nav bd-sidenav">
|
||||
|
||||
@@ -28,7 +28,7 @@
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-brands-400.woff2" />
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-regular-400.woff2" />
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/sphinx-book-theme.css?v=eba8b062" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css?v=13237357" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
|
||||
@@ -246,6 +246,16 @@
|
||||
|
||||
|
||||
|
||||
<li class="toctree-l1"><a class="reference internal" href="exercisesweek42.html">Exercises week 42</a></li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
|
||||
<ul class="nav bd-sidenav">
|
||||
|
||||
@@ -28,7 +28,7 @@
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-brands-400.woff2" />
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-regular-400.woff2" />
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/sphinx-book-theme.css?v=eba8b062" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css?v=13237357" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
|
||||
@@ -246,6 +246,16 @@
|
||||
|
||||
|
||||
|
||||
<li class="toctree-l1"><a class="reference internal" href="exercisesweek42.html">Exercises week 42</a></li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
|
||||
<ul class="nav bd-sidenav">
|
||||
|
||||
@@ -28,7 +28,7 @@
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-brands-400.woff2" />
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-regular-400.woff2" />
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/sphinx-book-theme.css?v=eba8b062" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css?v=13237357" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
|
||||
@@ -246,6 +246,16 @@
|
||||
|
||||
|
||||
|
||||
<li class="toctree-l1"><a class="reference internal" href="exercisesweek42.html">Exercises week 42</a></li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
|
||||
<ul class="nav bd-sidenav">
|
||||
|
||||
@@ -28,7 +28,7 @@
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-brands-400.woff2" />
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-regular-400.woff2" />
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/sphinx-book-theme.css?v=eba8b062" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css?v=13237357" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
|
||||
@@ -246,6 +246,16 @@
|
||||
|
||||
|
||||
|
||||
<li class="toctree-l1"><a class="reference internal" href="exercisesweek42.html">Exercises week 42</a></li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
|
||||
<ul class="nav bd-sidenav">
|
||||
|
||||
@@ -28,7 +28,7 @@
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-brands-400.woff2" />
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-regular-400.woff2" />
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/sphinx-book-theme.css?v=eba8b062" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css?v=13237357" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
|
||||
@@ -244,6 +244,16 @@
|
||||
|
||||
|
||||
|
||||
<li class="toctree-l1"><a class="reference internal" href="exercisesweek42.html">Exercises week 42</a></li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
|
||||
<ul class="nav bd-sidenav">
|
||||
@@ -491,11 +501,11 @@ document.write(`
|
||||
<p><strong>b)</strong> Compute the mean square error for the line model and for the second degree polynomial model.</p>
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span><span class="w"> </span><span class="nn">numpy</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">np</span>
|
||||
<span class="kn">import</span><span class="w"> </span><span class="nn">matplotlib.pyplot</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">plt</span>
|
||||
<span class="kn">from</span><span class="w"> </span><span class="nn">sklearn.preprocessing</span><span class="w"> </span><span class="kn">import</span> <span class="n">PolynomialFeatures</span> <span class="c1"># use the fit_transform method of the created object!</span>
|
||||
<span class="kn">from</span><span class="w"> </span><span class="nn">sklearn.linear_model</span><span class="w"> </span><span class="kn">import</span> <span class="n">LinearRegression</span>
|
||||
<span class="kn">from</span><span class="w"> </span><span class="nn">sklearn.metrics</span><span class="w"> </span><span class="kn">import</span> <span class="n">mean_squared_error</span>
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
|
||||
<span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span>
|
||||
<span class="kn">from</span> <span class="nn">sklearn.preprocessing</span> <span class="kn">import</span> <span class="n">PolynomialFeatures</span> <span class="c1"># use the fit_transform method of the created object!</span>
|
||||
<span class="kn">from</span> <span class="nn">sklearn.linear_model</span> <span class="kn">import</span> <span class="n">LinearRegression</span>
|
||||
<span class="kn">from</span> <span class="nn">sklearn.metrics</span> <span class="kn">import</span> <span class="n">mean_squared_error</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -532,7 +542,7 @@ document.write(`
|
||||
<p>Hopefully your model fit the data quite well, but to know how well the model actually generalizes to unseen data, which is most often what we care about, we need to split our data into training and testing data.</p>
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span><span class="w"> </span><span class="nn">sklearn.model_selection</span><span class="w"> </span><span class="kn">import</span> <span class="n">train_test_split</span>
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">sklearn.model_selection</span> <span class="kn">import</span> <span class="n">train_test_split</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@@ -28,7 +28,7 @@
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-brands-400.woff2" />
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-regular-400.woff2" />
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/sphinx-book-theme.css?v=eba8b062" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css?v=13237357" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
|
||||
@@ -246,6 +246,16 @@
|
||||
|
||||
|
||||
|
||||
<li class="toctree-l1"><a class="reference internal" href="exercisesweek42.html">Exercises week 42</a></li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
|
||||
<ul class="nav bd-sidenav">
|
||||
@@ -527,7 +537,7 @@ f_i =\sum_{j=0}^{n-1}a_{ij}x_j,
|
||||
<p>We calculate the optimal intercept by including a feature with the constant value of 1 in our model, which is then multplied by some parameter <span class="math notranslate nohighlight">\(\theta_0\)</span> from the OLS method into the optimal intercept value (which will be <span class="math notranslate nohighlight">\(\theta_0\)</span>). In practice, we include the intercept in our model by adding a column of ones to the start of our feature matrix.</p>
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span><span class="w"> </span><span class="nn">numpy</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">np</span>
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -556,7 +566,7 @@ f_i =\sum_{j=0}^{n-1}a_{ij}x_j,
|
||||
<p><strong>b)</strong> Use the expression from <strong>3d)</strong> to find the optimal parameters <span class="math notranslate nohighlight">\(\boldsymbol{\hat{\beta}_{OLS}}\)</span> for predicting spending based on these features. Create a function for this operation, as you are going to need to use it a lot.</p>
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="k">def</span><span class="w"> </span><span class="nf">OLS_parameters</span><span class="p">(</span><span class="n">X</span><span class="p">,</span> <span class="n">y</span><span class="p">):</span>
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">OLS_parameters</span><span class="p">(</span><span class="n">X</span><span class="p">,</span> <span class="n">y</span><span class="p">):</span>
|
||||
<span class="k">return</span> <span class="o">...</span>
|
||||
|
||||
<span class="c1">#beta = OLS_parameters(X, y)</span>
|
||||
@@ -581,7 +591,7 @@ f_i =\sum_{j=0}^{n-1}a_{ij}x_j,
|
||||
<p><strong>a)</strong> Create a feature matrix <span class="math notranslate nohighlight">\(\boldsymbol{X}\)</span> for the features <span class="math notranslate nohighlight">\(x, x^2, x^3, x^4, x^5\)</span>, including an intercept column of ones at the start. Make this into a function, as you will do this a lot over the next weeks.</p>
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="k">def</span><span class="w"> </span><span class="nf">polynomial_features</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">p</span><span class="p">):</span>
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">polynomial_features</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">p</span><span class="p">):</span>
|
||||
<span class="n">n</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">x</span><span class="p">)</span>
|
||||
<span class="n">X</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">zeros</span><span class="p">((</span><span class="n">n</span><span class="p">,</span> <span class="n">p</span> <span class="o">+</span> <span class="mi">1</span><span class="p">))</span>
|
||||
<span class="c1">#X[:, 0] = ...</span>
|
||||
@@ -605,7 +615,7 @@ f_i =\sum_{j=0}^{n-1}a_{ij}x_j,
|
||||
<p><strong>c)</strong> Like in exercise 4 last week, split your feature matrix and target data into a training split and test split.</p>
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span><span class="w"> </span><span class="nn">sklearn.model_selection</span><span class="w"> </span><span class="kn">import</span> <span class="n">train_test_split</span>
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">sklearn.model_selection</span> <span class="kn">import</span> <span class="n">train_test_split</span>
|
||||
|
||||
<span class="c1">#X_train, X_test, y_train, y_test = ...</span>
|
||||
</pre></div>
|
||||
|
||||
@@ -28,7 +28,7 @@
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-brands-400.woff2" />
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-regular-400.woff2" />
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/sphinx-book-theme.css?v=eba8b062" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css?v=13237357" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
|
||||
@@ -246,6 +246,16 @@
|
||||
|
||||
|
||||
|
||||
<li class="toctree-l1"><a class="reference internal" href="exercisesweek42.html">Exercises week 42</a></li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
|
||||
<ul class="nav bd-sidenav">
|
||||
@@ -479,10 +489,10 @@ defining a new cost function to be optimized, that is</p>
|
||||
<h2>Exercise 3 - Scaling data<a class="headerlink" href="#exercise-3-scaling-data" title="Link to this heading">#</a></h2>
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span><span class="w"> </span><span class="nn">numpy</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">np</span>
|
||||
<span class="kn">import</span><span class="w"> </span><span class="nn">matplotlib.pyplot</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">plt</span>
|
||||
<span class="kn">from</span><span class="w"> </span><span class="nn">sklearn.model_selection</span><span class="w"> </span><span class="kn">import</span> <span class="n">train_test_split</span>
|
||||
<span class="kn">from</span><span class="w"> </span><span class="nn">sklearn.preprocessing</span><span class="w"> </span><span class="kn">import</span> <span class="n">StandardScaler</span>
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
|
||||
<span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span>
|
||||
<span class="kn">from</span> <span class="nn">sklearn.model_selection</span> <span class="kn">import</span> <span class="n">train_test_split</span>
|
||||
<span class="kn">from</span> <span class="nn">sklearn.preprocessing</span> <span class="kn">import</span> <span class="n">StandardScaler</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -499,7 +509,7 @@ defining a new cost function to be optimized, that is</p>
|
||||
<p><strong>a)</strong> Adapt your function from last week to only include the intercept column if the boolean argument <code class="docutils literal notranslate"><span class="pre">intercept</span></code> is set to true.</p>
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="k">def</span><span class="w"> </span><span class="nf">polynomial_features</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">p</span><span class="p">,</span> <span class="n">intercept</span><span class="o">=</span><span class="kc">False</span><span class="p">):</span>
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">polynomial_features</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">p</span><span class="p">,</span> <span class="n">intercept</span><span class="o">=</span><span class="kc">False</span><span class="p">):</span>
|
||||
<span class="n">n</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">x</span><span class="p">)</span>
|
||||
<span class="n">X</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">zeros</span><span class="p">((</span><span class="n">n</span><span class="p">,</span> <span class="n">p</span> <span class="o">+</span> <span class="mi">1</span><span class="p">))</span>
|
||||
<span class="c1">#X[:, 0] = ...</span>
|
||||
@@ -512,7 +522,7 @@ defining a new cost function to be optimized, that is</p>
|
||||
</div>
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="k">def</span><span class="w"> </span><span class="nf">polynomial_features</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">p</span><span class="p">,</span> <span class="n">intercept</span><span class="o">=</span><span class="kc">False</span><span class="p">):</span>
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">polynomial_features</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">p</span><span class="p">,</span> <span class="n">intercept</span><span class="o">=</span><span class="kc">False</span><span class="p">):</span>
|
||||
<span class="n">n</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">x</span><span class="p">)</span>
|
||||
<span class="n">X</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">zeros</span><span class="p">((</span><span class="n">n</span><span class="p">,</span> <span class="n">p</span><span class="p">))</span>
|
||||
<span class="n">X</span><span class="p">[:,</span> <span class="mi">0</span><span class="p">]</span> <span class="o">=</span> <span class="n">x</span><span class="p">[:]</span>
|
||||
@@ -558,7 +568,7 @@ defining a new cost function to be optimized, that is</p>
|
||||
<p><strong>a)</strong> Implement a function for computing the optimal Ridge parameters using the expression from <strong>2a)</strong>.</p>
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="k">def</span><span class="w"> </span><span class="nf">Ridge_parameters</span><span class="p">(</span><span class="n">X</span><span class="p">,</span> <span class="n">y</span><span class="p">):</span>
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">Ridge_parameters</span><span class="p">(</span><span class="n">X</span><span class="p">,</span> <span class="n">y</span><span class="p">):</span>
|
||||
<span class="c1"># Assumes X is scaled and has no intercept column</span>
|
||||
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">linalg</span><span class="o">.</span><span class="n">inv</span><span class="p">(</span><span class="n">X</span><span class="o">.</span><span class="n">T</span> <span class="o">@</span> <span class="n">X</span><span class="p">)</span> <span class="o">@</span> <span class="n">X</span><span class="o">.</span><span class="n">T</span> <span class="o">@</span> <span class="n">y</span>
|
||||
|
||||
|
||||
@@ -28,7 +28,7 @@
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-brands-400.woff2" />
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-regular-400.woff2" />
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/sphinx-book-theme.css?v=eba8b062" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css?v=13237357" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
|
||||
@@ -246,6 +246,16 @@
|
||||
|
||||
|
||||
|
||||
<li class="toctree-l1"><a class="reference internal" href="exercisesweek42.html">Exercises week 42</a></li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
|
||||
<ul class="nav bd-sidenav">
|
||||
@@ -596,7 +606,7 @@ Then we compute the target values <span class="math notranslate nohighlight">\(y
|
||||
<p>Below is the code to generate the dataset:</p>
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span><span class="w"> </span><span class="nn">numpy</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">np</span>
|
||||
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
|
||||
|
||||
<span class="c1"># Set random seed for reproducibility</span>
|
||||
<span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">seed</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>
|
||||
|
||||
@@ -28,7 +28,7 @@
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-brands-400.woff2" />
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-regular-400.woff2" />
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/sphinx-book-theme.css?v=eba8b062" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css?v=13237357" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
|
||||
@@ -246,6 +246,16 @@
|
||||
|
||||
|
||||
|
||||
<li class="toctree-l1"><a class="reference internal" href="exercisesweek42.html">Exercises week 42</a></li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
|
||||
<ul class="nav bd-sidenav">
|
||||
@@ -537,7 +547,7 @@ C(\boldsymbol{X},\boldsymbol{\beta}) =\frac{1}{n}\sum_{i=0}^{n-1}(y_i-\tilde{y}_
|
||||
<p><strong>a)</strong> Using the expression above, compute the mean squared error, bias and variance of the given data. Check that the sum of the bias and variance correctly gives (approximately) the mean squared error.</p>
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span><span class="w"> </span><span class="nn">numpy</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">np</span>
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
|
||||
|
||||
<span class="n">n</span> <span class="o">=</span> <span class="mi">100</span>
|
||||
<span class="n">bootstraps</span> <span class="o">=</span> <span class="mi">1000</span>
|
||||
@@ -558,15 +568,15 @@ C(\boldsymbol{X},\boldsymbol{\beta}) =\frac{1}{n}\sum_{i=0}^{n-1}(y_i-\tilde{y}_
|
||||
<p><strong>d)</strong> Perform a bias-variance analysis of a polynomial OLS model fit to a one-dimensional function by computing and plotting the bias and variances values as a function of the polynomial degree of your model.</p>
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span><span class="w"> </span><span class="nn">numpy</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">np</span>
|
||||
<span class="kn">import</span><span class="w"> </span><span class="nn">matplotlib.pyplot</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">plt</span>
|
||||
<span class="kn">from</span><span class="w"> </span><span class="nn">sklearn.preprocessing</span><span class="w"> </span><span class="kn">import</span> <span class="p">(</span>
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
|
||||
<span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span>
|
||||
<span class="kn">from</span> <span class="nn">sklearn.preprocessing</span> <span class="kn">import</span> <span class="p">(</span>
|
||||
<span class="n">PolynomialFeatures</span><span class="p">,</span>
|
||||
<span class="p">)</span> <span class="c1"># use the fit_transform method of the created object!</span>
|
||||
<span class="kn">from</span><span class="w"> </span><span class="nn">sklearn.linear_model</span><span class="w"> </span><span class="kn">import</span> <span class="n">LinearRegression</span>
|
||||
<span class="kn">from</span><span class="w"> </span><span class="nn">sklearn.metrics</span><span class="w"> </span><span class="kn">import</span> <span class="n">mean_squared_error</span>
|
||||
<span class="kn">from</span><span class="w"> </span><span class="nn">sklearn.model_selection</span><span class="w"> </span><span class="kn">import</span> <span class="n">train_test_split</span>
|
||||
<span class="kn">from</span><span class="w"> </span><span class="nn">sklearn.utils</span><span class="w"> </span><span class="kn">import</span> <span class="n">resample</span>
|
||||
<span class="kn">from</span> <span class="nn">sklearn.linear_model</span> <span class="kn">import</span> <span class="n">LinearRegression</span>
|
||||
<span class="kn">from</span> <span class="nn">sklearn.metrics</span> <span class="kn">import</span> <span class="n">mean_squared_error</span>
|
||||
<span class="kn">from</span> <span class="nn">sklearn.model_selection</span> <span class="kn">import</span> <span class="n">train_test_split</span>
|
||||
<span class="kn">from</span> <span class="nn">sklearn.utils</span> <span class="kn">import</span> <span class="n">resample</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@@ -28,7 +28,7 @@
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-brands-400.woff2" />
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-regular-400.woff2" />
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/sphinx-book-theme.css?v=eba8b062" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css?v=13237357" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
|
||||
@@ -244,6 +244,16 @@
|
||||
|
||||
|
||||
|
||||
<li class="toctree-l1"><a class="reference internal" href="exercisesweek42.html">Exercises week 42</a></li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
|
||||
<ul class="nav bd-sidenav">
|
||||
|
||||
@@ -28,7 +28,7 @@
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-brands-400.woff2" />
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-regular-400.woff2" />
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/sphinx-book-theme.css?v=eba8b062" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css?v=13237357" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
|
||||
@@ -62,7 +62,7 @@
|
||||
<script>DOCUMENTATION_OPTIONS.pagename = 'exercisesweek41';</script>
|
||||
<link rel="index" title="Index" href="genindex.html" />
|
||||
<link rel="search" title="Search" href="search.html" />
|
||||
<link rel="next" title="Project 1 on Machine Learning, deadline October 6 (midnight), 2025" href="project1.html" />
|
||||
<link rel="next" title="Exercises week 42" href="exercisesweek42.html" />
|
||||
<link rel="prev" title="Week 41 Neural networks and constructing a neural network code" href="week41.html" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1"/>
|
||||
<meta name="docsearch:language" content="en"/>
|
||||
@@ -246,6 +246,16 @@
|
||||
|
||||
|
||||
|
||||
<li class="toctree-l1"><a class="reference internal" href="exercisesweek42.html">Exercises week 42</a></li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
|
||||
<ul class="nav bd-sidenav">
|
||||
@@ -435,29 +445,29 @@ doconce format html exercisesweek41.do.txt -->
|
||||
<p>First, here are some functions you are going to need, don’t change this cell. If you are unable to import autograd, just swap in normal numpy until you want to do the final optional exercise.</p>
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span><span class="w"> </span><span class="nn">autograd.numpy</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">np</span> <span class="c1"># We need to use this numpy wrapper to make automatic differentiation work later</span>
|
||||
<span class="kn">from</span><span class="w"> </span><span class="nn">sklearn</span><span class="w"> </span><span class="kn">import</span> <span class="n">datasets</span>
|
||||
<span class="kn">import</span><span class="w"> </span><span class="nn">matplotlib.pyplot</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">plt</span>
|
||||
<span class="kn">from</span><span class="w"> </span><span class="nn">sklearn.metrics</span><span class="w"> </span><span class="kn">import</span> <span class="n">accuracy_score</span>
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">autograd.numpy</span> <span class="k">as</span> <span class="nn">np</span> <span class="c1"># We need to use this numpy wrapper to make automatic differentiation work later</span>
|
||||
<span class="kn">from</span> <span class="nn">sklearn</span> <span class="kn">import</span> <span class="n">datasets</span>
|
||||
<span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span>
|
||||
<span class="kn">from</span> <span class="nn">sklearn.metrics</span> <span class="kn">import</span> <span class="n">accuracy_score</span>
|
||||
|
||||
|
||||
<span class="c1"># Defining some activation functions</span>
|
||||
<span class="k">def</span><span class="w"> </span><span class="nf">ReLU</span><span class="p">(</span><span class="n">z</span><span class="p">):</span>
|
||||
<span class="k">def</span> <span class="nf">ReLU</span><span class="p">(</span><span class="n">z</span><span class="p">):</span>
|
||||
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">z</span> <span class="o">></span> <span class="mi">0</span><span class="p">,</span> <span class="n">z</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span>
|
||||
|
||||
|
||||
<span class="k">def</span><span class="w"> </span><span class="nf">sigmoid</span><span class="p">(</span><span class="n">z</span><span class="p">):</span>
|
||||
<span class="k">def</span> <span class="nf">sigmoid</span><span class="p">(</span><span class="n">z</span><span class="p">):</span>
|
||||
<span class="k">return</span> <span class="mi">1</span> <span class="o">/</span> <span class="p">(</span><span class="mi">1</span> <span class="o">+</span> <span class="n">np</span><span class="o">.</span><span class="n">exp</span><span class="p">(</span><span class="o">-</span><span class="n">z</span><span class="p">))</span>
|
||||
|
||||
|
||||
<span class="k">def</span><span class="w"> </span><span class="nf">softmax</span><span class="p">(</span><span class="n">z</span><span class="p">):</span>
|
||||
<span class="k">def</span> <span class="nf">softmax</span><span class="p">(</span><span class="n">z</span><span class="p">):</span>
|
||||
<span class="w"> </span><span class="sd">"""Compute softmax values for each set of scores in the rows of the matrix z.</span>
|
||||
<span class="sd"> Used with batched input data."""</span>
|
||||
<span class="n">e_z</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">exp</span><span class="p">(</span><span class="n">z</span> <span class="o">-</span> <span class="n">np</span><span class="o">.</span><span class="n">max</span><span class="p">(</span><span class="n">z</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">0</span><span class="p">))</span>
|
||||
<span class="k">return</span> <span class="n">e_z</span> <span class="o">/</span> <span class="n">np</span><span class="o">.</span><span class="n">sum</span><span class="p">(</span><span class="n">e_z</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">)[:,</span> <span class="n">np</span><span class="o">.</span><span class="n">newaxis</span><span class="p">]</span>
|
||||
|
||||
|
||||
<span class="k">def</span><span class="w"> </span><span class="nf">softmax_vec</span><span class="p">(</span><span class="n">z</span><span class="p">):</span>
|
||||
<span class="k">def</span> <span class="nf">softmax_vec</span><span class="p">(</span><span class="n">z</span><span class="p">):</span>
|
||||
<span class="w"> </span><span class="sd">"""Compute softmax values for each set of scores in the vector z.</span>
|
||||
<span class="sd"> Use this function when you use the activation function on one vector at a time"""</span>
|
||||
<span class="n">e_z</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">exp</span><span class="p">(</span><span class="n">z</span> <span class="o">-</span> <span class="n">np</span><span class="o">.</span><span class="n">max</span><span class="p">(</span><span class="n">z</span><span class="p">))</span>
|
||||
@@ -555,7 +565,7 @@ doconce format html exercisesweek41.do.txt -->
|
||||
<p><strong>a)</strong> Complete the function below so that it returns a list <code class="docutils literal notranslate"><span class="pre">layers</span></code> of weight and bias tuples <code class="docutils literal notranslate"><span class="pre">(W,</span> <span class="pre">b)</span></code> for each layer, in order, with the correct shapes that we can use later as our network parameters.</p>
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="k">def</span><span class="w"> </span><span class="nf">create_layers</span><span class="p">(</span><span class="n">network_input_size</span><span class="p">,</span> <span class="n">layer_output_sizes</span><span class="p">):</span>
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">create_layers</span><span class="p">(</span><span class="n">network_input_size</span><span class="p">,</span> <span class="n">layer_output_sizes</span><span class="p">):</span>
|
||||
<span class="n">layers</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
|
||||
<span class="n">i_size</span> <span class="o">=</span> <span class="n">network_input_size</span>
|
||||
@@ -573,7 +583,7 @@ doconce format html exercisesweek41.do.txt -->
|
||||
<p><strong>b)</strong> Comple the function below so that it evaluates the intermediary <code class="docutils literal notranslate"><span class="pre">z</span></code> and activation <code class="docutils literal notranslate"><span class="pre">a</span></code> for each layer, with ReLU actication, and returns the final activation <code class="docutils literal notranslate"><span class="pre">a</span></code>. This is the complete feed-forward pass, a full neural network!</p>
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="k">def</span><span class="w"> </span><span class="nf">feed_forward_all_relu</span><span class="p">(</span><span class="n">layers</span><span class="p">,</span> <span class="nb">input</span><span class="p">):</span>
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">feed_forward_all_relu</span><span class="p">(</span><span class="n">layers</span><span class="p">,</span> <span class="nb">input</span><span class="p">):</span>
|
||||
<span class="n">a</span> <span class="o">=</span> <span class="nb">input</span>
|
||||
<span class="k">for</span> <span class="n">W</span><span class="p">,</span> <span class="n">b</span> <span class="ow">in</span> <span class="n">layers</span><span class="p">:</span>
|
||||
<span class="n">z</span> <span class="o">=</span> <span class="o">...</span>
|
||||
@@ -605,7 +615,7 @@ doconce format html exercisesweek41.do.txt -->
|
||||
<p><strong>a)</strong> Complete the <code class="docutils literal notranslate"><span class="pre">feed_forward</span></code> function which accepts a list of activation functions as an argument, and which evaluates these activation functions at each layer.</p>
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="k">def</span><span class="w"> </span><span class="nf">feed_forward</span><span class="p">(</span><span class="nb">input</span><span class="p">,</span> <span class="n">layers</span><span class="p">,</span> <span class="n">activation_funcs</span><span class="p">):</span>
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">feed_forward</span><span class="p">(</span><span class="nb">input</span><span class="p">,</span> <span class="n">layers</span><span class="p">,</span> <span class="n">activation_funcs</span><span class="p">):</span>
|
||||
<span class="n">a</span> <span class="o">=</span> <span class="nb">input</span>
|
||||
<span class="k">for</span> <span class="p">(</span><span class="n">W</span><span class="p">,</span> <span class="n">b</span><span class="p">),</span> <span class="n">activation_func</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">layers</span><span class="p">,</span> <span class="n">activation_funcs</span><span class="p">):</span>
|
||||
<span class="n">z</span> <span class="o">=</span> <span class="o">...</span>
|
||||
@@ -639,7 +649,7 @@ doconce format html exercisesweek41.do.txt -->
|
||||
<p><strong>a)</strong> Complete the function <code class="docutils literal notranslate"><span class="pre">create_layers_batch</span></code> so that the weight matrix is the transpose of what it was when you only sent in one input at a time.</p>
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="k">def</span><span class="w"> </span><span class="nf">create_layers_batch</span><span class="p">(</span><span class="n">network_input_size</span><span class="p">,</span> <span class="n">layer_output_sizes</span><span class="p">):</span>
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">create_layers_batch</span><span class="p">(</span><span class="n">network_input_size</span><span class="p">,</span> <span class="n">layer_output_sizes</span><span class="p">):</span>
|
||||
<span class="n">layers</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
|
||||
<span class="n">i_size</span> <span class="o">=</span> <span class="n">network_input_size</span>
|
||||
@@ -660,7 +670,7 @@ doconce format html exercisesweek41.do.txt -->
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="n">inputs</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">rand</span><span class="p">(</span><span class="mi">1000</span><span class="p">,</span> <span class="mi">4</span><span class="p">)</span>
|
||||
|
||||
|
||||
<span class="k">def</span><span class="w"> </span><span class="nf">feed_forward_batch</span><span class="p">(</span><span class="n">inputs</span><span class="p">,</span> <span class="n">layers</span><span class="p">,</span> <span class="n">activation_funcs</span><span class="p">):</span>
|
||||
<span class="k">def</span> <span class="nf">feed_forward_batch</span><span class="p">(</span><span class="n">inputs</span><span class="p">,</span> <span class="n">layers</span><span class="p">,</span> <span class="n">activation_funcs</span><span class="p">):</span>
|
||||
<span class="n">a</span> <span class="o">=</span> <span class="n">inputs</span>
|
||||
<span class="k">for</span> <span class="p">(</span><span class="n">W</span><span class="p">,</span> <span class="n">b</span><span class="p">),</span> <span class="n">activation_func</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">layers</span><span class="p">,</span> <span class="n">activation_funcs</span><span class="p">):</span>
|
||||
<span class="n">z</span> <span class="o">=</span> <span class="o">...</span>
|
||||
@@ -715,7 +725,7 @@ doconce format html exercisesweek41.do.txt -->
|
||||
<span class="n">targets</span><span class="p">[</span><span class="n">i</span><span class="p">,</span> <span class="n">t</span><span class="p">]</span> <span class="o">=</span> <span class="mi">1</span>
|
||||
|
||||
|
||||
<span class="k">def</span><span class="w"> </span><span class="nf">accuracy</span><span class="p">(</span><span class="n">predictions</span><span class="p">,</span> <span class="n">targets</span><span class="p">):</span>
|
||||
<span class="k">def</span> <span class="nf">accuracy</span><span class="p">(</span><span class="n">predictions</span><span class="p">,</span> <span class="n">targets</span><span class="p">):</span>
|
||||
<span class="n">one_hot_predictions</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">zeros</span><span class="p">(</span><span class="n">predictions</span><span class="o">.</span><span class="n">shape</span><span class="p">)</span>
|
||||
|
||||
<span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">prediction</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">predictions</span><span class="p">):</span>
|
||||
@@ -758,11 +768,11 @@ doconce format html exercisesweek41.do.txt -->
|
||||
<p>Since we are doing a classification task with multiple output classes, we use the cross-entropy loss function, which can evaluate performance on classification tasks. It sees if your prediction is “most certain” on the correct target.</p>
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="k">def</span><span class="w"> </span><span class="nf">cross_entropy</span><span class="p">(</span><span class="n">predict</span><span class="p">,</span> <span class="n">target</span><span class="p">):</span>
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">cross_entropy</span><span class="p">(</span><span class="n">predict</span><span class="p">,</span> <span class="n">target</span><span class="p">):</span>
|
||||
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">sum</span><span class="p">(</span><span class="o">-</span><span class="n">target</span> <span class="o">*</span> <span class="n">np</span><span class="o">.</span><span class="n">log</span><span class="p">(</span><span class="n">predict</span><span class="p">))</span>
|
||||
|
||||
|
||||
<span class="k">def</span><span class="w"> </span><span class="nf">cost</span><span class="p">(</span><span class="nb">input</span><span class="p">,</span> <span class="n">layers</span><span class="p">,</span> <span class="n">activation_funcs</span><span class="p">,</span> <span class="n">target</span><span class="p">):</span>
|
||||
<span class="k">def</span> <span class="nf">cost</span><span class="p">(</span><span class="nb">input</span><span class="p">,</span> <span class="n">layers</span><span class="p">,</span> <span class="n">activation_funcs</span><span class="p">,</span> <span class="n">target</span><span class="p">):</span>
|
||||
<span class="n">predict</span> <span class="o">=</span> <span class="n">feed_forward_batch</span><span class="p">(</span><span class="nb">input</span><span class="p">,</span> <span class="n">layers</span><span class="p">,</span> <span class="n">activation_funcs</span><span class="p">)</span>
|
||||
<span class="k">return</span> <span class="n">cross_entropy</span><span class="p">(</span><span class="n">predict</span><span class="p">,</span> <span class="n">target</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
@@ -777,7 +787,7 @@ doconce format html exercisesweek41.do.txt -->
|
||||
<p>Now we need to compute these gradients. This is pretty hard to do for a neural network, we will use most of next week to do this, but we can also use autograd to just do it for us, which is what we always do in practice. With the code cell below, we create a function which takes all of these gradients for us.</p>
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span><span class="w"> </span><span class="nn">autograd</span><span class="w"> </span><span class="kn">import</span> <span class="n">grad</span>
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">autograd</span> <span class="kn">import</span> <span class="n">grad</span>
|
||||
|
||||
|
||||
<span class="n">gradient_func</span> <span class="o">=</span> <span class="n">grad</span><span class="p">(</span>
|
||||
@@ -801,7 +811,7 @@ doconce format html exercisesweek41.do.txt -->
|
||||
<p><strong>c)</strong> Finish the <code class="docutils literal notranslate"><span class="pre">train_network</span></code> function.</p>
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="k">def</span><span class="w"> </span><span class="nf">train_network</span><span class="p">(</span>
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">train_network</span><span class="p">(</span>
|
||||
<span class="n">inputs</span><span class="p">,</span> <span class="n">layers</span><span class="p">,</span> <span class="n">activation_funcs</span><span class="p">,</span> <span class="n">targets</span><span class="p">,</span> <span class="n">learning_rate</span><span class="o">=</span><span class="mf">0.001</span><span class="p">,</span> <span class="n">epochs</span><span class="o">=</span><span class="mi">100</span>
|
||||
<span class="p">):</span>
|
||||
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">epochs</span><span class="p">):</span>
|
||||
@@ -865,11 +875,11 @@ doconce format html exercisesweek41.do.txt -->
|
||||
</div>
|
||||
</a>
|
||||
<a class="right-next"
|
||||
href="project1.html"
|
||||
href="exercisesweek42.html"
|
||||
title="next page">
|
||||
<div class="prev-next-info">
|
||||
<p class="prev-next-subtitle">next</p>
|
||||
<p class="prev-next-title">Project 1 on Machine Learning, deadline October 6 (midnight), 2025</p>
|
||||
<p class="prev-next-title">Exercises week 42</p>
|
||||
</div>
|
||||
<i class="fa-solid fa-angle-right"></i>
|
||||
</a>
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -27,7 +27,7 @@
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-brands-400.woff2" />
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-regular-400.woff2" />
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/sphinx-book-theme.css?v=eba8b062" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css?v=13237357" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
|
||||
@@ -243,6 +243,16 @@
|
||||
|
||||
|
||||
|
||||
<li class="toctree-l1"><a class="reference internal" href="exercisesweek42.html">Exercises week 42</a></li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
|
||||
<ul class="nav bd-sidenav">
|
||||
|
||||
@@ -28,7 +28,7 @@
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-brands-400.woff2" />
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-regular-400.woff2" />
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/sphinx-book-theme.css?v=eba8b062" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css?v=13237357" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
|
||||
@@ -247,6 +247,16 @@
|
||||
|
||||
|
||||
|
||||
<li class="toctree-l1"><a class="reference internal" href="exercisesweek42.html">Exercises week 42</a></li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
|
||||
<ul class="nav bd-sidenav">
|
||||
|
||||
@@ -28,7 +28,7 @@
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-brands-400.woff2" />
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-regular-400.woff2" />
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/sphinx-book-theme.css?v=eba8b062" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css?v=13237357" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
|
||||
@@ -246,6 +246,16 @@
|
||||
|
||||
|
||||
|
||||
<li class="toctree-l1"><a class="reference internal" href="exercisesweek42.html">Exercises week 42</a></li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
|
||||
<ul class="nav bd-sidenav">
|
||||
|
||||
Binary file not shown.
@@ -28,7 +28,7 @@
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-brands-400.woff2" />
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-regular-400.woff2" />
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/sphinx-book-theme.css?v=eba8b062" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css?v=13237357" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
|
||||
@@ -62,7 +62,7 @@
|
||||
<script>DOCUMENTATION_OPTIONS.pagename = 'project1';</script>
|
||||
<link rel="index" title="Index" href="genindex.html" />
|
||||
<link rel="search" title="Search" href="search.html" />
|
||||
<link rel="prev" title="Exercises week 41" href="exercisesweek41.html" />
|
||||
<link rel="prev" title="Exercises week 42" href="exercisesweek42.html" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1"/>
|
||||
<meta name="docsearch:language" content="en"/>
|
||||
</head>
|
||||
@@ -245,6 +245,16 @@
|
||||
|
||||
|
||||
|
||||
<li class="toctree-l1"><a class="reference internal" href="exercisesweek42.html">Exercises week 42</a></li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
|
||||
<ul class="current nav bd-sidenav">
|
||||
@@ -795,12 +805,12 @@ of code developers and contributors keeps increasing.</p>
|
||||
|
||||
<div class="prev-next-area">
|
||||
<a class="left-prev"
|
||||
href="exercisesweek41.html"
|
||||
href="exercisesweek42.html"
|
||||
title="previous page">
|
||||
<i class="fa-solid fa-angle-left"></i>
|
||||
<div class="prev-next-info">
|
||||
<p class="prev-next-subtitle">previous</p>
|
||||
<p class="prev-next-title">Exercises week 41</p>
|
||||
<p class="prev-next-title">Exercises week 42</p>
|
||||
</div>
|
||||
</a>
|
||||
</div>
|
||||
|
||||
@@ -28,7 +28,7 @@
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-brands-400.woff2" />
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-regular-400.woff2" />
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/sphinx-book-theme.css?v=eba8b062" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css?v=13237357" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
|
||||
@@ -244,6 +244,16 @@
|
||||
|
||||
|
||||
|
||||
<li class="toctree-l1"><a class="reference internal" href="exercisesweek42.html">Exercises week 42</a></li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
|
||||
<ul class="nav bd-sidenav">
|
||||
|
||||
@@ -26,7 +26,7 @@
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-brands-400.woff2" />
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-regular-400.woff2" />
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/sphinx-book-theme.css?v=eba8b062" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css?v=13237357" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
|
||||
@@ -245,6 +245,16 @@
|
||||
|
||||
|
||||
|
||||
<li class="toctree-l1"><a class="reference internal" href="exercisesweek42.html">Exercises week 42</a></li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
|
||||
<ul class="nav bd-sidenav">
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -28,7 +28,7 @@
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-brands-400.woff2" />
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-regular-400.woff2" />
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/sphinx-book-theme.css?v=eba8b062" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css?v=13237357" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
|
||||
@@ -246,6 +246,16 @@
|
||||
|
||||
|
||||
|
||||
<li class="toctree-l1"><a class="reference internal" href="exercisesweek42.html">Exercises week 42</a></li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
|
||||
<ul class="nav bd-sidenav">
|
||||
|
||||
@@ -28,7 +28,7 @@
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-brands-400.woff2" />
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-regular-400.woff2" />
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/sphinx-book-theme.css?v=eba8b062" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css?v=13237357" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
|
||||
@@ -244,6 +244,16 @@
|
||||
|
||||
|
||||
|
||||
<li class="toctree-l1"><a class="reference internal" href="exercisesweek42.html">Exercises week 42</a></li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
|
||||
<ul class="nav bd-sidenav">
|
||||
|
||||
@@ -28,7 +28,7 @@
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-brands-400.woff2" />
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-regular-400.woff2" />
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/sphinx-book-theme.css?v=eba8b062" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css?v=13237357" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
|
||||
@@ -244,6 +244,16 @@
|
||||
|
||||
|
||||
|
||||
<li class="toctree-l1"><a class="reference internal" href="exercisesweek42.html">Exercises week 42</a></li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
|
||||
<ul class="nav bd-sidenav">
|
||||
|
||||
@@ -28,7 +28,7 @@
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-brands-400.woff2" />
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-regular-400.woff2" />
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/sphinx-book-theme.css?v=eba8b062" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css?v=13237357" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
|
||||
@@ -246,6 +246,16 @@
|
||||
|
||||
|
||||
|
||||
<li class="toctree-l1"><a class="reference internal" href="exercisesweek42.html">Exercises week 42</a></li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
|
||||
<ul class="nav bd-sidenav">
|
||||
|
||||
@@ -28,7 +28,7 @@
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-brands-400.woff2" />
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-regular-400.woff2" />
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/sphinx-book-theme.css?v=eba8b062" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css?v=13237357" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
|
||||
@@ -246,6 +246,16 @@
|
||||
|
||||
|
||||
|
||||
<li class="toctree-l1"><a class="reference internal" href="exercisesweek42.html">Exercises week 42</a></li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
|
||||
<ul class="nav bd-sidenav">
|
||||
@@ -907,7 +917,7 @@ We assume our data can represented by a fourth-order polynomial. For the <span c
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="c1"># matrix inversion to find theta</span>
|
||||
<span class="c1"># First we set up the data</span>
|
||||
<span class="kn">import</span><span class="w"> </span><span class="nn">numpy</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">np</span>
|
||||
<span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
|
||||
<span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">rand</span><span class="p">(</span><span class="mi">100</span><span class="p">)</span>
|
||||
<span class="n">y</span> <span class="o">=</span> <span class="mf">2.0</span><span class="o">+</span><span class="mi">5</span><span class="o">*</span><span class="n">x</span><span class="o">*</span><span class="n">x</span><span class="o">+</span><span class="mf">0.1</span><span class="o">*</span><span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">randn</span><span class="p">(</span><span class="mi">100</span><span class="p">)</span>
|
||||
<span class="c1"># and then the design matrix X including the intercept</span>
|
||||
@@ -941,7 +951,7 @@ We assume our data can represented by a fourth-order polynomial. For the <span c
|
||||
Since we are not using <strong>Scikit-Learn</strong> here we can define our own <span class="math notranslate nohighlight">\(R2\)</span> function as</p>
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="k">def</span><span class="w"> </span><span class="nf">R2</span><span class="p">(</span><span class="n">y_data</span><span class="p">,</span> <span class="n">y_model</span><span class="p">):</span>
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">R2</span><span class="p">(</span><span class="n">y_data</span><span class="p">,</span> <span class="n">y_model</span><span class="p">):</span>
|
||||
<span class="k">return</span> <span class="mi">1</span> <span class="o">-</span> <span class="n">np</span><span class="o">.</span><span class="n">sum</span><span class="p">((</span><span class="n">y_data</span> <span class="o">-</span> <span class="n">y_model</span><span class="p">)</span> <span class="o">**</span> <span class="mi">2</span><span class="p">)</span> <span class="o">/</span> <span class="n">np</span><span class="o">.</span><span class="n">sum</span><span class="p">((</span><span class="n">y_data</span> <span class="o">-</span> <span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">y_data</span><span class="p">))</span> <span class="o">**</span> <span class="mi">2</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -958,7 +968,7 @@ Since we are not using <strong>Scikit-Learn</strong> here we can define our own
|
||||
<p>We can easily add our <strong>MSE</strong> score as</p>
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="k">def</span><span class="w"> </span><span class="nf">MSE</span><span class="p">(</span><span class="n">y_data</span><span class="p">,</span><span class="n">y_model</span><span class="p">):</span>
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">MSE</span><span class="p">(</span><span class="n">y_data</span><span class="p">,</span><span class="n">y_model</span><span class="p">):</span>
|
||||
<span class="n">n</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">size</span><span class="p">(</span><span class="n">y_model</span><span class="p">)</span>
|
||||
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">sum</span><span class="p">((</span><span class="n">y_data</span><span class="o">-</span><span class="n">y_model</span><span class="p">)</span><span class="o">**</span><span class="mi">2</span><span class="p">)</span><span class="o">/</span><span class="n">n</span>
|
||||
|
||||
@@ -970,7 +980,7 @@ Since we are not using <strong>Scikit-Learn</strong> here we can define our own
|
||||
<p>and finally the relative error as</p>
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="k">def</span><span class="w"> </span><span class="nf">RelativeError</span><span class="p">(</span><span class="n">y_data</span><span class="p">,</span><span class="n">y_model</span><span class="p">):</span>
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">RelativeError</span><span class="p">(</span><span class="n">y_data</span><span class="p">,</span><span class="n">y_model</span><span class="p">):</span>
|
||||
<span class="k">return</span> <span class="nb">abs</span><span class="p">((</span><span class="n">y_data</span><span class="o">-</span><span class="n">y_model</span><span class="p">)</span><span class="o">/</span><span class="n">y_data</span><span class="p">)</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">RelativeError</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="n">ytilde</span><span class="p">))</span>
|
||||
</pre></div>
|
||||
@@ -997,16 +1007,16 @@ but now splitting the data into a training set and a test set.</p>
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="o">%</span><span class="k">matplotlib</span> inline
|
||||
|
||||
<span class="kn">import</span><span class="w"> </span><span class="nn">os</span>
|
||||
<span class="kn">import</span><span class="w"> </span><span class="nn">numpy</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">np</span>
|
||||
<span class="kn">import</span><span class="w"> </span><span class="nn">pandas</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">pd</span>
|
||||
<span class="kn">import</span><span class="w"> </span><span class="nn">matplotlib.pyplot</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">plt</span>
|
||||
<span class="kn">from</span><span class="w"> </span><span class="nn">sklearn.model_selection</span><span class="w"> </span><span class="kn">import</span> <span class="n">train_test_split</span>
|
||||
<span class="kn">import</span> <span class="nn">os</span>
|
||||
<span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
|
||||
<span class="kn">import</span> <span class="nn">pandas</span> <span class="k">as</span> <span class="nn">pd</span>
|
||||
<span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span>
|
||||
<span class="kn">from</span> <span class="nn">sklearn.model_selection</span> <span class="kn">import</span> <span class="n">train_test_split</span>
|
||||
|
||||
|
||||
<span class="k">def</span><span class="w"> </span><span class="nf">R2</span><span class="p">(</span><span class="n">y_data</span><span class="p">,</span> <span class="n">y_model</span><span class="p">):</span>
|
||||
<span class="k">def</span> <span class="nf">R2</span><span class="p">(</span><span class="n">y_data</span><span class="p">,</span> <span class="n">y_model</span><span class="p">):</span>
|
||||
<span class="k">return</span> <span class="mi">1</span> <span class="o">-</span> <span class="n">np</span><span class="o">.</span><span class="n">sum</span><span class="p">((</span><span class="n">y_data</span> <span class="o">-</span> <span class="n">y_model</span><span class="p">)</span> <span class="o">**</span> <span class="mi">2</span><span class="p">)</span> <span class="o">/</span> <span class="n">np</span><span class="o">.</span><span class="n">sum</span><span class="p">((</span><span class="n">y_data</span> <span class="o">-</span> <span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">y_data</span><span class="p">))</span> <span class="o">**</span> <span class="mi">2</span><span class="p">)</span>
|
||||
<span class="k">def</span><span class="w"> </span><span class="nf">MSE</span><span class="p">(</span><span class="n">y_data</span><span class="p">,</span><span class="n">y_model</span><span class="p">):</span>
|
||||
<span class="k">def</span> <span class="nf">MSE</span><span class="p">(</span><span class="n">y_data</span><span class="p">,</span><span class="n">y_model</span><span class="p">):</span>
|
||||
<span class="n">n</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">size</span><span class="p">(</span><span class="n">y_model</span><span class="p">)</span>
|
||||
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">sum</span><span class="p">((</span><span class="n">y_data</span><span class="o">-</span><span class="n">y_model</span><span class="p">)</span><span class="o">**</span><span class="mi">2</span><span class="p">)</span><span class="o">/</span><span class="n">n</span>
|
||||
|
||||
@@ -1047,7 +1057,7 @@ but now splitting the data into a training set and a test set.</p>
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="c1"># equivalently in numpy</span>
|
||||
<span class="k">def</span><span class="w"> </span><span class="nf">train_test_split_numpy</span><span class="p">(</span><span class="n">inputs</span><span class="p">,</span> <span class="n">labels</span><span class="p">,</span> <span class="n">train_size</span><span class="p">,</span> <span class="n">test_size</span><span class="p">):</span>
|
||||
<span class="k">def</span> <span class="nf">train_test_split_numpy</span><span class="p">(</span><span class="n">inputs</span><span class="p">,</span> <span class="n">labels</span><span class="p">,</span> <span class="n">train_size</span><span class="p">,</span> <span class="n">test_size</span><span class="p">):</span>
|
||||
<span class="n">n_inputs</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">inputs</span><span class="p">)</span>
|
||||
<span class="n">inputs_shuffled</span> <span class="o">=</span> <span class="n">inputs</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>
|
||||
<span class="n">labels_shuffled</span> <span class="o">=</span> <span class="n">labels</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>
|
||||
@@ -1152,13 +1162,13 @@ simple test design matrix with random numbers. Each column could then
|
||||
represent a specific feature whose mean value is subracted.</p>
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span><span class="w"> </span><span class="nn">sklearn.linear_model</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">skl</span>
|
||||
<span class="kn">from</span><span class="w"> </span><span class="nn">sklearn.metrics</span><span class="w"> </span><span class="kn">import</span> <span class="n">mean_squared_error</span>
|
||||
<span class="kn">from</span><span class="w"> </span><span class="nn">sklearn.model_selection</span><span class="w"> </span><span class="kn">import</span> <span class="n">train_test_split</span>
|
||||
<span class="kn">from</span><span class="w"> </span><span class="nn">sklearn.preprocessing</span><span class="w"> </span><span class="kn">import</span> <span class="n">MinMaxScaler</span><span class="p">,</span> <span class="n">StandardScaler</span><span class="p">,</span> <span class="n">Normalizer</span>
|
||||
<span class="kn">import</span><span class="w"> </span><span class="nn">numpy</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">np</span>
|
||||
<span class="kn">import</span><span class="w"> </span><span class="nn">pandas</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">pd</span>
|
||||
<span class="kn">from</span><span class="w"> </span><span class="nn">IPython.display</span><span class="w"> </span><span class="kn">import</span> <span class="n">display</span>
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">sklearn.linear_model</span> <span class="k">as</span> <span class="nn">skl</span>
|
||||
<span class="kn">from</span> <span class="nn">sklearn.metrics</span> <span class="kn">import</span> <span class="n">mean_squared_error</span>
|
||||
<span class="kn">from</span> <span class="nn">sklearn.model_selection</span> <span class="kn">import</span> <span class="n">train_test_split</span>
|
||||
<span class="kn">from</span> <span class="nn">sklearn.preprocessing</span> <span class="kn">import</span> <span class="n">MinMaxScaler</span><span class="p">,</span> <span class="n">StandardScaler</span><span class="p">,</span> <span class="n">Normalizer</span>
|
||||
<span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
|
||||
<span class="kn">import</span> <span class="nn">pandas</span> <span class="k">as</span> <span class="nn">pd</span>
|
||||
<span class="kn">from</span> <span class="nn">IPython.display</span> <span class="kn">import</span> <span class="n">display</span>
|
||||
<span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">seed</span><span class="p">(</span><span class="mi">100</span><span class="p">)</span>
|
||||
<span class="c1"># setting up a 10 x 5 matrix</span>
|
||||
<span class="n">rows</span> <span class="o">=</span> <span class="mi">10</span>
|
||||
@@ -1214,12 +1224,12 @@ the aims is to reproduce Figure 2.11 of <a class="reference external" href="http
|
||||
<p>Write a first code which sets up a design matrix <span class="math notranslate nohighlight">\(X\)</span> defined by a fourth-order polynomial. Scale your data and split it in training and test data.</p>
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span><span class="w"> </span><span class="nn">matplotlib.pyplot</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">plt</span>
|
||||
<span class="kn">import</span><span class="w"> </span><span class="nn">numpy</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">np</span>
|
||||
<span class="kn">from</span><span class="w"> </span><span class="nn">sklearn.linear_model</span><span class="w"> </span><span class="kn">import</span> <span class="n">LinearRegression</span>
|
||||
<span class="kn">from</span><span class="w"> </span><span class="nn">sklearn.preprocessing</span><span class="w"> </span><span class="kn">import</span> <span class="n">PolynomialFeatures</span>
|
||||
<span class="kn">from</span><span class="w"> </span><span class="nn">sklearn.model_selection</span><span class="w"> </span><span class="kn">import</span> <span class="n">train_test_split</span>
|
||||
<span class="kn">from</span><span class="w"> </span><span class="nn">sklearn.pipeline</span><span class="w"> </span><span class="kn">import</span> <span class="n">make_pipeline</span>
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span>
|
||||
<span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
|
||||
<span class="kn">from</span> <span class="nn">sklearn.linear_model</span> <span class="kn">import</span> <span class="n">LinearRegression</span>
|
||||
<span class="kn">from</span> <span class="nn">sklearn.preprocessing</span> <span class="kn">import</span> <span class="n">PolynomialFeatures</span>
|
||||
<span class="kn">from</span> <span class="nn">sklearn.model_selection</span> <span class="kn">import</span> <span class="n">train_test_split</span>
|
||||
<span class="kn">from</span> <span class="nn">sklearn.pipeline</span> <span class="kn">import</span> <span class="n">make_pipeline</span>
|
||||
|
||||
|
||||
<span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">seed</span><span class="p">(</span><span class="mi">2018</span><span class="p">)</span>
|
||||
@@ -1499,13 +1509,13 @@ discussion of Ridge regression.</p>
|
||||
<p>The code here is a simple demonstration of how to implement Ridge regression with our own code and compare this with scikit-learn.</p>
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span><span class="w"> </span><span class="nn">numpy</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">np</span>
|
||||
<span class="kn">import</span><span class="w"> </span><span class="nn">pandas</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">pd</span>
|
||||
<span class="kn">import</span><span class="w"> </span><span class="nn">matplotlib.pyplot</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">plt</span>
|
||||
<span class="kn">from</span><span class="w"> </span><span class="nn">sklearn.model_selection</span><span class="w"> </span><span class="kn">import</span> <span class="n">train_test_split</span>
|
||||
<span class="kn">from</span><span class="w"> </span><span class="nn">sklearn</span><span class="w"> </span><span class="kn">import</span> <span class="n">linear_model</span>
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
|
||||
<span class="kn">import</span> <span class="nn">pandas</span> <span class="k">as</span> <span class="nn">pd</span>
|
||||
<span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span>
|
||||
<span class="kn">from</span> <span class="nn">sklearn.model_selection</span> <span class="kn">import</span> <span class="n">train_test_split</span>
|
||||
<span class="kn">from</span> <span class="nn">sklearn</span> <span class="kn">import</span> <span class="n">linear_model</span>
|
||||
|
||||
<span class="k">def</span><span class="w"> </span><span class="nf">MSE</span><span class="p">(</span><span class="n">y_data</span><span class="p">,</span><span class="n">y_model</span><span class="p">):</span>
|
||||
<span class="k">def</span> <span class="nf">MSE</span><span class="p">(</span><span class="n">y_data</span><span class="p">,</span><span class="n">y_model</span><span class="p">):</span>
|
||||
<span class="n">n</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">size</span><span class="p">(</span><span class="n">y_model</span><span class="p">)</span>
|
||||
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">sum</span><span class="p">((</span><span class="n">y_data</span><span class="o">-</span><span class="n">y_model</span><span class="p">)</span><span class="o">**</span><span class="mi">2</span><span class="p">)</span><span class="o">/</span><span class="n">n</span>
|
||||
|
||||
@@ -1663,9 +1673,9 @@ In general the economy-size SVD leads to less FLOPS and still conserving the des
|
||||
<h2>Codes for the SVD<a class="headerlink" href="#codes-for-the-svd" title="Link to this heading">#</a></h2>
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span><span class="w"> </span><span class="nn">numpy</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">np</span>
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
|
||||
<span class="c1"># SVD inversion</span>
|
||||
<span class="k">def</span><span class="w"> </span><span class="nf">SVD</span><span class="p">(</span><span class="n">A</span><span class="p">):</span>
|
||||
<span class="k">def</span> <span class="nf">SVD</span><span class="p">(</span><span class="n">A</span><span class="p">):</span>
|
||||
<span class="w"> </span><span class="sd">''' Takes as input a numpy matrix A and returns inv(A) based on singular value decomposition (SVD).</span>
|
||||
<span class="sd"> SVD is numerically more stable than the inversion algorithms provided by</span>
|
||||
<span class="sd"> numpy and scipy.linalg at the cost of being slower.</span>
|
||||
@@ -2038,7 +2048,7 @@ covariance matrix through the <strong>np.linalg.eig()</strong> function.</p>
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="c1"># Importing various packages</span>
|
||||
<span class="kn">import</span><span class="w"> </span><span class="nn">numpy</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">np</span>
|
||||
<span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
|
||||
<span class="n">n</span> <span class="o">=</span> <span class="mi">100</span>
|
||||
<span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">normal</span><span class="p">(</span><span class="n">size</span><span class="o">=</span><span class="n">n</span><span class="p">)</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">x</span><span class="p">))</span>
|
||||
@@ -2061,7 +2071,7 @@ code which sets up the correlations matrix for the previous example in
|
||||
a more brute force way. Here we scale the mean values for each column of the design matrix, calculate the relevant mean values and variances and then finally set up the <span class="math notranslate nohighlight">\(2\times 2\)</span> correlation matrix (since we have only two vectors).</p>
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span><span class="w"> </span><span class="nn">numpy</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">np</span>
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
|
||||
<span class="n">n</span> <span class="o">=</span> <span class="mi">100</span>
|
||||
<span class="c1"># define two vectors </span>
|
||||
<span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">random</span><span class="p">(</span><span class="n">size</span><span class="o">=</span><span class="n">n</span><span class="p">)</span>
|
||||
@@ -2096,8 +2106,8 @@ this matrix we easily see that it is a positive definite matrix.</p>
|
||||
<p>We whow here how we can set up the correlation matrix using <strong>pandas</strong>, as done in this simple code</p>
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span><span class="w"> </span><span class="nn">numpy</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">np</span>
|
||||
<span class="kn">import</span><span class="w"> </span><span class="nn">pandas</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">pd</span>
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
|
||||
<span class="kn">import</span> <span class="nn">pandas</span> <span class="k">as</span> <span class="nn">pd</span>
|
||||
<span class="n">n</span> <span class="o">=</span> <span class="mi">10</span>
|
||||
<span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">normal</span><span class="p">(</span><span class="n">size</span><span class="o">=</span><span class="n">n</span><span class="p">)</span>
|
||||
<span class="n">x</span> <span class="o">=</span> <span class="n">x</span> <span class="o">-</span> <span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">x</span><span class="p">)</span>
|
||||
@@ -2535,20 +2545,20 @@ and <span class="math notranslate nohighlight">\(\tilde{X}_{ij} = X_{ij} - \frac
|
||||
Note also that we do not split the data into training and test.</p>
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span><span class="w"> </span><span class="nn">numpy</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">np</span>
|
||||
<span class="kn">import</span><span class="w"> </span><span class="nn">matplotlib.pyplot</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">plt</span>
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
|
||||
<span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span>
|
||||
|
||||
<span class="kn">from</span><span class="w"> </span><span class="nn">sklearn.linear_model</span><span class="w"> </span><span class="kn">import</span> <span class="n">LinearRegression</span>
|
||||
<span class="kn">from</span> <span class="nn">sklearn.linear_model</span> <span class="kn">import</span> <span class="n">LinearRegression</span>
|
||||
|
||||
|
||||
<span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">seed</span><span class="p">(</span><span class="mi">2021</span><span class="p">)</span>
|
||||
|
||||
<span class="k">def</span><span class="w"> </span><span class="nf">MSE</span><span class="p">(</span><span class="n">y_data</span><span class="p">,</span><span class="n">y_model</span><span class="p">):</span>
|
||||
<span class="k">def</span> <span class="nf">MSE</span><span class="p">(</span><span class="n">y_data</span><span class="p">,</span><span class="n">y_model</span><span class="p">):</span>
|
||||
<span class="n">n</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">size</span><span class="p">(</span><span class="n">y_model</span><span class="p">)</span>
|
||||
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">sum</span><span class="p">((</span><span class="n">y_data</span><span class="o">-</span><span class="n">y_model</span><span class="p">)</span><span class="o">**</span><span class="mi">2</span><span class="p">)</span><span class="o">/</span><span class="n">n</span>
|
||||
|
||||
|
||||
<span class="k">def</span><span class="w"> </span><span class="nf">fit_beta</span><span class="p">(</span><span class="n">X</span><span class="p">,</span> <span class="n">y</span><span class="p">):</span>
|
||||
<span class="k">def</span> <span class="nf">fit_beta</span><span class="p">(</span><span class="n">X</span><span class="p">,</span> <span class="n">y</span><span class="p">):</span>
|
||||
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">linalg</span><span class="o">.</span><span class="n">pinv</span><span class="p">(</span><span class="n">X</span><span class="o">.</span><span class="n">T</span> <span class="o">@</span> <span class="n">X</span><span class="p">)</span> <span class="o">@</span> <span class="n">X</span><span class="o">.</span><span class="n">T</span> <span class="o">@</span> <span class="n">y</span>
|
||||
|
||||
|
||||
@@ -2678,13 +2688,13 @@ intercept.</p>
|
||||
<p>Armed with this wisdom, we attempt first to simply set the intercept equal to <strong>False</strong> in our implementation of Ridge regression for our well-known vanilla data set.</p>
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span><span class="w"> </span><span class="nn">numpy</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">np</span>
|
||||
<span class="kn">import</span><span class="w"> </span><span class="nn">pandas</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">pd</span>
|
||||
<span class="kn">import</span><span class="w"> </span><span class="nn">matplotlib.pyplot</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">plt</span>
|
||||
<span class="kn">from</span><span class="w"> </span><span class="nn">sklearn.model_selection</span><span class="w"> </span><span class="kn">import</span> <span class="n">train_test_split</span>
|
||||
<span class="kn">from</span><span class="w"> </span><span class="nn">sklearn</span><span class="w"> </span><span class="kn">import</span> <span class="n">linear_model</span>
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
|
||||
<span class="kn">import</span> <span class="nn">pandas</span> <span class="k">as</span> <span class="nn">pd</span>
|
||||
<span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span>
|
||||
<span class="kn">from</span> <span class="nn">sklearn.model_selection</span> <span class="kn">import</span> <span class="n">train_test_split</span>
|
||||
<span class="kn">from</span> <span class="nn">sklearn</span> <span class="kn">import</span> <span class="n">linear_model</span>
|
||||
|
||||
<span class="k">def</span><span class="w"> </span><span class="nf">MSE</span><span class="p">(</span><span class="n">y_data</span><span class="p">,</span><span class="n">y_model</span><span class="p">):</span>
|
||||
<span class="k">def</span> <span class="nf">MSE</span><span class="p">(</span><span class="n">y_data</span><span class="p">,</span><span class="n">y_model</span><span class="p">):</span>
|
||||
<span class="n">n</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">size</span><span class="p">(</span><span class="n">y_model</span><span class="p">)</span>
|
||||
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">sum</span><span class="p">((</span><span class="n">y_data</span><span class="o">-</span><span class="n">y_model</span><span class="p">)</span><span class="o">**</span><span class="mi">2</span><span class="p">)</span><span class="o">/</span><span class="n">n</span>
|
||||
|
||||
@@ -2753,14 +2763,14 @@ What happens if we do not include the intercept in our fit?
|
||||
Let us see how we can change this code by zero centering.</p>
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span><span class="w"> </span><span class="nn">numpy</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">np</span>
|
||||
<span class="kn">import</span><span class="w"> </span><span class="nn">pandas</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">pd</span>
|
||||
<span class="kn">import</span><span class="w"> </span><span class="nn">matplotlib.pyplot</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">plt</span>
|
||||
<span class="kn">from</span><span class="w"> </span><span class="nn">sklearn.model_selection</span><span class="w"> </span><span class="kn">import</span> <span class="n">train_test_split</span>
|
||||
<span class="kn">from</span><span class="w"> </span><span class="nn">sklearn</span><span class="w"> </span><span class="kn">import</span> <span class="n">linear_model</span>
|
||||
<span class="kn">from</span><span class="w"> </span><span class="nn">sklearn.preprocessing</span><span class="w"> </span><span class="kn">import</span> <span class="n">StandardScaler</span>
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
|
||||
<span class="kn">import</span> <span class="nn">pandas</span> <span class="k">as</span> <span class="nn">pd</span>
|
||||
<span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span>
|
||||
<span class="kn">from</span> <span class="nn">sklearn.model_selection</span> <span class="kn">import</span> <span class="n">train_test_split</span>
|
||||
<span class="kn">from</span> <span class="nn">sklearn</span> <span class="kn">import</span> <span class="n">linear_model</span>
|
||||
<span class="kn">from</span> <span class="nn">sklearn.preprocessing</span> <span class="kn">import</span> <span class="n">StandardScaler</span>
|
||||
|
||||
<span class="k">def</span><span class="w"> </span><span class="nf">MSE</span><span class="p">(</span><span class="n">y_data</span><span class="p">,</span><span class="n">y_model</span><span class="p">):</span>
|
||||
<span class="k">def</span> <span class="nf">MSE</span><span class="p">(</span><span class="n">y_data</span><span class="p">,</span><span class="n">y_model</span><span class="p">):</span>
|
||||
<span class="n">n</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">size</span><span class="p">(</span><span class="n">y_model</span><span class="p">)</span>
|
||||
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">sum</span><span class="p">((</span><span class="n">y_data</span><span class="o">-</span><span class="n">y_model</span><span class="p">)</span><span class="o">**</span><span class="mi">2</span><span class="p">)</span><span class="o">/</span><span class="n">n</span>
|
||||
<span class="c1"># A seed just to ensure that the random numbers are the same for every run.</span>
|
||||
|
||||
@@ -28,7 +28,7 @@
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-brands-400.woff2" />
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-regular-400.woff2" />
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/sphinx-book-theme.css?v=eba8b062" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css?v=13237357" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
|
||||
@@ -246,6 +246,16 @@
|
||||
|
||||
|
||||
|
||||
<li class="toctree-l1"><a class="reference internal" href="exercisesweek42.html">Exercises week 42</a></li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
|
||||
<ul class="nav bd-sidenav">
|
||||
|
||||
@@ -28,7 +28,7 @@
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-brands-400.woff2" />
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-regular-400.woff2" />
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/sphinx-book-theme.css?v=eba8b062" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css?v=13237357" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
|
||||
@@ -246,6 +246,16 @@
|
||||
|
||||
|
||||
|
||||
<li class="toctree-l1"><a class="reference internal" href="exercisesweek42.html">Exercises week 42</a></li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
|
||||
<ul class="nav bd-sidenav">
|
||||
|
||||
@@ -28,7 +28,7 @@
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-brands-400.woff2" />
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-regular-400.woff2" />
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/sphinx-book-theme.css?v=eba8b062" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css?v=13237357" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
|
||||
@@ -246,6 +246,16 @@
|
||||
|
||||
|
||||
|
||||
<li class="toctree-l1"><a class="reference internal" href="exercisesweek42.html">Exercises week 42</a></li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
|
||||
<ul class="nav bd-sidenav">
|
||||
|
||||
@@ -28,7 +28,7 @@
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-brands-400.woff2" />
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-regular-400.woff2" />
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/sphinx-book-theme.css?v=eba8b062" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css?v=13237357" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
|
||||
@@ -246,6 +246,16 @@
|
||||
|
||||
|
||||
|
||||
<li class="toctree-l1"><a class="reference internal" href="exercisesweek42.html">Exercises week 42</a></li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
|
||||
<ul class="nav bd-sidenav">
|
||||
|
||||
@@ -28,7 +28,7 @@
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-brands-400.woff2" />
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-regular-400.woff2" />
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/sphinx-book-theme.css?v=eba8b062" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css?v=13237357" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
|
||||
@@ -246,6 +246,16 @@
|
||||
|
||||
|
||||
|
||||
<li class="toctree-l1"><a class="reference internal" href="exercisesweek42.html">Exercises week 42</a></li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
|
||||
<ul class="nav bd-sidenav">
|
||||
|
||||
@@ -28,7 +28,7 @@
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-brands-400.woff2" />
|
||||
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-regular-400.woff2" />
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/styles/sphinx-book-theme.css?v=eba8b062" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css?v=13237357" />
|
||||
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
|
||||
@@ -246,6 +246,16 @@
|
||||
|
||||
|
||||
|
||||
<li class="toctree-l1"><a class="reference internal" href="exercisesweek42.html">Exercises week 42</a></li>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
|
||||
<ul class="nav bd-sidenav">
|
||||
|
||||
Reference in New Issue
Block a user