+
+
+
+Exercises week 43#
+October 20-24, 2025
+Date: Deadline Friday October 24 at midnight
+Overarching aims of the exercises for week 43#
+The aim of the exercises this week is to gain some confidence with +ways to visualize the results of a classification problem. We will +target three ways of setting up the analysis. The first and simplest +one is the
+-
+
so-called confusion matrix. The next one is the so-called
+ROC curve. Finally we have the
+Cumulative gain curve.
+
We will use Logistic Regression as method for the classification in +this exercise. You can compare these results with those obtained with +your neural network code from project 2 without a hidden layer.
+In these exercises we will use binary and multi-class data sets +(the Iris data set from week 41).
+The underlying mathematics is described here.
+Confusion Matrix#
+A confusion matrix summarizes a classifier’s performance by +tabulating predictions versus true labels. For binary classification, +it is a \(2\times2\) table whose entries are counts of outcomes:
+Here TP (true positives) is the number of cases correctly predicted as +positive, FP (false positives) is the number incorrectly predicted as +positive, TN (true negatives) is correctly predicted negative, and FN +(false negatives) is incorrectly predicted negative . In other words, +“positive” means class 1 and “negative” means class 0; for example, TP +occurs when the prediction and actual are both positive. Formally:
+where TPR and FPR are the true and false positive rates defined below.
+In multiclass classification with \(K\) classes, the confusion matrix +generalizes to a \(K\times K\) table. Entry \(N_{ij}\) in the table is +the count of instances whose true class is \(i\) and whose predicted +class is \(j\). For example, a three-class confusion matrix can be written +as:
+Here the diagonal entries \(N_{ii}\) are the true positives for each +class, and off-diagonal entries are misclassifications. This matrix +allows computation of per-class metrics: e.g. for class \(i\), +\(\mathrm{TP}_i=N_{ii}\), \(\mathrm{FN}_i=\sum_{j\neq i}N_{ij}\), +\(\mathrm{FP}_i=\sum_{j\neq i}N_{ji}\), and \(\mathrm{TN}_i\) is the sum of +all remaining entries.
+As defined above, TPR and FPR come from the binary case. In binary +terms with \(P\) actual positives and \(N\) actual negatives, one has
+as used in standard confusion-matrix +formulations. These rates will be used in constructing ROC curves.
+ROC Curve#
+The Receiver Operating Characteristic (ROC) curve plots the trade-off +between true positives and false positives as a discrimination +threshold varies. Specifically, for a binary classifier that outputs +a score or probability, one varies the threshold \(t\) for declaring +positive, and computes at each \(t\) the true positive rate +\(\mathrm{TPR}(t)\) and false positive rate \(\mathrm{FPR}(t)\) using the +confusion matrix at that threshold. The ROC curve is then the graph +of TPR versus FPR. By definition,
+where \(TP,FP,TN,FN\) are counts determined by threshold \(t\). A perfect +classifier would reach the point (FPR=0, TPR=1) at some threshold.
+Formally, the ROC curve is obtained by plotting +\((\mathrm{FPR}(t),\mathrm{TPR}(t))\) for all \(t\in[0,1]\) (or as \(t\) +sweeps through the sorted scores). The Area Under the ROC Curve (AUC) +quantifies the average performance over all thresholds. It can be +interpreted probabilistically: \(\mathrm{AUC} = +\Pr\bigl(s(X^+)>s(X^-)\bigr)\), the probability that a random positive +instance \(X^+\) receives a higher score \(s\) than a random negative +instance \(X^-\) . Equivalently, the AUC is the integral under the ROC +curve:
+where \(f\) ranges over FPR (or fraction of negatives). A model that guesses at random yields a diagonal ROC (AUC=0.5), whereas a perfect model yields AUC=1.0.
+Cumulative Gain#
+The cumulative gain curve (or gains chart) evaluates how many +positives are captured as one targets an increasing fraction of the +population, sorted by model confidence. To construct it, sort all +instances by decreasing predicted probability of the positive class. +Then, for the top \(\alpha\) fraction of instances, compute the fraction +of all actual positives that fall in this subset. In formula form, if +\(P\) is the total number of positive instances and \(P(\alpha)\) is the +number of positives among the top \(\alpha\) of the data, the cumulative +gain at level \(\alpha\) is
+For example, cutting off at the top 10% of predictions yields a gain +equal to (positives in top 10%) divided by (total positives) . +Plotting \(\mathrm{Gain}(\alpha)\) versus \(\alpha\) (often in percent) +gives the gain curve. The baseline (random) curve is the diagonal +\(\mathrm{Gain}(\alpha)=\alpha\), while an ideal model has a steep climb +toward 1.
+A related measure is the {\em lift}, often called the gain ratio. It is the ratio of the model’s capture rate to that of random selection. Equivalently,
+A lift \(>1\) indicates better-than-random targeting. In practice, gain +and lift charts (used e.g.\ in marketing or imbalanced classification) +show how many positives can be “gained” by focusing on a fraction of +the population .
+Other measures: Precision, Recall, and the F\(_1\) Measure#
+Precision and recall (sensitivity) quantify binary classification +accuracy in terms of positive predictions. They are defined from the +confusion matrix as:
+Precision is the fraction of predicted positives that are correct, and +recall is the fraction of actual positives that are correctly +identified . A high-precision classifier makes few false-positive +errors, while a high-recall classifier makes few false-negative +errors.
+The F\(_1\) score (balanced F-measure) combines precision and recall into a single metric via their harmonic mean. The usual formula is:
+This can be shown to equal
+The F\(_1\) score ranges from 0 (worst) to 1 (best), and balances the +trade-off between precision and recall.
+For multi-class classification, one computes per-class +precision/recall/F\(_1\) (treating each class as “positive” in a +one-vs-rest manner) and then averages. Common averaging methods are:
+Micro-averaging: Sum all true positives, false positives, and false negatives across classes, then compute precision/recall/F\(_1\) from these totals. +Macro-averaging: Compute the F\(1\) score \(F{1,i}\) for each class \(i\) separately, then take the unweighted mean: \(F_{1,\mathrm{macro}} = \frac{1}{K}\sum_{i=1}^K F_{1,i}\) . This treats all classes equally regardless of size. +Weighted-averaging: Like macro-average, but weight each class’s \(F_{1,i}\) by its support \(n_i\) (true count): \(F_{1,\mathrm{weighted}} = \frac{1}{N}\sum_{i=1}^K n_i F_{1,i}\), where \(N=\sum_i n_i\). This accounts for class imbalance by giving more weight to larger classes .
+Each of these averages has different use-cases. Micro-average is +dominated by common classes, macro-average highlights performance on +rare classes, and weighted-average is a compromise. These formulas +and concepts allow rigorous evaluation of classifier performance in +both binary and multi-class settings.
+Exercises#
+Here is a simple code example which uses the Logistic regression machinery from scikit-learn. +At the end it sets up the confusion matrix and the ROC and cumulative gain curves. +Feel free to use these functionalities (we don’t expect you to write your own code for say the confusion matrix).
+%matplotlib inline
+
+import matplotlib.pyplot as plt
+import numpy as np
+from sklearn.model_selection import train_test_split
+# from sklearn.datasets import fill in the data set
+from sklearn.linear_model import LogisticRegression
+
+# Load the data, fill inn
+mydata.data = ?
+
+X_train, X_test, y_train, y_test = train_test_split(mydata.data,cancer.target,random_state=0)
+print(X_train.shape)
+print(X_test.shape)
+# Logistic Regression
+# define which type of problem, binary or multiclass
+logreg = LogisticRegression(solver='lbfgs')
+logreg.fit(X_train, y_train)
+
+from sklearn.preprocessing import LabelEncoder
+from sklearn.model_selection import cross_validate
+#Cross validation
+accuracy = cross_validate(logreg,X_test,y_test,cv=10)['test_score']
+print(accuracy)
+print("Test set accuracy with Logistic Regression: {:.2f}".format(logreg.score(X_test,y_test)))
+
+import scikitplot as skplt
+y_pred = logreg.predict(X_test)
+skplt.metrics.plot_confusion_matrix(y_test, y_pred, normalize=True)
+plt.show()
+y_probas = logreg.predict_proba(X_test)
+skplt.metrics.plot_roc(y_test, y_probas)
+plt.show()
+skplt.metrics.plot_cumulative_gain(y_test, y_probas)
+plt.show()
+Exercise a)#
+Convince yourself about the mathematics for the confusion matrix, the ROC and the cumlative gain curves for both a binary and a multiclass classification problem.
+Exercise b)#
+Use a binary classification data available from scikit-learn. As an example you can use +the MNIST data set and just specialize to two numbers. To do so you can use the following code lines
+from sklearn.datasets import load_digits
+digits = load_digits(n_class=2) # Load only two classes, e.g., 0 and 1
+X, y = digits.data, digits.target
+Alternatively, you can use the make\(\_\)classification +functionality. This function generates a random \(n\)-class classification +dataset, which can be configured for binary classification by setting +n_classes=2. You can also control the number of samples, features, +informative features, redundant features, and more.
+from sklearn.datasets import make_classification
+X, y = make_classification(n_samples=1000, n_features=20, n_informative=10, n_redundant=5, n_classes=2, random_state=42)
+You can use this option for the multiclass case as well, see the next exercise. +If you prefer to study other binary classification datasets, feel free +to replace the above suggestions with your own dataset.
+Make plots of the confusion matrix, the ROC curve and the cumulative gain curve.
+Exercise c) week 43#
+As a multiclass problem, we will use the Iris data set discussed in +the exercises from weeks 41 and 42. This is a three-class data set and +you can set it up using scikit-learn,
+from sklearn.datasets import load_iris
+iris = load_iris()
+X = iris.data # Features
+y = iris.target # Target labels
+Make plots of the confusion matrix, the ROC curve and the cumulative +gain curve for this (or other) multiclass data set.
+![](\"figslides/nn.jpeg\")
\n",
- "\n",
- "\n",
- "\n",
- "\n",
- "## 3D volumes of neurons\n",
- "\n",
- "Convolutional Neural Networks take advantage of the fact that the\n",
- "input consists of images and they constrain the architecture in a more\n",
- "sensible way. \n",
- "\n",
- "In particular, unlike a regular Neural Network, the\n",
- "layers of a CNN have neurons arranged in 3 dimensions: width,\n",
- "height, depth. (Note that the word depth here refers to the third\n",
- "dimension of an activation volume, not to the depth of a full Neural\n",
- "Network, which can refer to the total number of layers in a network.)\n",
- "\n",
- "To understand it better, the above example of an image \n",
- "with an input volume of\n",
- "activations has dimensions $32\\times 32\\times 3$ (width, height,\n",
- "depth respectively). \n",
- "\n",
- "The neurons in a layer will\n",
- "only be connected to a small region of the layer before it, instead of\n",
- "all of the neurons in a fully-connected manner. Moreover, the final\n",
- "output layer could for this specific image have dimensions $1\\times 1 \\times 10$, \n",
- "because by the\n",
- "end of the CNN architecture we will reduce the full image into a\n",
- "single vector of class scores, arranged along the depth\n",
- "dimension. \n",
- "\n",
- "\n",
- "\n",
- "\n",
- "![](\"figslides/cnn.jpeg\")
\n",
- "\n",
- "\n",
- "\n",
- "\n",
- "\n",
- "\n",
- "\n",
- "## Layers used to build CNNs\n",
- "\n",
- "\n",
- "A simple CNN is a sequence of layers, and every layer of a CNN\n",
- "transforms one volume of activations to another through a\n",
- "differentiable function. We use three main types of layers to build\n",
- "CNN architectures: Convolutional Layer, Pooling Layer, and\n",
- "Fully-Connected Layer (exactly as seen in regular Neural Networks). We\n",
- "will stack these layers to form a full CNN architecture.\n",
- "\n",
- "A simple CNN for image classification could have the architecture:\n",
- "\n",
- "* **INPUT** ($32\\times 32 \\times 3$) will hold the raw pixel values of the image, in this case an image of width 32, height 32, and with three color channels R,G,B.\n",
- "\n",
- "* **CONV** (convolutional )layer will compute the output of neurons that are connected to local regions in the input, each computing a dot product between their weights and a small region they are connected to in the input volume. This may result in volume such as $[32\\times 32\\times 12]$ if we decided to use 12 filters.\n",
- "\n",
- "* **RELU** layer will apply an elementwise activation function, such as the $max(0,x)$ thresholding at zero. This leaves the size of the volume unchanged ($[32\\times 32\\times 12]$).\n",
- "\n",
- "* **POOL** (pooling) layer will perform a downsampling operation along the spatial dimensions (width, height), resulting in volume such as $[16\\times 16\\times 12]$.\n",
- "\n",
- "* **FC** (i.e. fully-connected) layer will compute the class scores, resulting in volume of size $[1\\times 1\\times 10]$, where each of the 10 numbers correspond to a class score, such as among the 10 categories of the MNIST images we considered above . As with ordinary Neural Networks and as the name implies, each neuron in this layer will be connected to all the numbers in the previous volume.\n",
- "\n",
- "## Transforming images\n",
- "\n",
- "CNNs transform the original image layer by layer from the original\n",
- "pixel values to the final class scores. \n",
- "\n",
- "Observe that some layers contain\n",
- "parameters and other don’t. In particular, the CNN layers perform\n",
- "transformations that are a function of not only the activations in the\n",
- "input volume, but also of the parameters (the weights and biases of\n",
- "the neurons). On the other hand, the RELU/POOL layers will implement a\n",
- "fixed function. The parameters in the CONV/FC layers will be trained\n",
- "with gradient descent so that the class scores that the CNN computes\n",
- "are consistent with the labels in the training set for each image.\n",
- "\n",
- "\n",
- "## CNNs in brief\n",
- "\n",
- "In summary:\n",
- "\n",
- "* A CNN architecture is in the simplest case a list of Layers that transform the image volume into an output volume (e.g. holding the class scores)\n",
- "\n",
- "* There are a few distinct types of Layers (e.g. CONV/FC/RELU/POOL are by far the most popular)\n",
- "\n",
- "* Each Layer accepts an input 3D volume and transforms it to an output 3D volume through a differentiable function\n",
- "\n",
- "* Each Layer may or may not have parameters (e.g. CONV/FC do, RELU/POOL don’t)\n",
- "\n",
- "* Each Layer may or may not have additional hyperparameters (e.g. CONV/FC/POOL do, RELU doesn’t)\n",
- "\n",
- "For more material on convolutional networks, we strongly recommend\n",
- "the course\n",
- "[IN5400 – Machine Learning for Image Analysis](https://www.uio.no/studier/emner/matnat/ifi/IN5400/index-eng.html)\n",
- "and the slides of [CS231](http://cs231n.github.io/convolutional-networks/) which is taught at Stanford University (consistently ranked as one of the top computer science programs in the world). [Michael Nielsen's book is a must read, in particular chapter 6 which deals with CNNs](http://neuralnetworksanddeeplearning.com/chap6.html).\n",
- "\n",
- "\n",
- "\n",
- "## CNNs in more detail, building convolutional neural networks in Tensorflow and Keras\n",
- "\n",
- "\n",
- "As discussed above, CNNs are neural networks built from the assumption that the inputs\n",
- "to the network are 2D images. This is important because the number of features or pixels in images\n",
- "grows very fast with the image size, and an enormous number of weights and biases are needed in order to build an accurate network. \n",
- "\n",
- "As before, we still have our input, a hidden layer and an output. What's novel about convolutional networks\n",
- "are the **convolutional** and **pooling** layers stacked in pairs between the input and the hidden layer.\n",
- "In addition, the data is no longer represented as a 2D feature matrix, instead each input is a number of 2D\n",
- "matrices, typically 1 for each color dimension (Red, Green, Blue). \n",
- "\n",
- "\n",
- "## Setting it up\n",
- "\n",
- "It means that to represent the entire\n",
- "dataset of images, we require a 4D matrix or **tensor**. This tensor has the dimensions:"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "$$\n",
- "(n_{inputs},\\, n_{pixels, width},\\, n_{pixels, height},\\, depth) .\n",
- "$$"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## The MNIST dataset again\n",
- "\n",
- "The MNIST dataset consists of grayscale images with a pixel size of\n",
- "$28\\times 28$, meaning we require $28 \\times 28 = 724$ weights to each\n",
- "neuron in the first hidden layer.\n",
- "\n",
- "If we were to analyze images of size $128\\times 128$ we would require\n",
- "$128 \\times 128 = 16384$ weights to each neuron. Even worse if we were\n",
- "dealing with color images, as most images are, we have an image matrix\n",
- "of size $128\\times 128$ for each color dimension (Red, Green, Blue),\n",
- "meaning 3 times the number of weights $= 49152$ are required for every\n",
- "single neuron in the first hidden layer.\n",
- "\n",
- "\n",
- "## Strong correlations\n",
- "\n",
- "Images typically have strong local correlations, meaning that a small\n",
- "part of the image varies little from its neighboring regions. If for\n",
- "example we have an image of a blue car, we can roughly assume that a\n",
- "small blue part of the image is surrounded by other blue regions.\n",
- "\n",
- "Therefore, instead of connecting every single pixel to a neuron in the\n",
- "first hidden layer, as we have previously done with deep neural\n",
- "networks, we can instead connect each neuron to a small part of the\n",
- "image (in all 3 RGB depth dimensions). The size of each small area is\n",
- "fixed, and known as a [receptive](https://en.wikipedia.org/wiki/Receptive_field).\n",
- "\n",
- "\n",
- "\n",
- "## Layers of a CNN\n",
- "The layers of a convolutional neural network arrange neurons in 3D: width, height and depth. \n",
- "The input image is typically a square matrix of depth 3. \n",
- "\n",
- "A **convolution** is performed on the image which outputs\n",
- "a 3D volume of neurons. The weights to the input are arranged in a number of 2D matrices, known as **filters**.\n",
- "\n",
- "\n",
- "Each filter slides along the input image, taking the dot product\n",
- "between each small part of the image and the filter, in all depth\n",
- "dimensions. This is then passed through a non-linear function,\n",
- "typically the **Rectified Linear (ReLu)** function, which serves as the\n",
- "activation of the neurons in the first convolutional layer. This is\n",
- "further passed through a **pooling layer**, which reduces the size of the\n",
- "convolutional layer, e.g. by taking the maximum or average across some\n",
- "small regions, and this serves as input to the next convolutional\n",
- "layer.\n",
- "\n",
- "\n",
- "## Systematic reduction\n",
- "\n",
- "By systematically reducing the size of the input volume, through\n",
- "convolution and pooling, the network should create representations of\n",
- "small parts of the input, and then from them assemble representations\n",
- "of larger areas. The final pooling layer is flattened to serve as\n",
- "input to a hidden layer, such that each neuron in the final pooling\n",
- "layer is connected to every single neuron in the hidden layer. This\n",
- "then serves as input to the output layer, e.g. a softmax output for\n",
- "classification.\n",
- "\n",
- "\n",
- "## Prerequisites: Collect and pre-process data"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 1,
- "metadata": {},
-<<<<<<< HEAD
- "outputs": [],
-=======
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "inputs = (n_inputs, pixel_width, pixel_height, depth) = (1797, 8, 8, 1)\n",
- "labels = (n_inputs) = (1797,)\n"
- ]
- },
- {
- "data": {
- "image/png": "iVBORw0KGgoAAAANSUhEUgAAAqwAAACRCAYAAAAGuepqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAALD0lEQVR4nO3dX2jl6VkH8OdpZ6HWP3O6KKJ2d6ItFbYXmxtRsTJnQBAESWBZEaw7GWnBK2eW6o0gyUi9EC82I17o1Wa6ghUtJKCLIDqJ1lW0sBnozYKUtGux2MqeuKsiWl8vTgbDMH+yz++cnDcznw8EMnvy/J73l/Oc93zPLyfZbK0FAAD06j2LXgAAADyIwAoAQNcEVgAAuiawAgDQNYEVAICuCawAAHTtkQ+smbmbmZ847VrOFnPCSZkVTsqscBLm5GTOTGDNzIPM/IlFr+N+curTmfnVzDw8GqKPLnpdj5szMCe/m5nvHPv4r8x8e9HrehyZFU7qDMzKz2bmG0fPPf+SmTcz8zsWva7HjTmZrzMTWM+A5yPiFyLixyPiyYj424h4ZaErojuttV9srX3bnY+I+IOI+KNFr4v+mBXehb+JiB9rrZ2PiB+IiHMR8enFLokOnek5OfOBNTM/kJl/kplfz8y3jj7/4F1f9qHM/PujVxU7mfnksfofyczXMnOSmbczc1xcyvdHxOdba19qrX0zIn4/Ip4pHosZ62hOjq/pWyPiuYi4OfRYzI5Z4aR6mZXW2puttW8c+0/fjIgPV47F7JmT2TjzgTWm5/ByRFyIiKcj4j8j4nfu+poXYnr183sj4n8i4rcjIjLz+yLiT2P6CuPJiPjliPhcZn7X3U0y8+mjYXn6Puv4bER8ODM/kplPRMTliPizgefG7PQyJ8c9FxFfj4i/qpwQc2NWOKluZiUzP5aZhxHxdkznZXPYqTFD5mQGznxgba39a2vtc621/2itvR0RvxERF+/6sldaa19srf17RPxaRPxMZr43Ij4eEa+21l5trf1va+3PI+ILEfFT9+jzldbaqLX2lfss5Z8j4q8j4o2YDuPzEfHiTE6SwTqak+MuR8RnWmtt0MkxU2aFk+ppVlprnz/6Ue8HI+K3IuJgJifJYOZkNs58YM3M92fm72XmlzPz32J6BWJ0dEff8eaxz78cEU9ExHfG9NXO80evSCaZOYmIj0XE9xSWsh4RPxQRT0XE+yLiekT8ZWa+v3AsZqyjObmznqdiumF9pnoM5sOscFK9zUpERGvtqzH96d5nhxyH2TEns3Fu0QuYgU9FxA9GxA+31r6WmcsR8XpE5LGveerY509HxH9HxDdiOiCvtNY+OYN1PBsRf9ha+6ejf29l5mZM38f6hRkcn2F6mZM7XoiI11prX5rhMZkNs8JJ9TYrd5yLiA/N4bjUmJMZOGtXWJ/IzPcd+zgXEd8e0x/BT47epLx+j7qPZ+YzR1c7fz0i/vjYL0b9dGb+ZGa+9+iY43u8Gfok/iGmr4K+OzPfk5k/H9NXSP9YOlOG6HlO7nghIrYG1DMbZoWT6nZWMvPnjt6/mJl5IaY/cv6L8pkyhDmZk7MWWF+N6Z1+52Mjpm8Y/paYvhL5u7j3Lzq9EtMN/2sx/XH9L0VMf2MuIlYi4ldj+gsNb0bEr8Q9vi9Hd/I7ef83M/9mRNyOiP2ImMT0/avPtdYm7/40GajnOYnM/NGYvn/InyhaPLPCSfU8K89ExGsR8U5M/3TRGxExjytyPJw5mZP0Hn4AAHp21q6wAgDwmBFYAQDomsAKAEDXBFYAALr2sL/Deuq/kbW1tVWq29jYKPccjUalus3N+v/RbDwel2sHyId/Sdmpz8ru7m6prjpjERHb29ulusPDw3LPW7duleoGzti8ZuXU52RnZ6dUd/Xq1Rmv5OGqMx0RsbS0NLN1vAvd7SkHBwflhtU9fcieUt0bzp8/X+65v79fqhs4Y13tKZNJ/Q/4LGJOquut3tcR/e0prrACANA1gRUAgK4JrAAAdE1gBQCgawIrAABdE1gBAOiawAoAQNcEVgAAuiawAgDQNYEVAICuCawAAHRNYAUAoGsCKwAAXTs3j4Pu7u6Wa69cuVKqW1lZKfccjUalutXV1XLPyWRSrmXq2rVrpboh3/u1tbVS3Y0bN8o9q/P5KDk4OCjXDnmcnrbt7e1ybfXx8KhZxPfh5s2b5dpbt26V6obsKZ5/hn0Pqo/TIXtRtefW1la558bGRrl2HlxhBQCgawIrAABdE1gBAOiawAoAQNcEVgAAuiawAgDQNYEVAICuCawAAHRNYAUAoGsCKwAAXRNYAQDomsAKAEDXBFYAALomsAIA0LVsrT3o9gfeeD/Xrl2rrSYiDg4OSnXb29vlnuPxuFQ3Go3KPYesd4Cc47FLszJEdVaG3G97e3ulusuXL5d7TiaTcu0A85qVU5+Tzc3NUt3y8nK556VLl0p1Fy9eLPfc3d0t1w7wSO0pi1B9vtzf3y/3fMRm5bGYk2pOGbKPVffOge47J66wAgDQNYEVAICuCawAAHRNYAUAoGsCKwAAXRNYAQDomsAKAEDXBFYAALomsAIA0DWBFQCArgmsAAB0TWAFAKBrAisAAF07N4+DLi0tlWsPDg5KdRsbG+Wee3t7pbrXX3+93JPhJpNJqa46YxER6+vrpbrRaFTuWV3vkMfho2Rtba1UN2RPqaruRRH19S7iPPl/y8vLpbqtra1yz+reOWQfe5RU9+TV1dXZLuQENjc3T73nvLjCCgBA1wRWAAC6JrACANA1gRUAgK4JrAAAdE1gBQCgawIrAABdE1gBAOiawAoAQNcEVgAAuiawAgDQNYEVAICuCawAAHRNYAUAoGvZWnvQ7Q+8cR6Wl5dLdbdv3y73vHz5cqlua2ur3HNBco7HLs3Kzs5OueHq6mq59ixZX18v1W1sbAxpO69ZKc3J/v5+ueF4PC7VHR4elntWVfeiiPr9vbS0VO4ZHe4pj4sh91t179zc3Cz3jM72lCGq+1F1L4qo70cvv/xyuefa2lq5doD7zokrrAAAdE1gBQCgawIrAABdE1gBAOiawAoAQNcEVgAAuiawAgDQNYEVAICuCawAAHRNYAUAoGsCKwAAXRNYAQDomsAKAEDXsrX2oNsfeOM8LC8vn3bLGI1Gpboha93c3CzXDpBzPHZpVnZ3d8sNt7e3S3X7+/vlngcHB6feszqfA81rVk59Ti5dulSurVpZWSnVVWd6gbrbUx4X4/H41HsOeRxGZ3vKZDKZ9ToeasheXr2/q89ZQ2sHuO+cuMIKAEDXBFYAALomsAIA0DWBFQCArgmsAAB0TWAFAKBrAisAAF0TWAEA6JrACgBA1wRWAAC6JrACANA1gRUAgK4JrAAAdE1gBQCga+cWvYC7jUajUt14PC733NjYKNVV17qonj0acr8dHh6W6ra2tso9V1dXS3WP2v122obMydWrV0t1N27cKPe8cuVKuZbF2dnZKdVduHCh3HN/f/9U6yLqzz+Pkr29vXLt+vp6qe769evlnmtra6W6IXvRZDIp1c3r+c4VVgAAuiawAgDQNYEVAICuCawAAHRNYAUAoGsCKwAAXRNYAQDomsAKAEDXBFYAALomsAIA0DWBFQCArgmsAAB0TWAFAKBr5xa9gLu9+OKLpbrV1dVyz+vXr5fqVlZWyj1Ho1G5lqm33nqrVHd4eFjuuba2Vq7lbHn22WfLtUP2BhbnpZdeKtXt7e2Ve54/f75UN2Qvso9FXLx4sVw7Ho9LddX5ioiYTCaluqtXr5Z79pZTXGEFAKBrAisAAF0TWAEA6JrACgBA1wRWAAC6JrACANA1gRUAgK4JrAAAdE1gBQCgawIrAABdE1gBAOiawAoAQNcEVgAAuiawAgDQtWytLXoNAABwX66wAgDQNYEVAICuCawAAHRNYAUAoGsCKwAAXRNYAQDo2v8BlXpliBTuHMEAAAAASUVORK5CYII=\n",
- "text/plain": [
- "![](\"figures/simplenn1.png\")
![](\"figures/simplenn2.png\")
![](\"figures/simplenn3.png\")
![](\"figures/nn2.png\")
![](\"figures/structure.png\")