diff --git a/doc/pub/week44/ipynb/week44.ipynb b/doc/pub/week44/ipynb/week44.ipynb
index 3836b8722..e8457b8a3 100644
--- a/doc/pub/week44/ipynb/week44.ipynb
+++ b/doc/pub/week44/ipynb/week44.ipynb
@@ -3,9 +3,7 @@
{
"cell_type": "markdown",
"id": "32b868e8",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"\n",
@@ -15,9 +13,7 @@
{
"cell_type": "markdown",
"id": "f49ca0a8",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"# Week 44, Convolutional Neural Networks (CNN)\n",
"**Morten Hjorth-Jensen**, Department of Physics, University of Oslo and Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University\n",
@@ -28,9 +24,7 @@
{
"cell_type": "markdown",
"id": "18b6439b",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## Plan for week 44\n",
"\n",
@@ -76,9 +70,7 @@
{
"cell_type": "markdown",
"id": "4d15ec9a",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## Material for Lecture Thursday November 2"
]
@@ -86,9 +78,7 @@
{
"cell_type": "markdown",
"id": "d059dfe4",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## Convolutional Neural Networks (recognizing images)\n",
"\n",
@@ -113,9 +103,7 @@
{
"cell_type": "markdown",
"id": "11298ed6",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## What is the Difference\n",
"\n",
@@ -135,9 +123,7 @@
{
"cell_type": "markdown",
"id": "adf7491b",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## Neural Networks vs CNNs\n",
"\n",
@@ -153,9 +139,7 @@
{
"cell_type": "markdown",
"id": "0c91aae7",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## Why CNNS for images, sound files, medical images from CT scans etc?\n",
"\n",
@@ -183,9 +167,7 @@
{
"cell_type": "markdown",
"id": "44f4900f",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## Regular NNs don’t scale well to full images\n",
"\n",
@@ -213,9 +195,7 @@
{
"cell_type": "markdown",
"id": "65dfc36c",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## 3D volumes of neurons\n",
"\n",
@@ -253,9 +233,7 @@
{
"cell_type": "markdown",
"id": "9cc13745",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## Layers used to build CNNs\n",
"\n",
@@ -282,9 +260,7 @@
{
"cell_type": "markdown",
"id": "2b29f8fd",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## Transforming images\n",
"\n",
@@ -304,9 +280,7 @@
{
"cell_type": "markdown",
"id": "b863e7a7",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## CNNs in brief\n",
"\n",
@@ -333,9 +307,7 @@
{
"cell_type": "markdown",
"id": "d00f2c70",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## Key Idea\n",
"\n",
@@ -350,9 +322,7 @@
{
"cell_type": "markdown",
"id": "b449a003",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## Mathematics of CNNs\n",
"\n",
@@ -370,9 +340,7 @@
{
"cell_type": "markdown",
"id": "cc14fe57",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"$$\n",
"y(t) = \\int x(a) w(t-a) da,\n",
@@ -382,9 +350,7 @@
{
"cell_type": "markdown",
"id": "756045db",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"where $x(a)$ represents a so-called input and $w(t-a)$ is normally called the weight function or kernel.\n",
"\n",
@@ -394,9 +360,7 @@
{
"cell_type": "markdown",
"id": "ce837df3",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"$$\n",
"y(t) = \\left(x * w\\right)(t).\n",
@@ -406,9 +370,7 @@
{
"cell_type": "markdown",
"id": "9783d27c",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"The discretized version reads"
]
@@ -416,9 +378,7 @@
{
"cell_type": "markdown",
"id": "ab5e9780",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"$$\n",
"y(t) = \\sum_{a=-\\infty}^{a=\\infty}x(a)w(t-a).\n",
@@ -428,9 +388,7 @@
{
"cell_type": "markdown",
"id": "464f7836",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"Computing the inverse of the above convolution operations is known as deconvolution.\n",
"\n",
@@ -440,9 +398,7 @@
{
"cell_type": "markdown",
"id": "154be50a",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## Convolution Examples: Polynomial multiplication\n",
"\n",
@@ -455,9 +411,7 @@
{
"cell_type": "markdown",
"id": "b4d21b2f",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"$$\n",
"p(t) = \\alpha_0+\\alpha_1 t+\\alpha_2 t^2,\n",
@@ -467,9 +421,7 @@
{
"cell_type": "markdown",
"id": "ff44d22a",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"and"
]
@@ -477,9 +429,7 @@
{
"cell_type": "markdown",
"id": "1c7d0d6e",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"$$\n",
"s(t) = \\beta_0+\\beta_1 t+\\beta_2 t^2+\\beta_3 t^3.\n",
@@ -489,9 +439,7 @@
{
"cell_type": "markdown",
"id": "214f4775",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"The polynomial multiplication gives us a new polynomial of degree $5$"
]
@@ -499,9 +447,7 @@
{
"cell_type": "markdown",
"id": "e2d3411d",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"$$\n",
"z(t) = \\delta_0+\\delta_1 t+\\delta_2 t^2+\\delta_3 t^3+\\delta_4 t^4+\\delta_5 t^5.\n",
@@ -511,9 +457,7 @@
{
"cell_type": "markdown",
"id": "2ddb06e5",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## Efficient Polynomial Multiplication\n",
"\n",
@@ -524,9 +468,7 @@
{
"cell_type": "markdown",
"id": "6c188f5d",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"$$\n",
"\\begin{split}\n",
@@ -543,9 +485,7 @@
{
"cell_type": "markdown",
"id": "0b315303",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"We note that $\\alpha_i=0$ except for $i\\in \\left\\{0,1,2\\right\\}$ and $\\beta_i=0$ except for $i\\in\\left\\{0,1,2,3\\right\\}$.\n",
"\n",
@@ -555,9 +495,7 @@
{
"cell_type": "markdown",
"id": "69576ac3",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"$$\n",
"\\delta_j = \\sum_{i=-\\infty}^{i=\\infty}\\alpha_i\\beta_{j-i}=(\\alpha * \\beta)_j,\n",
@@ -567,9 +505,7 @@
{
"cell_type": "markdown",
"id": "2613b8a5",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"or as a double sum with restriction $l=i+j$"
]
@@ -577,9 +513,7 @@
{
"cell_type": "markdown",
"id": "d41c3500",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"$$\n",
"\\delta_l = \\sum_{ij}\\alpha_i\\beta_{j}.\n",
@@ -589,9 +523,7 @@
{
"cell_type": "markdown",
"id": "1568b770",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"Do you see a potential drawback with these equations?"
]
@@ -599,9 +531,7 @@
{
"cell_type": "markdown",
"id": "a1a0651f",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## A more efficient way of coding the above Convolution\n",
"\n",
@@ -613,9 +543,7 @@
{
"cell_type": "markdown",
"id": "86a7a43b",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"$$\n",
"\\boldsymbol{\\delta}=\\begin{bmatrix}\\alpha_0 & 0 & 0 & 0 \\\\\n",
@@ -631,9 +559,7 @@
{
"cell_type": "markdown",
"id": "683b8af2",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"The process is commutative and we can easily see that we can rewrite the multiplication in terms of a matrix holding $\\beta$ and a vector holding $\\alpha$.\n",
"In this case we have"
@@ -642,9 +568,7 @@
{
"cell_type": "markdown",
"id": "6137c505",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"$$\n",
"\\boldsymbol{\\delta}=\\begin{bmatrix}\\beta_0 & 0 & 0 \\\\\n",
@@ -660,9 +584,7 @@
{
"cell_type": "markdown",
"id": "eb91fb37",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"Note that the use of these matrices is for mathematical purposes only and not implementation purposes.\n",
"When implementing the above equation we do not encode (and allocate memory) the matrices explicitely.\n",
@@ -674,9 +596,7 @@
{
"cell_type": "markdown",
"id": "aabbac3a",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)\n",
"\n",
@@ -686,9 +606,7 @@
{
"cell_type": "markdown",
"id": "4fe1f6a7",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"$$\n",
"m\\frac{d^2x}{dt^2}+\\eta\\frac{dx}{dt}+x(t)=F(t),\n",
@@ -698,9 +616,7 @@
{
"cell_type": "markdown",
"id": "44c67471",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"where $F(t)$ is an applied external force acting on the system (often called a driving force), one can use the theory of Fourier transformations to find the solutions of this type of equations.\n",
"\n",
@@ -712,9 +628,7 @@
{
"cell_type": "markdown",
"id": "76d28174",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"\n",
"\n",
@@ -730,9 +644,7 @@
{
"cell_type": "markdown",
"id": "e86171f0",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## Principle of Superposition\n",
"\n",
@@ -751,9 +663,7 @@
{
"cell_type": "markdown",
"id": "680dd1f2",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"$$\n",
"\\begin{eqnarray}\n",
@@ -765,9 +675,7 @@
{
"cell_type": "markdown",
"id": "513afa38",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"One example of a non-sinusoidal periodic force is a square wave. Many\n",
"components in electric circuits are non-linear, e.g. diodes, which\n",
@@ -778,9 +686,7 @@
{
"cell_type": "markdown",
"id": "ae689c51",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## Simple Code Example\n",
"\n",
@@ -791,10 +697,7 @@
"cell_type": "code",
"execution_count": 1,
"id": "f555270a",
- "metadata": {
- "collapsed": false,
- "editable": true
- },
+ "metadata": {},
"outputs": [],
"source": [
"%matplotlib inline\n",
@@ -821,9 +724,7 @@
{
"cell_type": "markdown",
"id": "16b8d140",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"For the sinusoidal example the\n",
"period is $\\tau=2\\pi/\\omega$. However, higher harmonics can also\n",
@@ -835,9 +736,7 @@
{
"cell_type": "markdown",
"id": "ad49e93d",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"\n",
"\n",
@@ -853,9 +752,7 @@
{
"cell_type": "markdown",
"id": "3eece618",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## Wrapping up Fourier transforms\n",
"\n",
@@ -868,9 +765,7 @@
{
"cell_type": "markdown",
"id": "d7d5881a",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"\n",
"\n",
@@ -886,9 +781,7 @@
{
"cell_type": "markdown",
"id": "83b03221",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"The solutions for $x(t)$ then come from replacing $\\omega$ with\n",
"$n\\omega$ for each term in the particular solution,"
@@ -897,9 +790,7 @@
{
"cell_type": "markdown",
"id": "010f1781",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"$$\n",
"\\begin{eqnarray}\n",
@@ -917,9 +808,7 @@
{
"cell_type": "markdown",
"id": "f02aa278",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## Finding the Coefficients\n",
"\n",
@@ -936,9 +825,7 @@
{
"cell_type": "markdown",
"id": "b11ae77a",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"\n",
"\n",
@@ -956,9 +843,7 @@
{
"cell_type": "markdown",
"id": "f874de4b",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"To check the consistency of these expressions and to verify\n",
"Eq. ([4](#eq:fourierdef2)), one can insert the expansion of $F(t)$ in\n",
@@ -969,9 +854,7 @@
{
"cell_type": "markdown",
"id": "597579d6",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"$$\n",
"\\begin{eqnarray}\n",
@@ -985,9 +868,7 @@
{
"cell_type": "markdown",
"id": "4e055320",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"Immediately, one can throw away all the terms with $g_m$ because they\n",
"convolute an even and an odd function. The term with $f_0/2$\n",
@@ -1002,9 +883,7 @@
{
"cell_type": "markdown",
"id": "a133e3b1",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"\n",
"\n",
@@ -1020,9 +899,7 @@
{
"cell_type": "markdown",
"id": "c14919a7",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"and"
]
@@ -1030,9 +907,7 @@
{
"cell_type": "markdown",
"id": "5650b08e",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"$$\n",
"\\begin{eqnarray}\n",
@@ -1046,9 +921,7 @@
{
"cell_type": "markdown",
"id": "91006368",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"The same method can be used to check for the consistency of $g_n$."
]
@@ -1056,9 +929,7 @@
{
"cell_type": "markdown",
"id": "c1b5a978",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## Final words on Fourier Transforms\n",
"\n",
@@ -1075,10 +946,7 @@
"cell_type": "code",
"execution_count": 2,
"id": "021490c8",
- "metadata": {
- "collapsed": false,
- "editable": true
- },
+ "metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
@@ -1115,9 +983,7 @@
{
"cell_type": "markdown",
"id": "b8006631",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## Two-dimensional Objects\n",
"\n",
@@ -1129,9 +995,7 @@
{
"cell_type": "markdown",
"id": "de188b6b",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"$$\n",
"S_(i,j)=(I * K)(i,j) = \\sum_m\\sum_n I(m,n)K(i-m,j-n).\n",
@@ -1141,9 +1005,7 @@
{
"cell_type": "markdown",
"id": "ca5e4985",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"Convolution is a commutatitave process, which means we can rewrite this equation as"
]
@@ -1151,9 +1013,7 @@
{
"cell_type": "markdown",
"id": "e3baf95c",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"$$\n",
"S_(i,j)=(I * K)(i,j) = \\sum_m\\sum_n I(i-m,j-n)K(m,n).\n",
@@ -1163,9 +1023,7 @@
{
"cell_type": "markdown",
"id": "b1aff500",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"Normally the latter is more straightforward to implement in a machine elarning library since there is less variation in the range of values of $m$ and $n$."
]
@@ -1173,9 +1031,7 @@
{
"cell_type": "markdown",
"id": "0adf4a73",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## Cross-Correlation\n",
"\n",
@@ -1185,9 +1041,7 @@
{
"cell_type": "markdown",
"id": "943b3641",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"$$\n",
"S_(i,j)=(I * K)(i,j) = \\sum_m\\sum_n I(i+m,j-+)K(m,n).\n",
@@ -1197,9 +1051,7 @@
{
"cell_type": "markdown",
"id": "d4a4565f",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## More on Dimensionalities\n",
"\n",
@@ -1224,9 +1076,7 @@
{
"cell_type": "markdown",
"id": "90c5983f",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"$$\n",
"\\mathrm{NumberParameters}=10^{10}+10^4+10^4+1 \\approx 10^{10},\n",
@@ -1236,9 +1086,7 @@
{
"cell_type": "markdown",
"id": "b9dc706b",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"that is ten billion parameters to determine."
]
@@ -1246,9 +1094,7 @@
{
"cell_type": "markdown",
"id": "4e700e3b",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## Further Dimensionality Remarks\n",
"\n",
@@ -1272,9 +1118,7 @@
{
"cell_type": "markdown",
"id": "76a62fa2",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## CNNs in more detail, Lecture from IN5400\n",
"\n",
@@ -1284,9 +1128,7 @@
{
"cell_type": "markdown",
"id": "f70c1ce4",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## CNNs in more detail, building convolutional neural networks in Tensorflow and Keras\n",
"\n",
@@ -1303,9 +1145,7 @@
{
"cell_type": "markdown",
"id": "8ad25ed3",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## Setting it up\n",
"\n",
@@ -1316,9 +1156,7 @@
{
"cell_type": "markdown",
"id": "5a56eff1",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"$$\n",
"(n_{inputs},\\, n_{pixels, width},\\, n_{pixels, height},\\, depth) .\n",
@@ -1328,9 +1166,7 @@
{
"cell_type": "markdown",
"id": "2302c58c",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## The MNIST dataset again\n",
"\n",
@@ -1349,9 +1185,7 @@
{
"cell_type": "markdown",
"id": "b24c6544",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## Strong correlations\n",
"\n",
@@ -1370,9 +1204,7 @@
{
"cell_type": "markdown",
"id": "1e4682a2",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## Layers of a CNN\n",
"The layers of a convolutional neural network arrange neurons in 3D: width, height and depth. \n",
@@ -1395,9 +1227,7 @@
{
"cell_type": "markdown",
"id": "317a594e",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## Systematic reduction\n",
"\n",
@@ -1414,9 +1244,7 @@
{
"cell_type": "markdown",
"id": "5bcbc7de",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## Prerequisites: Collect and pre-process data"
]
@@ -1425,10 +1253,7 @@
"cell_type": "code",
"execution_count": 3,
"id": "32c6d683",
- "metadata": {
- "collapsed": false,
- "editable": true
- },
+ "metadata": {},
"outputs": [],
"source": [
"# import necessary packages\n",
@@ -1476,9 +1301,7 @@
{
"cell_type": "markdown",
"id": "10efc6e2",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## Importing Keras and Tensorflow"
]
@@ -1487,10 +1310,7 @@
"cell_type": "code",
"execution_count": 4,
"id": "d8b243fe",
- "metadata": {
- "collapsed": false,
- "editable": true
- },
+ "metadata": {},
"outputs": [],
"source": [
"from tensorflow.keras import datasets, layers, models\n",
@@ -1520,9 +1340,7 @@
{
"cell_type": "markdown",
"id": "8231a073",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## Running with Keras"
]
@@ -1531,10 +1349,7 @@
"cell_type": "code",
"execution_count": 5,
"id": "e2d8b11c",
- "metadata": {
- "collapsed": false,
- "editable": true
- },
+ "metadata": {},
"outputs": [],
"source": [
"def create_convolutional_neural_network_keras(input_shape, receptive_field,\n",
@@ -1568,9 +1383,7 @@
{
"cell_type": "markdown",
"id": "1c599535",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## Final part"
]
@@ -1579,10 +1392,7 @@
"cell_type": "code",
"execution_count": 6,
"id": "c7718d55",
- "metadata": {
- "collapsed": false,
- "editable": true
- },
+ "metadata": {},
"outputs": [],
"source": [
"CNN_keras = np.zeros((len(eta_vals), len(lmbd_vals)), dtype=object)\n",
@@ -1606,9 +1416,7 @@
{
"cell_type": "markdown",
"id": "b02c6452",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## Final visualization"
]
@@ -1617,10 +1425,7 @@
"cell_type": "code",
"execution_count": 7,
"id": "ba14f397",
- "metadata": {
- "collapsed": false,
- "editable": true
- },
+ "metadata": {},
"outputs": [],
"source": [
"# visual representation of grid search\n",
@@ -1658,9 +1463,7 @@
{
"cell_type": "markdown",
"id": "8f433eb8",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## The CIFAR01 data set\n",
"\n",
@@ -1674,10 +1477,7 @@
"cell_type": "code",
"execution_count": 8,
"id": "ab03b0e5",
- "metadata": {
- "collapsed": false,
- "editable": true
- },
+ "metadata": {},
"outputs": [],
"source": [
"import tensorflow as tf\n",
@@ -1695,9 +1495,7 @@
{
"cell_type": "markdown",
"id": "bfba77ad",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## Verifying the data set\n",
"\n",
@@ -1708,10 +1506,7 @@
"cell_type": "code",
"execution_count": 9,
"id": "bcde11c7",
- "metadata": {
- "collapsed": false,
- "editable": true
- },
+ "metadata": {},
"outputs": [],
"source": [
"class_names = ['airplane', 'automobile', 'bird', 'cat', 'deer',\n",
@@ -1733,9 +1528,7 @@
{
"cell_type": "markdown",
"id": "cf5bc34e",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## Set up the model\n",
"\n",
@@ -1748,10 +1541,7 @@
"cell_type": "code",
"execution_count": 10,
"id": "e0ded37d",
- "metadata": {
- "collapsed": false,
- "editable": true
- },
+ "metadata": {},
"outputs": [],
"source": [
"model = models.Sequential()\n",
@@ -1769,9 +1559,7 @@
{
"cell_type": "markdown",
"id": "6eb288a2",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"You can see that the output of every Conv2D and MaxPooling2D layer is a 3D tensor of shape (height, width, channels). The width and height dimensions tend to shrink as you go deeper in the network. The number of output channels for each Conv2D layer is controlled by the first argument (e.g., 32 or 64). Typically, as the width and height shrink, you can afford (computationally) to add more output channels in each Conv2D layer."
]
@@ -1779,9 +1567,7 @@
{
"cell_type": "markdown",
"id": "ff6eb6ef",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## Add Dense layers on top\n",
"\n",
@@ -1798,10 +1584,7 @@
"cell_type": "code",
"execution_count": 11,
"id": "a2b050dc",
- "metadata": {
- "collapsed": false,
- "editable": true
- },
+ "metadata": {},
"outputs": [],
"source": [
"model.add(layers.Flatten())\n",
@@ -1815,9 +1598,7 @@
{
"cell_type": "markdown",
"id": "01af49fe",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"As you can see, our (4, 4, 64) outputs were flattened into vectors of shape (1024) before going through two Dense layers."
]
@@ -1825,9 +1606,7 @@
{
"cell_type": "markdown",
"id": "5293bdae",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## Compile and train the model"
]
@@ -1836,10 +1615,7 @@
"cell_type": "code",
"execution_count": 12,
"id": "4adcb51c",
- "metadata": {
- "collapsed": false,
- "editable": true
- },
+ "metadata": {},
"outputs": [],
"source": [
"model.compile(optimizer='adam',\n",
@@ -1853,9 +1629,7 @@
{
"cell_type": "markdown",
"id": "60534f56",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## Finally, evaluate the model"
]
@@ -1864,10 +1638,7 @@
"cell_type": "code",
"execution_count": 13,
"id": "396b9a0a",
- "metadata": {
- "collapsed": false,
- "editable": true
- },
+ "metadata": {},
"outputs": [],
"source": [
"plt.plot(history.history['accuracy'], label='accuracy')\n",
@@ -1885,9 +1656,7 @@
{
"cell_type": "markdown",
"id": "117079e0",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"## Building our own CNN code\n",
"\n",
@@ -1916,9 +1685,7 @@
{
"cell_type": "markdown",
"id": "4387296c",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"### List of contents:\n",
"\n",
@@ -1940,9 +1707,7 @@
{
"cell_type": "markdown",
"id": "6051bce6",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"### Schedulers\n",
"\n",
@@ -1961,12 +1726,9 @@
},
{
"cell_type": "code",
- "execution_count": 14,
+ "execution_count": 1,
"id": "b687d5af",
- "metadata": {
- "collapsed": false,
- "editable": true
- },
+ "metadata": {},
"outputs": [],
"source": [
"import autograd.numpy as np\n",
@@ -2104,9 +1866,7 @@
{
"cell_type": "markdown",
"id": "ab9955ee",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"### Usage of schedulers\n",
"\n",
@@ -2115,12 +1875,9 @@
},
{
"cell_type": "code",
- "execution_count": 15,
+ "execution_count": 2,
"id": "99c69ccb",
- "metadata": {
- "collapsed": false,
- "editable": true
- },
+ "metadata": {},
"outputs": [],
"source": [
"momentum_scheduler = Momentum(eta=1e-3, momentum=0.9)\n",
@@ -2130,22 +1887,33 @@
{
"cell_type": "markdown",
"id": "02508975",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"Here is a small example for how a segment of code using schedulers could look. Switching out the schedulers is simple."
]
},
{
"cell_type": "code",
- "execution_count": 16,
+ "execution_count": 3,
"id": "5ac22cfd",
- "metadata": {
- "collapsed": false,
- "editable": true
- },
- "outputs": [],
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Before scheduler:\n",
+ "weights=array([[1., 1., 1.],\n",
+ " [1., 1., 1.],\n",
+ " [1., 1., 1.]])\n",
+ "\n",
+ "After scheduler:\n",
+ "weights=array([[0.993993 , 0.99399301, 0.99399301],\n",
+ " [0.993993 , 0.993993 , 0.99399301],\n",
+ " [0.99399301, 0.99399301, 0.993993 ]])\n"
+ ]
+ }
+ ],
"source": [
"weights = np.ones((3,3))\n",
"print(f\"Before scheduler:\\n{weights=}\")\n",
@@ -2163,9 +1931,7 @@
{
"cell_type": "markdown",
"id": "233c256f",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"### Cost functions\n",
"\n",
@@ -2177,12 +1943,9 @@
},
{
"cell_type": "code",
- "execution_count": 17,
+ "execution_count": 4,
"id": "27a2af46",
- "metadata": {
- "collapsed": false,
- "editable": true
- },
+ "metadata": {},
"outputs": [],
"source": [
"def CostOLS(target):\n",
@@ -2226,9 +1989,7 @@
{
"cell_type": "markdown",
"id": "48ae2ab4",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"### Usage of cost functions\n",
"\n",
@@ -2239,13 +2000,21 @@
},
{
"cell_type": "code",
- "execution_count": 18,
+ "execution_count": 5,
"id": "99302d7b",
- "metadata": {
- "collapsed": false,
- "editable": true
- },
- "outputs": [],
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Derivative of cost function CostCrossEntropy valued at a:\n",
+ "[[-0.08333333]\n",
+ " [-0.13333333]\n",
+ " [-0.16666667]]\n"
+ ]
+ }
+ ],
"source": [
"from autograd import grad\n",
"\n",
@@ -2262,9 +2031,7 @@
{
"cell_type": "markdown",
"id": "beb76a0e",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"### Activation functions\n",
"\n",
@@ -2277,12 +2044,9 @@
},
{
"cell_type": "code",
- "execution_count": 19,
+ "execution_count": 6,
"id": "057b4aac",
- "metadata": {
- "collapsed": false,
- "editable": true
- },
+ "metadata": {},
"outputs": [],
"source": [
"\n",
@@ -2338,9 +2102,7 @@
{
"cell_type": "markdown",
"id": "e5e39beb",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"### Usage of activation functions\n",
"\n",
@@ -2353,13 +2115,31 @@
},
{
"cell_type": "code",
- "execution_count": 20,
+ "execution_count": 7,
"id": "dca3e6a9",
- "metadata": {
- "collapsed": false,
- "editable": true
- },
- "outputs": [],
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Input to activation function:\n",
+ "[[4]\n",
+ " [5]\n",
+ " [6]]\n",
+ "\n",
+ "Output from sigmoid activation function:\n",
+ "[[0.98201379]\n",
+ " [0.99330715]\n",
+ " [0.99752738]]\n",
+ "\n",
+ "Derivative of sigmoid activation function valued at z:\n",
+ "[[0.19824029]\n",
+ " [0.19721923]\n",
+ " [0.19683648]]\n"
+ ]
+ }
+ ],
"source": [
"z = np.array([[4, 5, 6]]).T\n",
"print(f\"Input to activation function:\\n{z}\")\n",
@@ -2376,9 +2156,7 @@
{
"cell_type": "markdown",
"id": "41430661",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"### Convolution\n",
"\n",
@@ -2393,9 +2171,7 @@
{
"cell_type": "markdown",
"id": "815920b5",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"$$\n",
"(f \\ast g)(t):=\\int_{-\\infty}^{\\infty} f(\\tau) g(t-\\tau) d \\tau.\n",
@@ -2405,9 +2181,7 @@
{
"cell_type": "markdown",
"id": "826ae26d",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"Here, f and g are the two functions on which we want to perform an\n",
"operation. The outcome of the convolution operation is represented by\n",
@@ -2421,9 +2195,7 @@
{
"cell_type": "markdown",
"id": "ae984b90",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"$$\n",
"(f \\ast g)[n]=\\sum_{m=0}^{n-1} f[m] g[n-m].\n",
@@ -2433,9 +2205,7 @@
{
"cell_type": "markdown",
"id": "c74ad806",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"The key idea we utilize to extract the information contained in an\n",
"image is to slide an $m \\times n$ matrix *g* over an $m \\times n$\n",
@@ -2448,9 +2218,7 @@
{
"cell_type": "markdown",
"id": "91b53f99",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"$$\n",
"(f \\ast g)[i, j]\\sum_{m=0}^{M-1}\\sum_{n=0}^{N-1} f[m,n] g[i-m, j-n].\n",
@@ -2460,9 +2228,7 @@
{
"cell_type": "markdown",
"id": "404c8cac",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"It is imperative to note that the size of the kernel g is\n",
"significantly smaller than the size of the input image f, thereby\n",
@@ -2478,9 +2244,7 @@
{
"cell_type": "markdown",
"id": "dcbbd719",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"$$\n",
"f = \\begin{bmatrix}\n",
@@ -2497,9 +2261,7 @@
{
"cell_type": "markdown",
"id": "823269cb",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"and a $3 \\times 3$ kernel *g* called a low-pass filter. Note that the\n",
"kernel is usually rotated by 180 degrees during convolution, however\n",
@@ -2509,9 +2271,7 @@
{
"cell_type": "markdown",
"id": "c251f583",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"$$\n",
"g = \\frac{1}{9}\n",
@@ -2526,9 +2286,7 @@
{
"cell_type": "markdown",
"id": "41218fa0",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"In order to filter the image, we have to extract a $3 \\times 3$\n",
"element from the upper left corner of *f*, and perform element-wise\n",
@@ -2539,9 +2297,7 @@
{
"cell_type": "markdown",
"id": "fd6333be",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"$$\n",
"\\begin{bmatrix}\n",
@@ -2568,9 +2324,7 @@
{
"cell_type": "markdown",
"id": "990703e2",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"Then, following the multiplication, we summarize all the elements of the resulting matrix A:"
]
@@ -2578,9 +2332,7 @@
{
"cell_type": "markdown",
"id": "9f2e335f",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"$$\n",
"(f \\ast g)[0, 0]= \\sum_{i=0}^{2} \\sum_{j=0}^{2} a_{i,j} = 5\n",
@@ -2590,9 +2342,7 @@
{
"cell_type": "markdown",
"id": "7e6ceca6",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"Which corresponds to the first element of the filtered image $(f \\ast g)$.\n",
"\n",
@@ -2611,9 +2361,7 @@
{
"cell_type": "markdown",
"id": "023394e0",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"$$\n",
"(f \\ast g) =\n",
@@ -2629,9 +2377,7 @@
{
"cell_type": "markdown",
"id": "9a974887",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"The result is markedly smaller in shape than the original image. This occurs when using convolution without first padding the image with additional columns and rows, allowing us to keep the original image shape after sliding the kernel over the image.\n",
"How many rows and columns we wish to pad the image with depends strictly on the shape of the kernel, as we wish to pad the image with *r* additional rows and *c* additional columns."
@@ -2640,9 +2386,7 @@
{
"cell_type": "markdown",
"id": "d0fcdb54",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"$$\n",
"r =\\lfloor \\frac{kernel\\ height}{2} \\rfloor \\cdot 2 \\\\\n",
@@ -2653,9 +2397,7 @@
{
"cell_type": "markdown",
"id": "70eeff71",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"Note the notation $\\lfloor \\frac{kernel width}{2} \\rfloor$ means that\n",
"we floor the result of the division, meaning we round down to a whole\n",
@@ -2677,9 +2419,7 @@
{
"cell_type": "markdown",
"id": "da8af146",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"$$\n",
"\\begin{bmatrix}\n",
@@ -2699,21 +2439,16 @@
{
"cell_type": "markdown",
"id": "ae69a9ad",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"Below we have provided code that demonstrates padding and convolution. As you will see when we run the code, the size of the image will remain unchanged when using padding.~"
]
},
{
"cell_type": "code",
- "execution_count": 21,
+ "execution_count": 8,
"id": "e6bf3cc7",
- "metadata": {
- "collapsed": false,
- "editable": true
- },
+ "metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
@@ -2792,9 +2527,7 @@
{
"cell_type": "markdown",
"id": "2f4eeaae",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"Fun fact: When filtering images, you will see that convolution involves rotating the kernel by 180 degrees. \n",
"However, this is not the case when applying convolution in a CNN, where the same operation not rotated by 180 degrees is called \n",
@@ -2803,13 +2536,19 @@
},
{
"cell_type": "code",
- "execution_count": 22,
+ "execution_count": 9,
"id": "97f47d4f",
- "metadata": {
- "collapsed": false,
- "editable": true
- },
- "outputs": [],
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "original_image.shape=(6, 6)\n",
+ "convolved_image.shape=(6, 6)\n"
+ ]
+ }
+ ],
"source": [
"\n",
"original_image = np.array([[4, 1, 2, 9, 8, 6],\n",
@@ -2832,9 +2571,7 @@
{
"cell_type": "markdown",
"id": "31222f43",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"As you can see, the resulting image is of the same size as the\n",
"original image. To round of our demonstration of convolution, we will\n",
@@ -2851,13 +2588,26 @@
},
{
"cell_type": "code",
- "execution_count": 23,
+ "execution_count": 10,
"id": "430389e0",
- "metadata": {
- "collapsed": false,
- "editable": true
- },
- "outputs": [],
+ "metadata": {},
+ "outputs": [
+ {
+ "ename": "FileNotFoundError",
+ "evalue": "No such file: '/Users/mhjensen/Teaching/MachineLearning/doc/pub/week44/data/IMG-2167.JPG'",
+ "output_type": "error",
+ "traceback": [
+ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
+ "\u001b[0;31mFileNotFoundError\u001b[0m Traceback (most recent call last)",
+ "Input \u001b[0;32mIn [10]\u001b[0m, in \u001b[0;36m\u001b[0;34m()\u001b[0m\n\u001b[1;32m 13\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m g, ker_coef\n\u001b[1;32m 16\u001b[0m img_path \u001b[38;5;241m=\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m../data/IMG-2167.JPG\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[0;32m---> 17\u001b[0m image_of_cute_dog \u001b[38;5;241m=\u001b[39m \u001b[43mimageio\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mimread\u001b[49m\u001b[43m(\u001b[49m\u001b[43mimg_path\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mmode\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43mL\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[43m)\u001b[49m\n\u001b[1;32m 19\u001b[0m plt\u001b[38;5;241m.\u001b[39mimshow(image_of_cute_dog, cmap\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mgray\u001b[39m\u001b[38;5;124m\"\u001b[39m, vmin\u001b[38;5;241m=\u001b[39m\u001b[38;5;241m0\u001b[39m, vmax\u001b[38;5;241m=\u001b[39m\u001b[38;5;241m255\u001b[39m, aspect\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mauto\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n\u001b[1;32m 20\u001b[0m plt\u001b[38;5;241m.\u001b[39mtitle(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mOriginal image\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n",
+ "File \u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/imageio/v3.py:53\u001b[0m, in \u001b[0;36mimread\u001b[0;34m(uri, index, plugin, extension, format_hint, **kwargs)\u001b[0m\n\u001b[1;32m 50\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m index \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[1;32m 51\u001b[0m call_kwargs[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mindex\u001b[39m\u001b[38;5;124m\"\u001b[39m] \u001b[38;5;241m=\u001b[39m index\n\u001b[0;32m---> 53\u001b[0m \u001b[38;5;28;01mwith\u001b[39;00m \u001b[43mimopen\u001b[49m\u001b[43m(\u001b[49m\u001b[43muri\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mr\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mplugin_kwargs\u001b[49m\u001b[43m)\u001b[49m \u001b[38;5;28;01mas\u001b[39;00m img_file:\n\u001b[1;32m 54\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m np\u001b[38;5;241m.\u001b[39masarray(img_file\u001b[38;5;241m.\u001b[39mread(\u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mcall_kwargs))\n",
+ "File \u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/imageio/core/imopen.py:113\u001b[0m, in \u001b[0;36mimopen\u001b[0;34m(uri, io_mode, plugin, extension, format_hint, legacy_mode, **kwargs)\u001b[0m\n\u001b[1;32m 111\u001b[0m request\u001b[38;5;241m.\u001b[39mformat_hint \u001b[38;5;241m=\u001b[39m format_hint\n\u001b[1;32m 112\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[0;32m--> 113\u001b[0m request \u001b[38;5;241m=\u001b[39m \u001b[43mRequest\u001b[49m\u001b[43m(\u001b[49m\u001b[43muri\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mio_mode\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mformat_hint\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mformat_hint\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mextension\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mextension\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 115\u001b[0m source \u001b[38;5;241m=\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m\u001b[39m\u001b[38;5;124m\"\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(uri, \u001b[38;5;28mbytes\u001b[39m) \u001b[38;5;28;01melse\u001b[39;00m uri\n\u001b[1;32m 117\u001b[0m \u001b[38;5;66;03m# fast-path based on plugin\u001b[39;00m\n\u001b[1;32m 118\u001b[0m \u001b[38;5;66;03m# (except in legacy mode)\u001b[39;00m\n",
+ "File \u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/imageio/core/request.py:247\u001b[0m, in \u001b[0;36mRequest.__init__\u001b[0;34m(self, uri, mode, extension, format_hint, **kwargs)\u001b[0m\n\u001b[1;32m 244\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mValueError\u001b[39;00m(\u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mInvalid Request.Mode: \u001b[39m\u001b[38;5;132;01m{\u001b[39;00mmode\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m\"\u001b[39m)\n\u001b[1;32m 246\u001b[0m \u001b[38;5;66;03m# Parse what was given\u001b[39;00m\n\u001b[0;32m--> 247\u001b[0m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_parse_uri\u001b[49m\u001b[43m(\u001b[49m\u001b[43muri\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 249\u001b[0m \u001b[38;5;66;03m# Set extension\u001b[39;00m\n\u001b[1;32m 250\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m extension \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n",
+ "File \u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/imageio/core/request.py:407\u001b[0m, in \u001b[0;36mRequest._parse_uri\u001b[0;34m(self, uri)\u001b[0m\n\u001b[1;32m 404\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m is_read_request:\n\u001b[1;32m 405\u001b[0m \u001b[38;5;66;03m# Reading: check that the file exists (but is allowed a dir)\u001b[39;00m\n\u001b[1;32m 406\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m os\u001b[38;5;241m.\u001b[39mpath\u001b[38;5;241m.\u001b[39mexists(fn):\n\u001b[0;32m--> 407\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mFileNotFoundError\u001b[39;00m(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mNo such file: \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;132;01m%s\u001b[39;00m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m\"\u001b[39m \u001b[38;5;241m%\u001b[39m fn)\n\u001b[1;32m 408\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m 409\u001b[0m \u001b[38;5;66;03m# Writing: check that the directory to write to does exist\u001b[39;00m\n\u001b[1;32m 410\u001b[0m dn \u001b[38;5;241m=\u001b[39m os\u001b[38;5;241m.\u001b[39mpath\u001b[38;5;241m.\u001b[39mdirname(fn)\n",
+ "\u001b[0;31mFileNotFoundError\u001b[0m: No such file: '/Users/mhjensen/Teaching/MachineLearning/doc/pub/week44/data/IMG-2167.JPG'"
+ ]
+ }
+ ],
"source": [
"# Now an example using a real image and first a gaussian low-pass filter and then a sobel filter\n",
"import numpy as np\n",
@@ -2903,9 +2653,7 @@
{
"cell_type": "markdown",
"id": "5df25adc",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"### Layers\n",
"\n",
@@ -2916,12 +2664,9 @@
},
{
"cell_type": "code",
- "execution_count": 24,
+ "execution_count": 11,
"id": "96ef1a9d",
- "metadata": {
- "collapsed": false,
- "editable": true
- },
+ "metadata": {},
"outputs": [],
"source": [
"import math\n",
@@ -2959,9 +2704,7 @@
{
"cell_type": "markdown",
"id": "7b3820e4",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"### Convolution2DLayer: convolution in a hidden layer\n",
"\n",
@@ -2999,12 +2742,9 @@
},
{
"cell_type": "code",
- "execution_count": 25,
+ "execution_count": 12,
"id": "8775922b",
- "metadata": {
- "collapsed": false,
- "editable": true
- },
+ "metadata": {},
"outputs": [],
"source": [
"class Convolution2DLayer(Layer):\n",
@@ -3287,9 +3027,7 @@
{
"cell_type": "markdown",
"id": "730b8338",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"### Backpropagation in the convolutional layer\n",
"\n",
@@ -3310,9 +3048,7 @@
{
"cell_type": "markdown",
"id": "54d1a07d",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"### Demonstration\n",
"\n",
@@ -3321,13 +3057,26 @@
},
{
"cell_type": "code",
- "execution_count": 26,
+ "execution_count": 13,
"id": "ac10f107",
- "metadata": {
- "collapsed": false,
- "editable": true
- },
- "outputs": [],
+ "metadata": {},
+ "outputs": [
+ {
+ "ename": "FileNotFoundError",
+ "evalue": "No such file: '/Users/mhjensen/Teaching/MachineLearning/doc/pub/week44/data/IMG-2167.JPG'",
+ "output_type": "error",
+ "traceback": [
+ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
+ "\u001b[0;31mFileNotFoundError\u001b[0m Traceback (most recent call last)",
+ "Input \u001b[0;32mIn [13]\u001b[0m, in \u001b[0;36m| \u001b[0;34m()\u001b[0m\n\u001b[1;32m 29\u001b[0m \u001b[38;5;66;03m# read in image path, make data correct format\u001b[39;00m\n\u001b[1;32m 30\u001b[0m img_path \u001b[38;5;241m=\u001b[39m img_path \u001b[38;5;241m=\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m../data/IMG-2167.JPG\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[0;32m---> 31\u001b[0m image_of_cute_dog \u001b[38;5;241m=\u001b[39m \u001b[43mimageio\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mimread\u001b[49m\u001b[43m(\u001b[49m\u001b[43mimg_path\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 32\u001b[0m image_shape \u001b[38;5;241m=\u001b[39m image_of_cute_dog\u001b[38;5;241m.\u001b[39mshape\n\u001b[1;32m 33\u001b[0m image_of_cute_dog \u001b[38;5;241m=\u001b[39m image_of_cute_dog\u001b[38;5;241m.\u001b[39mreshape(\u001b[38;5;241m1\u001b[39m, image_shape[\u001b[38;5;241m0\u001b[39m], image_shape[\u001b[38;5;241m1\u001b[39m], image_shape[\u001b[38;5;241m2\u001b[39m])\n",
+ "File \u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/imageio/v3.py:53\u001b[0m, in \u001b[0;36mimread\u001b[0;34m(uri, index, plugin, extension, format_hint, **kwargs)\u001b[0m\n\u001b[1;32m 50\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m index \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[1;32m 51\u001b[0m call_kwargs[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mindex\u001b[39m\u001b[38;5;124m\"\u001b[39m] \u001b[38;5;241m=\u001b[39m index\n\u001b[0;32m---> 53\u001b[0m \u001b[38;5;28;01mwith\u001b[39;00m \u001b[43mimopen\u001b[49m\u001b[43m(\u001b[49m\u001b[43muri\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mr\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mplugin_kwargs\u001b[49m\u001b[43m)\u001b[49m \u001b[38;5;28;01mas\u001b[39;00m img_file:\n\u001b[1;32m 54\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m np\u001b[38;5;241m.\u001b[39masarray(img_file\u001b[38;5;241m.\u001b[39mread(\u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mcall_kwargs))\n",
+ "File \u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/imageio/core/imopen.py:113\u001b[0m, in \u001b[0;36mimopen\u001b[0;34m(uri, io_mode, plugin, extension, format_hint, legacy_mode, **kwargs)\u001b[0m\n\u001b[1;32m 111\u001b[0m request\u001b[38;5;241m.\u001b[39mformat_hint \u001b[38;5;241m=\u001b[39m format_hint\n\u001b[1;32m 112\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[0;32m--> 113\u001b[0m request \u001b[38;5;241m=\u001b[39m \u001b[43mRequest\u001b[49m\u001b[43m(\u001b[49m\u001b[43muri\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mio_mode\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mformat_hint\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mformat_hint\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mextension\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mextension\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 115\u001b[0m source \u001b[38;5;241m=\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m\u001b[39m\u001b[38;5;124m\"\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(uri, \u001b[38;5;28mbytes\u001b[39m) \u001b[38;5;28;01melse\u001b[39;00m uri\n\u001b[1;32m 117\u001b[0m \u001b[38;5;66;03m# fast-path based on plugin\u001b[39;00m\n\u001b[1;32m 118\u001b[0m \u001b[38;5;66;03m# (except in legacy mode)\u001b[39;00m\n",
+ "File \u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/imageio/core/request.py:247\u001b[0m, in \u001b[0;36mRequest.__init__\u001b[0;34m(self, uri, mode, extension, format_hint, **kwargs)\u001b[0m\n\u001b[1;32m 244\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mValueError\u001b[39;00m(\u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mInvalid Request.Mode: \u001b[39m\u001b[38;5;132;01m{\u001b[39;00mmode\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m\"\u001b[39m)\n\u001b[1;32m 246\u001b[0m \u001b[38;5;66;03m# Parse what was given\u001b[39;00m\n\u001b[0;32m--> 247\u001b[0m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_parse_uri\u001b[49m\u001b[43m(\u001b[49m\u001b[43muri\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 249\u001b[0m \u001b[38;5;66;03m# Set extension\u001b[39;00m\n\u001b[1;32m 250\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m extension \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n",
+ "File \u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/imageio/core/request.py:407\u001b[0m, in \u001b[0;36mRequest._parse_uri\u001b[0;34m(self, uri)\u001b[0m\n\u001b[1;32m 404\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m is_read_request:\n\u001b[1;32m 405\u001b[0m \u001b[38;5;66;03m# Reading: check that the file exists (but is allowed a dir)\u001b[39;00m\n\u001b[1;32m 406\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m os\u001b[38;5;241m.\u001b[39mpath\u001b[38;5;241m.\u001b[39mexists(fn):\n\u001b[0;32m--> 407\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mFileNotFoundError\u001b[39;00m(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mNo such file: \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;132;01m%s\u001b[39;00m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m\"\u001b[39m \u001b[38;5;241m%\u001b[39m fn)\n\u001b[1;32m 408\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m 409\u001b[0m \u001b[38;5;66;03m# Writing: check that the directory to write to does exist\u001b[39;00m\n\u001b[1;32m 410\u001b[0m dn \u001b[38;5;241m=\u001b[39m os\u001b[38;5;241m.\u001b[39mpath\u001b[38;5;241m.\u001b[39mdirname(fn)\n",
+ "\u001b[0;31mFileNotFoundError\u001b[0m: No such file: '/Users/mhjensen/Teaching/MachineLearning/doc/pub/week44/data/IMG-2167.JPG'"
+ ]
+ }
+ ],
"source": [
"import numpy as np\n",
"import imageio.v3 as imageio\n",
@@ -3371,9 +3120,7 @@
{
"cell_type": "markdown",
"id": "cf95bae4",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"We cobserve that the result has half the pixels on each axis due to\n",
"the fact that we've used a horizontal and vertical stride of 2. The\n",
@@ -3393,9 +3140,7 @@
{
"cell_type": "markdown",
"id": "d02b9c44",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"### Pooling Layer\n",
"\n",
@@ -3415,10 +3160,7 @@
"cell_type": "code",
"execution_count": 27,
"id": "f53d233d",
- "metadata": {
- "collapsed": false,
- "editable": true
- },
+ "metadata": {},
"outputs": [],
"source": [
"class Pooling2DLayer(Layer):\n",
@@ -3577,9 +3319,7 @@
{
"cell_type": "markdown",
"id": "d6d73c7c",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"### Flattening Layer\n",
"\n",
@@ -3598,10 +3338,7 @@
"cell_type": "code",
"execution_count": 28,
"id": "c5ae6332",
- "metadata": {
- "collapsed": false,
- "editable": true
- },
+ "metadata": {},
"outputs": [],
"source": [
"class FlattenLayer(Layer):\n",
@@ -3661,9 +3398,7 @@
{
"cell_type": "markdown",
"id": "09dc88fd",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"### Fully Connected Layers\n",
"\n",
@@ -3685,10 +3420,7 @@
"cell_type": "code",
"execution_count": 29,
"id": "00672cce",
- "metadata": {
- "collapsed": false,
- "editable": true
- },
+ "metadata": {},
"outputs": [],
"source": [
"class FullyConnectedLayer(Layer):\n",
@@ -3913,9 +3645,7 @@
{
"cell_type": "markdown",
"id": "d12f3649",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"### Optimized Convolution2DLayer\n",
"\n",
@@ -3934,10 +3664,7 @@
"cell_type": "code",
"execution_count": 30,
"id": "53a308e9",
- "metadata": {
- "collapsed": false,
- "editable": true
- },
+ "metadata": {},
"outputs": [],
"source": [
"class Convolution2DLayerOPT(Convolution2DLayer):\n",
@@ -4238,9 +3965,7 @@
{
"cell_type": "markdown",
"id": "3729d0f3",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"### The Convolutional Neural Network (CNN)\n",
"\n",
@@ -4251,10 +3976,7 @@
"cell_type": "code",
"execution_count": 31,
"id": "4c63d821",
- "metadata": {
- "collapsed": false,
- "editable": true
- },
+ "metadata": {},
"outputs": [],
"source": [
"import math\n",
@@ -4786,9 +4508,7 @@
{
"cell_type": "markdown",
"id": "6b0ce5da",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"### Usage of CNN code\n",
"\n",
@@ -4805,10 +4525,7 @@
"cell_type": "code",
"execution_count": 32,
"id": "ebb4b539",
- "metadata": {
- "collapsed": false,
- "editable": true
- },
+ "metadata": {},
"outputs": [],
"source": [
"adam_scheduler = Adam(eta=1e-3, rho=0.9, rho2=0.999)\n",
@@ -4818,9 +4535,7 @@
{
"cell_type": "markdown",
"id": "a530c013",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"Now that we have our CNN object, we can begin to add layers to it!\n",
"Many of the add_layer functions have default values, for example\n",
@@ -4834,10 +4549,7 @@
"cell_type": "code",
"execution_count": 33,
"id": "a0dea1a7",
- "metadata": {
- "collapsed": false,
- "editable": true
- },
+ "metadata": {},
"outputs": [],
"source": [
"cnn.add_Convolution2DLayer(\n",
@@ -4860,9 +4572,7 @@
{
"cell_type": "markdown",
"id": "82fbc808",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"Here we have created a CNN with the following architecture:\n",
"\n",
@@ -4884,10 +4594,7 @@
"cell_type": "code",
"execution_count": 34,
"id": "ff126b57",
- "metadata": {
- "collapsed": false,
- "editable": true
- },
+ "metadata": {},
"outputs": [],
"source": [
"from sklearn.datasets import fetch_openml\n",
@@ -4919,9 +4626,7 @@
{
"cell_type": "markdown",
"id": "8bd37284",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"Now we may train our model. Note that we can utilize regularization in\n",
"the CNN by using the lam (lambda) parameter in fit(), and utilize\n",
@@ -4938,10 +4643,7 @@
"cell_type": "code",
"execution_count": 35,
"id": "b273c240",
- "metadata": {
- "collapsed": false,
- "editable": true
- },
+ "metadata": {},
"outputs": [],
"source": [
"scores = cnn.fit(\n",
@@ -4966,9 +4668,7 @@
{
"cell_type": "markdown",
"id": "9f246cc2",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"Considering we only trained the model for 100 epochs without any tuning of the hyperparameters, this result is pretty good.\n",
"\n",
@@ -4995,10 +4695,7 @@
"cell_type": "code",
"execution_count": 36,
"id": "6684df44",
- "metadata": {
- "collapsed": false,
- "editable": true
- },
+ "metadata": {},
"outputs": [],
"source": [
"adam_scheduler = Adam(eta=1e-3, rho=0.9, rho2=0.999)\n",
@@ -5064,9 +4761,7 @@
{
"cell_type": "markdown",
"id": "a20d9002",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"Here we see the use of asymmetrical 1D kernels such as the $7 \\times\n",
"1$ kernel in the first convolutional layer, both max and average\n",
@@ -5082,9 +4777,7 @@
{
"cell_type": "markdown",
"id": "2eac2c7e",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"### Additional Remarks\n",
"\n",
@@ -5106,10 +4799,7 @@
"cell_type": "code",
"execution_count": 37,
"id": "12387ab5",
- "metadata": {
- "collapsed": false,
- "editable": true
- },
+ "metadata": {},
"outputs": [],
"source": [
"def convolve(image, kernel, stride=1):\n",
@@ -5137,9 +4827,7 @@
{
"cell_type": "markdown",
"id": "0a08b11f",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"### Remarks on the speed\n",
"\n",
@@ -5167,9 +4855,7 @@
{
"cell_type": "markdown",
"id": "596d27ad",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"### Convolution using separable kernels"
]
@@ -5178,10 +4864,7 @@
"cell_type": "code",
"execution_count": 38,
"id": "0ab0cb35",
- "metadata": {
- "collapsed": false,
- "editable": true
- },
+ "metadata": {},
"outputs": [],
"source": [
"def conv2DSep(image, kernel, coef, stride=1, pad=\"zero\"):\n",
@@ -5228,9 +4911,7 @@
{
"cell_type": "markdown",
"id": "8896f1b3",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"By taking advantage of the capabilities of separable kernels, we can\n",
"effectively cut the computational expense of filtering an image in\n",
@@ -5249,9 +4930,7 @@
{
"cell_type": "markdown",
"id": "2021cf7f",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"### Convolution in the Fourier domain"
]
@@ -5260,10 +4939,7 @@
"cell_type": "code",
"execution_count": 39,
"id": "27aee28e",
- "metadata": {
- "collapsed": false,
- "editable": true
- },
+ "metadata": {},
"outputs": [],
"source": [
"start_time = time.time()\n",
@@ -5281,9 +4957,7 @@
{
"cell_type": "markdown",
"id": "0b4ae049",
- "metadata": {
- "editable": true
- },
+ "metadata": {},
"source": [
"It is evident that executing convolution in the Fourier domain yields\n",
"the quickest computation time. Nonetheless, one should exercise\n",
@@ -5298,7 +4972,25 @@
]
}
],
- "metadata": {},
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.9.10"
+ }
+ },
"nbformat": 4,
"nbformat_minor": 5
}
| |