lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

correcting slides

Browse Source
This commit is contained in:
Morten Hjorth-Jensen
2023-10-23 07:25:11 +02:00
parent 3fbf55bdc9
commit 36c4cd352b
8 changed files with 266 additions and 420 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/src/week43/exercisesweek43.ipynb
-169
View File
@@ -1,169 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "40653618",
"metadata": {
"editable": true
},
"source": [
"<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)\n",
"doconce format html exercisesweek43.do.txt -->\n",
"<!-- dom:TITLE: Exercises weeks 43 and 44 -->"
]
},
{
"cell_type": "markdown",
"id": "b661b3a0",
"metadata": {
"editable": true
},
"source": [
"# Exercises weeks 43 and 44 \n",
"**October 9-13, 2023**\n",
"\n",
"Date: **Deadline is Sunday November 5 at midnight**\n",
"\n",
"You can hand in the exercises from week 43 and week 44 as one exercise and get a total score of two additional points."
]
},
{
"cell_type": "markdown",
"id": "01487b34",
"metadata": {
"editable": true
},
"source": [
"# Overarching aims of the exercises weeks 43 and 44\n",
"\n",
"The aim of the exercises this week and next week is to get started with writing a neural network code\n",
"of relevance for project 2. \n",
"\n",
"During week 41 we discussed three different types of gates, the\n",
"so-called XOR, the OR and the AND gates. In order to develop a code\n",
"for neural networks, it can be useful to set up a simpler system with\n",
"only two inputs and one output. This can make it easier to debug and\n",
"study the feed forward pass and the back propagation part. In the\n",
"exercise this and next week, we propose to study this system with just\n",
"one hidden layer and two hidden nodes. There is only one output node\n",
"and we can choose to use either a simple regression case (fitting a\n",
"line) or just a binary classification case with the cross-entropy as\n",
"cost function.\n",
"\n",
"Their inputs and outputs can be\n",
"summarized using the following tables, first for the OR gate with\n",
"inputs $x_1$ and $x_2$ and outputs $y$:\n",
"\n",
"<table class=\"dotable\" border=\"1\">\n",
"<thead>\n",
"<tr><th align=\"center\">$x_1$</th> <th align=\"center\">$x_2$</th> <th align=\"center\">$y$</th> </tr>\n",
"</thead>\n",
"<tbody>\n",
"<tr><td align=\"center\"> 0 </td> <td align=\"center\"> 0 </td> <td align=\"center\"> 0 </td> </tr>\n",
"<tr><td align=\"center\"> 0 </td> <td align=\"center\"> 1 </td> <td align=\"center\"> 1 </td> </tr>\n",
"<tr><td align=\"center\"> 1 </td> <td align=\"center\"> 0 </td> <td align=\"center\"> 1 </td> </tr>\n",
"<tr><td align=\"center\"> 1 </td> <td align=\"center\"> 1 </td> <td align=\"center\"> 1 </td> </tr>\n",
"</tbody>\n",
"</table>"
]
},
{
"cell_type": "markdown",
"id": "e8f2df30",
"metadata": {
"editable": true
},
"source": [
"## The AND and XOR Gates\n",
"\n",
"The AND gate is defined as\n",
"\n",
"<table class=\"dotable\" border=\"1\">\n",
"<thead>\n",
"<tr><th align=\"center\">$x_1$</th> <th align=\"center\">$x_2$</th> <th align=\"center\">$y$</th> </tr>\n",
"</thead>\n",
"<tbody>\n",
"<tr><td align=\"center\"> 0 </td> <td align=\"center\"> 0 </td> <td align=\"center\"> 0 </td> </tr>\n",
"<tr><td align=\"center\"> 0 </td> <td align=\"center\"> 1 </td> <td align=\"center\"> 0 </td> </tr>\n",
"<tr><td align=\"center\"> 1 </td> <td align=\"center\"> 0 </td> <td align=\"center\"> 0 </td> </tr>\n",
"<tr><td align=\"center\"> 1 </td> <td align=\"center\"> 1 </td> <td align=\"center\"> 1 </td> </tr>\n",
"</tbody>\n",
"</table>\n",
"\n",
"And finally we have the XOR gate\n",
"\n",
"<table class=\"dotable\" border=\"1\">\n",
"<thead>\n",
"<tr><th align=\"center\">$x_1$</th> <th align=\"center\">$x_2$</th> <th align=\"center\">$y$</th> </tr>\n",
"</thead>\n",
"<tbody>\n",
"<tr><td align=\"center\"> 0 </td> <td align=\"center\"> 0 </td> <td align=\"center\"> 0 </td> </tr>\n",
"<tr><td align=\"center\"> 0 </td> <td align=\"center\"> 1 </td> <td align=\"center\"> 1 </td> </tr>\n",
"<tr><td align=\"center\"> 1 </td> <td align=\"center\"> 0 </td> <td align=\"center\"> 1 </td> </tr>\n",
"<tr><td align=\"center\"> 1 </td> <td align=\"center\"> 1 </td> <td align=\"center\"> 0 </td> </tr>\n",
"</tbody>\n",
"</table>"
]
},
{
"cell_type": "markdown",
"id": "a3d25110",
"metadata": {
"editable": true
},
"source": [
"## Representing the Data Sets\n",
"\n",
"Our design matrix is defined by the input values $x_1$ and $x_2$. Since we have four possible outputs, our design matrix reads"
]
},
{
"cell_type": "markdown",
"id": "abdf765d",
"metadata": {
"editable": true
},
"source": [
"$$\n",
"\\boldsymbol{X}=\\begin{bmatrix} 0 & 0 \\\\\n",
" 0 & 1 \\\\\n",
"\t\t 1 & 0 \\\\\n",
"\t\t 1 & 1 \\end{bmatrix},\n",
"$$"
]
},
{
"cell_type": "markdown",
"id": "684ff136",
"metadata": {
"editable": true
},
"source": [
"while the vector of outputs is $\\boldsymbol{y}^T=[0,1,1,0]$ for the XOR gate, $\\boldsymbol{y}^T=[0,0,0,1]$ for the AND gate and $\\boldsymbol{y}^T=[0,1,1,1]$ for the OR gate.\n",
"\n",
"Your tasks here are\n",
"\n",
"1. Set up the design matrix with the inputs as discussed above and a vector containing the output, the so-called targets. Note that the design matrix is the same for all gates. You need just to define different outputs.\n",
"\n",
"2. Construct a neural network with only one hidden layer and two hidden nodes using the Sigmoid function as activation function.\n",
"\n",
"3. Set up the output layer with only one output node and use again the Sigmoid function as activation function for the output.\n",
"\n",
"4. Initialize the weights and biases and perform a feed forward pass and compare the outputs with the targets.\n",
"\n",
"5. Set up the cost function (cross entropy for classification of binary cases).\n",
"\n",
"6. Calculate the gradients needed for the back propagation part.\n",
"\n",
"7. Use the gradients to train the network in the back propagation part. Think of using automatic differentiation.\n",
"\n",
"8. Train the network and study your results and compare with results obtained either with **scikit-learn** or **TensorFlow**.\n",
"\n",
"Everything you develop here can be used directly into the code for the project."
]
}
],
"metadata": {},
"nbformat": 4,
"nbformat_minor": 5
}
doc/src/week43/week43.do.txt
+2 -2
View File
@@ -46,7 +46,7 @@ o Slides 12-44 at URL":http://cs231n.stanford.edu/slides/2017/cs231n_2017_lectur
===== Material for exercises week 43 and week 44 =====
!split
===== Writing our first neural network code, Testing our code for the OR and XOR gates =====
===== Writing our first neural network code, testing it for the OR and XOR gates =====
During week 41 we discussed three different types of gates, the
so-called XOR, the OR and the AND gates. In order to develop a code
@@ -252,7 +252,7 @@ plt.show()
!ec
How do we interpret these results?