Testing our code for the XOR, OR and AND gates
+ ++Last week we discussed three different types of gates, the so-called +XOR, the OR and the AND gates. Their inputs and outputs can be +summarized using the following tables, first for the OR gate with +inputs \( x_1 \) and \( x_2 \) and outputs \( y \): + +
+
| \( x_1 \) | \( x_2 \) | \( y \) |
| 0 | 0 | 0 |
| 0 | 1 | 1 |
| 1 | 0 | 1 |
| 1 | 1 | 1 |
The AND and XOR Gates
+ ++The AND gate is defined as + +
+
| \( x_1 \) | \( x_2 \) | \( y \) |
| 0 | 0 | 0 |
| 0 | 1 | 0 |
| 1 | 0 | 0 |
| 1 | 1 | 1 |
+And finally we have the XOR gate + +
+
| \( x_1 \) | \( x_2 \) | \( y \) |
| 0 | 0 | 0 |
| 0 | 1 | 1 |
| 1 | 0 | 1 |
| 1 | 1 | 0 |
Representing the Data Sets
+ ++Our design matrix is defined by the input values \( x_1 \) and \( x_2 \). Since we have four possible outputs, our design matrix reads + +
+$$
+\boldsymbol{X}=\begin{bmatrix} 0 & 0 \\
+ 0 & 1 \\
+ 1 & 0 \\
+ 1 & 1 \end{bmatrix},
+$$
+
+
+while the vector of outputs is \( \boldsymbol{y}^T=[0,1,1,0] \) for the XOR gate, \( \boldsymbol{y}^T=[0,0,0,1] \) for the AND gate and \( \boldsymbol{y}^T=[0,1,1,1] \) for the OR gate.
+
Setting up the Neural Network
+ ++We define first our design matrix and the various input vectors. + +
+ + +
"""
+Simple code that tests XOR, OR and AND gates with linear regression
+"""
+
+import numpy as np
+# Design matrix
+X = np.array([ [1, 0, 0], [1, 0, 1], [1, 1, 0],[1, 1, 1]],dtype=np.float64)
+
+# The XOR gate
+yXOR = np.array( [ 0, 1 ,1, 0])
+# The OR gate
+yOR = np.array( [ 0, 1 ,1, 1])
+# The AND gate
+yAND = np.array( [ 0, 0 ,0, 1])
+
+#print(f"The values of theta for the AND gate:{ThetaAND}")
+#print(f"The linear regression prediction for the AND gate:{X @ ThetaAND}")
+Building neural networks in Tensorflow and Keras
@@ -2226,6 +2335,45 @@ Some of these remarks are particular to DNNs, others are shared by all superviseOverarching Views, a personal note
+ ++The author of these lecture notes has an overarching take on many of +the machine learning algorithms we discuss here. + +
+If we wish to understand complex systems, we need to find some +effective degrees of freedom or features that we find essential, +simply in order to reduce the complexity of the systems we are +studying. This leads, in one way or the other to dimensionality +reductions. Most of the Machine Learning methods we encounter deal +with this, whether we opt for a principal component analysis, or +clustering, or convolutional neural networks, or Ridge or Lasso +regression or random forest, yes, perhaps most machine learning +methods at large. + +
+For neural networks and our previous discussion, we have seen that we +in essence end up with matrix-matrix and matrix-vector +multiplications. In all cases, our matrices are dense ones, and the +more data we deal with the larger the dimensionalities of the matrices +and vectors. How can we reduce such dimensionalities? One possible +answer is offered by convolutional neural networks (CNN), as +discussed below. The figure here shows a typical situation of the +reduction of information in an image and is typical of what CNNs +actually end up doing. +
From a Spherical Cow to a real one
+ +
+
+
Convolutional Neural Networks (recognizing images)
diff --git a/doc/pub/week41/html/week41-solarized.html b/doc/pub/week41/html/week41-solarized.html index 2c58ec1d8..93ae88671 100644 --- a/doc/pub/week41/html/week41-solarized.html +++ b/doc/pub/week41/html/week41-solarized.html @@ -127,6 +127,19 @@ div { text-align: justify; text-justify: inter-word; } None, 'scikit-learn-implementation'), ('Visualization', 2, None, 'visualization'), + ('Testing our code for the XOR, OR and AND gates', + 2, + None, + 'testing-our-code-for-the-xor-or-and-and-gates'), + ('The AND and XOR Gates', 2, None, 'the-and-and-xor-gates'), + ('Representing the Data Sets', + 2, + None, + 'representing-the-data-sets'), + ('Setting up the Neural Network', + 2, + None, + 'setting-up-the-neural-network'), ('Building neural networks in Tensorflow and Keras', 2, None, @@ -182,6 +195,14 @@ div { text-align: justify; text-justify: inter-word; } 2, None, 'limitations-of-supervised-learning-with-deep-networks'), + ('Overarching Views, a personal note', + 2, + None, + 'overarching-views-a-personal-note'), + ('From a Spherical Cow to a real one', + 2, + None, + 'from-a-spherical-cow-to-a-real-one'), ('Convolutional Neural Networks (recognizing images)', 2, None, @@ -260,7 +281,7 @@ MathJax.Hub.Config({-
Oct 11, 2021
Oct 13, 2021
@@ -1457,6 +1478,110 @@ plt.show()
+
Testing our code for the XOR, OR and AND gates
+ ++Last week we discussed three different types of gates, the so-called +XOR, the OR and the AND gates. Their inputs and outputs can be +summarized using the following tables, first for the OR gate with +inputs \( x_1 \) and \( x_2 \) and outputs \( y \): + +
+
| \( x_1 \) | \( x_2 \) | \( y \) |
| 0 | 0 | 0 |
| 0 | 1 | 1 |
| 1 | 0 | 1 |
| 1 | 1 | 1 |
+
+
+
The AND and XOR Gates
+ ++The AND gate is defined as + +
+
| \( x_1 \) | \( x_2 \) | \( y \) |
| 0 | 0 | 0 |
| 0 | 1 | 0 |
| 1 | 0 | 0 |
| 1 | 1 | 1 |
+And finally we have the XOR gate + +
+
| \( x_1 \) | \( x_2 \) | \( y \) |
| 0 | 0 | 0 |
| 0 | 1 | 1 |
| 1 | 0 | 1 |
| 1 | 1 | 0 |
+
+
+
Representing the Data Sets
+ ++Our design matrix is defined by the input values \( x_1 \) and \( x_2 \). Since we have four possible outputs, our design matrix reads + +$$ +\boldsymbol{X}=\begin{bmatrix} 0 & 0 \\ + 0 & 1 \\ + 1 & 0 \\ + 1 & 1 \end{bmatrix}, +$$ + +while the vector of outputs is \( \boldsymbol{y}^T=[0,1,1,0] \) for the XOR gate, \( \boldsymbol{y}^T=[0,0,0,1] \) for the AND gate and \( \boldsymbol{y}^T=[0,1,1,1] \) for the OR gate. + +
+
+
+
Setting up the Neural Network
+ ++We define first our design matrix and the various input vectors. + +
+ + +
"""
+Simple code that tests XOR, OR and AND gates with linear regression
+"""
+
+import numpy as np
+# Design matrix
+X = np.array([ [1, 0, 0], [1, 0, 1], [1, 1, 0],[1, 1, 1]],dtype=np.float64)
+
+# The XOR gate
+yXOR = np.array( [ 0, 1 ,1, 0])
+# The OR gate
+yOR = np.array( [ 0, 1 ,1, 1])
+# The AND gate
+yAND = np.array( [ 0, 0 ,0, 1])
+
+#print(f"The values of theta for the AND gate:{ThetaAND}")
+#print(f"The linear regression prediction for the AND gate:{X @ ThetaAND}")
+
+
+
Building neural networks in Tensorflow and Keras
@@ -2205,6 +2330,45 @@ Some of these remarks are particular to DNNs, others are shared by all supervise
+
Overarching Views, a personal note
+ ++The author of these lecture notes has an overarching take on many of +the machine learning algorithms we discuss here. + +
+If we wish to understand complex systems, we need to find some +effective degrees of freedom or features that we find essential, +simply in order to reduce the complexity of the systems we are +studying. This leads, in one way or the other to dimensionality +reductions. Most of the Machine Learning methods we encounter deal +with this, whether we opt for a principal component analysis, or +clustering, or convolutional neural networks, or Ridge or Lasso +regression or random forest, yes, perhaps most machine learning +methods at large. + +
+For neural networks and our previous discussion, we have seen that we +in essence end up with matrix-matrix and matrix-vector +multiplications. In all cases, our matrices are dense ones, and the +more data we deal with the larger the dimensionalities of the matrices +and vectors. How can we reduce such dimensionalities? One possible +answer is offered by convolutional neural networks (CNN), as +discussed below. The figure here shows a typical situation of the +reduction of information in an image and is typical of what CNNs +actually end up doing. + +
+
+
+
From a Spherical Cow to a real one
+ +
+
+ +
+
+
Convolutional Neural Networks (recognizing images)
diff --git a/doc/pub/week41/html/week41.html b/doc/pub/week41/html/week41.html index d676fed29..c152409d4 100644 --- a/doc/pub/week41/html/week41.html +++ b/doc/pub/week41/html/week41.html @@ -132,6 +132,19 @@ div { text-align: justify; text-justify: inter-word; } None, 'scikit-learn-implementation'), ('Visualization', 2, None, 'visualization'), + ('Testing our code for the XOR, OR and AND gates', + 2, + None, + 'testing-our-code-for-the-xor-or-and-and-gates'), + ('The AND and XOR Gates', 2, None, 'the-and-and-xor-gates'), + ('Representing the Data Sets', + 2, + None, + 'representing-the-data-sets'), + ('Setting up the Neural Network', + 2, + None, + 'setting-up-the-neural-network'), ('Building neural networks in Tensorflow and Keras', 2, None, @@ -187,6 +200,14 @@ div { text-align: justify; text-justify: inter-word; } 2, None, 'limitations-of-supervised-learning-with-deep-networks'), + ('Overarching Views, a personal note', + 2, + None, + 'overarching-views-a-personal-note'), + ('From a Spherical Cow to a real one', + 2, + None, + 'from-a-spherical-cow-to-a-real-one'), ('Convolutional Neural Networks (recognizing images)', 2, None, @@ -265,7 +286,7 @@ MathJax.Hub.Config({
-
Oct 11, 2021
Oct 13, 2021
@@ -1462,6 +1483,110 @@ plt.show()
+
Testing our code for the XOR, OR and AND gates
+ ++Last week we discussed three different types of gates, the so-called +XOR, the OR and the AND gates. Their inputs and outputs can be +summarized using the following tables, first for the OR gate with +inputs \( x_1 \) and \( x_2 \) and outputs \( y \): + +
+
| \( x_1 \) | \( x_2 \) | \( y \) |
| 0 | 0 | 0 |
| 0 | 1 | 1 |
| 1 | 0 | 1 |
| 1 | 1 | 1 |
+
+
+
The AND and XOR Gates
+ ++The AND gate is defined as + +
+
| \( x_1 \) | \( x_2 \) | \( y \) |
| 0 | 0 | 0 |
| 0 | 1 | 0 |
| 1 | 0 | 0 |
| 1 | 1 | 1 |
+And finally we have the XOR gate + +
+
| \( x_1 \) | \( x_2 \) | \( y \) |
| 0 | 0 | 0 |
| 0 | 1 | 1 |
| 1 | 0 | 1 |
| 1 | 1 | 0 |
+
+
+
Representing the Data Sets
+ ++Our design matrix is defined by the input values \( x_1 \) and \( x_2 \). Since we have four possible outputs, our design matrix reads + +$$ +\boldsymbol{X}=\begin{bmatrix} 0 & 0 \\ + 0 & 1 \\ + 1 & 0 \\ + 1 & 1 \end{bmatrix}, +$$ + +while the vector of outputs is \( \boldsymbol{y}^T=[0,1,1,0] \) for the XOR gate, \( \boldsymbol{y}^T=[0,0,0,1] \) for the AND gate and \( \boldsymbol{y}^T=[0,1,1,1] \) for the OR gate. + +
+
+
+
Setting up the Neural Network
+ ++We define first our design matrix and the various input vectors. + +
+ + +
"""
+Simple code that tests XOR, OR and AND gates with linear regression
+"""
+
+import numpy as np
+# Design matrix
+X = np.array([ [1, 0, 0], [1, 0, 1], [1, 1, 0],[1, 1, 1]],dtype=np.float64)
+
+# The XOR gate
+yXOR = np.array( [ 0, 1 ,1, 0])
+# The OR gate
+yOR = np.array( [ 0, 1 ,1, 1])
+# The AND gate
+yAND = np.array( [ 0, 0 ,0, 1])
+
+#print(f"The values of theta for the AND gate:{ThetaAND}")
+#print(f"The linear regression prediction for the AND gate:{X @ ThetaAND}")
+
+
+
Building neural networks in Tensorflow and Keras
@@ -2210,6 +2335,45 @@ Some of these remarks are particular to DNNs, others are shared by all supervise
+
Overarching Views, a personal note
+ ++The author of these lecture notes has an overarching take on many of +the machine learning algorithms we discuss here. + +
+If we wish to understand complex systems, we need to find some +effective degrees of freedom or features that we find essential, +simply in order to reduce the complexity of the systems we are +studying. This leads, in one way or the other to dimensionality +reductions. Most of the Machine Learning methods we encounter deal +with this, whether we opt for a principal component analysis, or +clustering, or convolutional neural networks, or Ridge or Lasso +regression or random forest, yes, perhaps most machine learning +methods at large. + +
+For neural networks and our previous discussion, we have seen that we +in essence end up with matrix-matrix and matrix-vector +multiplications. In all cases, our matrices are dense ones, and the +more data we deal with the larger the dimensionalities of the matrices +and vectors. How can we reduce such dimensionalities? One possible +answer is offered by convolutional neural networks (CNN), as +discussed below. The figure here shows a typical situation of the +reduction of information in an image and is typical of what CNNs +actually end up doing. + +
+
+
+
From a Spherical Cow to a real one
+ +
+
+ +
+
+
Convolutional Neural Networks (recognizing images)
diff --git a/doc/pub/week41/ipynb/ipynb-week41-src.tar.gz b/doc/pub/week41/ipynb/ipynb-week41-src.tar.gz index 0eecbf512..f66080706 100644 Binary files a/doc/pub/week41/ipynb/ipynb-week41-src.tar.gz and b/doc/pub/week41/ipynb/ipynb-week41-src.tar.gz differ diff --git a/doc/pub/week41/ipynb/week41.ipynb b/doc/pub/week41/ipynb/week41.ipynb index 763a6c2c6..04023feac 100644 --- a/doc/pub/week41/ipynb/week41.ipynb +++ b/doc/pub/week41/ipynb/week41.ipynb @@ -10,7 +10,7 @@ " \n", "**Morten Hjorth-Jensen**, Department of Physics, University of Oslo and Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University\n", "\n", - "Date: **Oct 11, 2021**\n", + "Date: **Oct 13, 2021**\n", "\n", "Copyright 1999-2021, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license\n", "\n", @@ -1383,6 +1383,112 @@ "plt.show()" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Testing our code for the XOR, OR and AND gates\n", + "\n", + "Last week we discussed three different types of gates, the so-called\n", + "XOR, the OR and the AND gates. Their inputs and outputs can be\n", + "summarized using the following tables, first for the OR gate with\n", + "inputs $x_1$ and $x_2$ and outputs $y$:\n", + "\n", + "
| $x_1$ | $x_2$ | $y$ |
|---|---|---|
| 0 | 0 | 0 |
| 0 | 1 | 1 |
| 1 | 0 | 1 |
| 1 | 1 | 1 |
| $x_1$ | $x_2$ | $y$ |
|---|---|---|
| 0 | 0 | 0 |
| 0 | 1 | 0 |
| 1 | 0 | 0 |
| 1 | 1 | 1 |
| $x_1$ | $x_2$ | $y$ |
|---|---|---|
| 0 | 0 | 0 |
| 0 | 1 | 1 |
| 1 | 0 | 1 |
| 1 | 1 | 0 |
![](\"figslides/ImageReduction.png\")
Figure 1:
\n", "\n", "\n", "\n", diff --git a/doc/src/week41/figslides/ImageReduction.png b/doc/src/week41/figslides/ImageReduction.png new file mode 100644 index 000000000..04f4bb973 Binary files /dev/null and b/doc/src/week41/figslides/ImageReduction.png differ diff --git a/doc/src/week41/week41.do.txt b/doc/src/week41/week41.do.txt index 2b7976610..0830ab643 100644 --- a/doc/src/week41/week41.do.txt +++ b/doc/src/week41/week41.do.txt @@ -1067,6 +1067,88 @@ plt.show() !ec +!split +===== Testing our code for the XOR, OR and AND gates ===== + +Last week we discussed three different types of gates, the so-called +XOR, the OR and the AND gates. Their inputs and outputs can be +summarized using the following tables, first for the OR gate with +inputs $x_1$ and $x_2$ and outputs $y$: + +|---------------------| +| $x_1$ | $x_2$ | $y$ | +|---------------------| +| 0 | 0 | 0 | +| 0 | 1 | 1 | +| 1 | 0 | 1 | +| 1 | 1 | 1 | +|---------------------| + +!split +===== The AND and XOR Gates ===== + +The _AND_ gate is defined as + +|---------------------| +| $x_1$ | $x_2$ | $y$ | +|---------------------| +| 0 | 0 | 0 | +| 0 | 1 | 0 | +| 1 | 0 | 0 | +| 1 | 1 | 1 | +|---------------------| + +And finally we have the _XOR_ gate + +|---------------------| +| $x_1$ | $x_2$ | $y$ | +|---------------------| +| 0 | 0 | 0 | +| 0 | 1 | 1 | +| 1 | 0 | 1 | +| 1 | 1 | 0 | +|---------------------| + +!split +===== Representing the Data Sets ===== + +Our design matrix is defined by the input values $x_1$ and $x_2$. Since we have four possible outputs, our design matrix reads + +!bt +\bm{X}=\begin{bmatrix} 0 & 0 \\ + 0 & 1 \\ + 1 & 0 \\ + 1 & 1 \end{bmatrix}, +!et +while the vector of outputs is $\bm{y}^T=[0,1,1,0]$ for the XOR gate, $\bm{y}^T=[0,0,0,1]$ for the AND gate and $\bm{y}^T=[0,1,1,1]$ for the OR gate. + +!split +===== Setting up the Neural Network ===== + +We define first our design matrix and the various input vectors. + +!bc pycod +""" +Simple code that tests XOR, OR and AND gates with linear regression +""" + +import numpy as np +# Design matrix +X = np.array([ [1, 0, 0], [1, 0, 1], [1, 1, 0],[1, 1, 1]],dtype=np.float64) + +# The XOR gate +yXOR = np.array( [ 0, 1 ,1, 0]) +# The OR gate +yOR = np.array( [ 0, 1 ,1, 1]) +# The AND gate +yAND = np.array( [ 0, 0 ,0, 1]) + +#print(f"The values of theta for the AND gate:{ThetaAND}") +#print(f"The linear regression prediction for the AND gate:{X @ ThetaAND}") +!ec + + + !split ===== Building neural networks in Tensorflow and Keras ===== @@ -1749,7 +1831,37 @@ Some of these remarks are particular to DNNs, others are shared by all supervise +!split +===== Overarching Views, a personal note ===== +The author of these lecture notes has an overarching take on many of +the machine learning algorithms we discuss here. + + +If we wish to understand complex systems, we need to find some +effective degrees of freedom or features that we find essential, +simply in order to reduce the complexity of the systems we are +studying. This leads, in one way or the other to dimensionality +reductions. Most of the Machine Learning methods we encounter deal +with this, whether we opt for a principal component analysis, or +clustering, or convolutional neural networks, or Ridge or Lasso +regression or random forest, yes, perhaps most machine learning +methods at large. + +For neural networks and our previous discussion, we have seen that we +in essence end up with matrix-matrix and matrix-vector +multiplications. In all cases, our matrices are dense ones, and the +more data we deal with the larger the dimensionalities of the matrices +and vectors. How can we reduce such dimensionalities? One possible +answer is offered by _convolutional neural networks_ (CNN), as +discussed below. The figure here shows a typical situation of the +reduction of information in an image and is typical of what CNNs +actually end up doing. + +!split +===== From a Spherical Cow to a real one ===== + +FIGURE: [figslides/ImageReduction.png, width=500 frac=0.6] !split @@ -2148,3 +2260,5 @@ o "Abstract art using convolutional neural networks":"https://deepdreamgenerator + +