lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update

Browse Source
This commit is contained in:
Morten Hjorth-Jensen
2023-10-23 07:07:40 +02:00
parent f16d3eefc4
commit 197a4dc38d
7 changed files with 24091 additions and 12900 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/pub/week43/html/week43-bs.html
+4801 -2641
View File
File diff suppressed because it is too large Load Diff
doc/pub/week43/html/week43-reveal.html
+4610 -2427
View File
File diff suppressed because it is too large Load Diff
doc/pub/week43/html/week43-solarized.html
+4691 -2581
View File
File diff suppressed because it is too large Load Diff
doc/pub/week43/html/week43.html
+4691 -2581
View File
File diff suppressed because it is too large Load Diff
doc/pub/week43/ipynb/ipynb-week43-src.tar.gz
BIN
View File
Binary file not shown.
doc/pub/week43/ipynb/week43.ipynb
+5263 -2628
View File
File diff suppressed because it is too large Load Diff
doc/src/week43/week43.do.txt
+35 -42
View File
@@ -5,7 +5,6 @@ DATE: today
!split
===== Plans for week 43 =====
!bblock Material for the active learning sessions on Tuesday and Wednesday
* Exercise on writing your own neural network code, application to the OR and XOR gates
* The exercises this week will be continued next week as well
@@ -13,7 +12,8 @@ DATE: today
!eblock
!bblock Material for the lecture on Thursday October 26, 2023
* Building our own Feed-forward Neural Network and discussion of project 2
* Building our own Feed-forward Neural Network and discussion of project 2, continuation from last week
* Solving differential equations with Neural Networks and intro to _Tensorflow_ with examples.
* Readings and Videos:
* These lecture notes
* "Aurelien Geron's chapters 10-11":"https://github.com/CompPhysics/MachineLearning/blob/master/doc/Textbooks/TensorflowML.pdf"
@@ -25,6 +25,21 @@ DATE: today
I also recommend Michael Nielsen's intuitive approach to the neural networks and the universal approximation theorem, see the slides at URL:"http://neuralnetworksanddeeplearning.com/chap4.html".
!eblock
!split
===== Using Automatic differentiation =====
a
In our discussions of ordinary differential equations
we will also study the usage of "Autograd":"https://www.youtube.com/watch?v=fRf4l5qaX1M&ab_channel=AlexSmola" in computing gradients for deep learning. For the documentation of Autograd and examples see the lectures slides from "week 39":"https://compphysics.github.io/MachineLearning/doc/pub/week39/html/week39.html" and the "Autograd documentation":"https://github.com/HIPS/autograd".
t
!split
===== Back propagation and automatic differentiation =====
For more details on the back propagation algorithm and automatic differentiation see
o URL:"https://www.jmlr.org/papers/volume18/17-468/17-468.pdf"
o URL:"https://deepimaging.github.io/lectures/lecture_11_Backpropagation.pdf"
o Slides 12-44 at URL":http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture4.pdf"
!split
@@ -33,8 +48,19 @@ I also recommend Michael Nielsen's intuitive approach to the neural networks an
!split
===== Writing our first neural network code, Testing our code for the OR and XOR gates =====
During week 41, we discussed three different types of gates, the so-called
XOR, the OR and the AND gates. Their inputs and outputs can be
During week 41 we discussed three different types of gates, the
so-called XOR, the OR and the AND gates. In order to develop a code
for neural networks, it can be useful to set up a simpler system with
only two inputs and one output. This can make it easier to debug and
study the feed forward pass and the back propagation part. In the
exercise this and next week, we propose to study this system with just
one hidden layer and two hidden nodes. There is only one output node
and we can choose to use either a simple regression case (fitting a
line) or just a binary classification case with the corss-entropy as
cost function.
Their inputs and outputs can be
summarized using the following tables, first for the OR gate with
inputs $x_1$ and $x_2$ and outputs $y$:
@@ -237,7 +263,10 @@ plt.show()
!split
===== Developing a code for doing neural networks with back propagation =====
We repeat some of the elements discussed last week.
We repeat some of the elements discussed last week. The first part of
the material for Thursday was contained in the slides for last
week as well. We will repeat some of the topics here before we move into
applications to differential equations and other examples.
One can identify a set of key steps when using neural networks to solve supervised learning problems:
@@ -1118,7 +1147,7 @@ conda install keras
!ec
You can look up the "instructions here":"https://keras.io/" for more information.
We will to a large extent use _keras_ in this course.
We will to a large extent use _keras_ in our examples..
!split
===== Collect and pre-process data =====
@@ -1740,44 +1769,8 @@ Here we list some of the important limitations of supervised neural network base
Some of these remarks are particular to DNNs, others are shared by all supervised learning methods. This motivates the use of unsupervised methods which in part circumvent these problems.
TITLE: Week 42 Solving differential equations and Convolutional (CNN)
AUTHOR: Morten Hjorth-Jensen {copyright, 1999-present|CC BY-NC} at Department of Physics, University of Oslo & Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University
DATE: today
!split
===== Plan for week 42 =====
!bblock
* Thursday: Solving differential equations with Neural Networks and intro to _Tensorflow_ with examples.
* "Video of lecture":"https://youtu.be/MdYT6uwOkT0"
* Friday: Convolutional Neural Networks.
* "Video of lecture":"https://youtu.be/3bDkrB-E7cU"
* Reading recommendations:
o See lecture notes for week 42 at https://compphysics.github.io/MachineLearning/doc/web/course.html.
o For Tensorflow and Keras, see lecture notes from week 41
o For neural networks we recommend Goodfellow et al chapters 6 and 7. For CNNs, see Goodfellow et al chapter 9. See also chapter 11 and 12 on practicalities and applications
o Reading suggestions for implementation of CNNs: "Aurelien Geron's chapter 13":"https://github.com/CompPhysics/MachineLearning/blob/master/doc/Textbooks/TensorflowML.pdf".
!eblock
!split
===== Using Automatic differentiation =====
a
In our discussions of ordinary differential equations
we will also study the usage of "Autograd":"https://www.youtube.com/watch?v=fRf4l5qaX1M&ab_channel=AlexSmola" in computing gradients for deep learning. For the documentation of Autograd and examples see the lectures slides from "week 39":"https://compphysics.github.io/MachineLearning/doc/pub/week39/html/week39.html" and the "Autograd documentation":"https://github.com/HIPS/autograd".
t
!split
===== Back propagation and automatic differentiation =====
For more details on the back propagation algorithm and automatic differentiation see
o URL:"https://www.jmlr.org/papers/volume18/17-468/17-468.pdf"
o URL:"https://deepimaging.github.io/lectures/lecture_11_Backpropagation.pdf"
o Slides 12-44 at URL":http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture4.pdf"
!split
===== Solving ODEs with Deep Learning =====