This commit is contained in:
Morten Hjorth-Jensen
2021-09-30 10:31:21 +02:00
parent 57c1fcee57
commit 43877d18ff
7 changed files with 104 additions and 91 deletions
+11 -6
View File
@@ -14,6 +14,9 @@ DATE: today
See "lecture notes for week 39":"https://compphysics.github.io/MachineLearning/doc/web/course.html".
For a good discussion on gradient methods, see Goodfellow et al section 4.3-4.5 and chapter 8. We will come back to the latter chapter in our discussion of Neural networks as well.
_For project 1, chapter 5 of Goodfellow et al is a good read, in particular sections 5.1-5.55 and 5.7-5.11_.
These sections summarize neatly what we have done till now and point to what is coming with respect to deep learning.
!split
===== Thursday September 30 =====
@@ -264,12 +267,6 @@ $p(y_i\vert x_i,\bm{\beta})(1-p(y_i\vert x_i,\bm{\beta})$, we can obtain a compa
This defines what is called the Hessian matrix.
!split
===== To be added =====
We will add here an example which computes the likelihood $p_i$, sets up the gradient and the Hessian matrix.
Make link with linear regression and the Hessian matrix from linear regression.
@@ -1231,6 +1228,14 @@ plt.show()
* GD can take exponential time to escape saddle points, even with random initialization. As we mentioned, GD is extremely sensitive to initial condition since it determines the particular local minimum GD would eventually reach. However, even with a good initialization scheme, through the introduction of randomness, GD can still take exponential time to escape saddle points.
!split
===== To be added =====
We will add here an example which computes the likelihood $p_i$, sets up the gradient and the Hessian matrix.
!split
===== Friday October 1 =====