update
This commit is contained in:
@@ -14,6 +14,9 @@ DATE: today
|
||||
See "lecture notes for week 39":"https://compphysics.github.io/MachineLearning/doc/web/course.html".
|
||||
For a good discussion on gradient methods, see Goodfellow et al section 4.3-4.5 and chapter 8. We will come back to the latter chapter in our discussion of Neural networks as well.
|
||||
|
||||
_For project 1, chapter 5 of Goodfellow et al is a good read, in particular sections 5.1-5.55 and 5.7-5.11_.
|
||||
These sections summarize neatly what we have done till now and point to what is coming with respect to deep learning.
|
||||
|
||||
!split
|
||||
===== Thursday September 30 =====
|
||||
|
||||
@@ -264,12 +267,6 @@ $p(y_i\vert x_i,\bm{\beta})(1-p(y_i\vert x_i,\bm{\beta})$, we can obtain a compa
|
||||
This defines what is called the Hessian matrix.
|
||||
|
||||
|
||||
!split
|
||||
===== To be added =====
|
||||
|
||||
We will add here an example which computes the likelihood $p_i$, sets up the gradient and the Hessian matrix.
|
||||
|
||||
Make link with linear regression and the Hessian matrix from linear regression.
|
||||
|
||||
|
||||
|
||||
@@ -1231,6 +1228,14 @@ plt.show()
|
||||
* GD can take exponential time to escape saddle points, even with random initialization. As we mentioned, GD is extremely sensitive to initial condition since it determines the particular local minimum GD would eventually reach. However, even with a good initialization scheme, through the introduction of randomness, GD can still take exponential time to escape saddle points.
|
||||
|
||||
|
||||
!split
|
||||
===== To be added =====
|
||||
|
||||
We will add here an example which computes the likelihood $p_i$, sets up the gradient and the Hessian matrix.
|
||||
|
||||
|
||||
|
||||
|
||||
!split
|
||||
===== Friday October 1 =====
|
||||
|
||||
|
||||
Reference in New Issue
Block a user