Update week43.do.txt

doc/src/week43/week43.do.txt

@@ -30,6 +30,21 @@ DATE: today
 
 
 
+neural nets will be very large: impractical to write down gradient formula
+by hand for all parameters
+● backpropagation = recursive application of the chain rule along a
+computational graph to compute the gradients of all
+inputs/parameters/intermediates
+● implementations maintain a graph structure, where the nodes implement
+the forward() / backward() API
+● forward: compute result of an operation and save any intermediates
+needed for gradient computation in memory
+● backward: apply the chain rule to compute the gradient of the loss
+function with respect to the inputs
+
+
+
+
 !split
 ===== CNNs in brief =====