From 2471589234a419b5de6e4810c6cdbb8b406d76e2 Mon Sep 17 00:00:00 2001
From: Morten Hjorth-Jensen <morten.hjorth-jensen@fys.uio.no>
Date: Sun, 27 Aug 2023 09:57:53 +0200
Subject: [PATCH] Update week35.do.txt

---
 doc/src/week35/week35.do.txt | 54 ++++++++++++++++++++++++++++++++++--
 1 file changed, 51 insertions(+), 3 deletions(-)

diff --git a/doc/src/week35/week35.do.txt b/doc/src/week35/week35.do.txt
index 9cfe3c267..dba20299e 100644
--- a/doc/src/week35/week35.do.txt
+++ b/doc/src/week35/week35.do.txt
@@ -270,7 +270,7 @@ our matrix as $\bm{X}\in {\mathbb{R}}^{n\times p}$, with the predictors refering
 
 
 !split
-===== Our model for the nuclear binding energies =====
+===== Examples relevant for the exercises  =====
 
 In our "introductory notes":"https://compphysics.github.io/MachineLearning/doc/pub/How2ReadData/html/How2ReadData.html" we looked at the so-called "liquid drop model":"https://en.wikipedia.org/wiki/Semi-empirical_mass_formula". Let us remind ourselves about what we did by looking at the code.
 
@@ -491,8 +491,56 @@ _Small question_: Do you think the example we have at hand here (the nuclear bin
 !split
 ===== Some useful matrix and vector expressions =====
 
-The following matrix and vector relation will be useful here and for the rest of the course. Vectors are always written as boldfaced lower case letters and 
-matrices as upper case boldfaced letters. 
+
+The following matrix and vector relation will be useful here and for
+the rest of the course. Vectors are always written as boldfaced lower
+case letters and matrices as upper case boldfaced letters.  In the
+following we will discuss how to calculate derivatives of various
+matrices relevant for machine learning. We will often represent our
+data in terms of matrices and vectors.
+
+Let us introduce first some conventions. We assume that $\bm{y}$ is a
+vector of length $m$, that is it has $m$ elements $y_0,y_1,\dots,
+y_{m-1}$. By convention we start labeling vectors with the zeroth
+element, as are arrays in Python and C++/C, for example. Similarly, we
+have a vector $\bm{x}$ of length $n$, that is
+$\bm{x}^T=[x_0,x_1,\dots, x_{n-1}]$.
+
+We assume also that $\bm{y}$ is a function of $\bm{x}$ through some
+given function $f$
+
+!bt
+\[
+\bm{y}=f(\bm{x}).
+\]
+!et
+
+!split
+===== The Jacobian =====
+
+We define the partial derivatives of the various components of $\bm{y}$ as functions of $x_i$ in terms of the so-called "Jacobian matrix":"https://en.wikipedia.org/wiki/Jacobian_matrix_and_determinant"
+
+!bt
+\[
+\bm{J}=\frac{\partial \bm{y}}{\partial \bm{x}}=\begin{bmatrix} \frac{\partial y_0}{\partial x_0} & \frac{\partial y_0}{\partial x_1} & \frac{\partial y_0}{\partial x_2} & \dots & \dots & \frac{\partial y_0}{\partial x_{n-1}} \\ \frac{\partial y_0}{\partial x_0} & \frac{\partial y_1}{\partial x_1} & \frac{\partial y_1}{\partial x_2} & \dots & \dots & \frac{\partial y_1}{\partial x_{n-1}} \\
+\frac{\partial y_2}{\partial x_0} & \frac{\partial y_2}{\partial x_1} & \frac{\partial y_2}{\partial x_2} & \dots & \dots & \frac{\partial y_2}{\partial x_{n-1}} \\
+\dots & \dots & \dots & \dots & \dots & \dots \\
+\dots & \dots & \dots & \dots & \dots & \dots \\
+\frac{\partial y_{m-1}}{\partial x_0} & \frac{\partial y_{m-1}}{\partial x_1} & \frac{\partial y_{m-1}}{\partial x_2} & \dots & \dots & \frac{\partial y_{m-1}}{\partial x_{n-1}} \end{bmatrix},
+\]
+!et
+
+which is an $m\times n$ matrix. If $\bm{x}$ is a scalar, then the
+Jacobian is only a single-column vector, or an $m\times 1$ matrix. If
+on the other hand $\bm{y}$ is a scalar, the Jacobian becomes a
+$1\times n$ matrix.
+
+
+!split
+===== Derivatives, example 1 =====
+
+Let now $\bm{y}=\bm{A}\bm{x}$, where $\bm{A}$ is an $m\times n$ matrix and the matrix does not depend on $\bm{x}$. If we write out the vector $\bm{y}$
+
 
 !bt
 \[