diff --git a/doc/pub/week43/html/._week43-bs000.html b/doc/pub/week43/html/._week43-bs000.html index 6340568db..94361c0f2 100644 --- a/doc/pub/week43/html/._week43-bs000.html +++ b/doc/pub/week43/html/._week43-bs000.html @@ -67,7 +67,7 @@ Automatically generated HTML file from DocOnce source ('Final technicalities II', 2, None, '___sec16'), ('Final technicalities III', 2, None, '___sec17'), ('Final technicalities IV', 2, None, '___sec18'), - ('Backpropagation', 2, None, '___sec19'), + ('Back propagation', 2, None, '___sec19'), ('Gradient descent', 2, None, '___sec20'), ('The code for solving the ODE', 2, None, '___sec21'), ('The network with one input layer, specified number of hidden ' @@ -80,100 +80,100 @@ Automatically generated HTML file from DocOnce source ('The trial solution', 2, None, '___sec25'), ('The program using Autograd', 2, None, '___sec26'), ('Using forward Euler to solve the ODE', 2, None, '___sec27'), - ('Example: Solving the one dimensional Poisson equation using ' - 'Autograd and TensorFlow', + ('Example: Solving the one dimensional Poisson equation', 2, None, '___sec28'), ('The specific equation to solve for', 2, None, '___sec29'), ('Solving the equation using Autograd', 2, None, '___sec30'), ('Comparing with a numerical scheme', 2, None, '___sec31'), - ('Partial Differential Equations', 2, None, '___sec32'), - ('Type of problem', 2, None, '___sec33'), - ('Network requirements', 2, None, '___sec34'), - ('More details', 2, None, '___sec35'), - ('Example: The diffusion equation', 2, None, '___sec36'), - ('Defining the problem', 2, None, '___sec37'), - ('Setting up the network using Autograd', 2, None, '___sec38'), + ('Setting up the code', 2, None, '___sec32'), + ('Partial Differential Equations', 2, None, '___sec33'), + ('Type of problem', 2, None, '___sec34'), + ('Network requirements', 2, None, '___sec35'), + ('More details', 2, None, '___sec36'), + ('Example: The diffusion equation', 2, None, '___sec37'), + ('Defining the problem', 2, None, '___sec38'), + ('Setting up the network using Autograd', 2, None, '___sec39'), ('Setting up the network using Autograd; The trial solution', 2, None, - '___sec39'), - ('Why the jacobian?', 2, None, '___sec40'), + '___sec40'), + ('Why the jacobian?', 2, None, '___sec41'), ('Setting up the network using Autograd; The full program', 2, None, - '___sec41'), + '___sec42'), ('Example: Solving the wave equation with Neural Networks', 2, None, - '___sec42'), - ('The problem to solve for', 2, None, '___sec43'), - ('The trial solution', 2, None, '___sec44'), - ('The analytical solution', 2, None, '___sec45'), + '___sec43'), + ('The problem to solve for', 2, None, '___sec44'), + ('The trial solution', 2, None, '___sec45'), + ('The analytical solution', 2, None, '___sec46'), ('Solving the wave equation - the full program using Autograd', 2, None, - '___sec46'), + '___sec47'), ('Resources on differential equations and deep learning', 2, None, - '___sec47'), - ('Friday, Principal Component Analysis', 2, None, '___sec48'), + '___sec48'), + ('Friday, Principal Component Analysis', 2, None, '___sec49'), ('Basic ideas of the Principal Component Analysis (PCA)', 2, None, - '___sec49'), + '___sec50'), ('Introducing the Covariance and Correlation functions', 2, None, - '___sec50'), + '___sec51'), ('Correlation Function and Design/Feature Matrix', 2, None, - '___sec51'), - ('Covariance Matrix Examples', 2, None, '___sec52'), - ('Correlation Matrix', 2, None, '___sec53'), - ('Correlation Matrix with Pandas', 2, None, '___sec54'), + '___sec52'), + ('Covariance Matrix Examples', 2, None, '___sec53'), + ('Correlation Matrix', 2, None, '___sec54'), + ('Correlation Matrix with Pandas', 2, None, '___sec55'), ('Correlation Matrix with Pandas and the Franke function', 2, None, - '___sec55'), + '___sec56'), ('Rewriting the Covariance and/or Correlation Matrix', 2, None, - '___sec56'), - ('Towards the PCA theorem', 2, None, '___sec57'), - ('The Algorithm before the Theorem', 2, None, '___sec58'), - ('Writing our own PCA code', 2, None, '___sec59'), + '___sec57'), + ('Towards the PCA theorem', 2, None, '___sec58'), + ('The Algorithm before the Theorem', 2, None, '___sec59'), + ('Writing our own PCA code', 2, None, '___sec60'), ('Compute the sample mean and center the data', 3, None, - '___sec60'), - ('Compute the sample covariance', 3, None, '___sec61'), + '___sec61'), + ('Compute the sample covariance', 3, None, '___sec62'), ('Diagonalize the sample covariance matrix to obtain the ' 'principal components', 3, None, - '___sec62'), - ('Classical PCA Theorem', 2, None, '___sec63'), - ('Proof of the PCA Theorem', 2, None, '___sec64'), - ('PCA Proof continued', 2, None, '___sec65'), - ('The final step', 2, None, '___sec66'), + '___sec63'), + ('Classical PCA Theorem', 2, None, '___sec64'), + ('Proof of the PCA Theorem', 2, None, '___sec65'), + ('PCA Proof continued', 2, None, '___sec66'), + ('The final step', 2, None, '___sec67'), ('Geometric Interpretation and link with Singular Value ' 'Decomposition', 2, None, - '___sec67'), - ('Principal Component Analysis', 2, None, '___sec68'), - ('PCA and scikit-learn', 2, None, '___sec69'), - ('Back to the Cancer Data', 2, None, '___sec70'), - ('More on the PCA', 2, None, '___sec71'), - ('Incremental PCA', 2, None, '___sec72'), - ('Randomized PCA', 2, None, '___sec73'), - ('Kernel PCA', 2, None, '___sec74'), - ('LLE', 2, None, '___sec75'), - ('Other techniques', 2, None, '___sec76')]} + '___sec68'), + ('Principal Component Analysis', 2, None, '___sec69'), + ('PCA and scikit-learn', 2, None, '___sec70'), + ('Back to the Cancer Data', 2, None, '___sec71'), + ('More on the PCA', 2, None, '___sec72'), + ('Incremental PCA', 2, None, '___sec73'), + ('Randomized PCA', 2, None, '___sec74'), + ('Kernel PCA', 2, None, '___sec75'), + ('LLE', 2, None, '___sec76'), + ('Other techniques', 2, None, '___sec77')]} end of tocinfo --> @@ -230,7 +230,7 @@ MathJax.Hub.Config({
  • Final technicalities II
  • Final technicalities III
  • Final technicalities IV
    • -
  • Backpropagation
    • +
  • Back propagation
  • Gradient descent
  • The code for solving the ODE
  • The network with one input layer, specified number of hidden layers, and one output layer
    • @@ -239,55 +239,56 @@ MathJax.Hub.Config({
  • The trial solution
  • The program using Autograd
  • Using forward Euler to solve the ODE
    • -
  • Example: Solving the one dimensional Poisson equation using Autograd and TensorFlow
    • +
  • Example: Solving the one dimensional Poisson equation
  • The specific equation to solve for
  • Solving the equation using Autograd
  • Comparing with a numerical scheme
    • -
  • Partial Differential Equations
    • -
  • Type of problem
    • -
  • Network requirements
    • -
  • More details
    • -
  • Example: The diffusion equation
    • -
  • Defining the problem
    • -
  • Setting up the network using Autograd
    • -
  • Setting up the network using Autograd; The trial solution
    • -
  • Why the jacobian?
    • -
  • Setting up the network using Autograd; The full program
    • -
  • Example: Solving the wave equation with Neural Networks
    • -
  • The problem to solve for
    • -
  • The trial solution
    • -
  • The analytical solution
    • -
  • Solving the wave equation - the full program using Autograd
    • -
  • Resources on differential equations and deep learning
    • -
  • Friday, Principal Component Analysis
    • -
  • Basic ideas of the Principal Component Analysis (PCA)
    • -
  • Introducing the Covariance and Correlation functions
    • -
  • Correlation Function and Design/Feature Matrix
    • -
  • Covariance Matrix Examples
    • -
  • Correlation Matrix
    • -
  • Correlation Matrix with Pandas
    • -
  • Correlation Matrix with Pandas and the Franke function
    • -
  • Rewriting the Covariance and/or Correlation Matrix
    • -
  • Towards the PCA theorem
    • -
  • The Algorithm before the Theorem
    • -
  • Writing our own PCA code
    • -
  •    Compute the sample mean and center the data
    • -
  •    Compute the sample covariance
    • -
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • -
  • Classical PCA Theorem
    • -
  • Proof of the PCA Theorem
    • -
  • PCA Proof continued
    • -
  • The final step
    • -
  • Geometric Interpretation and link with Singular Value Decomposition
    • -
  • Principal Component Analysis
    • -
  • PCA and scikit-learn
    • -
  • Back to the Cancer Data
    • -
  • More on the PCA
    • -
  • Incremental PCA
    • -
  • Randomized PCA
    • -
  • Kernel PCA
    • -
  • LLE
    • -
  • Other techniques
    • +
  • Setting up the code
    • +
  • Partial Differential Equations
    • +
  • Type of problem
    • +
  • Network requirements
    • +
  • More details
    • +
  • Example: The diffusion equation
    • +
  • Defining the problem
    • +
  • Setting up the network using Autograd
    • +
  • Setting up the network using Autograd; The trial solution
    • +
  • Why the jacobian?
    • +
  • Setting up the network using Autograd; The full program
    • +
  • Example: Solving the wave equation with Neural Networks
    • +
  • The problem to solve for
    • +
  • The trial solution
    • +
  • The analytical solution
    • +
  • Solving the wave equation - the full program using Autograd
    • +
  • Resources on differential equations and deep learning
    • +
  • Friday, Principal Component Analysis
    • +
  • Basic ideas of the Principal Component Analysis (PCA)
    • +
  • Introducing the Covariance and Correlation functions
    • +
  • Correlation Function and Design/Feature Matrix
    • +
  • Covariance Matrix Examples
    • +
  • Correlation Matrix
    • +
  • Correlation Matrix with Pandas
    • +
  • Correlation Matrix with Pandas and the Franke function
    • +
  • Rewriting the Covariance and/or Correlation Matrix
    • +
  • Towards the PCA theorem
    • +
  • The Algorithm before the Theorem
    • +
  • Writing our own PCA code
    • +
  •    Compute the sample mean and center the data
    • +
  •    Compute the sample covariance
    • +
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • +
  • Classical PCA Theorem
    • +
  • Proof of the PCA Theorem
    • +
  • PCA Proof continued
    • +
  • The final step
    • +
  • Geometric Interpretation and link with Singular Value Decomposition
    • +
  • Principal Component Analysis
    • +
  • PCA and scikit-learn
    • +
  • Back to the Cancer Data
    • +
  • More on the PCA
    • +
  • Incremental PCA
    • +
  • Randomized PCA
    • +
  • Kernel PCA
    • +
  • LLE
    • +
  • Other techniques
    • @@ -346,7 +347,7 @@ MathJax.Hub.Config({
  • 9
  • 10
  • ...
    • -
  • 76
    • +
  • 77
  • »
    • diff --git a/doc/pub/week43/html/._week43-bs001.html b/doc/pub/week43/html/._week43-bs001.html index e202d2c1c..575b914bb 100644 --- a/doc/pub/week43/html/._week43-bs001.html +++ b/doc/pub/week43/html/._week43-bs001.html @@ -67,7 +67,7 @@ Automatically generated HTML file from DocOnce source ('Final technicalities II', 2, None, '___sec16'), ('Final technicalities III', 2, None, '___sec17'), ('Final technicalities IV', 2, None, '___sec18'), - ('Backpropagation', 2, None, '___sec19'), + ('Back propagation', 2, None, '___sec19'), ('Gradient descent', 2, None, '___sec20'), ('The code for solving the ODE', 2, None, '___sec21'), ('The network with one input layer, specified number of hidden ' @@ -80,100 +80,100 @@ Automatically generated HTML file from DocOnce source ('The trial solution', 2, None, '___sec25'), ('The program using Autograd', 2, None, '___sec26'), ('Using forward Euler to solve the ODE', 2, None, '___sec27'), - ('Example: Solving the one dimensional Poisson equation using ' - 'Autograd and TensorFlow', + ('Example: Solving the one dimensional Poisson equation', 2, None, '___sec28'), ('The specific equation to solve for', 2, None, '___sec29'), ('Solving the equation using Autograd', 2, None, '___sec30'), ('Comparing with a numerical scheme', 2, None, '___sec31'), - ('Partial Differential Equations', 2, None, '___sec32'), - ('Type of problem', 2, None, '___sec33'), - ('Network requirements', 2, None, '___sec34'), - ('More details', 2, None, '___sec35'), - ('Example: The diffusion equation', 2, None, '___sec36'), - ('Defining the problem', 2, None, '___sec37'), - ('Setting up the network using Autograd', 2, None, '___sec38'), + ('Setting up the code', 2, None, '___sec32'), + ('Partial Differential Equations', 2, None, '___sec33'), + ('Type of problem', 2, None, '___sec34'), + ('Network requirements', 2, None, '___sec35'), + ('More details', 2, None, '___sec36'), + ('Example: The diffusion equation', 2, None, '___sec37'), + ('Defining the problem', 2, None, '___sec38'), + ('Setting up the network using Autograd', 2, None, '___sec39'), ('Setting up the network using Autograd; The trial solution', 2, None, - '___sec39'), - ('Why the jacobian?', 2, None, '___sec40'), + '___sec40'), + ('Why the jacobian?', 2, None, '___sec41'), ('Setting up the network using Autograd; The full program', 2, None, - '___sec41'), + '___sec42'), ('Example: Solving the wave equation with Neural Networks', 2, None, - '___sec42'), - ('The problem to solve for', 2, None, '___sec43'), - ('The trial solution', 2, None, '___sec44'), - ('The analytical solution', 2, None, '___sec45'), + '___sec43'), + ('The problem to solve for', 2, None, '___sec44'), + ('The trial solution', 2, None, '___sec45'), + ('The analytical solution', 2, None, '___sec46'), ('Solving the wave equation - the full program using Autograd', 2, None, - '___sec46'), + '___sec47'), ('Resources on differential equations and deep learning', 2, None, - '___sec47'), - ('Friday, Principal Component Analysis', 2, None, '___sec48'), + '___sec48'), + ('Friday, Principal Component Analysis', 2, None, '___sec49'), ('Basic ideas of the Principal Component Analysis (PCA)', 2, None, - '___sec49'), + '___sec50'), ('Introducing the Covariance and Correlation functions', 2, None, - '___sec50'), + '___sec51'), ('Correlation Function and Design/Feature Matrix', 2, None, - '___sec51'), - ('Covariance Matrix Examples', 2, None, '___sec52'), - ('Correlation Matrix', 2, None, '___sec53'), - ('Correlation Matrix with Pandas', 2, None, '___sec54'), + '___sec52'), + ('Covariance Matrix Examples', 2, None, '___sec53'), + ('Correlation Matrix', 2, None, '___sec54'), + ('Correlation Matrix with Pandas', 2, None, '___sec55'), ('Correlation Matrix with Pandas and the Franke function', 2, None, - '___sec55'), + '___sec56'), ('Rewriting the Covariance and/or Correlation Matrix', 2, None, - '___sec56'), - ('Towards the PCA theorem', 2, None, '___sec57'), - ('The Algorithm before the Theorem', 2, None, '___sec58'), - ('Writing our own PCA code', 2, None, '___sec59'), + '___sec57'), + ('Towards the PCA theorem', 2, None, '___sec58'), + ('The Algorithm before the Theorem', 2, None, '___sec59'), + ('Writing our own PCA code', 2, None, '___sec60'), ('Compute the sample mean and center the data', 3, None, - '___sec60'), - ('Compute the sample covariance', 3, None, '___sec61'), + '___sec61'), + ('Compute the sample covariance', 3, None, '___sec62'), ('Diagonalize the sample covariance matrix to obtain the ' 'principal components', 3, None, - '___sec62'), - ('Classical PCA Theorem', 2, None, '___sec63'), - ('Proof of the PCA Theorem', 2, None, '___sec64'), - ('PCA Proof continued', 2, None, '___sec65'), - ('The final step', 2, None, '___sec66'), + '___sec63'), + ('Classical PCA Theorem', 2, None, '___sec64'), + ('Proof of the PCA Theorem', 2, None, '___sec65'), + ('PCA Proof continued', 2, None, '___sec66'), + ('The final step', 2, None, '___sec67'), ('Geometric Interpretation and link with Singular Value ' 'Decomposition', 2, None, - '___sec67'), - ('Principal Component Analysis', 2, None, '___sec68'), - ('PCA and scikit-learn', 2, None, '___sec69'), - ('Back to the Cancer Data', 2, None, '___sec70'), - ('More on the PCA', 2, None, '___sec71'), - ('Incremental PCA', 2, None, '___sec72'), - ('Randomized PCA', 2, None, '___sec73'), - ('Kernel PCA', 2, None, '___sec74'), - ('LLE', 2, None, '___sec75'), - ('Other techniques', 2, None, '___sec76')]} + '___sec68'), + ('Principal Component Analysis', 2, None, '___sec69'), + ('PCA and scikit-learn', 2, None, '___sec70'), + ('Back to the Cancer Data', 2, None, '___sec71'), + ('More on the PCA', 2, None, '___sec72'), + ('Incremental PCA', 2, None, '___sec73'), + ('Randomized PCA', 2, None, '___sec74'), + ('Kernel PCA', 2, None, '___sec75'), + ('LLE', 2, None, '___sec76'), + ('Other techniques', 2, None, '___sec77')]} end of tocinfo --> @@ -230,7 +230,7 @@ MathJax.Hub.Config({
  • Final technicalities II
  • Final technicalities III
  • Final technicalities IV
    • -
  • Backpropagation
    • +
  • Back propagation
  • Gradient descent
  • The code for solving the ODE
  • The network with one input layer, specified number of hidden layers, and one output layer
    • @@ -239,55 +239,56 @@ MathJax.Hub.Config({
  • The trial solution
  • The program using Autograd
  • Using forward Euler to solve the ODE
    • -
  • Example: Solving the one dimensional Poisson equation using Autograd and TensorFlow
    • +
  • Example: Solving the one dimensional Poisson equation
  • The specific equation to solve for
  • Solving the equation using Autograd
  • Comparing with a numerical scheme
    • -
  • Partial Differential Equations
    • -
  • Type of problem
    • -
  • Network requirements
    • -
  • More details
    • -
  • Example: The diffusion equation
    • -
  • Defining the problem
    • -
  • Setting up the network using Autograd
    • -
  • Setting up the network using Autograd; The trial solution
    • -
  • Why the jacobian?
    • -
  • Setting up the network using Autograd; The full program
    • -
  • Example: Solving the wave equation with Neural Networks
    • -
  • The problem to solve for
    • -
  • The trial solution
    • -
  • The analytical solution
    • -
  • Solving the wave equation - the full program using Autograd
    • -
  • Resources on differential equations and deep learning
    • -
  • Friday, Principal Component Analysis
    • -
  • Basic ideas of the Principal Component Analysis (PCA)
    • -
  • Introducing the Covariance and Correlation functions
    • -
  • Correlation Function and Design/Feature Matrix
    • -
  • Covariance Matrix Examples
    • -
  • Correlation Matrix
    • -
  • Correlation Matrix with Pandas
    • -
  • Correlation Matrix with Pandas and the Franke function
    • -
  • Rewriting the Covariance and/or Correlation Matrix
    • -
  • Towards the PCA theorem
    • -
  • The Algorithm before the Theorem
    • -
  • Writing our own PCA code
    • -
  •    Compute the sample mean and center the data
    • -
  •    Compute the sample covariance
    • -
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • -
  • Classical PCA Theorem
    • -
  • Proof of the PCA Theorem
    • -
  • PCA Proof continued
    • -
  • The final step
    • -
  • Geometric Interpretation and link with Singular Value Decomposition
    • -
  • Principal Component Analysis
    • -
  • PCA and scikit-learn
    • -
  • Back to the Cancer Data
    • -
  • More on the PCA
    • -
  • Incremental PCA
    • -
  • Randomized PCA
    • -
  • Kernel PCA
    • -
  • LLE
    • -
  • Other techniques
    • +
  • Setting up the code
    • +
  • Partial Differential Equations
    • +
  • Type of problem
    • +
  • Network requirements
    • +
  • More details
    • +
  • Example: The diffusion equation
    • +
  • Defining the problem
    • +
  • Setting up the network using Autograd
    • +
  • Setting up the network using Autograd; The trial solution
    • +
  • Why the jacobian?
    • +
  • Setting up the network using Autograd; The full program
    • +
  • Example: Solving the wave equation with Neural Networks
    • +
  • The problem to solve for
    • +
  • The trial solution
    • +
  • The analytical solution
    • +
  • Solving the wave equation - the full program using Autograd
    • +
  • Resources on differential equations and deep learning
    • +
  • Friday, Principal Component Analysis
    • +
  • Basic ideas of the Principal Component Analysis (PCA)
    • +
  • Introducing the Covariance and Correlation functions
    • +
  • Correlation Function and Design/Feature Matrix
    • +
  • Covariance Matrix Examples
    • +
  • Correlation Matrix
    • +
  • Correlation Matrix with Pandas
    • +
  • Correlation Matrix with Pandas and the Franke function
    • +
  • Rewriting the Covariance and/or Correlation Matrix
    • +
  • Towards the PCA theorem
    • +
  • The Algorithm before the Theorem
    • +
  • Writing our own PCA code
    • +
  •    Compute the sample mean and center the data
    • +
  •    Compute the sample covariance
    • +
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • +
  • Classical PCA Theorem
    • +
  • Proof of the PCA Theorem
    • +
  • PCA Proof continued
    • +
  • The final step
    • +
  • Geometric Interpretation and link with Singular Value Decomposition
    • +
  • Principal Component Analysis
    • +
  • PCA and scikit-learn
    • +
  • Back to the Cancer Data
    • +
  • More on the PCA
    • +
  • Incremental PCA
    • +
  • Randomized PCA
    • +
  • Kernel PCA
    • +
  • LLE
    • +
  • Other techniques
    • @@ -308,7 +309,7 @@ MathJax.Hub.Config({
  • Friday: Principal Component Analysis and Dimensionality Reduction
    • -Reading suggestions for both days: "Aurelien Geron's chapters 8 +We will also study the usage of Autograd in computing gradients for deep learning. For the documentation of Autograd and examples see the lectures slides from week 40 and the Autograd doucmentation.

    @@ -327,7 +328,7 @@ Reading suggestions for both days: "Aurelien Geron's chapters 8

  • 10
  • 11
  • ...
    • -
  • 76
    • +
  • 77
  • »
    • diff --git a/doc/pub/week43/html/._week43-bs002.html b/doc/pub/week43/html/._week43-bs002.html index 19c0857d9..1f4a76de0 100644 --- a/doc/pub/week43/html/._week43-bs002.html +++ b/doc/pub/week43/html/._week43-bs002.html @@ -67,7 +67,7 @@ Automatically generated HTML file from DocOnce source ('Final technicalities II', 2, None, '___sec16'), ('Final technicalities III', 2, None, '___sec17'), ('Final technicalities IV', 2, None, '___sec18'), - ('Backpropagation', 2, None, '___sec19'), + ('Back propagation', 2, None, '___sec19'), ('Gradient descent', 2, None, '___sec20'), ('The code for solving the ODE', 2, None, '___sec21'), ('The network with one input layer, specified number of hidden ' @@ -80,100 +80,100 @@ Automatically generated HTML file from DocOnce source ('The trial solution', 2, None, '___sec25'), ('The program using Autograd', 2, None, '___sec26'), ('Using forward Euler to solve the ODE', 2, None, '___sec27'), - ('Example: Solving the one dimensional Poisson equation using ' - 'Autograd and TensorFlow', + ('Example: Solving the one dimensional Poisson equation', 2, None, '___sec28'), ('The specific equation to solve for', 2, None, '___sec29'), ('Solving the equation using Autograd', 2, None, '___sec30'), ('Comparing with a numerical scheme', 2, None, '___sec31'), - ('Partial Differential Equations', 2, None, '___sec32'), - ('Type of problem', 2, None, '___sec33'), - ('Network requirements', 2, None, '___sec34'), - ('More details', 2, None, '___sec35'), - ('Example: The diffusion equation', 2, None, '___sec36'), - ('Defining the problem', 2, None, '___sec37'), - ('Setting up the network using Autograd', 2, None, '___sec38'), + ('Setting up the code', 2, None, '___sec32'), + ('Partial Differential Equations', 2, None, '___sec33'), + ('Type of problem', 2, None, '___sec34'), + ('Network requirements', 2, None, '___sec35'), + ('More details', 2, None, '___sec36'), + ('Example: The diffusion equation', 2, None, '___sec37'), + ('Defining the problem', 2, None, '___sec38'), + ('Setting up the network using Autograd', 2, None, '___sec39'), ('Setting up the network using Autograd; The trial solution', 2, None, - '___sec39'), - ('Why the jacobian?', 2, None, '___sec40'), + '___sec40'), + ('Why the jacobian?', 2, None, '___sec41'), ('Setting up the network using Autograd; The full program', 2, None, - '___sec41'), + '___sec42'), ('Example: Solving the wave equation with Neural Networks', 2, None, - '___sec42'), - ('The problem to solve for', 2, None, '___sec43'), - ('The trial solution', 2, None, '___sec44'), - ('The analytical solution', 2, None, '___sec45'), + '___sec43'), + ('The problem to solve for', 2, None, '___sec44'), + ('The trial solution', 2, None, '___sec45'), + ('The analytical solution', 2, None, '___sec46'), ('Solving the wave equation - the full program using Autograd', 2, None, - '___sec46'), + '___sec47'), ('Resources on differential equations and deep learning', 2, None, - '___sec47'), - ('Friday, Principal Component Analysis', 2, None, '___sec48'), + '___sec48'), + ('Friday, Principal Component Analysis', 2, None, '___sec49'), ('Basic ideas of the Principal Component Analysis (PCA)', 2, None, - '___sec49'), + '___sec50'), ('Introducing the Covariance and Correlation functions', 2, None, - '___sec50'), + '___sec51'), ('Correlation Function and Design/Feature Matrix', 2, None, - '___sec51'), - ('Covariance Matrix Examples', 2, None, '___sec52'), - ('Correlation Matrix', 2, None, '___sec53'), - ('Correlation Matrix with Pandas', 2, None, '___sec54'), + '___sec52'), + ('Covariance Matrix Examples', 2, None, '___sec53'), + ('Correlation Matrix', 2, None, '___sec54'), + ('Correlation Matrix with Pandas', 2, None, '___sec55'), ('Correlation Matrix with Pandas and the Franke function', 2, None, - '___sec55'), + '___sec56'), ('Rewriting the Covariance and/or Correlation Matrix', 2, None, - '___sec56'), - ('Towards the PCA theorem', 2, None, '___sec57'), - ('The Algorithm before the Theorem', 2, None, '___sec58'), - ('Writing our own PCA code', 2, None, '___sec59'), + '___sec57'), + ('Towards the PCA theorem', 2, None, '___sec58'), + ('The Algorithm before the Theorem', 2, None, '___sec59'), + ('Writing our own PCA code', 2, None, '___sec60'), ('Compute the sample mean and center the data', 3, None, - '___sec60'), - ('Compute the sample covariance', 3, None, '___sec61'), + '___sec61'), + ('Compute the sample covariance', 3, None, '___sec62'), ('Diagonalize the sample covariance matrix to obtain the ' 'principal components', 3, None, - '___sec62'), - ('Classical PCA Theorem', 2, None, '___sec63'), - ('Proof of the PCA Theorem', 2, None, '___sec64'), - ('PCA Proof continued', 2, None, '___sec65'), - ('The final step', 2, None, '___sec66'), + '___sec63'), + ('Classical PCA Theorem', 2, None, '___sec64'), + ('Proof of the PCA Theorem', 2, None, '___sec65'), + ('PCA Proof continued', 2, None, '___sec66'), + ('The final step', 2, None, '___sec67'), ('Geometric Interpretation and link with Singular Value ' 'Decomposition', 2, None, - '___sec67'), - ('Principal Component Analysis', 2, None, '___sec68'), - ('PCA and scikit-learn', 2, None, '___sec69'), - ('Back to the Cancer Data', 2, None, '___sec70'), - ('More on the PCA', 2, None, '___sec71'), - ('Incremental PCA', 2, None, '___sec72'), - ('Randomized PCA', 2, None, '___sec73'), - ('Kernel PCA', 2, None, '___sec74'), - ('LLE', 2, None, '___sec75'), - ('Other techniques', 2, None, '___sec76')]} + '___sec68'), + ('Principal Component Analysis', 2, None, '___sec69'), + ('PCA and scikit-learn', 2, None, '___sec70'), + ('Back to the Cancer Data', 2, None, '___sec71'), + ('More on the PCA', 2, None, '___sec72'), + ('Incremental PCA', 2, None, '___sec73'), + ('Randomized PCA', 2, None, '___sec74'), + ('Kernel PCA', 2, None, '___sec75'), + ('LLE', 2, None, '___sec76'), + ('Other techniques', 2, None, '___sec77')]} end of tocinfo --> @@ -230,7 +230,7 @@ MathJax.Hub.Config({
  • Final technicalities II
  • Final technicalities III
  • Final technicalities IV
    • -
  • Backpropagation
    • +
  • Back propagation
  • Gradient descent
  • The code for solving the ODE
  • The network with one input layer, specified number of hidden layers, and one output layer
    • @@ -239,55 +239,56 @@ MathJax.Hub.Config({
  • The trial solution
  • The program using Autograd
  • Using forward Euler to solve the ODE
    • -
  • Example: Solving the one dimensional Poisson equation using Autograd and TensorFlow
    • +
  • Example: Solving the one dimensional Poisson equation
  • The specific equation to solve for
  • Solving the equation using Autograd
  • Comparing with a numerical scheme
    • -
  • Partial Differential Equations
    • -
  • Type of problem
    • -
  • Network requirements
    • -
  • More details
    • -
  • Example: The diffusion equation
    • -
  • Defining the problem
    • -
  • Setting up the network using Autograd
    • -
  • Setting up the network using Autograd; The trial solution
    • -
  • Why the jacobian?
    • -
  • Setting up the network using Autograd; The full program
    • -
  • Example: Solving the wave equation with Neural Networks
    • -
  • The problem to solve for
    • -
  • The trial solution
    • -
  • The analytical solution
    • -
  • Solving the wave equation - the full program using Autograd
    • -
  • Resources on differential equations and deep learning
    • -
  • Friday, Principal Component Analysis
    • -
  • Basic ideas of the Principal Component Analysis (PCA)
    • -
  • Introducing the Covariance and Correlation functions
    • -
  • Correlation Function and Design/Feature Matrix
    • -
  • Covariance Matrix Examples
    • -
  • Correlation Matrix
    • -
  • Correlation Matrix with Pandas
    • -
  • Correlation Matrix with Pandas and the Franke function
    • -
  • Rewriting the Covariance and/or Correlation Matrix
    • -
  • Towards the PCA theorem
    • -
  • The Algorithm before the Theorem
    • -
  • Writing our own PCA code
    • -
  •    Compute the sample mean and center the data
    • -
  •    Compute the sample covariance
    • -
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • -
  • Classical PCA Theorem
    • -
  • Proof of the PCA Theorem
    • -
  • PCA Proof continued
    • -
  • The final step
    • -
  • Geometric Interpretation and link with Singular Value Decomposition
    • -
  • Principal Component Analysis
    • -
  • PCA and scikit-learn
    • -
  • Back to the Cancer Data
    • -
  • More on the PCA
    • -
  • Incremental PCA
    • -
  • Randomized PCA
    • -
  • Kernel PCA
    • -
  • LLE
    • -
  • Other techniques
    • +
  • Setting up the code
    • +
  • Partial Differential Equations
    • +
  • Type of problem
    • +
  • Network requirements
    • +
  • More details
    • +
  • Example: The diffusion equation
    • +
  • Defining the problem
    • +
  • Setting up the network using Autograd
    • +
  • Setting up the network using Autograd; The trial solution
    • +
  • Why the jacobian?
    • +
  • Setting up the network using Autograd; The full program
    • +
  • Example: Solving the wave equation with Neural Networks
    • +
  • The problem to solve for
    • +
  • The trial solution
    • +
  • The analytical solution
    • +
  • Solving the wave equation - the full program using Autograd
    • +
  • Resources on differential equations and deep learning
    • +
  • Friday, Principal Component Analysis
    • +
  • Basic ideas of the Principal Component Analysis (PCA)
    • +
  • Introducing the Covariance and Correlation functions
    • +
  • Correlation Function and Design/Feature Matrix
    • +
  • Covariance Matrix Examples
    • +
  • Correlation Matrix
    • +
  • Correlation Matrix with Pandas
    • +
  • Correlation Matrix with Pandas and the Franke function
    • +
  • Rewriting the Covariance and/or Correlation Matrix
    • +
  • Towards the PCA theorem
    • +
  • The Algorithm before the Theorem
    • +
  • Writing our own PCA code
    • +
  •    Compute the sample mean and center the data
    • +
  •    Compute the sample covariance
    • +
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • +
  • Classical PCA Theorem
    • +
  • Proof of the PCA Theorem
    • +
  • PCA Proof continued
    • +
  • The final step
    • +
  • Geometric Interpretation and link with Singular Value Decomposition
    • +
  • Principal Component Analysis
    • +
  • PCA and scikit-learn
    • +
  • Back to the Cancer Data
    • +
  • More on the PCA
    • +
  • Incremental PCA
    • +
  • Randomized PCA
    • +
  • Kernel PCA
    • +
  • LLE
    • +
  • Other techniques
    • @@ -305,6 +306,10 @@ MathJax.Hub.Config({

    Recurrent Neural Networks

    +

    +Overview video. +See also lecture on Thursday October 22 and examples from week 42. +

    @@ -323,7 +328,7 @@ MathJax.Hub.Config({

  • 11
  • 12
  • ...
    • -
  • 76
    • +
  • 77
  • »
    • diff --git a/doc/pub/week43/html/._week43-bs003.html b/doc/pub/week43/html/._week43-bs003.html index 1a10efdf0..9c2adb8d4 100644 --- a/doc/pub/week43/html/._week43-bs003.html +++ b/doc/pub/week43/html/._week43-bs003.html @@ -67,7 +67,7 @@ Automatically generated HTML file from DocOnce source ('Final technicalities II', 2, None, '___sec16'), ('Final technicalities III', 2, None, '___sec17'), ('Final technicalities IV', 2, None, '___sec18'), - ('Backpropagation', 2, None, '___sec19'), + ('Back propagation', 2, None, '___sec19'), ('Gradient descent', 2, None, '___sec20'), ('The code for solving the ODE', 2, None, '___sec21'), ('The network with one input layer, specified number of hidden ' @@ -80,100 +80,100 @@ Automatically generated HTML file from DocOnce source ('The trial solution', 2, None, '___sec25'), ('The program using Autograd', 2, None, '___sec26'), ('Using forward Euler to solve the ODE', 2, None, '___sec27'), - ('Example: Solving the one dimensional Poisson equation using ' - 'Autograd and TensorFlow', + ('Example: Solving the one dimensional Poisson equation', 2, None, '___sec28'), ('The specific equation to solve for', 2, None, '___sec29'), ('Solving the equation using Autograd', 2, None, '___sec30'), ('Comparing with a numerical scheme', 2, None, '___sec31'), - ('Partial Differential Equations', 2, None, '___sec32'), - ('Type of problem', 2, None, '___sec33'), - ('Network requirements', 2, None, '___sec34'), - ('More details', 2, None, '___sec35'), - ('Example: The diffusion equation', 2, None, '___sec36'), - ('Defining the problem', 2, None, '___sec37'), - ('Setting up the network using Autograd', 2, None, '___sec38'), + ('Setting up the code', 2, None, '___sec32'), + ('Partial Differential Equations', 2, None, '___sec33'), + ('Type of problem', 2, None, '___sec34'), + ('Network requirements', 2, None, '___sec35'), + ('More details', 2, None, '___sec36'), + ('Example: The diffusion equation', 2, None, '___sec37'), + ('Defining the problem', 2, None, '___sec38'), + ('Setting up the network using Autograd', 2, None, '___sec39'), ('Setting up the network using Autograd; The trial solution', 2, None, - '___sec39'), - ('Why the jacobian?', 2, None, '___sec40'), + '___sec40'), + ('Why the jacobian?', 2, None, '___sec41'), ('Setting up the network using Autograd; The full program', 2, None, - '___sec41'), + '___sec42'), ('Example: Solving the wave equation with Neural Networks', 2, None, - '___sec42'), - ('The problem to solve for', 2, None, '___sec43'), - ('The trial solution', 2, None, '___sec44'), - ('The analytical solution', 2, None, '___sec45'), + '___sec43'), + ('The problem to solve for', 2, None, '___sec44'), + ('The trial solution', 2, None, '___sec45'), + ('The analytical solution', 2, None, '___sec46'), ('Solving the wave equation - the full program using Autograd', 2, None, - '___sec46'), + '___sec47'), ('Resources on differential equations and deep learning', 2, None, - '___sec47'), - ('Friday, Principal Component Analysis', 2, None, '___sec48'), + '___sec48'), + ('Friday, Principal Component Analysis', 2, None, '___sec49'), ('Basic ideas of the Principal Component Analysis (PCA)', 2, None, - '___sec49'), + '___sec50'), ('Introducing the Covariance and Correlation functions', 2, None, - '___sec50'), + '___sec51'), ('Correlation Function and Design/Feature Matrix', 2, None, - '___sec51'), - ('Covariance Matrix Examples', 2, None, '___sec52'), - ('Correlation Matrix', 2, None, '___sec53'), - ('Correlation Matrix with Pandas', 2, None, '___sec54'), + '___sec52'), + ('Covariance Matrix Examples', 2, None, '___sec53'), + ('Correlation Matrix', 2, None, '___sec54'), + ('Correlation Matrix with Pandas', 2, None, '___sec55'), ('Correlation Matrix with Pandas and the Franke function', 2, None, - '___sec55'), + '___sec56'), ('Rewriting the Covariance and/or Correlation Matrix', 2, None, - '___sec56'), - ('Towards the PCA theorem', 2, None, '___sec57'), - ('The Algorithm before the Theorem', 2, None, '___sec58'), - ('Writing our own PCA code', 2, None, '___sec59'), + '___sec57'), + ('Towards the PCA theorem', 2, None, '___sec58'), + ('The Algorithm before the Theorem', 2, None, '___sec59'), + ('Writing our own PCA code', 2, None, '___sec60'), ('Compute the sample mean and center the data', 3, None, - '___sec60'), - ('Compute the sample covariance', 3, None, '___sec61'), + '___sec61'), + ('Compute the sample covariance', 3, None, '___sec62'), ('Diagonalize the sample covariance matrix to obtain the ' 'principal components', 3, None, - '___sec62'), - ('Classical PCA Theorem', 2, None, '___sec63'), - ('Proof of the PCA Theorem', 2, None, '___sec64'), - ('PCA Proof continued', 2, None, '___sec65'), - ('The final step', 2, None, '___sec66'), + '___sec63'), + ('Classical PCA Theorem', 2, None, '___sec64'), + ('Proof of the PCA Theorem', 2, None, '___sec65'), + ('PCA Proof continued', 2, None, '___sec66'), + ('The final step', 2, None, '___sec67'), ('Geometric Interpretation and link with Singular Value ' 'Decomposition', 2, None, - '___sec67'), - ('Principal Component Analysis', 2, None, '___sec68'), - ('PCA and scikit-learn', 2, None, '___sec69'), - ('Back to the Cancer Data', 2, None, '___sec70'), - ('More on the PCA', 2, None, '___sec71'), - ('Incremental PCA', 2, None, '___sec72'), - ('Randomized PCA', 2, None, '___sec73'), - ('Kernel PCA', 2, None, '___sec74'), - ('LLE', 2, None, '___sec75'), - ('Other techniques', 2, None, '___sec76')]} + '___sec68'), + ('Principal Component Analysis', 2, None, '___sec69'), + ('PCA and scikit-learn', 2, None, '___sec70'), + ('Back to the Cancer Data', 2, None, '___sec71'), + ('More on the PCA', 2, None, '___sec72'), + ('Incremental PCA', 2, None, '___sec73'), + ('Randomized PCA', 2, None, '___sec74'), + ('Kernel PCA', 2, None, '___sec75'), + ('LLE', 2, None, '___sec76'), + ('Other techniques', 2, None, '___sec77')]} end of tocinfo --> @@ -230,7 +230,7 @@ MathJax.Hub.Config({
  • Final technicalities II
  • Final technicalities III
  • Final technicalities IV
    • -
  • Backpropagation
    • +
  • Back propagation
  • Gradient descent
  • The code for solving the ODE
  • The network with one input layer, specified number of hidden layers, and one output layer
    • @@ -239,55 +239,56 @@ MathJax.Hub.Config({
  • The trial solution
  • The program using Autograd
  • Using forward Euler to solve the ODE
    • -
  • Example: Solving the one dimensional Poisson equation using Autograd and TensorFlow
    • +
  • Example: Solving the one dimensional Poisson equation
  • The specific equation to solve for
  • Solving the equation using Autograd
  • Comparing with a numerical scheme
    • -
  • Partial Differential Equations
    • -
  • Type of problem
    • -
  • Network requirements
    • -
  • More details
    • -
  • Example: The diffusion equation
    • -
  • Defining the problem
    • -
  • Setting up the network using Autograd
    • -
  • Setting up the network using Autograd; The trial solution
    • -
  • Why the jacobian?
    • -
  • Setting up the network using Autograd; The full program
    • -
  • Example: Solving the wave equation with Neural Networks
    • -
  • The problem to solve for
    • -
  • The trial solution
    • -
  • The analytical solution
    • -
  • Solving the wave equation - the full program using Autograd
    • -
  • Resources on differential equations and deep learning
    • -
  • Friday, Principal Component Analysis
    • -
  • Basic ideas of the Principal Component Analysis (PCA)
    • -
  • Introducing the Covariance and Correlation functions
    • -
  • Correlation Function and Design/Feature Matrix
    • -
  • Covariance Matrix Examples
    • -
  • Correlation Matrix
    • -
  • Correlation Matrix with Pandas
    • -
  • Correlation Matrix with Pandas and the Franke function
    • -
  • Rewriting the Covariance and/or Correlation Matrix
    • -
  • Towards the PCA theorem
    • -
  • The Algorithm before the Theorem
    • -
  • Writing our own PCA code
    • -
  •    Compute the sample mean and center the data
    • -
  •    Compute the sample covariance
    • -
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • -
  • Classical PCA Theorem
    • -
  • Proof of the PCA Theorem
    • -
  • PCA Proof continued
    • -
  • The final step
    • -
  • Geometric Interpretation and link with Singular Value Decomposition
    • -
  • Principal Component Analysis
    • -
  • PCA and scikit-learn
    • -
  • Back to the Cancer Data
    • -
  • More on the PCA
    • -
  • Incremental PCA
    • -
  • Randomized PCA
    • -
  • Kernel PCA
    • -
  • LLE
    • -
  • Other techniques
    • +
  • Setting up the code
    • +
  • Partial Differential Equations
    • +
  • Type of problem
    • +
  • Network requirements
    • +
  • More details
    • +
  • Example: The diffusion equation
    • +
  • Defining the problem
    • +
  • Setting up the network using Autograd
    • +
  • Setting up the network using Autograd; The trial solution
    • +
  • Why the jacobian?
    • +
  • Setting up the network using Autograd; The full program
    • +
  • Example: Solving the wave equation with Neural Networks
    • +
  • The problem to solve for
    • +
  • The trial solution
    • +
  • The analytical solution
    • +
  • Solving the wave equation - the full program using Autograd
    • +
  • Resources on differential equations and deep learning
    • +
  • Friday, Principal Component Analysis
    • +
  • Basic ideas of the Principal Component Analysis (PCA)
    • +
  • Introducing the Covariance and Correlation functions
    • +
  • Correlation Function and Design/Feature Matrix
    • +
  • Covariance Matrix Examples
    • +
  • Correlation Matrix
    • +
  • Correlation Matrix with Pandas
    • +
  • Correlation Matrix with Pandas and the Franke function
    • +
  • Rewriting the Covariance and/or Correlation Matrix
    • +
  • Towards the PCA theorem
    • +
  • The Algorithm before the Theorem
    • +
  • Writing our own PCA code
    • +
  •    Compute the sample mean and center the data
    • +
  •    Compute the sample covariance
    • +
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • +
  • Classical PCA Theorem
    • +
  • Proof of the PCA Theorem
    • +
  • PCA Proof continued
    • +
  • The final step
    • +
  • Geometric Interpretation and link with Singular Value Decomposition
    • +
  • Principal Component Analysis
    • +
  • PCA and scikit-learn
    • +
  • Back to the Cancer Data
    • +
  • More on the PCA
    • +
  • Incremental PCA
    • +
  • Randomized PCA
    • +
  • Kernel PCA
    • +
  • LLE
    • +
  • Other techniques
    • @@ -329,7 +330,7 @@ and output layer to any given precision.
  • 12
  • 13
  • ...
    • -
  • 76
    • +
  • 77
  • »
    • diff --git a/doc/pub/week43/html/._week43-bs004.html b/doc/pub/week43/html/._week43-bs004.html index fc746cbef..39cf34075 100644 --- a/doc/pub/week43/html/._week43-bs004.html +++ b/doc/pub/week43/html/._week43-bs004.html @@ -67,7 +67,7 @@ Automatically generated HTML file from DocOnce source ('Final technicalities II', 2, None, '___sec16'), ('Final technicalities III', 2, None, '___sec17'), ('Final technicalities IV', 2, None, '___sec18'), - ('Backpropagation', 2, None, '___sec19'), + ('Back propagation', 2, None, '___sec19'), ('Gradient descent', 2, None, '___sec20'), ('The code for solving the ODE', 2, None, '___sec21'), ('The network with one input layer, specified number of hidden ' @@ -80,100 +80,100 @@ Automatically generated HTML file from DocOnce source ('The trial solution', 2, None, '___sec25'), ('The program using Autograd', 2, None, '___sec26'), ('Using forward Euler to solve the ODE', 2, None, '___sec27'), - ('Example: Solving the one dimensional Poisson equation using ' - 'Autograd and TensorFlow', + ('Example: Solving the one dimensional Poisson equation', 2, None, '___sec28'), ('The specific equation to solve for', 2, None, '___sec29'), ('Solving the equation using Autograd', 2, None, '___sec30'), ('Comparing with a numerical scheme', 2, None, '___sec31'), - ('Partial Differential Equations', 2, None, '___sec32'), - ('Type of problem', 2, None, '___sec33'), - ('Network requirements', 2, None, '___sec34'), - ('More details', 2, None, '___sec35'), - ('Example: The diffusion equation', 2, None, '___sec36'), - ('Defining the problem', 2, None, '___sec37'), - ('Setting up the network using Autograd', 2, None, '___sec38'), + ('Setting up the code', 2, None, '___sec32'), + ('Partial Differential Equations', 2, None, '___sec33'), + ('Type of problem', 2, None, '___sec34'), + ('Network requirements', 2, None, '___sec35'), + ('More details', 2, None, '___sec36'), + ('Example: The diffusion equation', 2, None, '___sec37'), + ('Defining the problem', 2, None, '___sec38'), + ('Setting up the network using Autograd', 2, None, '___sec39'), ('Setting up the network using Autograd; The trial solution', 2, None, - '___sec39'), - ('Why the jacobian?', 2, None, '___sec40'), + '___sec40'), + ('Why the jacobian?', 2, None, '___sec41'), ('Setting up the network using Autograd; The full program', 2, None, - '___sec41'), + '___sec42'), ('Example: Solving the wave equation with Neural Networks', 2, None, - '___sec42'), - ('The problem to solve for', 2, None, '___sec43'), - ('The trial solution', 2, None, '___sec44'), - ('The analytical solution', 2, None, '___sec45'), + '___sec43'), + ('The problem to solve for', 2, None, '___sec44'), + ('The trial solution', 2, None, '___sec45'), + ('The analytical solution', 2, None, '___sec46'), ('Solving the wave equation - the full program using Autograd', 2, None, - '___sec46'), + '___sec47'), ('Resources on differential equations and deep learning', 2, None, - '___sec47'), - ('Friday, Principal Component Analysis', 2, None, '___sec48'), + '___sec48'), + ('Friday, Principal Component Analysis', 2, None, '___sec49'), ('Basic ideas of the Principal Component Analysis (PCA)', 2, None, - '___sec49'), + '___sec50'), ('Introducing the Covariance and Correlation functions', 2, None, - '___sec50'), + '___sec51'), ('Correlation Function and Design/Feature Matrix', 2, None, - '___sec51'), - ('Covariance Matrix Examples', 2, None, '___sec52'), - ('Correlation Matrix', 2, None, '___sec53'), - ('Correlation Matrix with Pandas', 2, None, '___sec54'), + '___sec52'), + ('Covariance Matrix Examples', 2, None, '___sec53'), + ('Correlation Matrix', 2, None, '___sec54'), + ('Correlation Matrix with Pandas', 2, None, '___sec55'), ('Correlation Matrix with Pandas and the Franke function', 2, None, - '___sec55'), + '___sec56'), ('Rewriting the Covariance and/or Correlation Matrix', 2, None, - '___sec56'), - ('Towards the PCA theorem', 2, None, '___sec57'), - ('The Algorithm before the Theorem', 2, None, '___sec58'), - ('Writing our own PCA code', 2, None, '___sec59'), + '___sec57'), + ('Towards the PCA theorem', 2, None, '___sec58'), + ('The Algorithm before the Theorem', 2, None, '___sec59'), + ('Writing our own PCA code', 2, None, '___sec60'), ('Compute the sample mean and center the data', 3, None, - '___sec60'), - ('Compute the sample covariance', 3, None, '___sec61'), + '___sec61'), + ('Compute the sample covariance', 3, None, '___sec62'), ('Diagonalize the sample covariance matrix to obtain the ' 'principal components', 3, None, - '___sec62'), - ('Classical PCA Theorem', 2, None, '___sec63'), - ('Proof of the PCA Theorem', 2, None, '___sec64'), - ('PCA Proof continued', 2, None, '___sec65'), - ('The final step', 2, None, '___sec66'), + '___sec63'), + ('Classical PCA Theorem', 2, None, '___sec64'), + ('Proof of the PCA Theorem', 2, None, '___sec65'), + ('PCA Proof continued', 2, None, '___sec66'), + ('The final step', 2, None, '___sec67'), ('Geometric Interpretation and link with Singular Value ' 'Decomposition', 2, None, - '___sec67'), - ('Principal Component Analysis', 2, None, '___sec68'), - ('PCA and scikit-learn', 2, None, '___sec69'), - ('Back to the Cancer Data', 2, None, '___sec70'), - ('More on the PCA', 2, None, '___sec71'), - ('Incremental PCA', 2, None, '___sec72'), - ('Randomized PCA', 2, None, '___sec73'), - ('Kernel PCA', 2, None, '___sec74'), - ('LLE', 2, None, '___sec75'), - ('Other techniques', 2, None, '___sec76')]} + '___sec68'), + ('Principal Component Analysis', 2, None, '___sec69'), + ('PCA and scikit-learn', 2, None, '___sec70'), + ('Back to the Cancer Data', 2, None, '___sec71'), + ('More on the PCA', 2, None, '___sec72'), + ('Incremental PCA', 2, None, '___sec73'), + ('Randomized PCA', 2, None, '___sec74'), + ('Kernel PCA', 2, None, '___sec75'), + ('LLE', 2, None, '___sec76'), + ('Other techniques', 2, None, '___sec77')]} end of tocinfo --> @@ -230,7 +230,7 @@ MathJax.Hub.Config({
  • Final technicalities II
  • Final technicalities III
  • Final technicalities IV
    • -
  • Backpropagation
    • +
  • Back propagation
  • Gradient descent
  • The code for solving the ODE
  • The network with one input layer, specified number of hidden layers, and one output layer
    • @@ -239,55 +239,56 @@ MathJax.Hub.Config({
  • The trial solution
  • The program using Autograd
  • Using forward Euler to solve the ODE
    • -
  • Example: Solving the one dimensional Poisson equation using Autograd and TensorFlow
    • +
  • Example: Solving the one dimensional Poisson equation
  • The specific equation to solve for
  • Solving the equation using Autograd
  • Comparing with a numerical scheme
    • -
  • Partial Differential Equations
    • -
  • Type of problem
    • -
  • Network requirements
    • -
  • More details
    • -
  • Example: The diffusion equation
    • -
  • Defining the problem
    • -
  • Setting up the network using Autograd
    • -
  • Setting up the network using Autograd; The trial solution
    • -
  • Why the jacobian?
    • -
  • Setting up the network using Autograd; The full program
    • -
  • Example: Solving the wave equation with Neural Networks
    • -
  • The problem to solve for
    • -
  • The trial solution
    • -
  • The analytical solution
    • -
  • Solving the wave equation - the full program using Autograd
    • -
  • Resources on differential equations and deep learning
    • -
  • Friday, Principal Component Analysis
    • -
  • Basic ideas of the Principal Component Analysis (PCA)
    • -
  • Introducing the Covariance and Correlation functions
    • -
  • Correlation Function and Design/Feature Matrix
    • -
  • Covariance Matrix Examples
    • -
  • Correlation Matrix
    • -
  • Correlation Matrix with Pandas
    • -
  • Correlation Matrix with Pandas and the Franke function
    • -
  • Rewriting the Covariance and/or Correlation Matrix
    • -
  • Towards the PCA theorem
    • -
  • The Algorithm before the Theorem
    • -
  • Writing our own PCA code
    • -
  •    Compute the sample mean and center the data
    • -
  •    Compute the sample covariance
    • -
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • -
  • Classical PCA Theorem
    • -
  • Proof of the PCA Theorem
    • -
  • PCA Proof continued
    • -
  • The final step
    • -
  • Geometric Interpretation and link with Singular Value Decomposition
    • -
  • Principal Component Analysis
    • -
  • PCA and scikit-learn
    • -
  • Back to the Cancer Data
    • -
  • More on the PCA
    • -
  • Incremental PCA
    • -
  • Randomized PCA
    • -
  • Kernel PCA
    • -
  • LLE
    • -
  • Other techniques
    • +
  • Setting up the code
    • +
  • Partial Differential Equations
    • +
  • Type of problem
    • +
  • Network requirements
    • +
  • More details
    • +
  • Example: The diffusion equation
    • +
  • Defining the problem
    • +
  • Setting up the network using Autograd
    • +
  • Setting up the network using Autograd; The trial solution
    • +
  • Why the jacobian?
    • +
  • Setting up the network using Autograd; The full program
    • +
  • Example: Solving the wave equation with Neural Networks
    • +
  • The problem to solve for
    • +
  • The trial solution
    • +
  • The analytical solution
    • +
  • Solving the wave equation - the full program using Autograd
    • +
  • Resources on differential equations and deep learning
    • +
  • Friday, Principal Component Analysis
    • +
  • Basic ideas of the Principal Component Analysis (PCA)
    • +
  • Introducing the Covariance and Correlation functions
    • +
  • Correlation Function and Design/Feature Matrix
    • +
  • Covariance Matrix Examples
    • +
  • Correlation Matrix
    • +
  • Correlation Matrix with Pandas
    • +
  • Correlation Matrix with Pandas and the Franke function
    • +
  • Rewriting the Covariance and/or Correlation Matrix
    • +
  • Towards the PCA theorem
    • +
  • The Algorithm before the Theorem
    • +
  • Writing our own PCA code
    • +
  •    Compute the sample mean and center the data
    • +
  •    Compute the sample covariance
    • +
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • +
  • Classical PCA Theorem
    • +
  • Proof of the PCA Theorem
    • +
  • PCA Proof continued
    • +
  • The final step
    • +
  • Geometric Interpretation and link with Singular Value Decomposition
    • +
  • Principal Component Analysis
    • +
  • PCA and scikit-learn
    • +
  • Back to the Cancer Data
    • +
  • More on the PCA
    • +
  • Incremental PCA
    • +
  • Randomized PCA
    • +
  • Kernel PCA
    • +
  • LLE
    • +
  • Other techniques
    • @@ -347,7 +348,7 @@ for the solution to be unique.
  • 13
  • 14
  • ...
    • -
  • 76
    • +
  • 77
  • »
    • diff --git a/doc/pub/week43/html/._week43-bs005.html b/doc/pub/week43/html/._week43-bs005.html index 10cc92a61..61d6bd835 100644 --- a/doc/pub/week43/html/._week43-bs005.html +++ b/doc/pub/week43/html/._week43-bs005.html @@ -67,7 +67,7 @@ Automatically generated HTML file from DocOnce source ('Final technicalities II', 2, None, '___sec16'), ('Final technicalities III', 2, None, '___sec17'), ('Final technicalities IV', 2, None, '___sec18'), - ('Backpropagation', 2, None, '___sec19'), + ('Back propagation', 2, None, '___sec19'), ('Gradient descent', 2, None, '___sec20'), ('The code for solving the ODE', 2, None, '___sec21'), ('The network with one input layer, specified number of hidden ' @@ -80,100 +80,100 @@ Automatically generated HTML file from DocOnce source ('The trial solution', 2, None, '___sec25'), ('The program using Autograd', 2, None, '___sec26'), ('Using forward Euler to solve the ODE', 2, None, '___sec27'), - ('Example: Solving the one dimensional Poisson equation using ' - 'Autograd and TensorFlow', + ('Example: Solving the one dimensional Poisson equation', 2, None, '___sec28'), ('The specific equation to solve for', 2, None, '___sec29'), ('Solving the equation using Autograd', 2, None, '___sec30'), ('Comparing with a numerical scheme', 2, None, '___sec31'), - ('Partial Differential Equations', 2, None, '___sec32'), - ('Type of problem', 2, None, '___sec33'), - ('Network requirements', 2, None, '___sec34'), - ('More details', 2, None, '___sec35'), - ('Example: The diffusion equation', 2, None, '___sec36'), - ('Defining the problem', 2, None, '___sec37'), - ('Setting up the network using Autograd', 2, None, '___sec38'), + ('Setting up the code', 2, None, '___sec32'), + ('Partial Differential Equations', 2, None, '___sec33'), + ('Type of problem', 2, None, '___sec34'), + ('Network requirements', 2, None, '___sec35'), + ('More details', 2, None, '___sec36'), + ('Example: The diffusion equation', 2, None, '___sec37'), + ('Defining the problem', 2, None, '___sec38'), + ('Setting up the network using Autograd', 2, None, '___sec39'), ('Setting up the network using Autograd; The trial solution', 2, None, - '___sec39'), - ('Why the jacobian?', 2, None, '___sec40'), + '___sec40'), + ('Why the jacobian?', 2, None, '___sec41'), ('Setting up the network using Autograd; The full program', 2, None, - '___sec41'), + '___sec42'), ('Example: Solving the wave equation with Neural Networks', 2, None, - '___sec42'), - ('The problem to solve for', 2, None, '___sec43'), - ('The trial solution', 2, None, '___sec44'), - ('The analytical solution', 2, None, '___sec45'), + '___sec43'), + ('The problem to solve for', 2, None, '___sec44'), + ('The trial solution', 2, None, '___sec45'), + ('The analytical solution', 2, None, '___sec46'), ('Solving the wave equation - the full program using Autograd', 2, None, - '___sec46'), + '___sec47'), ('Resources on differential equations and deep learning', 2, None, - '___sec47'), - ('Friday, Principal Component Analysis', 2, None, '___sec48'), + '___sec48'), + ('Friday, Principal Component Analysis', 2, None, '___sec49'), ('Basic ideas of the Principal Component Analysis (PCA)', 2, None, - '___sec49'), + '___sec50'), ('Introducing the Covariance and Correlation functions', 2, None, - '___sec50'), + '___sec51'), ('Correlation Function and Design/Feature Matrix', 2, None, - '___sec51'), - ('Covariance Matrix Examples', 2, None, '___sec52'), - ('Correlation Matrix', 2, None, '___sec53'), - ('Correlation Matrix with Pandas', 2, None, '___sec54'), + '___sec52'), + ('Covariance Matrix Examples', 2, None, '___sec53'), + ('Correlation Matrix', 2, None, '___sec54'), + ('Correlation Matrix with Pandas', 2, None, '___sec55'), ('Correlation Matrix with Pandas and the Franke function', 2, None, - '___sec55'), + '___sec56'), ('Rewriting the Covariance and/or Correlation Matrix', 2, None, - '___sec56'), - ('Towards the PCA theorem', 2, None, '___sec57'), - ('The Algorithm before the Theorem', 2, None, '___sec58'), - ('Writing our own PCA code', 2, None, '___sec59'), + '___sec57'), + ('Towards the PCA theorem', 2, None, '___sec58'), + ('The Algorithm before the Theorem', 2, None, '___sec59'), + ('Writing our own PCA code', 2, None, '___sec60'), ('Compute the sample mean and center the data', 3, None, - '___sec60'), - ('Compute the sample covariance', 3, None, '___sec61'), + '___sec61'), + ('Compute the sample covariance', 3, None, '___sec62'), ('Diagonalize the sample covariance matrix to obtain the ' 'principal components', 3, None, - '___sec62'), - ('Classical PCA Theorem', 2, None, '___sec63'), - ('Proof of the PCA Theorem', 2, None, '___sec64'), - ('PCA Proof continued', 2, None, '___sec65'), - ('The final step', 2, None, '___sec66'), + '___sec63'), + ('Classical PCA Theorem', 2, None, '___sec64'), + ('Proof of the PCA Theorem', 2, None, '___sec65'), + ('PCA Proof continued', 2, None, '___sec66'), + ('The final step', 2, None, '___sec67'), ('Geometric Interpretation and link with Singular Value ' 'Decomposition', 2, None, - '___sec67'), - ('Principal Component Analysis', 2, None, '___sec68'), - ('PCA and scikit-learn', 2, None, '___sec69'), - ('Back to the Cancer Data', 2, None, '___sec70'), - ('More on the PCA', 2, None, '___sec71'), - ('Incremental PCA', 2, None, '___sec72'), - ('Randomized PCA', 2, None, '___sec73'), - ('Kernel PCA', 2, None, '___sec74'), - ('LLE', 2, None, '___sec75'), - ('Other techniques', 2, None, '___sec76')]} + '___sec68'), + ('Principal Component Analysis', 2, None, '___sec69'), + ('PCA and scikit-learn', 2, None, '___sec70'), + ('Back to the Cancer Data', 2, None, '___sec71'), + ('More on the PCA', 2, None, '___sec72'), + ('Incremental PCA', 2, None, '___sec73'), + ('Randomized PCA', 2, None, '___sec74'), + ('Kernel PCA', 2, None, '___sec75'), + ('LLE', 2, None, '___sec76'), + ('Other techniques', 2, None, '___sec77')]} end of tocinfo --> @@ -230,7 +230,7 @@ MathJax.Hub.Config({
  • Final technicalities II
  • Final technicalities III
  • Final technicalities IV
    • -
  • Backpropagation
    • +
  • Back propagation
  • Gradient descent
  • The code for solving the ODE
  • The network with one input layer, specified number of hidden layers, and one output layer
    • @@ -239,55 +239,56 @@ MathJax.Hub.Config({
  • The trial solution
  • The program using Autograd
  • Using forward Euler to solve the ODE
    • -
  • Example: Solving the one dimensional Poisson equation using Autograd and TensorFlow
    • +
  • Example: Solving the one dimensional Poisson equation
  • The specific equation to solve for
  • Solving the equation using Autograd
  • Comparing with a numerical scheme
    • -
  • Partial Differential Equations
    • -
  • Type of problem
    • -
  • Network requirements
    • -
  • More details
    • -
  • Example: The diffusion equation
    • -
  • Defining the problem
    • -
  • Setting up the network using Autograd
    • -
  • Setting up the network using Autograd; The trial solution
    • -
  • Why the jacobian?
    • -
  • Setting up the network using Autograd; The full program
    • -
  • Example: Solving the wave equation with Neural Networks
    • -
  • The problem to solve for
    • -
  • The trial solution
    • -
  • The analytical solution
    • -
  • Solving the wave equation - the full program using Autograd
    • -
  • Resources on differential equations and deep learning
    • -
  • Friday, Principal Component Analysis
    • -
  • Basic ideas of the Principal Component Analysis (PCA)
    • -
  • Introducing the Covariance and Correlation functions
    • -
  • Correlation Function and Design/Feature Matrix
    • -
  • Covariance Matrix Examples
    • -
  • Correlation Matrix
    • -
  • Correlation Matrix with Pandas
    • -
  • Correlation Matrix with Pandas and the Franke function
    • -
  • Rewriting the Covariance and/or Correlation Matrix
    • -
  • Towards the PCA theorem
    • -
  • The Algorithm before the Theorem
    • -
  • Writing our own PCA code
    • -
  •    Compute the sample mean and center the data
    • -
  •    Compute the sample covariance
    • -
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • -
  • Classical PCA Theorem
    • -
  • Proof of the PCA Theorem
    • -
  • PCA Proof continued
    • -
  • The final step
    • -
  • Geometric Interpretation and link with Singular Value Decomposition
    • -
  • Principal Component Analysis
    • -
  • PCA and scikit-learn
    • -
  • Back to the Cancer Data
    • -
  • More on the PCA
    • -
  • Incremental PCA
    • -
  • Randomized PCA
    • -
  • Kernel PCA
    • -
  • LLE
    • -
  • Other techniques
    • +
  • Setting up the code
    • +
  • Partial Differential Equations
    • +
  • Type of problem
    • +
  • Network requirements
    • +
  • More details
    • +
  • Example: The diffusion equation
    • +
  • Defining the problem
    • +
  • Setting up the network using Autograd
    • +
  • Setting up the network using Autograd; The trial solution
    • +
  • Why the jacobian?
    • +
  • Setting up the network using Autograd; The full program
    • +
  • Example: Solving the wave equation with Neural Networks
    • +
  • The problem to solve for
    • +
  • The trial solution
    • +
  • The analytical solution
    • +
  • Solving the wave equation - the full program using Autograd
    • +
  • Resources on differential equations and deep learning
    • +
  • Friday, Principal Component Analysis
    • +
  • Basic ideas of the Principal Component Analysis (PCA)
    • +
  • Introducing the Covariance and Correlation functions
    • +
  • Correlation Function and Design/Feature Matrix
    • +
  • Covariance Matrix Examples
    • +
  • Correlation Matrix
    • +
  • Correlation Matrix with Pandas
    • +
  • Correlation Matrix with Pandas and the Franke function
    • +
  • Rewriting the Covariance and/or Correlation Matrix
    • +
  • Towards the PCA theorem
    • +
  • The Algorithm before the Theorem
    • +
  • Writing our own PCA code
    • +
  •    Compute the sample mean and center the data
    • +
  •    Compute the sample covariance
    • +
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • +
  • Classical PCA Theorem
    • +
  • Proof of the PCA Theorem
    • +
  • PCA Proof continued
    • +
  • The final step
    • +
  • Geometric Interpretation and link with Singular Value Decomposition
    • +
  • Principal Component Analysis
    • +
  • PCA and scikit-learn
    • +
  • Back to the Cancer Data
    • +
  • More on the PCA
    • +
  • Incremental PCA
    • +
  • Randomized PCA
    • +
  • Kernel PCA
    • +
  • LLE
    • +
  • Other techniques
    • @@ -352,7 +353,7 @@ As described previously, an optimization method could be used to minimize the pa
  • 14
  • 15
  • ...
    • -
  • 76
    • +
  • 77
  • »
    • diff --git a/doc/pub/week43/html/._week43-bs006.html b/doc/pub/week43/html/._week43-bs006.html index 625c20e3e..82f9aa18e 100644 --- a/doc/pub/week43/html/._week43-bs006.html +++ b/doc/pub/week43/html/._week43-bs006.html @@ -67,7 +67,7 @@ Automatically generated HTML file from DocOnce source ('Final technicalities II', 2, None, '___sec16'), ('Final technicalities III', 2, None, '___sec17'), ('Final technicalities IV', 2, None, '___sec18'), - ('Backpropagation', 2, None, '___sec19'), + ('Back propagation', 2, None, '___sec19'), ('Gradient descent', 2, None, '___sec20'), ('The code for solving the ODE', 2, None, '___sec21'), ('The network with one input layer, specified number of hidden ' @@ -80,100 +80,100 @@ Automatically generated HTML file from DocOnce source ('The trial solution', 2, None, '___sec25'), ('The program using Autograd', 2, None, '___sec26'), ('Using forward Euler to solve the ODE', 2, None, '___sec27'), - ('Example: Solving the one dimensional Poisson equation using ' - 'Autograd and TensorFlow', + ('Example: Solving the one dimensional Poisson equation', 2, None, '___sec28'), ('The specific equation to solve for', 2, None, '___sec29'), ('Solving the equation using Autograd', 2, None, '___sec30'), ('Comparing with a numerical scheme', 2, None, '___sec31'), - ('Partial Differential Equations', 2, None, '___sec32'), - ('Type of problem', 2, None, '___sec33'), - ('Network requirements', 2, None, '___sec34'), - ('More details', 2, None, '___sec35'), - ('Example: The diffusion equation', 2, None, '___sec36'), - ('Defining the problem', 2, None, '___sec37'), - ('Setting up the network using Autograd', 2, None, '___sec38'), + ('Setting up the code', 2, None, '___sec32'), + ('Partial Differential Equations', 2, None, '___sec33'), + ('Type of problem', 2, None, '___sec34'), + ('Network requirements', 2, None, '___sec35'), + ('More details', 2, None, '___sec36'), + ('Example: The diffusion equation', 2, None, '___sec37'), + ('Defining the problem', 2, None, '___sec38'), + ('Setting up the network using Autograd', 2, None, '___sec39'), ('Setting up the network using Autograd; The trial solution', 2, None, - '___sec39'), - ('Why the jacobian?', 2, None, '___sec40'), + '___sec40'), + ('Why the jacobian?', 2, None, '___sec41'), ('Setting up the network using Autograd; The full program', 2, None, - '___sec41'), + '___sec42'), ('Example: Solving the wave equation with Neural Networks', 2, None, - '___sec42'), - ('The problem to solve for', 2, None, '___sec43'), - ('The trial solution', 2, None, '___sec44'), - ('The analytical solution', 2, None, '___sec45'), + '___sec43'), + ('The problem to solve for', 2, None, '___sec44'), + ('The trial solution', 2, None, '___sec45'), + ('The analytical solution', 2, None, '___sec46'), ('Solving the wave equation - the full program using Autograd', 2, None, - '___sec46'), + '___sec47'), ('Resources on differential equations and deep learning', 2, None, - '___sec47'), - ('Friday, Principal Component Analysis', 2, None, '___sec48'), + '___sec48'), + ('Friday, Principal Component Analysis', 2, None, '___sec49'), ('Basic ideas of the Principal Component Analysis (PCA)', 2, None, - '___sec49'), + '___sec50'), ('Introducing the Covariance and Correlation functions', 2, None, - '___sec50'), + '___sec51'), ('Correlation Function and Design/Feature Matrix', 2, None, - '___sec51'), - ('Covariance Matrix Examples', 2, None, '___sec52'), - ('Correlation Matrix', 2, None, '___sec53'), - ('Correlation Matrix with Pandas', 2, None, '___sec54'), + '___sec52'), + ('Covariance Matrix Examples', 2, None, '___sec53'), + ('Correlation Matrix', 2, None, '___sec54'), + ('Correlation Matrix with Pandas', 2, None, '___sec55'), ('Correlation Matrix with Pandas and the Franke function', 2, None, - '___sec55'), + '___sec56'), ('Rewriting the Covariance and/or Correlation Matrix', 2, None, - '___sec56'), - ('Towards the PCA theorem', 2, None, '___sec57'), - ('The Algorithm before the Theorem', 2, None, '___sec58'), - ('Writing our own PCA code', 2, None, '___sec59'), + '___sec57'), + ('Towards the PCA theorem', 2, None, '___sec58'), + ('The Algorithm before the Theorem', 2, None, '___sec59'), + ('Writing our own PCA code', 2, None, '___sec60'), ('Compute the sample mean and center the data', 3, None, - '___sec60'), - ('Compute the sample covariance', 3, None, '___sec61'), + '___sec61'), + ('Compute the sample covariance', 3, None, '___sec62'), ('Diagonalize the sample covariance matrix to obtain the ' 'principal components', 3, None, - '___sec62'), - ('Classical PCA Theorem', 2, None, '___sec63'), - ('Proof of the PCA Theorem', 2, None, '___sec64'), - ('PCA Proof continued', 2, None, '___sec65'), - ('The final step', 2, None, '___sec66'), + '___sec63'), + ('Classical PCA Theorem', 2, None, '___sec64'), + ('Proof of the PCA Theorem', 2, None, '___sec65'), + ('PCA Proof continued', 2, None, '___sec66'), + ('The final step', 2, None, '___sec67'), ('Geometric Interpretation and link with Singular Value ' 'Decomposition', 2, None, - '___sec67'), - ('Principal Component Analysis', 2, None, '___sec68'), - ('PCA and scikit-learn', 2, None, '___sec69'), - ('Back to the Cancer Data', 2, None, '___sec70'), - ('More on the PCA', 2, None, '___sec71'), - ('Incremental PCA', 2, None, '___sec72'), - ('Randomized PCA', 2, None, '___sec73'), - ('Kernel PCA', 2, None, '___sec74'), - ('LLE', 2, None, '___sec75'), - ('Other techniques', 2, None, '___sec76')]} + '___sec68'), + ('Principal Component Analysis', 2, None, '___sec69'), + ('PCA and scikit-learn', 2, None, '___sec70'), + ('Back to the Cancer Data', 2, None, '___sec71'), + ('More on the PCA', 2, None, '___sec72'), + ('Incremental PCA', 2, None, '___sec73'), + ('Randomized PCA', 2, None, '___sec74'), + ('Kernel PCA', 2, None, '___sec75'), + ('LLE', 2, None, '___sec76'), + ('Other techniques', 2, None, '___sec77')]} end of tocinfo --> @@ -230,7 +230,7 @@ MathJax.Hub.Config({
  • Final technicalities II
  • Final technicalities III
  • Final technicalities IV
    • -
  • Backpropagation
    • +
  • Back propagation
  • Gradient descent
  • The code for solving the ODE
  • The network with one input layer, specified number of hidden layers, and one output layer
    • @@ -239,55 +239,56 @@ MathJax.Hub.Config({
  • The trial solution
  • The program using Autograd
  • Using forward Euler to solve the ODE
    • -
  • Example: Solving the one dimensional Poisson equation using Autograd and TensorFlow
    • +
  • Example: Solving the one dimensional Poisson equation
  • The specific equation to solve for
  • Solving the equation using Autograd
  • Comparing with a numerical scheme
    • -
  • Partial Differential Equations
    • -
  • Type of problem
    • -
  • Network requirements
    • -
  • More details
    • -
  • Example: The diffusion equation
    • -
  • Defining the problem
    • -
  • Setting up the network using Autograd
    • -
  • Setting up the network using Autograd; The trial solution
    • -
  • Why the jacobian?
    • -
  • Setting up the network using Autograd; The full program
    • -
  • Example: Solving the wave equation with Neural Networks
    • -
  • The problem to solve for
    • -
  • The trial solution
    • -
  • The analytical solution
    • -
  • Solving the wave equation - the full program using Autograd
    • -
  • Resources on differential equations and deep learning
    • -
  • Friday, Principal Component Analysis
    • -
  • Basic ideas of the Principal Component Analysis (PCA)
    • -
  • Introducing the Covariance and Correlation functions
    • -
  • Correlation Function and Design/Feature Matrix
    • -
  • Covariance Matrix Examples
    • -
  • Correlation Matrix
    • -
  • Correlation Matrix with Pandas
    • -
  • Correlation Matrix with Pandas and the Franke function
    • -
  • Rewriting the Covariance and/or Correlation Matrix
    • -
  • Towards the PCA theorem
    • -
  • The Algorithm before the Theorem
    • -
  • Writing our own PCA code
    • -
  •    Compute the sample mean and center the data
    • -
  •    Compute the sample covariance
    • -
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • -
  • Classical PCA Theorem
    • -
  • Proof of the PCA Theorem
    • -
  • PCA Proof continued
    • -
  • The final step
    • -
  • Geometric Interpretation and link with Singular Value Decomposition
    • -
  • Principal Component Analysis
    • -
  • PCA and scikit-learn
    • -
  • Back to the Cancer Data
    • -
  • More on the PCA
    • -
  • Incremental PCA
    • -
  • Randomized PCA
    • -
  • Kernel PCA
    • -
  • LLE
    • -
  • Other techniques
    • +
  • Setting up the code
    • +
  • Partial Differential Equations
    • +
  • Type of problem
    • +
  • Network requirements
    • +
  • More details
    • +
  • Example: The diffusion equation
    • +
  • Defining the problem
    • +
  • Setting up the network using Autograd
    • +
  • Setting up the network using Autograd; The trial solution
    • +
  • Why the jacobian?
    • +
  • Setting up the network using Autograd; The full program
    • +
  • Example: Solving the wave equation with Neural Networks
    • +
  • The problem to solve for
    • +
  • The trial solution
    • +
  • The analytical solution
    • +
  • Solving the wave equation - the full program using Autograd
    • +
  • Resources on differential equations and deep learning
    • +
  • Friday, Principal Component Analysis
    • +
  • Basic ideas of the Principal Component Analysis (PCA)
    • +
  • Introducing the Covariance and Correlation functions
    • +
  • Correlation Function and Design/Feature Matrix
    • +
  • Covariance Matrix Examples
    • +
  • Correlation Matrix
    • +
  • Correlation Matrix with Pandas
    • +
  • Correlation Matrix with Pandas and the Franke function
    • +
  • Rewriting the Covariance and/or Correlation Matrix
    • +
  • Towards the PCA theorem
    • +
  • The Algorithm before the Theorem
    • +
  • Writing our own PCA code
    • +
  •    Compute the sample mean and center the data
    • +
  •    Compute the sample covariance
    • +
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • +
  • Classical PCA Theorem
    • +
  • Proof of the PCA Theorem
    • +
  • PCA Proof continued
    • +
  • The final step
    • +
  • Geometric Interpretation and link with Singular Value Decomposition
    • +
  • Principal Component Analysis
    • +
  • PCA and scikit-learn
    • +
  • Back to the Cancer Data
    • +
  • More on the PCA
    • +
  • Incremental PCA
    • +
  • Randomized PCA
    • +
  • Kernel PCA
    • +
  • LLE
    • +
  • Other techniques
    • @@ -354,7 +355,7 @@ The neural net should then find the parameters \( P \) that minimizes the cost f
  • 15
  • 16
  • ...
    • -
  • 76
    • +
  • 77
  • »
    • diff --git a/doc/pub/week43/html/._week43-bs007.html b/doc/pub/week43/html/._week43-bs007.html index 386eeff90..f867887a5 100644 --- a/doc/pub/week43/html/._week43-bs007.html +++ b/doc/pub/week43/html/._week43-bs007.html @@ -67,7 +67,7 @@ Automatically generated HTML file from DocOnce source ('Final technicalities II', 2, None, '___sec16'), ('Final technicalities III', 2, None, '___sec17'), ('Final technicalities IV', 2, None, '___sec18'), - ('Backpropagation', 2, None, '___sec19'), + ('Back propagation', 2, None, '___sec19'), ('Gradient descent', 2, None, '___sec20'), ('The code for solving the ODE', 2, None, '___sec21'), ('The network with one input layer, specified number of hidden ' @@ -80,100 +80,100 @@ Automatically generated HTML file from DocOnce source ('The trial solution', 2, None, '___sec25'), ('The program using Autograd', 2, None, '___sec26'), ('Using forward Euler to solve the ODE', 2, None, '___sec27'), - ('Example: Solving the one dimensional Poisson equation using ' - 'Autograd and TensorFlow', + ('Example: Solving the one dimensional Poisson equation', 2, None, '___sec28'), ('The specific equation to solve for', 2, None, '___sec29'), ('Solving the equation using Autograd', 2, None, '___sec30'), ('Comparing with a numerical scheme', 2, None, '___sec31'), - ('Partial Differential Equations', 2, None, '___sec32'), - ('Type of problem', 2, None, '___sec33'), - ('Network requirements', 2, None, '___sec34'), - ('More details', 2, None, '___sec35'), - ('Example: The diffusion equation', 2, None, '___sec36'), - ('Defining the problem', 2, None, '___sec37'), - ('Setting up the network using Autograd', 2, None, '___sec38'), + ('Setting up the code', 2, None, '___sec32'), + ('Partial Differential Equations', 2, None, '___sec33'), + ('Type of problem', 2, None, '___sec34'), + ('Network requirements', 2, None, '___sec35'), + ('More details', 2, None, '___sec36'), + ('Example: The diffusion equation', 2, None, '___sec37'), + ('Defining the problem', 2, None, '___sec38'), + ('Setting up the network using Autograd', 2, None, '___sec39'), ('Setting up the network using Autograd; The trial solution', 2, None, - '___sec39'), - ('Why the jacobian?', 2, None, '___sec40'), + '___sec40'), + ('Why the jacobian?', 2, None, '___sec41'), ('Setting up the network using Autograd; The full program', 2, None, - '___sec41'), + '___sec42'), ('Example: Solving the wave equation with Neural Networks', 2, None, - '___sec42'), - ('The problem to solve for', 2, None, '___sec43'), - ('The trial solution', 2, None, '___sec44'), - ('The analytical solution', 2, None, '___sec45'), + '___sec43'), + ('The problem to solve for', 2, None, '___sec44'), + ('The trial solution', 2, None, '___sec45'), + ('The analytical solution', 2, None, '___sec46'), ('Solving the wave equation - the full program using Autograd', 2, None, - '___sec46'), + '___sec47'), ('Resources on differential equations and deep learning', 2, None, - '___sec47'), - ('Friday, Principal Component Analysis', 2, None, '___sec48'), + '___sec48'), + ('Friday, Principal Component Analysis', 2, None, '___sec49'), ('Basic ideas of the Principal Component Analysis (PCA)', 2, None, - '___sec49'), + '___sec50'), ('Introducing the Covariance and Correlation functions', 2, None, - '___sec50'), + '___sec51'), ('Correlation Function and Design/Feature Matrix', 2, None, - '___sec51'), - ('Covariance Matrix Examples', 2, None, '___sec52'), - ('Correlation Matrix', 2, None, '___sec53'), - ('Correlation Matrix with Pandas', 2, None, '___sec54'), + '___sec52'), + ('Covariance Matrix Examples', 2, None, '___sec53'), + ('Correlation Matrix', 2, None, '___sec54'), + ('Correlation Matrix with Pandas', 2, None, '___sec55'), ('Correlation Matrix with Pandas and the Franke function', 2, None, - '___sec55'), + '___sec56'), ('Rewriting the Covariance and/or Correlation Matrix', 2, None, - '___sec56'), - ('Towards the PCA theorem', 2, None, '___sec57'), - ('The Algorithm before the Theorem', 2, None, '___sec58'), - ('Writing our own PCA code', 2, None, '___sec59'), + '___sec57'), + ('Towards the PCA theorem', 2, None, '___sec58'), + ('The Algorithm before the Theorem', 2, None, '___sec59'), + ('Writing our own PCA code', 2, None, '___sec60'), ('Compute the sample mean and center the data', 3, None, - '___sec60'), - ('Compute the sample covariance', 3, None, '___sec61'), + '___sec61'), + ('Compute the sample covariance', 3, None, '___sec62'), ('Diagonalize the sample covariance matrix to obtain the ' 'principal components', 3, None, - '___sec62'), - ('Classical PCA Theorem', 2, None, '___sec63'), - ('Proof of the PCA Theorem', 2, None, '___sec64'), - ('PCA Proof continued', 2, None, '___sec65'), - ('The final step', 2, None, '___sec66'), + '___sec63'), + ('Classical PCA Theorem', 2, None, '___sec64'), + ('Proof of the PCA Theorem', 2, None, '___sec65'), + ('PCA Proof continued', 2, None, '___sec66'), + ('The final step', 2, None, '___sec67'), ('Geometric Interpretation and link with Singular Value ' 'Decomposition', 2, None, - '___sec67'), - ('Principal Component Analysis', 2, None, '___sec68'), - ('PCA and scikit-learn', 2, None, '___sec69'), - ('Back to the Cancer Data', 2, None, '___sec70'), - ('More on the PCA', 2, None, '___sec71'), - ('Incremental PCA', 2, None, '___sec72'), - ('Randomized PCA', 2, None, '___sec73'), - ('Kernel PCA', 2, None, '___sec74'), - ('LLE', 2, None, '___sec75'), - ('Other techniques', 2, None, '___sec76')]} + '___sec68'), + ('Principal Component Analysis', 2, None, '___sec69'), + ('PCA and scikit-learn', 2, None, '___sec70'), + ('Back to the Cancer Data', 2, None, '___sec71'), + ('More on the PCA', 2, None, '___sec72'), + ('Incremental PCA', 2, None, '___sec73'), + ('Randomized PCA', 2, None, '___sec74'), + ('Kernel PCA', 2, None, '___sec75'), + ('LLE', 2, None, '___sec76'), + ('Other techniques', 2, None, '___sec77')]} end of tocinfo --> @@ -230,7 +230,7 @@ MathJax.Hub.Config({
  • Final technicalities II
  • Final technicalities III
  • Final technicalities IV
    • -
  • Backpropagation
    • +
  • Back propagation
  • Gradient descent
  • The code for solving the ODE
  • The network with one input layer, specified number of hidden layers, and one output layer
    • @@ -239,55 +239,56 @@ MathJax.Hub.Config({
  • The trial solution
  • The program using Autograd
  • Using forward Euler to solve the ODE
    • -
  • Example: Solving the one dimensional Poisson equation using Autograd and TensorFlow
    • +
  • Example: Solving the one dimensional Poisson equation
  • The specific equation to solve for
  • Solving the equation using Autograd
  • Comparing with a numerical scheme
    • -
  • Partial Differential Equations
    • -
  • Type of problem
    • -
  • Network requirements
    • -
  • More details
    • -
  • Example: The diffusion equation
    • -
  • Defining the problem
    • -
  • Setting up the network using Autograd
    • -
  • Setting up the network using Autograd; The trial solution
    • -
  • Why the jacobian?
    • -
  • Setting up the network using Autograd; The full program
    • -
  • Example: Solving the wave equation with Neural Networks
    • -
  • The problem to solve for
    • -
  • The trial solution
    • -
  • The analytical solution
    • -
  • Solving the wave equation - the full program using Autograd
    • -
  • Resources on differential equations and deep learning
    • -
  • Friday, Principal Component Analysis
    • -
  • Basic ideas of the Principal Component Analysis (PCA)
    • -
  • Introducing the Covariance and Correlation functions
    • -
  • Correlation Function and Design/Feature Matrix
    • -
  • Covariance Matrix Examples
    • -
  • Correlation Matrix
    • -
  • Correlation Matrix with Pandas
    • -
  • Correlation Matrix with Pandas and the Franke function
    • -
  • Rewriting the Covariance and/or Correlation Matrix
    • -
  • Towards the PCA theorem
    • -
  • The Algorithm before the Theorem
    • -
  • Writing our own PCA code
    • -
  •    Compute the sample mean and center the data
    • -
  •    Compute the sample covariance
    • -
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • -
  • Classical PCA Theorem
    • -
  • Proof of the PCA Theorem
    • -
  • PCA Proof continued
    • -
  • The final step
    • -
  • Geometric Interpretation and link with Singular Value Decomposition
    • -
  • Principal Component Analysis
    • -
  • PCA and scikit-learn
    • -
  • Back to the Cancer Data
    • -
  • More on the PCA
    • -
  • Incremental PCA
    • -
  • Randomized PCA
    • -
  • Kernel PCA
    • -
  • LLE
    • -
  • Other techniques
    • +
  • Setting up the code
    • +
  • Partial Differential Equations
    • +
  • Type of problem
    • +
  • Network requirements
    • +
  • More details
    • +
  • Example: The diffusion equation
    • +
  • Defining the problem
    • +
  • Setting up the network using Autograd
    • +
  • Setting up the network using Autograd; The trial solution
    • +
  • Why the jacobian?
    • +
  • Setting up the network using Autograd; The full program
    • +
  • Example: Solving the wave equation with Neural Networks
    • +
  • The problem to solve for
    • +
  • The trial solution
    • +
  • The analytical solution
    • +
  • Solving the wave equation - the full program using Autograd
    • +
  • Resources on differential equations and deep learning
    • +
  • Friday, Principal Component Analysis
    • +
  • Basic ideas of the Principal Component Analysis (PCA)
    • +
  • Introducing the Covariance and Correlation functions
    • +
  • Correlation Function and Design/Feature Matrix
    • +
  • Covariance Matrix Examples
    • +
  • Correlation Matrix
    • +
  • Correlation Matrix with Pandas
    • +
  • Correlation Matrix with Pandas and the Franke function
    • +
  • Rewriting the Covariance and/or Correlation Matrix
    • +
  • Towards the PCA theorem
    • +
  • The Algorithm before the Theorem
    • +
  • Writing our own PCA code
    • +
  •    Compute the sample mean and center the data
    • +
  •    Compute the sample covariance
    • +
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • +
  • Classical PCA Theorem
    • +
  • Proof of the PCA Theorem
    • +
  • PCA Proof continued
    • +
  • The final step
    • +
  • Geometric Interpretation and link with Singular Value Decomposition
    • +
  • Principal Component Analysis
    • +
  • PCA and scikit-learn
    • +
  • Back to the Cancer Data
    • +
  • More on the PCA
    • +
  • Incremental PCA
    • +
  • Randomized PCA
    • +
  • Kernel PCA
    • +
  • LLE
    • +
  • Other techniques
    • @@ -336,7 +337,7 @@ Automatic differentiation is a method of finding the derivatives numerically wit
  • 16
  • 17
  • ...
    • -
  • 76
    • +
  • 77
  • »
    • diff --git a/doc/pub/week43/html/._week43-bs008.html b/doc/pub/week43/html/._week43-bs008.html index f1a122605..53beb5a0b 100644 --- a/doc/pub/week43/html/._week43-bs008.html +++ b/doc/pub/week43/html/._week43-bs008.html @@ -67,7 +67,7 @@ Automatically generated HTML file from DocOnce source ('Final technicalities II', 2, None, '___sec16'), ('Final technicalities III', 2, None, '___sec17'), ('Final technicalities IV', 2, None, '___sec18'), - ('Backpropagation', 2, None, '___sec19'), + ('Back propagation', 2, None, '___sec19'), ('Gradient descent', 2, None, '___sec20'), ('The code for solving the ODE', 2, None, '___sec21'), ('The network with one input layer, specified number of hidden ' @@ -80,100 +80,100 @@ Automatically generated HTML file from DocOnce source ('The trial solution', 2, None, '___sec25'), ('The program using Autograd', 2, None, '___sec26'), ('Using forward Euler to solve the ODE', 2, None, '___sec27'), - ('Example: Solving the one dimensional Poisson equation using ' - 'Autograd and TensorFlow', + ('Example: Solving the one dimensional Poisson equation', 2, None, '___sec28'), ('The specific equation to solve for', 2, None, '___sec29'), ('Solving the equation using Autograd', 2, None, '___sec30'), ('Comparing with a numerical scheme', 2, None, '___sec31'), - ('Partial Differential Equations', 2, None, '___sec32'), - ('Type of problem', 2, None, '___sec33'), - ('Network requirements', 2, None, '___sec34'), - ('More details', 2, None, '___sec35'), - ('Example: The diffusion equation', 2, None, '___sec36'), - ('Defining the problem', 2, None, '___sec37'), - ('Setting up the network using Autograd', 2, None, '___sec38'), + ('Setting up the code', 2, None, '___sec32'), + ('Partial Differential Equations', 2, None, '___sec33'), + ('Type of problem', 2, None, '___sec34'), + ('Network requirements', 2, None, '___sec35'), + ('More details', 2, None, '___sec36'), + ('Example: The diffusion equation', 2, None, '___sec37'), + ('Defining the problem', 2, None, '___sec38'), + ('Setting up the network using Autograd', 2, None, '___sec39'), ('Setting up the network using Autograd; The trial solution', 2, None, - '___sec39'), - ('Why the jacobian?', 2, None, '___sec40'), + '___sec40'), + ('Why the jacobian?', 2, None, '___sec41'), ('Setting up the network using Autograd; The full program', 2, None, - '___sec41'), + '___sec42'), ('Example: Solving the wave equation with Neural Networks', 2, None, - '___sec42'), - ('The problem to solve for', 2, None, '___sec43'), - ('The trial solution', 2, None, '___sec44'), - ('The analytical solution', 2, None, '___sec45'), + '___sec43'), + ('The problem to solve for', 2, None, '___sec44'), + ('The trial solution', 2, None, '___sec45'), + ('The analytical solution', 2, None, '___sec46'), ('Solving the wave equation - the full program using Autograd', 2, None, - '___sec46'), + '___sec47'), ('Resources on differential equations and deep learning', 2, None, - '___sec47'), - ('Friday, Principal Component Analysis', 2, None, '___sec48'), + '___sec48'), + ('Friday, Principal Component Analysis', 2, None, '___sec49'), ('Basic ideas of the Principal Component Analysis (PCA)', 2, None, - '___sec49'), + '___sec50'), ('Introducing the Covariance and Correlation functions', 2, None, - '___sec50'), + '___sec51'), ('Correlation Function and Design/Feature Matrix', 2, None, - '___sec51'), - ('Covariance Matrix Examples', 2, None, '___sec52'), - ('Correlation Matrix', 2, None, '___sec53'), - ('Correlation Matrix with Pandas', 2, None, '___sec54'), + '___sec52'), + ('Covariance Matrix Examples', 2, None, '___sec53'), + ('Correlation Matrix', 2, None, '___sec54'), + ('Correlation Matrix with Pandas', 2, None, '___sec55'), ('Correlation Matrix with Pandas and the Franke function', 2, None, - '___sec55'), + '___sec56'), ('Rewriting the Covariance and/or Correlation Matrix', 2, None, - '___sec56'), - ('Towards the PCA theorem', 2, None, '___sec57'), - ('The Algorithm before the Theorem', 2, None, '___sec58'), - ('Writing our own PCA code', 2, None, '___sec59'), + '___sec57'), + ('Towards the PCA theorem', 2, None, '___sec58'), + ('The Algorithm before the Theorem', 2, None, '___sec59'), + ('Writing our own PCA code', 2, None, '___sec60'), ('Compute the sample mean and center the data', 3, None, - '___sec60'), - ('Compute the sample covariance', 3, None, '___sec61'), + '___sec61'), + ('Compute the sample covariance', 3, None, '___sec62'), ('Diagonalize the sample covariance matrix to obtain the ' 'principal components', 3, None, - '___sec62'), - ('Classical PCA Theorem', 2, None, '___sec63'), - ('Proof of the PCA Theorem', 2, None, '___sec64'), - ('PCA Proof continued', 2, None, '___sec65'), - ('The final step', 2, None, '___sec66'), + '___sec63'), + ('Classical PCA Theorem', 2, None, '___sec64'), + ('Proof of the PCA Theorem', 2, None, '___sec65'), + ('PCA Proof continued', 2, None, '___sec66'), + ('The final step', 2, None, '___sec67'), ('Geometric Interpretation and link with Singular Value ' 'Decomposition', 2, None, - '___sec67'), - ('Principal Component Analysis', 2, None, '___sec68'), - ('PCA and scikit-learn', 2, None, '___sec69'), - ('Back to the Cancer Data', 2, None, '___sec70'), - ('More on the PCA', 2, None, '___sec71'), - ('Incremental PCA', 2, None, '___sec72'), - ('Randomized PCA', 2, None, '___sec73'), - ('Kernel PCA', 2, None, '___sec74'), - ('LLE', 2, None, '___sec75'), - ('Other techniques', 2, None, '___sec76')]} + '___sec68'), + ('Principal Component Analysis', 2, None, '___sec69'), + ('PCA and scikit-learn', 2, None, '___sec70'), + ('Back to the Cancer Data', 2, None, '___sec71'), + ('More on the PCA', 2, None, '___sec72'), + ('Incremental PCA', 2, None, '___sec73'), + ('Randomized PCA', 2, None, '___sec74'), + ('Kernel PCA', 2, None, '___sec75'), + ('LLE', 2, None, '___sec76'), + ('Other techniques', 2, None, '___sec77')]} end of tocinfo --> @@ -230,7 +230,7 @@ MathJax.Hub.Config({
  • Final technicalities II
  • Final technicalities III
  • Final technicalities IV
    • -
  • Backpropagation
    • +
  • Back propagation
  • Gradient descent
  • The code for solving the ODE
  • The network with one input layer, specified number of hidden layers, and one output layer
    • @@ -239,55 +239,56 @@ MathJax.Hub.Config({
  • The trial solution
  • The program using Autograd
  • Using forward Euler to solve the ODE
    • -
  • Example: Solving the one dimensional Poisson equation using Autograd and TensorFlow
    • +
  • Example: Solving the one dimensional Poisson equation
  • The specific equation to solve for
  • Solving the equation using Autograd
  • Comparing with a numerical scheme
    • -
  • Partial Differential Equations
    • -
  • Type of problem
    • -
  • Network requirements
    • -
  • More details
    • -
  • Example: The diffusion equation
    • -
  • Defining the problem
    • -
  • Setting up the network using Autograd
    • -
  • Setting up the network using Autograd; The trial solution
    • -
  • Why the jacobian?
    • -
  • Setting up the network using Autograd; The full program
    • -
  • Example: Solving the wave equation with Neural Networks
    • -
  • The problem to solve for
    • -
  • The trial solution
    • -
  • The analytical solution
    • -
  • Solving the wave equation - the full program using Autograd
    • -
  • Resources on differential equations and deep learning
    • -
  • Friday, Principal Component Analysis
    • -
  • Basic ideas of the Principal Component Analysis (PCA)
    • -
  • Introducing the Covariance and Correlation functions
    • -
  • Correlation Function and Design/Feature Matrix
    • -
  • Covariance Matrix Examples
    • -
  • Correlation Matrix
    • -
  • Correlation Matrix with Pandas
    • -
  • Correlation Matrix with Pandas and the Franke function
    • -
  • Rewriting the Covariance and/or Correlation Matrix
    • -
  • Towards the PCA theorem
    • -
  • The Algorithm before the Theorem
    • -
  • Writing our own PCA code
    • -
  •    Compute the sample mean and center the data
    • -
  •    Compute the sample covariance
    • -
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • -
  • Classical PCA Theorem
    • -
  • Proof of the PCA Theorem
    • -
  • PCA Proof continued
    • -
  • The final step
    • -
  • Geometric Interpretation and link with Singular Value Decomposition
    • -
  • Principal Component Analysis
    • -
  • PCA and scikit-learn
    • -
  • Back to the Cancer Data
    • -
  • More on the PCA
    • -
  • Incremental PCA
    • -
  • Randomized PCA
    • -
  • Kernel PCA
    • -
  • LLE
    • -
  • Other techniques
    • +
  • Setting up the code
    • +
  • Partial Differential Equations
    • +
  • Type of problem
    • +
  • Network requirements
    • +
  • More details
    • +
  • Example: The diffusion equation
    • +
  • Defining the problem
    • +
  • Setting up the network using Autograd
    • +
  • Setting up the network using Autograd; The trial solution
    • +
  • Why the jacobian?
    • +
  • Setting up the network using Autograd; The full program
    • +
  • Example: Solving the wave equation with Neural Networks
    • +
  • The problem to solve for
    • +
  • The trial solution
    • +
  • The analytical solution
    • +
  • Solving the wave equation - the full program using Autograd
    • +
  • Resources on differential equations and deep learning
    • +
  • Friday, Principal Component Analysis
    • +
  • Basic ideas of the Principal Component Analysis (PCA)
    • +
  • Introducing the Covariance and Correlation functions
    • +
  • Correlation Function and Design/Feature Matrix
    • +
  • Covariance Matrix Examples
    • +
  • Correlation Matrix
    • +
  • Correlation Matrix with Pandas
    • +
  • Correlation Matrix with Pandas and the Franke function
    • +
  • Rewriting the Covariance and/or Correlation Matrix
    • +
  • Towards the PCA theorem
    • +
  • The Algorithm before the Theorem
    • +
  • Writing our own PCA code
    • +
  •    Compute the sample mean and center the data
    • +
  •    Compute the sample covariance
    • +
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • +
  • Classical PCA Theorem
    • +
  • Proof of the PCA Theorem
    • +
  • PCA Proof continued
    • +
  • The final step
    • +
  • Geometric Interpretation and link with Singular Value Decomposition
    • +
  • Principal Component Analysis
    • +
  • PCA and scikit-learn
    • +
  • Back to the Cancer Data
    • +
  • More on the PCA
    • +
  • Incremental PCA
    • +
  • Randomized PCA
    • +
  • Kernel PCA
    • +
  • LLE
    • +
  • Other techniques
    • @@ -354,7 +355,7 @@ Having an analytical solution at hand, it is possible to use it to compare how w
  • 17
  • 18
  • ...
    • -
  • 76
    • +
  • 77
  • »
    • diff --git a/doc/pub/week43/html/._week43-bs009.html b/doc/pub/week43/html/._week43-bs009.html index 3879b20cf..d731e390f 100644 --- a/doc/pub/week43/html/._week43-bs009.html +++ b/doc/pub/week43/html/._week43-bs009.html @@ -67,7 +67,7 @@ Automatically generated HTML file from DocOnce source ('Final technicalities II', 2, None, '___sec16'), ('Final technicalities III', 2, None, '___sec17'), ('Final technicalities IV', 2, None, '___sec18'), - ('Backpropagation', 2, None, '___sec19'), + ('Back propagation', 2, None, '___sec19'), ('Gradient descent', 2, None, '___sec20'), ('The code for solving the ODE', 2, None, '___sec21'), ('The network with one input layer, specified number of hidden ' @@ -80,100 +80,100 @@ Automatically generated HTML file from DocOnce source ('The trial solution', 2, None, '___sec25'), ('The program using Autograd', 2, None, '___sec26'), ('Using forward Euler to solve the ODE', 2, None, '___sec27'), - ('Example: Solving the one dimensional Poisson equation using ' - 'Autograd and TensorFlow', + ('Example: Solving the one dimensional Poisson equation', 2, None, '___sec28'), ('The specific equation to solve for', 2, None, '___sec29'), ('Solving the equation using Autograd', 2, None, '___sec30'), ('Comparing with a numerical scheme', 2, None, '___sec31'), - ('Partial Differential Equations', 2, None, '___sec32'), - ('Type of problem', 2, None, '___sec33'), - ('Network requirements', 2, None, '___sec34'), - ('More details', 2, None, '___sec35'), - ('Example: The diffusion equation', 2, None, '___sec36'), - ('Defining the problem', 2, None, '___sec37'), - ('Setting up the network using Autograd', 2, None, '___sec38'), + ('Setting up the code', 2, None, '___sec32'), + ('Partial Differential Equations', 2, None, '___sec33'), + ('Type of problem', 2, None, '___sec34'), + ('Network requirements', 2, None, '___sec35'), + ('More details', 2, None, '___sec36'), + ('Example: The diffusion equation', 2, None, '___sec37'), + ('Defining the problem', 2, None, '___sec38'), + ('Setting up the network using Autograd', 2, None, '___sec39'), ('Setting up the network using Autograd; The trial solution', 2, None, - '___sec39'), - ('Why the jacobian?', 2, None, '___sec40'), + '___sec40'), + ('Why the jacobian?', 2, None, '___sec41'), ('Setting up the network using Autograd; The full program', 2, None, - '___sec41'), + '___sec42'), ('Example: Solving the wave equation with Neural Networks', 2, None, - '___sec42'), - ('The problem to solve for', 2, None, '___sec43'), - ('The trial solution', 2, None, '___sec44'), - ('The analytical solution', 2, None, '___sec45'), + '___sec43'), + ('The problem to solve for', 2, None, '___sec44'), + ('The trial solution', 2, None, '___sec45'), + ('The analytical solution', 2, None, '___sec46'), ('Solving the wave equation - the full program using Autograd', 2, None, - '___sec46'), + '___sec47'), ('Resources on differential equations and deep learning', 2, None, - '___sec47'), - ('Friday, Principal Component Analysis', 2, None, '___sec48'), + '___sec48'), + ('Friday, Principal Component Analysis', 2, None, '___sec49'), ('Basic ideas of the Principal Component Analysis (PCA)', 2, None, - '___sec49'), + '___sec50'), ('Introducing the Covariance and Correlation functions', 2, None, - '___sec50'), + '___sec51'), ('Correlation Function and Design/Feature Matrix', 2, None, - '___sec51'), - ('Covariance Matrix Examples', 2, None, '___sec52'), - ('Correlation Matrix', 2, None, '___sec53'), - ('Correlation Matrix with Pandas', 2, None, '___sec54'), + '___sec52'), + ('Covariance Matrix Examples', 2, None, '___sec53'), + ('Correlation Matrix', 2, None, '___sec54'), + ('Correlation Matrix with Pandas', 2, None, '___sec55'), ('Correlation Matrix with Pandas and the Franke function', 2, None, - '___sec55'), + '___sec56'), ('Rewriting the Covariance and/or Correlation Matrix', 2, None, - '___sec56'), - ('Towards the PCA theorem', 2, None, '___sec57'), - ('The Algorithm before the Theorem', 2, None, '___sec58'), - ('Writing our own PCA code', 2, None, '___sec59'), + '___sec57'), + ('Towards the PCA theorem', 2, None, '___sec58'), + ('The Algorithm before the Theorem', 2, None, '___sec59'), + ('Writing our own PCA code', 2, None, '___sec60'), ('Compute the sample mean and center the data', 3, None, - '___sec60'), - ('Compute the sample covariance', 3, None, '___sec61'), + '___sec61'), + ('Compute the sample covariance', 3, None, '___sec62'), ('Diagonalize the sample covariance matrix to obtain the ' 'principal components', 3, None, - '___sec62'), - ('Classical PCA Theorem', 2, None, '___sec63'), - ('Proof of the PCA Theorem', 2, None, '___sec64'), - ('PCA Proof continued', 2, None, '___sec65'), - ('The final step', 2, None, '___sec66'), + '___sec63'), + ('Classical PCA Theorem', 2, None, '___sec64'), + ('Proof of the PCA Theorem', 2, None, '___sec65'), + ('PCA Proof continued', 2, None, '___sec66'), + ('The final step', 2, None, '___sec67'), ('Geometric Interpretation and link with Singular Value ' 'Decomposition', 2, None, - '___sec67'), - ('Principal Component Analysis', 2, None, '___sec68'), - ('PCA and scikit-learn', 2, None, '___sec69'), - ('Back to the Cancer Data', 2, None, '___sec70'), - ('More on the PCA', 2, None, '___sec71'), - ('Incremental PCA', 2, None, '___sec72'), - ('Randomized PCA', 2, None, '___sec73'), - ('Kernel PCA', 2, None, '___sec74'), - ('LLE', 2, None, '___sec75'), - ('Other techniques', 2, None, '___sec76')]} + '___sec68'), + ('Principal Component Analysis', 2, None, '___sec69'), + ('PCA and scikit-learn', 2, None, '___sec70'), + ('Back to the Cancer Data', 2, None, '___sec71'), + ('More on the PCA', 2, None, '___sec72'), + ('Incremental PCA', 2, None, '___sec73'), + ('Randomized PCA', 2, None, '___sec74'), + ('Kernel PCA', 2, None, '___sec75'), + ('LLE', 2, None, '___sec76'), + ('Other techniques', 2, None, '___sec77')]} end of tocinfo --> @@ -230,7 +230,7 @@ MathJax.Hub.Config({
  • Final technicalities II
  • Final technicalities III
  • Final technicalities IV
    • -
  • Backpropagation
    • +
  • Back propagation
  • Gradient descent
  • The code for solving the ODE
  • The network with one input layer, specified number of hidden layers, and one output layer
    • @@ -239,55 +239,56 @@ MathJax.Hub.Config({
  • The trial solution
  • The program using Autograd
  • Using forward Euler to solve the ODE
    • -
  • Example: Solving the one dimensional Poisson equation using Autograd and TensorFlow
    • +
  • Example: Solving the one dimensional Poisson equation
  • The specific equation to solve for
  • Solving the equation using Autograd
  • Comparing with a numerical scheme
    • -
  • Partial Differential Equations
    • -
  • Type of problem
    • -
  • Network requirements
    • -
  • More details
    • -
  • Example: The diffusion equation
    • -
  • Defining the problem
    • -
  • Setting up the network using Autograd
    • -
  • Setting up the network using Autograd; The trial solution
    • -
  • Why the jacobian?
    • -
  • Setting up the network using Autograd; The full program
    • -
  • Example: Solving the wave equation with Neural Networks
    • -
  • The problem to solve for
    • -
  • The trial solution
    • -
  • The analytical solution
    • -
  • Solving the wave equation - the full program using Autograd
    • -
  • Resources on differential equations and deep learning
    • -
  • Friday, Principal Component Analysis
    • -
  • Basic ideas of the Principal Component Analysis (PCA)
    • -
  • Introducing the Covariance and Correlation functions
    • -
  • Correlation Function and Design/Feature Matrix
    • -
  • Covariance Matrix Examples
    • -
  • Correlation Matrix
    • -
  • Correlation Matrix with Pandas
    • -
  • Correlation Matrix with Pandas and the Franke function
    • -
  • Rewriting the Covariance and/or Correlation Matrix
    • -
  • Towards the PCA theorem
    • -
  • The Algorithm before the Theorem
    • -
  • Writing our own PCA code
    • -
  •    Compute the sample mean and center the data
    • -
  •    Compute the sample covariance
    • -
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • -
  • Classical PCA Theorem
    • -
  • Proof of the PCA Theorem
    • -
  • PCA Proof continued
    • -
  • The final step
    • -
  • Geometric Interpretation and link with Singular Value Decomposition
    • -
  • Principal Component Analysis
    • -
  • PCA and scikit-learn
    • -
  • Back to the Cancer Data
    • -
  • More on the PCA
    • -
  • Incremental PCA
    • -
  • Randomized PCA
    • -
  • Kernel PCA
    • -
  • LLE
    • -
  • Other techniques
    • +
  • Setting up the code
    • +
  • Partial Differential Equations
    • +
  • Type of problem
    • +
  • Network requirements
    • +
  • More details
    • +
  • Example: The diffusion equation
    • +
  • Defining the problem
    • +
  • Setting up the network using Autograd
    • +
  • Setting up the network using Autograd; The trial solution
    • +
  • Why the jacobian?
    • +
  • Setting up the network using Autograd; The full program
    • +
  • Example: Solving the wave equation with Neural Networks
    • +
  • The problem to solve for
    • +
  • The trial solution
    • +
  • The analytical solution
    • +
  • Solving the wave equation - the full program using Autograd
    • +
  • Resources on differential equations and deep learning
    • +
  • Friday, Principal Component Analysis
    • +
  • Basic ideas of the Principal Component Analysis (PCA)
    • +
  • Introducing the Covariance and Correlation functions
    • +
  • Correlation Function and Design/Feature Matrix
    • +
  • Covariance Matrix Examples
    • +
  • Correlation Matrix
    • +
  • Correlation Matrix with Pandas
    • +
  • Correlation Matrix with Pandas and the Franke function
    • +
  • Rewriting the Covariance and/or Correlation Matrix
    • +
  • Towards the PCA theorem
    • +
  • The Algorithm before the Theorem
    • +
  • Writing our own PCA code
    • +
  •    Compute the sample mean and center the data
    • +
  •    Compute the sample covariance
    • +
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • +
  • Classical PCA Theorem
    • +
  • Proof of the PCA Theorem
    • +
  • PCA Proof continued
    • +
  • The final step
    • +
  • Geometric Interpretation and link with Singular Value Decomposition
    • +
  • Principal Component Analysis
    • +
  • PCA and scikit-learn
    • +
  • Back to the Cancer Data
    • +
  • More on the PCA
    • +
  • Incremental PCA
    • +
  • Randomized PCA
    • +
  • Kernel PCA
    • +
  • LLE
    • +
  • Other techniques
    • @@ -345,7 +346,7 @@ In this example, \( \gamma = 2 \) and \( g_0 = 10 \).
  • 18
  • 19
  • ...
    • -
  • 76
    • +
  • 77
  • »
    • diff --git a/doc/pub/week43/html/._week43-bs010.html b/doc/pub/week43/html/._week43-bs010.html index 096ab291c..2438c3260 100644 --- a/doc/pub/week43/html/._week43-bs010.html +++ b/doc/pub/week43/html/._week43-bs010.html @@ -67,7 +67,7 @@ Automatically generated HTML file from DocOnce source ('Final technicalities II', 2, None, '___sec16'), ('Final technicalities III', 2, None, '___sec17'), ('Final technicalities IV', 2, None, '___sec18'), - ('Backpropagation', 2, None, '___sec19'), + ('Back propagation', 2, None, '___sec19'), ('Gradient descent', 2, None, '___sec20'), ('The code for solving the ODE', 2, None, '___sec21'), ('The network with one input layer, specified number of hidden ' @@ -80,100 +80,100 @@ Automatically generated HTML file from DocOnce source ('The trial solution', 2, None, '___sec25'), ('The program using Autograd', 2, None, '___sec26'), ('Using forward Euler to solve the ODE', 2, None, '___sec27'), - ('Example: Solving the one dimensional Poisson equation using ' - 'Autograd and TensorFlow', + ('Example: Solving the one dimensional Poisson equation', 2, None, '___sec28'), ('The specific equation to solve for', 2, None, '___sec29'), ('Solving the equation using Autograd', 2, None, '___sec30'), ('Comparing with a numerical scheme', 2, None, '___sec31'), - ('Partial Differential Equations', 2, None, '___sec32'), - ('Type of problem', 2, None, '___sec33'), - ('Network requirements', 2, None, '___sec34'), - ('More details', 2, None, '___sec35'), - ('Example: The diffusion equation', 2, None, '___sec36'), - ('Defining the problem', 2, None, '___sec37'), - ('Setting up the network using Autograd', 2, None, '___sec38'), + ('Setting up the code', 2, None, '___sec32'), + ('Partial Differential Equations', 2, None, '___sec33'), + ('Type of problem', 2, None, '___sec34'), + ('Network requirements', 2, None, '___sec35'), + ('More details', 2, None, '___sec36'), + ('Example: The diffusion equation', 2, None, '___sec37'), + ('Defining the problem', 2, None, '___sec38'), + ('Setting up the network using Autograd', 2, None, '___sec39'), ('Setting up the network using Autograd; The trial solution', 2, None, - '___sec39'), - ('Why the jacobian?', 2, None, '___sec40'), + '___sec40'), + ('Why the jacobian?', 2, None, '___sec41'), ('Setting up the network using Autograd; The full program', 2, None, - '___sec41'), + '___sec42'), ('Example: Solving the wave equation with Neural Networks', 2, None, - '___sec42'), - ('The problem to solve for', 2, None, '___sec43'), - ('The trial solution', 2, None, '___sec44'), - ('The analytical solution', 2, None, '___sec45'), + '___sec43'), + ('The problem to solve for', 2, None, '___sec44'), + ('The trial solution', 2, None, '___sec45'), + ('The analytical solution', 2, None, '___sec46'), ('Solving the wave equation - the full program using Autograd', 2, None, - '___sec46'), + '___sec47'), ('Resources on differential equations and deep learning', 2, None, - '___sec47'), - ('Friday, Principal Component Analysis', 2, None, '___sec48'), + '___sec48'), + ('Friday, Principal Component Analysis', 2, None, '___sec49'), ('Basic ideas of the Principal Component Analysis (PCA)', 2, None, - '___sec49'), + '___sec50'), ('Introducing the Covariance and Correlation functions', 2, None, - '___sec50'), + '___sec51'), ('Correlation Function and Design/Feature Matrix', 2, None, - '___sec51'), - ('Covariance Matrix Examples', 2, None, '___sec52'), - ('Correlation Matrix', 2, None, '___sec53'), - ('Correlation Matrix with Pandas', 2, None, '___sec54'), + '___sec52'), + ('Covariance Matrix Examples', 2, None, '___sec53'), + ('Correlation Matrix', 2, None, '___sec54'), + ('Correlation Matrix with Pandas', 2, None, '___sec55'), ('Correlation Matrix with Pandas and the Franke function', 2, None, - '___sec55'), + '___sec56'), ('Rewriting the Covariance and/or Correlation Matrix', 2, None, - '___sec56'), - ('Towards the PCA theorem', 2, None, '___sec57'), - ('The Algorithm before the Theorem', 2, None, '___sec58'), - ('Writing our own PCA code', 2, None, '___sec59'), + '___sec57'), + ('Towards the PCA theorem', 2, None, '___sec58'), + ('The Algorithm before the Theorem', 2, None, '___sec59'), + ('Writing our own PCA code', 2, None, '___sec60'), ('Compute the sample mean and center the data', 3, None, - '___sec60'), - ('Compute the sample covariance', 3, None, '___sec61'), + '___sec61'), + ('Compute the sample covariance', 3, None, '___sec62'), ('Diagonalize the sample covariance matrix to obtain the ' 'principal components', 3, None, - '___sec62'), - ('Classical PCA Theorem', 2, None, '___sec63'), - ('Proof of the PCA Theorem', 2, None, '___sec64'), - ('PCA Proof continued', 2, None, '___sec65'), - ('The final step', 2, None, '___sec66'), + '___sec63'), + ('Classical PCA Theorem', 2, None, '___sec64'), + ('Proof of the PCA Theorem', 2, None, '___sec65'), + ('PCA Proof continued', 2, None, '___sec66'), + ('The final step', 2, None, '___sec67'), ('Geometric Interpretation and link with Singular Value ' 'Decomposition', 2, None, - '___sec67'), - ('Principal Component Analysis', 2, None, '___sec68'), - ('PCA and scikit-learn', 2, None, '___sec69'), - ('Back to the Cancer Data', 2, None, '___sec70'), - ('More on the PCA', 2, None, '___sec71'), - ('Incremental PCA', 2, None, '___sec72'), - ('Randomized PCA', 2, None, '___sec73'), - ('Kernel PCA', 2, None, '___sec74'), - ('LLE', 2, None, '___sec75'), - ('Other techniques', 2, None, '___sec76')]} + '___sec68'), + ('Principal Component Analysis', 2, None, '___sec69'), + ('PCA and scikit-learn', 2, None, '___sec70'), + ('Back to the Cancer Data', 2, None, '___sec71'), + ('More on the PCA', 2, None, '___sec72'), + ('Incremental PCA', 2, None, '___sec73'), + ('Randomized PCA', 2, None, '___sec74'), + ('Kernel PCA', 2, None, '___sec75'), + ('LLE', 2, None, '___sec76'), + ('Other techniques', 2, None, '___sec77')]} end of tocinfo --> @@ -230,7 +230,7 @@ MathJax.Hub.Config({
  • Final technicalities II
  • Final technicalities III
  • Final technicalities IV
    • -
  • Backpropagation
    • +
  • Back propagation
  • Gradient descent
  • The code for solving the ODE
  • The network with one input layer, specified number of hidden layers, and one output layer
    • @@ -239,55 +239,56 @@ MathJax.Hub.Config({
  • The trial solution
  • The program using Autograd
  • Using forward Euler to solve the ODE
    • -
  • Example: Solving the one dimensional Poisson equation using Autograd and TensorFlow
    • +
  • Example: Solving the one dimensional Poisson equation
  • The specific equation to solve for
  • Solving the equation using Autograd
  • Comparing with a numerical scheme
    • -
  • Partial Differential Equations
    • -
  • Type of problem
    • -
  • Network requirements
    • -
  • More details
    • -
  • Example: The diffusion equation
    • -
  • Defining the problem
    • -
  • Setting up the network using Autograd
    • -
  • Setting up the network using Autograd; The trial solution
    • -
  • Why the jacobian?
    • -
  • Setting up the network using Autograd; The full program
    • -
  • Example: Solving the wave equation with Neural Networks
    • -
  • The problem to solve for
    • -
  • The trial solution
    • -
  • The analytical solution
    • -
  • Solving the wave equation - the full program using Autograd
    • -
  • Resources on differential equations and deep learning
    • -
  • Friday, Principal Component Analysis
    • -
  • Basic ideas of the Principal Component Analysis (PCA)
    • -
  • Introducing the Covariance and Correlation functions
    • -
  • Correlation Function and Design/Feature Matrix
    • -
  • Covariance Matrix Examples
    • -
  • Correlation Matrix
    • -
  • Correlation Matrix with Pandas
    • -
  • Correlation Matrix with Pandas and the Franke function
    • -
  • Rewriting the Covariance and/or Correlation Matrix
    • -
  • Towards the PCA theorem
    • -
  • The Algorithm before the Theorem
    • -
  • Writing our own PCA code
    • -
  •    Compute the sample mean and center the data
    • -
  •    Compute the sample covariance
    • -
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • -
  • Classical PCA Theorem
    • -
  • Proof of the PCA Theorem
    • -
  • PCA Proof continued
    • -
  • The final step
    • -
  • Geometric Interpretation and link with Singular Value Decomposition
    • -
  • Principal Component Analysis
    • -
  • PCA and scikit-learn
    • -
  • Back to the Cancer Data
    • -
  • More on the PCA
    • -
  • Incremental PCA
    • -
  • Randomized PCA
    • -
  • Kernel PCA
    • -
  • LLE
    • -
  • Other techniques
    • +
  • Setting up the code
    • +
  • Partial Differential Equations
    • +
  • Type of problem
    • +
  • Network requirements
    • +
  • More details
    • +
  • Example: The diffusion equation
    • +
  • Defining the problem
    • +
  • Setting up the network using Autograd
    • +
  • Setting up the network using Autograd; The trial solution
    • +
  • Why the jacobian?
    • +
  • Setting up the network using Autograd; The full program
    • +
  • Example: Solving the wave equation with Neural Networks
    • +
  • The problem to solve for
    • +
  • The trial solution
    • +
  • The analytical solution
    • +
  • Solving the wave equation - the full program using Autograd
    • +
  • Resources on differential equations and deep learning
    • +
  • Friday, Principal Component Analysis
    • +
  • Basic ideas of the Principal Component Analysis (PCA)
    • +
  • Introducing the Covariance and Correlation functions
    • +
  • Correlation Function and Design/Feature Matrix
    • +
  • Covariance Matrix Examples
    • +
  • Correlation Matrix
    • +
  • Correlation Matrix with Pandas
    • +
  • Correlation Matrix with Pandas and the Franke function
    • +
  • Rewriting the Covariance and/or Correlation Matrix
    • +
  • Towards the PCA theorem
    • +
  • The Algorithm before the Theorem
    • +
  • Writing our own PCA code
    • +
  •    Compute the sample mean and center the data
    • +
  •    Compute the sample covariance
    • +
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • +
  • Classical PCA Theorem
    • +
  • Proof of the PCA Theorem
    • +
  • PCA Proof continued
    • +
  • The final step
    • +
  • Geometric Interpretation and link with Singular Value Decomposition
    • +
  • Principal Component Analysis
    • +
  • PCA and scikit-learn
    • +
  • Back to the Cancer Data
    • +
  • More on the PCA
    • +
  • Incremental PCA
    • +
  • Randomized PCA
    • +
  • Kernel PCA
    • +
  • LLE
    • +
  • Other techniques
    • @@ -339,7 +340,7 @@ with \( h_1(x) \) ensuring that \( g_t(x) \) satisfies some conditions and \( h_
  • 19
  • 20
  • ...
    • -
  • 76
    • +
  • 77
  • »
    • diff --git a/doc/pub/week43/html/._week43-bs011.html b/doc/pub/week43/html/._week43-bs011.html index a74838958..dd39cdf93 100644 --- a/doc/pub/week43/html/._week43-bs011.html +++ b/doc/pub/week43/html/._week43-bs011.html @@ -67,7 +67,7 @@ Automatically generated HTML file from DocOnce source ('Final technicalities II', 2, None, '___sec16'), ('Final technicalities III', 2, None, '___sec17'), ('Final technicalities IV', 2, None, '___sec18'), - ('Backpropagation', 2, None, '___sec19'), + ('Back propagation', 2, None, '___sec19'), ('Gradient descent', 2, None, '___sec20'), ('The code for solving the ODE', 2, None, '___sec21'), ('The network with one input layer, specified number of hidden ' @@ -80,100 +80,100 @@ Automatically generated HTML file from DocOnce source ('The trial solution', 2, None, '___sec25'), ('The program using Autograd', 2, None, '___sec26'), ('Using forward Euler to solve the ODE', 2, None, '___sec27'), - ('Example: Solving the one dimensional Poisson equation using ' - 'Autograd and TensorFlow', + ('Example: Solving the one dimensional Poisson equation', 2, None, '___sec28'), ('The specific equation to solve for', 2, None, '___sec29'), ('Solving the equation using Autograd', 2, None, '___sec30'), ('Comparing with a numerical scheme', 2, None, '___sec31'), - ('Partial Differential Equations', 2, None, '___sec32'), - ('Type of problem', 2, None, '___sec33'), - ('Network requirements', 2, None, '___sec34'), - ('More details', 2, None, '___sec35'), - ('Example: The diffusion equation', 2, None, '___sec36'), - ('Defining the problem', 2, None, '___sec37'), - ('Setting up the network using Autograd', 2, None, '___sec38'), + ('Setting up the code', 2, None, '___sec32'), + ('Partial Differential Equations', 2, None, '___sec33'), + ('Type of problem', 2, None, '___sec34'), + ('Network requirements', 2, None, '___sec35'), + ('More details', 2, None, '___sec36'), + ('Example: The diffusion equation', 2, None, '___sec37'), + ('Defining the problem', 2, None, '___sec38'), + ('Setting up the network using Autograd', 2, None, '___sec39'), ('Setting up the network using Autograd; The trial solution', 2, None, - '___sec39'), - ('Why the jacobian?', 2, None, '___sec40'), + '___sec40'), + ('Why the jacobian?', 2, None, '___sec41'), ('Setting up the network using Autograd; The full program', 2, None, - '___sec41'), + '___sec42'), ('Example: Solving the wave equation with Neural Networks', 2, None, - '___sec42'), - ('The problem to solve for', 2, None, '___sec43'), - ('The trial solution', 2, None, '___sec44'), - ('The analytical solution', 2, None, '___sec45'), + '___sec43'), + ('The problem to solve for', 2, None, '___sec44'), + ('The trial solution', 2, None, '___sec45'), + ('The analytical solution', 2, None, '___sec46'), ('Solving the wave equation - the full program using Autograd', 2, None, - '___sec46'), + '___sec47'), ('Resources on differential equations and deep learning', 2, None, - '___sec47'), - ('Friday, Principal Component Analysis', 2, None, '___sec48'), + '___sec48'), + ('Friday, Principal Component Analysis', 2, None, '___sec49'), ('Basic ideas of the Principal Component Analysis (PCA)', 2, None, - '___sec49'), + '___sec50'), ('Introducing the Covariance and Correlation functions', 2, None, - '___sec50'), + '___sec51'), ('Correlation Function and Design/Feature Matrix', 2, None, - '___sec51'), - ('Covariance Matrix Examples', 2, None, '___sec52'), - ('Correlation Matrix', 2, None, '___sec53'), - ('Correlation Matrix with Pandas', 2, None, '___sec54'), + '___sec52'), + ('Covariance Matrix Examples', 2, None, '___sec53'), + ('Correlation Matrix', 2, None, '___sec54'), + ('Correlation Matrix with Pandas', 2, None, '___sec55'), ('Correlation Matrix with Pandas and the Franke function', 2, None, - '___sec55'), + '___sec56'), ('Rewriting the Covariance and/or Correlation Matrix', 2, None, - '___sec56'), - ('Towards the PCA theorem', 2, None, '___sec57'), - ('The Algorithm before the Theorem', 2, None, '___sec58'), - ('Writing our own PCA code', 2, None, '___sec59'), + '___sec57'), + ('Towards the PCA theorem', 2, None, '___sec58'), + ('The Algorithm before the Theorem', 2, None, '___sec59'), + ('Writing our own PCA code', 2, None, '___sec60'), ('Compute the sample mean and center the data', 3, None, - '___sec60'), - ('Compute the sample covariance', 3, None, '___sec61'), + '___sec61'), + ('Compute the sample covariance', 3, None, '___sec62'), ('Diagonalize the sample covariance matrix to obtain the ' 'principal components', 3, None, - '___sec62'), - ('Classical PCA Theorem', 2, None, '___sec63'), - ('Proof of the PCA Theorem', 2, None, '___sec64'), - ('PCA Proof continued', 2, None, '___sec65'), - ('The final step', 2, None, '___sec66'), + '___sec63'), + ('Classical PCA Theorem', 2, None, '___sec64'), + ('Proof of the PCA Theorem', 2, None, '___sec65'), + ('PCA Proof continued', 2, None, '___sec66'), + ('The final step', 2, None, '___sec67'), ('Geometric Interpretation and link with Singular Value ' 'Decomposition', 2, None, - '___sec67'), - ('Principal Component Analysis', 2, None, '___sec68'), - ('PCA and scikit-learn', 2, None, '___sec69'), - ('Back to the Cancer Data', 2, None, '___sec70'), - ('More on the PCA', 2, None, '___sec71'), - ('Incremental PCA', 2, None, '___sec72'), - ('Randomized PCA', 2, None, '___sec73'), - ('Kernel PCA', 2, None, '___sec74'), - ('LLE', 2, None, '___sec75'), - ('Other techniques', 2, None, '___sec76')]} + '___sec68'), + ('Principal Component Analysis', 2, None, '___sec69'), + ('PCA and scikit-learn', 2, None, '___sec70'), + ('Back to the Cancer Data', 2, None, '___sec71'), + ('More on the PCA', 2, None, '___sec72'), + ('Incremental PCA', 2, None, '___sec73'), + ('Randomized PCA', 2, None, '___sec74'), + ('Kernel PCA', 2, None, '___sec75'), + ('LLE', 2, None, '___sec76'), + ('Other techniques', 2, None, '___sec77')]} end of tocinfo --> @@ -230,7 +230,7 @@ MathJax.Hub.Config({
  • Final technicalities II
  • Final technicalities III
  • Final technicalities IV
    • -
  • Backpropagation
    • +
  • Back propagation
  • Gradient descent
  • The code for solving the ODE
  • The network with one input layer, specified number of hidden layers, and one output layer
    • @@ -239,55 +239,56 @@ MathJax.Hub.Config({
  • The trial solution
  • The program using Autograd
  • Using forward Euler to solve the ODE
    • -
  • Example: Solving the one dimensional Poisson equation using Autograd and TensorFlow
    • +
  • Example: Solving the one dimensional Poisson equation
  • The specific equation to solve for
  • Solving the equation using Autograd
  • Comparing with a numerical scheme
    • -
  • Partial Differential Equations
    • -
  • Type of problem
    • -
  • Network requirements
    • -
  • More details
    • -
  • Example: The diffusion equation
    • -
  • Defining the problem
    • -
  • Setting up the network using Autograd
    • -
  • Setting up the network using Autograd; The trial solution
    • -
  • Why the jacobian?
    • -
  • Setting up the network using Autograd; The full program
    • -
  • Example: Solving the wave equation with Neural Networks
    • -
  • The problem to solve for
    • -
  • The trial solution
    • -
  • The analytical solution
    • -
  • Solving the wave equation - the full program using Autograd
    • -
  • Resources on differential equations and deep learning
    • -
  • Friday, Principal Component Analysis
    • -
  • Basic ideas of the Principal Component Analysis (PCA)
    • -
  • Introducing the Covariance and Correlation functions
    • -
  • Correlation Function and Design/Feature Matrix
    • -
  • Covariance Matrix Examples
    • -
  • Correlation Matrix
    • -
  • Correlation Matrix with Pandas
    • -
  • Correlation Matrix with Pandas and the Franke function
    • -
  • Rewriting the Covariance and/or Correlation Matrix
    • -
  • Towards the PCA theorem
    • -
  • The Algorithm before the Theorem
    • -
  • Writing our own PCA code
    • -
  •    Compute the sample mean and center the data
    • -
  •    Compute the sample covariance
    • -
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • -
  • Classical PCA Theorem
    • -
  • Proof of the PCA Theorem
    • -
  • PCA Proof continued
    • -
  • The final step
    • -
  • Geometric Interpretation and link with Singular Value Decomposition
    • -
  • Principal Component Analysis
    • -
  • PCA and scikit-learn
    • -
  • Back to the Cancer Data
    • -
  • More on the PCA
    • -
  • Incremental PCA
    • -
  • Randomized PCA
    • -
  • Kernel PCA
    • -
  • LLE
    • -
  • Other techniques
    • +
  • Setting up the code
    • +
  • Partial Differential Equations
    • +
  • Type of problem
    • +
  • Network requirements
    • +
  • More details
    • +
  • Example: The diffusion equation
    • +
  • Defining the problem
    • +
  • Setting up the network using Autograd
    • +
  • Setting up the network using Autograd; The trial solution
    • +
  • Why the jacobian?
    • +
  • Setting up the network using Autograd; The full program
    • +
  • Example: Solving the wave equation with Neural Networks
    • +
  • The problem to solve for
    • +
  • The trial solution
    • +
  • The analytical solution
    • +
  • Solving the wave equation - the full program using Autograd
    • +
  • Resources on differential equations and deep learning
    • +
  • Friday, Principal Component Analysis
    • +
  • Basic ideas of the Principal Component Analysis (PCA)
    • +
  • Introducing the Covariance and Correlation functions
    • +
  • Correlation Function and Design/Feature Matrix
    • +
  • Covariance Matrix Examples
    • +
  • Correlation Matrix
    • +
  • Correlation Matrix with Pandas
    • +
  • Correlation Matrix with Pandas and the Franke function
    • +
  • Rewriting the Covariance and/or Correlation Matrix
    • +
  • Towards the PCA theorem
    • +
  • The Algorithm before the Theorem
    • +
  • Writing our own PCA code
    • +
  •    Compute the sample mean and center the data
    • +
  •    Compute the sample covariance
    • +
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • +
  • Classical PCA Theorem
    • +
  • Proof of the PCA Theorem
    • +
  • PCA Proof continued
    • +
  • The final step
    • +
  • Geometric Interpretation and link with Singular Value Decomposition
    • +
  • Principal Component Analysis
    • +
  • PCA and scikit-learn
    • +
  • Back to the Cancer Data
    • +
  • More on the PCA
    • +
  • Incremental PCA
    • +
  • Randomized PCA
    • +
  • Kernel PCA
    • +
  • LLE
    • +
  • Other techniques
    • @@ -351,7 +352,7 @@ $$
  • 20
  • 21
  • ...
    • -
  • 76
    • +
  • 77
  • »
    • diff --git a/doc/pub/week43/html/._week43-bs012.html b/doc/pub/week43/html/._week43-bs012.html index 9f48880c3..1ad0f8a89 100644 --- a/doc/pub/week43/html/._week43-bs012.html +++ b/doc/pub/week43/html/._week43-bs012.html @@ -67,7 +67,7 @@ Automatically generated HTML file from DocOnce source ('Final technicalities II', 2, None, '___sec16'), ('Final technicalities III', 2, None, '___sec17'), ('Final technicalities IV', 2, None, '___sec18'), - ('Backpropagation', 2, None, '___sec19'), + ('Back propagation', 2, None, '___sec19'), ('Gradient descent', 2, None, '___sec20'), ('The code for solving the ODE', 2, None, '___sec21'), ('The network with one input layer, specified number of hidden ' @@ -80,100 +80,100 @@ Automatically generated HTML file from DocOnce source ('The trial solution', 2, None, '___sec25'), ('The program using Autograd', 2, None, '___sec26'), ('Using forward Euler to solve the ODE', 2, None, '___sec27'), - ('Example: Solving the one dimensional Poisson equation using ' - 'Autograd and TensorFlow', + ('Example: Solving the one dimensional Poisson equation', 2, None, '___sec28'), ('The specific equation to solve for', 2, None, '___sec29'), ('Solving the equation using Autograd', 2, None, '___sec30'), ('Comparing with a numerical scheme', 2, None, '___sec31'), - ('Partial Differential Equations', 2, None, '___sec32'), - ('Type of problem', 2, None, '___sec33'), - ('Network requirements', 2, None, '___sec34'), - ('More details', 2, None, '___sec35'), - ('Example: The diffusion equation', 2, None, '___sec36'), - ('Defining the problem', 2, None, '___sec37'), - ('Setting up the network using Autograd', 2, None, '___sec38'), + ('Setting up the code', 2, None, '___sec32'), + ('Partial Differential Equations', 2, None, '___sec33'), + ('Type of problem', 2, None, '___sec34'), + ('Network requirements', 2, None, '___sec35'), + ('More details', 2, None, '___sec36'), + ('Example: The diffusion equation', 2, None, '___sec37'), + ('Defining the problem', 2, None, '___sec38'), + ('Setting up the network using Autograd', 2, None, '___sec39'), ('Setting up the network using Autograd; The trial solution', 2, None, - '___sec39'), - ('Why the jacobian?', 2, None, '___sec40'), + '___sec40'), + ('Why the jacobian?', 2, None, '___sec41'), ('Setting up the network using Autograd; The full program', 2, None, - '___sec41'), + '___sec42'), ('Example: Solving the wave equation with Neural Networks', 2, None, - '___sec42'), - ('The problem to solve for', 2, None, '___sec43'), - ('The trial solution', 2, None, '___sec44'), - ('The analytical solution', 2, None, '___sec45'), + '___sec43'), + ('The problem to solve for', 2, None, '___sec44'), + ('The trial solution', 2, None, '___sec45'), + ('The analytical solution', 2, None, '___sec46'), ('Solving the wave equation - the full program using Autograd', 2, None, - '___sec46'), + '___sec47'), ('Resources on differential equations and deep learning', 2, None, - '___sec47'), - ('Friday, Principal Component Analysis', 2, None, '___sec48'), + '___sec48'), + ('Friday, Principal Component Analysis', 2, None, '___sec49'), ('Basic ideas of the Principal Component Analysis (PCA)', 2, None, - '___sec49'), + '___sec50'), ('Introducing the Covariance and Correlation functions', 2, None, - '___sec50'), + '___sec51'), ('Correlation Function and Design/Feature Matrix', 2, None, - '___sec51'), - ('Covariance Matrix Examples', 2, None, '___sec52'), - ('Correlation Matrix', 2, None, '___sec53'), - ('Correlation Matrix with Pandas', 2, None, '___sec54'), + '___sec52'), + ('Covariance Matrix Examples', 2, None, '___sec53'), + ('Correlation Matrix', 2, None, '___sec54'), + ('Correlation Matrix with Pandas', 2, None, '___sec55'), ('Correlation Matrix with Pandas and the Franke function', 2, None, - '___sec55'), + '___sec56'), ('Rewriting the Covariance and/or Correlation Matrix', 2, None, - '___sec56'), - ('Towards the PCA theorem', 2, None, '___sec57'), - ('The Algorithm before the Theorem', 2, None, '___sec58'), - ('Writing our own PCA code', 2, None, '___sec59'), + '___sec57'), + ('Towards the PCA theorem', 2, None, '___sec58'), + ('The Algorithm before the Theorem', 2, None, '___sec59'), + ('Writing our own PCA code', 2, None, '___sec60'), ('Compute the sample mean and center the data', 3, None, - '___sec60'), - ('Compute the sample covariance', 3, None, '___sec61'), + '___sec61'), + ('Compute the sample covariance', 3, None, '___sec62'), ('Diagonalize the sample covariance matrix to obtain the ' 'principal components', 3, None, - '___sec62'), - ('Classical PCA Theorem', 2, None, '___sec63'), - ('Proof of the PCA Theorem', 2, None, '___sec64'), - ('PCA Proof continued', 2, None, '___sec65'), - ('The final step', 2, None, '___sec66'), + '___sec63'), + ('Classical PCA Theorem', 2, None, '___sec64'), + ('Proof of the PCA Theorem', 2, None, '___sec65'), + ('PCA Proof continued', 2, None, '___sec66'), + ('The final step', 2, None, '___sec67'), ('Geometric Interpretation and link with Singular Value ' 'Decomposition', 2, None, - '___sec67'), - ('Principal Component Analysis', 2, None, '___sec68'), - ('PCA and scikit-learn', 2, None, '___sec69'), - ('Back to the Cancer Data', 2, None, '___sec70'), - ('More on the PCA', 2, None, '___sec71'), - ('Incremental PCA', 2, None, '___sec72'), - ('Randomized PCA', 2, None, '___sec73'), - ('Kernel PCA', 2, None, '___sec74'), - ('LLE', 2, None, '___sec75'), - ('Other techniques', 2, None, '___sec76')]} + '___sec68'), + ('Principal Component Analysis', 2, None, '___sec69'), + ('PCA and scikit-learn', 2, None, '___sec70'), + ('Back to the Cancer Data', 2, None, '___sec71'), + ('More on the PCA', 2, None, '___sec72'), + ('Incremental PCA', 2, None, '___sec73'), + ('Randomized PCA', 2, None, '___sec74'), + ('Kernel PCA', 2, None, '___sec75'), + ('LLE', 2, None, '___sec76'), + ('Other techniques', 2, None, '___sec77')]} end of tocinfo --> @@ -230,7 +230,7 @@ MathJax.Hub.Config({
  • Final technicalities II
  • Final technicalities III
  • Final technicalities IV
    • -
  • Backpropagation
    • +
  • Back propagation
  • Gradient descent
  • The code for solving the ODE
  • The network with one input layer, specified number of hidden layers, and one output layer
    • @@ -239,55 +239,56 @@ MathJax.Hub.Config({
  • The trial solution
  • The program using Autograd
  • Using forward Euler to solve the ODE
    • -
  • Example: Solving the one dimensional Poisson equation using Autograd and TensorFlow
    • +
  • Example: Solving the one dimensional Poisson equation
  • The specific equation to solve for
  • Solving the equation using Autograd
  • Comparing with a numerical scheme
    • -
  • Partial Differential Equations
    • -
  • Type of problem
    • -
  • Network requirements
    • -
  • More details
    • -
  • Example: The diffusion equation
    • -
  • Defining the problem
    • -
  • Setting up the network using Autograd
    • -
  • Setting up the network using Autograd; The trial solution
    • -
  • Why the jacobian?
    • -
  • Setting up the network using Autograd; The full program
    • -
  • Example: Solving the wave equation with Neural Networks
    • -
  • The problem to solve for
    • -
  • The trial solution
    • -
  • The analytical solution
    • -
  • Solving the wave equation - the full program using Autograd
    • -
  • Resources on differential equations and deep learning
    • -
  • Friday, Principal Component Analysis
    • -
  • Basic ideas of the Principal Component Analysis (PCA)
    • -
  • Introducing the Covariance and Correlation functions
    • -
  • Correlation Function and Design/Feature Matrix
    • -
  • Covariance Matrix Examples
    • -
  • Correlation Matrix
    • -
  • Correlation Matrix with Pandas
    • -
  • Correlation Matrix with Pandas and the Franke function
    • -
  • Rewriting the Covariance and/or Correlation Matrix
    • -
  • Towards the PCA theorem
    • -
  • The Algorithm before the Theorem
    • -
  • Writing our own PCA code
    • -
  •    Compute the sample mean and center the data
    • -
  •    Compute the sample covariance
    • -
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • -
  • Classical PCA Theorem
    • -
  • Proof of the PCA Theorem
    • -
  • PCA Proof continued
    • -
  • The final step
    • -
  • Geometric Interpretation and link with Singular Value Decomposition
    • -
  • Principal Component Analysis
    • -
  • PCA and scikit-learn
    • -
  • Back to the Cancer Data
    • -
  • More on the PCA
    • -
  • Incremental PCA
    • -
  • Randomized PCA
    • -
  • Kernel PCA
    • -
  • LLE
    • -
  • Other techniques
    • +
  • Setting up the code
    • +
  • Partial Differential Equations
    • +
  • Type of problem
    • +
  • Network requirements
    • +
  • More details
    • +
  • Example: The diffusion equation
    • +
  • Defining the problem
    • +
  • Setting up the network using Autograd
    • +
  • Setting up the network using Autograd; The trial solution
    • +
  • Why the jacobian?
    • +
  • Setting up the network using Autograd; The full program
    • +
  • Example: Solving the wave equation with Neural Networks
    • +
  • The problem to solve for
    • +
  • The trial solution
    • +
  • The analytical solution
    • +
  • Solving the wave equation - the full program using Autograd
    • +
  • Resources on differential equations and deep learning
    • +
  • Friday, Principal Component Analysis
    • +
  • Basic ideas of the Principal Component Analysis (PCA)
    • +
  • Introducing the Covariance and Correlation functions
    • +
  • Correlation Function and Design/Feature Matrix
    • +
  • Covariance Matrix Examples
    • +
  • Correlation Matrix
    • +
  • Correlation Matrix with Pandas
    • +
  • Correlation Matrix with Pandas and the Franke function
    • +
  • Rewriting the Covariance and/or Correlation Matrix
    • +
  • Towards the PCA theorem
    • +
  • The Algorithm before the Theorem
    • +
  • Writing our own PCA code
    • +
  •    Compute the sample mean and center the data
    • +
  •    Compute the sample covariance
    • +
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • +
  • Classical PCA Theorem
    • +
  • Proof of the PCA Theorem
    • +
  • PCA Proof continued
    • +
  • The final step
    • +
  • Geometric Interpretation and link with Singular Value Decomposition
    • +
  • Principal Component Analysis
    • +
  • PCA and scikit-learn
    • +
  • Back to the Cancer Data
    • +
  • More on the PCA
    • +
  • Incremental PCA
    • +
  • Randomized PCA
    • +
  • Kernel PCA
    • +
  • LLE
    • +
  • Other techniques
    • @@ -360,7 +361,7 @@ is fulfilled as best as possible.
  • 21
  • 22
  • ...
    • -
  • 76
    • +
  • 77
  • »
    • diff --git a/doc/pub/week43/html/._week43-bs013.html b/doc/pub/week43/html/._week43-bs013.html index 2c109787d..3a49a07c6 100644 --- a/doc/pub/week43/html/._week43-bs013.html +++ b/doc/pub/week43/html/._week43-bs013.html @@ -67,7 +67,7 @@ Automatically generated HTML file from DocOnce source ('Final technicalities II', 2, None, '___sec16'), ('Final technicalities III', 2, None, '___sec17'), ('Final technicalities IV', 2, None, '___sec18'), - ('Backpropagation', 2, None, '___sec19'), + ('Back propagation', 2, None, '___sec19'), ('Gradient descent', 2, None, '___sec20'), ('The code for solving the ODE', 2, None, '___sec21'), ('The network with one input layer, specified number of hidden ' @@ -80,100 +80,100 @@ Automatically generated HTML file from DocOnce source ('The trial solution', 2, None, '___sec25'), ('The program using Autograd', 2, None, '___sec26'), ('Using forward Euler to solve the ODE', 2, None, '___sec27'), - ('Example: Solving the one dimensional Poisson equation using ' - 'Autograd and TensorFlow', + ('Example: Solving the one dimensional Poisson equation', 2, None, '___sec28'), ('The specific equation to solve for', 2, None, '___sec29'), ('Solving the equation using Autograd', 2, None, '___sec30'), ('Comparing with a numerical scheme', 2, None, '___sec31'), - ('Partial Differential Equations', 2, None, '___sec32'), - ('Type of problem', 2, None, '___sec33'), - ('Network requirements', 2, None, '___sec34'), - ('More details', 2, None, '___sec35'), - ('Example: The diffusion equation', 2, None, '___sec36'), - ('Defining the problem', 2, None, '___sec37'), - ('Setting up the network using Autograd', 2, None, '___sec38'), + ('Setting up the code', 2, None, '___sec32'), + ('Partial Differential Equations', 2, None, '___sec33'), + ('Type of problem', 2, None, '___sec34'), + ('Network requirements', 2, None, '___sec35'), + ('More details', 2, None, '___sec36'), + ('Example: The diffusion equation', 2, None, '___sec37'), + ('Defining the problem', 2, None, '___sec38'), + ('Setting up the network using Autograd', 2, None, '___sec39'), ('Setting up the network using Autograd; The trial solution', 2, None, - '___sec39'), - ('Why the jacobian?', 2, None, '___sec40'), + '___sec40'), + ('Why the jacobian?', 2, None, '___sec41'), ('Setting up the network using Autograd; The full program', 2, None, - '___sec41'), + '___sec42'), ('Example: Solving the wave equation with Neural Networks', 2, None, - '___sec42'), - ('The problem to solve for', 2, None, '___sec43'), - ('The trial solution', 2, None, '___sec44'), - ('The analytical solution', 2, None, '___sec45'), + '___sec43'), + ('The problem to solve for', 2, None, '___sec44'), + ('The trial solution', 2, None, '___sec45'), + ('The analytical solution', 2, None, '___sec46'), ('Solving the wave equation - the full program using Autograd', 2, None, - '___sec46'), + '___sec47'), ('Resources on differential equations and deep learning', 2, None, - '___sec47'), - ('Friday, Principal Component Analysis', 2, None, '___sec48'), + '___sec48'), + ('Friday, Principal Component Analysis', 2, None, '___sec49'), ('Basic ideas of the Principal Component Analysis (PCA)', 2, None, - '___sec49'), + '___sec50'), ('Introducing the Covariance and Correlation functions', 2, None, - '___sec50'), + '___sec51'), ('Correlation Function and Design/Feature Matrix', 2, None, - '___sec51'), - ('Covariance Matrix Examples', 2, None, '___sec52'), - ('Correlation Matrix', 2, None, '___sec53'), - ('Correlation Matrix with Pandas', 2, None, '___sec54'), + '___sec52'), + ('Covariance Matrix Examples', 2, None, '___sec53'), + ('Correlation Matrix', 2, None, '___sec54'), + ('Correlation Matrix with Pandas', 2, None, '___sec55'), ('Correlation Matrix with Pandas and the Franke function', 2, None, - '___sec55'), + '___sec56'), ('Rewriting the Covariance and/or Correlation Matrix', 2, None, - '___sec56'), - ('Towards the PCA theorem', 2, None, '___sec57'), - ('The Algorithm before the Theorem', 2, None, '___sec58'), - ('Writing our own PCA code', 2, None, '___sec59'), + '___sec57'), + ('Towards the PCA theorem', 2, None, '___sec58'), + ('The Algorithm before the Theorem', 2, None, '___sec59'), + ('Writing our own PCA code', 2, None, '___sec60'), ('Compute the sample mean and center the data', 3, None, - '___sec60'), - ('Compute the sample covariance', 3, None, '___sec61'), + '___sec61'), + ('Compute the sample covariance', 3, None, '___sec62'), ('Diagonalize the sample covariance matrix to obtain the ' 'principal components', 3, None, - '___sec62'), - ('Classical PCA Theorem', 2, None, '___sec63'), - ('Proof of the PCA Theorem', 2, None, '___sec64'), - ('PCA Proof continued', 2, None, '___sec65'), - ('The final step', 2, None, '___sec66'), + '___sec63'), + ('Classical PCA Theorem', 2, None, '___sec64'), + ('Proof of the PCA Theorem', 2, None, '___sec65'), + ('PCA Proof continued', 2, None, '___sec66'), + ('The final step', 2, None, '___sec67'), ('Geometric Interpretation and link with Singular Value ' 'Decomposition', 2, None, - '___sec67'), - ('Principal Component Analysis', 2, None, '___sec68'), - ('PCA and scikit-learn', 2, None, '___sec69'), - ('Back to the Cancer Data', 2, None, '___sec70'), - ('More on the PCA', 2, None, '___sec71'), - ('Incremental PCA', 2, None, '___sec72'), - ('Randomized PCA', 2, None, '___sec73'), - ('Kernel PCA', 2, None, '___sec74'), - ('LLE', 2, None, '___sec75'), - ('Other techniques', 2, None, '___sec76')]} + '___sec68'), + ('Principal Component Analysis', 2, None, '___sec69'), + ('PCA and scikit-learn', 2, None, '___sec70'), + ('Back to the Cancer Data', 2, None, '___sec71'), + ('More on the PCA', 2, None, '___sec72'), + ('Incremental PCA', 2, None, '___sec73'), + ('Randomized PCA', 2, None, '___sec74'), + ('Kernel PCA', 2, None, '___sec75'), + ('LLE', 2, None, '___sec76'), + ('Other techniques', 2, None, '___sec77')]} end of tocinfo --> @@ -230,7 +230,7 @@ MathJax.Hub.Config({
  • Final technicalities II
  • Final technicalities III
  • Final technicalities IV
    • -
  • Backpropagation
    • +
  • Back propagation
  • Gradient descent
  • The code for solving the ODE
  • The network with one input layer, specified number of hidden layers, and one output layer
    • @@ -239,55 +239,56 @@ MathJax.Hub.Config({
  • The trial solution
  • The program using Autograd
  • Using forward Euler to solve the ODE
    • -
  • Example: Solving the one dimensional Poisson equation using Autograd and TensorFlow
    • +
  • Example: Solving the one dimensional Poisson equation
  • The specific equation to solve for
  • Solving the equation using Autograd
  • Comparing with a numerical scheme
    • -
  • Partial Differential Equations
    • -
  • Type of problem
    • -
  • Network requirements
    • -
  • More details
    • -
  • Example: The diffusion equation
    • -
  • Defining the problem
    • -
  • Setting up the network using Autograd
    • -
  • Setting up the network using Autograd; The trial solution
    • -
  • Why the jacobian?
    • -
  • Setting up the network using Autograd; The full program
    • -
  • Example: Solving the wave equation with Neural Networks
    • -
  • The problem to solve for
    • -
  • The trial solution
    • -
  • The analytical solution
    • -
  • Solving the wave equation - the full program using Autograd
    • -
  • Resources on differential equations and deep learning
    • -
  • Friday, Principal Component Analysis
    • -
  • Basic ideas of the Principal Component Analysis (PCA)
    • -
  • Introducing the Covariance and Correlation functions
    • -
  • Correlation Function and Design/Feature Matrix
    • -
  • Covariance Matrix Examples
    • -
  • Correlation Matrix
    • -
  • Correlation Matrix with Pandas
    • -
  • Correlation Matrix with Pandas and the Franke function
    • -
  • Rewriting the Covariance and/or Correlation Matrix
    • -
  • Towards the PCA theorem
    • -
  • The Algorithm before the Theorem
    • -
  • Writing our own PCA code
    • -
  •    Compute the sample mean and center the data
    • -
  •    Compute the sample covariance
    • -
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • -
  • Classical PCA Theorem
    • -
  • Proof of the PCA Theorem
    • -
  • PCA Proof continued
    • -
  • The final step
    • -
  • Geometric Interpretation and link with Singular Value Decomposition
    • -
  • Principal Component Analysis
    • -
  • PCA and scikit-learn
    • -
  • Back to the Cancer Data
    • -
  • More on the PCA
    • -
  • Incremental PCA
    • -
  • Randomized PCA
    • -
  • Kernel PCA
    • -
  • LLE
    • -
  • Other techniques
    • +
  • Setting up the code
    • +
  • Partial Differential Equations
    • +
  • Type of problem
    • +
  • Network requirements
    • +
  • More details
    • +
  • Example: The diffusion equation
    • +
  • Defining the problem
    • +
  • Setting up the network using Autograd
    • +
  • Setting up the network using Autograd; The trial solution
    • +
  • Why the jacobian?
    • +
  • Setting up the network using Autograd; The full program
    • +
  • Example: Solving the wave equation with Neural Networks
    • +
  • The problem to solve for
    • +
  • The trial solution
    • +
  • The analytical solution
    • +
  • Solving the wave equation - the full program using Autograd
    • +
  • Resources on differential equations and deep learning
    • +
  • Friday, Principal Component Analysis
    • +
  • Basic ideas of the Principal Component Analysis (PCA)
    • +
  • Introducing the Covariance and Correlation functions
    • +
  • Correlation Function and Design/Feature Matrix
    • +
  • Covariance Matrix Examples
    • +
  • Correlation Matrix
    • +
  • Correlation Matrix with Pandas
    • +
  • Correlation Matrix with Pandas and the Franke function
    • +
  • Rewriting the Covariance and/or Correlation Matrix
    • +
  • Towards the PCA theorem
    • +
  • The Algorithm before the Theorem
    • +
  • Writing our own PCA code
    • +
  •    Compute the sample mean and center the data
    • +
  •    Compute the sample covariance
    • +
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • +
  • Classical PCA Theorem
    • +
  • Proof of the PCA Theorem
    • +
  • PCA Proof continued
    • +
  • The final step
    • +
  • Geometric Interpretation and link with Singular Value Decomposition
    • +
  • Principal Component Analysis
    • +
  • PCA and scikit-learn
    • +
  • Back to the Cancer Data
    • +
  • More on the PCA
    • +
  • Incremental PCA
    • +
  • Randomized PCA
    • +
  • Kernel PCA
    • +
  • LLE
    • +
  • Other techniques
    • @@ -356,7 +357,7 @@ for an input value \( x \).
  • 22
  • 23
  • ...
    • -
  • 76
    • +
  • 77
  • »
    • diff --git a/doc/pub/week43/html/._week43-bs014.html b/doc/pub/week43/html/._week43-bs014.html index 0b5138ed0..0da56dce0 100644 --- a/doc/pub/week43/html/._week43-bs014.html +++ b/doc/pub/week43/html/._week43-bs014.html @@ -67,7 +67,7 @@ Automatically generated HTML file from DocOnce source ('Final technicalities II', 2, None, '___sec16'), ('Final technicalities III', 2, None, '___sec17'), ('Final technicalities IV', 2, None, '___sec18'), - ('Backpropagation', 2, None, '___sec19'), + ('Back propagation', 2, None, '___sec19'), ('Gradient descent', 2, None, '___sec20'), ('The code for solving the ODE', 2, None, '___sec21'), ('The network with one input layer, specified number of hidden ' @@ -80,100 +80,100 @@ Automatically generated HTML file from DocOnce source ('The trial solution', 2, None, '___sec25'), ('The program using Autograd', 2, None, '___sec26'), ('Using forward Euler to solve the ODE', 2, None, '___sec27'), - ('Example: Solving the one dimensional Poisson equation using ' - 'Autograd and TensorFlow', + ('Example: Solving the one dimensional Poisson equation', 2, None, '___sec28'), ('The specific equation to solve for', 2, None, '___sec29'), ('Solving the equation using Autograd', 2, None, '___sec30'), ('Comparing with a numerical scheme', 2, None, '___sec31'), - ('Partial Differential Equations', 2, None, '___sec32'), - ('Type of problem', 2, None, '___sec33'), - ('Network requirements', 2, None, '___sec34'), - ('More details', 2, None, '___sec35'), - ('Example: The diffusion equation', 2, None, '___sec36'), - ('Defining the problem', 2, None, '___sec37'), - ('Setting up the network using Autograd', 2, None, '___sec38'), + ('Setting up the code', 2, None, '___sec32'), + ('Partial Differential Equations', 2, None, '___sec33'), + ('Type of problem', 2, None, '___sec34'), + ('Network requirements', 2, None, '___sec35'), + ('More details', 2, None, '___sec36'), + ('Example: The diffusion equation', 2, None, '___sec37'), + ('Defining the problem', 2, None, '___sec38'), + ('Setting up the network using Autograd', 2, None, '___sec39'), ('Setting up the network using Autograd; The trial solution', 2, None, - '___sec39'), - ('Why the jacobian?', 2, None, '___sec40'), + '___sec40'), + ('Why the jacobian?', 2, None, '___sec41'), ('Setting up the network using Autograd; The full program', 2, None, - '___sec41'), + '___sec42'), ('Example: Solving the wave equation with Neural Networks', 2, None, - '___sec42'), - ('The problem to solve for', 2, None, '___sec43'), - ('The trial solution', 2, None, '___sec44'), - ('The analytical solution', 2, None, '___sec45'), + '___sec43'), + ('The problem to solve for', 2, None, '___sec44'), + ('The trial solution', 2, None, '___sec45'), + ('The analytical solution', 2, None, '___sec46'), ('Solving the wave equation - the full program using Autograd', 2, None, - '___sec46'), + '___sec47'), ('Resources on differential equations and deep learning', 2, None, - '___sec47'), - ('Friday, Principal Component Analysis', 2, None, '___sec48'), + '___sec48'), + ('Friday, Principal Component Analysis', 2, None, '___sec49'), ('Basic ideas of the Principal Component Analysis (PCA)', 2, None, - '___sec49'), + '___sec50'), ('Introducing the Covariance and Correlation functions', 2, None, - '___sec50'), + '___sec51'), ('Correlation Function and Design/Feature Matrix', 2, None, - '___sec51'), - ('Covariance Matrix Examples', 2, None, '___sec52'), - ('Correlation Matrix', 2, None, '___sec53'), - ('Correlation Matrix with Pandas', 2, None, '___sec54'), + '___sec52'), + ('Covariance Matrix Examples', 2, None, '___sec53'), + ('Correlation Matrix', 2, None, '___sec54'), + ('Correlation Matrix with Pandas', 2, None, '___sec55'), ('Correlation Matrix with Pandas and the Franke function', 2, None, - '___sec55'), + '___sec56'), ('Rewriting the Covariance and/or Correlation Matrix', 2, None, - '___sec56'), - ('Towards the PCA theorem', 2, None, '___sec57'), - ('The Algorithm before the Theorem', 2, None, '___sec58'), - ('Writing our own PCA code', 2, None, '___sec59'), + '___sec57'), + ('Towards the PCA theorem', 2, None, '___sec58'), + ('The Algorithm before the Theorem', 2, None, '___sec59'), + ('Writing our own PCA code', 2, None, '___sec60'), ('Compute the sample mean and center the data', 3, None, - '___sec60'), - ('Compute the sample covariance', 3, None, '___sec61'), + '___sec61'), + ('Compute the sample covariance', 3, None, '___sec62'), ('Diagonalize the sample covariance matrix to obtain the ' 'principal components', 3, None, - '___sec62'), - ('Classical PCA Theorem', 2, None, '___sec63'), - ('Proof of the PCA Theorem', 2, None, '___sec64'), - ('PCA Proof continued', 2, None, '___sec65'), - ('The final step', 2, None, '___sec66'), + '___sec63'), + ('Classical PCA Theorem', 2, None, '___sec64'), + ('Proof of the PCA Theorem', 2, None, '___sec65'), + ('PCA Proof continued', 2, None, '___sec66'), + ('The final step', 2, None, '___sec67'), ('Geometric Interpretation and link with Singular Value ' 'Decomposition', 2, None, - '___sec67'), - ('Principal Component Analysis', 2, None, '___sec68'), - ('PCA and scikit-learn', 2, None, '___sec69'), - ('Back to the Cancer Data', 2, None, '___sec70'), - ('More on the PCA', 2, None, '___sec71'), - ('Incremental PCA', 2, None, '___sec72'), - ('Randomized PCA', 2, None, '___sec73'), - ('Kernel PCA', 2, None, '___sec74'), - ('LLE', 2, None, '___sec75'), - ('Other techniques', 2, None, '___sec76')]} + '___sec68'), + ('Principal Component Analysis', 2, None, '___sec69'), + ('PCA and scikit-learn', 2, None, '___sec70'), + ('Back to the Cancer Data', 2, None, '___sec71'), + ('More on the PCA', 2, None, '___sec72'), + ('Incremental PCA', 2, None, '___sec73'), + ('Randomized PCA', 2, None, '___sec74'), + ('Kernel PCA', 2, None, '___sec75'), + ('LLE', 2, None, '___sec76'), + ('Other techniques', 2, None, '___sec77')]} end of tocinfo --> @@ -230,7 +230,7 @@ MathJax.Hub.Config({
  • Final technicalities II
  • Final technicalities III
  • Final technicalities IV
    • -
  • Backpropagation
    • +
  • Back propagation
  • Gradient descent
  • The code for solving the ODE
  • The network with one input layer, specified number of hidden layers, and one output layer
    • @@ -239,55 +239,56 @@ MathJax.Hub.Config({
  • The trial solution
  • The program using Autograd
  • Using forward Euler to solve the ODE
    • -
  • Example: Solving the one dimensional Poisson equation using Autograd and TensorFlow
    • +
  • Example: Solving the one dimensional Poisson equation
  • The specific equation to solve for
  • Solving the equation using Autograd
  • Comparing with a numerical scheme
    • -
  • Partial Differential Equations
    • -
  • Type of problem
    • -
  • Network requirements
    • -
  • More details
    • -
  • Example: The diffusion equation
    • -
  • Defining the problem
    • -
  • Setting up the network using Autograd
    • -
  • Setting up the network using Autograd; The trial solution
    • -
  • Why the jacobian?
    • -
  • Setting up the network using Autograd; The full program
    • -
  • Example: Solving the wave equation with Neural Networks
    • -
  • The problem to solve for
    • -
  • The trial solution
    • -
  • The analytical solution
    • -
  • Solving the wave equation - the full program using Autograd
    • -
  • Resources on differential equations and deep learning
    • -
  • Friday, Principal Component Analysis
    • -
  • Basic ideas of the Principal Component Analysis (PCA)
    • -
  • Introducing the Covariance and Correlation functions
    • -
  • Correlation Function and Design/Feature Matrix
    • -
  • Covariance Matrix Examples
    • -
  • Correlation Matrix
    • -
  • Correlation Matrix with Pandas
    • -
  • Correlation Matrix with Pandas and the Franke function
    • -
  • Rewriting the Covariance and/or Correlation Matrix
    • -
  • Towards the PCA theorem
    • -
  • The Algorithm before the Theorem
    • -
  • Writing our own PCA code
    • -
  •    Compute the sample mean and center the data
    • -
  •    Compute the sample covariance
    • -
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • -
  • Classical PCA Theorem
    • -
  • Proof of the PCA Theorem
    • -
  • PCA Proof continued
    • -
  • The final step
    • -
  • Geometric Interpretation and link with Singular Value Decomposition
    • -
  • Principal Component Analysis
    • -
  • PCA and scikit-learn
    • -
  • Back to the Cancer Data
    • -
  • More on the PCA
    • -
  • Incremental PCA
    • -
  • Randomized PCA
    • -
  • Kernel PCA
    • -
  • LLE
    • -
  • Other techniques
    • +
  • Setting up the code
    • +
  • Partial Differential Equations
    • +
  • Type of problem
    • +
  • Network requirements
    • +
  • More details
    • +
  • Example: The diffusion equation
    • +
  • Defining the problem
    • +
  • Setting up the network using Autograd
    • +
  • Setting up the network using Autograd; The trial solution
    • +
  • Why the jacobian?
    • +
  • Setting up the network using Autograd; The full program
    • +
  • Example: Solving the wave equation with Neural Networks
    • +
  • The problem to solve for
    • +
  • The trial solution
    • +
  • The analytical solution
    • +
  • Solving the wave equation - the full program using Autograd
    • +
  • Resources on differential equations and deep learning
    • +
  • Friday, Principal Component Analysis
    • +
  • Basic ideas of the Principal Component Analysis (PCA)
    • +
  • Introducing the Covariance and Correlation functions
    • +
  • Correlation Function and Design/Feature Matrix
    • +
  • Covariance Matrix Examples
    • +
  • Correlation Matrix
    • +
  • Correlation Matrix with Pandas
    • +
  • Correlation Matrix with Pandas and the Franke function
    • +
  • Rewriting the Covariance and/or Correlation Matrix
    • +
  • Towards the PCA theorem
    • +
  • The Algorithm before the Theorem
    • +
  • Writing our own PCA code
    • +
  •    Compute the sample mean and center the data
    • +
  •    Compute the sample covariance
    • +
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • +
  • Classical PCA Theorem
    • +
  • Proof of the PCA Theorem
    • +
  • PCA Proof continued
    • +
  • The final step
    • +
  • Geometric Interpretation and link with Singular Value Decomposition
    • +
  • Principal Component Analysis
    • +
  • PCA and scikit-learn
    • +
  • Back to the Cancer Data
    • +
  • More on the PCA
    • +
  • Incremental PCA
    • +
  • Randomized PCA
    • +
  • Kernel PCA
    • +
  • LLE
    • +
  • Other techniques
    • @@ -354,7 +355,7 @@ $$
  • 23
  • 24
  • ...
    • -
  • 76
    • +
  • 77
  • »
    • diff --git a/doc/pub/week43/html/._week43-bs015.html b/doc/pub/week43/html/._week43-bs015.html index 94d3391c3..46cc77e10 100644 --- a/doc/pub/week43/html/._week43-bs015.html +++ b/doc/pub/week43/html/._week43-bs015.html @@ -67,7 +67,7 @@ Automatically generated HTML file from DocOnce source ('Final technicalities II', 2, None, '___sec16'), ('Final technicalities III', 2, None, '___sec17'), ('Final technicalities IV', 2, None, '___sec18'), - ('Backpropagation', 2, None, '___sec19'), + ('Back propagation', 2, None, '___sec19'), ('Gradient descent', 2, None, '___sec20'), ('The code for solving the ODE', 2, None, '___sec21'), ('The network with one input layer, specified number of hidden ' @@ -80,100 +80,100 @@ Automatically generated HTML file from DocOnce source ('The trial solution', 2, None, '___sec25'), ('The program using Autograd', 2, None, '___sec26'), ('Using forward Euler to solve the ODE', 2, None, '___sec27'), - ('Example: Solving the one dimensional Poisson equation using ' - 'Autograd and TensorFlow', + ('Example: Solving the one dimensional Poisson equation', 2, None, '___sec28'), ('The specific equation to solve for', 2, None, '___sec29'), ('Solving the equation using Autograd', 2, None, '___sec30'), ('Comparing with a numerical scheme', 2, None, '___sec31'), - ('Partial Differential Equations', 2, None, '___sec32'), - ('Type of problem', 2, None, '___sec33'), - ('Network requirements', 2, None, '___sec34'), - ('More details', 2, None, '___sec35'), - ('Example: The diffusion equation', 2, None, '___sec36'), - ('Defining the problem', 2, None, '___sec37'), - ('Setting up the network using Autograd', 2, None, '___sec38'), + ('Setting up the code', 2, None, '___sec32'), + ('Partial Differential Equations', 2, None, '___sec33'), + ('Type of problem', 2, None, '___sec34'), + ('Network requirements', 2, None, '___sec35'), + ('More details', 2, None, '___sec36'), + ('Example: The diffusion equation', 2, None, '___sec37'), + ('Defining the problem', 2, None, '___sec38'), + ('Setting up the network using Autograd', 2, None, '___sec39'), ('Setting up the network using Autograd; The trial solution', 2, None, - '___sec39'), - ('Why the jacobian?', 2, None, '___sec40'), + '___sec40'), + ('Why the jacobian?', 2, None, '___sec41'), ('Setting up the network using Autograd; The full program', 2, None, - '___sec41'), + '___sec42'), ('Example: Solving the wave equation with Neural Networks', 2, None, - '___sec42'), - ('The problem to solve for', 2, None, '___sec43'), - ('The trial solution', 2, None, '___sec44'), - ('The analytical solution', 2, None, '___sec45'), + '___sec43'), + ('The problem to solve for', 2, None, '___sec44'), + ('The trial solution', 2, None, '___sec45'), + ('The analytical solution', 2, None, '___sec46'), ('Solving the wave equation - the full program using Autograd', 2, None, - '___sec46'), + '___sec47'), ('Resources on differential equations and deep learning', 2, None, - '___sec47'), - ('Friday, Principal Component Analysis', 2, None, '___sec48'), + '___sec48'), + ('Friday, Principal Component Analysis', 2, None, '___sec49'), ('Basic ideas of the Principal Component Analysis (PCA)', 2, None, - '___sec49'), + '___sec50'), ('Introducing the Covariance and Correlation functions', 2, None, - '___sec50'), + '___sec51'), ('Correlation Function and Design/Feature Matrix', 2, None, - '___sec51'), - ('Covariance Matrix Examples', 2, None, '___sec52'), - ('Correlation Matrix', 2, None, '___sec53'), - ('Correlation Matrix with Pandas', 2, None, '___sec54'), + '___sec52'), + ('Covariance Matrix Examples', 2, None, '___sec53'), + ('Correlation Matrix', 2, None, '___sec54'), + ('Correlation Matrix with Pandas', 2, None, '___sec55'), ('Correlation Matrix with Pandas and the Franke function', 2, None, - '___sec55'), + '___sec56'), ('Rewriting the Covariance and/or Correlation Matrix', 2, None, - '___sec56'), - ('Towards the PCA theorem', 2, None, '___sec57'), - ('The Algorithm before the Theorem', 2, None, '___sec58'), - ('Writing our own PCA code', 2, None, '___sec59'), + '___sec57'), + ('Towards the PCA theorem', 2, None, '___sec58'), + ('The Algorithm before the Theorem', 2, None, '___sec59'), + ('Writing our own PCA code', 2, None, '___sec60'), ('Compute the sample mean and center the data', 3, None, - '___sec60'), - ('Compute the sample covariance', 3, None, '___sec61'), + '___sec61'), + ('Compute the sample covariance', 3, None, '___sec62'), ('Diagonalize the sample covariance matrix to obtain the ' 'principal components', 3, None, - '___sec62'), - ('Classical PCA Theorem', 2, None, '___sec63'), - ('Proof of the PCA Theorem', 2, None, '___sec64'), - ('PCA Proof continued', 2, None, '___sec65'), - ('The final step', 2, None, '___sec66'), + '___sec63'), + ('Classical PCA Theorem', 2, None, '___sec64'), + ('Proof of the PCA Theorem', 2, None, '___sec65'), + ('PCA Proof continued', 2, None, '___sec66'), + ('The final step', 2, None, '___sec67'), ('Geometric Interpretation and link with Singular Value ' 'Decomposition', 2, None, - '___sec67'), - ('Principal Component Analysis', 2, None, '___sec68'), - ('PCA and scikit-learn', 2, None, '___sec69'), - ('Back to the Cancer Data', 2, None, '___sec70'), - ('More on the PCA', 2, None, '___sec71'), - ('Incremental PCA', 2, None, '___sec72'), - ('Randomized PCA', 2, None, '___sec73'), - ('Kernel PCA', 2, None, '___sec74'), - ('LLE', 2, None, '___sec75'), - ('Other techniques', 2, None, '___sec76')]} + '___sec68'), + ('Principal Component Analysis', 2, None, '___sec69'), + ('PCA and scikit-learn', 2, None, '___sec70'), + ('Back to the Cancer Data', 2, None, '___sec71'), + ('More on the PCA', 2, None, '___sec72'), + ('Incremental PCA', 2, None, '___sec73'), + ('Randomized PCA', 2, None, '___sec74'), + ('Kernel PCA', 2, None, '___sec75'), + ('LLE', 2, None, '___sec76'), + ('Other techniques', 2, None, '___sec77')]} end of tocinfo --> @@ -230,7 +230,7 @@ MathJax.Hub.Config({
  • Final technicalities II
  • Final technicalities III
  • Final technicalities IV
    • -
  • Backpropagation
    • +
  • Back propagation
  • Gradient descent
  • The code for solving the ODE
  • The network with one input layer, specified number of hidden layers, and one output layer
    • @@ -239,55 +239,56 @@ MathJax.Hub.Config({
  • The trial solution
  • The program using Autograd
  • Using forward Euler to solve the ODE
    • -
  • Example: Solving the one dimensional Poisson equation using Autograd and TensorFlow
    • +
  • Example: Solving the one dimensional Poisson equation
  • The specific equation to solve for
  • Solving the equation using Autograd
  • Comparing with a numerical scheme
    • -
  • Partial Differential Equations
    • -
  • Type of problem
    • -
  • Network requirements
    • -
  • More details
    • -
  • Example: The diffusion equation
    • -
  • Defining the problem
    • -
  • Setting up the network using Autograd
    • -
  • Setting up the network using Autograd; The trial solution
    • -
  • Why the jacobian?
    • -
  • Setting up the network using Autograd; The full program
    • -
  • Example: Solving the wave equation with Neural Networks
    • -
  • The problem to solve for
    • -
  • The trial solution
    • -
  • The analytical solution
    • -
  • Solving the wave equation - the full program using Autograd
    • -
  • Resources on differential equations and deep learning
    • -
  • Friday, Principal Component Analysis
    • -
  • Basic ideas of the Principal Component Analysis (PCA)
    • -
  • Introducing the Covariance and Correlation functions
    • -
  • Correlation Function and Design/Feature Matrix
    • -
  • Covariance Matrix Examples
    • -
  • Correlation Matrix
    • -
  • Correlation Matrix with Pandas
    • -
  • Correlation Matrix with Pandas and the Franke function
    • -
  • Rewriting the Covariance and/or Correlation Matrix
    • -
  • Towards the PCA theorem
    • -
  • The Algorithm before the Theorem
    • -
  • Writing our own PCA code
    • -
  •    Compute the sample mean and center the data
    • -
  •    Compute the sample covariance
    • -
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • -
  • Classical PCA Theorem
    • -
  • Proof of the PCA Theorem
    • -
  • PCA Proof continued
    • -
  • The final step
    • -
  • Geometric Interpretation and link with Singular Value Decomposition
    • -
  • Principal Component Analysis
    • -
  • PCA and scikit-learn
    • -
  • Back to the Cancer Data
    • -
  • More on the PCA
    • -
  • Incremental PCA
    • -
  • Randomized PCA
    • -
  • Kernel PCA
    • -
  • LLE
    • -
  • Other techniques
    • +
  • Setting up the code
    • +
  • Partial Differential Equations
    • +
  • Type of problem
    • +
  • Network requirements
    • +
  • More details
    • +
  • Example: The diffusion equation
    • +
  • Defining the problem
    • +
  • Setting up the network using Autograd
    • +
  • Setting up the network using Autograd; The trial solution
    • +
  • Why the jacobian?
    • +
  • Setting up the network using Autograd; The full program
    • +
  • Example: Solving the wave equation with Neural Networks
    • +
  • The problem to solve for
    • +
  • The trial solution
    • +
  • The analytical solution
    • +
  • Solving the wave equation - the full program using Autograd
    • +
  • Resources on differential equations and deep learning
    • +
  • Friday, Principal Component Analysis
    • +
  • Basic ideas of the Principal Component Analysis (PCA)
    • +
  • Introducing the Covariance and Correlation functions
    • +
  • Correlation Function and Design/Feature Matrix
    • +
  • Covariance Matrix Examples
    • +
  • Correlation Matrix
    • +
  • Correlation Matrix with Pandas
    • +
  • Correlation Matrix with Pandas and the Franke function
    • +
  • Rewriting the Covariance and/or Correlation Matrix
    • +
  • Towards the PCA theorem
    • +
  • The Algorithm before the Theorem
    • +
  • Writing our own PCA code
    • +
  •    Compute the sample mean and center the data
    • +
  •    Compute the sample covariance
    • +
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • +
  • Classical PCA Theorem
    • +
  • Proof of the PCA Theorem
    • +
  • PCA Proof continued
    • +
  • The final step
    • +
  • Geometric Interpretation and link with Singular Value Decomposition
    • +
  • Principal Component Analysis
    • +
  • PCA and scikit-learn
    • +
  • Back to the Cancer Data
    • +
  • More on the PCA
    • +
  • Incremental PCA
    • +
  • Randomized PCA
    • +
  • Kernel PCA
    • +
  • LLE
    • +
  • Other techniques
    • @@ -339,7 +340,7 @@ The input layer will consist of \( N_{\text{input} } \) neurons, passing its ele
  • 24
  • 25
  • ...
    • -
  • 76
    • +
  • 77
  • »
    • diff --git a/doc/pub/week43/html/._week43-bs016.html b/doc/pub/week43/html/._week43-bs016.html index 5f233c75d..885aa95bb 100644 --- a/doc/pub/week43/html/._week43-bs016.html +++ b/doc/pub/week43/html/._week43-bs016.html @@ -67,7 +67,7 @@ Automatically generated HTML file from DocOnce source ('Final technicalities II', 2, None, '___sec16'), ('Final technicalities III', 2, None, '___sec17'), ('Final technicalities IV', 2, None, '___sec18'), - ('Backpropagation', 2, None, '___sec19'), + ('Back propagation', 2, None, '___sec19'), ('Gradient descent', 2, None, '___sec20'), ('The code for solving the ODE', 2, None, '___sec21'), ('The network with one input layer, specified number of hidden ' @@ -80,100 +80,100 @@ Automatically generated HTML file from DocOnce source ('The trial solution', 2, None, '___sec25'), ('The program using Autograd', 2, None, '___sec26'), ('Using forward Euler to solve the ODE', 2, None, '___sec27'), - ('Example: Solving the one dimensional Poisson equation using ' - 'Autograd and TensorFlow', + ('Example: Solving the one dimensional Poisson equation', 2, None, '___sec28'), ('The specific equation to solve for', 2, None, '___sec29'), ('Solving the equation using Autograd', 2, None, '___sec30'), ('Comparing with a numerical scheme', 2, None, '___sec31'), - ('Partial Differential Equations', 2, None, '___sec32'), - ('Type of problem', 2, None, '___sec33'), - ('Network requirements', 2, None, '___sec34'), - ('More details', 2, None, '___sec35'), - ('Example: The diffusion equation', 2, None, '___sec36'), - ('Defining the problem', 2, None, '___sec37'), - ('Setting up the network using Autograd', 2, None, '___sec38'), + ('Setting up the code', 2, None, '___sec32'), + ('Partial Differential Equations', 2, None, '___sec33'), + ('Type of problem', 2, None, '___sec34'), + ('Network requirements', 2, None, '___sec35'), + ('More details', 2, None, '___sec36'), + ('Example: The diffusion equation', 2, None, '___sec37'), + ('Defining the problem', 2, None, '___sec38'), + ('Setting up the network using Autograd', 2, None, '___sec39'), ('Setting up the network using Autograd; The trial solution', 2, None, - '___sec39'), - ('Why the jacobian?', 2, None, '___sec40'), + '___sec40'), + ('Why the jacobian?', 2, None, '___sec41'), ('Setting up the network using Autograd; The full program', 2, None, - '___sec41'), + '___sec42'), ('Example: Solving the wave equation with Neural Networks', 2, None, - '___sec42'), - ('The problem to solve for', 2, None, '___sec43'), - ('The trial solution', 2, None, '___sec44'), - ('The analytical solution', 2, None, '___sec45'), + '___sec43'), + ('The problem to solve for', 2, None, '___sec44'), + ('The trial solution', 2, None, '___sec45'), + ('The analytical solution', 2, None, '___sec46'), ('Solving the wave equation - the full program using Autograd', 2, None, - '___sec46'), + '___sec47'), ('Resources on differential equations and deep learning', 2, None, - '___sec47'), - ('Friday, Principal Component Analysis', 2, None, '___sec48'), + '___sec48'), + ('Friday, Principal Component Analysis', 2, None, '___sec49'), ('Basic ideas of the Principal Component Analysis (PCA)', 2, None, - '___sec49'), + '___sec50'), ('Introducing the Covariance and Correlation functions', 2, None, - '___sec50'), + '___sec51'), ('Correlation Function and Design/Feature Matrix', 2, None, - '___sec51'), - ('Covariance Matrix Examples', 2, None, '___sec52'), - ('Correlation Matrix', 2, None, '___sec53'), - ('Correlation Matrix with Pandas', 2, None, '___sec54'), + '___sec52'), + ('Covariance Matrix Examples', 2, None, '___sec53'), + ('Correlation Matrix', 2, None, '___sec54'), + ('Correlation Matrix with Pandas', 2, None, '___sec55'), ('Correlation Matrix with Pandas and the Franke function', 2, None, - '___sec55'), + '___sec56'), ('Rewriting the Covariance and/or Correlation Matrix', 2, None, - '___sec56'), - ('Towards the PCA theorem', 2, None, '___sec57'), - ('The Algorithm before the Theorem', 2, None, '___sec58'), - ('Writing our own PCA code', 2, None, '___sec59'), + '___sec57'), + ('Towards the PCA theorem', 2, None, '___sec58'), + ('The Algorithm before the Theorem', 2, None, '___sec59'), + ('Writing our own PCA code', 2, None, '___sec60'), ('Compute the sample mean and center the data', 3, None, - '___sec60'), - ('Compute the sample covariance', 3, None, '___sec61'), + '___sec61'), + ('Compute the sample covariance', 3, None, '___sec62'), ('Diagonalize the sample covariance matrix to obtain the ' 'principal components', 3, None, - '___sec62'), - ('Classical PCA Theorem', 2, None, '___sec63'), - ('Proof of the PCA Theorem', 2, None, '___sec64'), - ('PCA Proof continued', 2, None, '___sec65'), - ('The final step', 2, None, '___sec66'), + '___sec63'), + ('Classical PCA Theorem', 2, None, '___sec64'), + ('Proof of the PCA Theorem', 2, None, '___sec65'), + ('PCA Proof continued', 2, None, '___sec66'), + ('The final step', 2, None, '___sec67'), ('Geometric Interpretation and link with Singular Value ' 'Decomposition', 2, None, - '___sec67'), - ('Principal Component Analysis', 2, None, '___sec68'), - ('PCA and scikit-learn', 2, None, '___sec69'), - ('Back to the Cancer Data', 2, None, '___sec70'), - ('More on the PCA', 2, None, '___sec71'), - ('Incremental PCA', 2, None, '___sec72'), - ('Randomized PCA', 2, None, '___sec73'), - ('Kernel PCA', 2, None, '___sec74'), - ('LLE', 2, None, '___sec75'), - ('Other techniques', 2, None, '___sec76')]} + '___sec68'), + ('Principal Component Analysis', 2, None, '___sec69'), + ('PCA and scikit-learn', 2, None, '___sec70'), + ('Back to the Cancer Data', 2, None, '___sec71'), + ('More on the PCA', 2, None, '___sec72'), + ('Incremental PCA', 2, None, '___sec73'), + ('Randomized PCA', 2, None, '___sec74'), + ('Kernel PCA', 2, None, '___sec75'), + ('LLE', 2, None, '___sec76'), + ('Other techniques', 2, None, '___sec77')]} end of tocinfo --> @@ -230,7 +230,7 @@ MathJax.Hub.Config({
  • Final technicalities II
  • Final technicalities III
  • Final technicalities IV
    • -
  • Backpropagation
    • +
  • Back propagation
  • Gradient descent
  • The code for solving the ODE
  • The network with one input layer, specified number of hidden layers, and one output layer
    • @@ -239,55 +239,56 @@ MathJax.Hub.Config({
  • The trial solution
  • The program using Autograd
  • Using forward Euler to solve the ODE
    • -
  • Example: Solving the one dimensional Poisson equation using Autograd and TensorFlow
    • +
  • Example: Solving the one dimensional Poisson equation
  • The specific equation to solve for
  • Solving the equation using Autograd
  • Comparing with a numerical scheme
    • -
  • Partial Differential Equations
    • -
  • Type of problem
    • -
  • Network requirements
    • -
  • More details
    • -
  • Example: The diffusion equation
    • -
  • Defining the problem
    • -
  • Setting up the network using Autograd
    • -
  • Setting up the network using Autograd; The trial solution
    • -
  • Why the jacobian?
    • -
  • Setting up the network using Autograd; The full program
    • -
  • Example: Solving the wave equation with Neural Networks
    • -
  • The problem to solve for
    • -
  • The trial solution
    • -
  • The analytical solution
    • -
  • Solving the wave equation - the full program using Autograd
    • -
  • Resources on differential equations and deep learning
    • -
  • Friday, Principal Component Analysis
    • -
  • Basic ideas of the Principal Component Analysis (PCA)
    • -
  • Introducing the Covariance and Correlation functions
    • -
  • Correlation Function and Design/Feature Matrix
    • -
  • Covariance Matrix Examples
    • -
  • Correlation Matrix
    • -
  • Correlation Matrix with Pandas
    • -
  • Correlation Matrix with Pandas and the Franke function
    • -
  • Rewriting the Covariance and/or Correlation Matrix
    • -
  • Towards the PCA theorem
    • -
  • The Algorithm before the Theorem
    • -
  • Writing our own PCA code
    • -
  •    Compute the sample mean and center the data
    • -
  •    Compute the sample covariance
    • -
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • -
  • Classical PCA Theorem
    • -
  • Proof of the PCA Theorem
    • -
  • PCA Proof continued
    • -
  • The final step
    • -
  • Geometric Interpretation and link with Singular Value Decomposition
    • -
  • Principal Component Analysis
    • -
  • PCA and scikit-learn
    • -
  • Back to the Cancer Data
    • -
  • More on the PCA
    • -
  • Incremental PCA
    • -
  • Randomized PCA
    • -
  • Kernel PCA
    • -
  • LLE
    • -
  • Other techniques
    • +
  • Setting up the code
    • +
  • Partial Differential Equations
    • +
  • Type of problem
    • +
  • Network requirements
    • +
  • More details
    • +
  • Example: The diffusion equation
    • +
  • Defining the problem
    • +
  • Setting up the network using Autograd
    • +
  • Setting up the network using Autograd; The trial solution
    • +
  • Why the jacobian?
    • +
  • Setting up the network using Autograd; The full program
    • +
  • Example: Solving the wave equation with Neural Networks
    • +
  • The problem to solve for
    • +
  • The trial solution
    • +
  • The analytical solution
    • +
  • Solving the wave equation - the full program using Autograd
    • +
  • Resources on differential equations and deep learning
    • +
  • Friday, Principal Component Analysis
    • +
  • Basic ideas of the Principal Component Analysis (PCA)
    • +
  • Introducing the Covariance and Correlation functions
    • +
  • Correlation Function and Design/Feature Matrix
    • +
  • Covariance Matrix Examples
    • +
  • Correlation Matrix
    • +
  • Correlation Matrix with Pandas
    • +
  • Correlation Matrix with Pandas and the Franke function
    • +
  • Rewriting the Covariance and/or Correlation Matrix
    • +
  • Towards the PCA theorem
    • +
  • The Algorithm before the Theorem
    • +
  • Writing our own PCA code
    • +
  •    Compute the sample mean and center the data
    • +
  •    Compute the sample covariance
    • +
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • +
  • Classical PCA Theorem
    • +
  • Proof of the PCA Theorem
    • +
  • PCA Proof continued
    • +
  • The final step
    • +
  • Geometric Interpretation and link with Singular Value Decomposition
    • +
  • Principal Component Analysis
    • +
  • PCA and scikit-learn
    • +
  • Back to the Cancer Data
    • +
  • More on the PCA
    • +
  • Incremental PCA
    • +
  • Randomized PCA
    • +
  • Kernel PCA
    • +
  • LLE
    • +
  • Other techniques
    • @@ -348,7 +349,7 @@ $$
  • 25
  • 26
  • ...
    • -
  • 76
    • +
  • 77
  • »
    • diff --git a/doc/pub/week43/html/._week43-bs017.html b/doc/pub/week43/html/._week43-bs017.html index 1fa12636a..0dd5c4161 100644 --- a/doc/pub/week43/html/._week43-bs017.html +++ b/doc/pub/week43/html/._week43-bs017.html @@ -67,7 +67,7 @@ Automatically generated HTML file from DocOnce source ('Final technicalities II', 2, None, '___sec16'), ('Final technicalities III', 2, None, '___sec17'), ('Final technicalities IV', 2, None, '___sec18'), - ('Backpropagation', 2, None, '___sec19'), + ('Back propagation', 2, None, '___sec19'), ('Gradient descent', 2, None, '___sec20'), ('The code for solving the ODE', 2, None, '___sec21'), ('The network with one input layer, specified number of hidden ' @@ -80,100 +80,100 @@ Automatically generated HTML file from DocOnce source ('The trial solution', 2, None, '___sec25'), ('The program using Autograd', 2, None, '___sec26'), ('Using forward Euler to solve the ODE', 2, None, '___sec27'), - ('Example: Solving the one dimensional Poisson equation using ' - 'Autograd and TensorFlow', + ('Example: Solving the one dimensional Poisson equation', 2, None, '___sec28'), ('The specific equation to solve for', 2, None, '___sec29'), ('Solving the equation using Autograd', 2, None, '___sec30'), ('Comparing with a numerical scheme', 2, None, '___sec31'), - ('Partial Differential Equations', 2, None, '___sec32'), - ('Type of problem', 2, None, '___sec33'), - ('Network requirements', 2, None, '___sec34'), - ('More details', 2, None, '___sec35'), - ('Example: The diffusion equation', 2, None, '___sec36'), - ('Defining the problem', 2, None, '___sec37'), - ('Setting up the network using Autograd', 2, None, '___sec38'), + ('Setting up the code', 2, None, '___sec32'), + ('Partial Differential Equations', 2, None, '___sec33'), + ('Type of problem', 2, None, '___sec34'), + ('Network requirements', 2, None, '___sec35'), + ('More details', 2, None, '___sec36'), + ('Example: The diffusion equation', 2, None, '___sec37'), + ('Defining the problem', 2, None, '___sec38'), + ('Setting up the network using Autograd', 2, None, '___sec39'), ('Setting up the network using Autograd; The trial solution', 2, None, - '___sec39'), - ('Why the jacobian?', 2, None, '___sec40'), + '___sec40'), + ('Why the jacobian?', 2, None, '___sec41'), ('Setting up the network using Autograd; The full program', 2, None, - '___sec41'), + '___sec42'), ('Example: Solving the wave equation with Neural Networks', 2, None, - '___sec42'), - ('The problem to solve for', 2, None, '___sec43'), - ('The trial solution', 2, None, '___sec44'), - ('The analytical solution', 2, None, '___sec45'), + '___sec43'), + ('The problem to solve for', 2, None, '___sec44'), + ('The trial solution', 2, None, '___sec45'), + ('The analytical solution', 2, None, '___sec46'), ('Solving the wave equation - the full program using Autograd', 2, None, - '___sec46'), + '___sec47'), ('Resources on differential equations and deep learning', 2, None, - '___sec47'), - ('Friday, Principal Component Analysis', 2, None, '___sec48'), + '___sec48'), + ('Friday, Principal Component Analysis', 2, None, '___sec49'), ('Basic ideas of the Principal Component Analysis (PCA)', 2, None, - '___sec49'), + '___sec50'), ('Introducing the Covariance and Correlation functions', 2, None, - '___sec50'), + '___sec51'), ('Correlation Function and Design/Feature Matrix', 2, None, - '___sec51'), - ('Covariance Matrix Examples', 2, None, '___sec52'), - ('Correlation Matrix', 2, None, '___sec53'), - ('Correlation Matrix with Pandas', 2, None, '___sec54'), + '___sec52'), + ('Covariance Matrix Examples', 2, None, '___sec53'), + ('Correlation Matrix', 2, None, '___sec54'), + ('Correlation Matrix with Pandas', 2, None, '___sec55'), ('Correlation Matrix with Pandas and the Franke function', 2, None, - '___sec55'), + '___sec56'), ('Rewriting the Covariance and/or Correlation Matrix', 2, None, - '___sec56'), - ('Towards the PCA theorem', 2, None, '___sec57'), - ('The Algorithm before the Theorem', 2, None, '___sec58'), - ('Writing our own PCA code', 2, None, '___sec59'), + '___sec57'), + ('Towards the PCA theorem', 2, None, '___sec58'), + ('The Algorithm before the Theorem', 2, None, '___sec59'), + ('Writing our own PCA code', 2, None, '___sec60'), ('Compute the sample mean and center the data', 3, None, - '___sec60'), - ('Compute the sample covariance', 3, None, '___sec61'), + '___sec61'), + ('Compute the sample covariance', 3, None, '___sec62'), ('Diagonalize the sample covariance matrix to obtain the ' 'principal components', 3, None, - '___sec62'), - ('Classical PCA Theorem', 2, None, '___sec63'), - ('Proof of the PCA Theorem', 2, None, '___sec64'), - ('PCA Proof continued', 2, None, '___sec65'), - ('The final step', 2, None, '___sec66'), + '___sec63'), + ('Classical PCA Theorem', 2, None, '___sec64'), + ('Proof of the PCA Theorem', 2, None, '___sec65'), + ('PCA Proof continued', 2, None, '___sec66'), + ('The final step', 2, None, '___sec67'), ('Geometric Interpretation and link with Singular Value ' 'Decomposition', 2, None, - '___sec67'), - ('Principal Component Analysis', 2, None, '___sec68'), - ('PCA and scikit-learn', 2, None, '___sec69'), - ('Back to the Cancer Data', 2, None, '___sec70'), - ('More on the PCA', 2, None, '___sec71'), - ('Incremental PCA', 2, None, '___sec72'), - ('Randomized PCA', 2, None, '___sec73'), - ('Kernel PCA', 2, None, '___sec74'), - ('LLE', 2, None, '___sec75'), - ('Other techniques', 2, None, '___sec76')]} + '___sec68'), + ('Principal Component Analysis', 2, None, '___sec69'), + ('PCA and scikit-learn', 2, None, '___sec70'), + ('Back to the Cancer Data', 2, None, '___sec71'), + ('More on the PCA', 2, None, '___sec72'), + ('Incremental PCA', 2, None, '___sec73'), + ('Randomized PCA', 2, None, '___sec74'), + ('Kernel PCA', 2, None, '___sec75'), + ('LLE', 2, None, '___sec76'), + ('Other techniques', 2, None, '___sec77')]} end of tocinfo --> @@ -230,7 +230,7 @@ MathJax.Hub.Config({
  • Final technicalities II
  • Final technicalities III
  • Final technicalities IV
    • -
  • Backpropagation
    • +
  • Back propagation
  • Gradient descent
  • The code for solving the ODE
  • The network with one input layer, specified number of hidden layers, and one output layer
    • @@ -239,55 +239,56 @@ MathJax.Hub.Config({
  • The trial solution
  • The program using Autograd
  • Using forward Euler to solve the ODE
    • -
  • Example: Solving the one dimensional Poisson equation using Autograd and TensorFlow
    • +
  • Example: Solving the one dimensional Poisson equation
  • The specific equation to solve for
  • Solving the equation using Autograd
  • Comparing with a numerical scheme
    • -
  • Partial Differential Equations
    • -
  • Type of problem
    • -
  • Network requirements
    • -
  • More details
    • -
  • Example: The diffusion equation
    • -
  • Defining the problem
    • -
  • Setting up the network using Autograd
    • -
  • Setting up the network using Autograd; The trial solution
    • -
  • Why the jacobian?
    • -
  • Setting up the network using Autograd; The full program
    • -
  • Example: Solving the wave equation with Neural Networks
    • -
  • The problem to solve for
    • -
  • The trial solution
    • -
  • The analytical solution
    • -
  • Solving the wave equation - the full program using Autograd
    • -
  • Resources on differential equations and deep learning
    • -
  • Friday, Principal Component Analysis
    • -
  • Basic ideas of the Principal Component Analysis (PCA)
    • -
  • Introducing the Covariance and Correlation functions
    • -
  • Correlation Function and Design/Feature Matrix
    • -
  • Covariance Matrix Examples
    • -
  • Correlation Matrix
    • -
  • Correlation Matrix with Pandas
    • -
  • Correlation Matrix with Pandas and the Franke function
    • -
  • Rewriting the Covariance and/or Correlation Matrix
    • -
  • Towards the PCA theorem
    • -
  • The Algorithm before the Theorem
    • -
  • Writing our own PCA code
    • -
  •    Compute the sample mean and center the data
    • -
  •    Compute the sample covariance
    • -
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • -
  • Classical PCA Theorem
    • -
  • Proof of the PCA Theorem
    • -
  • PCA Proof continued
    • -
  • The final step
    • -
  • Geometric Interpretation and link with Singular Value Decomposition
    • -
  • Principal Component Analysis
    • -
  • PCA and scikit-learn
    • -
  • Back to the Cancer Data
    • -
  • More on the PCA
    • -
  • Incremental PCA
    • -
  • Randomized PCA
    • -
  • Kernel PCA
    • -
  • LLE
    • -
  • Other techniques
    • +
  • Setting up the code
    • +
  • Partial Differential Equations
    • +
  • Type of problem
    • +
  • Network requirements
    • +
  • More details
    • +
  • Example: The diffusion equation
    • +
  • Defining the problem
    • +
  • Setting up the network using Autograd
    • +
  • Setting up the network using Autograd; The trial solution
    • +
  • Why the jacobian?
    • +
  • Setting up the network using Autograd; The full program
    • +
  • Example: Solving the wave equation with Neural Networks
    • +
  • The problem to solve for
    • +
  • The trial solution
    • +
  • The analytical solution
    • +
  • Solving the wave equation - the full program using Autograd
    • +
  • Resources on differential equations and deep learning
    • +
  • Friday, Principal Component Analysis
    • +
  • Basic ideas of the Principal Component Analysis (PCA)
    • +
  • Introducing the Covariance and Correlation functions
    • +
  • Correlation Function and Design/Feature Matrix
    • +
  • Covariance Matrix Examples
    • +
  • Correlation Matrix
    • +
  • Correlation Matrix with Pandas
    • +
  • Correlation Matrix with Pandas and the Franke function
    • +
  • Rewriting the Covariance and/or Correlation Matrix
    • +
  • Towards the PCA theorem
    • +
  • The Algorithm before the Theorem
    • +
  • Writing our own PCA code
    • +
  •    Compute the sample mean and center the data
    • +
  •    Compute the sample covariance
    • +
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • +
  • Classical PCA Theorem
    • +
  • Proof of the PCA Theorem
    • +
  • PCA Proof continued
    • +
  • The final step
    • +
  • Geometric Interpretation and link with Singular Value Decomposition
    • +
  • Principal Component Analysis
    • +
  • PCA and scikit-learn
    • +
  • Back to the Cancer Data
    • +
  • More on the PCA
    • +
  • Incremental PCA
    • +
  • Randomized PCA
    • +
  • Kernel PCA
    • +
  • LLE
    • +
  • Other techniques
    • @@ -349,7 +350,7 @@ $$
  • 26
  • 27
  • ...
    • -
  • 76
    • +
  • 77
  • »
    • diff --git a/doc/pub/week43/html/._week43-bs018.html b/doc/pub/week43/html/._week43-bs018.html index 4345d7abf..f68d4ab27 100644 --- a/doc/pub/week43/html/._week43-bs018.html +++ b/doc/pub/week43/html/._week43-bs018.html @@ -67,7 +67,7 @@ Automatically generated HTML file from DocOnce source ('Final technicalities II', 2, None, '___sec16'), ('Final technicalities III', 2, None, '___sec17'), ('Final technicalities IV', 2, None, '___sec18'), - ('Backpropagation', 2, None, '___sec19'), + ('Back propagation', 2, None, '___sec19'), ('Gradient descent', 2, None, '___sec20'), ('The code for solving the ODE', 2, None, '___sec21'), ('The network with one input layer, specified number of hidden ' @@ -80,100 +80,100 @@ Automatically generated HTML file from DocOnce source ('The trial solution', 2, None, '___sec25'), ('The program using Autograd', 2, None, '___sec26'), ('Using forward Euler to solve the ODE', 2, None, '___sec27'), - ('Example: Solving the one dimensional Poisson equation using ' - 'Autograd and TensorFlow', + ('Example: Solving the one dimensional Poisson equation', 2, None, '___sec28'), ('The specific equation to solve for', 2, None, '___sec29'), ('Solving the equation using Autograd', 2, None, '___sec30'), ('Comparing with a numerical scheme', 2, None, '___sec31'), - ('Partial Differential Equations', 2, None, '___sec32'), - ('Type of problem', 2, None, '___sec33'), - ('Network requirements', 2, None, '___sec34'), - ('More details', 2, None, '___sec35'), - ('Example: The diffusion equation', 2, None, '___sec36'), - ('Defining the problem', 2, None, '___sec37'), - ('Setting up the network using Autograd', 2, None, '___sec38'), + ('Setting up the code', 2, None, '___sec32'), + ('Partial Differential Equations', 2, None, '___sec33'), + ('Type of problem', 2, None, '___sec34'), + ('Network requirements', 2, None, '___sec35'), + ('More details', 2, None, '___sec36'), + ('Example: The diffusion equation', 2, None, '___sec37'), + ('Defining the problem', 2, None, '___sec38'), + ('Setting up the network using Autograd', 2, None, '___sec39'), ('Setting up the network using Autograd; The trial solution', 2, None, - '___sec39'), - ('Why the jacobian?', 2, None, '___sec40'), + '___sec40'), + ('Why the jacobian?', 2, None, '___sec41'), ('Setting up the network using Autograd; The full program', 2, None, - '___sec41'), + '___sec42'), ('Example: Solving the wave equation with Neural Networks', 2, None, - '___sec42'), - ('The problem to solve for', 2, None, '___sec43'), - ('The trial solution', 2, None, '___sec44'), - ('The analytical solution', 2, None, '___sec45'), + '___sec43'), + ('The problem to solve for', 2, None, '___sec44'), + ('The trial solution', 2, None, '___sec45'), + ('The analytical solution', 2, None, '___sec46'), ('Solving the wave equation - the full program using Autograd', 2, None, - '___sec46'), + '___sec47'), ('Resources on differential equations and deep learning', 2, None, - '___sec47'), - ('Friday, Principal Component Analysis', 2, None, '___sec48'), + '___sec48'), + ('Friday, Principal Component Analysis', 2, None, '___sec49'), ('Basic ideas of the Principal Component Analysis (PCA)', 2, None, - '___sec49'), + '___sec50'), ('Introducing the Covariance and Correlation functions', 2, None, - '___sec50'), + '___sec51'), ('Correlation Function and Design/Feature Matrix', 2, None, - '___sec51'), - ('Covariance Matrix Examples', 2, None, '___sec52'), - ('Correlation Matrix', 2, None, '___sec53'), - ('Correlation Matrix with Pandas', 2, None, '___sec54'), + '___sec52'), + ('Covariance Matrix Examples', 2, None, '___sec53'), + ('Correlation Matrix', 2, None, '___sec54'), + ('Correlation Matrix with Pandas', 2, None, '___sec55'), ('Correlation Matrix with Pandas and the Franke function', 2, None, - '___sec55'), + '___sec56'), ('Rewriting the Covariance and/or Correlation Matrix', 2, None, - '___sec56'), - ('Towards the PCA theorem', 2, None, '___sec57'), - ('The Algorithm before the Theorem', 2, None, '___sec58'), - ('Writing our own PCA code', 2, None, '___sec59'), + '___sec57'), + ('Towards the PCA theorem', 2, None, '___sec58'), + ('The Algorithm before the Theorem', 2, None, '___sec59'), + ('Writing our own PCA code', 2, None, '___sec60'), ('Compute the sample mean and center the data', 3, None, - '___sec60'), - ('Compute the sample covariance', 3, None, '___sec61'), + '___sec61'), + ('Compute the sample covariance', 3, None, '___sec62'), ('Diagonalize the sample covariance matrix to obtain the ' 'principal components', 3, None, - '___sec62'), - ('Classical PCA Theorem', 2, None, '___sec63'), - ('Proof of the PCA Theorem', 2, None, '___sec64'), - ('PCA Proof continued', 2, None, '___sec65'), - ('The final step', 2, None, '___sec66'), + '___sec63'), + ('Classical PCA Theorem', 2, None, '___sec64'), + ('Proof of the PCA Theorem', 2, None, '___sec65'), + ('PCA Proof continued', 2, None, '___sec66'), + ('The final step', 2, None, '___sec67'), ('Geometric Interpretation and link with Singular Value ' 'Decomposition', 2, None, - '___sec67'), - ('Principal Component Analysis', 2, None, '___sec68'), - ('PCA and scikit-learn', 2, None, '___sec69'), - ('Back to the Cancer Data', 2, None, '___sec70'), - ('More on the PCA', 2, None, '___sec71'), - ('Incremental PCA', 2, None, '___sec72'), - ('Randomized PCA', 2, None, '___sec73'), - ('Kernel PCA', 2, None, '___sec74'), - ('LLE', 2, None, '___sec75'), - ('Other techniques', 2, None, '___sec76')]} + '___sec68'), + ('Principal Component Analysis', 2, None, '___sec69'), + ('PCA and scikit-learn', 2, None, '___sec70'), + ('Back to the Cancer Data', 2, None, '___sec71'), + ('More on the PCA', 2, None, '___sec72'), + ('Incremental PCA', 2, None, '___sec73'), + ('Randomized PCA', 2, None, '___sec74'), + ('Kernel PCA', 2, None, '___sec75'), + ('LLE', 2, None, '___sec76'), + ('Other techniques', 2, None, '___sec77')]} end of tocinfo --> @@ -230,7 +230,7 @@ MathJax.Hub.Config({
  • Final technicalities II
  • Final technicalities III
  • Final technicalities IV
    • -
  • Backpropagation
    • +
  • Back propagation
  • Gradient descent
  • The code for solving the ODE
  • The network with one input layer, specified number of hidden layers, and one output layer
    • @@ -239,55 +239,56 @@ MathJax.Hub.Config({
  • The trial solution
  • The program using Autograd
  • Using forward Euler to solve the ODE
    • -
  • Example: Solving the one dimensional Poisson equation using Autograd and TensorFlow
    • +
  • Example: Solving the one dimensional Poisson equation
  • The specific equation to solve for
  • Solving the equation using Autograd
  • Comparing with a numerical scheme
    • -
  • Partial Differential Equations
    • -
  • Type of problem
    • -
  • Network requirements
    • -
  • More details
    • -
  • Example: The diffusion equation
    • -
  • Defining the problem
    • -
  • Setting up the network using Autograd
    • -
  • Setting up the network using Autograd; The trial solution
    • -
  • Why the jacobian?
    • -
  • Setting up the network using Autograd; The full program
    • -
  • Example: Solving the wave equation with Neural Networks
    • -
  • The problem to solve for
    • -
  • The trial solution
    • -
  • The analytical solution
    • -
  • Solving the wave equation - the full program using Autograd
    • -
  • Resources on differential equations and deep learning
    • -
  • Friday, Principal Component Analysis
    • -
  • Basic ideas of the Principal Component Analysis (PCA)
    • -
  • Introducing the Covariance and Correlation functions
    • -
  • Correlation Function and Design/Feature Matrix
    • -
  • Covariance Matrix Examples
    • -
  • Correlation Matrix
    • -
  • Correlation Matrix with Pandas
    • -
  • Correlation Matrix with Pandas and the Franke function
    • -
  • Rewriting the Covariance and/or Correlation Matrix
    • -
  • Towards the PCA theorem
    • -
  • The Algorithm before the Theorem
    • -
  • Writing our own PCA code
    • -
  •    Compute the sample mean and center the data
    • -
  •    Compute the sample covariance
    • -
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • -
  • Classical PCA Theorem
    • -
  • Proof of the PCA Theorem
    • -
  • PCA Proof continued
    • -
  • The final step
    • -
  • Geometric Interpretation and link with Singular Value Decomposition
    • -
  • Principal Component Analysis
    • -
  • PCA and scikit-learn
    • -
  • Back to the Cancer Data
    • -
  • More on the PCA
    • -
  • Incremental PCA
    • -
  • Randomized PCA
    • -
  • Kernel PCA
    • -
  • LLE
    • -
  • Other techniques
    • +
  • Setting up the code
    • +
  • Partial Differential Equations
    • +
  • Type of problem
    • +
  • Network requirements
    • +
  • More details
    • +
  • Example: The diffusion equation
    • +
  • Defining the problem
    • +
  • Setting up the network using Autograd
    • +
  • Setting up the network using Autograd; The trial solution
    • +
  • Why the jacobian?
    • +
  • Setting up the network using Autograd; The full program
    • +
  • Example: Solving the wave equation with Neural Networks
    • +
  • The problem to solve for
    • +
  • The trial solution
    • +
  • The analytical solution
    • +
  • Solving the wave equation - the full program using Autograd
    • +
  • Resources on differential equations and deep learning
    • +
  • Friday, Principal Component Analysis
    • +
  • Basic ideas of the Principal Component Analysis (PCA)
    • +
  • Introducing the Covariance and Correlation functions
    • +
  • Correlation Function and Design/Feature Matrix
    • +
  • Covariance Matrix Examples
    • +
  • Correlation Matrix
    • +
  • Correlation Matrix with Pandas
    • +
  • Correlation Matrix with Pandas and the Franke function
    • +
  • Rewriting the Covariance and/or Correlation Matrix
    • +
  • Towards the PCA theorem
    • +
  • The Algorithm before the Theorem
    • +
  • Writing our own PCA code
    • +
  •    Compute the sample mean and center the data
    • +
  •    Compute the sample covariance
    • +
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • +
  • Classical PCA Theorem
    • +
  • Proof of the PCA Theorem
    • +
  • PCA Proof continued
    • +
  • The final step
    • +
  • Geometric Interpretation and link with Singular Value Decomposition
    • +
  • Principal Component Analysis
    • +
  • PCA and scikit-learn
    • +
  • Back to the Cancer Data
    • +
  • More on the PCA
    • +
  • Incremental PCA
    • +
  • Randomized PCA
    • +
  • Kernel PCA
    • +
  • LLE
    • +
  • Other techniques
    • @@ -322,7 +323,7 @@ $$ It is possible to use other activations functions for the hidden layer also.

    -The output $\boldsymbol{x}_i^{\text{hidden} }$from each \( i \)-th hidden neuron is: +The output \( \boldsymbol{x}_i^{\text{hidden}} \) from each \( i \)-th hidden neuron is: $$ \boldsymbol{x}_i^{\text{hidden} } = f\big( \boldsymbol{z}_{i}^{\text{hidden}} \big) @@ -332,7 +333,11 @@ $$ The outputs \( \boldsymbol{x}_i^{\text{hidden} } \) are then sent to the output layer.

    -The output layer consists of one neuron in this case, and combines the output from each of the neurons in the hidden layers. The output layer combines the results from the hidden layer using some weights $ w_i^{\text{output}}$ and biases \( b_i^{\text{output}} \). In this case, it is assumes that the number of neurons in the output layer is one. +The output layer consists of one neuron in this case, and combines the +output from each of the neurons in the hidden layers. The output layer +combines the results from the hidden layer using some weights \( w_i^{\text{output}} \) +and biases \( b_i^{\text{output}} \). In this case, +it is assumes that the number of neurons in the output layer is one.

    @@ -360,7 +365,7 @@ The output layer consists of one neuron in this case, and combines the output fr

  • 27
  • 28
  • ...
    • -
  • 76
    • +
  • 77
  • »
    • diff --git a/doc/pub/week43/html/._week43-bs019.html b/doc/pub/week43/html/._week43-bs019.html index 4c92be862..5c42f6b3e 100644 --- a/doc/pub/week43/html/._week43-bs019.html +++ b/doc/pub/week43/html/._week43-bs019.html @@ -67,7 +67,7 @@ Automatically generated HTML file from DocOnce source ('Final technicalities II', 2, None, '___sec16'), ('Final technicalities III', 2, None, '___sec17'), ('Final technicalities IV', 2, None, '___sec18'), - ('Backpropagation', 2, None, '___sec19'), + ('Back propagation', 2, None, '___sec19'), ('Gradient descent', 2, None, '___sec20'), ('The code for solving the ODE', 2, None, '___sec21'), ('The network with one input layer, specified number of hidden ' @@ -80,100 +80,100 @@ Automatically generated HTML file from DocOnce source ('The trial solution', 2, None, '___sec25'), ('The program using Autograd', 2, None, '___sec26'), ('Using forward Euler to solve the ODE', 2, None, '___sec27'), - ('Example: Solving the one dimensional Poisson equation using ' - 'Autograd and TensorFlow', + ('Example: Solving the one dimensional Poisson equation', 2, None, '___sec28'), ('The specific equation to solve for', 2, None, '___sec29'), ('Solving the equation using Autograd', 2, None, '___sec30'), ('Comparing with a numerical scheme', 2, None, '___sec31'), - ('Partial Differential Equations', 2, None, '___sec32'), - ('Type of problem', 2, None, '___sec33'), - ('Network requirements', 2, None, '___sec34'), - ('More details', 2, None, '___sec35'), - ('Example: The diffusion equation', 2, None, '___sec36'), - ('Defining the problem', 2, None, '___sec37'), - ('Setting up the network using Autograd', 2, None, '___sec38'), + ('Setting up the code', 2, None, '___sec32'), + ('Partial Differential Equations', 2, None, '___sec33'), + ('Type of problem', 2, None, '___sec34'), + ('Network requirements', 2, None, '___sec35'), + ('More details', 2, None, '___sec36'), + ('Example: The diffusion equation', 2, None, '___sec37'), + ('Defining the problem', 2, None, '___sec38'), + ('Setting up the network using Autograd', 2, None, '___sec39'), ('Setting up the network using Autograd; The trial solution', 2, None, - '___sec39'), - ('Why the jacobian?', 2, None, '___sec40'), + '___sec40'), + ('Why the jacobian?', 2, None, '___sec41'), ('Setting up the network using Autograd; The full program', 2, None, - '___sec41'), + '___sec42'), ('Example: Solving the wave equation with Neural Networks', 2, None, - '___sec42'), - ('The problem to solve for', 2, None, '___sec43'), - ('The trial solution', 2, None, '___sec44'), - ('The analytical solution', 2, None, '___sec45'), + '___sec43'), + ('The problem to solve for', 2, None, '___sec44'), + ('The trial solution', 2, None, '___sec45'), + ('The analytical solution', 2, None, '___sec46'), ('Solving the wave equation - the full program using Autograd', 2, None, - '___sec46'), + '___sec47'), ('Resources on differential equations and deep learning', 2, None, - '___sec47'), - ('Friday, Principal Component Analysis', 2, None, '___sec48'), + '___sec48'), + ('Friday, Principal Component Analysis', 2, None, '___sec49'), ('Basic ideas of the Principal Component Analysis (PCA)', 2, None, - '___sec49'), + '___sec50'), ('Introducing the Covariance and Correlation functions', 2, None, - '___sec50'), + '___sec51'), ('Correlation Function and Design/Feature Matrix', 2, None, - '___sec51'), - ('Covariance Matrix Examples', 2, None, '___sec52'), - ('Correlation Matrix', 2, None, '___sec53'), - ('Correlation Matrix with Pandas', 2, None, '___sec54'), + '___sec52'), + ('Covariance Matrix Examples', 2, None, '___sec53'), + ('Correlation Matrix', 2, None, '___sec54'), + ('Correlation Matrix with Pandas', 2, None, '___sec55'), ('Correlation Matrix with Pandas and the Franke function', 2, None, - '___sec55'), + '___sec56'), ('Rewriting the Covariance and/or Correlation Matrix', 2, None, - '___sec56'), - ('Towards the PCA theorem', 2, None, '___sec57'), - ('The Algorithm before the Theorem', 2, None, '___sec58'), - ('Writing our own PCA code', 2, None, '___sec59'), + '___sec57'), + ('Towards the PCA theorem', 2, None, '___sec58'), + ('The Algorithm before the Theorem', 2, None, '___sec59'), + ('Writing our own PCA code', 2, None, '___sec60'), ('Compute the sample mean and center the data', 3, None, - '___sec60'), - ('Compute the sample covariance', 3, None, '___sec61'), + '___sec61'), + ('Compute the sample covariance', 3, None, '___sec62'), ('Diagonalize the sample covariance matrix to obtain the ' 'principal components', 3, None, - '___sec62'), - ('Classical PCA Theorem', 2, None, '___sec63'), - ('Proof of the PCA Theorem', 2, None, '___sec64'), - ('PCA Proof continued', 2, None, '___sec65'), - ('The final step', 2, None, '___sec66'), + '___sec63'), + ('Classical PCA Theorem', 2, None, '___sec64'), + ('Proof of the PCA Theorem', 2, None, '___sec65'), + ('PCA Proof continued', 2, None, '___sec66'), + ('The final step', 2, None, '___sec67'), ('Geometric Interpretation and link with Singular Value ' 'Decomposition', 2, None, - '___sec67'), - ('Principal Component Analysis', 2, None, '___sec68'), - ('PCA and scikit-learn', 2, None, '___sec69'), - ('Back to the Cancer Data', 2, None, '___sec70'), - ('More on the PCA', 2, None, '___sec71'), - ('Incremental PCA', 2, None, '___sec72'), - ('Randomized PCA', 2, None, '___sec73'), - ('Kernel PCA', 2, None, '___sec74'), - ('LLE', 2, None, '___sec75'), - ('Other techniques', 2, None, '___sec76')]} + '___sec68'), + ('Principal Component Analysis', 2, None, '___sec69'), + ('PCA and scikit-learn', 2, None, '___sec70'), + ('Back to the Cancer Data', 2, None, '___sec71'), + ('More on the PCA', 2, None, '___sec72'), + ('Incremental PCA', 2, None, '___sec73'), + ('Randomized PCA', 2, None, '___sec74'), + ('Kernel PCA', 2, None, '___sec75'), + ('LLE', 2, None, '___sec76'), + ('Other techniques', 2, None, '___sec77')]} end of tocinfo --> @@ -230,7 +230,7 @@ MathJax.Hub.Config({
  • Final technicalities II
  • Final technicalities III
  • Final technicalities IV
    • -
  • Backpropagation
    • +
  • Back propagation
  • Gradient descent
  • The code for solving the ODE
  • The network with one input layer, specified number of hidden layers, and one output layer
    • @@ -239,55 +239,56 @@ MathJax.Hub.Config({
  • The trial solution
  • The program using Autograd
  • Using forward Euler to solve the ODE
    • -
  • Example: Solving the one dimensional Poisson equation using Autograd and TensorFlow
    • +
  • Example: Solving the one dimensional Poisson equation
  • The specific equation to solve for
  • Solving the equation using Autograd
  • Comparing with a numerical scheme
    • -
  • Partial Differential Equations
    • -
  • Type of problem
    • -
  • Network requirements
    • -
  • More details
    • -
  • Example: The diffusion equation
    • -
  • Defining the problem
    • -
  • Setting up the network using Autograd
    • -
  • Setting up the network using Autograd; The trial solution
    • -
  • Why the jacobian?
    • -
  • Setting up the network using Autograd; The full program
    • -
  • Example: Solving the wave equation with Neural Networks
    • -
  • The problem to solve for
    • -
  • The trial solution
    • -
  • The analytical solution
    • -
  • Solving the wave equation - the full program using Autograd
    • -
  • Resources on differential equations and deep learning
    • -
  • Friday, Principal Component Analysis
    • -
  • Basic ideas of the Principal Component Analysis (PCA)
    • -
  • Introducing the Covariance and Correlation functions
    • -
  • Correlation Function and Design/Feature Matrix
    • -
  • Covariance Matrix Examples
    • -
  • Correlation Matrix
    • -
  • Correlation Matrix with Pandas
    • -
  • Correlation Matrix with Pandas and the Franke function
    • -
  • Rewriting the Covariance and/or Correlation Matrix
    • -
  • Towards the PCA theorem
    • -
  • The Algorithm before the Theorem
    • -
  • Writing our own PCA code
    • -
  •    Compute the sample mean and center the data
    • -
  •    Compute the sample covariance
    • -
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • -
  • Classical PCA Theorem
    • -
  • Proof of the PCA Theorem
    • -
  • PCA Proof continued
    • -
  • The final step
    • -
  • Geometric Interpretation and link with Singular Value Decomposition
    • -
  • Principal Component Analysis
    • -
  • PCA and scikit-learn
    • -
  • Back to the Cancer Data
    • -
  • More on the PCA
    • -
  • Incremental PCA
    • -
  • Randomized PCA
    • -
  • Kernel PCA
    • -
  • LLE
    • -
  • Other techniques
    • +
  • Setting up the code
    • +
  • Partial Differential Equations
    • +
  • Type of problem
    • +
  • Network requirements
    • +
  • More details
    • +
  • Example: The diffusion equation
    • +
  • Defining the problem
    • +
  • Setting up the network using Autograd
    • +
  • Setting up the network using Autograd; The trial solution
    • +
  • Why the jacobian?
    • +
  • Setting up the network using Autograd; The full program
    • +
  • Example: Solving the wave equation with Neural Networks
    • +
  • The problem to solve for
    • +
  • The trial solution
    • +
  • The analytical solution
    • +
  • Solving the wave equation - the full program using Autograd
    • +
  • Resources on differential equations and deep learning
    • +
  • Friday, Principal Component Analysis
    • +
  • Basic ideas of the Principal Component Analysis (PCA)
    • +
  • Introducing the Covariance and Correlation functions
    • +
  • Correlation Function and Design/Feature Matrix
    • +
  • Covariance Matrix Examples
    • +
  • Correlation Matrix
    • +
  • Correlation Matrix with Pandas
    • +
  • Correlation Matrix with Pandas and the Franke function
    • +
  • Rewriting the Covariance and/or Correlation Matrix
    • +
  • Towards the PCA theorem
    • +
  • The Algorithm before the Theorem
    • +
  • Writing our own PCA code
    • +
  •    Compute the sample mean and center the data
    • +
  •    Compute the sample covariance
    • +
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • +
  • Classical PCA Theorem
    • +
  • Proof of the PCA Theorem
    • +
  • PCA Proof continued
    • +
  • The final step
    • +
  • Geometric Interpretation and link with Singular Value Decomposition
    • +
  • Principal Component Analysis
    • +
  • PCA and scikit-learn
    • +
  • Back to the Cancer Data
    • +
  • More on the PCA
    • +
  • Incremental PCA
    • +
  • Randomized PCA
    • +
  • Kernel PCA
    • +
  • LLE
    • +
  • Other techniques
    • @@ -347,7 +348,7 @@ $$
  • 28
  • 29
  • ...
    • -
  • 76
    • +
  • 77
  • »
    • diff --git a/doc/pub/week43/html/._week43-bs020.html b/doc/pub/week43/html/._week43-bs020.html index 509bafc33..3fd6c690c 100644 --- a/doc/pub/week43/html/._week43-bs020.html +++ b/doc/pub/week43/html/._week43-bs020.html @@ -67,7 +67,7 @@ Automatically generated HTML file from DocOnce source ('Final technicalities II', 2, None, '___sec16'), ('Final technicalities III', 2, None, '___sec17'), ('Final technicalities IV', 2, None, '___sec18'), - ('Backpropagation', 2, None, '___sec19'), + ('Back propagation', 2, None, '___sec19'), ('Gradient descent', 2, None, '___sec20'), ('The code for solving the ODE', 2, None, '___sec21'), ('The network with one input layer, specified number of hidden ' @@ -80,100 +80,100 @@ Automatically generated HTML file from DocOnce source ('The trial solution', 2, None, '___sec25'), ('The program using Autograd', 2, None, '___sec26'), ('Using forward Euler to solve the ODE', 2, None, '___sec27'), - ('Example: Solving the one dimensional Poisson equation using ' - 'Autograd and TensorFlow', + ('Example: Solving the one dimensional Poisson equation', 2, None, '___sec28'), ('The specific equation to solve for', 2, None, '___sec29'), ('Solving the equation using Autograd', 2, None, '___sec30'), ('Comparing with a numerical scheme', 2, None, '___sec31'), - ('Partial Differential Equations', 2, None, '___sec32'), - ('Type of problem', 2, None, '___sec33'), - ('Network requirements', 2, None, '___sec34'), - ('More details', 2, None, '___sec35'), - ('Example: The diffusion equation', 2, None, '___sec36'), - ('Defining the problem', 2, None, '___sec37'), - ('Setting up the network using Autograd', 2, None, '___sec38'), + ('Setting up the code', 2, None, '___sec32'), + ('Partial Differential Equations', 2, None, '___sec33'), + ('Type of problem', 2, None, '___sec34'), + ('Network requirements', 2, None, '___sec35'), + ('More details', 2, None, '___sec36'), + ('Example: The diffusion equation', 2, None, '___sec37'), + ('Defining the problem', 2, None, '___sec38'), + ('Setting up the network using Autograd', 2, None, '___sec39'), ('Setting up the network using Autograd; The trial solution', 2, None, - '___sec39'), - ('Why the jacobian?', 2, None, '___sec40'), + '___sec40'), + ('Why the jacobian?', 2, None, '___sec41'), ('Setting up the network using Autograd; The full program', 2, None, - '___sec41'), + '___sec42'), ('Example: Solving the wave equation with Neural Networks', 2, None, - '___sec42'), - ('The problem to solve for', 2, None, '___sec43'), - ('The trial solution', 2, None, '___sec44'), - ('The analytical solution', 2, None, '___sec45'), + '___sec43'), + ('The problem to solve for', 2, None, '___sec44'), + ('The trial solution', 2, None, '___sec45'), + ('The analytical solution', 2, None, '___sec46'), ('Solving the wave equation - the full program using Autograd', 2, None, - '___sec46'), + '___sec47'), ('Resources on differential equations and deep learning', 2, None, - '___sec47'), - ('Friday, Principal Component Analysis', 2, None, '___sec48'), + '___sec48'), + ('Friday, Principal Component Analysis', 2, None, '___sec49'), ('Basic ideas of the Principal Component Analysis (PCA)', 2, None, - '___sec49'), + '___sec50'), ('Introducing the Covariance and Correlation functions', 2, None, - '___sec50'), + '___sec51'), ('Correlation Function and Design/Feature Matrix', 2, None, - '___sec51'), - ('Covariance Matrix Examples', 2, None, '___sec52'), - ('Correlation Matrix', 2, None, '___sec53'), - ('Correlation Matrix with Pandas', 2, None, '___sec54'), + '___sec52'), + ('Covariance Matrix Examples', 2, None, '___sec53'), + ('Correlation Matrix', 2, None, '___sec54'), + ('Correlation Matrix with Pandas', 2, None, '___sec55'), ('Correlation Matrix with Pandas and the Franke function', 2, None, - '___sec55'), + '___sec56'), ('Rewriting the Covariance and/or Correlation Matrix', 2, None, - '___sec56'), - ('Towards the PCA theorem', 2, None, '___sec57'), - ('The Algorithm before the Theorem', 2, None, '___sec58'), - ('Writing our own PCA code', 2, None, '___sec59'), + '___sec57'), + ('Towards the PCA theorem', 2, None, '___sec58'), + ('The Algorithm before the Theorem', 2, None, '___sec59'), + ('Writing our own PCA code', 2, None, '___sec60'), ('Compute the sample mean and center the data', 3, None, - '___sec60'), - ('Compute the sample covariance', 3, None, '___sec61'), + '___sec61'), + ('Compute the sample covariance', 3, None, '___sec62'), ('Diagonalize the sample covariance matrix to obtain the ' 'principal components', 3, None, - '___sec62'), - ('Classical PCA Theorem', 2, None, '___sec63'), - ('Proof of the PCA Theorem', 2, None, '___sec64'), - ('PCA Proof continued', 2, None, '___sec65'), - ('The final step', 2, None, '___sec66'), + '___sec63'), + ('Classical PCA Theorem', 2, None, '___sec64'), + ('Proof of the PCA Theorem', 2, None, '___sec65'), + ('PCA Proof continued', 2, None, '___sec66'), + ('The final step', 2, None, '___sec67'), ('Geometric Interpretation and link with Singular Value ' 'Decomposition', 2, None, - '___sec67'), - ('Principal Component Analysis', 2, None, '___sec68'), - ('PCA and scikit-learn', 2, None, '___sec69'), - ('Back to the Cancer Data', 2, None, '___sec70'), - ('More on the PCA', 2, None, '___sec71'), - ('Incremental PCA', 2, None, '___sec72'), - ('Randomized PCA', 2, None, '___sec73'), - ('Kernel PCA', 2, None, '___sec74'), - ('LLE', 2, None, '___sec75'), - ('Other techniques', 2, None, '___sec76')]} + '___sec68'), + ('Principal Component Analysis', 2, None, '___sec69'), + ('PCA and scikit-learn', 2, None, '___sec70'), + ('Back to the Cancer Data', 2, None, '___sec71'), + ('More on the PCA', 2, None, '___sec72'), + ('Incremental PCA', 2, None, '___sec73'), + ('Randomized PCA', 2, None, '___sec74'), + ('Kernel PCA', 2, None, '___sec75'), + ('LLE', 2, None, '___sec76'), + ('Other techniques', 2, None, '___sec77')]} end of tocinfo --> @@ -230,7 +230,7 @@ MathJax.Hub.Config({
  • Final technicalities II
  • Final technicalities III
  • Final technicalities IV
    • -
  • Backpropagation
    • +
  • Back propagation
  • Gradient descent
  • The code for solving the ODE
  • The network with one input layer, specified number of hidden layers, and one output layer
    • @@ -239,55 +239,56 @@ MathJax.Hub.Config({
  • The trial solution
  • The program using Autograd
  • Using forward Euler to solve the ODE
    • -
  • Example: Solving the one dimensional Poisson equation using Autograd and TensorFlow
    • +
  • Example: Solving the one dimensional Poisson equation
  • The specific equation to solve for
  • Solving the equation using Autograd
  • Comparing with a numerical scheme
    • -
  • Partial Differential Equations
    • -
  • Type of problem
    • -
  • Network requirements
    • -
  • More details
    • -
  • Example: The diffusion equation
    • -
  • Defining the problem
    • -
  • Setting up the network using Autograd
    • -
  • Setting up the network using Autograd; The trial solution
    • -
  • Why the jacobian?
    • -
  • Setting up the network using Autograd; The full program
    • -
  • Example: Solving the wave equation with Neural Networks
    • -
  • The problem to solve for
    • -
  • The trial solution
    • -
  • The analytical solution
    • -
  • Solving the wave equation - the full program using Autograd
    • -
  • Resources on differential equations and deep learning
    • -
  • Friday, Principal Component Analysis
    • -
  • Basic ideas of the Principal Component Analysis (PCA)
    • -
  • Introducing the Covariance and Correlation functions
    • -
  • Correlation Function and Design/Feature Matrix
    • -
  • Covariance Matrix Examples
    • -
  • Correlation Matrix
    • -
  • Correlation Matrix with Pandas
    • -
  • Correlation Matrix with Pandas and the Franke function
    • -
  • Rewriting the Covariance and/or Correlation Matrix
    • -
  • Towards the PCA theorem
    • -
  • The Algorithm before the Theorem
    • -
  • Writing our own PCA code
    • -
  •    Compute the sample mean and center the data
    • -
  •    Compute the sample covariance
    • -
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • -
  • Classical PCA Theorem
    • -
  • Proof of the PCA Theorem
    • -
  • PCA Proof continued
    • -
  • The final step
    • -
  • Geometric Interpretation and link with Singular Value Decomposition
    • -
  • Principal Component Analysis
    • -
  • PCA and scikit-learn
    • -
  • Back to the Cancer Data
    • -
  • More on the PCA
    • -
  • Incremental PCA
    • -
  • Randomized PCA
    • -
  • Kernel PCA
    • -
  • LLE
    • -
  • Other techniques
    • +
  • Setting up the code
    • +
  • Partial Differential Equations
    • +
  • Type of problem
    • +
  • Network requirements
    • +
  • More details
    • +
  • Example: The diffusion equation
    • +
  • Defining the problem
    • +
  • Setting up the network using Autograd
    • +
  • Setting up the network using Autograd; The trial solution
    • +
  • Why the jacobian?
    • +
  • Setting up the network using Autograd; The full program
    • +
  • Example: Solving the wave equation with Neural Networks
    • +
  • The problem to solve for
    • +
  • The trial solution
    • +
  • The analytical solution
    • +
  • Solving the wave equation - the full program using Autograd
    • +
  • Resources on differential equations and deep learning
    • +
  • Friday, Principal Component Analysis
    • +
  • Basic ideas of the Principal Component Analysis (PCA)
    • +
  • Introducing the Covariance and Correlation functions
    • +
  • Correlation Function and Design/Feature Matrix
    • +
  • Covariance Matrix Examples
    • +
  • Correlation Matrix
    • +
  • Correlation Matrix with Pandas
    • +
  • Correlation Matrix with Pandas and the Franke function
    • +
  • Rewriting the Covariance and/or Correlation Matrix
    • +
  • Towards the PCA theorem
    • +
  • The Algorithm before the Theorem
    • +
  • Writing our own PCA code
    • +
  •    Compute the sample mean and center the data
    • +
  •    Compute the sample covariance
    • +
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • +
  • Classical PCA Theorem
    • +
  • Proof of the PCA Theorem
    • +
  • PCA Proof continued
    • +
  • The final step
    • +
  • Geometric Interpretation and link with Singular Value Decomposition
    • +
  • Principal Component Analysis
    • +
  • PCA and scikit-learn
    • +
  • Back to the Cancer Data
    • +
  • More on the PCA
    • +
  • Incremental PCA
    • +
  • Randomized PCA
    • +
  • Kernel PCA
    • +
  • LLE
    • +
  • Other techniques
    • @@ -348,7 +349,7 @@ In this case we seek a continuous range of values since we are approximating a f
  • 29
  • 30
  • ...
    • -
  • 76
    • +
  • 77
  • »
    • diff --git a/doc/pub/week43/html/._week43-bs021.html b/doc/pub/week43/html/._week43-bs021.html index baaaf2e57..6536855d8 100644 --- a/doc/pub/week43/html/._week43-bs021.html +++ b/doc/pub/week43/html/._week43-bs021.html @@ -67,7 +67,7 @@ Automatically generated HTML file from DocOnce source ('Final technicalities II', 2, None, '___sec16'), ('Final technicalities III', 2, None, '___sec17'), ('Final technicalities IV', 2, None, '___sec18'), - ('Backpropagation', 2, None, '___sec19'), + ('Back propagation', 2, None, '___sec19'), ('Gradient descent', 2, None, '___sec20'), ('The code for solving the ODE', 2, None, '___sec21'), ('The network with one input layer, specified number of hidden ' @@ -80,100 +80,100 @@ Automatically generated HTML file from DocOnce source ('The trial solution', 2, None, '___sec25'), ('The program using Autograd', 2, None, '___sec26'), ('Using forward Euler to solve the ODE', 2, None, '___sec27'), - ('Example: Solving the one dimensional Poisson equation using ' - 'Autograd and TensorFlow', + ('Example: Solving the one dimensional Poisson equation', 2, None, '___sec28'), ('The specific equation to solve for', 2, None, '___sec29'), ('Solving the equation using Autograd', 2, None, '___sec30'), ('Comparing with a numerical scheme', 2, None, '___sec31'), - ('Partial Differential Equations', 2, None, '___sec32'), - ('Type of problem', 2, None, '___sec33'), - ('Network requirements', 2, None, '___sec34'), - ('More details', 2, None, '___sec35'), - ('Example: The diffusion equation', 2, None, '___sec36'), - ('Defining the problem', 2, None, '___sec37'), - ('Setting up the network using Autograd', 2, None, '___sec38'), + ('Setting up the code', 2, None, '___sec32'), + ('Partial Differential Equations', 2, None, '___sec33'), + ('Type of problem', 2, None, '___sec34'), + ('Network requirements', 2, None, '___sec35'), + ('More details', 2, None, '___sec36'), + ('Example: The diffusion equation', 2, None, '___sec37'), + ('Defining the problem', 2, None, '___sec38'), + ('Setting up the network using Autograd', 2, None, '___sec39'), ('Setting up the network using Autograd; The trial solution', 2, None, - '___sec39'), - ('Why the jacobian?', 2, None, '___sec40'), + '___sec40'), + ('Why the jacobian?', 2, None, '___sec41'), ('Setting up the network using Autograd; The full program', 2, None, - '___sec41'), + '___sec42'), ('Example: Solving the wave equation with Neural Networks', 2, None, - '___sec42'), - ('The problem to solve for', 2, None, '___sec43'), - ('The trial solution', 2, None, '___sec44'), - ('The analytical solution', 2, None, '___sec45'), + '___sec43'), + ('The problem to solve for', 2, None, '___sec44'), + ('The trial solution', 2, None, '___sec45'), + ('The analytical solution', 2, None, '___sec46'), ('Solving the wave equation - the full program using Autograd', 2, None, - '___sec46'), + '___sec47'), ('Resources on differential equations and deep learning', 2, None, - '___sec47'), - ('Friday, Principal Component Analysis', 2, None, '___sec48'), + '___sec48'), + ('Friday, Principal Component Analysis', 2, None, '___sec49'), ('Basic ideas of the Principal Component Analysis (PCA)', 2, None, - '___sec49'), + '___sec50'), ('Introducing the Covariance and Correlation functions', 2, None, - '___sec50'), + '___sec51'), ('Correlation Function and Design/Feature Matrix', 2, None, - '___sec51'), - ('Covariance Matrix Examples', 2, None, '___sec52'), - ('Correlation Matrix', 2, None, '___sec53'), - ('Correlation Matrix with Pandas', 2, None, '___sec54'), + '___sec52'), + ('Covariance Matrix Examples', 2, None, '___sec53'), + ('Correlation Matrix', 2, None, '___sec54'), + ('Correlation Matrix with Pandas', 2, None, '___sec55'), ('Correlation Matrix with Pandas and the Franke function', 2, None, - '___sec55'), + '___sec56'), ('Rewriting the Covariance and/or Correlation Matrix', 2, None, - '___sec56'), - ('Towards the PCA theorem', 2, None, '___sec57'), - ('The Algorithm before the Theorem', 2, None, '___sec58'), - ('Writing our own PCA code', 2, None, '___sec59'), + '___sec57'), + ('Towards the PCA theorem', 2, None, '___sec58'), + ('The Algorithm before the Theorem', 2, None, '___sec59'), + ('Writing our own PCA code', 2, None, '___sec60'), ('Compute the sample mean and center the data', 3, None, - '___sec60'), - ('Compute the sample covariance', 3, None, '___sec61'), + '___sec61'), + ('Compute the sample covariance', 3, None, '___sec62'), ('Diagonalize the sample covariance matrix to obtain the ' 'principal components', 3, None, - '___sec62'), - ('Classical PCA Theorem', 2, None, '___sec63'), - ('Proof of the PCA Theorem', 2, None, '___sec64'), - ('PCA Proof continued', 2, None, '___sec65'), - ('The final step', 2, None, '___sec66'), + '___sec63'), + ('Classical PCA Theorem', 2, None, '___sec64'), + ('Proof of the PCA Theorem', 2, None, '___sec65'), + ('PCA Proof continued', 2, None, '___sec66'), + ('The final step', 2, None, '___sec67'), ('Geometric Interpretation and link with Singular Value ' 'Decomposition', 2, None, - '___sec67'), - ('Principal Component Analysis', 2, None, '___sec68'), - ('PCA and scikit-learn', 2, None, '___sec69'), - ('Back to the Cancer Data', 2, None, '___sec70'), - ('More on the PCA', 2, None, '___sec71'), - ('Incremental PCA', 2, None, '___sec72'), - ('Randomized PCA', 2, None, '___sec73'), - ('Kernel PCA', 2, None, '___sec74'), - ('LLE', 2, None, '___sec75'), - ('Other techniques', 2, None, '___sec76')]} + '___sec68'), + ('Principal Component Analysis', 2, None, '___sec69'), + ('PCA and scikit-learn', 2, None, '___sec70'), + ('Back to the Cancer Data', 2, None, '___sec71'), + ('More on the PCA', 2, None, '___sec72'), + ('Incremental PCA', 2, None, '___sec73'), + ('Randomized PCA', 2, None, '___sec74'), + ('Kernel PCA', 2, None, '___sec75'), + ('LLE', 2, None, '___sec76'), + ('Other techniques', 2, None, '___sec77')]} end of tocinfo --> @@ -230,7 +230,7 @@ MathJax.Hub.Config({
  • Final technicalities II
  • Final technicalities III
  • Final technicalities IV
    • -
  • Backpropagation
    • +
  • Back propagation
  • Gradient descent
  • The code for solving the ODE
  • The network with one input layer, specified number of hidden layers, and one output layer
    • @@ -239,55 +239,56 @@ MathJax.Hub.Config({
  • The trial solution
  • The program using Autograd
  • Using forward Euler to solve the ODE
    • -
  • Example: Solving the one dimensional Poisson equation using Autograd and TensorFlow
    • +
  • Example: Solving the one dimensional Poisson equation
  • The specific equation to solve for
  • Solving the equation using Autograd
  • Comparing with a numerical scheme
    • -
  • Partial Differential Equations
    • -
  • Type of problem
    • -
  • Network requirements
    • -
  • More details
    • -
  • Example: The diffusion equation
    • -
  • Defining the problem
    • -
  • Setting up the network using Autograd
    • -
  • Setting up the network using Autograd; The trial solution
    • -
  • Why the jacobian?
    • -
  • Setting up the network using Autograd; The full program
    • -
  • Example: Solving the wave equation with Neural Networks
    • -
  • The problem to solve for
    • -
  • The trial solution
    • -
  • The analytical solution
    • -
  • Solving the wave equation - the full program using Autograd
    • -
  • Resources on differential equations and deep learning
    • -
  • Friday, Principal Component Analysis
    • -
  • Basic ideas of the Principal Component Analysis (PCA)
    • -
  • Introducing the Covariance and Correlation functions
    • -
  • Correlation Function and Design/Feature Matrix
    • -
  • Covariance Matrix Examples
    • -
  • Correlation Matrix
    • -
  • Correlation Matrix with Pandas
    • -
  • Correlation Matrix with Pandas and the Franke function
    • -
  • Rewriting the Covariance and/or Correlation Matrix
    • -
  • Towards the PCA theorem
    • -
  • The Algorithm before the Theorem
    • -
  • Writing our own PCA code
    • -
  •    Compute the sample mean and center the data
    • -
  •    Compute the sample covariance
    • -
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • -
  • Classical PCA Theorem
    • -
  • Proof of the PCA Theorem
    • -
  • PCA Proof continued
    • -
  • The final step
    • -
  • Geometric Interpretation and link with Singular Value Decomposition
    • -
  • Principal Component Analysis
    • -
  • PCA and scikit-learn
    • -
  • Back to the Cancer Data
    • -
  • More on the PCA
    • -
  • Incremental PCA
    • -
  • Randomized PCA
    • -
  • Kernel PCA
    • -
  • LLE
    • -
  • Other techniques
    • +
  • Setting up the code
    • +
  • Partial Differential Equations
    • +
  • Type of problem
    • +
  • Network requirements
    • +
  • More details
    • +
  • Example: The diffusion equation
    • +
  • Defining the problem
    • +
  • Setting up the network using Autograd
    • +
  • Setting up the network using Autograd; The trial solution
    • +
  • Why the jacobian?
    • +
  • Setting up the network using Autograd; The full program
    • +
  • Example: Solving the wave equation with Neural Networks
    • +
  • The problem to solve for
    • +
  • The trial solution
    • +
  • The analytical solution
    • +
  • Solving the wave equation - the full program using Autograd
    • +
  • Resources on differential equations and deep learning
    • +
  • Friday, Principal Component Analysis
    • +
  • Basic ideas of the Principal Component Analysis (PCA)
    • +
  • Introducing the Covariance and Correlation functions
    • +
  • Correlation Function and Design/Feature Matrix
    • +
  • Covariance Matrix Examples
    • +
  • Correlation Matrix
    • +
  • Correlation Matrix with Pandas
    • +
  • Correlation Matrix with Pandas and the Franke function
    • +
  • Rewriting the Covariance and/or Correlation Matrix
    • +
  • Towards the PCA theorem
    • +
  • The Algorithm before the Theorem
    • +
  • Writing our own PCA code
    • +
  •    Compute the sample mean and center the data
    • +
  •    Compute the sample covariance
    • +
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • +
  • Classical PCA Theorem
    • +
  • Proof of the PCA Theorem
    • +
  • PCA Proof continued
    • +
  • The final step
    • +
  • Geometric Interpretation and link with Singular Value Decomposition
    • +
  • Principal Component Analysis
    • +
  • PCA and scikit-learn
    • +
  • Back to the Cancer Data
    • +
  • More on the PCA
    • +
  • Incremental PCA
    • +
  • Randomized PCA
    • +
  • Kernel PCA
    • +
  • LLE
    • +
  • Other techniques
    • @@ -303,7 +304,7 @@ MathJax.Hub.Config({ -

    Backpropagation

    +

    Back propagation

    The next step is to decide how the parameters should be changed such that they minimize the cost function. @@ -347,7 +348,7 @@ Here, gradient descent with a constant step size has been chosen.

  • 30
  • 31
  • ...
    • -
  • 76
    • +
  • 77
  • »
    • diff --git a/doc/pub/week43/html/._week43-bs022.html b/doc/pub/week43/html/._week43-bs022.html index 6e6640e9f..9192d9756 100644 --- a/doc/pub/week43/html/._week43-bs022.html +++ b/doc/pub/week43/html/._week43-bs022.html @@ -67,7 +67,7 @@ Automatically generated HTML file from DocOnce source ('Final technicalities II', 2, None, '___sec16'), ('Final technicalities III', 2, None, '___sec17'), ('Final technicalities IV', 2, None, '___sec18'), - ('Backpropagation', 2, None, '___sec19'), + ('Back propagation', 2, None, '___sec19'), ('Gradient descent', 2, None, '___sec20'), ('The code for solving the ODE', 2, None, '___sec21'), ('The network with one input layer, specified number of hidden ' @@ -80,100 +80,100 @@ Automatically generated HTML file from DocOnce source ('The trial solution', 2, None, '___sec25'), ('The program using Autograd', 2, None, '___sec26'), ('Using forward Euler to solve the ODE', 2, None, '___sec27'), - ('Example: Solving the one dimensional Poisson equation using ' - 'Autograd and TensorFlow', + ('Example: Solving the one dimensional Poisson equation', 2, None, '___sec28'), ('The specific equation to solve for', 2, None, '___sec29'), ('Solving the equation using Autograd', 2, None, '___sec30'), ('Comparing with a numerical scheme', 2, None, '___sec31'), - ('Partial Differential Equations', 2, None, '___sec32'), - ('Type of problem', 2, None, '___sec33'), - ('Network requirements', 2, None, '___sec34'), - ('More details', 2, None, '___sec35'), - ('Example: The diffusion equation', 2, None, '___sec36'), - ('Defining the problem', 2, None, '___sec37'), - ('Setting up the network using Autograd', 2, None, '___sec38'), + ('Setting up the code', 2, None, '___sec32'), + ('Partial Differential Equations', 2, None, '___sec33'), + ('Type of problem', 2, None, '___sec34'), + ('Network requirements', 2, None, '___sec35'), + ('More details', 2, None, '___sec36'), + ('Example: The diffusion equation', 2, None, '___sec37'), + ('Defining the problem', 2, None, '___sec38'), + ('Setting up the network using Autograd', 2, None, '___sec39'), ('Setting up the network using Autograd; The trial solution', 2, None, - '___sec39'), - ('Why the jacobian?', 2, None, '___sec40'), + '___sec40'), + ('Why the jacobian?', 2, None, '___sec41'), ('Setting up the network using Autograd; The full program', 2, None, - '___sec41'), + '___sec42'), ('Example: Solving the wave equation with Neural Networks', 2, None, - '___sec42'), - ('The problem to solve for', 2, None, '___sec43'), - ('The trial solution', 2, None, '___sec44'), - ('The analytical solution', 2, None, '___sec45'), + '___sec43'), + ('The problem to solve for', 2, None, '___sec44'), + ('The trial solution', 2, None, '___sec45'), + ('The analytical solution', 2, None, '___sec46'), ('Solving the wave equation - the full program using Autograd', 2, None, - '___sec46'), + '___sec47'), ('Resources on differential equations and deep learning', 2, None, - '___sec47'), - ('Friday, Principal Component Analysis', 2, None, '___sec48'), + '___sec48'), + ('Friday, Principal Component Analysis', 2, None, '___sec49'), ('Basic ideas of the Principal Component Analysis (PCA)', 2, None, - '___sec49'), + '___sec50'), ('Introducing the Covariance and Correlation functions', 2, None, - '___sec50'), + '___sec51'), ('Correlation Function and Design/Feature Matrix', 2, None, - '___sec51'), - ('Covariance Matrix Examples', 2, None, '___sec52'), - ('Correlation Matrix', 2, None, '___sec53'), - ('Correlation Matrix with Pandas', 2, None, '___sec54'), + '___sec52'), + ('Covariance Matrix Examples', 2, None, '___sec53'), + ('Correlation Matrix', 2, None, '___sec54'), + ('Correlation Matrix with Pandas', 2, None, '___sec55'), ('Correlation Matrix with Pandas and the Franke function', 2, None, - '___sec55'), + '___sec56'), ('Rewriting the Covariance and/or Correlation Matrix', 2, None, - '___sec56'), - ('Towards the PCA theorem', 2, None, '___sec57'), - ('The Algorithm before the Theorem', 2, None, '___sec58'), - ('Writing our own PCA code', 2, None, '___sec59'), + '___sec57'), + ('Towards the PCA theorem', 2, None, '___sec58'), + ('The Algorithm before the Theorem', 2, None, '___sec59'), + ('Writing our own PCA code', 2, None, '___sec60'), ('Compute the sample mean and center the data', 3, None, - '___sec60'), - ('Compute the sample covariance', 3, None, '___sec61'), + '___sec61'), + ('Compute the sample covariance', 3, None, '___sec62'), ('Diagonalize the sample covariance matrix to obtain the ' 'principal components', 3, None, - '___sec62'), - ('Classical PCA Theorem', 2, None, '___sec63'), - ('Proof of the PCA Theorem', 2, None, '___sec64'), - ('PCA Proof continued', 2, None, '___sec65'), - ('The final step', 2, None, '___sec66'), + '___sec63'), + ('Classical PCA Theorem', 2, None, '___sec64'), + ('Proof of the PCA Theorem', 2, None, '___sec65'), + ('PCA Proof continued', 2, None, '___sec66'), + ('The final step', 2, None, '___sec67'), ('Geometric Interpretation and link with Singular Value ' 'Decomposition', 2, None, - '___sec67'), - ('Principal Component Analysis', 2, None, '___sec68'), - ('PCA and scikit-learn', 2, None, '___sec69'), - ('Back to the Cancer Data', 2, None, '___sec70'), - ('More on the PCA', 2, None, '___sec71'), - ('Incremental PCA', 2, None, '___sec72'), - ('Randomized PCA', 2, None, '___sec73'), - ('Kernel PCA', 2, None, '___sec74'), - ('LLE', 2, None, '___sec75'), - ('Other techniques', 2, None, '___sec76')]} + '___sec68'), + ('Principal Component Analysis', 2, None, '___sec69'), + ('PCA and scikit-learn', 2, None, '___sec70'), + ('Back to the Cancer Data', 2, None, '___sec71'), + ('More on the PCA', 2, None, '___sec72'), + ('Incremental PCA', 2, None, '___sec73'), + ('Randomized PCA', 2, None, '___sec74'), + ('Kernel PCA', 2, None, '___sec75'), + ('LLE', 2, None, '___sec76'), + ('Other techniques', 2, None, '___sec77')]} end of tocinfo --> @@ -230,7 +230,7 @@ MathJax.Hub.Config({
  • Final technicalities II
  • Final technicalities III
  • Final technicalities IV
    • -
  • Backpropagation
    • +
  • Back propagation
  • Gradient descent
  • The code for solving the ODE
  • The network with one input layer, specified number of hidden layers, and one output layer
    • @@ -239,55 +239,56 @@ MathJax.Hub.Config({
  • The trial solution
  • The program using Autograd
  • Using forward Euler to solve the ODE
    • -
  • Example: Solving the one dimensional Poisson equation using Autograd and TensorFlow
    • +
  • Example: Solving the one dimensional Poisson equation
  • The specific equation to solve for
  • Solving the equation using Autograd
  • Comparing with a numerical scheme
    • -
  • Partial Differential Equations
    • -
  • Type of problem
    • -
  • Network requirements
    • -
  • More details
    • -
  • Example: The diffusion equation
    • -
  • Defining the problem
    • -
  • Setting up the network using Autograd
    • -
  • Setting up the network using Autograd; The trial solution
    • -
  • Why the jacobian?
    • -
  • Setting up the network using Autograd; The full program
    • -
  • Example: Solving the wave equation with Neural Networks
    • -
  • The problem to solve for
    • -
  • The trial solution
    • -
  • The analytical solution
    • -
  • Solving the wave equation - the full program using Autograd
    • -
  • Resources on differential equations and deep learning
    • -
  • Friday, Principal Component Analysis
    • -
  • Basic ideas of the Principal Component Analysis (PCA)
    • -
  • Introducing the Covariance and Correlation functions
    • -
  • Correlation Function and Design/Feature Matrix
    • -
  • Covariance Matrix Examples
    • -
  • Correlation Matrix
    • -
  • Correlation Matrix with Pandas
    • -
  • Correlation Matrix with Pandas and the Franke function
    • -
  • Rewriting the Covariance and/or Correlation Matrix
    • -
  • Towards the PCA theorem
    • -
  • The Algorithm before the Theorem
    • -
  • Writing our own PCA code
    • -
  •    Compute the sample mean and center the data
    • -
  •    Compute the sample covariance
    • -
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • -
  • Classical PCA Theorem
    • -
  • Proof of the PCA Theorem
    • -
  • PCA Proof continued
    • -
  • The final step
    • -
  • Geometric Interpretation and link with Singular Value Decomposition
    • -
  • Principal Component Analysis
    • -
  • PCA and scikit-learn
    • -
  • Back to the Cancer Data
    • -
  • More on the PCA
    • -
  • Incremental PCA
    • -
  • Randomized PCA
    • -
  • Kernel PCA
    • -
  • LLE
    • -
  • Other techniques
    • +
  • Setting up the code
    • +
  • Partial Differential Equations
    • +
  • Type of problem
    • +
  • Network requirements
    • +
  • More details
    • +
  • Example: The diffusion equation
    • +
  • Defining the problem
    • +
  • Setting up the network using Autograd
    • +
  • Setting up the network using Autograd; The trial solution
    • +
  • Why the jacobian?
    • +
  • Setting up the network using Autograd; The full program
    • +
  • Example: Solving the wave equation with Neural Networks
    • +
  • The problem to solve for
    • +
  • The trial solution
    • +
  • The analytical solution
    • +
  • Solving the wave equation - the full program using Autograd
    • +
  • Resources on differential equations and deep learning
    • +
  • Friday, Principal Component Analysis
    • +
  • Basic ideas of the Principal Component Analysis (PCA)
    • +
  • Introducing the Covariance and Correlation functions
    • +
  • Correlation Function and Design/Feature Matrix
    • +
  • Covariance Matrix Examples
    • +
  • Correlation Matrix
    • +
  • Correlation Matrix with Pandas
    • +
  • Correlation Matrix with Pandas and the Franke function
    • +
  • Rewriting the Covariance and/or Correlation Matrix
    • +
  • Towards the PCA theorem
    • +
  • The Algorithm before the Theorem
    • +
  • Writing our own PCA code
    • +
  •    Compute the sample mean and center the data
    • +
  •    Compute the sample covariance
    • +
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • +
  • Classical PCA Theorem
    • +
  • Proof of the PCA Theorem
    • +
  • PCA Proof continued
    • +
  • The final step
    • +
  • Geometric Interpretation and link with Singular Value Decomposition
    • +
  • Principal Component Analysis
    • +
  • PCA and scikit-learn
    • +
  • Back to the Cancer Data
    • +
  • More on the PCA
    • +
  • Incremental PCA
    • +
  • Randomized PCA
    • +
  • Kernel PCA
    • +
  • LLE
    • +
  • Other techniques
    • @@ -325,7 +326,7 @@ for a number of iterations or until $ \big|\big| \boldsymbol{\omega}_{\text{new} The value of \( \lambda \) decides how large steps the algorithm must take in the direction of $ \nabla_{\boldsymbol{\omega}} C(\boldsymbol{x}, \boldsymbol{\omega})$. The notation \( \nabla_{\boldsymbol{\omega}} \) express the gradient with respect -to the elements in \( \boldsymbol{\omega \). +to the elements in \( \boldsymbol{\omega} \).

    In our case, we have to minimize the cost function \( C(\boldsymbol{x}, P) \) with @@ -369,7 +370,7 @@ $$

  • 31
  • 32
  • ...
    • -
  • 76
    • +
  • 77
  • »
    • diff --git a/doc/pub/week43/html/._week43-bs023.html b/doc/pub/week43/html/._week43-bs023.html index 34ba61dda..0bad80e07 100644 --- a/doc/pub/week43/html/._week43-bs023.html +++ b/doc/pub/week43/html/._week43-bs023.html @@ -67,7 +67,7 @@ Automatically generated HTML file from DocOnce source ('Final technicalities II', 2, None, '___sec16'), ('Final technicalities III', 2, None, '___sec17'), ('Final technicalities IV', 2, None, '___sec18'), - ('Backpropagation', 2, None, '___sec19'), + ('Back propagation', 2, None, '___sec19'), ('Gradient descent', 2, None, '___sec20'), ('The code for solving the ODE', 2, None, '___sec21'), ('The network with one input layer, specified number of hidden ' @@ -80,100 +80,100 @@ Automatically generated HTML file from DocOnce source ('The trial solution', 2, None, '___sec25'), ('The program using Autograd', 2, None, '___sec26'), ('Using forward Euler to solve the ODE', 2, None, '___sec27'), - ('Example: Solving the one dimensional Poisson equation using ' - 'Autograd and TensorFlow', + ('Example: Solving the one dimensional Poisson equation', 2, None, '___sec28'), ('The specific equation to solve for', 2, None, '___sec29'), ('Solving the equation using Autograd', 2, None, '___sec30'), ('Comparing with a numerical scheme', 2, None, '___sec31'), - ('Partial Differential Equations', 2, None, '___sec32'), - ('Type of problem', 2, None, '___sec33'), - ('Network requirements', 2, None, '___sec34'), - ('More details', 2, None, '___sec35'), - ('Example: The diffusion equation', 2, None, '___sec36'), - ('Defining the problem', 2, None, '___sec37'), - ('Setting up the network using Autograd', 2, None, '___sec38'), + ('Setting up the code', 2, None, '___sec32'), + ('Partial Differential Equations', 2, None, '___sec33'), + ('Type of problem', 2, None, '___sec34'), + ('Network requirements', 2, None, '___sec35'), + ('More details', 2, None, '___sec36'), + ('Example: The diffusion equation', 2, None, '___sec37'), + ('Defining the problem', 2, None, '___sec38'), + ('Setting up the network using Autograd', 2, None, '___sec39'), ('Setting up the network using Autograd; The trial solution', 2, None, - '___sec39'), - ('Why the jacobian?', 2, None, '___sec40'), + '___sec40'), + ('Why the jacobian?', 2, None, '___sec41'), ('Setting up the network using Autograd; The full program', 2, None, - '___sec41'), + '___sec42'), ('Example: Solving the wave equation with Neural Networks', 2, None, - '___sec42'), - ('The problem to solve for', 2, None, '___sec43'), - ('The trial solution', 2, None, '___sec44'), - ('The analytical solution', 2, None, '___sec45'), + '___sec43'), + ('The problem to solve for', 2, None, '___sec44'), + ('The trial solution', 2, None, '___sec45'), + ('The analytical solution', 2, None, '___sec46'), ('Solving the wave equation - the full program using Autograd', 2, None, - '___sec46'), + '___sec47'), ('Resources on differential equations and deep learning', 2, None, - '___sec47'), - ('Friday, Principal Component Analysis', 2, None, '___sec48'), + '___sec48'), + ('Friday, Principal Component Analysis', 2, None, '___sec49'), ('Basic ideas of the Principal Component Analysis (PCA)', 2, None, - '___sec49'), + '___sec50'), ('Introducing the Covariance and Correlation functions', 2, None, - '___sec50'), + '___sec51'), ('Correlation Function and Design/Feature Matrix', 2, None, - '___sec51'), - ('Covariance Matrix Examples', 2, None, '___sec52'), - ('Correlation Matrix', 2, None, '___sec53'), - ('Correlation Matrix with Pandas', 2, None, '___sec54'), + '___sec52'), + ('Covariance Matrix Examples', 2, None, '___sec53'), + ('Correlation Matrix', 2, None, '___sec54'), + ('Correlation Matrix with Pandas', 2, None, '___sec55'), ('Correlation Matrix with Pandas and the Franke function', 2, None, - '___sec55'), + '___sec56'), ('Rewriting the Covariance and/or Correlation Matrix', 2, None, - '___sec56'), - ('Towards the PCA theorem', 2, None, '___sec57'), - ('The Algorithm before the Theorem', 2, None, '___sec58'), - ('Writing our own PCA code', 2, None, '___sec59'), + '___sec57'), + ('Towards the PCA theorem', 2, None, '___sec58'), + ('The Algorithm before the Theorem', 2, None, '___sec59'), + ('Writing our own PCA code', 2, None, '___sec60'), ('Compute the sample mean and center the data', 3, None, - '___sec60'), - ('Compute the sample covariance', 3, None, '___sec61'), + '___sec61'), + ('Compute the sample covariance', 3, None, '___sec62'), ('Diagonalize the sample covariance matrix to obtain the ' 'principal components', 3, None, - '___sec62'), - ('Classical PCA Theorem', 2, None, '___sec63'), - ('Proof of the PCA Theorem', 2, None, '___sec64'), - ('PCA Proof continued', 2, None, '___sec65'), - ('The final step', 2, None, '___sec66'), + '___sec63'), + ('Classical PCA Theorem', 2, None, '___sec64'), + ('Proof of the PCA Theorem', 2, None, '___sec65'), + ('PCA Proof continued', 2, None, '___sec66'), + ('The final step', 2, None, '___sec67'), ('Geometric Interpretation and link with Singular Value ' 'Decomposition', 2, None, - '___sec67'), - ('Principal Component Analysis', 2, None, '___sec68'), - ('PCA and scikit-learn', 2, None, '___sec69'), - ('Back to the Cancer Data', 2, None, '___sec70'), - ('More on the PCA', 2, None, '___sec71'), - ('Incremental PCA', 2, None, '___sec72'), - ('Randomized PCA', 2, None, '___sec73'), - ('Kernel PCA', 2, None, '___sec74'), - ('LLE', 2, None, '___sec75'), - ('Other techniques', 2, None, '___sec76')]} + '___sec68'), + ('Principal Component Analysis', 2, None, '___sec69'), + ('PCA and scikit-learn', 2, None, '___sec70'), + ('Back to the Cancer Data', 2, None, '___sec71'), + ('More on the PCA', 2, None, '___sec72'), + ('Incremental PCA', 2, None, '___sec73'), + ('Randomized PCA', 2, None, '___sec74'), + ('Kernel PCA', 2, None, '___sec75'), + ('LLE', 2, None, '___sec76'), + ('Other techniques', 2, None, '___sec77')]} end of tocinfo --> @@ -230,7 +230,7 @@ MathJax.Hub.Config({
  • Final technicalities II
  • Final technicalities III
  • Final technicalities IV
    • -
  • Backpropagation
    • +
  • Back propagation
  • Gradient descent
  • The code for solving the ODE
  • The network with one input layer, specified number of hidden layers, and one output layer
    • @@ -239,55 +239,56 @@ MathJax.Hub.Config({
  • The trial solution
  • The program using Autograd
  • Using forward Euler to solve the ODE
    • -
  • Example: Solving the one dimensional Poisson equation using Autograd and TensorFlow
    • +
  • Example: Solving the one dimensional Poisson equation
  • The specific equation to solve for
  • Solving the equation using Autograd
  • Comparing with a numerical scheme
    • -
  • Partial Differential Equations
    • -
  • Type of problem
    • -
  • Network requirements
    • -
  • More details
    • -
  • Example: The diffusion equation
    • -
  • Defining the problem
    • -
  • Setting up the network using Autograd
    • -
  • Setting up the network using Autograd; The trial solution
    • -
  • Why the jacobian?
    • -
  • Setting up the network using Autograd; The full program
    • -
  • Example: Solving the wave equation with Neural Networks
    • -
  • The problem to solve for
    • -
  • The trial solution
    • -
  • The analytical solution
    • -
  • Solving the wave equation - the full program using Autograd
    • -
  • Resources on differential equations and deep learning
    • -
  • Friday, Principal Component Analysis
    • -
  • Basic ideas of the Principal Component Analysis (PCA)
    • -
  • Introducing the Covariance and Correlation functions
    • -
  • Correlation Function and Design/Feature Matrix
    • -
  • Covariance Matrix Examples
    • -
  • Correlation Matrix
    • -
  • Correlation Matrix with Pandas
    • -
  • Correlation Matrix with Pandas and the Franke function
    • -
  • Rewriting the Covariance and/or Correlation Matrix
    • -
  • Towards the PCA theorem
    • -
  • The Algorithm before the Theorem
    • -
  • Writing our own PCA code
    • -
  •    Compute the sample mean and center the data
    • -
  •    Compute the sample covariance
    • -
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • -
  • Classical PCA Theorem
    • -
  • Proof of the PCA Theorem
    • -
  • PCA Proof continued
    • -
  • The final step
    • -
  • Geometric Interpretation and link with Singular Value Decomposition
    • -
  • Principal Component Analysis
    • -
  • PCA and scikit-learn
    • -
  • Back to the Cancer Data
    • -
  • More on the PCA
    • -
  • Incremental PCA
    • -
  • Randomized PCA
    • -
  • Kernel PCA
    • -
  • LLE
    • -
  • Other techniques
    • +
  • Setting up the code
    • +
  • Partial Differential Equations
    • +
  • Type of problem
    • +
  • Network requirements
    • +
  • More details
    • +
  • Example: The diffusion equation
    • +
  • Defining the problem
    • +
  • Setting up the network using Autograd
    • +
  • Setting up the network using Autograd; The trial solution
    • +
  • Why the jacobian?
    • +
  • Setting up the network using Autograd; The full program
    • +
  • Example: Solving the wave equation with Neural Networks
    • +
  • The problem to solve for
    • +
  • The trial solution
    • +
  • The analytical solution
    • +
  • Solving the wave equation - the full program using Autograd
    • +
  • Resources on differential equations and deep learning
    • +
  • Friday, Principal Component Analysis
    • +
  • Basic ideas of the Principal Component Analysis (PCA)
    • +
  • Introducing the Covariance and Correlation functions
    • +
  • Correlation Function and Design/Feature Matrix
    • +
  • Covariance Matrix Examples
    • +
  • Correlation Matrix
    • +
  • Correlation Matrix with Pandas
    • +
  • Correlation Matrix with Pandas and the Franke function
    • +
  • Rewriting the Covariance and/or Correlation Matrix
    • +
  • Towards the PCA theorem
    • +
  • The Algorithm before the Theorem
    • +
  • Writing our own PCA code
    • +
  •    Compute the sample mean and center the data
    • +
  •    Compute the sample covariance
    • +
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • +
  • Classical PCA Theorem
    • +
  • Proof of the PCA Theorem
    • +
  • PCA Proof continued
    • +
  • The final step
    • +
  • Geometric Interpretation and link with Singular Value Decomposition
    • +
  • Principal Component Analysis
    • +
  • PCA and scikit-learn
    • +
  • Back to the Cancer Data
    • +
  • More on the PCA
    • +
  • Incremental PCA
    • +
  • Randomized PCA
    • +
  • Kernel PCA
    • +
  • LLE
    • +
  • Other techniques
    • @@ -479,7 +480,7 @@ MathJax.Hub.Config({
  • 32
  • 33
  • ...
    • -
  • 76
    • +
  • 77
  • »
    • diff --git a/doc/pub/week43/html/._week43-bs024.html b/doc/pub/week43/html/._week43-bs024.html index 8e616d0a1..a20407e5d 100644 --- a/doc/pub/week43/html/._week43-bs024.html +++ b/doc/pub/week43/html/._week43-bs024.html @@ -67,7 +67,7 @@ Automatically generated HTML file from DocOnce source ('Final technicalities II', 2, None, '___sec16'), ('Final technicalities III', 2, None, '___sec17'), ('Final technicalities IV', 2, None, '___sec18'), - ('Backpropagation', 2, None, '___sec19'), + ('Back propagation', 2, None, '___sec19'), ('Gradient descent', 2, None, '___sec20'), ('The code for solving the ODE', 2, None, '___sec21'), ('The network with one input layer, specified number of hidden ' @@ -80,100 +80,100 @@ Automatically generated HTML file from DocOnce source ('The trial solution', 2, None, '___sec25'), ('The program using Autograd', 2, None, '___sec26'), ('Using forward Euler to solve the ODE', 2, None, '___sec27'), - ('Example: Solving the one dimensional Poisson equation using ' - 'Autograd and TensorFlow', + ('Example: Solving the one dimensional Poisson equation', 2, None, '___sec28'), ('The specific equation to solve for', 2, None, '___sec29'), ('Solving the equation using Autograd', 2, None, '___sec30'), ('Comparing with a numerical scheme', 2, None, '___sec31'), - ('Partial Differential Equations', 2, None, '___sec32'), - ('Type of problem', 2, None, '___sec33'), - ('Network requirements', 2, None, '___sec34'), - ('More details', 2, None, '___sec35'), - ('Example: The diffusion equation', 2, None, '___sec36'), - ('Defining the problem', 2, None, '___sec37'), - ('Setting up the network using Autograd', 2, None, '___sec38'), + ('Setting up the code', 2, None, '___sec32'), + ('Partial Differential Equations', 2, None, '___sec33'), + ('Type of problem', 2, None, '___sec34'), + ('Network requirements', 2, None, '___sec35'), + ('More details', 2, None, '___sec36'), + ('Example: The diffusion equation', 2, None, '___sec37'), + ('Defining the problem', 2, None, '___sec38'), + ('Setting up the network using Autograd', 2, None, '___sec39'), ('Setting up the network using Autograd; The trial solution', 2, None, - '___sec39'), - ('Why the jacobian?', 2, None, '___sec40'), + '___sec40'), + ('Why the jacobian?', 2, None, '___sec41'), ('Setting up the network using Autograd; The full program', 2, None, - '___sec41'), + '___sec42'), ('Example: Solving the wave equation with Neural Networks', 2, None, - '___sec42'), - ('The problem to solve for', 2, None, '___sec43'), - ('The trial solution', 2, None, '___sec44'), - ('The analytical solution', 2, None, '___sec45'), + '___sec43'), + ('The problem to solve for', 2, None, '___sec44'), + ('The trial solution', 2, None, '___sec45'), + ('The analytical solution', 2, None, '___sec46'), ('Solving the wave equation - the full program using Autograd', 2, None, - '___sec46'), + '___sec47'), ('Resources on differential equations and deep learning', 2, None, - '___sec47'), - ('Friday, Principal Component Analysis', 2, None, '___sec48'), + '___sec48'), + ('Friday, Principal Component Analysis', 2, None, '___sec49'), ('Basic ideas of the Principal Component Analysis (PCA)', 2, None, - '___sec49'), + '___sec50'), ('Introducing the Covariance and Correlation functions', 2, None, - '___sec50'), + '___sec51'), ('Correlation Function and Design/Feature Matrix', 2, None, - '___sec51'), - ('Covariance Matrix Examples', 2, None, '___sec52'), - ('Correlation Matrix', 2, None, '___sec53'), - ('Correlation Matrix with Pandas', 2, None, '___sec54'), + '___sec52'), + ('Covariance Matrix Examples', 2, None, '___sec53'), + ('Correlation Matrix', 2, None, '___sec54'), + ('Correlation Matrix with Pandas', 2, None, '___sec55'), ('Correlation Matrix with Pandas and the Franke function', 2, None, - '___sec55'), + '___sec56'), ('Rewriting the Covariance and/or Correlation Matrix', 2, None, - '___sec56'), - ('Towards the PCA theorem', 2, None, '___sec57'), - ('The Algorithm before the Theorem', 2, None, '___sec58'), - ('Writing our own PCA code', 2, None, '___sec59'), + '___sec57'), + ('Towards the PCA theorem', 2, None, '___sec58'), + ('The Algorithm before the Theorem', 2, None, '___sec59'), + ('Writing our own PCA code', 2, None, '___sec60'), ('Compute the sample mean and center the data', 3, None, - '___sec60'), - ('Compute the sample covariance', 3, None, '___sec61'), + '___sec61'), + ('Compute the sample covariance', 3, None, '___sec62'), ('Diagonalize the sample covariance matrix to obtain the ' 'principal components', 3, None, - '___sec62'), - ('Classical PCA Theorem', 2, None, '___sec63'), - ('Proof of the PCA Theorem', 2, None, '___sec64'), - ('PCA Proof continued', 2, None, '___sec65'), - ('The final step', 2, None, '___sec66'), + '___sec63'), + ('Classical PCA Theorem', 2, None, '___sec64'), + ('Proof of the PCA Theorem', 2, None, '___sec65'), + ('PCA Proof continued', 2, None, '___sec66'), + ('The final step', 2, None, '___sec67'), ('Geometric Interpretation and link with Singular Value ' 'Decomposition', 2, None, - '___sec67'), - ('Principal Component Analysis', 2, None, '___sec68'), - ('PCA and scikit-learn', 2, None, '___sec69'), - ('Back to the Cancer Data', 2, None, '___sec70'), - ('More on the PCA', 2, None, '___sec71'), - ('Incremental PCA', 2, None, '___sec72'), - ('Randomized PCA', 2, None, '___sec73'), - ('Kernel PCA', 2, None, '___sec74'), - ('LLE', 2, None, '___sec75'), - ('Other techniques', 2, None, '___sec76')]} + '___sec68'), + ('Principal Component Analysis', 2, None, '___sec69'), + ('PCA and scikit-learn', 2, None, '___sec70'), + ('Back to the Cancer Data', 2, None, '___sec71'), + ('More on the PCA', 2, None, '___sec72'), + ('Incremental PCA', 2, None, '___sec73'), + ('Randomized PCA', 2, None, '___sec74'), + ('Kernel PCA', 2, None, '___sec75'), + ('LLE', 2, None, '___sec76'), + ('Other techniques', 2, None, '___sec77')]} end of tocinfo --> @@ -230,7 +230,7 @@ MathJax.Hub.Config({
  • Final technicalities II
  • Final technicalities III
  • Final technicalities IV
    • -
  • Backpropagation
    • +
  • Back propagation
  • Gradient descent
  • The code for solving the ODE
  • The network with one input layer, specified number of hidden layers, and one output layer
    • @@ -239,55 +239,56 @@ MathJax.Hub.Config({
  • The trial solution
  • The program using Autograd
  • Using forward Euler to solve the ODE
    • -
  • Example: Solving the one dimensional Poisson equation using Autograd and TensorFlow
    • +
  • Example: Solving the one dimensional Poisson equation
  • The specific equation to solve for
  • Solving the equation using Autograd
  • Comparing with a numerical scheme
    • -
  • Partial Differential Equations
    • -
  • Type of problem
    • -
  • Network requirements
    • -
  • More details
    • -
  • Example: The diffusion equation
    • -
  • Defining the problem
    • -
  • Setting up the network using Autograd
    • -
  • Setting up the network using Autograd; The trial solution
    • -
  • Why the jacobian?
    • -
  • Setting up the network using Autograd; The full program
    • -
  • Example: Solving the wave equation with Neural Networks
    • -
  • The problem to solve for
    • -
  • The trial solution
    • -
  • The analytical solution
    • -
  • Solving the wave equation - the full program using Autograd
    • -
  • Resources on differential equations and deep learning
    • -
  • Friday, Principal Component Analysis
    • -
  • Basic ideas of the Principal Component Analysis (PCA)
    • -
  • Introducing the Covariance and Correlation functions
    • -
  • Correlation Function and Design/Feature Matrix
    • -
  • Covariance Matrix Examples
    • -
  • Correlation Matrix
    • -
  • Correlation Matrix with Pandas
    • -
  • Correlation Matrix with Pandas and the Franke function
    • -
  • Rewriting the Covariance and/or Correlation Matrix
    • -
  • Towards the PCA theorem
    • -
  • The Algorithm before the Theorem
    • -
  • Writing our own PCA code
    • -
  •    Compute the sample mean and center the data
    • -
  •    Compute the sample covariance
    • -
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • -
  • Classical PCA Theorem
    • -
  • Proof of the PCA Theorem
    • -
  • PCA Proof continued
    • -
  • The final step
    • -
  • Geometric Interpretation and link with Singular Value Decomposition
    • -
  • Principal Component Analysis
    • -
  • PCA and scikit-learn
    • -
  • Back to the Cancer Data
    • -
  • More on the PCA
    • -
  • Incremental PCA
    • -
  • Randomized PCA
    • -
  • Kernel PCA
    • -
  • LLE
    • -
  • Other techniques
    • +
  • Setting up the code
    • +
  • Partial Differential Equations
    • +
  • Type of problem
    • +
  • Network requirements
    • +
  • More details
    • +
  • Example: The diffusion equation
    • +
  • Defining the problem
    • +
  • Setting up the network using Autograd
    • +
  • Setting up the network using Autograd; The trial solution
    • +
  • Why the jacobian?
    • +
  • Setting up the network using Autograd; The full program
    • +
  • Example: Solving the wave equation with Neural Networks
    • +
  • The problem to solve for
    • +
  • The trial solution
    • +
  • The analytical solution
    • +
  • Solving the wave equation - the full program using Autograd
    • +
  • Resources on differential equations and deep learning
    • +
  • Friday, Principal Component Analysis
    • +
  • Basic ideas of the Principal Component Analysis (PCA)
    • +
  • Introducing the Covariance and Correlation functions
    • +
  • Correlation Function and Design/Feature Matrix
    • +
  • Covariance Matrix Examples
    • +
  • Correlation Matrix
    • +
  • Correlation Matrix with Pandas
    • +
  • Correlation Matrix with Pandas and the Franke function
    • +
  • Rewriting the Covariance and/or Correlation Matrix
    • +
  • Towards the PCA theorem
    • +
  • The Algorithm before the Theorem
    • +
  • Writing our own PCA code
    • +
  •    Compute the sample mean and center the data
    • +
  •    Compute the sample covariance
    • +
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • +
  • Classical PCA Theorem
    • +
  • Proof of the PCA Theorem
    • +
  • PCA Proof continued
    • +
  • The final step
    • +
  • Geometric Interpretation and link with Singular Value Decomposition
    • +
  • Principal Component Analysis
    • +
  • PCA and scikit-learn
    • +
  • Back to the Cancer Data
    • +
  • More on the PCA
    • +
  • Incremental PCA
    • +
  • Randomized PCA
    • +
  • Kernel PCA
    • +
  • LLE
    • +
  • Other techniques
    • @@ -499,7 +500,7 @@ The number of neurons within each hidden layer are given as a list of integers i
  • 33
  • 34
  • ...
    • -
  • 76
    • +
  • 77
  • »
    • diff --git a/doc/pub/week43/html/._week43-bs025.html b/doc/pub/week43/html/._week43-bs025.html index 9948969e0..32c4027f0 100644 --- a/doc/pub/week43/html/._week43-bs025.html +++ b/doc/pub/week43/html/._week43-bs025.html @@ -67,7 +67,7 @@ Automatically generated HTML file from DocOnce source ('Final technicalities II', 2, None, '___sec16'), ('Final technicalities III', 2, None, '___sec17'), ('Final technicalities IV', 2, None, '___sec18'), - ('Backpropagation', 2, None, '___sec19'), + ('Back propagation', 2, None, '___sec19'), ('Gradient descent', 2, None, '___sec20'), ('The code for solving the ODE', 2, None, '___sec21'), ('The network with one input layer, specified number of hidden ' @@ -80,100 +80,100 @@ Automatically generated HTML file from DocOnce source ('The trial solution', 2, None, '___sec25'), ('The program using Autograd', 2, None, '___sec26'), ('Using forward Euler to solve the ODE', 2, None, '___sec27'), - ('Example: Solving the one dimensional Poisson equation using ' - 'Autograd and TensorFlow', + ('Example: Solving the one dimensional Poisson equation', 2, None, '___sec28'), ('The specific equation to solve for', 2, None, '___sec29'), ('Solving the equation using Autograd', 2, None, '___sec30'), ('Comparing with a numerical scheme', 2, None, '___sec31'), - ('Partial Differential Equations', 2, None, '___sec32'), - ('Type of problem', 2, None, '___sec33'), - ('Network requirements', 2, None, '___sec34'), - ('More details', 2, None, '___sec35'), - ('Example: The diffusion equation', 2, None, '___sec36'), - ('Defining the problem', 2, None, '___sec37'), - ('Setting up the network using Autograd', 2, None, '___sec38'), + ('Setting up the code', 2, None, '___sec32'), + ('Partial Differential Equations', 2, None, '___sec33'), + ('Type of problem', 2, None, '___sec34'), + ('Network requirements', 2, None, '___sec35'), + ('More details', 2, None, '___sec36'), + ('Example: The diffusion equation', 2, None, '___sec37'), + ('Defining the problem', 2, None, '___sec38'), + ('Setting up the network using Autograd', 2, None, '___sec39'), ('Setting up the network using Autograd; The trial solution', 2, None, - '___sec39'), - ('Why the jacobian?', 2, None, '___sec40'), + '___sec40'), + ('Why the jacobian?', 2, None, '___sec41'), ('Setting up the network using Autograd; The full program', 2, None, - '___sec41'), + '___sec42'), ('Example: Solving the wave equation with Neural Networks', 2, None, - '___sec42'), - ('The problem to solve for', 2, None, '___sec43'), - ('The trial solution', 2, None, '___sec44'), - ('The analytical solution', 2, None, '___sec45'), + '___sec43'), + ('The problem to solve for', 2, None, '___sec44'), + ('The trial solution', 2, None, '___sec45'), + ('The analytical solution', 2, None, '___sec46'), ('Solving the wave equation - the full program using Autograd', 2, None, - '___sec46'), + '___sec47'), ('Resources on differential equations and deep learning', 2, None, - '___sec47'), - ('Friday, Principal Component Analysis', 2, None, '___sec48'), + '___sec48'), + ('Friday, Principal Component Analysis', 2, None, '___sec49'), ('Basic ideas of the Principal Component Analysis (PCA)', 2, None, - '___sec49'), + '___sec50'), ('Introducing the Covariance and Correlation functions', 2, None, - '___sec50'), + '___sec51'), ('Correlation Function and Design/Feature Matrix', 2, None, - '___sec51'), - ('Covariance Matrix Examples', 2, None, '___sec52'), - ('Correlation Matrix', 2, None, '___sec53'), - ('Correlation Matrix with Pandas', 2, None, '___sec54'), + '___sec52'), + ('Covariance Matrix Examples', 2, None, '___sec53'), + ('Correlation Matrix', 2, None, '___sec54'), + ('Correlation Matrix with Pandas', 2, None, '___sec55'), ('Correlation Matrix with Pandas and the Franke function', 2, None, - '___sec55'), + '___sec56'), ('Rewriting the Covariance and/or Correlation Matrix', 2, None, - '___sec56'), - ('Towards the PCA theorem', 2, None, '___sec57'), - ('The Algorithm before the Theorem', 2, None, '___sec58'), - ('Writing our own PCA code', 2, None, '___sec59'), + '___sec57'), + ('Towards the PCA theorem', 2, None, '___sec58'), + ('The Algorithm before the Theorem', 2, None, '___sec59'), + ('Writing our own PCA code', 2, None, '___sec60'), ('Compute the sample mean and center the data', 3, None, - '___sec60'), - ('Compute the sample covariance', 3, None, '___sec61'), + '___sec61'), + ('Compute the sample covariance', 3, None, '___sec62'), ('Diagonalize the sample covariance matrix to obtain the ' 'principal components', 3, None, - '___sec62'), - ('Classical PCA Theorem', 2, None, '___sec63'), - ('Proof of the PCA Theorem', 2, None, '___sec64'), - ('PCA Proof continued', 2, None, '___sec65'), - ('The final step', 2, None, '___sec66'), + '___sec63'), + ('Classical PCA Theorem', 2, None, '___sec64'), + ('Proof of the PCA Theorem', 2, None, '___sec65'), + ('PCA Proof continued', 2, None, '___sec66'), + ('The final step', 2, None, '___sec67'), ('Geometric Interpretation and link with Singular Value ' 'Decomposition', 2, None, - '___sec67'), - ('Principal Component Analysis', 2, None, '___sec68'), - ('PCA and scikit-learn', 2, None, '___sec69'), - ('Back to the Cancer Data', 2, None, '___sec70'), - ('More on the PCA', 2, None, '___sec71'), - ('Incremental PCA', 2, None, '___sec72'), - ('Randomized PCA', 2, None, '___sec73'), - ('Kernel PCA', 2, None, '___sec74'), - ('LLE', 2, None, '___sec75'), - ('Other techniques', 2, None, '___sec76')]} + '___sec68'), + ('Principal Component Analysis', 2, None, '___sec69'), + ('PCA and scikit-learn', 2, None, '___sec70'), + ('Back to the Cancer Data', 2, None, '___sec71'), + ('More on the PCA', 2, None, '___sec72'), + ('Incremental PCA', 2, None, '___sec73'), + ('Randomized PCA', 2, None, '___sec74'), + ('Kernel PCA', 2, None, '___sec75'), + ('LLE', 2, None, '___sec76'), + ('Other techniques', 2, None, '___sec77')]} end of tocinfo --> @@ -230,7 +230,7 @@ MathJax.Hub.Config({
  • Final technicalities II
  • Final technicalities III
  • Final technicalities IV
    • -
  • Backpropagation
    • +
  • Back propagation
  • Gradient descent
  • The code for solving the ODE
  • The network with one input layer, specified number of hidden layers, and one output layer
    • @@ -239,55 +239,56 @@ MathJax.Hub.Config({
  • The trial solution
  • The program using Autograd
  • Using forward Euler to solve the ODE
    • -
  • Example: Solving the one dimensional Poisson equation using Autograd and TensorFlow
    • +
  • Example: Solving the one dimensional Poisson equation
  • The specific equation to solve for
  • Solving the equation using Autograd
  • Comparing with a numerical scheme
    • -
  • Partial Differential Equations
    • -
  • Type of problem
    • -
  • Network requirements
    • -
  • More details
    • -
  • Example: The diffusion equation
    • -
  • Defining the problem
    • -
  • Setting up the network using Autograd
    • -
  • Setting up the network using Autograd; The trial solution
    • -
  • Why the jacobian?
    • -
  • Setting up the network using Autograd; The full program
    • -
  • Example: Solving the wave equation with Neural Networks
    • -
  • The problem to solve for
    • -
  • The trial solution
    • -
  • The analytical solution
    • -
  • Solving the wave equation - the full program using Autograd
    • -
  • Resources on differential equations and deep learning
    • -
  • Friday, Principal Component Analysis
    • -
  • Basic ideas of the Principal Component Analysis (PCA)
    • -
  • Introducing the Covariance and Correlation functions
    • -
  • Correlation Function and Design/Feature Matrix
    • -
  • Covariance Matrix Examples
    • -
  • Correlation Matrix
    • -
  • Correlation Matrix with Pandas
    • -
  • Correlation Matrix with Pandas and the Franke function
    • -
  • Rewriting the Covariance and/or Correlation Matrix
    • -
  • Towards the PCA theorem
    • -
  • The Algorithm before the Theorem
    • -
  • Writing our own PCA code
    • -
  •    Compute the sample mean and center the data
    • -
  •    Compute the sample covariance
    • -
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • -
  • Classical PCA Theorem
    • -
  • Proof of the PCA Theorem
    • -
  • PCA Proof continued
    • -
  • The final step
    • -
  • Geometric Interpretation and link with Singular Value Decomposition
    • -
  • Principal Component Analysis
    • -
  • PCA and scikit-learn
    • -
  • Back to the Cancer Data
    • -
  • More on the PCA
    • -
  • Incremental PCA
    • -
  • Randomized PCA
    • -
  • Kernel PCA
    • -
  • LLE
    • -
  • Other techniques
    • +
  • Setting up the code
    • +
  • Partial Differential Equations
    • +
  • Type of problem
    • +
  • Network requirements
    • +
  • More details
    • +
  • Example: The diffusion equation
    • +
  • Defining the problem
    • +
  • Setting up the network using Autograd
    • +
  • Setting up the network using Autograd; The trial solution
    • +
  • Why the jacobian?
    • +
  • Setting up the network using Autograd; The full program
    • +
  • Example: Solving the wave equation with Neural Networks
    • +
  • The problem to solve for
    • +
  • The trial solution
    • +
  • The analytical solution
    • +
  • Solving the wave equation - the full program using Autograd
    • +
  • Resources on differential equations and deep learning
    • +
  • Friday, Principal Component Analysis
    • +
  • Basic ideas of the Principal Component Analysis (PCA)
    • +
  • Introducing the Covariance and Correlation functions
    • +
  • Correlation Function and Design/Feature Matrix
    • +
  • Covariance Matrix Examples
    • +
  • Correlation Matrix
    • +
  • Correlation Matrix with Pandas
    • +
  • Correlation Matrix with Pandas and the Franke function
    • +
  • Rewriting the Covariance and/or Correlation Matrix
    • +
  • Towards the PCA theorem
    • +
  • The Algorithm before the Theorem
    • +
  • Writing our own PCA code
    • +
  •    Compute the sample mean and center the data
    • +
  •    Compute the sample covariance
    • +
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • +
  • Classical PCA Theorem
    • +
  • Proof of the PCA Theorem
    • +
  • PCA Proof continued
    • +
  • The final step
    • +
  • Geometric Interpretation and link with Singular Value Decomposition
    • +
  • Principal Component Analysis
    • +
  • PCA and scikit-learn
    • +
  • Back to the Cancer Data
    • +
  • More on the PCA
    • +
  • Incremental PCA
    • +
  • Randomized PCA
    • +
  • Kernel PCA
    • +
  • LLE
    • +
  • Other techniques
    • @@ -322,8 +323,8 @@ Also, at \( t = 0 \) the population has the size \( g(0) = g_0 \), where \( g_0

    In this example, similar network as for the exponential decay using Autograd has been used to solve the equation. However, as the implementation might suffer from e.g numerical instability and high execution time (this might be more apparent in the examples solving PDEs), -a network has been constructed using TensorFlow also. -For comparison, the forward Euler method has been implemented in order to see how the networks performs compared to a numerical scheme. +using a library like TensorFlow is recommended. +Here, we stay with a more simple approach and implement for comparison, the simple forward Euler method.

    @@ -351,7 +352,7 @@ For comparison, the forward Euler method has been implemented in order to see ho

  • 34
  • 35
  • ...
    • -
  • 76
    • +
  • 77
  • »
    • diff --git a/doc/pub/week43/html/._week43-bs026.html b/doc/pub/week43/html/._week43-bs026.html index a9647bd00..5009c0dc5 100644 --- a/doc/pub/week43/html/._week43-bs026.html +++ b/doc/pub/week43/html/._week43-bs026.html @@ -67,7 +67,7 @@ Automatically generated HTML file from DocOnce source ('Final technicalities II', 2, None, '___sec16'), ('Final technicalities III', 2, None, '___sec17'), ('Final technicalities IV', 2, None, '___sec18'), - ('Backpropagation', 2, None, '___sec19'), + ('Back propagation', 2, None, '___sec19'), ('Gradient descent', 2, None, '___sec20'), ('The code for solving the ODE', 2, None, '___sec21'), ('The network with one input layer, specified number of hidden ' @@ -80,100 +80,100 @@ Automatically generated HTML file from DocOnce source ('The trial solution', 2, None, '___sec25'), ('The program using Autograd', 2, None, '___sec26'), ('Using forward Euler to solve the ODE', 2, None, '___sec27'), - ('Example: Solving the one dimensional Poisson equation using ' - 'Autograd and TensorFlow', + ('Example: Solving the one dimensional Poisson equation', 2, None, '___sec28'), ('The specific equation to solve for', 2, None, '___sec29'), ('Solving the equation using Autograd', 2, None, '___sec30'), ('Comparing with a numerical scheme', 2, None, '___sec31'), - ('Partial Differential Equations', 2, None, '___sec32'), - ('Type of problem', 2, None, '___sec33'), - ('Network requirements', 2, None, '___sec34'), - ('More details', 2, None, '___sec35'), - ('Example: The diffusion equation', 2, None, '___sec36'), - ('Defining the problem', 2, None, '___sec37'), - ('Setting up the network using Autograd', 2, None, '___sec38'), + ('Setting up the code', 2, None, '___sec32'), + ('Partial Differential Equations', 2, None, '___sec33'), + ('Type of problem', 2, None, '___sec34'), + ('Network requirements', 2, None, '___sec35'), + ('More details', 2, None, '___sec36'), + ('Example: The diffusion equation', 2, None, '___sec37'), + ('Defining the problem', 2, None, '___sec38'), + ('Setting up the network using Autograd', 2, None, '___sec39'), ('Setting up the network using Autograd; The trial solution', 2, None, - '___sec39'), - ('Why the jacobian?', 2, None, '___sec40'), + '___sec40'), + ('Why the jacobian?', 2, None, '___sec41'), ('Setting up the network using Autograd; The full program', 2, None, - '___sec41'), + '___sec42'), ('Example: Solving the wave equation with Neural Networks', 2, None, - '___sec42'), - ('The problem to solve for', 2, None, '___sec43'), - ('The trial solution', 2, None, '___sec44'), - ('The analytical solution', 2, None, '___sec45'), + '___sec43'), + ('The problem to solve for', 2, None, '___sec44'), + ('The trial solution', 2, None, '___sec45'), + ('The analytical solution', 2, None, '___sec46'), ('Solving the wave equation - the full program using Autograd', 2, None, - '___sec46'), + '___sec47'), ('Resources on differential equations and deep learning', 2, None, - '___sec47'), - ('Friday, Principal Component Analysis', 2, None, '___sec48'), + '___sec48'), + ('Friday, Principal Component Analysis', 2, None, '___sec49'), ('Basic ideas of the Principal Component Analysis (PCA)', 2, None, - '___sec49'), + '___sec50'), ('Introducing the Covariance and Correlation functions', 2, None, - '___sec50'), + '___sec51'), ('Correlation Function and Design/Feature Matrix', 2, None, - '___sec51'), - ('Covariance Matrix Examples', 2, None, '___sec52'), - ('Correlation Matrix', 2, None, '___sec53'), - ('Correlation Matrix with Pandas', 2, None, '___sec54'), + '___sec52'), + ('Covariance Matrix Examples', 2, None, '___sec53'), + ('Correlation Matrix', 2, None, '___sec54'), + ('Correlation Matrix with Pandas', 2, None, '___sec55'), ('Correlation Matrix with Pandas and the Franke function', 2, None, - '___sec55'), + '___sec56'), ('Rewriting the Covariance and/or Correlation Matrix', 2, None, - '___sec56'), - ('Towards the PCA theorem', 2, None, '___sec57'), - ('The Algorithm before the Theorem', 2, None, '___sec58'), - ('Writing our own PCA code', 2, None, '___sec59'), + '___sec57'), + ('Towards the PCA theorem', 2, None, '___sec58'), + ('The Algorithm before the Theorem', 2, None, '___sec59'), + ('Writing our own PCA code', 2, None, '___sec60'), ('Compute the sample mean and center the data', 3, None, - '___sec60'), - ('Compute the sample covariance', 3, None, '___sec61'), + '___sec61'), + ('Compute the sample covariance', 3, None, '___sec62'), ('Diagonalize the sample covariance matrix to obtain the ' 'principal components', 3, None, - '___sec62'), - ('Classical PCA Theorem', 2, None, '___sec63'), - ('Proof of the PCA Theorem', 2, None, '___sec64'), - ('PCA Proof continued', 2, None, '___sec65'), - ('The final step', 2, None, '___sec66'), + '___sec63'), + ('Classical PCA Theorem', 2, None, '___sec64'), + ('Proof of the PCA Theorem', 2, None, '___sec65'), + ('PCA Proof continued', 2, None, '___sec66'), + ('The final step', 2, None, '___sec67'), ('Geometric Interpretation and link with Singular Value ' 'Decomposition', 2, None, - '___sec67'), - ('Principal Component Analysis', 2, None, '___sec68'), - ('PCA and scikit-learn', 2, None, '___sec69'), - ('Back to the Cancer Data', 2, None, '___sec70'), - ('More on the PCA', 2, None, '___sec71'), - ('Incremental PCA', 2, None, '___sec72'), - ('Randomized PCA', 2, None, '___sec73'), - ('Kernel PCA', 2, None, '___sec74'), - ('LLE', 2, None, '___sec75'), - ('Other techniques', 2, None, '___sec76')]} + '___sec68'), + ('Principal Component Analysis', 2, None, '___sec69'), + ('PCA and scikit-learn', 2, None, '___sec70'), + ('Back to the Cancer Data', 2, None, '___sec71'), + ('More on the PCA', 2, None, '___sec72'), + ('Incremental PCA', 2, None, '___sec73'), + ('Randomized PCA', 2, None, '___sec74'), + ('Kernel PCA', 2, None, '___sec75'), + ('LLE', 2, None, '___sec76'), + ('Other techniques', 2, None, '___sec77')]} end of tocinfo --> @@ -230,7 +230,7 @@ MathJax.Hub.Config({
  • Final technicalities II
  • Final technicalities III
  • Final technicalities IV
    • -
  • Backpropagation
    • +
  • Back propagation
  • Gradient descent
  • The code for solving the ODE
  • The network with one input layer, specified number of hidden layers, and one output layer
    • @@ -239,55 +239,56 @@ MathJax.Hub.Config({
  • The trial solution
  • The program using Autograd
  • Using forward Euler to solve the ODE
    • -
  • Example: Solving the one dimensional Poisson equation using Autograd and TensorFlow
    • +
  • Example: Solving the one dimensional Poisson equation
  • The specific equation to solve for
  • Solving the equation using Autograd
  • Comparing with a numerical scheme
    • -
  • Partial Differential Equations
    • -
  • Type of problem
    • -
  • Network requirements
    • -
  • More details
    • -
  • Example: The diffusion equation
    • -
  • Defining the problem
    • -
  • Setting up the network using Autograd
    • -
  • Setting up the network using Autograd; The trial solution
    • -
  • Why the jacobian?
    • -
  • Setting up the network using Autograd; The full program
    • -
  • Example: Solving the wave equation with Neural Networks
    • -
  • The problem to solve for
    • -
  • The trial solution
    • -
  • The analytical solution
    • -
  • Solving the wave equation - the full program using Autograd
    • -
  • Resources on differential equations and deep learning
    • -
  • Friday, Principal Component Analysis
    • -
  • Basic ideas of the Principal Component Analysis (PCA)
    • -
  • Introducing the Covariance and Correlation functions
    • -
  • Correlation Function and Design/Feature Matrix
    • -
  • Covariance Matrix Examples
    • -
  • Correlation Matrix
    • -
  • Correlation Matrix with Pandas
    • -
  • Correlation Matrix with Pandas and the Franke function
    • -
  • Rewriting the Covariance and/or Correlation Matrix
    • -
  • Towards the PCA theorem
    • -
  • The Algorithm before the Theorem
    • -
  • Writing our own PCA code
    • -
  •    Compute the sample mean and center the data
    • -
  •    Compute the sample covariance
    • -
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • -
  • Classical PCA Theorem
    • -
  • Proof of the PCA Theorem
    • -
  • PCA Proof continued
    • -
  • The final step
    • -
  • Geometric Interpretation and link with Singular Value Decomposition
    • -
  • Principal Component Analysis
    • -
  • PCA and scikit-learn
    • -
  • Back to the Cancer Data
    • -
  • More on the PCA
    • -
  • Incremental PCA
    • -
  • Randomized PCA
    • -
  • Kernel PCA
    • -
  • LLE
    • -
  • Other techniques
    • +
  • Setting up the code
    • +
  • Partial Differential Equations
    • +
  • Type of problem
    • +
  • Network requirements
    • +
  • More details
    • +
  • Example: The diffusion equation
    • +
  • Defining the problem
    • +
  • Setting up the network using Autograd
    • +
  • Setting up the network using Autograd; The trial solution
    • +
  • Why the jacobian?
    • +
  • Setting up the network using Autograd; The full program
    • +
  • Example: Solving the wave equation with Neural Networks
    • +
  • The problem to solve for
    • +
  • The trial solution
    • +
  • The analytical solution
    • +
  • Solving the wave equation - the full program using Autograd
    • +
  • Resources on differential equations and deep learning
    • +
  • Friday, Principal Component Analysis
    • +
  • Basic ideas of the Principal Component Analysis (PCA)
    • +
  • Introducing the Covariance and Correlation functions
    • +
  • Correlation Function and Design/Feature Matrix
    • +
  • Covariance Matrix Examples
    • +
  • Correlation Matrix
    • +
  • Correlation Matrix with Pandas
    • +
  • Correlation Matrix with Pandas and the Franke function
    • +
  • Rewriting the Covariance and/or Correlation Matrix
    • +
  • Towards the PCA theorem
    • +
  • The Algorithm before the Theorem
    • +
  • Writing our own PCA code
    • +
  •    Compute the sample mean and center the data
    • +
  •    Compute the sample covariance
    • +
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • +
  • Classical PCA Theorem
    • +
  • Proof of the PCA Theorem
    • +
  • PCA Proof continued
    • +
  • The final step
    • +
  • Geometric Interpretation and link with Singular Value Decomposition
    • +
  • Principal Component Analysis
    • +
  • PCA and scikit-learn
    • +
  • Back to the Cancer Data
    • +
  • More on the PCA
    • +
  • Incremental PCA
    • +
  • Randomized PCA
    • +
  • Kernel PCA
    • +
  • LLE
    • +
  • Other techniques
    • @@ -347,7 +348,7 @@ In this example, we let \( \alpha = 2 \), \( A = 1 \), and \( g_0 = 1.2 \).
  • 35
  • 36
  • ...
    • -
  • 76
    • +
  • 77
  • »
    • diff --git a/doc/pub/week43/html/week43-bs.html b/doc/pub/week43/html/week43-bs.html index 6340568db..94361c0f2 100644 --- a/doc/pub/week43/html/week43-bs.html +++ b/doc/pub/week43/html/week43-bs.html @@ -67,7 +67,7 @@ Automatically generated HTML file from DocOnce source ('Final technicalities II', 2, None, '___sec16'), ('Final technicalities III', 2, None, '___sec17'), ('Final technicalities IV', 2, None, '___sec18'), - ('Backpropagation', 2, None, '___sec19'), + ('Back propagation', 2, None, '___sec19'), ('Gradient descent', 2, None, '___sec20'), ('The code for solving the ODE', 2, None, '___sec21'), ('The network with one input layer, specified number of hidden ' @@ -80,100 +80,100 @@ Automatically generated HTML file from DocOnce source ('The trial solution', 2, None, '___sec25'), ('The program using Autograd', 2, None, '___sec26'), ('Using forward Euler to solve the ODE', 2, None, '___sec27'), - ('Example: Solving the one dimensional Poisson equation using ' - 'Autograd and TensorFlow', + ('Example: Solving the one dimensional Poisson equation', 2, None, '___sec28'), ('The specific equation to solve for', 2, None, '___sec29'), ('Solving the equation using Autograd', 2, None, '___sec30'), ('Comparing with a numerical scheme', 2, None, '___sec31'), - ('Partial Differential Equations', 2, None, '___sec32'), - ('Type of problem', 2, None, '___sec33'), - ('Network requirements', 2, None, '___sec34'), - ('More details', 2, None, '___sec35'), - ('Example: The diffusion equation', 2, None, '___sec36'), - ('Defining the problem', 2, None, '___sec37'), - ('Setting up the network using Autograd', 2, None, '___sec38'), + ('Setting up the code', 2, None, '___sec32'), + ('Partial Differential Equations', 2, None, '___sec33'), + ('Type of problem', 2, None, '___sec34'), + ('Network requirements', 2, None, '___sec35'), + ('More details', 2, None, '___sec36'), + ('Example: The diffusion equation', 2, None, '___sec37'), + ('Defining the problem', 2, None, '___sec38'), + ('Setting up the network using Autograd', 2, None, '___sec39'), ('Setting up the network using Autograd; The trial solution', 2, None, - '___sec39'), - ('Why the jacobian?', 2, None, '___sec40'), + '___sec40'), + ('Why the jacobian?', 2, None, '___sec41'), ('Setting up the network using Autograd; The full program', 2, None, - '___sec41'), + '___sec42'), ('Example: Solving the wave equation with Neural Networks', 2, None, - '___sec42'), - ('The problem to solve for', 2, None, '___sec43'), - ('The trial solution', 2, None, '___sec44'), - ('The analytical solution', 2, None, '___sec45'), + '___sec43'), + ('The problem to solve for', 2, None, '___sec44'), + ('The trial solution', 2, None, '___sec45'), + ('The analytical solution', 2, None, '___sec46'), ('Solving the wave equation - the full program using Autograd', 2, None, - '___sec46'), + '___sec47'), ('Resources on differential equations and deep learning', 2, None, - '___sec47'), - ('Friday, Principal Component Analysis', 2, None, '___sec48'), + '___sec48'), + ('Friday, Principal Component Analysis', 2, None, '___sec49'), ('Basic ideas of the Principal Component Analysis (PCA)', 2, None, - '___sec49'), + '___sec50'), ('Introducing the Covariance and Correlation functions', 2, None, - '___sec50'), + '___sec51'), ('Correlation Function and Design/Feature Matrix', 2, None, - '___sec51'), - ('Covariance Matrix Examples', 2, None, '___sec52'), - ('Correlation Matrix', 2, None, '___sec53'), - ('Correlation Matrix with Pandas', 2, None, '___sec54'), + '___sec52'), + ('Covariance Matrix Examples', 2, None, '___sec53'), + ('Correlation Matrix', 2, None, '___sec54'), + ('Correlation Matrix with Pandas', 2, None, '___sec55'), ('Correlation Matrix with Pandas and the Franke function', 2, None, - '___sec55'), + '___sec56'), ('Rewriting the Covariance and/or Correlation Matrix', 2, None, - '___sec56'), - ('Towards the PCA theorem', 2, None, '___sec57'), - ('The Algorithm before the Theorem', 2, None, '___sec58'), - ('Writing our own PCA code', 2, None, '___sec59'), + '___sec57'), + ('Towards the PCA theorem', 2, None, '___sec58'), + ('The Algorithm before the Theorem', 2, None, '___sec59'), + ('Writing our own PCA code', 2, None, '___sec60'), ('Compute the sample mean and center the data', 3, None, - '___sec60'), - ('Compute the sample covariance', 3, None, '___sec61'), + '___sec61'), + ('Compute the sample covariance', 3, None, '___sec62'), ('Diagonalize the sample covariance matrix to obtain the ' 'principal components', 3, None, - '___sec62'), - ('Classical PCA Theorem', 2, None, '___sec63'), - ('Proof of the PCA Theorem', 2, None, '___sec64'), - ('PCA Proof continued', 2, None, '___sec65'), - ('The final step', 2, None, '___sec66'), + '___sec63'), + ('Classical PCA Theorem', 2, None, '___sec64'), + ('Proof of the PCA Theorem', 2, None, '___sec65'), + ('PCA Proof continued', 2, None, '___sec66'), + ('The final step', 2, None, '___sec67'), ('Geometric Interpretation and link with Singular Value ' 'Decomposition', 2, None, - '___sec67'), - ('Principal Component Analysis', 2, None, '___sec68'), - ('PCA and scikit-learn', 2, None, '___sec69'), - ('Back to the Cancer Data', 2, None, '___sec70'), - ('More on the PCA', 2, None, '___sec71'), - ('Incremental PCA', 2, None, '___sec72'), - ('Randomized PCA', 2, None, '___sec73'), - ('Kernel PCA', 2, None, '___sec74'), - ('LLE', 2, None, '___sec75'), - ('Other techniques', 2, None, '___sec76')]} + '___sec68'), + ('Principal Component Analysis', 2, None, '___sec69'), + ('PCA and scikit-learn', 2, None, '___sec70'), + ('Back to the Cancer Data', 2, None, '___sec71'), + ('More on the PCA', 2, None, '___sec72'), + ('Incremental PCA', 2, None, '___sec73'), + ('Randomized PCA', 2, None, '___sec74'), + ('Kernel PCA', 2, None, '___sec75'), + ('LLE', 2, None, '___sec76'), + ('Other techniques', 2, None, '___sec77')]} end of tocinfo --> @@ -230,7 +230,7 @@ MathJax.Hub.Config({
  • Final technicalities II
  • Final technicalities III
  • Final technicalities IV
    • -
  • Backpropagation
    • +
  • Back propagation
  • Gradient descent
  • The code for solving the ODE
  • The network with one input layer, specified number of hidden layers, and one output layer
    • @@ -239,55 +239,56 @@ MathJax.Hub.Config({
  • The trial solution
  • The program using Autograd
  • Using forward Euler to solve the ODE
    • -
  • Example: Solving the one dimensional Poisson equation using Autograd and TensorFlow
    • +
  • Example: Solving the one dimensional Poisson equation
  • The specific equation to solve for
  • Solving the equation using Autograd
  • Comparing with a numerical scheme
    • -
  • Partial Differential Equations
    • -
  • Type of problem
    • -
  • Network requirements
    • -
  • More details
    • -
  • Example: The diffusion equation
    • -
  • Defining the problem
    • -
  • Setting up the network using Autograd
    • -
  • Setting up the network using Autograd; The trial solution
    • -
  • Why the jacobian?
    • -
  • Setting up the network using Autograd; The full program
    • -
  • Example: Solving the wave equation with Neural Networks
    • -
  • The problem to solve for
    • -
  • The trial solution
    • -
  • The analytical solution
    • -
  • Solving the wave equation - the full program using Autograd
    • -
  • Resources on differential equations and deep learning
    • -
  • Friday, Principal Component Analysis
    • -
  • Basic ideas of the Principal Component Analysis (PCA)
    • -
  • Introducing the Covariance and Correlation functions
    • -
  • Correlation Function and Design/Feature Matrix
    • -
  • Covariance Matrix Examples
    • -
  • Correlation Matrix
    • -
  • Correlation Matrix with Pandas
    • -
  • Correlation Matrix with Pandas and the Franke function
    • -
  • Rewriting the Covariance and/or Correlation Matrix
    • -
  • Towards the PCA theorem
    • -
  • The Algorithm before the Theorem
    • -
  • Writing our own PCA code
    • -
  •    Compute the sample mean and center the data
    • -
  •    Compute the sample covariance
    • -
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • -
  • Classical PCA Theorem
    • -
  • Proof of the PCA Theorem
    • -
  • PCA Proof continued
    • -
  • The final step
    • -
  • Geometric Interpretation and link with Singular Value Decomposition
    • -
  • Principal Component Analysis
    • -
  • PCA and scikit-learn
    • -
  • Back to the Cancer Data
    • -
  • More on the PCA
    • -
  • Incremental PCA
    • -
  • Randomized PCA
    • -
  • Kernel PCA
    • -
  • LLE
    • -
  • Other techniques
    • +
  • Setting up the code
    • +
  • Partial Differential Equations
    • +
  • Type of problem
    • +
  • Network requirements
    • +
  • More details
    • +
  • Example: The diffusion equation
    • +
  • Defining the problem
    • +
  • Setting up the network using Autograd
    • +
  • Setting up the network using Autograd; The trial solution
    • +
  • Why the jacobian?
    • +
  • Setting up the network using Autograd; The full program
    • +
  • Example: Solving the wave equation with Neural Networks
    • +
  • The problem to solve for
    • +
  • The trial solution
    • +
  • The analytical solution
    • +
  • Solving the wave equation - the full program using Autograd
    • +
  • Resources on differential equations and deep learning
    • +
  • Friday, Principal Component Analysis
    • +
  • Basic ideas of the Principal Component Analysis (PCA)
    • +
  • Introducing the Covariance and Correlation functions
    • +
  • Correlation Function and Design/Feature Matrix
    • +
  • Covariance Matrix Examples
    • +
  • Correlation Matrix
    • +
  • Correlation Matrix with Pandas
    • +
  • Correlation Matrix with Pandas and the Franke function
    • +
  • Rewriting the Covariance and/or Correlation Matrix
    • +
  • Towards the PCA theorem
    • +
  • The Algorithm before the Theorem
    • +
  • Writing our own PCA code
    • +
  •    Compute the sample mean and center the data
    • +
  •    Compute the sample covariance
    • +
  •    Diagonalize the sample covariance matrix to obtain the principal components
    • +
  • Classical PCA Theorem
    • +
  • Proof of the PCA Theorem
    • +
  • PCA Proof continued
    • +
  • The final step
    • +
  • Geometric Interpretation and link with Singular Value Decomposition
    • +
  • Principal Component Analysis
    • +
  • PCA and scikit-learn
    • +
  • Back to the Cancer Data
    • +
  • More on the PCA
    • +
  • Incremental PCA
    • +
  • Randomized PCA
    • +
  • Kernel PCA
    • +
  • LLE
    • +
  • Other techniques
    • @@ -346,7 +347,7 @@ MathJax.Hub.Config({
  • 9
  • 10
  • ...
    • -
  • 76
    • +
  • 77
  • »
    • diff --git a/doc/pub/week43/html/week43-reveal.html b/doc/pub/week43/html/week43-reveal.html index 0441ffdf7..215903f9e 100644 --- a/doc/pub/week43/html/week43-reveal.html +++ b/doc/pub/week43/html/week43-reveal.html @@ -166,12 +166,16 @@ MathJax.Hub.Config({

    -Reading suggestions for both days: "Aurelien Geron's chapters 8 +We will also study the usage of Autograd in computing gradients for deep learning. For the documentation of Autograd and examples see the lectures slides from week 40 and the Autograd doucmentation.

    Recurrent Neural Networks

    + +

    +Overview video. +See also lecture on Thursday October 22 and examples from week 42.

    @@ -584,7 +588,7 @@ $$ It is possible to use other activations functions for the hidden layer also.

    -The output $\boldsymbol{x}_i^{\text{hidden} }$from each \( i \)-th hidden neuron is: +The output \( \boldsymbol{x}_i^{\text{hidden}} \) from each \( i \)-th hidden neuron is:

     
    $$ @@ -596,7 +600,11 @@ $$ The outputs \( \boldsymbol{x}_i^{\text{hidden} } \) are then sent to the output layer.

    -The output layer consists of one neuron in this case, and combines the output from each of the neurons in the hidden layers. The output layer combines the results from the hidden layer using some weights $ w_i^{\text{output}}$ and biases \( b_i^{\text{output}} \). In this case, it is assumes that the number of neurons in the output layer is one. +The output layer consists of one neuron in this case, and combines the +output from each of the neurons in the hidden layers. The output layer +combines the results from the hidden layer using some weights \( w_i^{\text{output}} \) +and biases \( b_i^{\text{output}} \). In this case, +it is assumes that the number of neurons in the output layer is one. @@ -648,7 +656,7 @@ In this case we seek a continuous range of values since we are approximating a f

    -

    Backpropagation

    +

    Back propagation

    The next step is to decide how the parameters should be changed such that they minimize the cost function. @@ -695,7 +703,7 @@ for a number of iterations or until $ \big|\big| \boldsymbol{\omega}_{\text{new} The value of \( \lambda \) decides how large steps the algorithm must take in the direction of $ \nabla_{\boldsymbol{\omega}} C(\boldsymbol{x}, \boldsymbol{\omega})$. The notation \( \nabla_{\boldsymbol{\omega}} \) express the gradient with respect -to the elements in \( \boldsymbol{\omega \). +to the elements in \( \boldsymbol{\omega} \).

    In our case, we have to minimize the cost function \( C(\boldsymbol{x}, P) \) with @@ -1067,8 +1075,8 @@ Also, at \( t = 0 \) the population has the size \( g(0) = g_0 \), where \( g_0

    In this example, similar network as for the exponential decay using Autograd has been used to solve the equation. However, as the implementation might suffer from e.g numerical instability and high execution time (this might be more apparent in the examples solving PDEs), -a network has been constructed using TensorFlow also. -For comparison, the forward Euler method has been implemented in order to see how the networks performs compared to a numerical scheme. +using a library like TensorFlow is recommended. +Here, we stay with a more simple approach and implement for comparison, the simple forward Euler method.

    @@ -1305,7 +1313,7 @@ The network will be the similar as for the exponential decay example, but with s

    Using forward Euler to solve the ODE

    -A straight-forward way of solving an ODE numerically, is to use Euler's method. +A straightforward way of solving an ODE numerically, is to use Euler's method.

    Euler's method uses Taylor series to approximate the value at a function \( f \) at a step \( \Delta x \) from \( x \): @@ -1436,13 +1444,11 @@ extending the program that uses the network using Autograd: plt.show() -

    -Running the program gives

    -

    Example: Solving the one dimensional Poisson equation using Autograd and TensorFlow

    +

    Example: Solving the one dimensional Poisson equation

    The Poisson equation for \( g(x) \) in one dimension is @@ -1782,13 +1788,18 @@ f(x_2) \\ f(x_{N_x - 3}) \\ f(x_{N_x - 2}) \end{pmatrix} \\ -A\boldsymbol{g} &= \boldsymbol{f} +\boldsymbol{A}\boldsymbol{g} &= \boldsymbol{f}, \end{aligned} $$

     

    which makes it possible to solve for the vector \( \boldsymbol{g} \). +

    + + +
    +

    Setting up the code

    We can then compare the result from this numerical scheme with the output from our network using Autograd: @@ -1987,23 +1998,17 @@ We can then compare the result from this numerical scheme with the output from o plt.legend(['numerical scheme','analytical','dnn']) plt.show() -

    -The program prints out: -

    - - -

    The max absolute difference between the analytical solution and DNN Autograd: 0.000464088
-The max absolute difference between the analytical solution and numerical scheme: 0.00266858
-
    -

    Partial Differential Equations

    +

    Partial Differential Equations

    -A partial differential equation (PDE) has a solution here the function is defined by multiple variables. -The equation may involve all kinds of combinations of which variables the function is differentiated with respect to. +A partial differential equation (PDE) has a solution here the function +is defined by multiple variables. The equation may involve all kinds +of combinations of which variables the function is differentiated with +respect to.

    In general, a partial differential equation for a function \( g(x_1,\dots,x_N) \) with \( N \) variables may be expressed as @@ -2022,7 +2027,7 @@ where \( f \) is an expression involving all kinds of possible mixed derivatives

    -

    Type of problem

    +

    Type of problem

    The problem our network must solve for, is similar to the ODE case. @@ -2047,7 +2052,7 @@ The role of the function \( h_2(x_1,\dots,x_N,N(x_1,\dots,x_N,P)) \), is to ensu

    -

    Network requirements

    +

    Network requirements

    The network tries then the minimize the cost function following the @@ -2064,7 +2069,7 @@ minimize is

     
    $$ \begin{equation*} -c\left(x_1, \dots, x_N, P\right) = \left( f\left(x_1, \, \dots \, , x_N, \frac{\partial g(x_1,\dots,x_N) }{\partial x_1}, \dots , \frac{\partial g(x_1,\dots,x_N) }{\partial x_N}, \frac{\partial g(x_1,\dots,x_N) }{\partial x_1\partial x_2}, \, \dots \, , \frac{\partial^n g(x_1,\dots,x_N) }{\partial x_N^n} \right) \right)^2 +C\left(x_1, \dots, x_N, P\right) = \left( f\left(x_1, \, \dots \, , x_N, \frac{\partial g(x_1,\dots,x_N) }{\partial x_1}, \dots , \frac{\partial g(x_1,\dots,x_N) }{\partial x_N}, \frac{\partial g(x_1,\dots,x_N) }{\partial x_1\partial x_2}, \, \dots \, , \frac{\partial^n g(x_1,\dots,x_N) }{\partial x_N^n} \right) \right)^2 \end{equation*} $$

     
    @@ -2072,15 +2077,13 @@ $$

    -

    More details

    +

    More details

    -If we let \( \boldsymbol{x = \big( x_1, \dots, x_N \big) \) be an array containing the values for \( x_1, \dots, x_N \) respectively, the cost function can be reformulated into the following: +If we let \( \boldsymbol{x} = \big( x_1, \dots, x_N \big) \) be an array containing the values for \( x_1, \dots, x_N \) respectively, the cost function can be reformulated into the following:

     
    $$ -\begin{equation*} - c\left(\boldsymbol{x}, P\right) = f\left( \left( \boldsymbol{x}, \frac{\partial g(\boldsymbol{x) }{\partial x_1}, \dots , \frac{\partial g(\boldsymbol{x) }{\partial x_N}, \frac{\partial g(\boldsymbol{x) }{\partial x_1\partial x_2}, \, \dots \, , \frac{\partial^n g(\boldsymbol{x) }{\partial x_N^n} \right) \right)^2 -\end{equation*} + C\left(\boldsymbol{x}, P\right) = f\left( \left( \boldsymbol{x}, \frac{\partial g(\boldsymbol{x}) }{\partial x_1}, \dots , \frac{\partial g(\boldsymbol{x}) }{\partial x_N}, \frac{\partial g(\boldsymbol{x}) }{\partial x_1\partial x_2}, \, \dots \, , \frac{\partial^n g(\boldsymbol{x}) }{\partial x_N^n} \right) \right)^2 $$

     
    @@ -2089,7 +2092,7 @@ If we also have \( M \) different sets of values for \( x_1, \dots, x_N \), that

     
    $$ \begin{equation*} -c\left(X, P \right) = \sum_{i=1}^M f\left( \left( \boldsymbol{x}_i, \frac{\partial g(\boldsymbol{x}_i) }{\partial x_1}, \dots , \frac{\partial g(\boldsymbol{x}_i) }{\partial x_N}, \frac{\partial g(\boldsymbol{x}_i) }{\partial x_1\partial x_2}, \, \dots \, , \frac{\partial^n g(\boldsymbol{x}_i) }{\partial x_N^n} \right) \right)^2 +C\left(X, P \right) = \sum_{i=1}^M f\left( \left( \boldsymbol{x}_i, \frac{\partial g(\boldsymbol{x}_i) }{\partial x_1}, \dots , \frac{\partial g(\boldsymbol{x}_i) }{\partial x_N}, \frac{\partial g(\boldsymbol{x}_i) }{\partial x_1\partial x_2}, \, \dots \, , \frac{\partial^n g(\boldsymbol{x}_i) }{\partial x_N^n} \right) \right)^2. \end{equation*} $$

     
    @@ -2097,7 +2100,7 @@ $$

    -

    Example: The diffusion equation

    +

    Example: The diffusion equation

    In one spatial dimension, the equation reads @@ -2126,7 +2129,7 @@ with \( u(x) \) being some given function.

    -

    Defining the problem

    +

    Defining the problem

    For this case, we want to find \( g(x,t) \) such that @@ -2162,7 +2165,7 @@ First, we will look into how Autograd could be used in a network tailored to sol

    -

    Setting up the network using Autograd

    +

    Setting up the network using Autograd

    The only change to do here, is to extend our network such that @@ -2225,7 +2228,7 @@ network at each possible pair \( (x,t) \), given an array for the desired

    -

    Setting up the network using Autograd; The trial solution

    +

    Setting up the network using Autograd; The trial solution

    The cost function must then iterate through the given arrays @@ -2258,7 +2261,7 @@ since \( (0) = u(1) = 0 \) and \( u(x) = \sin(\pi x) \).

    -

    Why the jacobian?

    +

    Why the jacobian?

    The Jacobian is used because the program must find the derivative of @@ -2326,7 +2329,7 @@ mixed derivatives of \( g(x,t) \).

    -

    Setting up the network using Autograd; The full program

    +

    Setting up the network using Autograd; The full program

    Having set up the network, along with the trial solution and cost function, we can now see how the deep neural network performs by comparing the results to the analytical solution. @@ -2580,7 +2583,7 @@ Using TensorFlow results in a much better execution time. Try it!

    -

    Example: Solving the wave equation with Neural Networks

    +

    Example: Solving the wave equation with Neural Networks

    The wave equation is @@ -2613,7 +2616,7 @@ where \( \frac{\partial g(x,t)}{\partial t} \Big |_{t = 0} \) means the derivati

    -

    The problem to solve for

    +

    The problem to solve for

    The wave equation to solve for, is @@ -2647,7 +2650,7 @@ In this example, let \( c = 1 \) and \( u(x) = \sin(\pi x) \) and \( v(x) = -\pi

    -

    The trial solution

    +

    The trial solution

    Setting up the network is done in similar matter as for the example of solving the diffusion equation. The only things we have to change, is the trial solution such that it satisfies the conditions from (20) and the cost function. @@ -2675,7 +2678,7 @@ Note that this trial solution satisfies the conditions only if \( u(0) = v(0) =
    -

    The analytical solution

    +

    The analytical solution

    The analytical solution for our specific problem, is @@ -2689,7 +2692,7 @@ $$

    -

    Solving the wave equation - the full program using Autograd

    +

    Solving the wave equation - the full program using Autograd

    @@ -2919,7 +2922,7 @@ $$

    -

    Resources on differential equations and deep learning

    +

    Resources on differential equations and deep learning

    1. Artificial neural networks for solving ordinary and partial differential equations by I.E. Lagaris et al
      2. @@ -2931,12 +2934,15 @@ $$
      -

      Friday, Principal Component Analysis

      +

      Friday, Principal Component Analysis

      + +

      +Overview video

      -

      Basic ideas of the Principal Component Analysis (PCA)

      +

      Basic ideas of the Principal Component Analysis (PCA)

      The principal component analysis deals with the problem of fitting a @@ -2959,7 +2965,7 @@ We have a data set defined by a design/feature matrix \( \boldsymbol{X} \) (see

      -

      Introducing the Covariance and Correlation functions

      +

      Introducing the Covariance and Correlation functions

      Before we discuss the PCA theorem, we need to remind ourselves about @@ -3032,7 +3038,7 @@ In the above example this is the function we constructed using pandas.

      -

      Correlation Function and Design/Feature Matrix

      +

      Correlation Function and Design/Feature Matrix

      In our derivation of the various regression algorithms like Ordinary Least Squares or Ridge regression @@ -3103,7 +3109,7 @@ $$

      -

      Covariance Matrix Examples

      +

      Covariance Matrix Examples

      The Numpy function np.cov calculates the covariance elements using @@ -3149,7 +3155,7 @@ C = np.cov(W)

      -

      Correlation Matrix

      +

      Correlation Matrix

      The previous example can be converted into the correlation matrix by @@ -3194,7 +3200,7 @@ The above procedure with numpy can be made more compact if we use pand

      -

      Correlation Matrix with Pandas

      +

      Correlation Matrix with Pandas

      We whow here how we can set up the correlation matrix using pandas, as done in this simple code @@ -3221,7 +3227,7 @@ We expand this model to the Franke function discussed above.

      -

      Correlation Matrix with Pandas and the Franke function

      +

      Correlation Matrix with Pandas and the Franke function

      @@ -3284,7 +3290,7 @@ matrix without these elements.

      -

      Rewriting the Covariance and/or Correlation Matrix

      +

      Rewriting the Covariance and/or Correlation Matrix

      We can rewrite the covariance matrix in a more compact form in terms of the design/feature matrix \( \boldsymbol{X} \) as @@ -3337,7 +3343,7 @@ It is easy to generalize this to a matrix \( \boldsymbol{X}\in {\mathbb{R}}^{n\t

      -

      Towards the PCA theorem

      +

      Towards the PCA theorem

      We have that the covariance matrix (the correlation matrix involves a simple rescaling) is given as @@ -3396,7 +3402,7 @@ features/predictors.

      -

      The Algorithm before theorem

      +

      The Algorithm before theorem

      Here's how we would proceed in setting up the algorithm for the PCA, see also discussion below here. @@ -3429,7 +3435,7 @@ $$

      -

      Writing our own PCA code

      +

      Writing our own PCA code

      We will use a simple example first with two-dimensional data @@ -3464,7 +3470,7 @@ X = np.random.multivariate_normal(mean, cov, n)

      Now we are going to implement the PCA algorithm. We will break it down into various substeps. -

      Compute the sample mean and center the data

      +

      Compute the sample mean and center the data

      The first step of PCA is to compute the sample mean of the data and use it to center the data. Recall that the sample mean is @@ -3505,7 +3511,7 @@ variance. The diagonal covariance matrix elements will then be one, while the non-diagonal ones need to be divided by \( 2\sqrt{2} \) for our specific case. -

      Compute the sample covariance

      +

      Compute the sample covariance

      Now we are going to use the mean centered data to compute the sample covariance of the data by using the following equation @@ -3549,7 +3555,7 @@ plt.show() Depending on the number of points \( n \), we will get results that are close to the covariance values defined above. The plot shows how the data are clustered around a line with slope close to one. Is this expected? -

      Diagonalize the sample covariance matrix to obtain the principal components

      +

      Diagonalize the sample covariance matrix to obtain the principal components

      Now we are ready to solve for the principal components! To do so we @@ -3614,7 +3620,7 @@ This code does not contain all the above elements, but it shows how we can use <

      -

      Classical PCA Theorem

      +

      Classical PCA Theorem

      We assume now that we have a design matrix \( \boldsymbol{X} \) which has been @@ -3649,7 +3655,7 @@ The proof which follows will be updated by mid January 2020.

      -

      Proof of the PCA Theorem

      +

      Proof of the PCA Theorem

      To show the PCA theorem let us start with the assumption that there is one vector \( \boldsymbol{w}_0 \) which corresponds to a solution which minimized the reconstruction error \( J \). This is an orthogonal vector. It means that we now approximate the reconstruction error in terms of \( \boldsymbol{w}_0 \) and \( \boldsymbol{z}_0 \) as @@ -3678,7 +3684,7 @@ where the vectors on the rhs are known.

      -

      PCA Proof continued

      +

      PCA Proof continued

      We have now found the unknown parameters \( z_{i0} \). These correspond to the projected coordinates and we can write @@ -3729,7 +3735,7 @@ of the projected data.

      -

      The final step

      +

      The final step

      We could trivially maximize the variance of the projection (and @@ -3791,7 +3797,7 @@ Additional part of the proof for the other eigenvectors will be added by mid Jan

      -

      Geometric Interpretation and link with Singular Value Decomposition

      +

      Geometric Interpretation and link with Singular Value Decomposition

      This material will be added by mid January 2020. @@ -3799,7 +3805,7 @@ This material will be added by mid January 2020.

      -

      Principal Component Analysis

      +

      Principal Component Analysis

      Principal Component Analysis (PCA) is by far the most popular dimensionality reduction algorithm. @@ -3855,7 +3861,7 @@ X2D = X_centered.dot(W2)

      -

      PCA and scikit-learn

      +

      PCA and scikit-learn

      Scikit-Learn’s PCA class implements PCA using SVD decomposition just like we did before. The @@ -3887,7 +3893,7 @@ variance that lies along the axis of each principal component.

      -

      Back to the Cancer Data

      +

      Back to the Cancer Data

      We can now repeat the above but applied to real data, in this case our breast cancer data. Here we compute performance scores on the training data using logistic regression.

      @@ -3928,7 +3934,7 @@ We see that our training data after the PCA decomposition has a performance simi

      -

      More on the PCA

      +

      More on the PCA

      Instead of arbitrarily choosing the number of dimensions to reduce down to, it is generally preferable to @@ -3959,7 +3965,7 @@ X_reduced = pca.fit_transform(X)

      -

      Incremental PCA

      +

      Incremental PCA

      One problem with the preceding implementation of PCA is that it requires the whole training set to fit in @@ -3971,7 +3977,7 @@ instances arrive).

      -

      Randomized PCA

      +

      Randomized PCA

      Scikit-Learn offers yet another option to perform PCA, called Randomized PCA. This is a stochastic @@ -3982,7 +3988,7 @@ previous algorithms when \( d \) is much smaller than \( n \).

      -

      Kernel PCA

      +

      Kernel PCA

      @@ -4008,7 +4014,7 @@ X_reduced = rbf_pca.fit_transform(X)

      -

      LLE

      +

      LLE

      Locally Linear Embedding (LLE) is another very powerful nonlinear dimensionality reduction @@ -4020,7 +4026,7 @@ these local relationships are best preserved (more details shortly).

      -

      Other techniques

      +

      Other techniques

      There are many other dimensionality reduction techniques, several of which are available in Scikit-Learn. diff --git a/doc/pub/week43/html/week43-solarized.html b/doc/pub/week43/html/week43-solarized.html index d561e511f..164136444 100644 --- a/doc/pub/week43/html/week43-solarized.html +++ b/doc/pub/week43/html/week43-solarized.html @@ -87,7 +87,7 @@ div { text-align: justify; text-justify: inter-word; } ('Final technicalities II', 2, None, '___sec16'), ('Final technicalities III', 2, None, '___sec17'), ('Final technicalities IV', 2, None, '___sec18'), - ('Backpropagation', 2, None, '___sec19'), + ('Back propagation', 2, None, '___sec19'), ('Gradient descent', 2, None, '___sec20'), ('The code for solving the ODE', 2, None, '___sec21'), ('The network with one input layer, specified number of hidden ' @@ -100,100 +100,100 @@ div { text-align: justify; text-justify: inter-word; } ('The trial solution', 2, None, '___sec25'), ('The program using Autograd', 2, None, '___sec26'), ('Using forward Euler to solve the ODE', 2, None, '___sec27'), - ('Example: Solving the one dimensional Poisson equation using ' - 'Autograd and TensorFlow', + ('Example: Solving the one dimensional Poisson equation', 2, None, '___sec28'), ('The specific equation to solve for', 2, None, '___sec29'), ('Solving the equation using Autograd', 2, None, '___sec30'), ('Comparing with a numerical scheme', 2, None, '___sec31'), - ('Partial Differential Equations', 2, None, '___sec32'), - ('Type of problem', 2, None, '___sec33'), - ('Network requirements', 2, None, '___sec34'), - ('More details', 2, None, '___sec35'), - ('Example: The diffusion equation', 2, None, '___sec36'), - ('Defining the problem', 2, None, '___sec37'), - ('Setting up the network using Autograd', 2, None, '___sec38'), + ('Setting up the code', 2, None, '___sec32'), + ('Partial Differential Equations', 2, None, '___sec33'), + ('Type of problem', 2, None, '___sec34'), + ('Network requirements', 2, None, '___sec35'), + ('More details', 2, None, '___sec36'), + ('Example: The diffusion equation', 2, None, '___sec37'), + ('Defining the problem', 2, None, '___sec38'), + ('Setting up the network using Autograd', 2, None, '___sec39'), ('Setting up the network using Autograd; The trial solution', 2, None, - '___sec39'), - ('Why the jacobian?', 2, None, '___sec40'), + '___sec40'), + ('Why the jacobian?', 2, None, '___sec41'), ('Setting up the network using Autograd; The full program', 2, None, - '___sec41'), + '___sec42'), ('Example: Solving the wave equation with Neural Networks', 2, None, - '___sec42'), - ('The problem to solve for', 2, None, '___sec43'), - ('The trial solution', 2, None, '___sec44'), - ('The analytical solution', 2, None, '___sec45'), + '___sec43'), + ('The problem to solve for', 2, None, '___sec44'), + ('The trial solution', 2, None, '___sec45'), + ('The analytical solution', 2, None, '___sec46'), ('Solving the wave equation - the full program using Autograd', 2, None, - '___sec46'), + '___sec47'), ('Resources on differential equations and deep learning', 2, None, - '___sec47'), - ('Friday, Principal Component Analysis', 2, None, '___sec48'), + '___sec48'), + ('Friday, Principal Component Analysis', 2, None, '___sec49'), ('Basic ideas of the Principal Component Analysis (PCA)', 2, None, - '___sec49'), + '___sec50'), ('Introducing the Covariance and Correlation functions', 2, None, - '___sec50'), + '___sec51'), ('Correlation Function and Design/Feature Matrix', 2, None, - '___sec51'), - ('Covariance Matrix Examples', 2, None, '___sec52'), - ('Correlation Matrix', 2, None, '___sec53'), - ('Correlation Matrix with Pandas', 2, None, '___sec54'), + '___sec52'), + ('Covariance Matrix Examples', 2, None, '___sec53'), + ('Correlation Matrix', 2, None, '___sec54'), + ('Correlation Matrix with Pandas', 2, None, '___sec55'), ('Correlation Matrix with Pandas and the Franke function', 2, None, - '___sec55'), + '___sec56'), ('Rewriting the Covariance and/or Correlation Matrix', 2, None, - '___sec56'), - ('Towards the PCA theorem', 2, None, '___sec57'), - ('The Algorithm before the Theorem', 2, None, '___sec58'), - ('Writing our own PCA code', 2, None, '___sec59'), + '___sec57'), + ('Towards the PCA theorem', 2, None, '___sec58'), + ('The Algorithm before the Theorem', 2, None, '___sec59'), + ('Writing our own PCA code', 2, None, '___sec60'), ('Compute the sample mean and center the data', 3, None, - '___sec60'), - ('Compute the sample covariance', 3, None, '___sec61'), + '___sec61'), + ('Compute the sample covariance', 3, None, '___sec62'), ('Diagonalize the sample covariance matrix to obtain the ' 'principal components', 3, None, - '___sec62'), - ('Classical PCA Theorem', 2, None, '___sec63'), - ('Proof of the PCA Theorem', 2, None, '___sec64'), - ('PCA Proof continued', 2, None, '___sec65'), - ('The final step', 2, None, '___sec66'), + '___sec63'), + ('Classical PCA Theorem', 2, None, '___sec64'), + ('Proof of the PCA Theorem', 2, None, '___sec65'), + ('PCA Proof continued', 2, None, '___sec66'), + ('The final step', 2, None, '___sec67'), ('Geometric Interpretation and link with Singular Value ' 'Decomposition', 2, None, - '___sec67'), - ('Principal Component Analysis', 2, None, '___sec68'), - ('PCA and scikit-learn', 2, None, '___sec69'), - ('Back to the Cancer Data', 2, None, '___sec70'), - ('More on the PCA', 2, None, '___sec71'), - ('Incremental PCA', 2, None, '___sec72'), - ('Randomized PCA', 2, None, '___sec73'), - ('Kernel PCA', 2, None, '___sec74'), - ('LLE', 2, None, '___sec75'), - ('Other techniques', 2, None, '___sec76')]} + '___sec68'), + ('Principal Component Analysis', 2, None, '___sec69'), + ('PCA and scikit-learn', 2, None, '___sec70'), + ('Back to the Cancer Data', 2, None, '___sec71'), + ('More on the PCA', 2, None, '___sec72'), + ('Incremental PCA', 2, None, '___sec73'), + ('Randomized PCA', 2, None, '___sec74'), + ('Kernel PCA', 2, None, '___sec75'), + ('LLE', 2, None, '___sec76'), + ('Other techniques', 2, None, '___sec77')]} end of tocinfo --> @@ -245,13 +245,17 @@ MathJax.Hub.Config({

    2. Friday: Principal Component Analysis and Dimensionality Reduction
      3. -Reading suggestions for both days: "Aurelien Geron's chapters 8 +We will also study the usage of Autograd in computing gradients for deep learning. For the documentation of Autograd and examples see the lectures slides from week 40 and the Autograd doucmentation.











      Recurrent Neural Networks

      +

      +Overview video. +See also lecture on Thursday October 22 and examples from week 42. +











      @@ -625,7 +629,7 @@ $$ It is possible to use other activations functions for the hidden layer also.

      -The output $\boldsymbol{x}_i^{\text{hidden} }$from each \( i \)-th hidden neuron is: +The output \( \boldsymbol{x}_i^{\text{hidden}} \) from each \( i \)-th hidden neuron is: $$ \boldsymbol{x}_i^{\text{hidden} } = f\big( \boldsymbol{z}_{i}^{\text{hidden}} \big) @@ -635,7 +639,11 @@ $$ The outputs \( \boldsymbol{x}_i^{\text{hidden} } \) are then sent to the output layer.

      -The output layer consists of one neuron in this case, and combines the output from each of the neurons in the hidden layers. The output layer combines the results from the hidden layer using some weights $ w_i^{\text{output}}$ and biases \( b_i^{\text{output}} \). In this case, it is assumes that the number of neurons in the output layer is one. +The output layer consists of one neuron in this case, and combines the +output from each of the neurons in the hidden layers. The output layer +combines the results from the hidden layer using some weights \( w_i^{\text{output}} \) +and biases \( b_i^{\text{output}} \). In this case, +it is assumes that the number of neurons in the output layer is one.











      @@ -683,7 +691,7 @@ In this case we seek a continuous range of values since we are approximating a f











      -

      Backpropagation

      +

      Back propagation

      The next step is to decide how the parameters should be changed such that they minimize the cost function. @@ -726,7 +734,7 @@ for a number of iterations or until $ \big|\big| \boldsymbol{\omega}_{\text{new} The value of \( \lambda \) decides how large steps the algorithm must take in the direction of $ \nabla_{\boldsymbol{\omega}} C(\boldsymbol{x}, \boldsymbol{\omega})$. The notation \( \nabla_{\boldsymbol{\omega}} \) express the gradient with respect -to the elements in \( \boldsymbol{\omega \). +to the elements in \( \boldsymbol{\omega} \).

      In our case, we have to minimize the cost function \( C(\boldsymbol{x}, P) \) with @@ -1092,8 +1100,8 @@ Also, at \( t = 0 \) the population has the size \( g(0) = g_0 \), where \( g_0

      In this example, similar network as for the exponential decay using Autograd has been used to solve the equation. However, as the implementation might suffer from e.g numerical instability and high execution time (this might be more apparent in the examples solving PDEs), -a network has been constructed using TensorFlow also. -For comparison, the forward Euler method has been implemented in order to see how the networks performs compared to a numerical scheme. +using a library like TensorFlow is recommended. +Here, we stay with a more simple approach and implement for comparison, the simple forward Euler method.











      @@ -1323,7 +1331,7 @@ The network will be the similar as for the exponential decay example, but with s

      Using forward Euler to solve the ODE

      -A straight-forward way of solving an ODE numerically, is to use Euler's method. +A straightforward way of solving an ODE numerically, is to use Euler's method.

      Euler's method uses Taylor series to approximate the value at a function \( f \) at a step \( \Delta x \) from \( x \): @@ -1446,13 +1454,10 @@ extending the program that uses the network using Autograd: plt.show()

      -

      -Running the program gives -











      -

      Example: Solving the one dimensional Poisson equation using Autograd and TensorFlow

      +

      Example: Solving the one dimensional Poisson equation

      The Poisson equation for \( g(x) \) in one dimension is @@ -1768,13 +1773,18 @@ f(x_2) \\ f(x_{N_x - 3}) \\ f(x_{N_x - 2}) \end{pmatrix} \\ -A\boldsymbol{g} &= \boldsymbol{f} +\boldsymbol{A}\boldsymbol{g} &= \boldsymbol{f}, \end{aligned} $$

      which makes it possible to solve for the vector \( \boldsymbol{g} \). +

      +









      + +

      Setting up the code

      +

      We can then compare the result from this numerical scheme with the output from our network using Autograd: @@ -1973,21 +1983,15 @@ We can then compare the result from this numerical scheme with the output from o plt.show()

      -The program prints out: -

      - - -

      The max absolute difference between the analytical solution and DNN Autograd: 0.000464088
-The max absolute difference between the analytical solution and numerical scheme: 0.00266858
-
      -











      -

      Partial Differential Equations

      +

      Partial Differential Equations

      -A partial differential equation (PDE) has a solution here the function is defined by multiple variables. -The equation may involve all kinds of combinations of which variables the function is differentiated with respect to. +A partial differential equation (PDE) has a solution here the function +is defined by multiple variables. The equation may involve all kinds +of combinations of which variables the function is differentiated with +respect to.

      In general, a partial differential equation for a function \( g(x_1,\dots,x_N) \) with \( N \) variables may be expressed as @@ -2004,7 +2008,7 @@ where \( f \) is an expression involving all kinds of possible mixed derivatives











      -

      Type of problem

      +

      Type of problem

      The problem our network must solve for, is similar to the ODE case. @@ -2027,7 +2031,7 @@ The role of the function \( h_2(x_1,\dots,x_N,N(x_1,\dots,x_N,P)) \), is to ensu











      -

      Network requirements

      +

      Network requirements

      The network tries then the minimize the cost function following the @@ -2043,35 +2047,33 @@ minimize is $$ \begin{equation*} -c\left(x_1, \dots, x_N, P\right) = \left( f\left(x_1, \, \dots \, , x_N, \frac{\partial g(x_1,\dots,x_N) }{\partial x_1}, \dots , \frac{\partial g(x_1,\dots,x_N) }{\partial x_N}, \frac{\partial g(x_1,\dots,x_N) }{\partial x_1\partial x_2}, \, \dots \, , \frac{\partial^n g(x_1,\dots,x_N) }{\partial x_N^n} \right) \right)^2 +C\left(x_1, \dots, x_N, P\right) = \left( f\left(x_1, \, \dots \, , x_N, \frac{\partial g(x_1,\dots,x_N) }{\partial x_1}, \dots , \frac{\partial g(x_1,\dots,x_N) }{\partial x_N}, \frac{\partial g(x_1,\dots,x_N) }{\partial x_1\partial x_2}, \, \dots \, , \frac{\partial^n g(x_1,\dots,x_N) }{\partial x_N^n} \right) \right)^2 \end{equation*} $$











      -

      More details

      +

      More details

      -If we let \( \boldsymbol{x = \big( x_1, \dots, x_N \big) \) be an array containing the values for \( x_1, \dots, x_N \) respectively, the cost function can be reformulated into the following: +If we let \( \boldsymbol{x} = \big( x_1, \dots, x_N \big) \) be an array containing the values for \( x_1, \dots, x_N \) respectively, the cost function can be reformulated into the following: $$ -\begin{equation*} - c\left(\boldsymbol{x}, P\right) = f\left( \left( \boldsymbol{x}, \frac{\partial g(\boldsymbol{x) }{\partial x_1}, \dots , \frac{\partial g(\boldsymbol{x) }{\partial x_N}, \frac{\partial g(\boldsymbol{x) }{\partial x_1\partial x_2}, \, \dots \, , \frac{\partial^n g(\boldsymbol{x) }{\partial x_N^n} \right) \right)^2 -\end{equation*} + C\left(\boldsymbol{x}, P\right) = f\left( \left( \boldsymbol{x}, \frac{\partial g(\boldsymbol{x}) }{\partial x_1}, \dots , \frac{\partial g(\boldsymbol{x}) }{\partial x_N}, \frac{\partial g(\boldsymbol{x}) }{\partial x_1\partial x_2}, \, \dots \, , \frac{\partial^n g(\boldsymbol{x}) }{\partial x_N^n} \right) \right)^2 $$

      If we also have \( M \) different sets of values for \( x_1, \dots, x_N \), that is \( \boldsymbol{x}_i = \big(x_1^{(i)}, \dots, x_N^{(i)}\big) \) for \( i = 1,\dots,M \) being the rows in matrix \( X \), the cost function can be generalized into $$ \begin{equation*} -c\left(X, P \right) = \sum_{i=1}^M f\left( \left( \boldsymbol{x}_i, \frac{\partial g(\boldsymbol{x}_i) }{\partial x_1}, \dots , \frac{\partial g(\boldsymbol{x}_i) }{\partial x_N}, \frac{\partial g(\boldsymbol{x}_i) }{\partial x_1\partial x_2}, \, \dots \, , \frac{\partial^n g(\boldsymbol{x}_i) }{\partial x_N^n} \right) \right)^2 +C\left(X, P \right) = \sum_{i=1}^M f\left( \left( \boldsymbol{x}_i, \frac{\partial g(\boldsymbol{x}_i) }{\partial x_1}, \dots , \frac{\partial g(\boldsymbol{x}_i) }{\partial x_N}, \frac{\partial g(\boldsymbol{x}_i) }{\partial x_1\partial x_2}, \, \dots \, , \frac{\partial^n g(\boldsymbol{x}_i) }{\partial x_N^n} \right) \right)^2. \end{equation*} $$











      -

      Example: The diffusion equation

      +

      Example: The diffusion equation

      In one spatial dimension, the equation reads @@ -2096,7 +2098,7 @@ with \( u(x) \) being some given function.











      -

      Defining the problem

      +

      Defining the problem

      For this case, we want to find \( g(x,t) \) such that @@ -2128,7 +2130,7 @@ First, we will look into how Autograd could be used in a network tailored to sol











      -

      Setting up the network using Autograd

      +

      Setting up the network using Autograd

      The only change to do here, is to extend our network such that @@ -2190,7 +2192,7 @@ network at each possible pair \( (x,t) \), given an array for the desired











      -

      Setting up the network using Autograd; The trial solution

      +

      Setting up the network using Autograd; The trial solution

      The cost function must then iterate through the given arrays @@ -2219,7 +2221,7 @@ since \( (0) = u(1) = 0 \) and \( u(x) = \sin(\pi x) \).











      -

      Why the jacobian?

      +

      Why the jacobian?

      The Jacobian is used because the program must find the derivative of @@ -2286,7 +2288,7 @@ mixed derivatives of \( g(x,t) \).











      -

      Setting up the network using Autograd; The full program

      +

      Setting up the network using Autograd; The full program

      Having set up the network, along with the trial solution and cost function, we can now see how the deep neural network performs by comparing the results to the analytical solution. @@ -2537,7 +2539,7 @@ Using TensorFlow results in a much better execution time. Try it!











      -

      Example: Solving the wave equation with Neural Networks

      +

      Example: Solving the wave equation with Neural Networks

      The wave equation is @@ -2566,7 +2568,7 @@ where \( \frac{\partial g(x,t)}{\partial t} \Big |_{t = 0} \) means the derivati











      -

      The problem to solve for

      +

      The problem to solve for

      The wave equation to solve for, is @@ -2596,7 +2598,7 @@ In this example, let \( c = 1 \) and \( u(x) = \sin(\pi x) \) and \( v(x) = -\pi











      -

      The trial solution

      +

      The trial solution

      Setting up the network is done in similar matter as for the example of solving the diffusion equation. The only things we have to change, is the trial solution such that it satisfies the conditions from \eqref{condwave} and the cost function. @@ -2620,7 +2622,7 @@ Note that this trial solution satisfies the conditions only if \( u(0) = v(0) =











      -

      The analytical solution

      +

      The analytical solution

      The analytical solution for our specific problem, is @@ -2632,7 +2634,7 @@ $$











      -

      Solving the wave equation - the full program using Autograd

      +

      Solving the wave equation - the full program using Autograd

      @@ -2861,7 +2863,7 @@ $$











      -

      Resources on differential equations and deep learning

      +

      Resources on differential equations and deep learning

      1. Artificial neural networks for solving ordinary and partial differential equations by I.E. Lagaris et al
        2. @@ -2872,12 +2874,15 @@ $$









        -

        Friday, Principal Component Analysis

        +

        Friday, Principal Component Analysis

        + +

        +Overview video











        -

        Basic ideas of the Principal Component Analysis (PCA)

        +

        Basic ideas of the Principal Component Analysis (PCA)

        The principal component analysis deals with the problem of fitting a @@ -2899,7 +2904,7 @@ We have a data set defined by a design/feature matrix \( \boldsymbol{X} \) (see









        -

        Introducing the Covariance and Correlation functions

        +

        Introducing the Covariance and Correlation functions

        Before we discuss the PCA theorem, we need to remind ourselves about @@ -2960,7 +2965,7 @@ In the above example this is the function we constructed using pandas.











        -

        Correlation Function and Design/Feature Matrix

        +

        Correlation Function and Design/Feature Matrix

        In our derivation of the various regression algorithms like Ordinary Least Squares or Ridge regression @@ -3021,7 +3026,7 @@ $$











        -

        Covariance Matrix Examples

        +

        Covariance Matrix Examples

        The Numpy function np.cov calculates the covariance elements using @@ -3064,7 +3069,7 @@ C = np.cov(W)











        -

        Correlation Matrix

        +

        Correlation Matrix

        The previous example can be converted into the correlation matrix by @@ -3109,7 +3114,7 @@ The above procedure with numpy can be made more compact if we use pand











        -

        Correlation Matrix with Pandas

        +

        Correlation Matrix with Pandas

        We whow here how we can set up the correlation matrix using pandas, as done in this simple code @@ -3136,7 +3141,7 @@ We expand this model to the Franke function discussed above.











        -

        Correlation Matrix with Pandas and the Franke function

        +

        Correlation Matrix with Pandas and the Franke function

        @@ -3199,7 +3204,7 @@ matrix without these elements.











        -

        Rewriting the Covariance and/or Correlation Matrix

        +

        Rewriting the Covariance and/or Correlation Matrix

        We can rewrite the covariance matrix in a more compact form in terms of the design/feature matrix \( \boldsymbol{X} \) as @@ -3242,7 +3247,7 @@ It is easy to generalize this to a matrix \( \boldsymbol{X}\in {\mathbb{R}}^{n\t











        -

        Towards the PCA theorem

        +

        Towards the PCA theorem

        We have that the covariance matrix (the correlation matrix involves a simple rescaling) is given as @@ -3293,7 +3298,7 @@ features/predictors.











        -

        The Algorithm before theorem

        +

        The Algorithm before theorem

        Here's how we would proceed in setting up the algorithm for the PCA, see also discussion below here. @@ -3324,7 +3329,7 @@ $$









        -

        Writing our own PCA code

        +

        Writing our own PCA code

        We will use a simple example first with two-dimensional data @@ -3357,7 +3362,7 @@ X = np.random.multivariate_normal(mean, cov, n)

        Now we are going to implement the PCA algorithm. We will break it down into various substeps. -

        Compute the sample mean and center the data

        +

        Compute the sample mean and center the data

        The first step of PCA is to compute the sample mean of the data and use it to center the data. Recall that the sample mean is @@ -3394,7 +3399,7 @@ variance. The diagonal covariance matrix elements will then be one, while the non-diagonal ones need to be divided by \( 2\sqrt{2} \) for our specific case. -

        Compute the sample covariance

        +

        Compute the sample covariance

        Now we are going to use the mean centered data to compute the sample covariance of the data by using the following equation @@ -3436,7 +3441,7 @@ plt.show() Depending on the number of points \( n \), we will get results that are close to the covariance values defined above. The plot shows how the data are clustered around a line with slope close to one. Is this expected? -

        Diagonalize the sample covariance matrix to obtain the principal components

        +

        Diagonalize the sample covariance matrix to obtain the principal components

        Now we are ready to solve for the principal components! To do so we @@ -3500,7 +3505,7 @@ This code does not contain all the above elements, but it shows how we can use <











        -

        Classical PCA Theorem

        +

        Classical PCA Theorem

        We assume now that we have a design matrix \( \boldsymbol{X} \) which has been @@ -3533,7 +3538,7 @@ The proof which follows will be updated by mid January 2020.











        -

        Proof of the PCA Theorem

        +

        Proof of the PCA Theorem

        To show the PCA theorem let us start with the assumption that there is one vector \( \boldsymbol{w}_0 \) which corresponds to a solution which minimized the reconstruction error \( J \). This is an orthogonal vector. It means that we now approximate the reconstruction error in terms of \( \boldsymbol{w}_0 \) and \( \boldsymbol{z}_0 \) as @@ -3556,7 +3561,7 @@ where the vectors on the rhs are known.











        -

        PCA Proof continued

        +

        PCA Proof continued

        We have now found the unknown parameters \( z_{i0} \). These correspond to the projected coordinates and we can write @@ -3597,7 +3602,7 @@ of the projected data.











        -

        The final step

        +

        The final step

        We could trivially maximize the variance of the projection (and @@ -3651,7 +3656,7 @@ Additional part of the proof for the other eigenvectors will be added by mid Jan











        -

        Geometric Interpretation and link with Singular Value Decomposition

        +

        Geometric Interpretation and link with Singular Value Decomposition

        This material will be added by mid January 2020. @@ -3659,7 +3664,7 @@ This material will be added by mid January 2020.











        -

        Principal Component Analysis

        +

        Principal Component Analysis

        Principal Component Analysis (PCA) is by far the most popular dimensionality reduction algorithm. @@ -3714,7 +3719,7 @@ X2D = X_centered.dot(W2)

        -

        PCA and scikit-learn

        +

        PCA and scikit-learn

        Scikit-Learn’s PCA class implements PCA using SVD decomposition just like we did before. The @@ -3746,7 +3751,7 @@ variance that lies along the axis of each principal component.











        -

        Back to the Cancer Data

        +

        Back to the Cancer Data

        We can now repeat the above but applied to real data, in this case our breast cancer data. Here we compute performance scores on the training data using logistic regression.

        @@ -3787,7 +3792,7 @@ We see that our training data after the PCA decomposition has a performance simi











        -

        More on the PCA

        +

        More on the PCA

        Instead of arbitrarily choosing the number of dimensions to reduce down to, it is generally preferable to @@ -3817,7 +3822,7 @@ X_reduced = pca.fit_transform(X)











        -

        Incremental PCA

        +

        Incremental PCA

        One problem with the preceding implementation of PCA is that it requires the whole training set to fit in @@ -3829,7 +3834,7 @@ instances arrive).











        -

        Randomized PCA

        +

        Randomized PCA

        Scikit-Learn offers yet another option to perform PCA, called Randomized PCA. This is a stochastic @@ -3840,7 +3845,7 @@ previous algorithms when \( d \) is much smaller than \( n \).











        -

        Kernel PCA

        +

        Kernel PCA

        @@ -3869,7 +3874,7 @@ X_reduced = rbf_pca.fit_transform(X)











        -

        LLE

        +

        LLE

        Locally Linear Embedding (LLE) is another very powerful nonlinear dimensionality reduction @@ -3881,7 +3886,7 @@ these local relationships are best preserved (more details shortly).











        -

        Other techniques

        +

        Other techniques

        There are many other dimensionality reduction techniques, several of which are available in Scikit-Learn. diff --git a/doc/pub/week43/html/week43.html b/doc/pub/week43/html/week43.html index b001118b9..1c21d2544 100644 --- a/doc/pub/week43/html/week43.html +++ b/doc/pub/week43/html/week43.html @@ -92,7 +92,7 @@ div { text-align: justify; text-justify: inter-word; } ('Final technicalities II', 2, None, '___sec16'), ('Final technicalities III', 2, None, '___sec17'), ('Final technicalities IV', 2, None, '___sec18'), - ('Backpropagation', 2, None, '___sec19'), + ('Back propagation', 2, None, '___sec19'), ('Gradient descent', 2, None, '___sec20'), ('The code for solving the ODE', 2, None, '___sec21'), ('The network with one input layer, specified number of hidden ' @@ -105,100 +105,100 @@ div { text-align: justify; text-justify: inter-word; } ('The trial solution', 2, None, '___sec25'), ('The program using Autograd', 2, None, '___sec26'), ('Using forward Euler to solve the ODE', 2, None, '___sec27'), - ('Example: Solving the one dimensional Poisson equation using ' - 'Autograd and TensorFlow', + ('Example: Solving the one dimensional Poisson equation', 2, None, '___sec28'), ('The specific equation to solve for', 2, None, '___sec29'), ('Solving the equation using Autograd', 2, None, '___sec30'), ('Comparing with a numerical scheme', 2, None, '___sec31'), - ('Partial Differential Equations', 2, None, '___sec32'), - ('Type of problem', 2, None, '___sec33'), - ('Network requirements', 2, None, '___sec34'), - ('More details', 2, None, '___sec35'), - ('Example: The diffusion equation', 2, None, '___sec36'), - ('Defining the problem', 2, None, '___sec37'), - ('Setting up the network using Autograd', 2, None, '___sec38'), + ('Setting up the code', 2, None, '___sec32'), + ('Partial Differential Equations', 2, None, '___sec33'), + ('Type of problem', 2, None, '___sec34'), + ('Network requirements', 2, None, '___sec35'), + ('More details', 2, None, '___sec36'), + ('Example: The diffusion equation', 2, None, '___sec37'), + ('Defining the problem', 2, None, '___sec38'), + ('Setting up the network using Autograd', 2, None, '___sec39'), ('Setting up the network using Autograd; The trial solution', 2, None, - '___sec39'), - ('Why the jacobian?', 2, None, '___sec40'), + '___sec40'), + ('Why the jacobian?', 2, None, '___sec41'), ('Setting up the network using Autograd; The full program', 2, None, - '___sec41'), + '___sec42'), ('Example: Solving the wave equation with Neural Networks', 2, None, - '___sec42'), - ('The problem to solve for', 2, None, '___sec43'), - ('The trial solution', 2, None, '___sec44'), - ('The analytical solution', 2, None, '___sec45'), + '___sec43'), + ('The problem to solve for', 2, None, '___sec44'), + ('The trial solution', 2, None, '___sec45'), + ('The analytical solution', 2, None, '___sec46'), ('Solving the wave equation - the full program using Autograd', 2, None, - '___sec46'), + '___sec47'), ('Resources on differential equations and deep learning', 2, None, - '___sec47'), - ('Friday, Principal Component Analysis', 2, None, '___sec48'), + '___sec48'), + ('Friday, Principal Component Analysis', 2, None, '___sec49'), ('Basic ideas of the Principal Component Analysis (PCA)', 2, None, - '___sec49'), + '___sec50'), ('Introducing the Covariance and Correlation functions', 2, None, - '___sec50'), + '___sec51'), ('Correlation Function and Design/Feature Matrix', 2, None, - '___sec51'), - ('Covariance Matrix Examples', 2, None, '___sec52'), - ('Correlation Matrix', 2, None, '___sec53'), - ('Correlation Matrix with Pandas', 2, None, '___sec54'), + '___sec52'), + ('Covariance Matrix Examples', 2, None, '___sec53'), + ('Correlation Matrix', 2, None, '___sec54'), + ('Correlation Matrix with Pandas', 2, None, '___sec55'), ('Correlation Matrix with Pandas and the Franke function', 2, None, - '___sec55'), + '___sec56'), ('Rewriting the Covariance and/or Correlation Matrix', 2, None, - '___sec56'), - ('Towards the PCA theorem', 2, None, '___sec57'), - ('The Algorithm before the Theorem', 2, None, '___sec58'), - ('Writing our own PCA code', 2, None, '___sec59'), + '___sec57'), + ('Towards the PCA theorem', 2, None, '___sec58'), + ('The Algorithm before the Theorem', 2, None, '___sec59'), + ('Writing our own PCA code', 2, None, '___sec60'), ('Compute the sample mean and center the data', 3, None, - '___sec60'), - ('Compute the sample covariance', 3, None, '___sec61'), + '___sec61'), + ('Compute the sample covariance', 3, None, '___sec62'), ('Diagonalize the sample covariance matrix to obtain the ' 'principal components', 3, None, - '___sec62'), - ('Classical PCA Theorem', 2, None, '___sec63'), - ('Proof of the PCA Theorem', 2, None, '___sec64'), - ('PCA Proof continued', 2, None, '___sec65'), - ('The final step', 2, None, '___sec66'), + '___sec63'), + ('Classical PCA Theorem', 2, None, '___sec64'), + ('Proof of the PCA Theorem', 2, None, '___sec65'), + ('PCA Proof continued', 2, None, '___sec66'), + ('The final step', 2, None, '___sec67'), ('Geometric Interpretation and link with Singular Value ' 'Decomposition', 2, None, - '___sec67'), - ('Principal Component Analysis', 2, None, '___sec68'), - ('PCA and scikit-learn', 2, None, '___sec69'), - ('Back to the Cancer Data', 2, None, '___sec70'), - ('More on the PCA', 2, None, '___sec71'), - ('Incremental PCA', 2, None, '___sec72'), - ('Randomized PCA', 2, None, '___sec73'), - ('Kernel PCA', 2, None, '___sec74'), - ('LLE', 2, None, '___sec75'), - ('Other techniques', 2, None, '___sec76')]} + '___sec68'), + ('Principal Component Analysis', 2, None, '___sec69'), + ('PCA and scikit-learn', 2, None, '___sec70'), + ('Back to the Cancer Data', 2, None, '___sec71'), + ('More on the PCA', 2, None, '___sec72'), + ('Incremental PCA', 2, None, '___sec73'), + ('Randomized PCA', 2, None, '___sec74'), + ('Kernel PCA', 2, None, '___sec75'), + ('LLE', 2, None, '___sec76'), + ('Other techniques', 2, None, '___sec77')]} end of tocinfo --> @@ -250,13 +250,17 @@ MathJax.Hub.Config({

      2. Friday: Principal Component Analysis and Dimensionality Reduction
        3. -Reading suggestions for both days: "Aurelien Geron's chapters 8 +We will also study the usage of Autograd in computing gradients for deep learning. For the documentation of Autograd and examples see the lectures slides from week 40 and the Autograd doucmentation.











        Recurrent Neural Networks

        +

        +Overview video. +See also lecture on Thursday October 22 and examples from week 42. +











        @@ -630,7 +634,7 @@ $$ It is possible to use other activations functions for the hidden layer also.

        -The output $\boldsymbol{x}_i^{\text{hidden} }$from each \( i \)-th hidden neuron is: +The output \( \boldsymbol{x}_i^{\text{hidden}} \) from each \( i \)-th hidden neuron is: $$ \boldsymbol{x}_i^{\text{hidden} } = f\big( \boldsymbol{z}_{i}^{\text{hidden}} \big) @@ -640,7 +644,11 @@ $$ The outputs \( \boldsymbol{x}_i^{\text{hidden} } \) are then sent to the output layer.

        -The output layer consists of one neuron in this case, and combines the output from each of the neurons in the hidden layers. The output layer combines the results from the hidden layer using some weights $ w_i^{\text{output}}$ and biases \( b_i^{\text{output}} \). In this case, it is assumes that the number of neurons in the output layer is one. +The output layer consists of one neuron in this case, and combines the +output from each of the neurons in the hidden layers. The output layer +combines the results from the hidden layer using some weights \( w_i^{\text{output}} \) +and biases \( b_i^{\text{output}} \). In this case, +it is assumes that the number of neurons in the output layer is one.











        @@ -688,7 +696,7 @@ In this case we seek a continuous range of values since we are approximating a f











        -

        Backpropagation

        +

        Back propagation

        The next step is to decide how the parameters should be changed such that they minimize the cost function. @@ -731,7 +739,7 @@ for a number of iterations or until $ \big|\big| \boldsymbol{\omega}_{\text{new} The value of \( \lambda \) decides how large steps the algorithm must take in the direction of $ \nabla_{\boldsymbol{\omega}} C(\boldsymbol{x}, \boldsymbol{\omega})$. The notation \( \nabla_{\boldsymbol{\omega}} \) express the gradient with respect -to the elements in \( \boldsymbol{\omega \). +to the elements in \( \boldsymbol{\omega} \).

        In our case, we have to minimize the cost function \( C(\boldsymbol{x}, P) \) with @@ -1097,8 +1105,8 @@ Also, at \( t = 0 \) the population has the size \( g(0) = g_0 \), where \( g_0

        In this example, similar network as for the exponential decay using Autograd has been used to solve the equation. However, as the implementation might suffer from e.g numerical instability and high execution time (this might be more apparent in the examples solving PDEs), -a network has been constructed using TensorFlow also. -For comparison, the forward Euler method has been implemented in order to see how the networks performs compared to a numerical scheme. +using a library like TensorFlow is recommended. +Here, we stay with a more simple approach and implement for comparison, the simple forward Euler method.











        @@ -1328,7 +1336,7 @@ The network will be the similar as for the exponential decay example, but with s

        Using forward Euler to solve the ODE

        -A straight-forward way of solving an ODE numerically, is to use Euler's method. +A straightforward way of solving an ODE numerically, is to use Euler's method.

        Euler's method uses Taylor series to approximate the value at a function \( f \) at a step \( \Delta x \) from \( x \): @@ -1451,13 +1459,10 @@ extending the program that uses the network using Autograd: plt.show()

        -

        -Running the program gives -











        -

        Example: Solving the one dimensional Poisson equation using Autograd and TensorFlow

        +

        Example: Solving the one dimensional Poisson equation

        The Poisson equation for \( g(x) \) in one dimension is @@ -1773,13 +1778,18 @@ f(x_2) \\ f(x_{N_x - 3}) \\ f(x_{N_x - 2}) \end{pmatrix} \\ -A\boldsymbol{g} &= \boldsymbol{f} +\boldsymbol{A}\boldsymbol{g} &= \boldsymbol{f}, \end{aligned} $$

        which makes it possible to solve for the vector \( \boldsymbol{g} \). +

        +









        + +

        Setting up the code

        +

        We can then compare the result from this numerical scheme with the output from our network using Autograd: @@ -1978,21 +1988,15 @@ We can then compare the result from this numerical scheme with the output from o plt.show()

        -The program prints out: -

        - - -

        The max absolute difference between the analytical solution and DNN Autograd: 0.000464088
-The max absolute difference between the analytical solution and numerical scheme: 0.00266858
-
        -











        -

        Partial Differential Equations

        +

        Partial Differential Equations

        -A partial differential equation (PDE) has a solution here the function is defined by multiple variables. -The equation may involve all kinds of combinations of which variables the function is differentiated with respect to. +A partial differential equation (PDE) has a solution here the function +is defined by multiple variables. The equation may involve all kinds +of combinations of which variables the function is differentiated with +respect to.

        In general, a partial differential equation for a function \( g(x_1,\dots,x_N) \) with \( N \) variables may be expressed as @@ -2009,7 +2013,7 @@ where \( f \) is an expression involving all kinds of possible mixed derivatives











        -

        Type of problem

        +

        Type of problem

        The problem our network must solve for, is similar to the ODE case. @@ -2032,7 +2036,7 @@ The role of the function \( h_2(x_1,\dots,x_N,N(x_1,\dots,x_N,P)) \), is to ensu











        -

        Network requirements

        +

        Network requirements

        The network tries then the minimize the cost function following the @@ -2048,35 +2052,33 @@ minimize is $$ \begin{equation*} -c\left(x_1, \dots, x_N, P\right) = \left( f\left(x_1, \, \dots \, , x_N, \frac{\partial g(x_1,\dots,x_N) }{\partial x_1}, \dots , \frac{\partial g(x_1,\dots,x_N) }{\partial x_N}, \frac{\partial g(x_1,\dots,x_N) }{\partial x_1\partial x_2}, \, \dots \, , \frac{\partial^n g(x_1,\dots,x_N) }{\partial x_N^n} \right) \right)^2 +C\left(x_1, \dots, x_N, P\right) = \left( f\left(x_1, \, \dots \, , x_N, \frac{\partial g(x_1,\dots,x_N) }{\partial x_1}, \dots , \frac{\partial g(x_1,\dots,x_N) }{\partial x_N}, \frac{\partial g(x_1,\dots,x_N) }{\partial x_1\partial x_2}, \, \dots \, , \frac{\partial^n g(x_1,\dots,x_N) }{\partial x_N^n} \right) \right)^2 \end{equation*} $$











        -

        More details

        +

        More details

        -If we let \( \boldsymbol{x = \big( x_1, \dots, x_N \big) \) be an array containing the values for \( x_1, \dots, x_N \) respectively, the cost function can be reformulated into the following: +If we let \( \boldsymbol{x} = \big( x_1, \dots, x_N \big) \) be an array containing the values for \( x_1, \dots, x_N \) respectively, the cost function can be reformulated into the following: $$ -\begin{equation*} - c\left(\boldsymbol{x}, P\right) = f\left( \left( \boldsymbol{x}, \frac{\partial g(\boldsymbol{x) }{\partial x_1}, \dots , \frac{\partial g(\boldsymbol{x) }{\partial x_N}, \frac{\partial g(\boldsymbol{x) }{\partial x_1\partial x_2}, \, \dots \, , \frac{\partial^n g(\boldsymbol{x) }{\partial x_N^n} \right) \right)^2 -\end{equation*} + C\left(\boldsymbol{x}, P\right) = f\left( \left( \boldsymbol{x}, \frac{\partial g(\boldsymbol{x}) }{\partial x_1}, \dots , \frac{\partial g(\boldsymbol{x}) }{\partial x_N}, \frac{\partial g(\boldsymbol{x}) }{\partial x_1\partial x_2}, \, \dots \, , \frac{\partial^n g(\boldsymbol{x}) }{\partial x_N^n} \right) \right)^2 $$

        If we also have \( M \) different sets of values for \( x_1, \dots, x_N \), that is \( \boldsymbol{x}_i = \big(x_1^{(i)}, \dots, x_N^{(i)}\big) \) for \( i = 1,\dots,M \) being the rows in matrix \( X \), the cost function can be generalized into $$ \begin{equation*} -c\left(X, P \right) = \sum_{i=1}^M f\left( \left( \boldsymbol{x}_i, \frac{\partial g(\boldsymbol{x}_i) }{\partial x_1}, \dots , \frac{\partial g(\boldsymbol{x}_i) }{\partial x_N}, \frac{\partial g(\boldsymbol{x}_i) }{\partial x_1\partial x_2}, \, \dots \, , \frac{\partial^n g(\boldsymbol{x}_i) }{\partial x_N^n} \right) \right)^2 +C\left(X, P \right) = \sum_{i=1}^M f\left( \left( \boldsymbol{x}_i, \frac{\partial g(\boldsymbol{x}_i) }{\partial x_1}, \dots , \frac{\partial g(\boldsymbol{x}_i) }{\partial x_N}, \frac{\partial g(\boldsymbol{x}_i) }{\partial x_1\partial x_2}, \, \dots \, , \frac{\partial^n g(\boldsymbol{x}_i) }{\partial x_N^n} \right) \right)^2. \end{equation*} $$











        -

        Example: The diffusion equation

        +

        Example: The diffusion equation

        In one spatial dimension, the equation reads @@ -2101,7 +2103,7 @@ with \( u(x) \) being some given function.











        -

        Defining the problem

        +

        Defining the problem

        For this case, we want to find \( g(x,t) \) such that @@ -2133,7 +2135,7 @@ First, we will look into how Autograd could be used in a network tailored to sol











        -

        Setting up the network using Autograd

        +

        Setting up the network using Autograd

        The only change to do here, is to extend our network such that @@ -2195,7 +2197,7 @@ network at each possible pair \( (x,t) \), given an array for the desired











        -

        Setting up the network using Autograd; The trial solution

        +

        Setting up the network using Autograd; The trial solution

        The cost function must then iterate through the given arrays @@ -2224,7 +2226,7 @@ since \( (0) = u(1) = 0 \) and \( u(x) = \sin(\pi x) \).











        -

        Why the jacobian?

        +

        Why the jacobian?

        The Jacobian is used because the program must find the derivative of @@ -2291,7 +2293,7 @@ mixed derivatives of \( g(x,t) \).











        -

        Setting up the network using Autograd; The full program

        +

        Setting up the network using Autograd; The full program

        Having set up the network, along with the trial solution and cost function, we can now see how the deep neural network performs by comparing the results to the analytical solution. @@ -2542,7 +2544,7 @@ Using TensorFlow results in a much better execution time. Try it!











        -

        Example: Solving the wave equation with Neural Networks

        +

        Example: Solving the wave equation with Neural Networks

        The wave equation is @@ -2571,7 +2573,7 @@ where \( \frac{\partial g(x,t)}{\partial t} \Big |_{t = 0} \) means the derivati











        -

        The problem to solve for

        +

        The problem to solve for

        The wave equation to solve for, is @@ -2601,7 +2603,7 @@ In this example, let \( c = 1 \) and \( u(x) = \sin(\pi x) \) and \( v(x) = -\pi











        -

        The trial solution

        +

        The trial solution

        Setting up the network is done in similar matter as for the example of solving the diffusion equation. The only things we have to change, is the trial solution such that it satisfies the conditions from \eqref{condwave} and the cost function. @@ -2625,7 +2627,7 @@ Note that this trial solution satisfies the conditions only if \( u(0) = v(0) =











        -

        The analytical solution

        +

        The analytical solution

        The analytical solution for our specific problem, is @@ -2637,7 +2639,7 @@ $$











        -

        Solving the wave equation - the full program using Autograd

        +

        Solving the wave equation - the full program using Autograd

        @@ -2866,7 +2868,7 @@ $$











        -

        Resources on differential equations and deep learning

        +

        Resources on differential equations and deep learning

        1. Artificial neural networks for solving ordinary and partial differential equations by I.E. Lagaris et al
          2. @@ -2877,12 +2879,15 @@ $$









          -

          Friday, Principal Component Analysis

          +

          Friday, Principal Component Analysis

          + +

          +Overview video











          -

          Basic ideas of the Principal Component Analysis (PCA)

          +

          Basic ideas of the Principal Component Analysis (PCA)

          The principal component analysis deals with the problem of fitting a @@ -2904,7 +2909,7 @@ We have a data set defined by a design/feature matrix \( \boldsymbol{X} \) (see









          -

          Introducing the Covariance and Correlation functions

          +

          Introducing the Covariance and Correlation functions

          Before we discuss the PCA theorem, we need to remind ourselves about @@ -2965,7 +2970,7 @@ In the above example this is the function we constructed using pandas.











          -

          Correlation Function and Design/Feature Matrix

          +

          Correlation Function and Design/Feature Matrix

          In our derivation of the various regression algorithms like Ordinary Least Squares or Ridge regression @@ -3026,7 +3031,7 @@ $$











          -

          Covariance Matrix Examples

          +

          Covariance Matrix Examples

          The Numpy function np.cov calculates the covariance elements using @@ -3069,7 +3074,7 @@ C = np.c











          -

          Correlation Matrix

          +

          Correlation Matrix

          The previous example can be converted into the correlation matrix by @@ -3114,7 +3119,7 @@ The above procedure with numpy can be made more compact if we use pand











          -

          Correlation Matrix with Pandas

          +

          Correlation Matrix with Pandas

          We whow here how we can set up the correlation matrix using pandas, as done in this simple code @@ -3141,7 +3146,7 @@ We expand this model to the Franke function discussed above.











          -

          Correlation Matrix with Pandas and the Franke function

          +

          Correlation Matrix with Pandas and the Franke function

          @@ -3204,7 +3209,7 @@ matrix without these elements.











          -

          Rewriting the Covariance and/or Correlation Matrix

          +

          Rewriting the Covariance and/or Correlation Matrix

          We can rewrite the covariance matrix in a more compact form in terms of the design/feature matrix \( \boldsymbol{X} \) as @@ -3247,7 +3252,7 @@ It is easy to generalize this to a matrix \( \boldsymbol{X}\in {\mathbb{R}}^{n\t











          -

          Towards the PCA theorem

          +

          Towards the PCA theorem

          We have that the covariance matrix (the correlation matrix involves a simple rescaling) is given as @@ -3298,7 +3303,7 @@ features/predictors.











          -

          The Algorithm before theorem

          +

          The Algorithm before theorem

          Here's how we would proceed in setting up the algorithm for the PCA, see also discussion below here. @@ -3329,7 +3334,7 @@ $$









          -

          Writing our own PCA code

          +

          Writing our own PCA code

          We will use a simple example first with two-dimensional data @@ -3362,7 +3367,7 @@ X = np.r

          Now we are going to implement the PCA algorithm. We will break it down into various substeps. -

          Compute the sample mean and center the data

          +

          Compute the sample mean and center the data

          The first step of PCA is to compute the sample mean of the data and use it to center the data. Recall that the sample mean is @@ -3399,7 +3404,7 @@ variance. The diagonal covariance matrix elements will then be one, while the non-diagonal ones need to be divided by \( 2\sqrt{2} \) for our specific case. -

          Compute the sample covariance

          +

          Compute the sample covariance

          Now we are going to use the mean centered data to compute the sample covariance of the data by using the following equation @@ -3441,7 +3446,7 @@ plt.show() Depending on the number of points \( n \), we will get results that are close to the covariance values defined above. The plot shows how the data are clustered around a line with slope close to one. Is this expected? -

          Diagonalize the sample covariance matrix to obtain the principal components

          +

          Diagonalize the sample covariance matrix to obtain the principal components

          Now we are ready to solve for the principal components! To do so we @@ -3505,7 +3510,7 @@ This code does not contain all the above elements, but it shows how we can use <











          -

          Classical PCA Theorem

          +

          Classical PCA Theorem

          We assume now that we have a design matrix \( \boldsymbol{X} \) which has been @@ -3538,7 +3543,7 @@ The proof which follows will be updated by mid January 2020.











          -

          Proof of the PCA Theorem

          +

          Proof of the PCA Theorem

          To show the PCA theorem let us start with the assumption that there is one vector \( \boldsymbol{w}_0 \) which corresponds to a solution which minimized the reconstruction error \( J \). This is an orthogonal vector. It means that we now approximate the reconstruction error in terms of \( \boldsymbol{w}_0 \) and \( \boldsymbol{z}_0 \) as @@ -3561,7 +3566,7 @@ where the vectors on the rhs are known.











          -

          PCA Proof continued

          +

          PCA Proof continued

          We have now found the unknown parameters \( z_{i0} \). These correspond to the projected coordinates and we can write @@ -3602,7 +3607,7 @@ of the projected data.











          -

          The final step

          +

          The final step

          We could trivially maximize the variance of the projection (and @@ -3656,7 +3661,7 @@ Additional part of the proof for the other eigenvectors will be added by mid Jan











          -

          Geometric Interpretation and link with Singular Value Decomposition

          +

          Geometric Interpretation and link with Singular Value Decomposition

          This material will be added by mid January 2020. @@ -3664,7 +3669,7 @@ This material will be added by mid January 2020.











          -

          Principal Component Analysis

          +

          Principal Component Analysis

          Principal Component Analysis (PCA) is by far the most popular dimensionality reduction algorithm. @@ -3719,7 +3724,7 @@ X2D = X_centered -

          PCA and scikit-learn

          +

          PCA and scikit-learn

          Scikit-Learn’s PCA class implements PCA using SVD decomposition just like we did before. The @@ -3751,7 +3756,7 @@ variance that lies along the axis of each principal component.











          -

          Back to the Cancer Data

          +

          Back to the Cancer Data

          We can now repeat the above but applied to real data, in this case our breast cancer data. Here we compute performance scores on the training data using logistic regression.

          @@ -3792,7 +3797,7 @@ We see that our training data after the PCA decomposition has a performance simi











          -

          More on the PCA

          +

          More on the PCA

          Instead of arbitrarily choosing the number of dimensions to reduce down to, it is generally preferable to @@ -3822,7 +3827,7 @@ X_reduced = pca











          -

          Incremental PCA

          +

          Incremental PCA

          One problem with the preceding implementation of PCA is that it requires the whole training set to fit in @@ -3834,7 +3839,7 @@ instances arrive).











          -

          Randomized PCA

          +

          Randomized PCA

          Scikit-Learn offers yet another option to perform PCA, called Randomized PCA. This is a stochastic @@ -3845,7 +3850,7 @@ previous algorithms when \( d \) is much smaller than \( n \).











          -

          Kernel PCA

          +

          Kernel PCA

          @@ -3874,7 +3879,7 @@ X_reduced = rbf_pcaLLE +

          LLE

          Locally Linear Embedding (LLE) is another very powerful nonlinear dimensionality reduction @@ -3886,7 +3891,7 @@ these local relationships are best preserved (more details shortly).











          -

          Other techniques

          +

          Other techniques

          There are many other dimensionality reduction techniques, several of which are available in Scikit-Learn. diff --git a/doc/pub/week43/ipynb/ipynb-week43-src.tar.gz b/doc/pub/week43/ipynb/ipynb-week43-src.tar.gz index 937f3b429..38c452deb 100644 Binary files a/doc/pub/week43/ipynb/ipynb-week43-src.tar.gz and b/doc/pub/week43/ipynb/ipynb-week43-src.tar.gz differ diff --git a/doc/pub/week43/ipynb/week43.ipynb b/doc/pub/week43/ipynb/week43.ipynb index d53967f04..cfc8b4395 100644 --- a/doc/pub/week43/ipynb/week43.ipynb +++ b/doc/pub/week43/ipynb/week43.ipynb @@ -21,10 +21,12 @@ "\n", "* Friday: Principal Component Analysis and Dimensionality Reduction\n", "\n", - "Reading suggestions for both days: \"Aurelien Geron's chapters 8 \n", + "We will also study the usage of [Autograd](https://www.youtube.com/watch?v=fRf4l5qaX1M&ab_channel=AlexSmola) in computing gradients for deep learning. For the documentation of Autograd and examples see the lectures slides from [week 40](https://compphysics.github.io/MachineLearning/doc/pub/week40/html/week40.html) and the [Autograd doucmentation](https://github.com/HIPS/autograd).\n", "\n", "## Recurrent Neural Networks\n", "\n", + "[Overview video](https://www.youtube.com/watch?v=SEnXr6v2ifU&ab_channel=AlexanderAmini).\n", + "See also lecture on Thursday October 22 and examples from [week 42](https://compphysics.github.io/MachineLearning/doc/pub/week42/html/week42.html).\n", "\n", "## Solving ODEs with Deep Learning\n", "\n", @@ -513,7 +515,7 @@ "source": [ "It is possible to use other activations functions for the hidden layer also.\n", "\n", - "The output $\\boldsymbol{x}_i^{\\text{hidden} }$from each $i$-th hidden neuron is:\n", + "The output $\\boldsymbol{x}_i^{\\text{hidden}}$ from each $i$-th hidden neuron is:\n", "\n", "$$\n", "\\boldsymbol{x}_i^{\\text{hidden} } = f\\big( \\boldsymbol{z}_{i}^{\\text{hidden}} \\big)\n", @@ -521,7 +523,11 @@ "\n", "The outputs $\\boldsymbol{x}_i^{\\text{hidden} } $ are then sent to the output layer.\n", "\n", - "The output layer consists of one neuron in this case, and combines the output from each of the neurons in the hidden layers. The output layer combines the results from the hidden layer using some weights $ w_i^{\\text{output}}$ and biases $b_i^{\\text{output}}$. In this case, it is assumes that the number of neurons in the output layer is one.\n", + "The output layer consists of one neuron in this case, and combines the\n", + "output from each of the neurons in the hidden layers. The output layer\n", + "combines the results from the hidden layer using some weights $w_i^{\\text{output}}$\n", + "and biases $b_i^{\\text{output}}$. In this case,\n", + "it is assumes that the number of neurons in the output layer is one.\n", "\n", "## Final technicalities III\n", "\n", @@ -578,7 +584,7 @@ "source": [ "In this case we seek a continuous range of values since we are approximating a function. This means that after computing $\\boldsymbol{z}_{1}^{\\text{output}}$ the neural network has finished its feed forward step, and $\\boldsymbol{z}_{1}^{\\text{output}}$ is the final output of the network.\n", "\n", - "## Backpropagation\n", + "## Back propagation\n", "\n", "The next step is to decide how the parameters should be changed such that they minimize the cost function.\n", "\n", @@ -630,7 +636,7 @@ "The value of $\\lambda$ decides how large steps the algorithm must take\n", "in the direction of $ \\nabla_{\\boldsymbol{\\omega}} C(\\boldsymbol{x}, \\boldsymbol{\\omega})$.\n", "The notation $\\nabla_{\\boldsymbol{\\omega}}$ express the gradient with respect\n", - "to the elements in $\\boldsymbol{\\omega$.\n", + "to the elements in $\\boldsymbol{\\omega}$.\n", "\n", "In our case, we have to minimize the cost function $C(\\boldsymbol{x}, P)$ with\n", "respect to the two sets of weights and biases, that is for the hidden\n", @@ -1027,8 +1033,8 @@ "\n", "In this example, similar network as for the exponential decay using Autograd has been used to solve the equation. However, as the implementation might suffer from e.g numerical instability\n", "and high execution time (this might be more apparent in the examples solving PDEs),\n", - "a network has been constructed using TensorFlow also.\n", - "For comparison, the forward Euler method has been implemented in order to see how the networks performs compared to a numerical scheme.\n", + "using a library like TensorFlow is recommended.\n", + "Here, we stay with a more simple approach and implement for comparison, the simple forward Euler method.\n", "\n", "## Setting up the problem\n", "\n", @@ -1261,7 +1267,7 @@ "source": [ "## Using forward Euler to solve the ODE\n", "\n", - "A straight-forward way of solving an ODE numerically, is to use Euler's method.\n", + "A straightforward way of solving an ODE numerically, is to use Euler's method.\n", "\n", "Euler's method uses Taylor series to approximate the value at a function $f$ at a step $\\Delta x$ from $x$:\n", "\n", @@ -1423,11 +1429,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Running the program gives\n", - "\n", - "\n", - "\n", - "## Example: Solving the one dimensional Poisson equation using Autograd and TensorFlow\n", + "## Example: Solving the one dimensional Poisson equation\n", "\n", "The Poisson equation for $g(x)$ in one dimension is" ] @@ -1845,7 +1847,7 @@ "f(x_{N_x - 3}) \\\\\n", "f(x_{N_x - 2})\n", "\\end{pmatrix} \\\\\n", - "A\\boldsymbol{g} &= \\boldsymbol{f}\n", + "\\boldsymbol{A}\\boldsymbol{g} &= \\boldsymbol{f},\n", "\\end{aligned}\n", "$$" ] @@ -1856,6 +1858,8 @@ "source": [ "which makes it possible to solve for the vector $\\boldsymbol{g}$.\n", "\n", + "## Setting up the code\n", + "\n", "We can then compare the result from this numerical scheme with the output from our network using Autograd:" ] }, @@ -2059,29 +2063,16 @@ " plt.show()" ] }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The program prints out:" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - " The max absolute difference between the analytical solution and DNN Autograd: 0.000464088\n", - " The max absolute difference between the analytical solution and numerical scheme: 0.00266858\n" - ] - }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Partial Differential Equations\n", "\n", - "A partial differential equation (PDE) has a solution here the function is defined by multiple variables.\n", - "The equation may involve all kinds of combinations of which variables the function is differentiated with respect to.\n", + "A partial differential equation (PDE) has a solution here the function\n", + "is defined by multiple variables. The equation may involve all kinds\n", + "of combinations of which variables the function is differentiated with\n", + "respect to.\n", "\n", "In general, a partial differential equation for a function $g(x_1,\\dots,x_N)$ with $N$ variables may be expressed as" ] @@ -2153,7 +2144,7 @@ "metadata": {}, "source": [ "$$\n", - "c\\left(x_1, \\dots, x_N, P\\right) = \\left( f\\left(x_1, \\, \\dots \\, , x_N, \\frac{\\partial g(x_1,\\dots,x_N) }{\\partial x_1}, \\dots , \\frac{\\partial g(x_1,\\dots,x_N) }{\\partial x_N}, \\frac{\\partial g(x_1,\\dots,x_N) }{\\partial x_1\\partial x_2}, \\, \\dots \\, , \\frac{\\partial^n g(x_1,\\dots,x_N) }{\\partial x_N^n} \\right) \\right)^2\n", + "C\\left(x_1, \\dots, x_N, P\\right) = \\left( f\\left(x_1, \\, \\dots \\, , x_N, \\frac{\\partial g(x_1,\\dots,x_N) }{\\partial x_1}, \\dots , \\frac{\\partial g(x_1,\\dots,x_N) }{\\partial x_N}, \\frac{\\partial g(x_1,\\dots,x_N) }{\\partial x_1\\partial x_2}, \\, \\dots \\, , \\frac{\\partial^n g(x_1,\\dots,x_N) }{\\partial x_N^n} \\right) \\right)^2\n", "$$" ] }, @@ -2163,7 +2154,7 @@ "source": [ "## More details\n", "\n", - "If we let $\\boldsymbol{x = \\big( x_1, \\dots, x_N \\big)$ be an array containing the values for $x_1, \\dots, x_N$ respectively, the cost function can be reformulated into the following:" + "If we let $\\boldsymbol{x} = \\big( x_1, \\dots, x_N \\big)$ be an array containing the values for $x_1, \\dots, x_N$ respectively, the cost function can be reformulated into the following:" ] }, { @@ -2171,7 +2162,7 @@ "metadata": {}, "source": [ "$$\n", - "c\\left(\\boldsymbol{x}, P\\right) = f\\left( \\left( \\boldsymbol{x}, \\frac{\\partial g(\\boldsymbol{x) }{\\partial x_1}, \\dots , \\frac{\\partial g(\\boldsymbol{x) }{\\partial x_N}, \\frac{\\partial g(\\boldsymbol{x) }{\\partial x_1\\partial x_2}, \\, \\dots \\, , \\frac{\\partial^n g(\\boldsymbol{x) }{\\partial x_N^n} \\right) \\right)^2\n", + "C\\left(\\boldsymbol{x}, P\\right) = f\\left( \\left( \\boldsymbol{x}, \\frac{\\partial g(\\boldsymbol{x}) }{\\partial x_1}, \\dots , \\frac{\\partial g(\\boldsymbol{x}) }{\\partial x_N}, \\frac{\\partial g(\\boldsymbol{x}) }{\\partial x_1\\partial x_2}, \\, \\dots \\, , \\frac{\\partial^n g(\\boldsymbol{x}) }{\\partial x_N^n} \\right) \\right)^2\n", "$$" ] }, @@ -2187,7 +2178,7 @@ "metadata": {}, "source": [ "$$\n", - "c\\left(X, P \\right) = \\sum_{i=1}^M f\\left( \\left( \\boldsymbol{x}_i, \\frac{\\partial g(\\boldsymbol{x}_i) }{\\partial x_1}, \\dots , \\frac{\\partial g(\\boldsymbol{x}_i) }{\\partial x_N}, \\frac{\\partial g(\\boldsymbol{x}_i) }{\\partial x_1\\partial x_2}, \\, \\dots \\, , \\frac{\\partial^n g(\\boldsymbol{x}_i) }{\\partial x_N^n} \\right) \\right)^2\n", + "C\\left(X, P \\right) = \\sum_{i=1}^M f\\left( \\left( \\boldsymbol{x}_i, \\frac{\\partial g(\\boldsymbol{x}_i) }{\\partial x_1}, \\dots , \\frac{\\partial g(\\boldsymbol{x}_i) }{\\partial x_N}, \\frac{\\partial g(\\boldsymbol{x}_i) }{\\partial x_1\\partial x_2}, \\, \\dots \\, , \\frac{\\partial^n g(\\boldsymbol{x}_i) }{\\partial x_N^n} \\right) \\right)^2.\n", "$$" ] }, @@ -3073,6 +3064,7 @@ "\n", "## Friday, Principal Component Analysis\n", "\n", + "[Overview video](https://www.youtube.com/watch?v=fkf4IBRSeEc&ab_channel=SteveBrunton)\n", "\n", "## Basic ideas of the Principal Component Analysis (PCA)\n", "\n", diff --git a/doc/src/week43/week43.do.txt b/doc/src/week43/week43.do.txt index b18cd0fcb..13aaf3c11 100644 --- a/doc/src/week43/week43.do.txt +++ b/doc/src/week43/week43.do.txt @@ -7,11 +7,14 @@ DATE: today * Thursday: Wrapping up Recurrent Neural Networks and solving differential equations. * Friday: Principal Component Analysis and Dimensionality Reduction -Reading suggestions for both days: "Aurelien Geron's chapters 8 + +We will also study the usage of "Autograd":"https://www.youtube.com/watch?v=fRf4l5qaX1M&ab_channel=AlexSmola" in computing gradients for deep learning. For the documentation of Autograd and examples see the lectures slides from "week 40":"https://compphysics.github.io/MachineLearning/doc/pub/week40/html/week40.html" and the "Autograd doucmentation":"https://github.com/HIPS/autograd". !split ===== Recurrent Neural Networks ===== +"Overview video":"https://www.youtube.com/watch?v=SEnXr6v2ifU&ab_channel=AlexanderAmini". +See also lecture on Thursday October 22 and examples from "week 42":"https://compphysics.github.io/MachineLearning/doc/pub/week42/html/week42.html". !split ===== Solving ODEs with Deep Learning ===== @@ -308,7 +311,7 @@ f(z) = \frac{1}{1 + \exp{(-z)}} It is possible to use other activations functions for the hidden layer also. -The output $\bm{x}_i^{\text{hidden} }$from each $i$-th hidden neuron is: +The output $\bm{x}_i^{\text{hidden}}$ from each $i$-th hidden neuron is: $$ \bm{x}_i^{\text{hidden} } = f\big( \bm{z}_{i}^{\text{hidden}} \big) @@ -316,7 +319,11 @@ $$ The outputs $\bm{x}_i^{\text{hidden} } $ are then sent to the output layer. -The output layer consists of one neuron in this case, and combines the output from each of the neurons in the hidden layers. The output layer combines the results from the hidden layer using some weights $ w_i^{\text{output}}$ and biases $b_i^{\text{output}}$. In this case, it is assumes that the number of neurons in the output layer is one. +The output layer consists of one neuron in this case, and combines the +output from each of the neurons in the hidden layers. The output layer +combines the results from the hidden layer using some weights $w_i^{\text{output}}$ +and biases $b_i^{\text{output}}$. In this case, +it is assumes that the number of neurons in the output layer is one. !split ===== Final technicalities III ===== @@ -356,7 +363,7 @@ b_1^{\text{output}} & \bm{w}_1^{\text{output}} In this case we seek a continuous range of values since we are approximating a function. This means that after computing $\bm{z}_{1}^{\text{output}}$ the neural network has finished its feed forward step, and $\bm{z}_{1}^{\text{output}}$ is the final output of the network. !split -===== Backpropagation ===== +===== Back propagation ===== The next step is to decide how the parameters should be changed such that they minimize the cost function. @@ -389,7 +396,7 @@ for a number of iterations or until $ \big|\big| \bm{\omega}_{\text{new} } - \bm The value of $\lambda$ decides how large steps the algorithm must take in the direction of $ \nabla_{\bm{\omega}} C(\bm{x}, \bm{\omega})$. The notation $\nabla_{\bm{\omega}}$ express the gradient with respect -to the elements in $\bm{\omega$. +to the elements in $\bm{\omega}$. In our case, we have to minimize the cost function $C(\bm{x}, P)$ with respect to the two sets of weights and biases, that is for the hidden @@ -742,8 +749,8 @@ Also, at $t = 0$ the population has the size $g(0) = g_0$, where $g_0$ is some c In this example, similar network as for the exponential decay using Autograd has been used to solve the equation. However, as the implementation might suffer from e.g numerical instability and high execution time (this might be more apparent in the examples solving PDEs), -a network has been constructed using TensorFlow also. -For comparison, the forward Euler method has been implemented in order to see how the networks performs compared to a numerical scheme. +using a library like TensorFlow is recommended. +Here, we stay with a more simple approach and implement for comparison, the simple forward Euler method. !split ===== Setting up the problem ===== @@ -955,7 +962,7 @@ if __name__ == '__main__': !split ===== Using forward Euler to solve the ODE ===== -A straight-forward way of solving an ODE numerically, is to use Euler's method. +A straightforward way of solving an ODE numerically, is to use Euler's method. Euler's method uses Taylor series to approximate the value at a function $f$ at a step $\Delta x$ from $x$: @@ -1069,12 +1076,10 @@ if __name__ == '__main__': plt.show() !ec -Running the program gives - !split -===== Example: Solving the one dimensional Poisson equation using Autograd and TensorFlow ===== +===== Example: Solving the one dimensional Poisson equation ===== The Poisson equation for $g(x)$ in one dimension is @@ -1365,12 +1370,15 @@ f(x_2) \\ f(x_{N_x - 3}) \\ f(x_{N_x - 2}) \end{pmatrix} \\ -A\bm{g} &= \bm{f} +\bm{A}\bm{g} &= \bm{f}, \end{aligned} !et which makes it possible to solve for the vector $\bm{g}$. +!split +===== Setting up the code ===== + We can then compare the result from this numerical scheme with the output from our network using Autograd: !bc pycod @@ -1567,18 +1575,15 @@ if __name__ == '__main__': !ec -The program prints out: -!bc -The max absolute difference between the analytical solution and DNN Autograd: 0.000464088 -The max absolute difference between the analytical solution and numerical scheme: 0.00266858 -!ec !split ===== Partial Differential Equations ===== -A partial differential equation (PDE) has a solution here the function is defined by multiple variables. -The equation may involve all kinds of combinations of which variables the function is differentiated with respect to. +A partial differential equation (PDE) has a solution here the function +is defined by multiple variables. The equation may involve all kinds +of combinations of which variables the function is differentiated with +respect to. In general, a partial differential equation for a function $g(x_1,\dots,x_N)$ with $N$ variables may be expressed as @@ -1623,24 +1628,24 @@ minimize is !bt \begin{equation*} -c\left(x_1, \dots, x_N, P\right) = \left( f\left(x_1, \, \dots \, , x_N, \frac{\partial g(x_1,\dots,x_N) }{\partial x_1}, \dots , \frac{\partial g(x_1,\dots,x_N) }{\partial x_N}, \frac{\partial g(x_1,\dots,x_N) }{\partial x_1\partial x_2}, \, \dots \, , \frac{\partial^n g(x_1,\dots,x_N) }{\partial x_N^n} \right) \right)^2 +C\left(x_1, \dots, x_N, P\right) = \left( f\left(x_1, \, \dots \, , x_N, \frac{\partial g(x_1,\dots,x_N) }{\partial x_1}, \dots , \frac{\partial g(x_1,\dots,x_N) }{\partial x_N}, \frac{\partial g(x_1,\dots,x_N) }{\partial x_1\partial x_2}, \, \dots \, , \frac{\partial^n g(x_1,\dots,x_N) }{\partial x_N^n} \right) \right)^2 \end{equation*} !et !split ===== More details ===== -If we let $\bm{x = \big( x_1, \dots, x_N \big)$ be an array containing the values for $x_1, \dots, x_N$ respectively, the cost function can be reformulated into the following: +If we let $\bm{x} = \big( x_1, \dots, x_N \big)$ be an array containing the values for $x_1, \dots, x_N$ respectively, the cost function can be reformulated into the following: !bt -\begin{equation*} - c\left(\bm{x}, P\right) = f\left( \left( \bm{x}, \frac{\partial g(\bm{x) }{\partial x_1}, \dots , \frac{\partial g(\bm{x) }{\partial x_N}, \frac{\partial g(\bm{x) }{\partial x_1\partial x_2}, \, \dots \, , \frac{\partial^n g(\bm{x) }{\partial x_N^n} \right) \right)^2 -\end{equation*} +\[ + C\left(\bm{x}, P\right) = f\left( \left( \bm{x}, \frac{\partial g(\bm{x}) }{\partial x_1}, \dots , \frac{\partial g(\bm{x}) }{\partial x_N}, \frac{\partial g(\bm{x}) }{\partial x_1\partial x_2}, \, \dots \, , \frac{\partial^n g(\bm{x}) }{\partial x_N^n} \right) \right)^2 +\] !et If we also have $M$ different sets of values for $x_1, \dots, x_N$, that is $\bm{x}_i = \big(x_1^{(i)}, \dots, x_N^{(i)}\big)$ for $i = 1,\dots,M$ being the rows in matrix $X$, the cost function can be generalized into !bt \begin{equation*} -c\left(X, P \right) = \sum_{i=1}^M f\left( \left( \bm{x}_i, \frac{\partial g(\bm{x}_i) }{\partial x_1}, \dots , \frac{\partial g(\bm{x}_i) }{\partial x_N}, \frac{\partial g(\bm{x}_i) }{\partial x_1\partial x_2}, \, \dots \, , \frac{\partial^n g(\bm{x}_i) }{\partial x_N^n} \right) \right)^2 +C\left(X, P \right) = \sum_{i=1}^M f\left( \left( \bm{x}_i, \frac{\partial g(\bm{x}_i) }{\partial x_1}, \dots , \frac{\partial g(\bm{x}_i) }{\partial x_N}, \frac{\partial g(\bm{x}_i) }{\partial x_1\partial x_2}, \, \dots \, , \frac{\partial^n g(\bm{x}_i) }{\partial x_N^n} \right) \right)^2. \end{equation*} !et @@ -2395,6 +2400,7 @@ o "Introduction to Partial Differential Equations by A. Tveito, R. Winther":"htt !split ===== Friday, Principal Component Analysis ===== +"Overview video":"https://www.youtube.com/watch?v=fkf4IBRSeEc&ab_channel=SteveBrunton" !split ===== Basic ideas of the Principal Component Analysis (PCA) =====