Data Analysis and Machine Learning: Neural networks, from the simple perceptron to deep learning and convolutional networks
Contents
Neural networks
Artificial neurons
Neural network types
Feed-forward neural networks
Convolutional Neural Network
Recurrent neural networks
Other types of networks
Multilayer perceptrons
Why multilayer perceptrons?
Mathematical model
Mathematical model
Mathematical model
Mathematical model
Mathematical model
Matrix-vector notation
Matrix-vector notation and activation
Activation functions
Activation functions, Logistic and Hyperbolic ones
Relevance
The multilayer perceptron (MLP)
From one to many layers, the universal approximation theorem
Deriving the back propagation code for a multilayer perceptron model
Definitions
Derivatives and the chain rule
Derivative of the cost function
Bringing it together, first back propagation equation
Derivatives in terms of \( z_j^L \)
Bringing it together
Final back propagating equation
Setting up the Back propagation algorithm
Setting up a Multi-layer perceptron model for classification
Defining the cost function
Example: binary classification problem
The Softmax function
Developing a code for doing neural networks with back propagation
Collect and pre-process data
Train and test datasets
Define model and architecture
Layers
Weights and biases
Feed-forward pass
Matrix multiplications
Choose cost function and optimizer
Optimizing the cost function
Regularization
Matrix multiplication
Improving performance
Full object-oriented implementation
Evaluate model performance on test data
Adjust hyperparameters
Visualization
scikit-learn implementation
Visualization
Building neural networks in Tensorflow and Keras
Tensorflow
Collect and pre-process data
Using TensorFlow backend
Optimizing and using gradient descent
Using Keras
Which activation function should I use?
Is the Logistic activation function (Sigmoid) our choice?
The derivative of the Logistic funtion
The RELU function family
Which activation function should we use?
A top-down perspective on Neural networks
Limitations of supervised learning with deep networks
Convolutional Neural Networks (recognizing images)
Regular NNs don’t scale well to full images
3D volumes of neurons
Layers used to build CNNs
Transforming images
CNNs in brief
CNNs in more detail, building convolutional neural networks in Tensorflow and Keras
Setting it up
The MNIST dataset again
Strong correlations
Layers of a CNN
Systematic reduction
Prerequisites: Collect and pre-process data
Importing Keras and Tensorflow
Using TensorFlow backend
Train the model
Visualizing the results
Running with Keras
Final part
Final visualization
Fun links
Applications: solving ordinary differential equations with Neural Networks
Trial solution
More details
Reformulating the problem
Estimating errors
Creating a simple Deep Neural Net
Setting up the code, feed forward part
Backpropagation
Gradient Descent
More on GD and cost function
An implementation of a Deep Neural Network
The final parts of the code
And adding Back propagation
Solving the ODE
Using neural network
Using a deep neural network
Wrapping it up
Fun links
Self-Driving cars using a convolutional neural network
Abstract art using convolutional neural networks
«
1
...
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
...
105
»