update on neural net

doc/src/NeuralNet/NeuralNet.do.txt

@@ -5,15 +5,19 @@ DATE: today
 !split
 ===== What is Machine Learning? =====
 
-Machine learning is the science of giving computers the ability to learn without being explicitly programmed. 
-The idea is that there exist generic algorithms which can be used to find patterns in a broad class of data sets without 
-having to write code specifically for each problem. The algorithm will build its own logic based on the data.  
+Machine learning is the science of giving computers the ability to
+learn without being explicitly programmed.  The idea is that there
+exist generic algorithms which can be used to find patterns in a broad
+class of data sets without having to write code specifically for each
+problem. The algorithm will build its own logic based on the data.
 
-Machine learning is a subfield of computer science, and is closely related to computational statistics. 
-It evolved from the study of pattern recognition in artificial intelligence (AI) research, and has made contributions to
-AI tasks like computer vision, natural language processing 
-and speech recognition. It has also, especially in later years, 
-found applications in a wide variety of other areas, including bioinformatics, economy, physics, finance and marketing. 
+Machine learning is a subfield of computer science, and is closely
+related to computational statistics.  It evolved from the study of
+pattern recognition in artificial intelligence (AI) research, and has
+made contributions to AI tasks like computer vision, natural language
+processing and speech recognition. It has also, especially in later
+years, found applications in a wide variety of other areas, including
+bioinformatics, economy, physics, finance and marketing.
 
 !split
 ===== Types of Machine Learning =====
@@ -39,14 +43,18 @@ Some of the most common tasks are:
 
 !split
 ===== Artificial neurons  =====
-The field of artificial neural networks has a long history of development, and is closely connected with 
-the advancement of computer science and computers in general. A model of artificial neurons 
-was first developed by McCulloch and Pitts in 1943  to study signal processing in the brain and 
-has later been refined by others. The general idea is to mimic neural networks in the human brain, which
-is composed of billions of neurons that communicate with each other by sending electrical signals. 
-Each neuron accumulates its incoming signals, 
-which must exceed an activation threshold to yield an output. If the threshold is not overcome, the neuron
-remains inactive, i.e. has zero output.  
+
+The field of artificial neural networks has a long history of
+development, and is closely connected with the advancement of computer
+science and computers in general. A model of artificial neurons was
+first developed by McCulloch and Pitts in 1943 to study signal
+processing in the brain and has later been refined by others. The
+general idea is to mimic neural networks in the human brain, which is
+composed of billions of neurons that communicate with each other by
+sending electrical signals.  Each neuron accumulates its incoming
+signals, which must exceed an activation threshold to yield an
+output. If the threshold is not overcome, the neuron remains inactive,
+i.e. has zero output.
 
 This behaviour has inspired a simple mathematical model for an artificial neuron.
 
@@ -59,21 +67,57 @@ This behaviour has inspired a simple mathematical model for an artificial neuron
 Here, the output $y$ of the neuron is the value of its activation function, which have as input
 a weighted sum of signals $x_i, \dots ,x_n$ received by $n$ other neurons.
 
+Conceptually, it is helpful to divide neural networks into four
+categories:
+o general purpose neural networks for supervised learning,
+o neural networks designed specifically for image processing, the most prominent example of this class being Convolutional Neural Networks (CNNs),
+o neural networks for sequential data such as Recurrent Neural Networks (RNNs), and
+o neural networks for unsupervised learning such as Deep Boltzmann Machines.
+
+
+In physics, DNNs and CNNs have already found numerous applications. In
+statistical physics, they have been applied to detect phase
+transitions in 2D Ising and Potts models, lattice gauge theories, and
+different phases of polymers.
+Deep learning has also found interesting applications in quantum
+physics. Various quantum phase transitions can be detected and studied
+using DNNs and CNNs, including the transverse-field Ising model,
+topological phases, and even non-equilibrium many-body
+localization. Representing quantum states as DNNs quantum state
+tomography are among some of the impressive
+achievements to reveal the potential of DNNs to facilitate the study
+of quantum systems.
+
+In quantum information theory, it has been shown that one can perform
+gate decompositions with the help of neural. In lattice quantum chromodynamics,
+DNNs have been used to learn action parameters in regions of parameter
+space where PCA fails. Last but not least,
+DNNs also found place in the study of quantum, and in scattering theory to learn
+$s$-wave scattering length of potentials.
+
 !split
 ===== Neural network types =====
 
-An artificial neural network (NN), is a computational model that consists of layers of connected neurons, or *nodes*. 
-It is supposed to mimic a biological nervous system by letting each neuron interact with other neurons
-by sending signals in the form of mathematical functions between layers. 
-A wide variety of different NNs have
-been developed, but most of them consist of an input layer, an output layer and eventual layers in-between, called
-*hidden layers*. All layers can contain an arbitrary number of nodes, and each connection between two nodes
-is associated with a weight variable. 
+An artificial neural network (NN), is a computational model that
+consists of layers of connected neurons, or *nodes*.  It is supposed
+to mimic a biological nervous system by letting each neuron interact
+with other neurons by sending signals in the form of mathematical
+functions between layers.  A wide variety of different NNs have been
+developed, but most of them consist of an input layer, an output layer
+and eventual layers in-between, called *hidden layers*. All layers can
+contain an arbitrary number of nodes, and each connection between two
+nodes is associated with a weight variable.
 
+Neural networks (also called neural nets) are neural-inspired
+nonlinear models for supervised learning.  As we will see, neural nets
+can be viewed as natural, more powerful extensions of supervised
+learning methods such as linear and logistic regression and soft-max
+methods.
 
 
 !split
 ===== Feed-forward neural networks =====
+
 The feed-forward neural network (FFNN) was the first and simplest type of NN devised. In this network, 
 the information moves in only one direction: forward through the layers.
 
@@ -351,3 +395,5 @@ performs better than the sigmoid for training MLPs.
 has become the most popular  for *deep neural networks*
 
 
+!split
+===