From 2a72f19a1c997e32fb0d225e86f0bff41e5d4aed Mon Sep 17 00:00:00 2001 From: mhjensen Date: Sat, 31 Mar 2018 18:48:38 -0400 Subject: [PATCH] update on neural net --- doc/src/NeuralNet/NeuralNet.do.txt | 92 ++++++++++++++++++++++-------- 1 file changed, 69 insertions(+), 23 deletions(-) diff --git a/doc/src/NeuralNet/NeuralNet.do.txt b/doc/src/NeuralNet/NeuralNet.do.txt index 111ca5635..cab0ba5fd 100644 --- a/doc/src/NeuralNet/NeuralNet.do.txt +++ b/doc/src/NeuralNet/NeuralNet.do.txt @@ -5,15 +5,19 @@ DATE: today !split ===== What is Machine Learning? ===== -Machine learning is the science of giving computers the ability to learn without being explicitly programmed. -The idea is that there exist generic algorithms which can be used to find patterns in a broad class of data sets without -having to write code specifically for each problem. The algorithm will build its own logic based on the data. +Machine learning is the science of giving computers the ability to +learn without being explicitly programmed. The idea is that there +exist generic algorithms which can be used to find patterns in a broad +class of data sets without having to write code specifically for each +problem. The algorithm will build its own logic based on the data. -Machine learning is a subfield of computer science, and is closely related to computational statistics. -It evolved from the study of pattern recognition in artificial intelligence (AI) research, and has made contributions to -AI tasks like computer vision, natural language processing -and speech recognition. It has also, especially in later years, -found applications in a wide variety of other areas, including bioinformatics, economy, physics, finance and marketing. +Machine learning is a subfield of computer science, and is closely +related to computational statistics. It evolved from the study of +pattern recognition in artificial intelligence (AI) research, and has +made contributions to AI tasks like computer vision, natural language +processing and speech recognition. It has also, especially in later +years, found applications in a wide variety of other areas, including +bioinformatics, economy, physics, finance and marketing. !split ===== Types of Machine Learning ===== @@ -39,14 +43,18 @@ Some of the most common tasks are: !split ===== Artificial neurons ===== -The field of artificial neural networks has a long history of development, and is closely connected with -the advancement of computer science and computers in general. A model of artificial neurons -was first developed by McCulloch and Pitts in 1943 to study signal processing in the brain and -has later been refined by others. The general idea is to mimic neural networks in the human brain, which -is composed of billions of neurons that communicate with each other by sending electrical signals. -Each neuron accumulates its incoming signals, -which must exceed an activation threshold to yield an output. If the threshold is not overcome, the neuron -remains inactive, i.e. has zero output. + +The field of artificial neural networks has a long history of +development, and is closely connected with the advancement of computer +science and computers in general. A model of artificial neurons was +first developed by McCulloch and Pitts in 1943 to study signal +processing in the brain and has later been refined by others. The +general idea is to mimic neural networks in the human brain, which is +composed of billions of neurons that communicate with each other by +sending electrical signals. Each neuron accumulates its incoming +signals, which must exceed an activation threshold to yield an +output. If the threshold is not overcome, the neuron remains inactive, +i.e. has zero output. This behaviour has inspired a simple mathematical model for an artificial neuron. @@ -59,21 +67,57 @@ This behaviour has inspired a simple mathematical model for an artificial neuron Here, the output $y$ of the neuron is the value of its activation function, which have as input a weighted sum of signals $x_i, \dots ,x_n$ received by $n$ other neurons. +Conceptually, it is helpful to divide neural networks into four +categories: +o general purpose neural networks for supervised learning, +o neural networks designed specifically for image processing, the most prominent example of this class being Convolutional Neural Networks (CNNs), +o neural networks for sequential data such as Recurrent Neural Networks (RNNs), and +o neural networks for unsupervised learning such as Deep Boltzmann Machines. + + +In physics, DNNs and CNNs have already found numerous applications. In +statistical physics, they have been applied to detect phase +transitions in 2D Ising and Potts models, lattice gauge theories, and +different phases of polymers. +Deep learning has also found interesting applications in quantum +physics. Various quantum phase transitions can be detected and studied +using DNNs and CNNs, including the transverse-field Ising model, +topological phases, and even non-equilibrium many-body +localization. Representing quantum states as DNNs quantum state +tomography are among some of the impressive +achievements to reveal the potential of DNNs to facilitate the study +of quantum systems. + +In quantum information theory, it has been shown that one can perform +gate decompositions with the help of neural. In lattice quantum chromodynamics, +DNNs have been used to learn action parameters in regions of parameter +space where PCA fails. Last but not least, +DNNs also found place in the study of quantum, and in scattering theory to learn +$s$-wave scattering length of potentials. + !split ===== Neural network types ===== -An artificial neural network (NN), is a computational model that consists of layers of connected neurons, or *nodes*. -It is supposed to mimic a biological nervous system by letting each neuron interact with other neurons -by sending signals in the form of mathematical functions between layers. -A wide variety of different NNs have -been developed, but most of them consist of an input layer, an output layer and eventual layers in-between, called -*hidden layers*. All layers can contain an arbitrary number of nodes, and each connection between two nodes -is associated with a weight variable. +An artificial neural network (NN), is a computational model that +consists of layers of connected neurons, or *nodes*. It is supposed +to mimic a biological nervous system by letting each neuron interact +with other neurons by sending signals in the form of mathematical +functions between layers. A wide variety of different NNs have been +developed, but most of them consist of an input layer, an output layer +and eventual layers in-between, called *hidden layers*. All layers can +contain an arbitrary number of nodes, and each connection between two +nodes is associated with a weight variable. +Neural networks (also called neural nets) are neural-inspired +nonlinear models for supervised learning. As we will see, neural nets +can be viewed as natural, more powerful extensions of supervised +learning methods such as linear and logistic regression and soft-max +methods. !split ===== Feed-forward neural networks ===== + The feed-forward neural network (FFNN) was the first and simplest type of NN devised. In this network, the information moves in only one direction: forward through the layers. @@ -351,3 +395,5 @@ performs better than the sigmoid for training MLPs. has become the most popular for *deep neural networks* +!split +===